Max level shown:
1 starting January 14, 2025 at 04:10:29 PM UTC
2 Creating 3 downstairs regions
3 Creating region directory at: /var/tmp/dsc/region
4 Creating dsc directory at: /tmp/dsc
5 Downstairs region 0 created at /var/tmp/dsc/region/8810 in 0.27181512
6 Downstairs region 1 created at /var/tmp/dsc/region/8820 in 0.02234496
7 Downstairs region 2 created at /var/tmp/dsc/region/8830 in 0.020097932
8 Created 3 regions
9 Starting 3 downstairs
10 Using existing output directory "/tmp/dsc"
11 Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 })
12 start ds: 8810
13 start ds: 8820
14 start access at:127.0.0.1:9998
15 start ds: 8830
16 Starting downstairs at port 8810
17 Starting downstairs at port 8820
18 Make output file at "/tmp/dsc/downstairs-8820.txt"
19 Make output file at "/tmp/dsc/downstairs-8810.txt"
20 Starting downstairs at port 8830
21 Make output file at "/tmp/dsc/downstairs-8830.txt"
22 [8810][0] initial start wait reports Starting
23 [8820][1] initial start wait reports Starting
24 [8830][2] initial start wait reports Starting
25 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1192)
26 [8820][1] initial start wait reports Running
27 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1193)
28 [8810][0] initial start wait reports Running
29 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1194)
30 [8830][2] initial start wait reports Running
31 All downstairs are running
32 Jan 14 16:10:30.132 INFO listening, local_addr: 127.0.0.1:9998
33 Control access at:127.0.0.1:9998
34 Running initial fill
35 Jan 14 16:10:35.194 INFO accepted connection, remote_addr: 127.0.0.1:40235, local_addr: 127.0.0.1:9998
36 Jan 14 16:10:35.196 INFO request completed, latency_us: 392, response_code: 200, uri: /regioninfo, method: GET, req_id: 1d6961da-dfc8-4c21-956a-417d567f2e15, remote_addr: 127.0.0.1:40235, local_addr: 127.0.0.1:9998
37 Jan 14 16:10:35.198 INFO request completed, latency_us: 114, response_code: 200, uri: /regioncount, method: GET, req_id: 09a6ecc5-4d02-401a-92ba-54d4c10d3444, remote_addr: 127.0.0.1:40235, local_addr: 127.0.0.1:9998
382025-01-14T16:10:35.198ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
392025-01-14T16:10:35.198ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
40 Jan 14 16:10:35.199 INFO request completed, latency_us: 148, response_code: 200, uri: /port/cid/0, method: GET, req_id: 932c9b9c-9503-4a17-bf4d-353898bdadbf, remote_addr: 127.0.0.1:40235, local_addr: 127.0.0.1:9998
41 Jan 14 16:10:35.199 INFO request completed, latency_us: 92, response_code: 200, uri: /port/cid/1, method: GET, req_id: 797d7a43-2167-43b3-8ce6-5c4dbe26055b, remote_addr: 127.0.0.1:40235, local_addr: 127.0.0.1:9998
42 Jan 14 16:10:35.200 INFO request completed, latency_us: 91, response_code: 200, uri: /port/cid/2, method: GET, req_id: 939e4fa5-bdea-4513-87ae-35858b4eb685, remote_addr: 127.0.0.1:40235, local_addr: 127.0.0.1:9998
432025-01-14T16:10:35.200ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
442025-01-14T16:10:35.200ZINFOcrucible: Upstairs starts
452025-01-14T16:10:35.201ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "42687c1346c9f5be1faf41362247db1201c426a8", git_commit_timestamp: "2025-01-14T15:54:39.000000000Z", git_branch: "mkeeter/separate-notify-task", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
462025-01-14T16:10:35.201ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
472025-01-14T16:10:35.201ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
482025-01-14T16:10:35.202ZINFOcrucible: Crucible 84246727-0985-443f-a6ca-67cdd46e167e has session id: 4d5c1a8c-a38a-4321-94df-ffcec47e41b3 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
492025-01-14T16:10:35.202ZINFOcrucible: Upstairs opts: Upstairs UUID: 84246727-0985-443f-a6ca-67cdd46e167e, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
502025-01-14T16:10:35.202ZINFOcrucible: Crucible stats registered with UUID: 84246727-0985-443f-a6ca-67cdd46e167e session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
512025-01-14T16:10:35.202ZWARNcrucible: could not find Downstairs address for Nexus session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
522025-01-14T16:10:35.203ZINFOcrucible: Downstairs in volume = 3 task = crutest
532025-01-14T16:10:35.203ZINFOcrucible: The guest has requested activation with gen:1
542025-01-14T16:10:35.203ZINFOcrucible: 84246727-0985-443f-a6ca-67cdd46e167e active request set session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
552025-01-14T16:10:35.203ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
562025-01-14T16:10:35.203ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
572025-01-14T16:10:35.203ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
582025-01-14T16:10:35.203ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
592025-01-14T16:10:35.203ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
602025-01-14T16:10:35.203ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
612025-01-14T16:10:35.203ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
622025-01-14T16:10:35.203ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
632025-01-14T16:10:35.203ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
642025-01-14T16:10:35.203ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38630) = io task client = 1 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
652025-01-14T16:10:35.203ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:47192) = io task client = 0 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
662025-01-14T16:10:35.203ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40717) = io task client = 2 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
672025-01-14T16:10:35.206ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
682025-01-14T16:10:35.206ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
692025-01-14T16:10:35.206ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
702025-01-14T16:10:35.206ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
712025-01-14T16:10:35.206ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
722025-01-14T16:10:35.207ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
732025-01-14T16:10:35.207ZINFOcrucible: Waiting for more clients to be ready session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
742025-01-14T16:10:35.207ZINFOcrucible: Waiting for more clients to be ready session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
752025-01-14T16:10:35.207ZINFOcrucible: [0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
762025-01-14T16:10:35.207ZINFOcrucible: [0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
772025-01-14T16:10:35.207ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
782025-01-14T16:10:35.207ZINFOcrucible: [1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
792025-01-14T16:10:35.207ZINFOcrucible: [1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
802025-01-14T16:10:35.207ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
812025-01-14T16:10:35.207ZINFOcrucible: [2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
822025-01-14T16:10:35.207ZINFOcrucible: [2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
832025-01-14T16:10:35.207ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
842025-01-14T16:10:35.207ZINFOcrucible: Max found gen is 1 = downstairs session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
852025-01-14T16:10:35.207ZINFOcrucible: Generation requested: 1 >= found:1 = downstairs session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
862025-01-14T16:10:35.207ZINFOcrucible: Next flush: 1 = downstairs session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
872025-01-14T16:10:35.208ZINFOcrucible: All extents match = downstairs session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
882025-01-14T16:10:35.208ZINFOcrucible: No downstairs reconciliation required session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
892025-01-14T16:10:35.208ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
902025-01-14T16:10:35.208ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
912025-01-14T16:10:35.208ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
922025-01-14T16:10:35.208ZINFOcrucible: All required reconciliation work is completed session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
93 {"msg":"Set Downstairs and Upstairs active after reconciliation","v":0,"name":"crucible","level":30Activate successful
94 Wait for a query_work_queue command to finish before sending IO
95 ,"time":"2025-01-14T16:10:35.208163962Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1198,"session_id":"4d5c1a8c-a38a-4321-94df-ffcec47e41b3"}
962025-01-14T16:10:35.208ZINFOcrucible: 84246727-0985-443f-a6ca-67cdd46e167e is now active with session: 4d5c1a8c-a38a-4321-94df-ffcec47e41b3 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
972025-01-14T16:10:35.208ZINFOcrucible: Set Active after no reconciliation session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
982025-01-14T16:10:35.208ZINFOcrucible: The guest has finished waiting for activation with:1
99 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
100 Fill test
101 Read and Verify all blocks (0..5000 range:false)
102 Wrote out file "/tmp/test_replay_verify.log"
103 CLIENT: Tests done. All submitted work has been ACK'd
104 ----------------------------------------------------------------
105 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:20
106 States: Active Active Active
107 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
108 1021 Acked Read 256 Done Done Done false
109 1022 Acked Read 256 Done Done Done false
110 1023 Acked Read 256 Done Done Done false
111 1024 Acked Read 256 Done Done Done false
112 1025 Acked Read 256 Done Done Done false
113 1026 Acked Read 256 Done Done Done false
114 1027 Acked Read 256 Done Done Done false
115 1028 Acked Read 256 Done Done Done false
116 1029 Acked Read 256 Done Done Done false
117 1030 Acked Read 256 Done Done Done false
118 1031 Acked Read 256 Done Done Done false
119 1032 Acked Read 256 Done Done Done false
120 1033 Acked Read 256 Done Done Done false
121 1034 Acked Read 256 Done Done Done false
122 1035 Acked Read 256 Done Done Done false
123 1036 Acked Read 256 Done Done Done false
124 1037 Acked Read 256 Done Done Done false
125 1038 Acked Read 256 Done Done Done false
126 1039 Acked Read 256 Done Done Done false
127 1040 Acked Read 136 Done Done Done false
128 STATES DS:0 DS:1 DS:2 TOTAL
129 Sent 0 0 0 0
130 Done 20 20 20 60
131 Skipped 0 0 0 0
132 Error 0 0 0 0
133 Last Flush: 1020 1020 1020
134 Downstairs last five completed: 1020 1019 1018 1017 1016
135 Upstairs last five completed: 1040 1039 1038 1037 1036
136 CLIENT: Up:0 ds:20 act:3
137 ----------------------------------------------------------------
138 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0
139 Downstairs last five completed: 1041 1040 1039 1038 1037
140 Upstairs last five completed: 1041 1040 1039 1038 1037
141 CLIENT: Up:0 ds:0 act:3
142 CLIENT: All crucible jobs finished, exiting program
1432025-01-14T16:10:39.807ZWARNcrucible: Guest handle has been dropped
1442025-01-14T16:10:39.807ZWARNcrucible: client request queue closed unexpectedly; is the program exiting? = io task client = 2 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
1452025-01-14T16:10:39.807ZWARNcrucible: client task is sending Done(QueueClosed) = io task client = 2 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
1462025-01-14T16:10:39.807ZWARNcrucible: client task could not reply to main task; shutting down? = io task client = 2 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
1472025-01-14T16:10:39.807ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4d5c1a8c-a38a-4321-94df-ffcec47e41b3
148 Replay loop starts now January 14, 2025 at 04:10:39 PM UTC
149 Jan 14 16:10:39.877 INFO accepted connection, remote_addr: 127.0.0.1:48102, local_addr: 127.0.0.1:9998
150 Jan 14 16:10:39.878 INFO request completed, latency_us: 143, response_code: 200, uri: /regioninfo, method: GET, req_id: e2bf1ec6-cc2b-40ee-a819-90ca9e064e18, remote_addr: 127.0.0.1:48102, local_addr: 127.0.0.1:9998
1512025-01-14T16:10:39.879ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
152 Jan 14 16:10:39.879 INFO request completed, latency_us: 96, response_code: 200, uri: /regioncount, method: GET, req_id: 7036559c-afd4-4008-850e-bbc192bad222, remote_addr: 127.0.0.1:48102, local_addr: 127.0.0.1:9998
1532025-01-14T16:10:39.879ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
154 Jan 14 16:10:39.880 INFO request completed, latency_us: 113, response_code: 200, uri: /port/cid/0, method: GET, req_id: 404c42e4-c73d-497a-ba03-7d878e3c42c6, remote_addr: 127.0.0.1:48102, local_addr: 127.0.0.1:9998
155 Jan 14 16:10:39.880 INFO request completed, latency_us: 101, response_code: 200, uri: /port/cid/1, method: GET, req_id: de7fe4b0-a72b-4984-913e-28c94f6e2874, remote_addr: 127.0.0.1:48102, local_addr: 127.0.0.1:9998
156 Jan 14 16:10:39.881 INFO request completed, latency_us: 94, response_code: 200, uri: /port/cid/2, method: GET, req_id: 6e08a2c6-7817-41c2-894d-76d97c8aed36, remote_addr: 127.0.0.1:48102, local_addr: 127.0.0.1:9998
1572025-01-14T16:10:39.881ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
1582025-01-14T16:10:39.881ZINFOcrucible: Upstairs starts
1592025-01-14T16:10:39.882ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "42687c1346c9f5be1faf41362247db1201c426a8", git_commit_timestamp: "2025-01-14T15:54:39.000000000Z", git_branch: "mkeeter/separate-notify-task", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
1602025-01-14T16:10:39.882ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
1612025-01-14T16:10:39.882ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
1622025-01-14T16:10:39.882ZINFOcrucible: Crucible 95474d24-b633-49f3-bc8a-d6ec632fff7e has session id: c2423ef0-728e-4895-997a-d9044c50948b session_id = c2423ef0-728e-4895-997a-d9044c50948b
1632025-01-14T16:10:39.882ZINFOcrucible: Upstairs opts: Upstairs UUID: 95474d24-b633-49f3-bc8a-d6ec632fff7e, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = c2423ef0-728e-4895-997a-d9044c50948b
1642025-01-14T16:10:39.882ZINFOcrucible: Crucible stats registered with UUID: 95474d24-b633-49f3-bc8a-d6ec632fff7e session_id = c2423ef0-728e-4895-997a-d9044c50948b
1652025-01-14T16:10:39.882ZWARNcrucible: could not find Downstairs address for Nexus session_id = c2423ef0-728e-4895-997a-d9044c50948b
1662025-01-14T16:10:39.883ZINFOcrucible: Downstairs in volume = 3 task = crutest
1672025-01-14T16:10:39.883ZINFOcrucible: The guest has requested activation with gen:2
1682025-01-14T16:10:39.883ZINFOcrucible: 95474d24-b633-49f3-bc8a-d6ec632fff7e active request set session_id = c2423ef0-728e-4895-997a-d9044c50948b
1692025-01-14T16:10:39.883ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1702025-01-14T16:10:39.883ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1712025-01-14T16:10:39.883ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1722025-01-14T16:10:39.883ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1732025-01-14T16:10:39.883ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1742025-01-14T16:10:39.883ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1752025-01-14T16:10:39.884ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1762025-01-14T16:10:39.884ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1772025-01-14T16:10:39.884ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1782025-01-14T16:10:39.884ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:43341) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1792025-01-14T16:10:39.884ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:56150) = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1802025-01-14T16:10:39.884ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:65416) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1812025-01-14T16:10:39.884ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1822025-01-14T16:10:39.884ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1832025-01-14T16:10:39.884ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1842025-01-14T16:10:39.885ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1852025-01-14T16:10:39.885ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1862025-01-14T16:10:39.885ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1872025-01-14T16:10:39.886ZINFOcrucible: Waiting for more clients to be ready session_id = c2423ef0-728e-4895-997a-d9044c50948b
1882025-01-14T16:10:39.886ZINFOcrucible: Waiting for more clients to be ready session_id = c2423ef0-728e-4895-997a-d9044c50948b
1892025-01-14T16:10:39.886ZINFOcrucible: [0]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
1902025-01-14T16:10:39.886ZINFOcrucible: [0]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
1912025-01-14T16:10:39.886ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
1922025-01-14T16:10:39.886ZINFOcrucible: [1]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
1932025-01-14T16:10:39.886ZINFOcrucible: [1]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
1942025-01-14T16:10:39.886ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
1952025-01-14T16:10:39.886ZINFOcrucible: [2]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
1962025-01-14T16:10:39.886ZINFOcrucible: [2]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
1972025-01-14T16:10:39.886ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
1982025-01-14T16:10:39.886ZINFOcrucible: Max found gen is 2 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
1992025-01-14T16:10:39.886ZINFOcrucible: Generation requested: 2 >= found:2 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
2002025-01-14T16:10:39.886ZINFOcrucible: Next flush: 2 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
2012025-01-14T16:10:39.886ZINFOcrucible: All extents match = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
2022025-01-14T16:10:39.886ZINFOcrucible: No downstairs reconciliation required session_id = c2423ef0-728e-4895-997a-d9044c50948b
2032025-01-14T16:10:39.886ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
204 {"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"Activate successful
205 2025-01-14T16:10:39.88640064ZWait for a query_work_queue command to finish before sending IO
206 ","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"client":"1","":"downstairs","session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
2072025-01-14T16:10:39.886ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
2082025-01-14T16:10:39.886ZINFOcrucible: All required reconciliation work is completed session_id = c2423ef0-728e-4895-997a-d9044c50948b
2092025-01-14T16:10:39.886ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = c2423ef0-728e-4895-997a-d9044c50948b
2102025-01-14T16:10:39.886ZINFOcrucible: 95474d24-b633-49f3-bc8a-d6ec632fff7e is now active with session: c2423ef0-728e-4895-997a-d9044c50948b session_id = c2423ef0-728e-4895-997a-d9044c50948b
2112025-01-14T16:10:39.886ZINFOcrucible: Set Active after no reconciliation session_id = c2423ef0-728e-4895-997a-d9044c50948b
2122025-01-14T16:10:39.886ZINFOcrucible: The guest has finished waiting for activation with:2
213 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
214 Loading write count information from file "/tmp/test_replay_verify.log"
215 Jan 14 16:10:39.919 INFO accepted connection, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
216 Main task has work to do, go find it
217 got dsc Stop(2)
218 stop 2
219 Jan 14 16:10:39.919 INFO request completed, latency_us: 193, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 81a8daf5-5f60-4843-b084-141f0de710fb, remote_addr: 127.0.0.1:61942, local_addr: 127.[8830] Got stop action so:false kr:false
220 0.0.1:9998
2212025-01-14T16:10:39.920ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
2222025-01-14T16:10:39.920ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
2232025-01-14T16:10:39.920ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
2242025-01-14T16:10:39.920ZINFOcrucible: client task is exiting = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
225 Jan 14 16:10:39.920 INFO request completed, latency_us: 239, response_code: 200, uri: /state/cid/2, method: GET, req_id: aab77604-3489-4443-9c0f-94caf6e52d0d, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
226 [8830] Exited with: ExitStatus(unix_wait_status(9))
227 [8830][2] reports Exit
228 Jan 14 16:10:43.923 INFO request completed, latency_us: 222, response_code: 200, uri: /state/cid/2, method: GET, req_id: 2d8eb6cb-fe7a-4672-897e-5ba540cb60d6, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
229 001/300 Read block 2519 len 12288
230 002/300 Write block 2029 len 24576 data: 2 2 2 2 2 2
231 003/300 Read block 4977 len 36864
232 004/300 Read block 3336 len 32768
233 005/300 Write block 3382 len 20480 data: 2 2 2 2 2
234 006/300 Read block 4772 len 36864
235 007/300 Flush
236 008/300 Write block 1645 len 32768 data: 2 2 2 2 2 2 2 2
237 009/300 Read block 2072 len 32768
238 010/300 Read block 3671 len 32768
239 011/300 Flush
240 012/300 Write block 187 len 8192 data: 2 2
241 013/300 Write block 480 len 20480 data: 2 2 2 2 2
242 014/300 Read block 609 len 32768
243 015/300 Read block 769 len 8192
244 016/300 Read block 871 len 24576
245 017/300 Read block 4492 len 36864
246 018/300 Read block 133 len 28672
247 019/300 Read block 902 len 28672
248 020/300 Read block 3096 len 20480
249 021/300 Write block 2004 len 28672 data: 2 2 2 2 2 2 2
250 022/300 Flush
251 023/300 Write block 2969 len 28672 data: 2 2 2 2 2 2 2
252 024/300 Read block 1133 len 40960
253 025/300 Read block 4331 len 28672
254 026/300 Write block 1006 len 4096 data: 2
255 027/300 Write block 3252 len 20480 data: 2 2 2 2 2
256 028/300 Read block 3941 len 28672
257 029/300 Flush
258 030/300 Read block 4569 len 8192
259 031/300 Write block 75 len 24576 data: 2 2 2 2 2 2
260 032/300 Read block 2892 len 36864
261 033/300 Read block 4857 len 28672
262 034/300 Write block 4050 len 4096 data: 2
263 035/300 Read block 4831 len 28672
264 036/300 Read block 1378 len 36864
265 037/300 Write block 2316 len 40960 data: 2 2 2 2 2 2 2 2 2 2
266 038/300 Read block 1345 len 32768
267 039/300 Read block 1846 len 20480
268 040/300 Write block 3365 len 40960 data: 2 2 2 2 2 2 2 2 2 2
269 041/300 Read block 2385 len 4096
270 042/300 Flush
271 043/300 Write block 3798 len 16384 data: 2 2 2 2
272 044/300 Write block 3656 len 32768 data: 2 2 2 2 2 2 2 2
273 045/300 Write block 1805 len 12288 data: 2 2 2
274 046/300 Flush
275 047/300 Flush
276 048/300 Read block 2070 len 8192
277 049/300 Read block 711 len 12288
278 050/300 Read block 1490 len 4096
279 051/300 Write block 1970 len 8192 data: 2 2
280 052/300 Write block 3281 len 32768 data: 2 2 2 2 2 2 2 2
281 053/300 Read block 941 len 16384
282 054/300 Read block 1011 len 20480
283 055/300 Read block 3797 len 8192
284 056/300 Write block 3710 len 36864 data: 2 2 2 2 2 2 2 2 2
285 057/300 Read block 55 len 32768
286 058/300 Read block 1537 len 12288
287 059/300 Write block 4598 len 20480 data: 2 2 2 2 2
288 060/300 Read block 453 len 20480
289 061/300 Read block 2783 len 28672
290 062/300 Write block 409 len 40960 data: 2 2 2 2 2 2 2 2 2 2
291 063/300 Flush
292 064/300 Write block 4956 len 16384 data: 2 2 2 2
293 065/300 Read block 2419 len 32768
294 066/300 Write block 4315 len 28672 data: 2 2 2 2 2 2 2
295 067/300 Write block 768 len 36864 data: 2 2 2 2 2 2 2 2 2
296 068/300 Flush
297 069/300 Read block 1869 len 20480
298 070/300 Read block 2277 len 40960
299 071/300 Read block 3576 len 4096
300 072/300 Write block 3510 len 36864 data: 2 2 2 2 2 2 2 2 2
301 073/300 Write block 2634 len 24576 data: 2 2 2 2 2 2
302 074/300 Write block 1610 len 32768 data: 2 2 2 2 2 2 2 2
303 075/300 Read block 4858 len 12288
304 076/300 Flush
305 077/300 Write block 3517 len 36864 data: 3 3 2 2 2 2 2 2 2
306 078/300 Read block 3260 len 16384
307 079/300 Read block 1248 len 32768
308 080/300 Write block 1797 len 28672 data: 2 2 2 2 2 2 2
309 081/300 Read block 3115 len 4096
310 082/300 Read block 828 len 16384
311 083/300 Read block 2529 len 4096
312 084/300 Write block 4872 len 28672 data: 2 2 2 2 2 2 2
313 085/300 Write block 4206 len 40960 data: 2 2 2 2 2 2 2 2 2 2
314 086/300 Read block 3374 len 24576
315 087/300 Read block 515 len 32768
316 088/300 Write block 3101 len 12288 data: 2 2 2
317 089/300 Read block 4334 len 32768
318 090/300 Flush
319 091/300 Write block 826 len 16384 data: 2 2 2 2
320 092/300 Write block 4631 len 16384 data: 2 2 2 2
321 093/300 Read block 1416 len 24576
322 094/300 Read block 903 len 40960
323 095/300 Read block 636 len 36864
324 096/300 Write block 2894 len 12288 data: 2 2 2
325 097/300 Read block 1645 len 8192
326 098/300 Read block 2654 len 4096
327 099/300 Write block 2044 len 32768 data: 2 2 2 2 2 2 2 2
328 100/300 Write block 3994 len 40960 data: 2 2 2 2 2 2 2 2 2 2
329 101/300 Read block 582 len 12288
330 102/300 Write block 4817 len 8192 data: 2 2
331 103/300 Read block 2931 len 4096
332 104/300 Read block 1692 len 28672
333 105/300 Read block 1544 len 28672
334 106/300 Read block 1128 len 12288
335 107/300 Write block 2715 len 24576 data: 2 2 2 2 2 2
336 108/300 Write block 757 len 36864 data: 2 2 2 2 2 2 2 2 2
337 109/300 Read block 3863 len 24576
338 110/300 Read block 3317 len 40960
339 111/300 Flush
340 112/300 Write block 387 len 4096 data: 2
341 113/300 Read block 4215 len 4096
342 114/300 Write block 4124 len 32768 data: 2 2 2 2 2 2 2 2
343 115/300 Write block 2559 len 24576 data: 2 2 2 2 2 2
344 116/300 Read block 652 len 4096
345 117/300 Read block 1527 len 40960
346 118/300 Flush
347 119/300 Read block 4059 len 36864
348 120/300 Flush
349 121/300 Read block 2466 len 4096
350 122/300 Read block 86 len 28672
351 123/300 Read block 4625 len 36864
352 124/300 Write block 2813 len 16384 data: 2 2 2 2
353 125/300 Write block 2512 len 12288 data: 2 2 2
354 126/300 Read block 544 len 8192
355 127/300 Write block 4390 len 40960 data: 2 2 2 2 2 2 2 2 2 2
356 128/300 Read block 1559 len 16384
357 129/300 Write block 497 len 20480 data: 2 2 2 2 2
358 130/300 Write block 3089 len 28672 data: 2 2 2 2 2 2 2
359 131/300 Write block 2268 len 4096 data: 2
360 132/300 Write block 1043 len 32768 data: 2 2 2 2 2 2 2 2
361 133/300 Read block 945 len 28672
362 134/300 Read block 1025 len 20480
363 135/300 Read block 999 len 12288
364 136/300 Read block 4780 len 8192
365 137/300 Read block 1461 len 40960
366 138/300 Read block 4914 len 8192
367 139/300 Write block 3446 len 4096 data: 2
368 140/300 Flush
369 141/300 Write block 1071 len 20480 data: 2 2 2 2 2
370 142/300 Write block 3478 len 8192 data: 2 2
371 143/300 Write block 4159 len 4096 data: 2
372 144/300 Read block 2890 len 8192
373 145/300 Write block 2810 len 32768 data: 2 2 2 3 3 3 3 2
374 146/300 Write block 3973 len 16384 data: 2 2 2 2
375 147/300 Read block 675 len 40960
376 148/300 Write block 376 len 8192 data: 2 2
377 149/300 Read block 1866 len 16384
378 150/300 Write block 2323 len 12288 data: 3 3 3
379 151/300 Write block 2725 len 32768 data: 2 2 2 2 2 2 2 2
380 152/300 Write block 799 len 32768 data: 2 2 2 2 2 2 2 2
381 153/300 Write block 1693 len 28672 data: 2 2 2 2 2 2 2
382 154/300 Write block 2678 len 28672 data: 2 2 2 2 2 2 2
383 155/300 Read block 1675 len 24576
384 156/300 Read block 3423 len 40960
385 157/300 Flush
386 158/300 Read block 764 len 32768
387 159/300 Read block 973 len 16384
388 160/300 Read block 1296 len 28672
389 161/300 Read block 2323 len 4096
390 162/300 Write block 2748 len 36864 data: 2 2 2 2 2 2 2 2 2
391 163/300 Read block 4302 len 36864
392 164/300 Read block 555 len 40960
393 165/300 Read block 3031 len 20480
394 166/300 Write block 2713 len 40960 data: 2 2 3 3 3 3 3 3 2 2
395 167/300 Write block 461 len 28672 data: 2 2 2 2 2 2 2
396 168/300 Write block 107 len 32768 data: 2 2 2 2 2 2 2 2
397 169/300 Read block 3921 len 16384
398 170/300 Read block 2601 len 4096
399 171/300 Write block 3310 len 4096 data: 2
400 172/300 Write block 704 len 40960 data: 2 2 2 2 2 2 2 2 2 2
401 173/300 Write block 1955 len 12288 data: 2 2 2
402 174/300 Read block 162 len 16384
403 175/300 Read block 3143 len 8192
404 176/300 Read block 3498 len 4096
405 177/300 Read block 1776 len 4096
406 178/300 Read block 1550 len 16384
407 179/300 Write block 23 len 16384 data: 2 2 2 2
408 180/300 Write block 2015 len 20480 data: 2 2 2 2 2
409 181/300 Read block 3030 len 16384
410 182/300 Write block 851 len 20480 data: 2 2 2 2 2
411 183/300 Write block 4663 len 16384 data: 2 2 2 2
412 184/300 Read block 4322 len 24576
413 185/300 Read block 2851 len 20480
414 186/300 Read block 752 len 36864
415 187/300 Write block 4006 len 8192 data: 2 2
416 188/300 Read block 618 len 8192
417 189/300 Write block 2927 len 36864 data: 2 2 2 2 2 2 2 2 2
418 190/300 Read block 91 len 16384
419 191/300 Flush
420 192/300 Read block 403 len 28672
421 193/300 Read block 815 len 4096
422 194/300 Read block 2871 len 40960
423 195/300 Read block 4165 len 32768
424 196/300 Write block 2537 len 4096 data: 2
425 197/300 Write block 2376 len 12288 data: 2 2 2
426 198/300 Write block 1304 len 8192 data: 2 2
427 199/300 Read block 823 len 32768
428 200/300 Write block 731 len 8192 data: 2 2
429 201/300 Write block 2759 len 20480 data: 2 2 2 2 2
430 202/300 Write block 1218 len 28672 data: 2 2 2 2 2 2 2
431 203/300 Write block 516 len 32768 data: 2 2 2 2 2 2 2 2
432 204/300 Write block 3005 len 12288 data: 2 2 2
433 205/300 Write block 2836 len 8192 data: 2 2
434 206/300 Write block 3063 len 12288 data: 2 2 2
435 207/300 Read block 2904 len 12288
436 208/300 Write block 4663 len 20480 data: 3 3 3 3 2
437 209/300 Write block 1198 len 8192 data: 2 2
438 210/300 Write block 1988 len 24576 data: 2 2 2 2 2 2
439 211/300 Flush
440 212/300 Read block 4218 len 40960
441 213/300 Write block 4543 len 28672 data: 2 2 2 2 2 2 2
442 214/300 Read block 835 len 32768
443 215/300 Flush
444 216/300 Write block 3356 len 24576 data: 2 2 2 2 2 2
445 217/300 Write block 4721 len 20480 data: 2 2 2 2 2
446 218/300 Write block 4138 len 40960 data: 2 2 2 2 2 2 2 2 2 2
447 219/300 Read block 3919 len 28672
448 220/300 Write block 3998 len 32768 data: 3 3 3 3 3 3 2 2
449 221/300 Flush
450 222/300 Read block 2587 len 36864
451 223/300 Read block 1765 len 16384
452 224/300 Flush
453 225/300 Write block 3253 len 16384 data: 3 3 3 3
454 226/300 Read block 1035 len 32768
455 227/300 Read block 526 len 32768
456 228/300 Write block 2810 len 4096 data: 3
457 229/300 Read block 1215 len 36864
458 230/300 Read block 3717 len 16384
459 231/300 Write block 3465 len 36864 data: 2 2 2 2 2 2 2 2 2
460 232/300 Write block 1364 len 16384 data: 2 2 2 2
461 233/300 Flush
462 234/300 Write block 21 len 36864 data: 2 2 3 3 3 3 2 2 2
463 235/300 Read block 4330 len 16384
464 236/300 Flush
465 237/300 Write block 3733 len 40960 data: 2 2 2 2 2 2 2 2 2 2
466 238/300 Flush
467 239/300 Write block 3396 len 32768 data: 2 2 2 2 2 2 2 2
468 240/300 Read block 106 len 32768
469 241/300 Write block 3844 len 20480 data: 2 2 2 2 2
470 242/300 Read block 685 len 20480
471 243/300 Read block 2058 len 28672
472 244/300 Read block 2388 len 20480
473 245/300 Read block 3674 len 8192
474 246/300 Read block 1094 len 8192
475 247/300 Read block 1792 len 24576
476 248/300 Read block 4032 len 8192
477 249/300 Read block 1722 len 12288
478 250/300 Write block 4077 len 40960 data: 2 2 2 2 2 2 2 2 2 2
479 251/300 Flush
480 252/300 Write block 2767 len 36864 data: 2 2 2 2 2 2 2 2 2
481 253/300 Read block 1792 len 12288
482 254/300 Read block 3636 len 36864
483 255/300 Write block 4324 len 24576 data: 2 2 2 2 2 2
484 256/300 Write block 4538 len 24576 data: 2 2 2 2 2 3
485 257/300 Read block 2407 len 32768
486 258/300 Flush
487 259/300 Read block 4462 len 36864
488 260/300 Read block 353 len 20480
489 261/300 Flush
490 262/300 Write block 1472 len 40960 data: 2 2 2 2 2 2 2 2 2 2
491 263/300 Read block 4882 len 36864
492 264/300 Flush
493 265/300 Read block 3748 len 12288
494 266/300 Read block 4572 len 16384
495 267/300 Read block 3592 len 24576
496 268/300 Read block 3851 len 40960
497 269/300 Write block 4176 len 36864 data: 2 2 2 2 2 2 2 2 2
498 270/300 Read block 1363 len 36864
499 271/300 Write block 1770 len 24576 data: 2 2 2 2 2 2
500 272/300 Read block 3911 len 4096
501 273/300 Flush
502 274/300 Write block 3261 len 40960 data: 2 2 2 2 2 2 2 2 2 2
503 275/300 Read block 825 len 4096
504 276/300 Read block 4293 len 36864
505 277/300 Read block 1488 len 28672
506 278/300 Read block 4317 len 28672
507 279/300 Read block 2463 len 40960
508 280/300 Read block 3387 len 28672
509 281/300 Flush
510 282/300 Write block 4485 len 20480 data: 2 2 2 2 2
511 283/300 Read block 2962 len 16384
512 284/300 Read block 813 len 16384
513 285/300 Flush
514 286/300 Write block 4980 len 24576 data: 2 2 2 2 2 2
515 287/300 Write block 1465 len 36864 data: 2 2 2 2 2 2 2 3 3
516 288/300 Flush
517 289/300 Write block 2489 len 20480 data: 2 2 2 2 2
518 290/300 Write block 3662 len 28672 data: 3 3 2 2 2 2 2
519 291/300 Read block 4033 len 32768
520 292/300 Write block 2640 len 24576 data: 2 2 2 2 2 2
521 293/300 Write block 4210 len 16384 data: 3 3 3 3
522 294/300 Read block 4180 len 4096
523 295/300 Write block 4783 len 32768 data: 2 2 2 2 2 2 2 2
524 296/300 Flush
525 297/300 Write block 1430 len 12288 data: 2 2 2
526 298/300 Write block 819 len 8192 data: 2 2
527 299/300 Read block 2725 len 40960
528 300/300 Write block 1148 len 20480 data: 2 2 2 2 2
529 301/300 Read block 3695 len 8192
530 Main task has work to do, go find it
531 got dsc Start(2)
532 start 2
533 Jan 14 16:10:44.161 INFO request completed, latency_us: 199, response_code: 204, uri: /start/cid/2, method: POST, req_id: f42addda-e74b-4f77-adfb-1dc56156027e, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
534 Replay: started 2, returned:Ok(())
535 ----------------------------------------------------------------
536 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
537 States: Active Active Offline
538 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
539 1000 Acked Read 3 Done Done Sent false
540 1001 Acked Write 6 Done Done Sent false
541 1002 Acked Read 9 Done Done Sent false
542 1003 Acked Read 8 Done Done Sent false
543 1004 Acked Write 5 Done Done Sent false
544 1005 Acked Read 9 Done Done Sent false
545 1006 Acked Flush 0 Done Done Sent false
546 1007 Acked Write 8 Done Done Sent false
547 1008 Acked Read 8 Done Done Sent false
548 1009 Acked Read 8 Done Done Sent false
549 1010 Acked Flush 0 Done Done Sent false
550 1011 Acked Write 2 Done Done Sent false
551 1012 Acked Write 5 Done Done Sent false
552 1013 Acked Read 8 Done Done Sent false
553 1014 Acked Read 2 Done Done Sent false
554 1015 Acked Read 6 Done Done Sent false
555 1016 Acked Read 9 Done Done Sent false
556 1017 Acked Read 7 Done Done Sent false
557 1018 Acked Read 7 Done Done Sent false
558 1019 Acked Read 5 Done Done Sent false
559 1020 Acked Write 7 Done Done Sent false
560 1021 Acked Flush 0 Done Done Sent false
561 1022 Acked Write 7 Done Done Sent false
562 1023 Acked Read 10 Done Done Sent false
563 1024 Acked Read 7 Done Done Sent false
564 1025 Acked Write 1 Done Done Sent false
565 1026 Acked Write 5 Done Done Sent false
566 1027 Acked Read 7 Done Done Sent false
567 1028 Acked Flush 0 Done Done Sent false
568 1029 Acked Read 2 Done Done Sent false
569 1030 Acked Write 6 Done Done Sent false
570 1031 Acked Read 9 Done Done Sent false
571 1032 Acked Read 7 Done Done Sent false
572 1033 Acked Write 1 Done Done Sent false
573 1034 Acked Read 7 Done Done Sent false
574 1035 Acked Read 9 Done Done Sent false
575 1036 Acked Write 10 Done Done Sent false
576 1037 Acked Read 8 Done Done Sent false
577 1038 Acked Read 5 Done Done Sent false
578 1039 Acked Write 10 Done Done Sent false
579 1040 Acked Read 1 Done Done Sent false
580 1041 Acked Flush 0 Done Done Sent false
581 1042 Acked Write 4 Done Done Sent false
582 1043 Acked Write 8 Done Done Sent false
583 1044 Acked Write 3 Done Done Sent false
584 1045 Acked Flush 0 Done Done Sent false
585 1046 Acked Flush 0 Done Done Sent false
586 1047 Acked Read 2 Done Done Sent false
587 1048 Acked Read 3 Done Done Sent false
588 1049 Acked Read 1 Done Done Sent false
589 1050 Acked Write 2 Done Done Sent false
590 1051 Acked Write 8 Done Done Sent false
591 1052 Acked Read 4 Done Done Sent false
592 1053 Acked Read 5 Done Done Sent false
593 1054 Acked Read 2 Done Done Sent false
594 1055 Acked Write 9 Done Done Sent false
595 1056 Acked Read 8 Done Done Sent false
596 1057 Acked Read 3 Done Done Sent false
597 1058 Acked Write 5 Done Done Sent false
598 1059 Acked Read 5 Done Done Sent false
599 1060 Acked Read 7 Done Done Sent false
600 1061 Acked Write 10 Done Done Sent false
601 1062 Acked Flush 0 Done Done Sent false
602 1063 Acked Write 4 Done Done Sent false
603 1064 Acked Read 8 Done Done Sent false
604 1065 Acked Write 7 Done Done Sent false
605 1066 Acked Write 9 Done Done Sent false
606 1067 Acked Flush 0 Done Done Sent false
607 1068 Acked Read 5 Done Done Sent false
608 1069 Acked Read 10 Done Done Sent false
609 1070 Acked Read 1 Done Done Sent false
610 1071 Acked Write 9 Done Done Sent false
611 1072 Acked Write 6 Done Done Sent false
612 1073 Acked Write 8 Done Done Sent false
613 1074 Acked Read 3 Done Done Sent false
614 1075 Acked Flush 0 Done Done Sent false
615 1076 Acked Write 9 Done Done Sent false
616 1077 Acked Read 4 Done Done Sent false
617 1078 Acked Read 8 Done Done Sent false
618 1079 Acked Write 7 Done Done Sent false
619 1080 Acked Read 1 Done Done Sent false
620 1081 Acked Read 4 Done Done Sent false
621 1082 Acked Read 1 Done Done Sent false
622 1083 Acked Write 7 Done Done Sent false
623 1084 Acked Write 10 Done Done Sent false
624 1085 Acked Read 6 Done Done Sent false
625 1086 Acked Read 8 Done Done Sent false
626 1087 Acked Write 3 Done Done Sent false
627 1088 Acked Read 8 Done Done Sent false
628 1089 Acked Flush 0 Done Done Sent false
629 1090 Acked Write 4 Done Done Sent false
630 1091 Acked Write 4 Done Done Sent false
631 1092 Acked Read 6 Done Done Sent false
632 1093 Acked Read 10 Done Done Sent false
633 1094 Acked Read 9 Done Done Sent false
634 1095 Acked Write 3 Done Done Sent false
635 1096 Acked Read 2 Done Done Sent false
636 1097 Acked Read 1 Done Done Sent false
637 1098 Acked Write 8 Done Done Sent false
638 1099 Acked Write 10 Done Done Sent false
639 1100 Acked Read 3 Done Done Sent false
640 1101 Acked Write 2 Done Done Sent false
641 1102 Acked Read 1 Done Done Sent false
642 1103 Acked Read 7 Done Done Sent false
643 1104 Acked Read 7 Done Done Sent false
644 1105 Acked Read 3 Done Done Sent false
645 1106 Acked Write 6 Done Done Sent false
646 1107 Acked Write 9 Done Done Sent false
647 1108 Acked Read 6 Done Done Sent false
648 1109 Acked Read 10 Done Done Sent false
649 1110 Acked Flush 0 Done Done Sent false
650 1111 Acked Write 1 Done Done Sent false
651 1112 Acked Read 1 Done Done Sent false
652 1113 Acked Write 8 Done Done Sent false
653 1114 Acked Write 6 Done Done Sent false
654 1115 Acked Read 1 Done Done Sent false
655 1116 Acked Read 10 Done Done Sent false
656 1117 Acked Flush 0 Done Done Sent false
657 1118 Acked Read 9 Done Done Sent false
658 1119 Acked Flush 0 Done Done Sent false
659 1120 Acked Read 1 Done Done Sent false
660 1121 Acked Read 7 Done Done Sent false
661 1122 Acked Read 9 Done Done Sent false
662 1123 Acked Write 4 Done Done Sent false
663 1124 Acked Write 3 Done Done Sent false
664 1125 Acked Read 2 Done Done Sent false
665 1126 Acked Write 10 Done Done Sent false
666 1127 Acked Read 4 Done Done Sent false
667 1128 Acked Write 5 Done Done Sent false
668 1129 Acked Write 7 Done Done Sent false
669 1130 Acked Write 1 Done Done Sent false
670 1131 Acked Write 8 Done Done Sent false
671 1132 Acked Read 7 Done Done Sent false
672 1133 Acked Read 5 Done Done Sent false
673 1134 Acked Read 3 Done Done Sent false
674 1135 Acked Read 2 Done Done Sent false
675 1136 Acked Read 10 Done Done Sent false
676 1137 Acked Read 2 Done Done Sent false
677 1138 Acked Write 1 Done Done Sent false
678 1139 Acked Flush 0 Done Done Sent false
679 1140 Acked Write 5 Done Done Sent false
680 1141 Acked Write 2 Done Done Sent false
681 1142 Acked Write 1 Done Done Sent false
682 1143 Acked Read 2 Done Done Sent false
683 1144 Acked Write 8 Done Done Sent false
684 1145 Acked Write 4 Done Done Sent false
685 1146 Acked Read 10 Done Done Sent false
686 1147 Acked Write 2 Done Done Sent false
687 1148 Acked Read 4 Done Done Sent false
688 1149 Acked Write 3 Done Done Sent false
689 1150 Acked Write 8 Done Done Sent false
690 1151 Acked Write 8 Done Done Sent false
691 1152 Acked Write 7 Done Done Sent false
692 1153 Acked Write 7 Done Done Sent false
693 1154 Acked Read 6 Done Done Sent false
694 1155 Acked Read 10 Done Done Sent false
695 1156 Acked Flush 0 Done Done Sent false
696 1157 Acked Read 8 Done Done Sent false
697 1158 Acked Read 4 Done Done Sent false
698 1159 Acked Read 7 Done Done Sent false
699 1160 Acked Read 1 Done Done Sent false
700 1161 Acked Write 9 Done Done Sent false
701 1162 Acked Read 9 Done Done Sent false
702 1163 Acked Read 10 Done Done Sent false
703 1164 Acked Read 5 Done Done Sent false
704 1165 Acked Write 10 Done Done Sent false
705 1166 Acked Write 7 Done Done Sent false
706 1167 Acked Write 8 Done Done Sent false
707 1168 Acked Read 4 Done Done Sent false
708 1169 Acked Read 1 Done Done Sent false
709 1170 Acked Write 1 Done Done Sent false
710 1171 Acked Write 10 Done Done Sent false
711 1172 Acked Write 3 Done Done Sent false
712 1173 Acked Read 4 Done Done Sent false
713 1174 Acked Read 2 Done Done Sent false
714 1175 Acked Read 1 Done Done Sent false
715 1176 Acked Read 1 Done Done Sent false
716 1177 Acked Read 4 Done Done Sent false
717 1178 Acked Write 4 Done Done Sent false
718 1179 Acked Write 5 Done Done Sent false
719 1180 Acked Read 4 Done Done Sent false
720 1181 Acked Write 5 Done Done Sent false
721 1182 Acked Write 4 Done Done Sent false
722 1183 Acked Read 6 Done Done Sent false
723 1184 Acked Read 5 Done Done Sent false
724 1185 Acked Read 9 Done Done Sent false
725 1186 Acked Write 2 Done Done Sent false
726 1187 Acked Read 2 Done Done Sent false
727 1188 Acked Write 9 Done Done Sent false
728 1189 Acked Read 4 Done Done Sent false
729 1190 Acked Flush 0 Done Done Sent false
730 1191 Acked Read 7 Done Done Sent false
731 1192 Acked Read 1 Done Done Sent false
732 1193 Acked Read 10 Done Done Sent false
733 1194 Acked Read 8 Done Done Sent false
734 1195 Acked Write 1 Done Done Sent false
735 1196 Acked Write 3 Done Done Sent false
736 1197 Acked Write 2 Done Done Sent false
737 1198 Acked Read 8 Done Done Sent false
738 1199 Acked Write 2 Done Done Sent false
739 1200 Acked Write 5 Done Done Sent false
740 1201 Acked Write 7 Done Done Sent false
741 1202 Acked Write 8 Done Done Sent false
742 1203 Acked Write 3 Done Done Sent false
743 1204 Acked Write 2 Done Done Sent false
744 1205 Acked Write 3 Done Done Sent false
745 1206 Acked Read 3 Done Done Sent false
746 1207 Acked Write 5 Done Done Sent false
747 1208 Acked Write 2 Done Done Sent false
748 1209 Acked Write 6 Done Done Sent false
749 1210 Acked Flush 0 Done Done Sent false
750 1211 Acked Read 10 Done Done Sent false
751 1212 Acked Write 7 Done Done Sent false
752 1213 Acked Read 8 Done Done Sent false
753 1214 Acked Flush 0 Done Done Sent false
754 1215 Acked Write 6 Done Done Sent false
755 1216 Acked Write 5 Done Done Sent false
756 1217 Acked Write 10 Done Done Sent false
757 1218 Acked Read 7 Done Done Sent false
758 1219 Acked Write 8 Done Done Sent false
759 1220 Acked Flush 0 Done Done Sent false
760 1221 Acked Read 9 Done Done Sent false
761 1222 Acked Read 4 Done Done Sent false
762 1223 Acked Flush 0 Done Done Sent false
763 1224 Acked Write 4 Done Done Sent false
764 1225 Acked Read 8 Done Done Sent false
765 1226 Acked Read 8 Done Done Sent false
766 1227 Acked Write 1 Done Done Sent false
767 1228 Acked Read 9 Done Done Sent false
768 1229 Acked Read 4 Done Done Sent false
769 1230 Acked Write 9 Done Done Sent false
770 1231 Acked Write 4 Done Done Sent false
771 1232 Acked Flush 0 Done Done Sent false
772 1233 Acked Write 9 Done Done Sent false
773 1234 Acked Read 4 Done Done Sent false
774 1235 Acked Flush 0 Done Done Sent false
775 1236 Acked Write 10 Done Done Sent false
776 1237 Acked Flush 0 Done Done Sent false
777 1238 Acked Write 8 Done Done Sent false
778 1239 Acked Read 8 Done Done Sent false
779 1240 Acked Write 5 Done Done Sent false
780 1241 Acked Read 5 Done Done Sent false
781 1242 Acked Read 7 Done Done Sent false
782 1243 Acked Read 5 Done Done Sent false
783 1244 Acked Read 2 Done Done Sent false
784 1245 Acked Read 2 Done Done Sent false
785 1246 Acked Read 6 Done Done Sent false
786 1247 Acked Read 2 Done Done Sent false
787 1248 Acked Read 3 Done Done Sent false
788 1249 Acked Write 10 Done Done Sent false
789 1250 Acked Flush 0 Done Done Sent false
790 1251 Acked Write 9 Done Done Sent false
791 1252 Acked Read 3 Done Done Sent false
792 1253 Acked Read 9 Done Done Sent false
793 1254 Acked Write 6 Done Done Sent false
794 1255 Acked Write 6 Done Done Sent false
795 1256 Acked Read 8 Done Done Sent false
796 1257 Acked Flush 0 Done Done Sent false
797 1258 Acked Read 9 Done Done Sent false
798 1259 Acked Read 5 Done Done Sent false
799 1260 Acked Flush 0 Done Done Sent false
800 1261 Acked Write 10 Done Done Sent false
801 1262 Acked Read 9 Done Done Sent false
802 1263 Acked Flush 0 Done Done Sent false
803 1264 Acked Read 3 Done Done Sent false
804 1265 Acked Read 4 Done Done Sent false
805 1266 Acked Read 6 Done Done Sent false
806 1267 Acked Read 10 Done Done Sent false
807 1268 Acked Write 9 Done Done Sent false
808 1269 Acked Read 9 Done Done Sent false
809 1270 Acked Write 6 Done Done Sent false
810 1271 Acked Read 1 Done Done Sent false
811 1272 Acked Flush 0 Done Done Sent false
812 1273 Acked Write 10 Done Done Sent false
813 1274 Acked Read 1 Done Done Sent false
814 1275 Acked Read 9 Done Done Sent false
815 1276 Acked Read 7 Done Done Sent false
816 1277 Acked Read 7 Done Done Sent false
817 1278 Acked Read 10 Done Done Sent false
818 1279 Acked Read 7 Done Done Sent false
819 1280 Acked Flush 0 Done Done Sent false
820 1281 Acked Write 5 Done Done Sent false
821 1282 Acked Read 4 Done Done Sent false
822 1283 Acked Read 4 Done Done Sent false
823 1284 Acked Flush 0 Done Done Sent false
824 1285 Acked Write 6 Done Done Sent false
825 1286 Acked Write 9 Done Done Sent false
826 1287 Acked Flush 0 Done Done Sent false
827 1288 Acked Write 5 Done Done Sent false
828 1289 Acked Write 7 Done Done Sent false
829 1290 Acked Read 8 Done Done Sent false
830 1291 Acked Write 6 Done Done Sent false
831 1292 Acked Write 4 Done Done Sent false
832 1293 Acked Read 1 Done Done Sent false
833 1294 Acked Write 8 Done Done Sent false
834 1295 Acked Flush 0 Done Done Sent false
835 1296 Acked Write 3 Done Done Sent false
836 1297 Acked Write 2 Done Done Sent false
837 1298 Acked Read 10 Done Done Sent false
838 1299 Acked Write 5 Done Done Sent false
839 1300 Acked Read 2 Done Done Sent false
840 STATES DS:0 DS:1 DS:2 TOTAL
841 Sent 0 0 301 301
842 Done 301 301 0 602
843 Skipped 0 0 0 0
844 Error 0 0 0 0
845 Last Flush: 1295 1295 0
846 Downstairs last five completed:
847 Upstairs last five completed: 1300 1299 1298 1297 1296
848 CLIENT: Up:0 ds:301 act:2
849 [8830] got start action
850 [8830] I am going to restart
851 Starting downstairs at port 8830
852 Make output file at "/tmp/dsc/downstairs-8830.txt"
853 [8830][2] reports Starting
854 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1203)
855 [8830][2] reports Running
856 ----------------------------------------------------------------
857 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
858 States: Active Active Offline
859 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
860 1000 Acked Read 3 Done Done Sent false
861 1001 Acked Write 6 Done Done Sent false
862 1002 Acked Read 9 Done Done Sent false
863 1003 Acked Read 8 Done Done Sent false
864 1004 Acked Write 5 Done Done Sent false
865 1005 Acked Read 9 Done Done Sent false
866 1006 Acked Flush 0 Done Done Sent false
867 1007 Acked Write 8 Done Done Sent false
868 1008 Acked Read 8 Done Done Sent false
869 1009 Acked Read 8 Done Done Sent false
870 1010 Acked Flush 0 Done Done Sent false
871 1011 Acked Write 2 Done Done Sent false
872 1012 Acked Write 5 Done Done Sent false
873 1013 Acked Read 8 Done Done Sent false
874 1014 Acked Read 2 Done Done Sent false
875 1015 Acked Read 6 Done Done Sent false
876 1016 Acked Read 9 Done Done Sent false
877 1017 Acked Read 7 Done Done Sent false
878 1018 Acked Read 7 Done Done Sent false
879 1019 Acked Read 5 Done Done Sent false
880 1020 Acked Write 7 Done Done Sent false
881 1021 Acked Flush 0 Done Done Sent false
882 1022 Acked Write 7 Done Done Sent false
883 1023 Acked Read 10 Done Done Sent false
884 1024 Acked Read 7 Done Done Sent false
885 1025 Acked Write 1 Done Done Sent false
886 1026 Acked Write 5 Done Done Sent false
887 1027 Acked Read 7 Done Done Sent false
888 1028 Acked Flush 0 Done Done Sent false
889 1029 Acked Read 2 Done Done Sent false
890 1030 Acked Write 6 Done Done Sent false
891 1031 Acked Read 9 Done Done Sent false
892 1032 Acked Read 7 Done Done Sent false
893 1033 Acked Write 1 Done Done Sent false
894 1034 Acked Read 7 Done Done Sent false
895 1035 Acked Read 9 Done Done Sent false
896 1036 Acked Write 10 Done Done Sent false
897 1037 Acked Read 8 Done Done Sent false
898 1038 Acked Read 5 Done Done Sent false
899 1039 Acked Write 10 Done Done Sent false
900 1040 Acked Read 1 Done Done Sent false
901 1041 Acked Flush 0 Done Done Sent false
902 1042 Acked Write 4 Done Done Sent false
903 1043 Acked Write 8 Done Done Sent false
904 1044 Acked Write 3 Done Done Sent false
905 1045 Acked Flush 0 Done Done Sent false
906 1046 Acked Flush 0 Done Done Sent false
907 1047 Acked Read 2 Done Done Sent false
908 1048 Acked Read 3 Done Done Sent false
909 1049 Acked Read 1 Done Done Sent false
910 1050 Acked Write 2 Done Done Sent false
911 1051 Acked Write 8 Done Done Sent false
912 1052 Acked Read 4 Done Done Sent false
913 1053 Acked Read 5 Done Done Sent false
914 1054 Acked Read 2 Done Done Sent false
915 1055 Acked Write 9 Done Done Sent false
916 1056 Acked Read 8 Done Done Sent false
917 1057 Acked Read 3 Done Done Sent false
918 1058 Acked Write 5 Done Done Sent false
919 1059 Acked Read 5 Done Done Sent false
920 1060 Acked Read 7 Done Done Sent false
921 1061 Acked Write 10 Done Done Sent false
922 1062 Acked Flush 0 Done Done Sent false
923 1063 Acked Write 4 Done Done Sent false
924 1064 Acked Read 8 Done Done Sent false
925 1065 Acked Write 7 Done Done Sent false
926 1066 Acked Write 9 Done Done Sent false
927 1067 Acked Flush 0 Done Done Sent false
928 1068 Acked Read 5 Done Done Sent false
929 1069 Acked Read 10 Done Done Sent false
930 1070 Acked Read 1 Done Done Sent false
931 1071 Acked Write 9 Done Done Sent false
932 1072 Acked Write 6 Done Done Sent false
933 1073 Acked Write 8 Done Done Sent false
934 1074 Acked Read 3 Done Done Sent false
935 1075 Acked Flush 0 Done Done Sent false
936 1076 Acked Write 9 Done Done Sent false
937 1077 Acked Read 4 Done Done Sent false
938 1078 Acked Read 8 Done Done Sent false
939 1079 Acked Write 7 Done Done Sent false
940 1080 Acked Read 1 Done Done Sent false
941 1081 Acked Read 4 Done Done Sent false
942 1082 Acked Read 1 Done Done Sent false
943 1083 Acked Write 7 Done Done Sent false
944 1084 Acked Write 10 Done Done Sent false
945 1085 Acked Read 6 Done Done Sent false
946 1086 Acked Read 8 Done Done Sent false
947 1087 Acked Write 3 Done Done Sent false
948 1088 Acked Read 8 Done Done Sent false
949 1089 Acked Flush 0 Done Done Sent false
950 1090 Acked Write 4 Done Done Sent false
951 1091 Acked Write 4 Done Done Sent false
952 1092 Acked Read 6 Done Done Sent false
953 1093 Acked Read 10 Done Done Sent false
954 1094 Acked Read 9 Done Done Sent false
955 1095 Acked Write 3 Done Done Sent false
956 1096 Acked Read 2 Done Done Sent false
957 1097 Acked Read 1 Done Done Sent false
958 1098 Acked Write 8 Done Done Sent false
959 1099 Acked Write 10 Done Done Sent false
960 1100 Acked Read 3 Done Done Sent false
961 1101 Acked Write 2 Done Done Sent false
962 1102 Acked Read 1 Done Done Sent false
963 1103 Acked Read 7 Done Done Sent false
964 1104 Acked Read 7 Done Done Sent false
965 1105 Acked Read 3 Done Done Sent false
966 1106 Acked Write 6 Done Done Sent false
967 1107 Acked Write 9 Done Done Sent false
968 1108 Acked Read 6 Done Done Sent false
969 1109 Acked Read 10 Done Done Sent false
970 1110 Acked Flush 0 Done Done Sent false
971 1111 Acked Write 1 Done Done Sent false
972 1112 Acked Read 1 Done Done Sent false
973 1113 Acked Write 8 Done Done Sent false
974 1114 Acked Write 6 Done Done Sent false
975 1115 Acked Read 1 Done Done Sent false
976 1116 Acked Read 10 Done Done Sent false
977 1117 Acked Flush 0 Done Done Sent false
978 1118 Acked Read 9 Done Done Sent false
979 1119 Acked Flush 0 Done Done Sent false
980 1120 Acked Read 1 Done Done Sent false
981 1121 Acked Read 7 Done Done Sent false
982 1122 Acked Read 9 Done Done Sent false
983 1123 Acked Write 4 Done Done Sent false
984 1124 Acked Write 3 Done Done Sent false
985 1125 Acked Read 2 Done Done Sent false
986 1126 Acked Write 10 Done Done Sent false
987 1127 Acked Read 4 Done Done Sent false
988 1128 Acked Write 5 Done Done Sent false
989 1129 Acked Write 7 Done Done Sent false
990 1130 Acked Write 1 Done Done Sent false
991 1131 Acked Write 8 Done Done Sent false
992 1132 Acked Read 7 Done Done Sent false
993 1133 Acked Read 5 Done Done Sent false
994 1134 Acked Read 3 Done Done Sent false
995 1135 Acked Read 2 Done Done Sent false
996 1136 Acked Read 10 Done Done Sent false
997 1137 Acked Read 2 Done Done Sent false
998 1138 Acked Write 1 Done Done Sent false
999 1139 Acked Flush 0 Done Done Sent false
1000 1140 Acked Write 5 Done Done Sent false
1001 1141 Acked Write 2 Done Done Sent false
1002 1142 Acked Write 1 Done Done Sent false
1003 1143 Acked Read 2 Done Done Sent false
1004 1144 Acked Write 8 Done Done Sent false
1005 1145 Acked Write 4 Done Done Sent false
1006 1146 Acked Read 10 Done Done Sent false
1007 1147 Acked Write 2 Done Done Sent false
1008 1148 Acked Read 4 Done Done Sent false
1009 1149 Acked Write 3 Done Done Sent false
1010 1150 Acked Write 8 Done Done Sent false
1011 1151 Acked Write 8 Done Done Sent false
1012 1152 Acked Write 7 Done Done Sent false
1013 1153 Acked Write 7 Done Done Sent false
1014 1154 Acked Read 6 Done Done Sent false
1015 1155 Acked Read 10 Done Done Sent false
1016 1156 Acked Flush 0 Done Done Sent false
1017 1157 Acked Read 8 Done Done Sent false
1018 1158 Acked Read 4 Done Done Sent false
1019 1159 Acked Read 7 Done Done Sent false
1020 1160 Acked Read 1 Done Done Sent false
1021 1161 Acked Write 9 Done Done Sent false
1022 1162 Acked Read 9 Done Done Sent false
1023 1163 Acked Read 10 Done Done Sent false
1024 1164 Acked Read 5 Done Done Sent false
1025 1165 Acked Write 10 Done Done Sent false
1026 1166 Acked Write 7 Done Done Sent false
1027 1167 Acked Write 8 Done Done Sent false
1028 1168 Acked Read 4 Done Done Sent false
1029 1169 Acked Read 1 Done Done Sent false
1030 1170 Acked Write 1 Done Done Sent false
1031 1171 Acked Write 10 Done Done Sent false
1032 1172 Acked Write 3 Done Done Sent false
1033 1173 Acked Read 4 Done Done Sent false
1034 1174 Acked Read 2 Done Done Sent false
1035 1175 Acked Read 1 Done Done Sent false
1036 1176 Acked Read 1 Done Done Sent false
1037 1177 Acked Read 4 Done Done Sent false
1038 1178 Acked Write 4 Done Done Sent false
1039 1179 Acked Write 5 Done Done Sent false
1040 1180 Acked Read 4 Done Done Sent false
1041 1181 Acked Write 5 Done Done Sent false
1042 1182 Acked Write 4 Done Done Sent false
1043 1183 Acked Read 6 Done Done Sent false
1044 1184 Acked Read 5 Done Done Sent false
1045 1185 Acked Read 9 Done Done Sent false
1046 1186 Acked Write 2 Done Done Sent false
1047 1187 Acked Read 2 Done Done Sent false
1048 1188 Acked Write 9 Done Done Sent false
1049 1189 Acked Read 4 Done Done Sent false
1050 1190 Acked Flush 0 Done Done Sent false
1051 1191 Acked Read 7 Done Done Sent false
1052 1192 Acked Read 1 Done Done Sent false
1053 1193 Acked Read 10 Done Done Sent false
1054 1194 Acked Read 8 Done Done Sent false
1055 1195 Acked Write 1 Done Done Sent false
1056 1196 Acked Write 3 Done Done Sent false
1057 1197 Acked Write 2 Done Done Sent false
1058 1198 Acked Read 8 Done Done Sent false
1059 1199 Acked Write 2 Done Done Sent false
1060 1200 Acked Write 5 Done Done Sent false
1061 1201 Acked Write 7 Done Done Sent false
1062 1202 Acked Write 8 Done Done Sent false
1063 1203 Acked Write 3 Done Done Sent false
1064 1204 Acked Write 2 Done Done Sent false
1065 1205 Acked Write 3 Done Done Sent false
1066 1206 Acked Read 3 Done Done Sent false
1067 1207 Acked Write 5 Done Done Sent false
1068 1208 Acked Write 2 Done Done Sent false
1069 1209 Acked Write 6 Done Done Sent false
1070 1210 Acked Flush 0 Done Done Sent false
1071 1211 Acked Read 10 Done Done Sent false
1072 1212 Acked Write 7 Done Done Sent false
1073 1213 Acked Read 8 Done Done Sent false
1074 1214 Acked Flush 0 Done Done Sent false
1075 1215 Acked Write 6 Done Done Sent false
1076 1216 Acked Write 5 Done Done Sent false
1077 1217 Acked Write 10 Done Done Sent false
1078 1218 Acked Read 7 Done Done Sent false
1079 1219 Acked Write 8 Done Done Sent false
1080 1220 Acked Flush 0 Done Done Sent false
1081 1221 Acked Read 9 Done Done Sent false
1082 1222 Acked Read 4 Done Done Sent false
1083 1223 Acked Flush 0 Done Done Sent false
1084 1224 Acked Write 4 Done Done Sent false
1085 1225 Acked Read 8 Done Done Sent false
1086 1226 Acked Read 8 Done Done Sent false
1087 1227 Acked Write 1 Done Done Sent false
1088 1228 Acked Read 9 Done Done Sent false
1089 1229 Acked Read 4 Done Done Sent false
1090 1230 Acked Write 9 Done Done Sent false
1091 1231 Acked Write 4 Done Done Sent false
1092 1232 Acked Flush 0 Done Done Sent false
1093 1233 Acked Write 9 Done Done Sent false
1094 1234 Acked Read 4 Done Done Sent false
1095 1235 Acked Flush 0 Done Done Sent false
1096 1236 Acked Write 10 Done Done Sent false
1097 1237 Acked Flush 0 Done Done Sent false
1098 1238 Acked Write 8 Done Done Sent false
1099 1239 Acked Read 8 Done Done Sent false
1100 1240 Acked Write 5 Done Done Sent false
1101 1241 Acked Read 5 Done Done Sent false
1102 1242 Acked Read 7 Done Done Sent false
1103 1243 Acked Read 5 Done Done Sent false
1104 1244 Acked Read 2 Done Done Sent false
1105 1245 Acked Read 2 Done Done Sent false
1106 1246 Acked Read 6 Done Done Sent false
1107 1247 Acked Read 2 Done Done Sent false
1108 1248 Acked Read 3 Done Done Sent false
1109 1249 Acked Write 10 Done Done Sent false
1110 1250 Acked Flush 0 Done Done Sent false
1111 1251 Acked Write 9 Done Done Sent false
1112 1252 Acked Read 3 Done Done Sent false
1113 1253 Acked Read 9 Done Done Sent false
1114 1254 Acked Write 6 Done Done Sent false
1115 1255 Acked Write 6 Done Done Sent false
1116 1256 Acked Read 8 Done Done Sent false
1117 1257 Acked Flush 0 Done Done Sent false
1118 1258 Acked Read 9 Done Done Sent false
1119 1259 Acked Read 5 Done Done Sent false
1120 1260 Acked Flush 0 Done Done Sent false
1121 1261 Acked Write 10 Done Done Sent false
1122 1262 Acked Read 9 Done Done Sent false
1123 1263 Acked Flush 0 Done Done Sent false
1124 1264 Acked Read 3 Done Done Sent false
1125 1265 Acked Read 4 Done Done Sent false
1126 1266 Acked Read 6 Done Done Sent false
1127 1267 Acked Read 10 Done Done Sent false
1128 1268 Acked Write 9 Done Done Sent false
1129 1269 Acked Read 9 Done Done Sent false
1130 1270 Acked Write 6 Done Done Sent false
1131 1271 Acked Read 1 Done Done Sent false
1132 1272 Acked Flush 0 Done Done Sent false
1133 1273 Acked Write 10 Done Done Sent false
1134 1274 Acked Read 1 Done Done Sent false
1135 1275 Acked Read 9 Done Done Sent false
1136 1276 Acked Read 7 Done Done Sent false
1137 1277 Acked Read 7 Done Done Sent false
1138 1278 Acked Read 10 Done Done Sent false
1139 1279 Acked Read 7 Done Done Sent false
1140 1280 Acked Flush 0 Done Done Sent false
1141 1281 Acked Write 5 Done Done Sent false
1142 1282 Acked Read 4 Done Done Sent false
1143 1283 Acked Read 4 Done Done Sent false
1144 1284 Acked Flush 0 Done Done Sent false
1145 1285 Acked Write 6 Done Done Sent false
1146 1286 Acked Write 9 Done Done Sent false
1147 1287 Acked Flush 0 Done Done Sent false
1148 1288 Acked Write 5 Done Done Sent false
1149 1289 Acked Write 7 Done Done Sent false
1150 1290 Acked Read 8 Done Done Sent false
1151 1291 Acked Write 6 Done Done Sent false
1152 1292 Acked Write 4 Done Done Sent false
1153 1293 Acked Read 1 Done Done Sent false
1154 1294 Acked Write 8 Done Done Sent false
1155 1295 Acked Flush 0 Done Done Sent false
1156 1296 Acked Write 3 Done Done Sent false
1157 1297 Acked Write 2 Done Done Sent false
1158 1298 Acked Read 10 Done Done Sent false
1159 1299 Acked Write 5 Done Done Sent false
1160 1300 Acked Read 2 Done Done Sent false
1161 1301 Acked Flush 0 Done Done Sent false
1162 STATES DS:0 DS:1 DS:2 TOTAL
1163 Sent 0 0 302 302
1164 Done 302 302 0 604
1165 Skipped 0 0 0 0
1166 Error 0 0 0 0
1167 Last Flush: 1301 1301 0
1168 Downstairs last five completed:
1169 Upstairs last five completed: 1301 1300 1299 1298 1297
1170 CLIENT: Up:0 ds:302 act:2
11712025-01-14T16:10:49.922ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
11722025-01-14T16:10:49.922ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:33739) = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
11732025-01-14T16:10:49.923ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
11742025-01-14T16:10:49.924ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
11752025-01-14T16:10:49.924ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
11762025-01-14T16:10:49.924ZINFOcrucible: send last flush ID to this DS: 0 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
11772025-01-14T16:10:49.924ZINFOcrucible: Replied this last flush ID: 0 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
11782025-01-14T16:10:49.924ZINFOcrucible: [2] client re-new 302 jobs since flush 0 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
11792025-01-14T16:10:49.924ZINFOcrucible: [2] Replayed 302 jobs since flush: 0 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
1180 ----------------------------------------------------------------
1181 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
1182 Downstairs last five completed: 1301 1300 1299 1298 1297
1183 Upstairs last five completed: 1301 1300 1299 1298 1297
1184 CLIENT: Up:0 ds:0 act:3
1185 Replay: All jobs finished
1186 Main task has work to do, go find it
1187 got dsc Stop(0)
1188 stop 0
1189 Jan 14 16:10:52.166 INFO request completed[8810] Got stop action so:false kr:false
1190 , latency_us: 315, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 348e85ed-5ac4-49b7-9477-09ae4015aec2, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
11912025-01-14T16:10:52.167ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
11922025-01-14T16:10:52.167ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
11932025-01-14T16:10:52.167ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
1194 Jan 14 16:10:52.167 INFO request completed, latency_us: 107, response_code: 200, uri: /state/cid/0, method: GET, req_id: 06e563f5-da3b-43ab-a60b-dea78d656d6c, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
11952025-01-14T16:10:52.167ZINFOcrucible: client task is exiting = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
1196 [8810] Exited with: ExitStatus(unix_wait_status(9))
1197 [8810][0] reports Exit
1198 Jan 14 16:10:56.170 INFO request completed, latency_us: 195, response_code: 200, uri: /state/cid/0, method: GET, req_id: 5e85422d-b2b3-4e06-b4ae-49b2ca0b3472, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
1199 001/300 Read block 3708 len 12288
1200 002/300 Write block 4036 len 4096 data: 2
1201 003/300 Flush
1202 004/300 Write block 2994 len 8192 data: 2 2
1203 005/300 Read block 525 len 40960
1204 006/300 Read block 210 len 40960
1205 007/300 Read block 2520 len 32768
1206 008/300 Read block 976 len 20480
1207 009/300 Read block 2343 len 8192
1208 010/300 Write block 584 len 24576 data: 2 2 2 2 2 2
1209 011/300 Flush
1210 012/300 Flush
1211 013/300 Read block 4136 len 36864
1212 014/300 Read block 1768 len 32768
1213 015/300 Write block 4014 len 28672 data: 2 2 2 2 2 2 2
1214 016/300 Read block 1367 len 32768
1215 017/300 Flush
1216 018/300 Read block 3057 len 8192
1217 019/300 Write block 1532 len 16384 data: 2 2 2 2
1218 020/300 Read block 2218 len 8192
1219 021/300 Write block 2366 len 32768 data: 2 2 2 2 2 2 2 2
1220 022/300 Flush
1221 023/300 Write block 399 len 24576 data: 2 2 2 2 2 2
1222 024/300 Read block 4594 len 4096
1223 025/300 Write block 4196 len 8192 data: 2 2
1224 026/300 Read block 1966 len 24576
1225 027/300 Read block 4321 len 28672
1226 028/300 Write block 3500 len 4096 data: 2
1227 029/300 Write block 3906 len 32768 data: 2 2 2 2 2 2 2 2
1228 030/300 Write block 3991 len 40960 data: 2 2 2 3 3 3 3 4 4 4
1229 031/300 Read block 2646 len 32768
1230 032/300 Write block 3263 len 24576 data: 3 3 3 3 3 3
1231 033/300 Read block 273 len 40960
1232 034/300 Flush
1233 035/300 Write block 3973 len 24576 data: 3 3 3 3 2 2
1234 036/300 Write block 826 len 8192 data: 3 3
1235 037/300 Write block 2454 len 4096 data: 2
1236 038/300 Write block 2399 len 28672 data: 2 2 2 2 2 2 2
1237 039/300 Write block 2424 len 24576 data: 2 2 2 2 2 2
1238 040/300 Read block 4095 len 20480
1239 041/300 Write block 3567 len 36864 data: 2 2 2 2 2 2 2 2 2
1240 042/300 Write block 1683 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1241 043/300 Read block 318 len 16384
1242 044/300 Read block 3962 len 32768
1243 045/300 Read block 3143 len 32768
1244 046/300 Write block 206 len 8192 data: 2 2
1245 047/300 Write block 2957 len 20480 data: 2 2 2 2 2
1246 048/300 Read block 3319 len 36864
1247 049/300 Write block 3155 len 16384 data: 2 2 2 2
1248 050/300 Write block 2161 len 32768 data: 2 2 2 2 2 2 2 2
1249 051/300 Read block 2822 len 28672
1250 052/300 Write block 3956 len 28672 data: 2 2 2 2 2 2 2
1251 053/300 Read block 2995 len 40960
1252 054/300 Read block 4877 len 36864
1253 055/300 Read block 653 len 8192
1254 056/300 Write block 3791 len 8192 data: 2 2
1255 057/300 Write block 2053 len 36864 data: 2 2 2 2 2 2 2 2 2
1256 058/300 Write block 1670 len 8192 data: 2 2
1257 059/300 Write block 39 len 28672 data: 2 2 2 2 2 2 2
1258 060/300 Flush
1259 061/300 Read block 4696 len 28672
1260 062/300 Read block 2570 len 8192
1261 063/300 Read block 369 len 36864
1262 064/300 Read block 3547 len 16384
1263 065/300 Read block 1995 len 28672
1264 066/300 Write block 3682 len 12288 data: 2 2 2
1265 067/300 Flush
1266 068/300 Write block 4084 len 36864 data: 3 3 3 2 2 2 2 2 2
1267 069/300 Read block 1889 len 32768
1268 070/300 Read block 1390 len 28672
1269 071/300 Read block 3624 len 36864
1270 072/300 Read block 2373 len 16384
1271 073/300 Write block 1986 len 32768 data: 2 2 3 3 3 3 3 3
1272 074/300 Read block 2790 len 24576
1273 075/300 Write block 4661 len 32768 data: 2 2 4 4 4 4 3 2
1274 076/300 Write block 2938 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1275 077/300 Read block 97 len 12288
1276 078/300 Read block 1983 len 24576
1277 079/300 Write block 2536 len 28672 data: 2 3 2 2 2 2 2
1278 080/300 Write block 1379 len 24576 data: 2 2 2 2 2 2
1279 081/300 Write block 3132 len 32768 data: 2 2 2 2 2 2 2 2
1280 082/300 Write block 3349 len 4096 data: 2
1281 083/300 Read block 4412 len 16384
1282 084/300 Read block 4445 len 20480
1283 085/300 Write block 3623 len 4096 data: 2
1284 086/300 Write block 4065 len 4096 data: 2
1285 087/300 Write block 207 len 40960 data: 3 2 2 2 2 2 2 2 2 2
1286 088/300 Write block 2903 len 32768 data: 2 2 2 2 2 2 2 2
1287 089/300 Read block 997 len 40960
1288 090/300 Flush
1289 091/300 Read block 3736 len 32768
1290 092/300 Write block 4842 len 36864 data: 2 2 2 2 2 2 2 2 2
1291 093/300 Write block 618 len 36864 data: 2 2 2 2 2 2 2 2 2
1292 094/300 Flush
1293 095/300 Read block 2602 len 16384
1294 096/300 Read block 3022 len 12288
1295 097/300 Read block 702 len 16384
1296 098/300 Read block 3956 len 24576
1297 099/300 Write block 1156 len 36864 data: 2 2 2 2 2 2 2 2 2
1298 100/300 Read block 75 len 8192
1299 101/300 Write block 3516 len 20480 data: 3 4 4 3 3
1300 102/300 Read block 4129 len 36864
1301 103/300 Read block 2733 len 16384
1302 104/300 Read block 2450 len 8192
1303 105/300 Read block 3436 len 32768
1304 106/300 Write block 4509 len 20480 data: 2 2 2 2 2
1305 107/300 Read block 4443 len 32768
1306 108/300 Flush
1307 109/300 Flush
1308 110/300 Write block 4524 len 8192 data: 2 2
1309 111/300 Read block 2903 len 4096
1310 112/300 Write block 2466 len 24576 data: 2 2 2 2 2 2
1311 113/300 Read block 1112 len 36864
1312 114/300 Write block 1193 len 12288 data: 2 2 2
1313 115/300 Write block 618 len 8192 data: 3 3
1314 116/300 Write block 2481 len 28672 data: 2 2 2 2 2 2 2
1315 117/300 Read block 4267 len 12288
1316 118/300 Read block 2646 len 4096
1317 119/300 Write block 4597 len 40960 data: 2 3 3 3 3 3 2 2 2 2
1318 120/300 Read block 1084 len 20480
1319 121/300 Flush
1320 122/300 Read block 1226 len 4096
1321 123/300 Read block 3510 len 4096
1322 124/300 Read block 3786 len 20480
1323 125/300 Flush
1324 126/300 Read block 4083 len 32768
1325 127/300 Write block 2733 len 8192 data: 2 2
1326 128/300 Write block 4505 len 4096 data: 2
1327 129/300 Read block 1714 len 20480
1328 130/300 Read block 4740 len 8192
1329 131/300 Read block 4590 len 20480
1330 132/300 Write block 1930 len 32768 data: 2 2 2 2 2 2 2 2
1331 133/300 Write block 2735 len 20480 data: 2 2 2 2 2
1332 134/300 Write block 3949 len 16384 data: 2 2 2 2
1333 135/300 Read block 2535 len 4096
1334 136/300 Read block 3561 len 24576
1335 137/300 Write block 2566 len 12288 data: 2 2 2
1336 138/300 Read block 4824 len 36864
1337 139/300 Read block 4715 len 32768
1338 140/300 Write block 3330 len 20480 data: 2 2 2 2 2
1339 141/300 Read block 3899 len 36864
1340 142/300 Write block 4941 len 20480 data: 2 2 2 2 2
1341 143/300 Write block 437 len 36864 data: 2 2 2 2 2 2 2 2 2
1342 144/300 Read block 726 len 8192
1343 145/300 Read block 4109 len 40960
1344 146/300 Write block 2998 len 28672 data: 2 2 2 2 2 2 2
1345 147/300 Read block 618 len 20480
1346 148/300 Write block 1463 len 36864 data: 2 2 3 3 3 3 3 3 3
1347 149/300 Write block 3382 len 32768 data: 3 3 3 3 3 2 2 2
1348 150/300 Read block 2462 len 36864
1349 151/300 Write block 3633 len 28672 data: 2 2 2 2 2 2 2
1350 152/300 Read block 4289 len 4096
1351 153/300 Read block 1218 len 4096
1352 154/300 Read block 3777 len 12288
1353 155/300 Read block 3195 len 36864
1354 156/300 Read block 2877 len 28672
1355 157/300 Flush
1356 158/300 Write block 3777 len 8192 data: 2 2
1357 159/300 Write block 4129 len 12288 data: 3 3 3
1358 160/300 Read block 2654 len 40960
1359 161/300 Write block 2925 len 12288 data: 2 2 3
1360 162/300 Write block 4246 len 12288 data: 2 2 2
1361 163/300 Read block 342 len 8192
1362 164/300 Read block 337 len 4096
1363 165/300 Write block 1696 len 8192 data: 3 3
1364 166/300 Read block 2651 len 4096
1365 167/300 Read block 3628 len 16384
1366 168/300 Read block 447 len 36864
1367 169/300 Write block 2214 len 16384 data: 2 2 2 2
1368 170/300 Read block 3663 len 40960
1369 171/300 Write block 4790 len 36864 data: 3 2 2 2 2 2 2 2 2
1370 172/300 Read block 3445 len 24576
1371 173/300 Flush
1372 174/300 Write block 319 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1373 175/300 Read block 654 len 8192
1374 176/300 Read block 1710 len 36864
1375 177/300 Write block 644 len 16384 data: 2 2 2 2
1376 178/300 Write block 3913 len 16384 data: 3 2 2 2
1377 179/300 Read block 3439 len 28672
1378 180/300 Read block 4207 len 36864
1379 181/300 Flush
1380 182/300 Read block 1234 len 12288
1381 183/300 Write block 2400 len 32768 data: 3 3 3 3 3 3 2 2
1382 184/300 Write block 2449 len 24576 data: 2 2 2 2 2 3
1383 185/300 Read block 2779 len 4096
1384 186/300 Write block 2107 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1385 187/300 Write block 1713 len 16384 data: 2 2 2 2
1386 188/300 Write block 881 len 36864 data: 2 2 2 2 2 2 2 2 2
1387 189/300 Write block 3896 len 20480 data: 2 2 2 2 2
1388 190/300 Read block 1143 len 8192
1389 191/300 Read block 3617 len 16384
1390 192/300 Read block 1486 len 20480
1391 193/300 Write block 1420 len 32768 data: 2 2 2 2 2 2 2 2
1392 194/300 Write block 168 len 8192 data: 2 2
1393 195/300 Write block 4001 len 32768 data: 4 4 4 3 3 3 3 2
1394 196/300 Read block 542 len 40960
1395 197/300 Write block 3282 len 40960 data: 3 3 3 3 3 3 3 2 2 2
1396 198/300 Read block 3298 len 24576
1397 199/300 Write block 95 len 4096 data: 2
1398 200/300 Write block 3193 len 24576 data: 2 2 2 2 2 2
1399 201/300 Read block 4022 len 32768
1400 202/300 Read block 505 len 28672
1401 203/300 Flush
1402 204/300 Flush
1403 205/300 Write block 993 len 4096 data: 2
1404 206/300 Write block 3757 len 8192 data: 2 2
1405 207/300 Flush
1406 208/300 Write block 2765 len 20480 data: 2 2 3 3 3
1407 209/300 Read block 1607 len 12288
1408 210/300 Read block 3450 len 20480
1409 211/300 Write block 1888 len 28672 data: 2 2 2 2 2 2 2
1410 212/300 Read block 285 len 36864
1411 213/300 Write block 24 len 4096 data: 4
1412 214/300 Write block 2245 len 20480 data: 2 2 2 2 2
1413 215/300 Write block 2185 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1414 216/300 Write block 2599 len 12288 data: 2 2 2
1415 217/300 Write block 966 len 32768 data: 2 2 2 2 2 2 2 2
1416 218/300 Read block 2601 len 28672
1417 219/300 Write block 1490 len 36864 data: 2 2 2 2 2 2 2 2 2
1418 220/300 Flush
1419 221/300 Write block 1646 len 12288 data: 3 3 3
1420 222/300 Write block 2697 len 28672 data: 2 2 2 2 2 2 2
1421 223/300 Write block 2030 len 8192 data: 3 3
1422 224/300 Read block 4596 len 28672
1423 225/300 Write block 3029 len 12288 data: 2 2 2
1424 226/300 Read block 2020 len 8192
1425 227/300 Write block 771 len 32768 data: 3 3 3 3 3 3 2 2
1426 228/300 Read block 1006 len 32768
1427 229/300 Read block 2335 len 16384
1428 230/300 Read block 1144 len 16384
1429 231/300 Write block 1385 len 36864 data: 2 2 2 2 2 2 2 2 2
1430 232/300 Write block 4101 len 24576 data: 2 2 2 2 2 2
1431 233/300 Read block 4302 len 12288
1432 234/300 Write block 2999 len 24576 data: 3 3 3 3 3 3
1433 235/300 Read block 1687 len 24576
1434 236/300 Read block 867 len 28672
1435 237/300 Read block 3308 len 24576
1436 238/300 Write block 822 len 12288 data: 2 2 2
1437 239/300 Write block 2813 len 32768 data: 4 4 4 4 3 2 2 2
1438 240/300 Read block 199 len 24576
1439 241/300 Read block 234 len 24576
1440 242/300 Read block 3654 len 4096
1441 243/300 Write block 2509 len 8192 data: 2 2
1442 244/300 Flush
1443 245/300 Read block 3537 len 12288
1444 246/300 Write block 4750 len 32768 data: 2 2 2 2 2 2 2 2
1445 247/300 Write block 342 len 36864 data: 2 2 2 2 2 2 2 2 2
1446 248/300 Read block 4333 len 16384
1447 249/300 Read block 427 len 12288
1448 250/300 Write block 3204 len 24576 data: 2 2 2 2 2 2
1449 251/300 Write block 2582 len 24576 data: 2 2 2 2 2 2
1450 252/300 Write block 4138 len 24576 data: 3 3 3 3 3 3
1451 253/300 Read block 2867 len 40960
1452 254/300 Read block 1897 len 32768
1453 255/300 Read block 1782 len 24576
1454 256/300 Write block 4830 len 8192 data: 2 2
1455 257/300 Read block 4080 len 40960
1456 258/300 Read block 3767 len 40960
1457 259/300 Write block 2830 len 8192 data: 2 2
1458 260/300 Flush
1459 261/300 Write block 1563 len 32768 data: 2 2 2 2 2 2 2 2
1460 262/300 Read block 1534 len 16384
1461 263/300 Write block 2560 len 28672 data: 3 3 3 3 3 2 3
1462 264/300 Read block 1098 len 40960
1463 265/300 Read block 1165 len 4096
1464 266/300 Write block 1217 len 32768 data: 2 3 3 3 3 3 3 3
1465 267/300 Write block 1891 len 32768 data: 3 3 3 3 2 2 2 2
1466 268/300 Write block 3858 len 4096 data: 2
1467 269/300 Write block 4846 len 4096 data: 3
1468 270/300 Read block 867 len 28672
1469 271/300 Write block 4127 len 32768 data: 3 3 4 4 4 2 2 2
1470 272/300 Write block 156 len 8192 data: 2 2
1471 273/300 Write block 2416 len 40960 data: 2 2 2 2 2 2 2 2 3 3
1472 274/300 Read block 1703 len 40960
1473 275/300 Write block 1948 len 40960 data: 2 2 2 2 2 2 2 3 3 3
1474 276/300 Write block 1190 len 12288 data: 2 2 2
1475 277/300 Write block 3943 len 8192 data: 2 2
1476 278/300 Write block 3806 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1477 279/300 Read block 2791 len 36864
1478 280/300 Write block 4188 len 16384 data: 2 2 2 2
1479 281/300 Flush
1480 282/300 Flush
1481 283/300 Read block 1148 len 40960
1482 284/300 Read block 2617 len 32768
1483 285/300 Read block 946 len 8192
1484 286/300 Read block 1606 len 32768
1485 287/300 Read block 4509 len 8192
1486 288/300 Read block 2828 len 12288
1487 289/300 Write block 3578 len 28672 data: 2 2 2 2 2 2 2
1488 290/300 Write block 3509 len 36864 data: 2 3 3 3 3 3 3 4 5
1489 291/300 Read block 486 len 28672
1490 292/300 Read block 602 len 16384
1491 293/300 Write block 4611 len 8192 data: 2 2
1492 294/300 Write block 4687 len 24576 data: 2 2 2 2 2 2
1493 295/300 Read block 175 len 40960
1494 296/300 Read block 4773 len 40960
1495 297/300 Read block 4800 len 36864
1496 298/300 Read block 4001 len 20480
1497 299/300 Read block 927 len 28672
1498 300/300 Flush
1499 301/300 Write block 2630 len 36864 data: 2 2 2 2 3 3 3 3 3
1500 Main task has work to do, go find it
1501 got dsc Start(0)
1502 start 0
1503 Jan 14 16:10:56.403 INFO request completed, latency_us: 199, response_code: 204, uri: /start/cid/0, method: POST, req_id: b16f8cee-7370-47bf-8cc2-a7c1c456f803, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
1504 Replay: started 0, returned:Ok(())
1505 ----------------------------------------------------------------
1506 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
1507 States: Offline Active Active
1508 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1509 1302 Acked Read 3 Sent Done Done false
1510 1303 Acked Write 1 Sent Done Done false
1511 1304 Acked Flush 0 Sent Done Done false
1512 1305 Acked Write 2 Sent Done Done false
1513 1306 Acked Read 10 Sent Done Done false
1514 1307 Acked Read 10 Sent Done Done false
1515 1308 Acked Read 8 Sent Done Done false
1516 1309 Acked Read 5 Sent Done Done false
1517 1310 Acked Read 2 Sent Done Done false
1518 1311 Acked Write 6 Sent Done Done false
1519 1312 Acked Flush 0 Sent Done Done false
1520 1313 Acked Flush 0 Sent Done Done false
1521 1314 Acked Read 9 Sent Done Done false
1522 1315 Acked Read 8 Sent Done Done false
1523 1316 Acked Write 7 Sent Done Done false
1524 1317 Acked Read 8 Sent Done Done false
1525 1318 Acked Flush 0 Sent Done Done false
1526 1319 Acked Read 2 Sent Done Done false
1527 1320 Acked Write 4 Sent Done Done false
1528 1321 Acked Read 2 Sent Done Done false
1529 1322 Acked Write 8 Sent Done Done false
1530 1323 Acked Flush 0 Sent Done Done false
1531 1324 Acked Write 6 Sent Done Done false
1532 1325 Acked Read 1 Sent Done Done false
1533 1326 Acked Write 2 Sent Done Done false
1534 1327 Acked Read 6 Sent Done Done false
1535 1328 Acked Read 7 Sent Done Done false
1536 1329 Acked Write 1 Sent Done Done false
1537 1330 Acked Write 8 Sent Done Done false
1538 1331 Acked Write 10 Sent Done Done false
1539 1332 Acked Read 8 Sent Done Done false
1540 1333 Acked Write 6 Sent Done Done false
1541 1334 Acked Read 10 Sent Done Done false
1542 1335 Acked Flush 0 Sent Done Done false
1543 1336 Acked Write 6 Sent Done Done false
1544 1337 Acked Write 2 Sent Done Done false
1545 1338 Acked Write 1 Sent Done Done false
1546 1339 Acked Write 7 Sent Done Done false
1547 1340 Acked Write 6 Sent Done Done false
1548 1341 Acked Read 5 Sent Done Done false
1549 1342 Acked Write 9 Sent Done Done false
1550 1343 Acked Write 10 Sent Done Done false
1551 1344 Acked Read 4 Sent Done Done false
1552 1345 Acked Read 8 Sent Done Done false
1553 1346 Acked Read 8 Sent Done Done false
1554 1347 Acked Write 2 Sent Done Done false
1555 1348 Acked Write 5 Sent Done Done false
1556 1349 Acked Read 9 Sent Done Done false
1557 1350 Acked Write 4 Sent Done Done false
1558 1351 Acked Write 8 Sent Done Done false
1559 1352 Acked Read 7 Sent Done Done false
1560 1353 Acked Write 7 Sent Done Done false
1561 1354 Acked Read 10 Sent Done Done false
1562 1355 Acked Read 9 Sent Done Done false
1563 1356 Acked Read 2 Sent Done Done false
1564 1357 Acked Write 2 Sent Done Done false
1565 1358 Acked Write 9 Sent Done Done false
1566 1359 Acked Write 2 Sent Done Done false
1567 1360 Acked Write 7 Sent Done Done false
1568 1361 Acked Flush 0 Sent Done Done false
1569 1362 Acked Read 7 Sent Done Done false
1570 1363 Acked Read 2 Sent Done Done false
1571 1364 Acked Read 9 Sent Done Done false
1572 1365 Acked Read 4 Sent Done Done false
1573 1366 Acked Read 7 Sent Done Done false
1574 1367 Acked Write 3 Sent Done Done false
1575 1368 Acked Flush 0 Sent Done Done false
1576 1369 Acked Write 9 Sent Done Done false
1577 1370 Acked Read 8 Sent Done Done false
1578 1371 Acked Read 7 Sent Done Done false
1579 1372 Acked Read 9 Sent Done Done false
1580 1373 Acked Read 4 Sent Done Done false
1581 1374 Acked Write 8 Sent Done Done false
1582 1375 Acked Read 6 Sent Done Done false
1583 1376 Acked Write 8 Sent Done Done false
1584 1377 Acked Write 10 Sent Done Done false
1585 1378 Acked Read 3 Sent Done Done false
1586 1379 Acked Read 6 Sent Done Done false
1587 1380 Acked Write 7 Sent Done Done false
1588 1381 Acked Write 6 Sent Done Done false
1589 1382 Acked Write 8 Sent Done Done false
1590 1383 Acked Write 1 Sent Done Done false
1591 1384 Acked Read 4 Sent Done Done false
1592 1385 Acked Read 5 Sent Done Done false
1593 1386 Acked Write 1 Sent Done Done false
1594 1387 Acked Write 1 Sent Done Done false
1595 1388 Acked Write 10 Sent Done Done false
1596 1389 Acked Write 8 Sent Done Done false
1597 1390 Acked Read 10 Sent Done Done false
1598 1391 Acked Flush 0 Sent Done Done false
1599 1392 Acked Read 8 Sent Done Done false
1600 1393 Acked Write 9 Sent Done Done false
1601 1394 Acked Write 9 Sent Done Done false
1602 1395 Acked Flush 0 Sent Done Done false
1603 1396 Acked Read 4 Sent Done Done false
1604 1397 Acked Read 3 Sent Done Done false
1605 1398 Acked Read 4 Sent Done Done false
1606 1399 Acked Read 6 Sent Done Done false
1607 1400 Acked Write 9 Sent Done Done false
1608 1401 Acked Read 2 Sent Done Done false
1609 1402 Acked Write 5 Sent Done Done false
1610 1403 Acked Read 9 Sent Done Done false
1611 1404 Acked Read 4 Sent Done Done false
1612 1405 Acked Read 2 Sent Done Done false
1613 1406 Acked Read 8 Sent Done Done false
1614 1407 Acked Write 5 Sent Done Done false
1615 1408 Acked Read 8 Sent Done Done false
1616 1409 Acked Flush 0 Sent Done Done false
1617 1410 Acked Flush 0 Sent Done Done false
1618 1411 Acked Write 2 Sent Done Done false
1619 1412 Acked Read 1 Sent Done Done false
1620 1413 Acked Write 6 Sent Done Done false
1621 1414 Acked Read 9 Sent Done Done false
1622 1415 Acked Write 3 Sent Done Done false
1623 1416 Acked Write 2 Sent Done Done false
1624 1417 Acked Write 7 Sent Done Done false
1625 1418 Acked Read 3 Sent Done Done false
1626 1419 Acked Read 1 Sent Done Done false
1627 1420 Acked Write 10 Sent Done Done false
1628 1421 Acked Read 5 Sent Done Done false
1629 1422 Acked Flush 0 Sent Done Done false
1630 1423 Acked Read 1 Sent Done Done false
1631 1424 Acked Read 1 Sent Done Done false
1632 1425 Acked Read 5 Sent Done Done false
1633 1426 Acked Flush 0 Sent Done Done false
1634 1427 Acked Read 8 Sent Done Done false
1635 1428 Acked Write 2 Sent Done Done false
1636 1429 Acked Write 1 Sent Done Done false
1637 1430 Acked Read 5 Sent Done Done false
1638 1431 Acked Read 2 Sent Done Done false
1639 1432 Acked Read 5 Sent Done Done false
1640 1433 Acked Write 8 Sent Done Done false
1641 1434 Acked Write 5 Sent Done Done false
1642 1435 Acked Write 4 Sent Done Done false
1643 1436 Acked Read 1 Sent Done Done false
1644 1437 Acked Read 6 Sent Done Done false
1645 1438 Acked Write 3 Sent Done Done false
1646 1439 Acked Read 9 Sent Done Done false
1647 1440 Acked Read 8 Sent Done Done false
1648 1441 Acked Write 5 Sent Done Done false
1649 1442 Acked Read 9 Sent Done Done false
1650 1443 Acked Write 5 Sent Done Done false
1651 1444 Acked Write 9 Sent Done Done false
1652 1445 Acked Read 2 Sent Done Done false
1653 1446 Acked Read 10 Sent Done Done false
1654 1447 Acked Write 7 Sent Done Done false
1655 1448 Acked Read 5 Sent Done Done false
1656 1449 Acked Write 9 Sent Done Done false
1657 1450 Acked Write 8 Sent Done Done false
1658 1451 Acked Read 9 Sent Done Done false
1659 1452 Acked Write 7 Sent Done Done false
1660 1453 Acked Read 1 Sent Done Done false
1661 1454 Acked Read 1 Sent Done Done false
1662 1455 Acked Read 3 Sent Done Done false
1663 1456 Acked Read 9 Sent Done Done false
1664 1457 Acked Read 7 Sent Done Done false
1665 1458 Acked Flush 0 Sent Done Done false
1666 1459 Acked Write 2 Sent Done Done false
1667 1460 Acked Write 3 Sent Done Done false
1668 1461 Acked Read 10 Sent Done Done false
1669 1462 Acked Write 3 Sent Done Done false
1670 1463 Acked Write 3 Sent Done Done false
1671 1464 Acked Read 2 Sent Done Done false
1672 1465 Acked Read 1 Sent Done Done false
1673 1466 Acked Write 2 Sent Done Done false
1674 1467 Acked Read 1 Sent Done Done false
1675 1468 Acked Read 4 Sent Done Done false
1676 1469 Acked Read 9 Sent Done Done false
1677 1470 Acked Write 4 Sent Done Done false
1678 1471 Acked Read 10 Sent Done Done false
1679 1472 Acked Write 9 Sent Done Done false
1680 1473 Acked Read 6 Sent Done Done false
1681 1474 Acked Flush 0 Sent Done Done false
1682 1475 Acked Write 10 Sent Done Done false
1683 1476 Acked Read 2 Sent Done Done false
1684 1477 Acked Read 9 Sent Done Done false
1685 1478 Acked Write 4 Sent Done Done false
1686 1479 Acked Write 4 Sent Done Done false
1687 1480 Acked Read 7 Sent Done Done false
1688 1481 Acked Read 9 Sent Done Done false
1689 1482 Acked Flush 0 Sent Done Done false
1690 1483 Acked Read 3 Sent Done Done false
1691 1484 Acked Write 8 Sent Done Done false
1692 1485 Acked Write 6 Sent Done Done false
1693 1486 Acked Read 1 Sent Done Done false
1694 1487 Acked Write 10 Sent Done Done false
1695 1488 Acked Write 4 Sent Done Done false
1696 1489 Acked Write 9 Sent Done Done false
1697 1490 Acked Write 5 Sent Done Done false
1698 1491 Acked Read 2 Sent Done Done false
1699 1492 Acked Read 4 Sent Done Done false
1700 1493 Acked Read 5 Sent Done Done false
1701 1494 Acked Write 8 Sent Done Done false
1702 1495 Acked Write 2 Sent Done Done false
1703 1496 Acked Write 8 Sent Done Done false
1704 1497 Acked Read 10 Sent Done Done false
1705 1498 Acked Write 10 Sent Done Done false
1706 1499 Acked Read 6 Sent Done Done false
1707 1500 Acked Write 1 Sent Done Done false
1708 1501 Acked Write 6 Sent Done Done false
1709 1502 Acked Read 8 Sent Done Done false
1710 1503 Acked Read 7 Sent Done Done false
1711 1504 Acked Flush 0 Sent Done Done false
1712 1505 Acked Flush 0 Sent Done Done false
1713 1506 Acked Write 1 Sent Done Done false
1714 1507 Acked Write 2 Sent Done Done false
1715 1508 Acked Flush 0 Sent Done Done false
1716 1509 Acked Write 5 Sent Done Done false
1717 1510 Acked Read 3 Sent Done Done false
1718 1511 Acked Read 5 Sent Done Done false
1719 1512 Acked Write 7 Sent Done Done false
1720 1513 Acked Read 9 Sent Done Done false
1721 1514 Acked Write 1 Sent Done Done false
1722 1515 Acked Write 5 Sent Done Done false
1723 1516 Acked Write 10 Sent Done Done false
1724 1517 Acked Write 3 Sent Done Done false
1725 1518 Acked Write 8 Sent Done Done false
1726 1519 Acked Read 7 Sent Done Done false
1727 1520 Acked Write 9 Sent Done Done false
1728 1521 Acked Flush 0 Sent Done Done false
1729 1522 Acked Write 3 Sent Done Done false
1730 1523 Acked Write 7 Sent Done Done false
1731 1524 Acked Write 2 Sent Done Done false
1732 1525 Acked Read 7 Sent Done Done false
1733 1526 Acked Write 3 Sent Done Done false
1734 1527 Acked Read 2 Sent Done Done false
1735 1528 Acked Write 8 Sent Done Done false
1736 1529 Acked Read 8 Sent Done Done false
1737 1530 Acked Read 4 Sent Done Done false
1738 1531 Acked Read 4 Sent Done Done false
1739 1532 Acked Write 9 Sent Done Done false
1740 1533 Acked Write 6 Sent Done Done false
1741 1534 Acked Read 3 Sent Done Done false
1742 1535 Acked Write 6 Sent Done Done false
1743 1536 Acked Read 6 Sent Done Done false
1744 1537 Acked Read 7 Sent Done Done false
1745 1538 Acked Read 6 Sent Done Done false
1746 1539 Acked Write 3 Sent Done Done false
1747 1540 Acked Write 8 Sent Done Done false
1748 1541 Acked Read 6 Sent Done Done false
1749 1542 Acked Read 6 Sent Done Done false
1750 1543 Acked Read 1 Sent Done Done false
1751 1544 Acked Write 2 Sent Done Done false
1752 1545 Acked Flush 0 Sent Done Done false
1753 1546 Acked Read 3 Sent Done Done false
1754 1547 Acked Write 8 Sent Done Done false
1755 1548 Acked Write 9 Sent Done Done false
1756 1549 Acked Read 4 Sent Done Done false
1757 1550 Acked Read 3 Sent Done Done false
1758 1551 Acked Write 6 Sent Done Done false
1759 1552 Acked Write 6 Sent Done Done false
1760 1553 Acked Write 6 Sent Done Done false
1761 1554 Acked Read 10 Sent Done Done false
1762 1555 Acked Read 8 Sent Done Done false
1763 1556 Acked Read 6 Sent Done Done false
1764 1557 Acked Write 2 Sent Done Done false
1765 1558 Acked Read 10 Sent Done Done false
1766 1559 Acked Read 10 Sent Done Done false
1767 1560 Acked Write 2 Sent Done Done false
1768 1561 Acked Flush 0 Sent Done Done false
1769 1562 Acked Write 8 Sent Done Done false
1770 1563 Acked Read 4 Sent Done Done false
1771 1564 Acked Write 7 Sent Done Done false
1772 1565 Acked Read 10 Sent Done Done false
1773 1566 Acked Read 1 Sent Done Done false
1774 1567 Acked Write 8 Sent Done Done false
1775 1568 Acked Write 8 Sent Done Done false
1776 1569 Acked Write 1 Sent Done Done false
1777 1570 Acked Write 1 Sent Done Done false
1778 1571 Acked Read 7 Sent Done Done false
1779 1572 Acked Write 8 Sent Done Done false
1780 1573 Acked Write 2 Sent Done Done false
1781 1574 Acked Write 10 Sent Done Done false
1782 1575 Acked Read 10 Sent Done Done false
1783 1576 Acked Write 10 Sent Done Done false
1784 1577 Acked Write 3 Sent Done Done false
1785 1578 Acked Write 2 Sent Done Done false
1786 1579 Acked Write 10 Sent Done Done false
1787 1580 Acked Read 9 Sent Done Done false
1788 1581 Acked Write 4 Sent Done Done false
1789 1582 Acked Flush 0 Sent Done Done false
1790 1583 Acked Flush 0 Sent Done Done false
1791 1584 Acked Read 10 Sent Done Done false
1792 1585 Acked Read 8 Sent Done Done false
1793 1586 Acked Read 2 Sent Done Done false
1794 1587 Acked Read 8 Sent Done Done false
1795 1588 Acked Read 2 Sent Done Done false
1796 1589 Acked Read 3 Sent Done Done false
1797 1590 Acked Write 7 Sent Done Done false
1798 1591 Acked Write 9 Sent Done Done false
1799 1592 Acked Read 7 Sent Done Done false
1800 1593 Acked Read 4 Sent Done Done false
1801 1594 Acked Write 2 Sent Done Done false
1802 1595 Acked Write 6 Sent Done Done false
1803 1596 Acked Read 10 Sent Done Done false
1804 1597 Acked Read 10 Sent Done Done false
1805 1598 Acked Read 9 Sent Done Done false
1806 1599 Acked Read 5 Sent Done Done false
1807 1600 Acked Read 7 Sent Done Done false
1808 1601 Acked Flush 0 Sent Done Done false
1809 1602 Acked Write 9 Sent Done Done false
1810 STATES DS:0 DS:1 DS:2 TOTAL
1811 Sent 301 0 0 301
1812 Done 0 301 301 602
1813 Skipped 0 0 0 0
1814 Error 0 0 0 0
1815 Last Flush: 1301 1601 1601
1816 Downstairs last five completed: 1301 1300 1299 1298 1297
1817 Upstairs last five completed: 1602 1601 1600 1599 1598
1818 CLIENT: Up:0 ds:301 act:2
1819 [8810] got start action
1820 [8810] I am going to restart
1821 Starting downstairs at port 8810
1822 Make output file at "/tmp/dsc/downstairs-8810.txt"
1823 [8810][0] reports Starting
1824 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1204)
1825 [8810][0] reports Running
1826 ----------------------------------------------------------------
1827 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
1828 States: Offline Active Active
1829 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1830 1302 Acked Read 3 Sent Done Done false
1831 1303 Acked Write 1 Sent Done Done false
1832 1304 Acked Flush 0 Sent Done Done false
1833 1305 Acked Write 2 Sent Done Done false
1834 1306 Acked Read 10 Sent Done Done false
1835 1307 Acked Read 10 Sent Done Done false
1836 1308 Acked Read 8 Sent Done Done false
1837 1309 Acked Read 5 Sent Done Done false
1838 1310 Acked Read 2 Sent Done Done false
1839 1311 Acked Write 6 Sent Done Done false
1840 1312 Acked Flush 0 Sent Done Done false
1841 1313 Acked Flush 0 Sent Done Done false
1842 1314 Acked Read 9 Sent Done Done false
1843 1315 Acked Read 8 Sent Done Done false
1844 1316 Acked Write 7 Sent Done Done false
1845 1317 Acked Read 8 Sent Done Done false
1846 1318 Acked Flush 0 Sent Done Done false
1847 1319 Acked Read 2 Sent Done Done false
1848 1320 Acked Write 4 Sent Done Done false
1849 1321 Acked Read 2 Sent Done Done false
1850 1322 Acked Write 8 Sent Done Done false
1851 1323 Acked Flush 0 Sent Done Done false
1852 1324 Acked Write 6 Sent Done Done false
1853 1325 Acked Read 1 Sent Done Done false
1854 1326 Acked Write 2 Sent Done Done false
1855 1327 Acked Read 6 Sent Done Done false
1856 1328 Acked Read 7 Sent Done Done false
1857 1329 Acked Write 1 Sent Done Done false
1858 1330 Acked Write 8 Sent Done Done false
1859 1331 Acked Write 10 Sent Done Done false
1860 1332 Acked Read 8 Sent Done Done false
1861 1333 Acked Write 6 Sent Done Done false
1862 1334 Acked Read 10 Sent Done Done false
1863 1335 Acked Flush 0 Sent Done Done false
1864 1336 Acked Write 6 Sent Done Done false
1865 1337 Acked Write 2 Sent Done Done false
1866 1338 Acked Write 1 Sent Done Done false
1867 1339 Acked Write 7 Sent Done Done false
1868 1340 Acked Write 6 Sent Done Done false
1869 1341 Acked Read 5 Sent Done Done false
1870 1342 Acked Write 9 Sent Done Done false
1871 1343 Acked Write 10 Sent Done Done false
1872 1344 Acked Read 4 Sent Done Done false
1873 1345 Acked Read 8 Sent Done Done false
1874 1346 Acked Read 8 Sent Done Done false
1875 1347 Acked Write 2 Sent Done Done false
1876 1348 Acked Write 5 Sent Done Done false
1877 1349 Acked Read 9 Sent Done Done false
1878 1350 Acked Write 4 Sent Done Done false
1879 1351 Acked Write 8 Sent Done Done false
1880 1352 Acked Read 7 Sent Done Done false
1881 1353 Acked Write 7 Sent Done Done false
1882 1354 Acked Read 10 Sent Done Done false
1883 1355 Acked Read 9 Sent Done Done false
1884 1356 Acked Read 2 Sent Done Done false
1885 1357 Acked Write 2 Sent Done Done false
1886 1358 Acked Write 9 Sent Done Done false
1887 1359 Acked Write 2 Sent Done Done false
1888 1360 Acked Write 7 Sent Done Done false
1889 1361 Acked Flush 0 Sent Done Done false
1890 1362 Acked Read 7 Sent Done Done false
1891 1363 Acked Read 2 Sent Done Done false
1892 1364 Acked Read 9 Sent Done Done false
1893 1365 Acked Read 4 Sent Done Done false
1894 1366 Acked Read 7 Sent Done Done false
1895 1367 Acked Write 3 Sent Done Done false
1896 1368 Acked Flush 0 Sent Done Done false
1897 1369 Acked Write 9 Sent Done Done false
1898 1370 Acked Read 8 Sent Done Done false
1899 1371 Acked Read 7 Sent Done Done false
1900 1372 Acked Read 9 Sent Done Done false
1901 1373 Acked Read 4 Sent Done Done false
1902 1374 Acked Write 8 Sent Done Done false
1903 1375 Acked Read 6 Sent Done Done false
1904 1376 Acked Write 8 Sent Done Done false
1905 1377 Acked Write 10 Sent Done Done false
1906 1378 Acked Read 3 Sent Done Done false
1907 1379 Acked Read 6 Sent Done Done false
1908 1380 Acked Write 7 Sent Done Done false
1909 1381 Acked Write 6 Sent Done Done false
1910 1382 Acked Write 8 Sent Done Done false
1911 1383 Acked Write 1 Sent Done Done false
1912 1384 Acked Read 4 Sent Done Done false
1913 1385 Acked Read 5 Sent Done Done false
1914 1386 Acked Write 1 Sent Done Done false
1915 1387 Acked Write 1 Sent Done Done false
1916 1388 Acked Write 10 Sent Done Done false
1917 1389 Acked Write 8 Sent Done Done false
1918 1390 Acked Read 10 Sent Done Done false
1919 1391 Acked Flush 0 Sent Done Done false
1920 1392 Acked Read 8 Sent Done Done false
1921 1393 Acked Write 9 Sent Done Done false
1922 1394 Acked Write 9 Sent Done Done false
1923 1395 Acked Flush 0 Sent Done Done false
1924 1396 Acked Read 4 Sent Done Done false
1925 1397 Acked Read 3 Sent Done Done false
1926 1398 Acked Read 4 Sent Done Done false
1927 1399 Acked Read 6 Sent Done Done false
1928 1400 Acked Write 9 Sent Done Done false
1929 1401 Acked Read 2 Sent Done Done false
1930 1402 Acked Write 5 Sent Done Done false
1931 1403 Acked Read 9 Sent Done Done false
1932 1404 Acked Read 4 Sent Done Done false
1933 1405 Acked Read 2 Sent Done Done false
1934 1406 Acked Read 8 Sent Done Done false
1935 1407 Acked Write 5 Sent Done Done false
1936 1408 Acked Read 8 Sent Done Done false
1937 1409 Acked Flush 0 Sent Done Done false
1938 1410 Acked Flush 0 Sent Done Done false
1939 1411 Acked Write 2 Sent Done Done false
1940 1412 Acked Read 1 Sent Done Done false
1941 1413 Acked Write 6 Sent Done Done false
1942 1414 Acked Read 9 Sent Done Done false
1943 1415 Acked Write 3 Sent Done Done false
1944 1416 Acked Write 2 Sent Done Done false
1945 1417 Acked Write 7 Sent Done Done false
1946 1418 Acked Read 3 Sent Done Done false
1947 1419 Acked Read 1 Sent Done Done false
1948 1420 Acked Write 10 Sent Done Done false
1949 1421 Acked Read 5 Sent Done Done false
1950 1422 Acked Flush 0 Sent Done Done false
1951 1423 Acked Read 1 Sent Done Done false
1952 1424 Acked Read 1 Sent Done Done false
1953 1425 Acked Read 5 Sent Done Done false
1954 1426 Acked Flush 0 Sent Done Done false
1955 1427 Acked Read 8 Sent Done Done false
1956 1428 Acked Write 2 Sent Done Done false
1957 1429 Acked Write 1 Sent Done Done false
1958 1430 Acked Read 5 Sent Done Done false
1959 1431 Acked Read 2 Sent Done Done false
1960 1432 Acked Read 5 Sent Done Done false
1961 1433 Acked Write 8 Sent Done Done false
1962 1434 Acked Write 5 Sent Done Done false
1963 1435 Acked Write 4 Sent Done Done false
1964 1436 Acked Read 1 Sent Done Done false
1965 1437 Acked Read 6 Sent Done Done false
1966 1438 Acked Write 3 Sent Done Done false
1967 1439 Acked Read 9 Sent Done Done false
1968 1440 Acked Read 8 Sent Done Done false
1969 1441 Acked Write 5 Sent Done Done false
1970 1442 Acked Read 9 Sent Done Done false
1971 1443 Acked Write 5 Sent Done Done false
1972 1444 Acked Write 9 Sent Done Done false
1973 1445 Acked Read 2 Sent Done Done false
1974 1446 Acked Read 10 Sent Done Done false
1975 1447 Acked Write 7 Sent Done Done false
1976 1448 Acked Read 5 Sent Done Done false
1977 1449 Acked Write 9 Sent Done Done false
1978 1450 Acked Write 8 Sent Done Done false
1979 1451 Acked Read 9 Sent Done Done false
1980 1452 Acked Write 7 Sent Done Done false
1981 1453 Acked Read 1 Sent Done Done false
1982 1454 Acked Read 1 Sent Done Done false
1983 1455 Acked Read 3 Sent Done Done false
1984 1456 Acked Read 9 Sent Done Done false
1985 1457 Acked Read 7 Sent Done Done false
1986 1458 Acked Flush 0 Sent Done Done false
1987 1459 Acked Write 2 Sent Done Done false
1988 1460 Acked Write 3 Sent Done Done false
1989 1461 Acked Read 10 Sent Done Done false
1990 1462 Acked Write 3 Sent Done Done false
1991 1463 Acked Write 3 Sent Done Done false
1992 1464 Acked Read 2 Sent Done Done false
1993 1465 Acked Read 1 Sent Done Done false
1994 1466 Acked Write 2 Sent Done Done false
1995 1467 Acked Read 1 Sent Done Done false
1996 1468 Acked Read 4 Sent Done Done false
1997 1469 Acked Read 9 Sent Done Done false
1998 1470 Acked Write 4 Sent Done Done false
1999 1471 Acked Read 10 Sent Done Done false
2000 1472 Acked Write 9 Sent Done Done false
2001 1473 Acked Read 6 Sent Done Done false
2002 1474 Acked Flush 0 Sent Done Done false
2003 1475 Acked Write 10 Sent Done Done false
2004 1476 Acked Read 2 Sent Done Done false
2005 1477 Acked Read 9 Sent Done Done false
2006 1478 Acked Write 4 Sent Done Done false
2007 1479 Acked Write 4 Sent Done Done false
2008 1480 Acked Read 7 Sent Done Done false
2009 1481 Acked Read 9 Sent Done Done false
2010 1482 Acked Flush 0 Sent Done Done false
2011 1483 Acked Read 3 Sent Done Done false
2012 1484 Acked Write 8 Sent Done Done false
2013 1485 Acked Write 6 Sent Done Done false
2014 1486 Acked Read 1 Sent Done Done false
2015 1487 Acked Write 10 Sent Done Done false
2016 1488 Acked Write 4 Sent Done Done false
2017 1489 Acked Write 9 Sent Done Done false
2018 1490 Acked Write 5 Sent Done Done false
2019 1491 Acked Read 2 Sent Done Done false
2020 1492 Acked Read 4 Sent Done Done false
2021 1493 Acked Read 5 Sent Done Done false
2022 1494 Acked Write 8 Sent Done Done false
2023 1495 Acked Write 2 Sent Done Done false
2024 1496 Acked Write 8 Sent Done Done false
2025 1497 Acked Read 10 Sent Done Done false
2026 1498 Acked Write 10 Sent Done Done false
2027 1499 Acked Read 6 Sent Done Done false
2028 1500 Acked Write 1 Sent Done Done false
2029 1501 Acked Write 6 Sent Done Done false
2030 1502 Acked Read 8 Sent Done Done false
2031 1503 Acked Read 7 Sent Done Done false
2032 1504 Acked Flush 0 Sent Done Done false
2033 1505 Acked Flush 0 Sent Done Done false
2034 1506 Acked Write 1 Sent Done Done false
2035 1507 Acked Write 2 Sent Done Done false
2036 1508 Acked Flush 0 Sent Done Done false
2037 1509 Acked Write 5 Sent Done Done false
2038 1510 Acked Read 3 Sent Done Done false
2039 1511 Acked Read 5 Sent Done Done false
2040 1512 Acked Write 7 Sent Done Done false
2041 1513 Acked Read 9 Sent Done Done false
2042 1514 Acked Write 1 Sent Done Done false
2043 1515 Acked Write 5 Sent Done Done false
2044 1516 Acked Write 10 Sent Done Done false
2045 1517 Acked Write 3 Sent Done Done false
2046 1518 Acked Write 8 Sent Done Done false
2047 1519 Acked Read 7 Sent Done Done false
2048 1520 Acked Write 9 Sent Done Done false
2049 1521 Acked Flush 0 Sent Done Done false
2050 1522 Acked Write 3 Sent Done Done false
2051 1523 Acked Write 7 Sent Done Done false
2052 1524 Acked Write 2 Sent Done Done false
2053 1525 Acked Read 7 Sent Done Done false
2054 1526 Acked Write 3 Sent Done Done false
2055 1527 Acked Read 2 Sent Done Done false
2056 1528 Acked Write 8 Sent Done Done false
2057 1529 Acked Read 8 Sent Done Done false
2058 1530 Acked Read 4 Sent Done Done false
2059 1531 Acked Read 4 Sent Done Done false
2060 1532 Acked Write 9 Sent Done Done false
2061 1533 Acked Write 6 Sent Done Done false
2062 1534 Acked Read 3 Sent Done Done false
2063 1535 Acked Write 6 Sent Done Done false
2064 1536 Acked Read 6 Sent Done Done false
2065 1537 Acked Read 7 Sent Done Done false
2066 1538 Acked Read 6 Sent Done Done false
2067 1539 Acked Write 3 Sent Done Done false
2068 1540 Acked Write 8 Sent Done Done false
2069 1541 Acked Read 6 Sent Done Done false
2070 1542 Acked Read 6 Sent Done Done false
2071 1543 Acked Read 1 Sent Done Done false
2072 1544 Acked Write 2 Sent Done Done false
2073 1545 Acked Flush 0 Sent Done Done false
2074 1546 Acked Read 3 Sent Done Done false
2075 1547 Acked Write 8 Sent Done Done false
2076 1548 Acked Write 9 Sent Done Done false
2077 1549 Acked Read 4 Sent Done Done false
2078 1550 Acked Read 3 Sent Done Done false
2079 1551 Acked Write 6 Sent Done Done false
2080 1552 Acked Write 6 Sent Done Done false
2081 1553 Acked Write 6 Sent Done Done false
2082 1554 Acked Read 10 Sent Done Done false
2083 1555 Acked Read 8 Sent Done Done false
2084 1556 Acked Read 6 Sent Done Done false
2085 1557 Acked Write 2 Sent Done Done false
2086 1558 Acked Read 10 Sent Done Done false
2087 1559 Acked Read 10 Sent Done Done false
2088 1560 Acked Write 2 Sent Done Done false
2089 1561 Acked Flush 0 Sent Done Done false
2090 1562 Acked Write 8 Sent Done Done false
2091 1563 Acked Read 4 Sent Done Done false
2092 1564 Acked Write 7 Sent Done Done false
2093 1565 Acked Read 10 Sent Done Done false
2094 1566 Acked Read 1 Sent Done Done false
2095 1567 Acked Write 8 Sent Done Done false
2096 1568 Acked Write 8 Sent Done Done false
2097 1569 Acked Write 1 Sent Done Done false
2098 1570 Acked Write 1 Sent Done Done false
2099 1571 Acked Read 7 Sent Done Done false
2100 1572 Acked Write 8 Sent Done Done false
2101 1573 Acked Write 2 Sent Done Done false
2102 1574 Acked Write 10 Sent Done Done false
2103 1575 Acked Read 10 Sent Done Done false
2104 1576 Acked Write 10 Sent Done Done false
2105 1577 Acked Write 3 Sent Done Done false
2106 1578 Acked Write 2 Sent Done Done false
2107 1579 Acked Write 10 Sent Done Done false
2108 1580 Acked Read 9 Sent Done Done false
2109 1581 Acked Write 4 Sent Done Done false
2110 1582 Acked Flush 0 Sent Done Done false
2111 1583 Acked Flush 0 Sent Done Done false
2112 1584 Acked Read 10 Sent Done Done false
2113 1585 Acked Read 8 Sent Done Done false
2114 1586 Acked Read 2 Sent Done Done false
2115 1587 Acked Read 8 Sent Done Done false
2116 1588 Acked Read 2 Sent Done Done false
2117 1589 Acked Read 3 Sent Done Done false
2118 1590 Acked Write 7 Sent Done Done false
2119 1591 Acked Write 9 Sent Done Done false
2120 1592 Acked Read 7 Sent Done Done false
2121 1593 Acked Read 4 Sent Done Done false
2122 1594 Acked Write 2 Sent Done Done false
2123 1595 Acked Write 6 Sent Done Done false
2124 1596 Acked Read 10 Sent Done Done false
2125 1597 Acked Read 10 Sent Done Done false
2126 1598 Acked Read 9 Sent Done Done false
2127 1599 Acked Read 5 Sent Done Done false
2128 1600 Acked Read 7 Sent Done Done false
2129 1601 Acked Flush 0 Sent Done Done false
2130 1602 Acked Write 9 Sent Done Done false
2131 1603 Acked Flush 0 Sent Done Done false
2132 STATES DS:0 DS:1 DS:2 TOTAL
2133 Sent 302 0 0 302
2134 Done 0 302 302 604
2135 Skipped 0 0 0 0
2136 Error 0 0 0 0
2137 Last Flush: 1301 1603 1603
2138 Downstairs last five completed: 1301 1300 1299 1298 1297
2139 Upstairs last five completed: 1603 1602 1601 1600 1599
2140 CLIENT: Up:0 ds:302 act:2
21412025-01-14T16:11:02.169ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
21422025-01-14T16:11:02.169ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:63368) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
21432025-01-14T16:11:02.170ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
21442025-01-14T16:11:02.171ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
21452025-01-14T16:11:02.171ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
21462025-01-14T16:11:02.171ZINFOcrucible: send last flush ID to this DS: 1301 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
21472025-01-14T16:11:02.171ZINFOcrucible: Replied this last flush ID: 1301 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
21482025-01-14T16:11:02.171ZINFOcrucible: [0] client re-new 302 jobs since flush 1301 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
21492025-01-14T16:11:02.172ZINFOcrucible: [0] Replayed 302 jobs since flush: 1301 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
2150 ----------------------------------------------------------------
2151 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
2152 Downstairs last five completed: 1603 1602 1601 1600 1599
2153 Upstairs last five completed: 1603 1602 1601 1600 1599
2154 CLIENT: Up:0 ds:0 act:3
2155 Replay: All jobs finished
2156 Main task has work to do, go find it
2157 got dsc Stop(0)
2158 stop 0
2159 Jan 14 16:11:04.408 INFO request completed, latency_us: [8810] Got stop action so:false kr:false
2160 358, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 2e79fea8-bf93-4485-9fc2-74a7ad54be1c, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
21612025-01-14T16:11:04.408ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
2162 Jan 14 16:11:04.409 INFO request completed, latency_us: 160, response_code: 200, uri: /state/cid/0, method: GET, req_id: 61d63638-cf0d-47af-9d93-ca9a886f5f2a{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-14T16:11:04.40915631Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
2163 , remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-14T16:11:04.409209463Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
2164 9998
21652025-01-14T16:11:04.409ZINFOcrucible: client task is exiting = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
2166 [8810] Exited with: ExitStatus(unix_wait_status(9))
2167 [8810][0] reports Exit
2168 Jan 14 16:11:08.410 INFO request completed, latency_us: 245, response_code: 200, uri: /state/cid/0, method: GET, req_id: bc8fb3d6-ff83-408a-baf5-979056f22493, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
2169 001/300 Write block 756 len 40960 data: 2 3 3 3 3 3 3 3 3 3
2170 002/300 Write block 4372 len 36864 data: 2 2 2 2 2 2 2 2 2
2171 003/300 Write block 3492 len 36864 data: 2 2 2 2 2 2 2 2 3
2172 004/300 Flush
2173 005/300 Write block 4621 len 16384 data: 2 2 2 2
2174 006/300 Write block 513 len 12288 data: 2 2 2
2175 007/300 Read block 1632 len 40960
2176 008/300 Read block 1748 len 28672
2177 009/300 Read block 2323 len 36864
2178 010/300 Write block 234 len 16384 data: 2 2 2 2
2179 011/300 Flush
2180 012/300 Read block 1196 len 4096
2181 013/300 Read block 2670 len 28672
2182 014/300 Read block 3128 len 20480
2183 015/300 Read block 1611 len 20480
2184 016/300 Read block 2142 len 8192
2185 017/300 Write block 441 len 4096 data: 3
2186 018/300 Read block 786 len 28672
2187 019/300 Read block 1034 len 20480
2188 020/300 Read block 1661 len 20480
2189 021/300 Read block 834 len 40960
2190 022/300 Read block 4182 len 8192
2191 023/300 Flush
2192 024/300 Write block 3096 len 8192 data: 2 2
2193 025/300 Write block 2037 len 20480 data: 2 2 2 2 2
2194 026/300 Write block 3471 len 28672 data: 3 3 3 2 2 2 2
2195 027/300 Write block 4862 len 12288 data: 2 2 2
2196 028/300 Read block 1041 len 40960
2197 029/300 Read block 1326 len 36864
2198 030/300 Write block 2008 len 16384 data: 3 3 3 2
2199 031/300 Write block 4988 len 16384 data: 2 2 2 2
2200 032/300 Flush
2201 033/300 Read block 1312 len 28672
2202 034/300 Read block 4890 len 24576
2203 035/300 Write block 3959 len 8192 data: 3 3
2204 036/300 Write block 2209 len 24576 data: 2 2 2 2 2 3
2205 037/300 Write block 4742 len 20480 data: 2 2 2 2 2
2206 038/300 Read block 2031 len 16384
2207 039/300 Read block 4140 len 16384
2208 040/300 Write block 4155 len 40960 data: 2 2 2 2 3 2 2 2 2 2
2209 041/300 Flush
2210 042/300 Write block 2720 len 20480 data: 4 3 3 2 2
2211 043/300 Read block 4249 len 36864
2212 044/300 Write block 1413 len 32768 data: 2 2 2 2 2 2 2 3
2213 045/300 Read block 849 len 24576
2214 046/300 Flush
2215 047/300 Write block 4804 len 20480 data: 2 2 2 2 2
2216 048/300 Write block 4242 len 36864 data: 2 2 2 2 3 3 3 2 2
2217 049/300 Read block 3353 len 40960
2218 050/300 Write block 250 len 28672 data: 2 2 2 2 2 2 2
2219 051/300 Flush
2220 052/300 Flush
2221 053/300 Read block 3235 len 8192
2222 054/300 Flush
2223 055/300 Read block 4078 len 24576
2224 056/300 Write block 4692 len 8192 data: 3 2
2225 057/300 Read block 2099 len 20480
2226 058/300 Flush
2227 059/300 Read block 774 len 20480
2228 060/300 Read block 1163 len 40960
2229 061/300 Write block 4394 len 4096 data: 3
2230 062/300 Read block 3374 len 4096
2231 063/300 Read block 3884 len 20480
2232 064/300 Write block 4749 len 20480 data: 2 3 3 3 3
2233 065/300 Write block 1724 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2234 066/300 Read block 1314 len 40960
2235 067/300 Read block 4944 len 40960
2236 068/300 Flush
2237 069/300 Write block 1753 len 20480 data: 2 2 2 2 2
2238 070/300 Write block 402 len 36864 data: 3 3 3 2 2 2 2 3 3
2239 071/300 Read block 3079 len 4096
2240 072/300 Flush
2241 073/300 Write block 3605 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2242 074/300 Read block 2264 len 16384
2243 075/300 Flush
2244 076/300 Read block 487 len 40960
2245 077/300 Write block 3253 len 20480 data: 4 4 4 4 2
2246 078/300 Read block 4007 len 40960
2247 079/300 Read block 811 len 20480
2248 080/300 Read block 486 len 12288
2249 081/300 Read block 2033 len 20480
2250 082/300 Read block 3364 len 8192
2251 083/300 Write block 3971 len 28672 data: 2 2 4 4 4 4 3
2252 084/300 Read block 555 len 4096
2253 085/300 Flush
2254 086/300 Read block 1405 len 28672
2255 087/300 Write block 4450 len 12288 data: 2 2 2
2256 088/300 Read block 1689 len 32768
2257 089/300 Read block 2885 len 28672
2258 090/300 Read block 106 len 16384
2259 091/300 Write block 2921 len 8192 data: 2 2
2260 092/300 Read block 141 len 28672
2261 093/300 Read block 1781 len 16384
2262 094/300 Read block 1826 len 20480
2263 095/300 Read block 431 len 12288
2264 096/300 Write block 4513 len 24576 data: 3 2 2 2 2 2
2265 097/300 Read block 195 len 8192
2266 098/300 Read block 2019 len 8192
2267 099/300 Read block 4377 len 4096
2268 100/300 Flush
2269 101/300 Read block 2196 len 40960
2270 102/300 Read block 4677 len 32768
2271 103/300 Write block 3960 len 40960 data: 4 3 3 2 2 2 2 2 2 2
2272 104/300 Write block 774 len 12288 data: 4 4 4
2273 105/300 Write block 3416 len 24576 data: 2 2 2 2 2 2
2274 106/300 Read block 666 len 24576
2275 107/300 Flush
2276 108/300 Flush
2277 109/300 Write block 1017 len 24576 data: 2 2 2 2 2 2
2278 110/300 Read block 98 len 16384
2279 111/300 Write block 669 len 28672 data: 2 2 2 2 2 2 2
2280 112/300 Write block 3016 len 8192 data: 2 2
2281 113/300 Read block 2207 len 28672
2282 114/300 Read block 493 len 32768
2283 115/300 Read block 3340 len 12288
2284 116/300 Write block 4715 len 12288 data: 2 2 2
2285 117/300 Write block 4953 len 4096 data: 2
2286 118/300 Read block 3200 len 4096
2287 119/300 Read block 3516 len 16384
2288 120/300 Read block 336 len 32768
2289 121/300 Write block 1271 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2290 122/300 Write block 1591 len 32768 data: 2 2 2 2 2 2 2 2
2291 123/300 Write block 1887 len 28672 data: 2 3 3 3 4 4 4
2292 124/300 Read block 2944 len 12288
2293 125/300 Write block 393 len 36864 data: 2 2 2 2 2 2 3 3 3
2294 126/300 Write block 3602 len 20480 data: 2 2 2 3 3
2295 127/300 Write block 3020 len 36864 data: 2 2 2 2 2 2 2 2 2
2296 128/300 Write block 2280 len 28672 data: 2 2 2 2 2 2 2
2297 129/300 Write block 3817 len 16384 data: 2 2 2 2
2298 130/300 Flush
2299 131/300 Write block 2284 len 8192 data: 3 3
2300 132/300 Read block 2771 len 40960
2301 133/300 Write block 1745 len 40960 data: 2 2 2 2 2 2 2 2 3 3
2302 134/300 Read block 1574 len 36864
2303 135/300 Flush
2304 136/300 Read block 1501 len 12288
2305 137/300 Flush
2306 138/300 Read block 298 len 4096
2307 139/300 Write block 2216 len 16384 data: 3 3 2 2
2308 140/300 Read block 299 len 36864
2309 141/300 Read block 2007 len 40960
2310 142/300 Write block 4609 len 4096 data: 2
2311 143/300 Read block 280 len 8192
2312 144/300 Write block 2710 len 12288 data: 2 2 2
2313 145/300 Flush
2314 146/300 Write block 1331 len 8192 data: 2 2
2315 147/300 Read block 4196 len 28672
2316 148/300 Read block 3143 len 12288
2317 149/300 Read block 2322 len 12288
2318 150/300 Write block 2726 len 12288 data: 3 3 3
2319 151/300 Flush
2320 152/300 Read block 3232 len 36864
2321 153/300 Write block 1323 len 16384 data: 2 2 2 2
2322 154/300 Write block 3354 len 36864 data: 2 2 3 3 3 3 3 3 2
2323 155/300 Write block 324 len 12288 data: 3 3 3
2324 156/300 Read block 4302 len 12288
2325 157/300 Write block 585 len 12288 data: 3 3 3
2326 158/300 Read block 435 len 12288
2327 159/300 Read block 675 len 16384
2328 160/300 Read block 4270 len 40960
2329 161/300 Read block 4207 len 32768
2330 162/300 Write block 1826 len 32768 data: 2 2 2 2 2 2 2 2
2331 163/300 Read block 1499 len 12288
2332 164/300 Write block 1358 len 4096 data: 2
2333 165/300 Read block 2418 len 40960
2334 166/300 Write block 4399 len 4096 data: 3
2335 167/300 Write block 3450 len 36864 data: 2 2 2 2 2 2 2 2 2
2336 168/300 Flush
2337 169/300 Read block 1174 len 24576
2338 170/300 Read block 3854 len 4096
2339 171/300 Write block 866 len 28672 data: 2 2 2 2 2 2 2
2340 172/300 Read block 3452 len 40960
2341 173/300 Read block 3673 len 4096
2342 174/300 Read block 3721 len 36864
2343 175/300 Write block 3374 len 8192 data: 3 2
2344 176/300 Flush
2345 177/300 Read block 1345 len 28672
2346 178/300 Read block 985 len 24576
2347 179/300 Write block 3822 len 20480 data: 2 2 2 2 2
2348 180/300 Read block 1979 len 12288
2349 181/300 Write block 3050 len 12288 data: 2 2 2
2350 182/300 Write block 1809 len 36864 data: 2 2 2 2 2 2 2 2 2
2351 183/300 Read block 1383 len 36864
2352 184/300 Read block 161 len 4096
2353 185/300 Flush
2354 186/300 Write block 2279 len 20480 data: 2 3 3 3 3
2355 187/300 Read block 2426 len 8192
2356 188/300 Write block 1735 len 28672 data: 2 2 2 2 2 2 2
2357 189/300 Read block 1333 len 12288
2358 190/300 Write block 3228 len 20480 data: 2 2 2 2 2
2359 191/300 Write block 2462 len 12288 data: 2 2 2
2360 192/300 Read block 1410 len 8192
2361 193/300 Write block 1403 len 12288 data: 2 2 2
2362 194/300 Write block 4251 len 4096 data: 2
2363 195/300 Write block 1312 len 12288 data: 2 2 2
2364 196/300 Write block 4136 len 12288 data: 2 2 4
2365 197/300 Read block 1991 len 8192
2366 198/300 Read block 483 len 8192
2367 199/300 Write block 1995 len 16384 data: 2 2 2 2
2368 200/300 Write block 4592 len 40960 data: 2 2 2 2 2 3 4 4 4 4
2369 201/300 Read block 2824 len 36864
2370 202/300 Flush
2371 203/300 Write block 1422 len 24576 data: 3 3 3 3 3 3
2372 204/300 Write block 2884 len 28672 data: 2 2 2 2 2 2 2
2373 205/300 Read block 1116 len 28672
2374 206/300 Read block 4505 len 20480
2375 207/300 Read block 2887 len 36864
2376 208/300 Read block 4240 len 8192
2377 209/300 Read block 2366 len 20480
2378 210/300 Read block 4741 len 16384
2379 211/300 Flush
2380 212/300 Flush
2381 213/300 Write block 905 len 36864 data: 2 2 2 2 2 2 2 2 2
2382 214/300 Write block 4963 len 4096 data: 2
2383 215/300 Read block 3200 len 4096
2384 216/300 Read block 3961 len 20480
2385 217/300 Write block 3280 len 32768 data: 2 3 4 4 4 4 4 4
2386 218/300 Write block 4405 len 36864 data: 2 2 2 2 2 2 2 2 2
2387 219/300 Read block 1372 len 36864
2388 220/300 Write block 208 len 32768 data: 3 3 3 3 3 3 3 3
2389 221/300 Write block 3867 len 32768 data: 2 2 2 2 2 2 2 2
2390 222/300 Read block 2978 len 40960
2391 223/300 Write block 57 len 16384 data: 2 2 2 2
2392 224/300 Read block 4882 len 32768
2393 225/300 Flush
2394 226/300 Read block 3596 len 24576
2395 227/300 Read block 124 len 28672
2396 228/300 Write block 4533 len 28672 data: 2 2 2 2 2 3 3
2397 229/300 Write block 4850 len 4096 data: 3
2398 230/300 Read block 1118 len 16384
2399 231/300 Read block 1708 len 8192
2400 232/300 Read block 4660 len 28672
2401 233/300 Write block 963 len 32768 data: 2 2 2 3 3 3 3 3
2402 234/300 Read block 2245 len 36864
2403 235/300 Write block 1175 len 36864 data: 2 2 2 2 2 2 2 2 2
2404 236/300 Read block 1862 len 8192
2405 237/300 Flush
2406 238/300 Read block 3302 len 32768
2407 239/300 Read block 4994 len 16384
2408 240/300 Read block 3035 len 12288
2409 241/300 Read block 599 len 12288
2410 242/300 Read block 856 len 8192
2411 243/300 Write block 1655 len 8192 data: 2 2
2412 244/300 Read block 4821 len 40960
2413 245/300 Read block 3206 len 4096
2414 246/300 Read block 4076 len 4096
2415 247/300 Write block 3767 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2416 248/300 Write block 725 len 24576 data: 2 2 2 2 2 2
2417 249/300 Read block 2678 len 12288
2418 250/300 Write block 2283 len 36864 data: 4 4 4 3 2 2 2 2 2
2419 251/300 Read block 4831 len 12288
2420 252/300 Read block 2997 len 20480
2421 253/300 Write block 3186 len 24576 data: 2 2 2 2 2 2
2422 254/300 Read block 2063 len 12288
2423 255/300 Read block 1498 len 4096
2424 256/300 Read block 3868 len 32768
2425 257/300 Read block 4910 len 12288
2426 258/300 Write block 2761 len 20480 data: 3 3 3 2 3
2427 259/300 Read block 4236 len 36864
2428 260/300 Read block 2362 len 16384
2429 261/300 Write block 196 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2430 262/300 Read block 1903 len 32768
2431 263/300 Read block 4591 len 8192
2432 264/300 Read block 34 len 8192
2433 265/300 Flush
2434 266/300 Write block 1051 len 4096 data: 2
2435 267/300 Write block 1868 len 12288 data: 2 2 2
2436 268/300 Write block 2243 len 24576 data: 2 2 3 3 3 3
2437 269/300 Read block 4746 len 40960
2438 270/300 Read block 4779 len 8192
2439 271/300 Write block 4280 len 24576 data: 2 2 2 2 2 2
2440 272/300 Flush
2441 273/300 Flush
2442 274/300 Write block 275 len 4096 data: 2
2443 275/300 Read block 1168 len 16384
2444 276/300 Read block 4430 len 28672
2445 277/300 Flush
2446 278/300 Write block 4330 len 12288 data: 2 2 2
2447 279/300 Read block 2664 len 4096
2448 280/300 Read block 793 len 32768
2449 281/300 Write block 2743 len 28672 data: 2 2 2 2 2 3 3
2450 282/300 Write block 2190 len 40960 data: 3 3 3 3 3 2 2 2 2 2
2451 283/300 Write block 201 len 36864 data: 3 3 3 3 3 3 4 4 4
2452 284/300 Read block 2407 len 28672
2453 285/300 Write block 2 len 36864 data: 2 2 2 2 2 2 2 2 2
2454 286/300 Read block 2036 len 16384
2455 287/300 Write block 1889 len 24576 data: 4 4 5 5 5 4
2456 288/300 Read block 2937 len 28672
2457 289/300 Read block 673 len 24576
2458 290/300 Read block 1753 len 24576
2459 291/300 Write block 1831 len 36864 data: 3 3 3 2 2 2 2 2 2
2460 292/300 Read block 1271 len 24576
2461 293/300 Read block 2904 len 16384
2462 294/300 Read block 2198 len 28672
2463 295/300 Write block 1351 len 12288 data: 2 2 2
2464 296/300 Flush
2465 297/300 Write block 925 len 32768 data: 2 2 2 2 2 2 2 2
2466 298/300 Read block 550 len 24576
2467 299/300 Write block 2251 len 20480 data: 2 2 2 2 2
2468 300/300 Write block 1960 len 12288 data: 2 2 2
2469 301/300 Flush
2470 Main task has work to do, go find it
2471 got dsc Start(0)
2472 start 0
2473 Jan 14 16:11:08.640 INFO request completed, latency_us: 196, response_code: 204, uri: /start/cid/0, method: POST, req_id: fe698883-6780-423b-988b-be27ae0e9620, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
2474 Replay: started 0, returned:Ok(())
2475 ----------------------------------------------------------------
2476 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
2477 States: Offline Active Active
2478 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2479 1604 Acked Write 10 Sent Done Done false
2480 1605 Acked Write 9 Sent Done Done false
2481 1606 Acked Write 9 Sent Done Done false
2482 1607 Acked Flush 0 Sent Done Done false
2483 1608 Acked Write 4 Sent Done Done false
2484 1609 Acked Write 3 Sent Done Done false
2485 1610 Acked Read 10 Sent Done Done false
2486 1611 Acked Read 7 Sent Done Done false
2487 1612 Acked Read 9 Sent Done Done false
2488 1613 Acked Write 4 Sent Done Done false
2489 1614 Acked Flush 0 Sent Done Done false
2490 1615 Acked Read 1 Sent Done Done false
2491 1616 Acked Read 7 Sent Done Done false
2492 1617 Acked Read 5 Sent Done Done false
2493 1618 Acked Read 5 Sent Done Done false
2494 1619 Acked Read 2 Sent Done Done false
2495 1620 Acked Write 1 Sent Done Done false
2496 1621 Acked Read 7 Sent Done Done false
2497 1622 Acked Read 5 Sent Done Done false
2498 1623 Acked Read 5 Sent Done Done false
2499 1624 Acked Read 10 Sent Done Done false
2500 1625 Acked Read 2 Sent Done Done false
2501 1626 Acked Flush 0 Sent Done Done false
2502 1627 Acked Write 2 Sent Done Done false
2503 1628 Acked Write 5 Sent Done Done false
2504 1629 Acked Write 7 Sent Done Done false
2505 1630 Acked Write 3 Sent Done Done false
2506 1631 Acked Read 10 Sent Done Done false
2507 1632 Acked Read 9 Sent Done Done false
2508 1633 Acked Write 4 Sent Done Done false
2509 1634 Acked Write 4 Sent Done Done false
2510 1635 Acked Flush 0 Sent Done Done false
2511 1636 Acked Read 7 Sent Done Done false
2512 1637 Acked Read 6 Sent Done Done false
2513 1638 Acked Write 2 Sent Done Done false
2514 1639 Acked Write 6 Sent Done Done false
2515 1640 Acked Write 5 Sent Done Done false
2516 1641 Acked Read 4 Sent Done Done false
2517 1642 Acked Read 4 Sent Done Done false
2518 1643 Acked Write 10 Sent Done Done false
2519 1644 Acked Flush 0 Sent Done Done false
2520 1645 Acked Write 5 Sent Done Done false
2521 1646 Acked Read 9 Sent Done Done false
2522 1647 Acked Write 8 Sent Done Done false
2523 1648 Acked Read 6 Sent Done Done false
2524 1649 Acked Flush 0 Sent Done Done false
2525 1650 Acked Write 5 Sent Done Done false
2526 1651 Acked Write 9 Sent Done Done false
2527 1652 Acked Read 10 Sent Done Done false
2528 1653 Acked Write 7 Sent Done Done false
2529 1654 Acked Flush 0 Sent Done Done false
2530 1655 Acked Flush 0 Sent Done Done false
2531 1656 Acked Read 2 Sent Done Done false
2532 1657 Acked Flush 0 Sent Done Done false
2533 1658 Acked Read 6 Sent Done Done false
2534 1659 Acked Write 2 Sent Done Done false
2535 1660 Acked Read 5 Sent Done Done false
2536 1661 Acked Flush 0 Sent Done Done false
2537 1662 Acked Read 5 Sent Done Done false
2538 1663 Acked Read 10 Sent Done Done false
2539 1664 Acked Write 1 Sent Done Done false
2540 1665 Acked Read 1 Sent Done Done false
2541 1666 Acked Read 5 Sent Done Done false
2542 1667 Acked Write 5 Sent Done Done false
2543 1668 Acked Write 10 Sent Done Done false
2544 1669 Acked Read 10 Sent Done Done false
2545 1670 Acked Read 10 Sent Done Done false
2546 1671 Acked Flush 0 Sent Done Done false
2547 1672 Acked Write 5 Sent Done Done false
2548 1673 Acked Write 9 Sent Done Done false
2549 1674 Acked Read 1 Sent Done Done false
2550 1675 Acked Flush 0 Sent Done Done false
2551 1676 Acked Write 10 Sent Done Done false
2552 1677 Acked Read 4 Sent Done Done false
2553 1678 Acked Flush 0 Sent Done Done false
2554 1679 Acked Read 10 Sent Done Done false
2555 1680 Acked Write 5 Sent Done Done false
2556 1681 Acked Read 10 Sent Done Done false
2557 1682 Acked Read 5 Sent Done Done false
2558 1683 Acked Read 3 Sent Done Done false
2559 1684 Acked Read 5 Sent Done Done false
2560 1685 Acked Read 2 Sent Done Done false
2561 1686 Acked Write 7 Sent Done Done false
2562 1687 Acked Read 1 Sent Done Done false
2563 1688 Acked Flush 0 Sent Done Done false
2564 1689 Acked Read 7 Sent Done Done false
2565 1690 Acked Write 3 Sent Done Done false
2566 1691 Acked Read 8 Sent Done Done false
2567 1692 Acked Read 7 Sent Done Done false
2568 1693 Acked Read 4 Sent Done Done false
2569 1694 Acked Write 2 Sent Done Done false
2570 1695 Acked Read 7 Sent Done Done false
2571 1696 Acked Read 4 Sent Done Done false
2572 1697 Acked Read 5 Sent Done Done false
2573 1698 Acked Read 3 Sent Done Done false
2574 1699 Acked Write 6 Sent Done Done false
2575 1700 Acked Read 2 Sent Done Done false
2576 1701 Acked Read 2 Sent Done Done false
2577 1702 Acked Read 1 Sent Done Done false
2578 1703 Acked Flush 0 Sent Done Done false
2579 1704 Acked Read 10 Sent Done Done false
2580 1705 Acked Read 8 Sent Done Done false
2581 1706 Acked Write 10 Sent Done Done false
2582 1707 Acked Write 3 Sent Done Done false
2583 1708 Acked Write 6 Sent Done Done false
2584 1709 Acked Read 6 Sent Done Done false
2585 1710 Acked Flush 0 Sent Done Done false
2586 1711 Acked Flush 0 Sent Done Done false
2587 1712 Acked Write 6 Sent Done Done false
2588 1713 Acked Read 4 Sent Done Done false
2589 1714 Acked Write 7 Sent Done Done false
2590 1715 Acked Write 2 Sent Done Done false
2591 1716 Acked Read 7 Sent Done Done false
2592 1717 Acked Read 8 Sent Done Done false
2593 1718 Acked Read 3 Sent Done Done false
2594 1719 Acked Write 3 Sent Done Done false
2595 1720 Acked Write 1 Sent Done Done false
2596 1721 Acked Read 1 Sent Done Done false
2597 1722 Acked Read 4 Sent Done Done false
2598 1723 Acked Read 8 Sent Done Done false
2599 1724 Acked Write 10 Sent Done Done false
2600 1725 Acked Write 8 Sent Done Done false
2601 1726 Acked Write 7 Sent Done Done false
2602 1727 Acked Read 3 Sent Done Done false
2603 1728 Acked Write 9 Sent Done Done false
2604 1729 Acked Write 5 Sent Done Done false
2605 1730 Acked Write 9 Sent Done Done false
2606 1731 Acked Write 7 Sent Done Done false
2607 1732 Acked Write 4 Sent Done Done false
2608 1733 Acked Flush 0 Sent Done Done false
2609 1734 Acked Write 2 Sent Done Done false
2610 1735 Acked Read 10 Sent Done Done false
2611 1736 Acked Write 10 Sent Done Done false
2612 1737 Acked Read 9 Sent Done Done false
2613 1738 Acked Flush 0 Sent Done Done false
2614 1739 Acked Read 3 Sent Done Done false
2615 1740 Acked Flush 0 Sent Done Done false
2616 1741 Acked Read 1 Sent Done Done false
2617 1742 Acked Write 4 Sent Done Done false
2618 1743 Acked Read 9 Sent Done Done false
2619 1744 Acked Read 10 Sent Done Done false
2620 1745 Acked Write 1 Sent Done Done false
2621 1746 Acked Read 2 Sent Done Done false
2622 1747 Acked Write 3 Sent Done Done false
2623 1748 Acked Flush 0 Sent Done Done false
2624 1749 Acked Write 2 Sent Done Done false
2625 1750 Acked Read 7 Sent Done Done false
2626 1751 Acked Read 3 Sent Done Done false
2627 1752 Acked Read 3 Sent Done Done false
2628 1753 Acked Write 3 Sent Done Done false
2629 1754 Acked Flush 0 Sent Done Done false
2630 1755 Acked Read 9 Sent Done Done false
2631 1756 Acked Write 4 Sent Done Done false
2632 1757 Acked Write 9 Sent Done Done false
2633 1758 Acked Write 3 Sent Done Done false
2634 1759 Acked Read 3 Sent Done Done false
2635 1760 Acked Write 3 Sent Done Done false
2636 1761 Acked Read 3 Sent Done Done false
2637 1762 Acked Read 4 Sent Done Done false
2638 1763 Acked Read 10 Sent Done Done false
2639 1764 Acked Read 8 Sent Done Done false
2640 1765 Acked Write 8 Sent Done Done false
2641 1766 Acked Read 3 Sent Done Done false
2642 1767 Acked Write 1 Sent Done Done false
2643 1768 Acked Read 10 Sent Done Done false
2644 1769 Acked Write 1 Sent Done Done false
2645 1770 Acked Write 9 Sent Done Done false
2646 1771 Acked Flush 0 Sent Done Done false
2647 1772 Acked Read 6 Sent Done Done false
2648 1773 Acked Read 1 Sent Done Done false
2649 1774 Acked Write 7 Sent Done Done false
2650 1775 Acked Read 10 Sent Done Done false
2651 1776 Acked Read 1 Sent Done Done false
2652 1777 Acked Read 9 Sent Done Done false
2653 1778 Acked Write 2 Sent Done Done false
2654 1779 Acked Flush 0 Sent Done Done false
2655 1780 Acked Read 7 Sent Done Done false
2656 1781 Acked Read 6 Sent Done Done false
2657 1782 Acked Write 5 Sent Done Done false
2658 1783 Acked Read 3 Sent Done Done false
2659 1784 Acked Write 3 Sent Done Done false
2660 1785 Acked Write 9 Sent Done Done false
2661 1786 Acked Read 9 Sent Done Done false
2662 1787 Acked Read 1 Sent Done Done false
2663 1788 Acked Flush 0 Sent Done Done false
2664 1789 Acked Write 5 Sent Done Done false
2665 1790 Acked Read 2 Sent Done Done false
2666 1791 Acked Write 7 Sent Done Done false
2667 1792 Acked Read 3 Sent Done Done false
2668 1793 Acked Write 5 Sent Done Done false
2669 1794 Acked Write 3 Sent Done Done false
2670 1795 Acked Read 2 Sent Done Done false
2671 1796 Acked Write 3 Sent Done Done false
2672 1797 Acked Write 1 Sent Done Done false
2673 1798 Acked Write 3 Sent Done Done false
2674 1799 Acked Write 3 Sent Done Done false
2675 1800 Acked Read 2 Sent Done Done false
2676 1801 Acked Read 2 Sent Done Done false
2677 1802 Acked Write 4 Sent Done Done false
2678 1803 Acked Write 10 Sent Done Done false
2679 1804 Acked Read 9 Sent Done Done false
2680 1805 Acked Flush 0 Sent Done Done false
2681 1806 Acked Write 6 Sent Done Done false
2682 1807 Acked Write 7 Sent Done Done false
2683 1808 Acked Read 7 Sent Done Done false
2684 1809 Acked Read 5 Sent Done Done false
2685 1810 Acked Read 9 Sent Done Done false
2686 1811 Acked Read 2 Sent Done Done false
2687 1812 Acked Read 5 Sent Done Done false
2688 1813 Acked Read 4 Sent Done Done false
2689 1814 Acked Flush 0 Sent Done Done false
2690 1815 Acked Flush 0 Sent Done Done false
2691 1816 Acked Write 9 Sent Done Done false
2692 1817 Acked Write 1 Sent Done Done false
2693 1818 Acked Read 1 Sent Done Done false
2694 1819 Acked Read 5 Sent Done Done false
2695 1820 Acked Write 8 Sent Done Done false
2696 1821 Acked Write 9 Sent Done Done false
2697 1822 Acked Read 9 Sent Done Done false
2698 1823 Acked Write 8 Sent Done Done false
2699 1824 Acked Write 8 Sent Done Done false
2700 1825 Acked Read 10 Sent Done Done false
2701 1826 Acked Write 4 Sent Done Done false
2702 1827 Acked Read 8 Sent Done Done false
2703 1828 Acked Flush 0 Sent Done Done false
2704 1829 Acked Read 6 Sent Done Done false
2705 1830 Acked Read 7 Sent Done Done false
2706 1831 Acked Write 7 Sent Done Done false
2707 1832 Acked Write 1 Sent Done Done false
2708 1833 Acked Read 4 Sent Done Done false
2709 1834 Acked Read 2 Sent Done Done false
2710 1835 Acked Read 7 Sent Done Done false
2711 1836 Acked Write 8 Sent Done Done false
2712 1837 Acked Read 9 Sent Done Done false
2713 1838 Acked Write 9 Sent Done Done false
2714 1839 Acked Read 2 Sent Done Done false
2715 1840 Acked Flush 0 Sent Done Done false
2716 1841 Acked Read 8 Sent Done Done false
2717 1842 Acked Read 4 Sent Done Done false
2718 1843 Acked Read 3 Sent Done Done false
2719 1844 Acked Read 3 Sent Done Done false
2720 1845 Acked Read 2 Sent Done Done false
2721 1846 Acked Write 2 Sent Done Done false
2722 1847 Acked Read 10 Sent Done Done false
2723 1848 Acked Read 1 Sent Done Done false
2724 1849 Acked Read 1 Sent Done Done false
2725 1850 Acked Write 10 Sent Done Done false
2726 1851 Acked Write 6 Sent Done Done false
2727 1852 Acked Read 3 Sent Done Done false
2728 1853 Acked Write 9 Sent Done Done false
2729 1854 Acked Read 3 Sent Done Done false
2730 1855 Acked Read 5 Sent Done Done false
2731 1856 Acked Write 6 Sent Done Done false
2732 1857 Acked Read 3 Sent Done Done false
2733 1858 Acked Read 1 Sent Done Done false
2734 1859 Acked Read 8 Sent Done Done false
2735 1860 Acked Read 3 Sent Done Done false
2736 1861 Acked Write 5 Sent Done Done false
2737 1862 Acked Read 9 Sent Done Done false
2738 1863 Acked Read 4 Sent Done Done false
2739 1864 Acked Write 10 Sent Done Done false
2740 1865 Acked Read 8 Sent Done Done false
2741 1866 Acked Read 2 Sent Done Done false
2742 1867 Acked Read 2 Sent Done Done false
2743 1868 Acked Flush 0 Sent Done Done false
2744 1869 Acked Write 1 Sent Done Done false
2745 1870 Acked Write 3 Sent Done Done false
2746 1871 Acked Write 6 Sent Done Done false
2747 1872 Acked Read 10 Sent Done Done false
2748 1873 Acked Read 2 Sent Done Done false
2749 1874 Acked Write 6 Sent Done Done false
2750 1875 Acked Flush 0 Sent Done Done false
2751 1876 Acked Flush 0 Sent Done Done false
2752 1877 Acked Write 1 Sent Done Done false
2753 1878 Acked Read 4 Sent Done Done false
2754 1879 Acked Read 7 Sent Done Done false
2755 1880 Acked Flush 0 Sent Done Done false
2756 1881 Acked Write 3 Sent Done Done false
2757 1882 Acked Read 1 Sent Done Done false
2758 1883 Acked Read 8 Sent Done Done false
2759 1884 Acked Write 7 Sent Done Done false
2760 1885 Acked Write 10 Sent Done Done false
2761 1886 Acked Write 9 Sent Done Done false
2762 1887 Acked Read 7 Sent Done Done false
2763 1888 Acked Write 9 Sent Done Done false
2764 1889 Acked Read 4 Sent Done Done false
2765 1890 Acked Write 6 Sent Done Done false
2766 1891 Acked Read 7 Sent Done Done false
2767 1892 Acked Read 6 Sent Done Done false
2768 1893 Acked Read 6 Sent Done Done false
2769 1894 Acked Write 9 Sent Done Done false
2770 1895 Acked Read 6 Sent Done Done false
2771 1896 Acked Read 4 Sent Done Done false
2772 1897 Acked Read 7 Sent Done Done false
2773 1898 Acked Write 3 Sent Done Done false
2774 1899 Acked Flush 0 Sent Done Done false
2775 1900 Acked Write 8 Sent Done Done false
2776 1901 Acked Read 6 Sent Done Done false
2777 1902 Acked Write 5 Sent Done Done false
2778 1903 Acked Write 3 Sent Done Done false
2779 1904 Acked Flush 0 Sent Done Done false
2780 STATES DS:0 DS:1 DS:2 TOTAL
2781 Sent 301 0 0 301
2782 Done 0 301 301 602
2783 Skipped 0 0 0 0
2784 Error 0 0 0 0
2785 Last Flush: 1603 1904 1904
2786 Downstairs last five completed: 1603 1602 1601 1600 1599
2787 Upstairs last five completed: 1904 1903 1902 1901 1900
2788 CLIENT: Up:0 ds:301 act:2
2789 [8810] got start action
2790 [8810] I am going to restart
2791 Starting downstairs at port 8810
2792 Make output file at "/tmp/dsc/downstairs-8810.txt"
2793 [8810][0] reports Starting
2794 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1205)
2795 [8810][0] reports Running
2796 ----------------------------------------------------------------
2797 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
2798 States: Offline Active Active
2799 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2800 1604 Acked Write 10 Sent Done Done false
2801 1605 Acked Write 9 Sent Done Done false
2802 1606 Acked Write 9 Sent Done Done false
2803 1607 Acked Flush 0 Sent Done Done false
2804 1608 Acked Write 4 Sent Done Done false
2805 1609 Acked Write 3 Sent Done Done false
2806 1610 Acked Read 10 Sent Done Done false
2807 1611 Acked Read 7 Sent Done Done false
2808 1612 Acked Read 9 Sent Done Done false
2809 1613 Acked Write 4 Sent Done Done false
2810 1614 Acked Flush 0 Sent Done Done false
2811 1615 Acked Read 1 Sent Done Done false
2812 1616 Acked Read 7 Sent Done Done false
2813 1617 Acked Read 5 Sent Done Done false
2814 1618 Acked Read 5 Sent Done Done false
2815 1619 Acked Read 2 Sent Done Done false
2816 1620 Acked Write 1 Sent Done Done false
2817 1621 Acked Read 7 Sent Done Done false
2818 1622 Acked Read 5 Sent Done Done false
2819 1623 Acked Read 5 Sent Done Done false
2820 1624 Acked Read 10 Sent Done Done false
2821 1625 Acked Read 2 Sent Done Done false
2822 1626 Acked Flush 0 Sent Done Done false
2823 1627 Acked Write 2 Sent Done Done false
2824 1628 Acked Write 5 Sent Done Done false
2825 1629 Acked Write 7 Sent Done Done false
2826 1630 Acked Write 3 Sent Done Done false
2827 1631 Acked Read 10 Sent Done Done false
2828 1632 Acked Read 9 Sent Done Done false
2829 1633 Acked Write 4 Sent Done Done false
2830 1634 Acked Write 4 Sent Done Done false
2831 1635 Acked Flush 0 Sent Done Done false
2832 1636 Acked Read 7 Sent Done Done false
2833 1637 Acked Read 6 Sent Done Done false
2834 1638 Acked Write 2 Sent Done Done false
2835 1639 Acked Write 6 Sent Done Done false
2836 1640 Acked Write 5 Sent Done Done false
2837 1641 Acked Read 4 Sent Done Done false
2838 1642 Acked Read 4 Sent Done Done false
2839 1643 Acked Write 10 Sent Done Done false
2840 1644 Acked Flush 0 Sent Done Done false
2841 1645 Acked Write 5 Sent Done Done false
2842 1646 Acked Read 9 Sent Done Done false
2843 1647 Acked Write 8 Sent Done Done false
2844 1648 Acked Read 6 Sent Done Done false
2845 1649 Acked Flush 0 Sent Done Done false
2846 1650 Acked Write 5 Sent Done Done false
2847 1651 Acked Write 9 Sent Done Done false
2848 1652 Acked Read 10 Sent Done Done false
2849 1653 Acked Write 7 Sent Done Done false
2850 1654 Acked Flush 0 Sent Done Done false
2851 1655 Acked Flush 0 Sent Done Done false
2852 1656 Acked Read 2 Sent Done Done false
2853 1657 Acked Flush 0 Sent Done Done false
2854 1658 Acked Read 6 Sent Done Done false
2855 1659 Acked Write 2 Sent Done Done false
2856 1660 Acked Read 5 Sent Done Done false
2857 1661 Acked Flush 0 Sent Done Done false
2858 1662 Acked Read 5 Sent Done Done false
2859 1663 Acked Read 10 Sent Done Done false
2860 1664 Acked Write 1 Sent Done Done false
2861 1665 Acked Read 1 Sent Done Done false
2862 1666 Acked Read 5 Sent Done Done false
2863 1667 Acked Write 5 Sent Done Done false
2864 1668 Acked Write 10 Sent Done Done false
2865 1669 Acked Read 10 Sent Done Done false
2866 1670 Acked Read 10 Sent Done Done false
2867 1671 Acked Flush 0 Sent Done Done false
2868 1672 Acked Write 5 Sent Done Done false
2869 1673 Acked Write 9 Sent Done Done false
2870 1674 Acked Read 1 Sent Done Done false
2871 1675 Acked Flush 0 Sent Done Done false
2872 1676 Acked Write 10 Sent Done Done false
2873 1677 Acked Read 4 Sent Done Done false
2874 1678 Acked Flush 0 Sent Done Done false
2875 1679 Acked Read 10 Sent Done Done false
2876 1680 Acked Write 5 Sent Done Done false
2877 1681 Acked Read 10 Sent Done Done false
2878 1682 Acked Read 5 Sent Done Done false
2879 1683 Acked Read 3 Sent Done Done false
2880 1684 Acked Read 5 Sent Done Done false
2881 1685 Acked Read 2 Sent Done Done false
2882 1686 Acked Write 7 Sent Done Done false
2883 1687 Acked Read 1 Sent Done Done false
2884 1688 Acked Flush 0 Sent Done Done false
2885 1689 Acked Read 7 Sent Done Done false
2886 1690 Acked Write 3 Sent Done Done false
2887 1691 Acked Read 8 Sent Done Done false
2888 1692 Acked Read 7 Sent Done Done false
2889 1693 Acked Read 4 Sent Done Done false
2890 1694 Acked Write 2 Sent Done Done false
2891 1695 Acked Read 7 Sent Done Done false
2892 1696 Acked Read 4 Sent Done Done false
2893 1697 Acked Read 5 Sent Done Done false
2894 1698 Acked Read 3 Sent Done Done false
2895 1699 Acked Write 6 Sent Done Done false
2896 1700 Acked Read 2 Sent Done Done false
2897 1701 Acked Read 2 Sent Done Done false
2898 1702 Acked Read 1 Sent Done Done false
2899 1703 Acked Flush 0 Sent Done Done false
2900 1704 Acked Read 10 Sent Done Done false
2901 1705 Acked Read 8 Sent Done Done false
2902 1706 Acked Write 10 Sent Done Done false
2903 1707 Acked Write 3 Sent Done Done false
2904 1708 Acked Write 6 Sent Done Done false
2905 1709 Acked Read 6 Sent Done Done false
2906 1710 Acked Flush 0 Sent Done Done false
2907 1711 Acked Flush 0 Sent Done Done false
2908 1712 Acked Write 6 Sent Done Done false
2909 1713 Acked Read 4 Sent Done Done false
2910 1714 Acked Write 7 Sent Done Done false
2911 1715 Acked Write 2 Sent Done Done false
2912 1716 Acked Read 7 Sent Done Done false
2913 1717 Acked Read 8 Sent Done Done false
2914 1718 Acked Read 3 Sent Done Done false
2915 1719 Acked Write 3 Sent Done Done false
2916 1720 Acked Write 1 Sent Done Done false
2917 1721 Acked Read 1 Sent Done Done false
2918 1722 Acked Read 4 Sent Done Done false
2919 1723 Acked Read 8 Sent Done Done false
2920 1724 Acked Write 10 Sent Done Done false
2921 1725 Acked Write 8 Sent Done Done false
2922 1726 Acked Write 7 Sent Done Done false
2923 1727 Acked Read 3 Sent Done Done false
2924 1728 Acked Write 9 Sent Done Done false
2925 1729 Acked Write 5 Sent Done Done false
2926 1730 Acked Write 9 Sent Done Done false
2927 1731 Acked Write 7 Sent Done Done false
2928 1732 Acked Write 4 Sent Done Done false
2929 1733 Acked Flush 0 Sent Done Done false
2930 1734 Acked Write 2 Sent Done Done false
2931 1735 Acked Read 10 Sent Done Done false
2932 1736 Acked Write 10 Sent Done Done false
2933 1737 Acked Read 9 Sent Done Done false
2934 1738 Acked Flush 0 Sent Done Done false
2935 1739 Acked Read 3 Sent Done Done false
2936 1740 Acked Flush 0 Sent Done Done false
2937 1741 Acked Read 1 Sent Done Done false
2938 1742 Acked Write 4 Sent Done Done false
2939 1743 Acked Read 9 Sent Done Done false
2940 1744 Acked Read 10 Sent Done Done false
2941 1745 Acked Write 1 Sent Done Done false
2942 1746 Acked Read 2 Sent Done Done false
2943 1747 Acked Write 3 Sent Done Done false
2944 1748 Acked Flush 0 Sent Done Done false
2945 1749 Acked Write 2 Sent Done Done false
2946 1750 Acked Read 7 Sent Done Done false
2947 1751 Acked Read 3 Sent Done Done false
2948 1752 Acked Read 3 Sent Done Done false
2949 1753 Acked Write 3 Sent Done Done false
2950 1754 Acked Flush 0 Sent Done Done false
2951 1755 Acked Read 9 Sent Done Done false
2952 1756 Acked Write 4 Sent Done Done false
2953 1757 Acked Write 9 Sent Done Done false
2954 1758 Acked Write 3 Sent Done Done false
2955 1759 Acked Read 3 Sent Done Done false
2956 1760 Acked Write 3 Sent Done Done false
2957 1761 Acked Read 3 Sent Done Done false
2958 1762 Acked Read 4 Sent Done Done false
2959 1763 Acked Read 10 Sent Done Done false
2960 1764 Acked Read 8 Sent Done Done false
2961 1765 Acked Write 8 Sent Done Done false
2962 1766 Acked Read 3 Sent Done Done false
2963 1767 Acked Write 1 Sent Done Done false
2964 1768 Acked Read 10 Sent Done Done false
2965 1769 Acked Write 1 Sent Done Done false
2966 1770 Acked Write 9 Sent Done Done false
2967 1771 Acked Flush 0 Sent Done Done false
2968 1772 Acked Read 6 Sent Done Done false
2969 1773 Acked Read 1 Sent Done Done false
2970 1774 Acked Write 7 Sent Done Done false
2971 1775 Acked Read 10 Sent Done Done false
2972 1776 Acked Read 1 Sent Done Done false
2973 1777 Acked Read 9 Sent Done Done false
2974 1778 Acked Write 2 Sent Done Done false
2975 1779 Acked Flush 0 Sent Done Done false
2976 1780 Acked Read 7 Sent Done Done false
2977 1781 Acked Read 6 Sent Done Done false
2978 1782 Acked Write 5 Sent Done Done false
2979 1783 Acked Read 3 Sent Done Done false
2980 1784 Acked Write 3 Sent Done Done false
2981 1785 Acked Write 9 Sent Done Done false
2982 1786 Acked Read 9 Sent Done Done false
2983 1787 Acked Read 1 Sent Done Done false
2984 1788 Acked Flush 0 Sent Done Done false
2985 1789 Acked Write 5 Sent Done Done false
2986 1790 Acked Read 2 Sent Done Done false
2987 1791 Acked Write 7 Sent Done Done false
2988 1792 Acked Read 3 Sent Done Done false
2989 1793 Acked Write 5 Sent Done Done false
2990 1794 Acked Write 3 Sent Done Done false
2991 1795 Acked Read 2 Sent Done Done false
2992 1796 Acked Write 3 Sent Done Done false
2993 1797 Acked Write 1 Sent Done Done false
2994 1798 Acked Write 3 Sent Done Done false
2995 1799 Acked Write 3 Sent Done Done false
2996 1800 Acked Read 2 Sent Done Done false
2997 1801 Acked Read 2 Sent Done Done false
2998 1802 Acked Write 4 Sent Done Done false
2999 1803 Acked Write 10 Sent Done Done false
3000 1804 Acked Read 9 Sent Done Done false
3001 1805 Acked Flush 0 Sent Done Done false
3002 1806 Acked Write 6 Sent Done Done false
3003 1807 Acked Write 7 Sent Done Done false
3004 1808 Acked Read 7 Sent Done Done false
3005 1809 Acked Read 5 Sent Done Done false
3006 1810 Acked Read 9 Sent Done Done false
3007 1811 Acked Read 2 Sent Done Done false
3008 1812 Acked Read 5 Sent Done Done false
3009 1813 Acked Read 4 Sent Done Done false
3010 1814 Acked Flush 0 Sent Done Done false
3011 1815 Acked Flush 0 Sent Done Done false
3012 1816 Acked Write 9 Sent Done Done false
3013 1817 Acked Write 1 Sent Done Done false
3014 1818 Acked Read 1 Sent Done Done false
3015 1819 Acked Read 5 Sent Done Done false
3016 1820 Acked Write 8 Sent Done Done false
3017 1821 Acked Write 9 Sent Done Done false
3018 1822 Acked Read 9 Sent Done Done false
3019 1823 Acked Write 8 Sent Done Done false
3020 1824 Acked Write 8 Sent Done Done false
3021 1825 Acked Read 10 Sent Done Done false
3022 1826 Acked Write 4 Sent Done Done false
3023 1827 Acked Read 8 Sent Done Done false
3024 1828 Acked Flush 0 Sent Done Done false
3025 1829 Acked Read 6 Sent Done Done false
3026 1830 Acked Read 7 Sent Done Done false
3027 1831 Acked Write 7 Sent Done Done false
3028 1832 Acked Write 1 Sent Done Done false
3029 1833 Acked Read 4 Sent Done Done false
3030 1834 Acked Read 2 Sent Done Done false
3031 1835 Acked Read 7 Sent Done Done false
3032 1836 Acked Write 8 Sent Done Done false
3033 1837 Acked Read 9 Sent Done Done false
3034 1838 Acked Write 9 Sent Done Done false
3035 1839 Acked Read 2 Sent Done Done false
3036 1840 Acked Flush 0 Sent Done Done false
3037 1841 Acked Read 8 Sent Done Done false
3038 1842 Acked Read 4 Sent Done Done false
3039 1843 Acked Read 3 Sent Done Done false
3040 1844 Acked Read 3 Sent Done Done false
3041 1845 Acked Read 2 Sent Done Done false
3042 1846 Acked Write 2 Sent Done Done false
3043 1847 Acked Read 10 Sent Done Done false
3044 1848 Acked Read 1 Sent Done Done false
3045 1849 Acked Read 1 Sent Done Done false
3046 1850 Acked Write 10 Sent Done Done false
3047 1851 Acked Write 6 Sent Done Done false
3048 1852 Acked Read 3 Sent Done Done false
3049 1853 Acked Write 9 Sent Done Done false
3050 1854 Acked Read 3 Sent Done Done false
3051 1855 Acked Read 5 Sent Done Done false
3052 1856 Acked Write 6 Sent Done Done false
3053 1857 Acked Read 3 Sent Done Done false
3054 1858 Acked Read 1 Sent Done Done false
3055 1859 Acked Read 8 Sent Done Done false
3056 1860 Acked Read 3 Sent Done Done false
3057 1861 Acked Write 5 Sent Done Done false
3058 1862 Acked Read 9 Sent Done Done false
3059 1863 Acked Read 4 Sent Done Done false
3060 1864 Acked Write 10 Sent Done Done false
3061 1865 Acked Read 8 Sent Done Done false
3062 1866 Acked Read 2 Sent Done Done false
3063 1867 Acked Read 2 Sent Done Done false
3064 1868 Acked Flush 0 Sent Done Done false
3065 1869 Acked Write 1 Sent Done Done false
3066 1870 Acked Write 3 Sent Done Done false
3067 1871 Acked Write 6 Sent Done Done false
3068 1872 Acked Read 10 Sent Done Done false
3069 1873 Acked Read 2 Sent Done Done false
3070 1874 Acked Write 6 Sent Done Done false
3071 1875 Acked Flush 0 Sent Done Done false
3072 1876 Acked Flush 0 Sent Done Done false
3073 1877 Acked Write 1 Sent Done Done false
3074 1878 Acked Read 4 Sent Done Done false
3075 1879 Acked Read 7 Sent Done Done false
3076 1880 Acked Flush 0 Sent Done Done false
3077 1881 Acked Write 3 Sent Done Done false
3078 1882 Acked Read 1 Sent Done Done false
3079 1883 Acked Read 8 Sent Done Done false
3080 1884 Acked Write 7 Sent Done Done false
3081 1885 Acked Write 10 Sent Done Done false
3082 1886 Acked Write 9 Sent Done Done false
3083 1887 Acked Read 7 Sent Done Done false
3084 1888 Acked Write 9 Sent Done Done false
3085 1889 Acked Read 4 Sent Done Done false
3086 1890 Acked Write 6 Sent Done Done false
3087 1891 Acked Read 7 Sent Done Done false
3088 1892 Acked Read 6 Sent Done Done false
3089 1893 Acked Read 6 Sent Done Done false
3090 1894 Acked Write 9 Sent Done Done false
3091 1895 Acked Read 6 Sent Done Done false
3092 1896 Acked Read 4 Sent Done Done false
3093 1897 Acked Read 7 Sent Done Done false
3094 1898 Acked Write 3 Sent Done Done false
3095 1899 Acked Flush 0 Sent Done Done false
3096 1900 Acked Write 8 Sent Done Done false
3097 1901 Acked Read 6 Sent Done Done false
3098 1902 Acked Write 5 Sent Done Done false
3099 1903 Acked Write 3 Sent Done Done false
3100 1904 Acked Flush 0 Sent Done Done false
3101 STATES DS:0 DS:1 DS:2 TOTAL
3102 Sent 301 0 0 301
3103 Done 0 301 301 602
3104 Skipped 0 0 0 0
3105 Error 0 0 0 0
3106 Last Flush: 1603 1904 1904
3107 Downstairs last five completed: 1603 1602 1601 1600 1599
3108 Upstairs last five completed: 1904 1903 1902 1901 1900
3109 CLIENT: Up:0 ds:301 act:2
31102025-01-14T16:11:14.410ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
31112025-01-14T16:11:14.410ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38874) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
31122025-01-14T16:11:14.411ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
31132025-01-14T16:11:14.412ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
31142025-01-14T16:11:14.412ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
31152025-01-14T16:11:14.412ZINFOcrucible: send last flush ID to this DS: 1603 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
31162025-01-14T16:11:14.413ZINFOcrucible: Replied this last flush ID: 1603 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
31172025-01-14T16:11:14.413ZINFOcrucible: [0] client re-new 301 jobs since flush 1603 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
31182025-01-14T16:11:14.413ZINFOcrucible: [0] Replayed 301 jobs since flush: 1603 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
3119 ----------------------------------------------------------------
3120 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
3121 Downstairs last five completed: 1904 1903 1902 1901 1900
3122 Upstairs last five completed: 1904 1903 1902 1901 1900
3123 CLIENT: Up:0 ds:0 act:3
3124 Replay: All jobs finished
3125 Main task has work to do, go find it
3126 got dsc Stop(0)
3127 stop 0
3128 [8810] Got stop action so:false kr:false
3129 Jan 14 16:11:16.645 INFO request completed, latency_us: 348, response_code: 204, uri: /stop/cid/0, method: POST, req_id: cd6b95d8-baa3-48fe-8f2f-76447d2b8189, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
31302025-01-14T16:11:16.645ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
31312025-01-14T16:11:16.645ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
31322025-01-14T16:11:16.645ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
3133 Jan 14 16:11:16.645 INFO request completed, latency_us: 145, response_code: 200, uri: /state/cid/0, method: GET, req_id: fe688435-81d1-473b-a4fe-35d14390adb9, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
31342025-01-14T16:11:16.646ZINFOcrucible: client task is exiting = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
3135 [8810] Exited with: ExitStatus(unix_wait_status(9))
3136 [8810][0] reports Exit
3137 Jan 14 16:11:20.647 INFO request completed, latency_us: 242, response_code: 200, uri: /state/cid/0, method: GET, req_id: 2f0efb86-a146-48fa-b297-c64390549229, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
3138 001/300 Read block 1480 len 4096
3139 002/300 Read block 3239 len 24576
3140 003/300 Read block 538 len 24576
3141 004/300 Read block 4596 len 24576
3142 005/300 Read block 3676 len 20480
3143 006/300 Write block 4953 len 32768 data: 3 2 2 3 3 3 3 2
3144 007/300 Write block 288 len 32768 data: 2 2 2 2 2 2 2 2
3145 008/300 Write block 4675 len 24576 data: 2 2 2 2 2 2
3146 009/300 Flush
3147 010/300 Read block 4993 len 12288
3148 011/300 Read block 4730 len 4096
3149 012/300 Read block 4892 len 28672
3150 013/300 Write block 449 len 4096 data: 2
3151 014/300 Write block 3633 len 16384 data: 3 3 3 3
3152 015/300 Write block 3937 len 28672 data: 2 2 2 2 2 2 3
3153 016/300 Write block 1432 len 16384 data: 3 2 2 2
3154 017/300 Read block 2979 len 28672
3155 018/300 Write block 1937 len 40960 data: 3 2 2 2 2 2 2 2 2 2
3156 019/300 Flush
3157 020/300 Write block 2836 len 4096 data: 3
3158 021/300 Read block 3029 len 36864
3159 022/300 Read block 902 len 4096
3160 023/300 Flush
3161 024/300 Write block 2701 len 36864 data: 3 3 3 2 2 2 2 2 2
3162 025/300 Write block 2573 len 12288 data: 2 2 2
3163 026/300 Flush
3164 027/300 Read block 2014 len 28672
3165 028/300 Write block 1524 len 32768 data: 2 2 2 2 2 2 2 2
3166 029/300 Write block 1012 len 40960 data: 2 2 2 2 2 3 3 3 3 3
3167 030/300 Write block 1695 len 32768 data: 3 4 4 3 3 2 2 2
3168 031/300 Read block 2003 len 20480
3169 032/300 Read block 3904 len 36864
3170 033/300 Read block 1375 len 36864
3171 034/300 Read block 382 len 8192
3172 035/300 Flush
3173 036/300 Write block 3762 len 36864 data: 2 2 2 2 2 3 3 3 3
3174 037/300 Read block 3376 len 36864
3175 038/300 Read block 1650 len 32768
3176 039/300 Read block 2062 len 28672
3177 040/300 Read block 550 len 20480
3178 041/300 Write block 3439 len 20480 data: 2 2 2 2 2
3179 042/300 Read block 3102 len 36864
3180 043/300 Write block 3896 len 4096 data: 3
3181 044/300 Read block 4140 len 8192
3182 045/300 Flush
3183 046/300 Read block 2589 len 8192
3184 047/300 Flush
3185 048/300 Write block 2562 len 36864 data: 4 4 4 3 4 3 3 2 2
3186 049/300 Read block 4260 len 28672
3187 050/300 Write block 1674 len 28672 data: 2 2 2 2 2 2 2
3188 051/300 Write block 1007 len 40960 data: 2 2 2 2 2 3 3 3 3 3
3189 052/300 Read block 252 len 40960
3190 053/300 Write block 479 len 32768 data: 2 3 3 3 3 3 2 2
3191 054/300 Write block 1069 len 16384 data: 2 2 3 3
3192 055/300 Read block 4662 len 20480
3193 056/300 Read block 4824 len 24576
3194 057/300 Read block 3063 len 24576
3195 058/300 Read block 1614 len 24576
3196 059/300 Read block 3914 len 20480
3197 060/300 Read block 2682 len 32768
3198 061/300 Write block 251 len 28672 data: 3 3 3 3 3 3 2
3199 062/300 Read block 1419 len 28672
3200 063/300 Read block 960 len 32768
3201 064/300 Write block 159 len 32768 data: 2 2 2 2 2 2 2 2
3202 065/300 Read block 442 len 4096
3203 066/300 Write block 4467 len 36864 data: 2 2 2 2 2 2 2 2 2
3204 067/300 Read block 146 len 24576
3205 068/300 Read block 166 len 28672
3206 069/300 Write block 3713 len 40960 data: 3 3 3 3 3 3 2 2 2 2
3207 070/300 Read block 4529 len 8192
3208 071/300 Flush
3209 072/300 Read block 1585 len 12288
3210 073/300 Read block 4642 len 40960
3211 074/300 Write block 170 len 20480 data: 2 2 2 2 2
3212 075/300 Read block 4168 len 4096
3213 076/300 Flush
3214 077/300 Read block 4361 len 24576
3215 078/300 Read block 871 len 32768
3216 079/300 Write block 334 len 20480 data: 2 2 2 2 2
3217 080/300 Write block 3196 len 32768 data: 3 3 3 2 2 2 2 2
3218 081/300 Read block 2786 len 8192
3219 082/300 Read block 4614 len 16384
3220 083/300 Read block 2775 len 8192
3221 084/300 Write block 407 len 20480 data: 3 3 4 4 3
3222 085/300 Read block 416 len 4096
3223 086/300 Read block 1406 len 36864
3224 087/300 Read block 3687 len 4096
3225 088/300 Read block 3990 len 8192
3226 089/300 Write block 307 len 32768 data: 2 2 2 2 2 2 2 2
3227 090/300 Flush
3228 091/300 Read block 4227 len 40960
3229 092/300 Write block 4888 len 16384 data: 2 2 2 2
3230 093/300 Flush
3231 094/300 Write block 1296 len 20480 data: 2 2 2 2 2
3232 095/300 Read block 4621 len 20480
3233 096/300 Write block 690 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3234 097/300 Read block 3794 len 40960
3235 098/300 Read block 1574 len 8192
3236 099/300 Flush
3237 100/300 Read block 1088 len 36864
3238 101/300 Write block 1808 len 16384 data: 2 3 3 3
3239 102/300 Read block 50 len 40960
3240 103/300 Write block 3665 len 8192 data: 3 3
3241 104/300 Read block 3062 len 12288
3242 105/300 Flush
3243 106/300 Read block 3295 len 40960
3244 107/300 Read block 1547 len 28672
3245 108/300 Read block 1190 len 32768
3246 109/300 Read block 2106 len 4096
3247 110/300 Read block 2082 len 32768
3248 111/300 Write block 2099 len 12288 data: 2 2 2
3249 112/300 Read block 2604 len 12288
3250 113/300 Write block 2432 len 12288 data: 2 2 2
3251 114/300 Flush
3252 115/300 Read block 4663 len 28672
3253 116/300 Write block 1652 len 32768 data: 3 2 2 3 3 2 2 2
3254 117/300 Write block 3704 len 40960 data: 2 2 2 2 2 2 3 3 3 4
3255 118/300 Read block 4574 len 24576
3256 119/300 Read block 3059 len 40960
3257 120/300 Read block 4670 len 40960
3258 121/300 Write block 1204 len 16384 data: 2 2 2 2
3259 122/300 Read block 3038 len 12288
3260 123/300 Write block 3234 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3261 124/300 Write block 260 len 20480 data: 2 2 2 2 2
3262 125/300 Flush
3263 126/300 Write block 2567 len 28672 data: 4 4 3 3 2 2 3
3264 127/300 Write block 4365 len 40960 data: 2 2 2 2 2 2 2 3 3 3
3265 128/300 Flush
3266 129/300 Write block 2330 len 16384 data: 2 2 2 2
3267 130/300 Write block 1355 len 36864 data: 2 2 2 3 2 2 2 2 2
3268 131/300 Write block 733 len 8192 data: 2 2
3269 132/300 Read block 3359 len 4096
3270 133/300 Read block 2588 len 4096
3271 134/300 Write block 2558 len 28672 data: 2 3 4 4 5 5 5
3272 135/300 Read block 348 len 36864
3273 136/300 Write block 2642 len 36864 data: 3 3 3 3 2 2 2 2 2
3274 137/300 Read block 4028 len 36864
3275 138/300 Write block 994 len 32768 data: 2 2 2 2 2 2 2 2
3276 139/300 Read block 1186 len 40960
3277 140/300 Write block 2354 len 8192 data: 2 2
3278 141/300 Write block 523 len 20480 data: 3 2 2 2 2
3279 142/300 Read block 1915 len 4096
3280 143/300 Flush
3281 144/300 Write block 4768 len 32768 data: 2 2 2 2 2 2 2 2
3282 145/300 Write block 3029 len 16384 data: 3 3 3 2
3283 146/300 Read block 1994 len 8192
3284 147/300 Write block 4653 len 4096 data: 2
3285 148/300 Read block 2742 len 20480
3286 149/300 Read block 4604 len 32768
3287 150/300 Read block 3757 len 20480
3288 151/300 Read block 2319 len 24576
3289 152/300 Read block 3493 len 20480
3290 153/300 Read block 599 len 4096
3291 154/300 Flush
3292 155/300 Write block 2000 len 8192 data: 2 2
3293 156/300 Write block 2831 len 36864 data: 3 2 2 2 2 4 3 2 2
3294 157/300 Write block 1900 len 16384 data: 2 2 2 2
3295 158/300 Read block 2303 len 4096
3296 159/300 Write block 2622 len 20480 data: 2 2 2 2 2
3297 160/300 Read block 4709 len 32768
3298 161/300 Write block 1978 len 28672 data: 2 2 2 2 2 2 2
3299 162/300 Read block 2510 len 40960
3300 163/300 Read block 126 len 28672
3301 164/300 Read block 2030 len 4096
3302 165/300 Write block 2474 len 36864 data: 2 2 2 2 2 2 2 3 3
3303 166/300 Read block 832 len 4096
3304 167/300 Read block 4034 len 28672
3305 168/300 Read block 4417 len 4096
3306 169/300 Write block 3760 len 36864 data: 2 2 3 3 3 3 3 4 4
3307 170/300 Write block 1833 len 24576 data: 4 3 3 3 3 3
3308 171/300 Write block 173 len 16384 data: 3 3 2 2
3309 172/300 Read block 4604 len 24576
3310 173/300 Read block 4169 len 28672
3311 174/300 Write block 1957 len 12288 data: 4 2 2
3312 175/300 Read block 657 len 16384
3313 176/300 Read block 3369 len 4096
3314 177/300 Write block 1876 len 16384 data: 2 2 2 2
3315 178/300 Read block 2070 len 32768
3316 179/300 Write block 1408 len 28672 data: 2 2 2 2 2 3 3
3317 180/300 Write block 905 len 28672 data: 3 3 3 3 3 3 3
3318 181/300 Write block 1567 len 28672 data: 3 3 3 3 2 2 2
3319 182/300 Write block 697 len 20480 data: 3 3 3 2 2
3320 183/300 Flush
3321 184/300 Read block 4037 len 28672
3322 185/300 Write block 4490 len 8192 data: 2 2
3323 186/300 Write block 1176 len 12288 data: 3 3 3
3324 187/300 Write block 4866 len 12288 data: 2 2 2
3325 188/300 Read block 3908 len 16384
3326 189/300 Read block 897 len 16384
3327 190/300 Read block 4033 len 40960
3328 191/300 Read block 1720 len 12288
3329 192/300 Write block 1030 len 32768 data: 2 2 2 2 2 2 2 2
3330 193/300 Read block 44 len 40960
3331 194/300 Read block 4243 len 16384
3332 195/300 Flush
3333 196/300 Read block 3863 len 32768
3334 197/300 Write block 408 len 20480 data: 4 5 5 4 3
3335 198/300 Write block 2679 len 28672 data: 3 3 3 3 3 3 2
3336 199/300 Write block 4543 len 8192 data: 4 3
3337 200/300 Write block 883 len 40960 data: 3 3 3 3 3 3 3 2 2 2
3338 201/300 Write block 322 len 32768 data: 3 3 4 4 4 3 3 2
3339 202/300 Read block 31 len 32768
3340 203/300 Read block 2299 len 12288
3341 204/300 Read block 1870 len 32768
3342 205/300 Read block 4195 len 36864
3343 206/300 Write block 4589 len 28672 data: 2 2 2 3 3 3 3
3344 207/300 Read block 1197 len 24576
3345 208/300 Write block 2763 len 4096 data: 4
3346 209/300 Write block 4069 len 36864 data: 2 2 2 2 2 2 2 2 3
3347 210/300 Read block 4676 len 24576
3348 211/300 Write block 892 len 32768 data: 3 2 2 2 2 2 2 2
3349 212/300 Read block 796 len 16384
3350 213/300 Write block 796 len 20480 data: 2 2 2 3 3
3351 214/300 Write block 1141 len 16384 data: 2 2 2 2
3352 215/300 Read block 3338 len 24576
3353 216/300 Read block 4802 len 20480
3354 217/300 Read block 1630 len 20480
3355 218/300 Write block 4875 len 12288 data: 3 3 3
3356 219/300 Flush
3357 220/300 Flush
3358 221/300 Read block 2930 len 40960
3359 222/300 Write block 2415 len 24576 data: 2 3 3 3 3 3
3360 223/300 Read block 4914 len 28672
3361 224/300 Read block 3824 len 12288
3362 225/300 Write block 218 len 24576 data: 2 2 2 2 2 2
3363 226/300 Write block 1290 len 32768 data: 2 2 2 2 2 2 3 3
3364 227/300 Write block 315 len 32768 data: 2 2 2 2 3 3 3 4
3365 228/300 Read block 3583 len 16384
3366 229/300 Write block 2577 len 24576 data: 2 2 2 2 2 3
3367 230/300 Read block 1941 len 28672
3368 231/300 Write block 1776 len 32768 data: 2 2 2 2 2 2 2 2
3369 232/300 Read block 2569 len 16384
3370 233/300 Write block 2703 len 12288 data: 4 3 3
3371 234/300 Read block 2930 len 8192
3372 235/300 Read block 2131 len 36864
3373 236/300 Write block 1333 len 36864 data: 2 2 2 2 2 2 2 2 2
3374 237/300 Write block 4844 len 24576 data: 3 3 4 3 3 3
3375 238/300 Write block 2192 len 24576 data: 4 4 4 3 3 3
3376 239/300 Read block 2985 len 20480
3377 240/300 Read block 2232 len 32768
3378 241/300 Write block 1818 len 40960 data: 2 2 2 2 2 2 2 2 3 3
3379 242/300 Write block 3320 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3380 243/300 Read block 3038 len 36864
3381 244/300 Write block 179 len 40960 data: 2 2 2 2 2 2 2 2 3 3
3382 245/300 Write block 153 len 4096 data: 2
3383 246/300 Read block 954 len 32768
3384 247/300 Write block 3073 len 36864 data: 2 2 2 2 2 2 2 2 2
3385 248/300 Read block 4776 len 12288
3386 249/300 Read block 915 len 4096
3387 250/300 Read block 3622 len 40960
3388 251/300 Write block 1618 len 20480 data: 2 2 2 2 2
3389 252/300 Read block 3698 len 20480
3390 253/300 Flush
3391 254/300 Write block 2920 len 40960 data: 2 3 3 2 2 3 3 4 3 3
3392 255/300 Read block 4982 len 28672
3393 256/300 Write block 1394 len 32768 data: 2 2 2 2 2 2 2 2
3394 257/300 Read block 4409 len 24576
3395 258/300 Write block 1895 len 28672 data: 3 3 3 3 2 3 3
3396 259/300 Read block 250 len 40960
3397 260/300 Write block 3352 len 28672 data: 2 2 3 3 4 4 4
3398 261/300 Write block 1646 len 24576 data: 4 4 4 3 3 3
3399 262/300 Read block 4691 len 36864
3400 263/300 Write block 2624 len 12288 data: 3 3 3
3401 264/300 Write block 2184 len 8192 data: 2 3
3402 265/300 Read block 4659 len 12288
3403 266/300 Read block 2081 len 36864
3404 267/300 Read block 1058 len 36864
3405 268/300 Read block 3031 len 32768
3406 269/300 Write block 1384 len 16384 data: 3 3 3 3
3407 270/300 Read block 4155 len 12288
3408 271/300 Read block 380 len 16384
3409 272/300 Read block 1668 len 16384
3410 273/300 Read block 4155 len 32768
3411 274/300 Read block 804 len 16384
3412 275/300 Write block 1902 len 12288 data: 3 3 2
3413 276/300 Read block 4798 len 28672
3414 277/300 Flush
3415 278/300 Write block 4986 len 8192 data: 2 2
3416 279/300 Write block 2390 len 32768 data: 2 2 2 2 2 2 2 2
3417 280/300 Read block 3769 len 20480
3418 281/300 Write block 2582 len 4096 data: 4
3419 282/300 Read block 2697 len 28672
3420 283/300 Write block 313 len 32768 data: 3 3 3 3 3 3 4 4
3421 284/300 Flush
3422 285/300 Read block 1787 len 20480
3423 286/300 Write block 581 len 40960 data: 2 2 2 3 4 4 4 3 3 2
3424 287/300 Read block 171 len 28672
3425 288/300 Read block 3314 len 24576
3426 289/300 Read block 4713 len 12288
3427 290/300 Read block 89 len 40960
3428 291/300 Write block 459 len 36864 data: 2 2 3 3 3 3 3 3 3
3429 292/300 Read block 330 len 36864
3430 293/300 Read block 4563 len 24576
3431 294/300 Write block 4218 len 20480 data: 2 2 2 2 2
3432 295/300 Read block 2320 len 20480
3433 296/300 Read block 3888 len 28672
3434 297/300 Write block 3207 len 40960 data: 3 3 3 2 2 2 2 2 2 2
3435 298/300 Read block 3665 len 32768
3436 299/300 Read block 2853 len 20480
3437 300/300 Flush
3438 301/300 Read block 2675 len 28672
3439 Main task has work to do, go find it
3440 got dsc Start(0)
3441 start 0
3442 Jan 14 16:11:20.894 INFO request completed, latency_us: 198, response_code: 204, uri: /start/cid/0, method: POST, req_id: b6edf558-1623-4d69-9385-0b5c491ea5e0, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
3443 Replay: started 0, returned:Ok(())
3444 ----------------------------------------------------------------
3445 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
3446 States: Offline Active Active
3447 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3448 1905 Acked Read 1 Sent Done Done false
3449 1906 Acked Read 6 Sent Done Done false
3450 1907 Acked Read 6 Sent Done Done false
3451 1908 Acked Read 6 Sent Done Done false
3452 1909 Acked Read 5 Sent Done Done false
3453 1910 Acked Write 8 Sent Done Done false
3454 1911 Acked Write 8 Sent Done Done false
3455 1912 Acked Write 6 Sent Done Done false
3456 1913 Acked Flush 0 Sent Done Done false
3457 1914 Acked Read 3 Sent Done Done false
3458 1915 Acked Read 1 Sent Done Done false
3459 1916 Acked Read 7 Sent Done Done false
3460 1917 Acked Write 1 Sent Done Done false
3461 1918 Acked Write 4 Sent Done Done false
3462 1919 Acked Write 7 Sent Done Done false
3463 1920 Acked Write 4 Sent Done Done false
3464 1921 Acked Read 7 Sent Done Done false
3465 1922 Acked Write 10 Sent Done Done false
3466 1923 Acked Flush 0 Sent Done Done false
3467 1924 Acked Write 1 Sent Done Done false
3468 1925 Acked Read 9 Sent Done Done false
3469 1926 Acked Read 1 Sent Done Done false
3470 1927 Acked Flush 0 Sent Done Done false
3471 1928 Acked Write 9 Sent Done Done false
3472 1929 Acked Write 3 Sent Done Done false
3473 1930 Acked Flush 0 Sent Done Done false
3474 1931 Acked Read 7 Sent Done Done false
3475 1932 Acked Write 8 Sent Done Done false
3476 1933 Acked Write 10 Sent Done Done false
3477 1934 Acked Write 8 Sent Done Done false
3478 1935 Acked Read 5 Sent Done Done false
3479 1936 Acked Read 9 Sent Done Done false
3480 1937 Acked Read 9 Sent Done Done false
3481 1938 Acked Read 2 Sent Done Done false
3482 1939 Acked Flush 0 Sent Done Done false
3483 1940 Acked Write 9 Sent Done Done false
3484 1941 Acked Read 9 Sent Done Done false
3485 1942 Acked Read 8 Sent Done Done false
3486 1943 Acked Read 7 Sent Done Done false
3487 1944 Acked Read 5 Sent Done Done false
3488 1945 Acked Write 5 Sent Done Done false
3489 1946 Acked Read 9 Sent Done Done false
3490 1947 Acked Write 1 Sent Done Done false
3491 1948 Acked Read 2 Sent Done Done false
3492 1949 Acked Flush 0 Sent Done Done false
3493 1950 Acked Read 2 Sent Done Done false
3494 1951 Acked Flush 0 Sent Done Done false
3495 1952 Acked Write 9 Sent Done Done false
3496 1953 Acked Read 7 Sent Done Done false
3497 1954 Acked Write 7 Sent Done Done false
3498 1955 Acked Write 10 Sent Done Done false
3499 1956 Acked Read 10 Sent Done Done false
3500 1957 Acked Write 8 Sent Done Done false
3501 1958 Acked Write 4 Sent Done Done false
3502 1959 Acked Read 5 Sent Done Done false
3503 1960 Acked Read 6 Sent Done Done false
3504 1961 Acked Read 6 Sent Done Done false
3505 1962 Acked Read 6 Sent Done Done false
3506 1963 Acked Read 5 Sent Done Done false
3507 1964 Acked Read 8 Sent Done Done false
3508 1965 Acked Write 7 Sent Done Done false
3509 1966 Acked Read 7 Sent Done Done false
3510 1967 Acked Read 8 Sent Done Done false
3511 1968 Acked Write 8 Sent Done Done false
3512 1969 Acked Read 1 Sent Done Done false
3513 1970 Acked Write 9 Sent Done Done false
3514 1971 Acked Read 6 Sent Done Done false
3515 1972 Acked Read 7 Sent Done Done false
3516 1973 Acked Write 10 Sent Done Done false
3517 1974 Acked Read 2 Sent Done Done false
3518 1975 Acked Flush 0 Sent Done Done false
3519 1976 Acked Read 3 Sent Done Done false
3520 1977 Acked Read 10 Sent Done Done false
3521 1978 Acked Write 5 Sent Done Done false
3522 1979 Acked Read 1 Sent Done Done false
3523 1980 Acked Flush 0 Sent Done Done false
3524 1981 Acked Read 6 Sent Done Done false
3525 1982 Acked Read 8 Sent Done Done false
3526 1983 Acked Write 5 Sent Done Done false
3527 1984 Acked Write 8 Sent Done Done false
3528 1985 Acked Read 2 Sent Done Done false
3529 1986 Acked Read 4 Sent Done Done false
3530 1987 Acked Read 2 Sent Done Done false
3531 1988 Acked Write 5 Sent Done Done false
3532 1989 Acked Read 1 Sent Done Done false
3533 1990 Acked Read 9 Sent Done Done false
3534 1991 Acked Read 1 Sent Done Done false
3535 1992 Acked Read 2 Sent Done Done false
3536 1993 Acked Write 8 Sent Done Done false
3537 1994 Acked Flush 0 Sent Done Done false
3538 1995 Acked Read 10 Sent Done Done false
3539 1996 Acked Write 4 Sent Done Done false
3540 1997 Acked Flush 0 Sent Done Done false
3541 1998 Acked Write 5 Sent Done Done false
3542 1999 Acked Read 5 Sent Done Done false
3543 2000 Acked Write 10 Sent Done Done false
3544 2001 Acked Read 10 Sent Done Done false
3545 2002 Acked Read 2 Sent Done Done false
3546 2003 Acked Flush 0 Sent Done Done false
3547 2004 Acked Read 9 Sent Done Done false
3548 2005 Acked Write 4 Sent Done Done false
3549 2006 Acked Read 10 Sent Done Done false
3550 2007 Acked Write 2 Sent Done Done false
3551 2008 Acked Read 3 Sent Done Done false
3552 2009 Acked Flush 0 Sent Done Done false
3553 2010 Acked Read 10 Sent Done Done false
3554 2011 Acked Read 7 Sent Done Done false
3555 2012 Acked Read 8 Sent Done Done false
3556 2013 Acked Read 1 Sent Done Done false
3557 2014 Acked Read 8 Sent Done Done false
3558 2015 Acked Write 3 Sent Done Done false
3559 2016 Acked Read 3 Sent Done Done false
3560 2017 Acked Write 3 Sent Done Done false
3561 2018 Acked Flush 0 Sent Done Done false
3562 2019 Acked Read 7 Sent Done Done false
3563 2020 Acked Write 8 Sent Done Done false
3564 2021 Acked Write 10 Sent Done Done false
3565 2022 Acked Read 6 Sent Done Done false
3566 2023 Acked Read 10 Sent Done Done false
3567 2024 Acked Read 10 Sent Done Done false
3568 2025 Acked Write 4 Sent Done Done false
3569 2026 Acked Read 3 Sent Done Done false
3570 2027 Acked Write 10 Sent Done Done false
3571 2028 Acked Write 5 Sent Done Done false
3572 2029 Acked Flush 0 Sent Done Done false
3573 2030 Acked Write 7 Sent Done Done false
3574 2031 Acked Write 10 Sent Done Done false
3575 2032 Acked Flush 0 Sent Done Done false
3576 2033 Acked Write 4 Sent Done Done false
3577 2034 Acked Write 9 Sent Done Done false
3578 2035 Acked Write 2 Sent Done Done false
3579 2036 Acked Read 1 Sent Done Done false
3580 2037 Acked Read 1 Sent Done Done false
3581 2038 Acked Write 7 Sent Done Done false
3582 2039 Acked Read 9 Sent Done Done false
3583 2040 Acked Write 9 Sent Done Done false
3584 2041 Acked Read 9 Sent Done Done false
3585 2042 Acked Write 8 Sent Done Done false
3586 2043 Acked Read 10 Sent Done Done false
3587 2044 Acked Write 2 Sent Done Done false
3588 2045 Acked Write 5 Sent Done Done false
3589 2046 Acked Read 1 Sent Done Done false
3590 2047 Acked Flush 0 Sent Done Done false
3591 2048 Acked Write 8 Sent Done Done false
3592 2049 Acked Write 4 Sent Done Done false
3593 2050 Acked Read 2 Sent Done Done false
3594 2051 Acked Write 1 Sent Done Done false
3595 2052 Acked Read 5 Sent Done Done false
3596 2053 Acked Read 8 Sent Done Done false
3597 2054 Acked Read 5 Sent Done Done false
3598 2055 Acked Read 6 Sent Done Done false
3599 2056 Acked Read 5 Sent Done Done false
3600 2057 Acked Read 1 Sent Done Done false
3601 2058 Acked Flush 0 Sent Done Done false
3602 2059 Acked Write 2 Sent Done Done false
3603 2060 Acked Write 9 Sent Done Done false
3604 2061 Acked Write 4 Sent Done Done false
3605 2062 Acked Read 1 Sent Done Done false
3606 2063 Acked Write 5 Sent Done Done false
3607 2064 Acked Read 8 Sent Done Done false
3608 2065 Acked Write 7 Sent Done Done false
3609 2066 Acked Read 10 Sent Done Done false
3610 2067 Acked Read 7 Sent Done Done false
3611 2068 Acked Read 1 Sent Done Done false
3612 2069 Acked Write 9 Sent Done Done false
3613 2070 Acked Read 1 Sent Done Done false
3614 2071 Acked Read 7 Sent Done Done false
3615 2072 Acked Read 1 Sent Done Done false
3616 2073 Acked Write 9 Sent Done Done false
3617 2074 Acked Write 6 Sent Done Done false
3618 2075 Acked Write 4 Sent Done Done false
3619 2076 Acked Read 6 Sent Done Done false
3620 2077 Acked Read 7 Sent Done Done false
3621 2078 Acked Write 3 Sent Done Done false
3622 2079 Acked Read 4 Sent Done Done false
3623 2080 Acked Read 1 Sent Done Done false
3624 2081 Acked Write 4 Sent Done Done false
3625 2082 Acked Read 8 Sent Done Done false
3626 2083 Acked Write 7 Sent Done Done false
3627 2084 Acked Write 7 Sent Done Done false
3628 2085 Acked Write 7 Sent Done Done false
3629 2086 Acked Write 5 Sent Done Done false
3630 2087 Acked Flush 0 Sent Done Done false
3631 2088 Acked Read 7 Sent Done Done false
3632 2089 Acked Write 2 Sent Done Done false
3633 2090 Acked Write 3 Sent Done Done false
3634 2091 Acked Write 3 Sent Done Done false
3635 2092 Acked Read 4 Sent Done Done false
3636 2093 Acked Read 4 Sent Done Done false
3637 2094 Acked Read 10 Sent Done Done false
3638 2095 Acked Read 3 Sent Done Done false
3639 2096 Acked Write 8 Sent Done Done false
3640 2097 Acked Read 10 Sent Done Done false
3641 2098 Acked Read 4 Sent Done Done false
3642 2099 Acked Flush 0 Sent Done Done false
3643 2100 Acked Read 8 Sent Done Done false
3644 2101 Acked Write 5 Sent Done Done false
3645 2102 Acked Write 7 Sent Done Done false
3646 2103 Acked Write 2 Sent Done Done false
3647 2104 Acked Write 10 Sent Done Done false
3648 2105 Acked Write 8 Sent Done Done false
3649 2106 Acked Read 8 Sent Done Done false
3650 2107 Acked Read 3 Sent Done Done false
3651 2108 Acked Read 8 Sent Done Done false
3652 2109 Acked Read 9 Sent Done Done false
3653 2110 Acked Write 7 Sent Done Done false
3654 2111 Acked Read 6 Sent Done Done false
3655 2112 Acked Write 1 Sent Done Done false
3656 2113 Acked Write 9 Sent Done Done false
3657 2114 Acked Read 6 Sent Done Done false
3658 2115 Acked Write 8 Sent Done Done false
3659 2116 Acked Read 4 Sent Done Done false
3660 2117 Acked Write 5 Sent Done Done false
3661 2118 Acked Write 4 Sent Done Done false
3662 2119 Acked Read 6 Sent Done Done false
3663 2120 Acked Read 5 Sent Done Done false
3664 2121 Acked Read 5 Sent Done Done false
3665 2122 Acked Write 3 Sent Done Done false
3666 2123 Acked Flush 0 Sent Done Done false
3667 2124 Acked Flush 0 Sent Done Done false
3668 2125 Acked Read 10 Sent Done Done false
3669 2126 Acked Write 6 Sent Done Done false
3670 2127 Acked Read 7 Sent Done Done false
3671 2128 Acked Read 3 Sent Done Done false
3672 2129 Acked Write 6 Sent Done Done false
3673 2130 Acked Write 8 Sent Done Done false
3674 2131 Acked Write 8 Sent Done Done false
3675 2132 Acked Read 4 Sent Done Done false
3676 2133 Acked Write 6 Sent Done Done false
3677 2134 Acked Read 7 Sent Done Done false
3678 2135 Acked Write 8 Sent Done Done false
3679 2136 Acked Read 4 Sent Done Done false
3680 2137 Acked Write 3 Sent Done Done false
3681 2138 Acked Read 2 Sent Done Done false
3682 2139 Acked Read 9 Sent Done Done false
3683 2140 Acked Write 9 Sent Done Done false
3684 2141 Acked Write 6 Sent Done Done false
3685 2142 Acked Write 6 Sent Done Done false
3686 2143 Acked Read 5 Sent Done Done false
3687 2144 Acked Read 8 Sent Done Done false
3688 2145 Acked Write 10 Sent Done Done false
3689 2146 Acked Write 10 Sent Done Done false
3690 2147 Acked Read 9 Sent Done Done false
3691 2148 Acked Write 10 Sent Done Done false
3692 2149 Acked Write 1 Sent Done Done false
3693 2150 Acked Read 8 Sent Done Done false
3694 2151 Acked Write 9 Sent Done Done false
3695 2152 Acked Read 3 Sent Done Done false
3696 2153 Acked Read 1 Sent Done Done false
3697 2154 Acked Read 10 Sent Done Done false
3698 2155 Acked Write 5 Sent Done Done false
3699 2156 Acked Read 5 Sent Done Done false
3700 2157 Acked Flush 0 Sent Done Done false
3701 2158 Acked Write 10 Sent Done Done false
3702 2159 Acked Read 7 Sent Done Done false
3703 2160 Acked Write 8 Sent Done Done false
3704 2161 Acked Read 6 Sent Done Done false
3705 2162 Acked Write 7 Sent Done Done false
3706 2163 Acked Read 10 Sent Done Done false
3707 2164 Acked Write 7 Sent Done Done false
3708 2165 Acked Write 6 Sent Done Done false
3709 2166 Acked Read 9 Sent Done Done false
3710 2167 Acked Write 3 Sent Done Done false
3711 2168 Acked Write 2 Sent Done Done false
3712 2169 Acked Read 3 Sent Done Done false
3713 2170 Acked Read 9 Sent Done Done false
3714 2171 Acked Read 9 Sent Done Done false
3715 2172 Acked Read 8 Sent Done Done false
3716 2173 Acked Write 4 Sent Done Done false
3717 2174 Acked Read 3 Sent Done Done false
3718 2175 Acked Read 4 Sent Done Done false
3719 2176 Acked Read 4 Sent Done Done false
3720 2177 Acked Read 8 Sent Done Done false
3721 2178 Acked Read 4 Sent Done Done false
3722 2179 Acked Write 3 Sent Done Done false
3723 2180 Acked Read 7 Sent Done Done false
3724 2181 Acked Flush 0 Sent Done Done false
3725 2182 Acked Write 2 Sent Done Done false
3726 2183 Acked Write 8 Sent Done Done false
3727 2184 Acked Read 5 Sent Done Done false
3728 2185 Acked Write 1 Sent Done Done false
3729 2186 Acked Read 7 Sent Done Done false
3730 2187 Acked Write 8 Sent Done Done false
3731 2188 Acked Flush 0 Sent Done Done false
3732 2189 Acked Read 5 Sent Done Done false
3733 2190 Acked Write 10 Sent Done Done false
3734 2191 Acked Read 7 Sent Done Done false
3735 2192 Acked Read 6 Sent Done Done false
3736 2193 Acked Read 3 Sent Done Done false
3737 2194 Acked Read 10 Sent Done Done false
3738 2195 Acked Write 9 Sent Done Done false
3739 2196 Acked Read 9 Sent Done Done false
3740 2197 Acked Read 6 Sent Done Done false
3741 2198 Acked Write 5 Sent Done Done false
3742 2199 Acked Read 5 Sent Done Done false
3743 2200 Acked Read 7 Sent Done Done false
3744 2201 Acked Write 10 Sent Done Done false
3745 2202 Acked Read 8 Sent Done Done false
3746 2203 Acked Read 5 Sent Done Done false
3747 2204 Acked Flush 0 Sent Done Done false
3748 2205 Acked Read 7 Sent Done Done false
3749 STATES DS:0 DS:1 DS:2 TOTAL
3750 Sent 301 0 0 301
3751 Done 0 301 301 602
3752 Skipped 0 0 0 0
3753 Error 0 0 0 0
3754 Last Flush: 1904 2204 2204
3755 Downstairs last five completed: 1904 1903 1902 1901 1900
3756 Upstairs last five completed: 2205 2204 2203 2202 2201
3757 CLIENT: Up:0 ds:301 act:2
3758 [8810] got start action
3759 [8810] I am going to restart
3760 Starting downstairs at port 8810
3761 Make output file at "/tmp/dsc/downstairs-8810.txt"
3762 [8810][0] reports Starting
3763 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1206)
3764 [8810][0] reports Running
3765 ----------------------------------------------------------------
3766 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
3767 States: Offline Active Active
3768 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3769 1905 Acked Read 1 Sent Done Done false
3770 1906 Acked Read 6 Sent Done Done false
3771 1907 Acked Read 6 Sent Done Done false
3772 1908 Acked Read 6 Sent Done Done false
3773 1909 Acked Read 5 Sent Done Done false
3774 1910 Acked Write 8 Sent Done Done false
3775 1911 Acked Write 8 Sent Done Done false
3776 1912 Acked Write 6 Sent Done Done false
3777 1913 Acked Flush 0 Sent Done Done false
3778 1914 Acked Read 3 Sent Done Done false
3779 1915 Acked Read 1 Sent Done Done false
3780 1916 Acked Read 7 Sent Done Done false
3781 1917 Acked Write 1 Sent Done Done false
3782 1918 Acked Write 4 Sent Done Done false
3783 1919 Acked Write 7 Sent Done Done false
3784 1920 Acked Write 4 Sent Done Done false
3785 1921 Acked Read 7 Sent Done Done false
3786 1922 Acked Write 10 Sent Done Done false
3787 1923 Acked Flush 0 Sent Done Done false
3788 1924 Acked Write 1 Sent Done Done false
3789 1925 Acked Read 9 Sent Done Done false
3790 1926 Acked Read 1 Sent Done Done false
3791 1927 Acked Flush 0 Sent Done Done false
3792 1928 Acked Write 9 Sent Done Done false
3793 1929 Acked Write 3 Sent Done Done false
3794 1930 Acked Flush 0 Sent Done Done false
3795 1931 Acked Read 7 Sent Done Done false
3796 1932 Acked Write 8 Sent Done Done false
3797 1933 Acked Write 10 Sent Done Done false
3798 1934 Acked Write 8 Sent Done Done false
3799 1935 Acked Read 5 Sent Done Done false
3800 1936 Acked Read 9 Sent Done Done false
3801 1937 Acked Read 9 Sent Done Done false
3802 1938 Acked Read 2 Sent Done Done false
3803 1939 Acked Flush 0 Sent Done Done false
3804 1940 Acked Write 9 Sent Done Done false
3805 1941 Acked Read 9 Sent Done Done false
3806 1942 Acked Read 8 Sent Done Done false
3807 1943 Acked Read 7 Sent Done Done false
3808 1944 Acked Read 5 Sent Done Done false
3809 1945 Acked Write 5 Sent Done Done false
3810 1946 Acked Read 9 Sent Done Done false
3811 1947 Acked Write 1 Sent Done Done false
3812 1948 Acked Read 2 Sent Done Done false
3813 1949 Acked Flush 0 Sent Done Done false
3814 1950 Acked Read 2 Sent Done Done false
3815 1951 Acked Flush 0 Sent Done Done false
3816 1952 Acked Write 9 Sent Done Done false
3817 1953 Acked Read 7 Sent Done Done false
3818 1954 Acked Write 7 Sent Done Done false
3819 1955 Acked Write 10 Sent Done Done false
3820 1956 Acked Read 10 Sent Done Done false
3821 1957 Acked Write 8 Sent Done Done false
3822 1958 Acked Write 4 Sent Done Done false
3823 1959 Acked Read 5 Sent Done Done false
3824 1960 Acked Read 6 Sent Done Done false
3825 1961 Acked Read 6 Sent Done Done false
3826 1962 Acked Read 6 Sent Done Done false
3827 1963 Acked Read 5 Sent Done Done false
3828 1964 Acked Read 8 Sent Done Done false
3829 1965 Acked Write 7 Sent Done Done false
3830 1966 Acked Read 7 Sent Done Done false
3831 1967 Acked Read 8 Sent Done Done false
3832 1968 Acked Write 8 Sent Done Done false
3833 1969 Acked Read 1 Sent Done Done false
3834 1970 Acked Write 9 Sent Done Done false
3835 1971 Acked Read 6 Sent Done Done false
3836 1972 Acked Read 7 Sent Done Done false
3837 1973 Acked Write 10 Sent Done Done false
3838 1974 Acked Read 2 Sent Done Done false
3839 1975 Acked Flush 0 Sent Done Done false
3840 1976 Acked Read 3 Sent Done Done false
3841 1977 Acked Read 10 Sent Done Done false
3842 1978 Acked Write 5 Sent Done Done false
3843 1979 Acked Read 1 Sent Done Done false
3844 1980 Acked Flush 0 Sent Done Done false
3845 1981 Acked Read 6 Sent Done Done false
3846 1982 Acked Read 8 Sent Done Done false
3847 1983 Acked Write 5 Sent Done Done false
3848 1984 Acked Write 8 Sent Done Done false
3849 1985 Acked Read 2 Sent Done Done false
3850 1986 Acked Read 4 Sent Done Done false
3851 1987 Acked Read 2 Sent Done Done false
3852 1988 Acked Write 5 Sent Done Done false
3853 1989 Acked Read 1 Sent Done Done false
3854 1990 Acked Read 9 Sent Done Done false
3855 1991 Acked Read 1 Sent Done Done false
3856 1992 Acked Read 2 Sent Done Done false
3857 1993 Acked Write 8 Sent Done Done false
3858 1994 Acked Flush 0 Sent Done Done false
3859 1995 Acked Read 10 Sent Done Done false
3860 1996 Acked Write 4 Sent Done Done false
3861 1997 Acked Flush 0 Sent Done Done false
3862 1998 Acked Write 5 Sent Done Done false
3863 1999 Acked Read 5 Sent Done Done false
3864 2000 Acked Write 10 Sent Done Done false
3865 2001 Acked Read 10 Sent Done Done false
3866 2002 Acked Read 2 Sent Done Done false
3867 2003 Acked Flush 0 Sent Done Done false
3868 2004 Acked Read 9 Sent Done Done false
3869 2005 Acked Write 4 Sent Done Done false
3870 2006 Acked Read 10 Sent Done Done false
3871 2007 Acked Write 2 Sent Done Done false
3872 2008 Acked Read 3 Sent Done Done false
3873 2009 Acked Flush 0 Sent Done Done false
3874 2010 Acked Read 10 Sent Done Done false
3875 2011 Acked Read 7 Sent Done Done false
3876 2012 Acked Read 8 Sent Done Done false
3877 2013 Acked Read 1 Sent Done Done false
3878 2014 Acked Read 8 Sent Done Done false
3879 2015 Acked Write 3 Sent Done Done false
3880 2016 Acked Read 3 Sent Done Done false
3881 2017 Acked Write 3 Sent Done Done false
3882 2018 Acked Flush 0 Sent Done Done false
3883 2019 Acked Read 7 Sent Done Done false
3884 2020 Acked Write 8 Sent Done Done false
3885 2021 Acked Write 10 Sent Done Done false
3886 2022 Acked Read 6 Sent Done Done false
3887 2023 Acked Read 10 Sent Done Done false
3888 2024 Acked Read 10 Sent Done Done false
3889 2025 Acked Write 4 Sent Done Done false
3890 2026 Acked Read 3 Sent Done Done false
3891 2027 Acked Write 10 Sent Done Done false
3892 2028 Acked Write 5 Sent Done Done false
3893 2029 Acked Flush 0 Sent Done Done false
3894 2030 Acked Write 7 Sent Done Done false
3895 2031 Acked Write 10 Sent Done Done false
3896 2032 Acked Flush 0 Sent Done Done false
3897 2033 Acked Write 4 Sent Done Done false
3898 2034 Acked Write 9 Sent Done Done false
3899 2035 Acked Write 2 Sent Done Done false
3900 2036 Acked Read 1 Sent Done Done false
3901 2037 Acked Read 1 Sent Done Done false
3902 2038 Acked Write 7 Sent Done Done false
3903 2039 Acked Read 9 Sent Done Done false
3904 2040 Acked Write 9 Sent Done Done false
3905 2041 Acked Read 9 Sent Done Done false
3906 2042 Acked Write 8 Sent Done Done false
3907 2043 Acked Read 10 Sent Done Done false
3908 2044 Acked Write 2 Sent Done Done false
3909 2045 Acked Write 5 Sent Done Done false
3910 2046 Acked Read 1 Sent Done Done false
3911 2047 Acked Flush 0 Sent Done Done false
3912 2048 Acked Write 8 Sent Done Done false
3913 2049 Acked Write 4 Sent Done Done false
3914 2050 Acked Read 2 Sent Done Done false
3915 2051 Acked Write 1 Sent Done Done false
3916 2052 Acked Read 5 Sent Done Done false
3917 2053 Acked Read 8 Sent Done Done false
3918 2054 Acked Read 5 Sent Done Done false
3919 2055 Acked Read 6 Sent Done Done false
3920 2056 Acked Read 5 Sent Done Done false
3921 2057 Acked Read 1 Sent Done Done false
3922 2058 Acked Flush 0 Sent Done Done false
3923 2059 Acked Write 2 Sent Done Done false
3924 2060 Acked Write 9 Sent Done Done false
3925 2061 Acked Write 4 Sent Done Done false
3926 2062 Acked Read 1 Sent Done Done false
3927 2063 Acked Write 5 Sent Done Done false
3928 2064 Acked Read 8 Sent Done Done false
3929 2065 Acked Write 7 Sent Done Done false
3930 2066 Acked Read 10 Sent Done Done false
3931 2067 Acked Read 7 Sent Done Done false
3932 2068 Acked Read 1 Sent Done Done false
3933 2069 Acked Write 9 Sent Done Done false
3934 2070 Acked Read 1 Sent Done Done false
3935 2071 Acked Read 7 Sent Done Done false
3936 2072 Acked Read 1 Sent Done Done false
3937 2073 Acked Write 9 Sent Done Done false
3938 2074 Acked Write 6 Sent Done Done false
3939 2075 Acked Write 4 Sent Done Done false
3940 2076 Acked Read 6 Sent Done Done false
3941 2077 Acked Read 7 Sent Done Done false
3942 2078 Acked Write 3 Sent Done Done false
3943 2079 Acked Read 4 Sent Done Done false
3944 2080 Acked Read 1 Sent Done Done false
3945 2081 Acked Write 4 Sent Done Done false
3946 2082 Acked Read 8 Sent Done Done false
3947 2083 Acked Write 7 Sent Done Done false
3948 2084 Acked Write 7 Sent Done Done false
3949 2085 Acked Write 7 Sent Done Done false
3950 2086 Acked Write 5 Sent Done Done false
3951 2087 Acked Flush 0 Sent Done Done false
3952 2088 Acked Read 7 Sent Done Done false
3953 2089 Acked Write 2 Sent Done Done false
3954 2090 Acked Write 3 Sent Done Done false
3955 2091 Acked Write 3 Sent Done Done false
3956 2092 Acked Read 4 Sent Done Done false
3957 2093 Acked Read 4 Sent Done Done false
3958 2094 Acked Read 10 Sent Done Done false
3959 2095 Acked Read 3 Sent Done Done false
3960 2096 Acked Write 8 Sent Done Done false
3961 2097 Acked Read 10 Sent Done Done false
3962 2098 Acked Read 4 Sent Done Done false
3963 2099 Acked Flush 0 Sent Done Done false
3964 2100 Acked Read 8 Sent Done Done false
3965 2101 Acked Write 5 Sent Done Done false
3966 2102 Acked Write 7 Sent Done Done false
3967 2103 Acked Write 2 Sent Done Done false
3968 2104 Acked Write 10 Sent Done Done false
3969 2105 Acked Write 8 Sent Done Done false
3970 2106 Acked Read 8 Sent Done Done false
3971 2107 Acked Read 3 Sent Done Done false
3972 2108 Acked Read 8 Sent Done Done false
3973 2109 Acked Read 9 Sent Done Done false
3974 2110 Acked Write 7 Sent Done Done false
3975 2111 Acked Read 6 Sent Done Done false
3976 2112 Acked Write 1 Sent Done Done false
3977 2113 Acked Write 9 Sent Done Done false
3978 2114 Acked Read 6 Sent Done Done false
3979 2115 Acked Write 8 Sent Done Done false
3980 2116 Acked Read 4 Sent Done Done false
3981 2117 Acked Write 5 Sent Done Done false
3982 2118 Acked Write 4 Sent Done Done false
3983 2119 Acked Read 6 Sent Done Done false
3984 2120 Acked Read 5 Sent Done Done false
3985 2121 Acked Read 5 Sent Done Done false
3986 2122 Acked Write 3 Sent Done Done false
3987 2123 Acked Flush 0 Sent Done Done false
3988 2124 Acked Flush 0 Sent Done Done false
3989 2125 Acked Read 10 Sent Done Done false
3990 2126 Acked Write 6 Sent Done Done false
3991 2127 Acked Read 7 Sent Done Done false
3992 2128 Acked Read 3 Sent Done Done false
3993 2129 Acked Write 6 Sent Done Done false
3994 2130 Acked Write 8 Sent Done Done false
3995 2131 Acked Write 8 Sent Done Done false
3996 2132 Acked Read 4 Sent Done Done false
3997 2133 Acked Write 6 Sent Done Done false
3998 2134 Acked Read 7 Sent Done Done false
3999 2135 Acked Write 8 Sent Done Done false
4000 2136 Acked Read 4 Sent Done Done false
4001 2137 Acked Write 3 Sent Done Done false
4002 2138 Acked Read 2 Sent Done Done false
4003 2139 Acked Read 9 Sent Done Done false
4004 2140 Acked Write 9 Sent Done Done false
4005 2141 Acked Write 6 Sent Done Done false
4006 2142 Acked Write 6 Sent Done Done false
4007 2143 Acked Read 5 Sent Done Done false
4008 2144 Acked Read 8 Sent Done Done false
4009 2145 Acked Write 10 Sent Done Done false
4010 2146 Acked Write 10 Sent Done Done false
4011 2147 Acked Read 9 Sent Done Done false
4012 2148 Acked Write 10 Sent Done Done false
4013 2149 Acked Write 1 Sent Done Done false
4014 2150 Acked Read 8 Sent Done Done false
4015 2151 Acked Write 9 Sent Done Done false
4016 2152 Acked Read 3 Sent Done Done false
4017 2153 Acked Read 1 Sent Done Done false
4018 2154 Acked Read 10 Sent Done Done false
4019 2155 Acked Write 5 Sent Done Done false
4020 2156 Acked Read 5 Sent Done Done false
4021 2157 Acked Flush 0 Sent Done Done false
4022 2158 Acked Write 10 Sent Done Done false
4023 2159 Acked Read 7 Sent Done Done false
4024 2160 Acked Write 8 Sent Done Done false
4025 2161 Acked Read 6 Sent Done Done false
4026 2162 Acked Write 7 Sent Done Done false
4027 2163 Acked Read 10 Sent Done Done false
4028 2164 Acked Write 7 Sent Done Done false
4029 2165 Acked Write 6 Sent Done Done false
4030 2166 Acked Read 9 Sent Done Done false
4031 2167 Acked Write 3 Sent Done Done false
4032 2168 Acked Write 2 Sent Done Done false
4033 2169 Acked Read 3 Sent Done Done false
4034 2170 Acked Read 9 Sent Done Done false
4035 2171 Acked Read 9 Sent Done Done false
4036 2172 Acked Read 8 Sent Done Done false
4037 2173 Acked Write 4 Sent Done Done false
4038 2174 Acked Read 3 Sent Done Done false
4039 2175 Acked Read 4 Sent Done Done false
4040 2176 Acked Read 4 Sent Done Done false
4041 2177 Acked Read 8 Sent Done Done false
4042 2178 Acked Read 4 Sent Done Done false
4043 2179 Acked Write 3 Sent Done Done false
4044 2180 Acked Read 7 Sent Done Done false
4045 2181 Acked Flush 0 Sent Done Done false
4046 2182 Acked Write 2 Sent Done Done false
4047 2183 Acked Write 8 Sent Done Done false
4048 2184 Acked Read 5 Sent Done Done false
4049 2185 Acked Write 1 Sent Done Done false
4050 2186 Acked Read 7 Sent Done Done false
4051 2187 Acked Write 8 Sent Done Done false
4052 2188 Acked Flush 0 Sent Done Done false
4053 2189 Acked Read 5 Sent Done Done false
4054 2190 Acked Write 10 Sent Done Done false
4055 2191 Acked Read 7 Sent Done Done false
4056 2192 Acked Read 6 Sent Done Done false
4057 2193 Acked Read 3 Sent Done Done false
4058 2194 Acked Read 10 Sent Done Done false
4059 2195 Acked Write 9 Sent Done Done false
4060 2196 Acked Read 9 Sent Done Done false
4061 2197 Acked Read 6 Sent Done Done false
4062 2198 Acked Write 5 Sent Done Done false
4063 2199 Acked Read 5 Sent Done Done false
4064 2200 Acked Read 7 Sent Done Done false
4065 2201 Acked Write 10 Sent Done Done false
4066 2202 Acked Read 8 Sent Done Done false
4067 2203 Acked Read 5 Sent Done Done false
4068 2204 Acked Flush 0 Sent Done Done false
4069 2205 Acked Read 7 Sent Done Done false
4070 2206 Acked Flush 0 Sent Done Done false
4071 STATES DS:0 DS:1 DS:2 TOTAL
4072 Sent 302 0 0 302
4073 Done 0 302 302 604
4074 Skipped 0 0 0 0
4075 Error 0 0 0 0
4076 Last Flush: 1904 2206 2206
4077 Downstairs last five completed: 1904 1903 1902 1901 1900
4078 Upstairs last five completed: 2206 2205 2204 2203 2202
4079 CLIENT: Up:0 ds:302 act:2
40802025-01-14T16:11:26.647ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
40812025-01-14T16:11:26.647ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:45927) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
40822025-01-14T16:11:26.648ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
40832025-01-14T16:11:26.649ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
40842025-01-14T16:11:26.649ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
40852025-01-14T16:11:26.649ZINFOcrucible: send last flush ID to this DS: 1904 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
40862025-01-14T16:11:26.649ZINFOcrucible: Replied this last flush ID: 1904 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
40872025-01-14T16:11:26.649ZINFOcrucible: [0] client re-new 302 jobs since flush 1904 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
40882025-01-14T16:11:26.649ZINFOcrucible: [0] Replayed 302 jobs since flush: 1904 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
4089 ----------------------------------------------------------------
4090 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
4091 Downstairs last five completed: 2206 2205 2204 2203 2202
4092 Upstairs last five completed: 2206 2205 2204 2203 2202
4093 CLIENT: Up:0 ds:0 act:3
4094 Replay: All jobs finished
4095 Main task has work to do, go find it
4096 got dsc Stop(1)
4097 stop 1
4098 Jan 14 16:11:28.900 INFO request completed, latency_us: 366, response_code: 204, uri: /stop/cid/1, method: POST, req_id: f30c422a-f29a-4816-a725-aeb9ca196309[8820] Got stop action so:false kr:false
4099 , remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
41002025-01-14T16:11:28.901ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
4101 Jan 14 16:11:28.901 INFO request completed, latency_us: 113, response_code: 200, uri: /state/cid/1{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-14T16:11:28.90147144Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
4102 , method: GET, req_id: a90e88b4-5f73-49c3-a925-4dd3fdd3f4e8, remote_addr: 127.0.0.1{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-14T16:11:28.901493587Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
4103 :61942, local_addr: 127.0.0.1:9998
41042025-01-14T16:11:28.901ZINFOcrucible: client task is exiting = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
4105 [8820] Exited with: ExitStatus(unix_wait_status(9))
4106 [8820][1] reports Exit
4107 Jan 14 16:11:32.904 INFO request completed, latency_us: 319, response_code: 200, uri: /state/cid/1, method: GET, req_id: b2e327a9-cba7-4d71-903d-f9087224e59c, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
4108 001/300 Read block 4490 len 36864
4109 002/300 Read block 501 len 24576
4110 003/300 Flush
4111 004/300 Read block 2438 len 32768
4112 005/300 Read block 611 len 4096
4113 006/300 Read block 4783 len 4096
4114 007/300 Write block 3165 len 20480 data: 2 2 2 2 2
4115 008/300 Write block 4305 len 12288 data: 2 2 2
4116 009/300 Read block 2047 len 40960
4117 010/300 Flush
4118 011/300 Read block 1599 len 20480
4119 012/300 Read block 4997 len 8192
4120 013/300 Read block 608 len 36864
4121 014/300 Read block 206 len 8192
4122 015/300 Write block 1508 len 12288 data: 2 2 2
4123 016/300 Read block 3172 len 32768
4124 017/300 Read block 2196 len 8192
4125 018/300 Read block 1085 len 12288
4126 019/300 Read block 4049 len 4096
4127 020/300 Write block 2887 len 36864 data: 3 3 3 3 2 2 2 3 3
4128 021/300 Read block 2355 len 4096
4129 022/300 Read block 4847 len 40960
4130 023/300 Read block 3572 len 36864
4131 024/300 Write block 342 len 12288 data: 3 3 3
4132 025/300 Read block 3752 len 28672
4133 026/300 Write block 1116 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4134 027/300 Read block 4531 len 36864
4135 028/300 Write block 701 len 20480 data: 3 2 2 3 3
4136 029/300 Write block 2039 len 16384 data: 3 3 3 2
4137 030/300 Read block 1426 len 32768
4138 031/300 Read block 1875 len 36864
4139 032/300 Read block 4041 len 8192
4140 033/300 Write block 1852 len 32768 data: 2 2 2 2 2 2 2 2
4141 034/300 Read block 3347 len 36864
4142 035/300 Read block 1188 len 40960
4143 036/300 Write block 1171 len 40960 data: 2 2 2 2 3 4 4 4 3 3
4144 037/300 Read block 458 len 40960
4145 038/300 Flush
4146 039/300 Read block 1697 len 4096
4147 040/300 Read block 3785 len 16384
4148 041/300 Write block 749 len 20480 data: 2 2 2 2 2
4149 042/300 Read block 717 len 36864
4150 043/300 Flush
4151 044/300 Read block 565 len 32768
4152 045/300 Read block 2226 len 20480
4153 046/300 Write block 3419 len 24576 data: 3 3 3 2 2 2
4154 047/300 Read block 2054 len 40960
4155 048/300 Read block 402 len 4096
4156 049/300 Write block 3653 len 12288 data: 2 2 2
4157 050/300 Read block 2132 len 24576
4158 051/300 Write block 4090 len 40960 data: 3 3 3 2 2 2 2 2 2 2
4159 052/300 Read block 942 len 40960
4160 053/300 Read block 4645 len 8192
4161 054/300 Write block 1752 len 4096 data: 3
4162 055/300 Read block 3904 len 20480
4163 056/300 Write block 1577 len 8192 data: 2 2
4164 057/300 Write block 1158 len 4096 data: 3
4165 058/300 Write block 4277 len 32768 data: 2 2 2 3 3 3 3 3
4166 059/300 Read block 4352 len 8192
4167 060/300 Write block 1353 len 20480 data: 3 2 3 3 3
4168 061/300 Write block 435 len 16384 data: 2 2 3 3
4169 062/300 Read block 1104 len 4096
4170 063/300 Read block 4293 len 12288
4171 064/300 Read block 1600 len 16384
4172 065/300 Read block 3115 len 36864
4173 066/300 Read block 279 len 40960
4174 067/300 Write block 3992 len 32768 data: 3 3 4 4 4 4 5 5
4175 068/300 Write block 2312 len 32768 data: 2 2 2 2 3 3 3 3
4176 069/300 Read block 3544 len 12288
4177 070/300 Read block 3397 len 20480
4178 071/300 Write block 3099 len 24576 data: 2 2 3 3 3 2
4179 072/300 Read block 2957 len 16384
4180 073/300 Read block 2114 len 28672
4181 074/300 Read block 505 len 4096
4182 075/300 Read block 187 len 32768
4183 076/300 Write block 3755 len 36864 data: 2 2 3 3 2 3 3 4 4
4184 077/300 Write block 240 len 16384 data: 2 2 2 2
4185 078/300 Write block 3034 len 8192 data: 2 2
4186 079/300 Read block 1637 len 40960
4187 080/300 Write block 4860 len 24576 data: 2 2 3 3 3 2
4188 081/300 Read block 332 len 12288
4189 082/300 Flush
4190 083/300 Write block 233 len 4096 data: 2
4191 084/300 Read block 3958 len 32768
4192 085/300 Write block 348 len 12288 data: 3 3 3
4193 086/300 Read block 2030 len 4096
4194 087/300 Flush
4195 088/300 Write block 4371 len 8192 data: 3 4
4196 089/300 Read block 3018 len 16384
4197 090/300 Read block 2794 len 4096
4198 091/300 Read block 4873 len 24576
4199 092/300 Write block 682 len 32768 data: 2 2 2 2 2 2 2 2
4200 093/300 Read block 4079 len 40960
4201 094/300 Write block 2801 len 12288 data: 2 2 2
4202 095/300 Write block 3141 len 32768 data: 2 2 2 2 2 2 2 2
4203 096/300 Flush
4204 097/300 Write block 1859 len 12288 data: 3 2 2
4205 098/300 Write block 14 len 20480 data: 2 2 2 2 2
4206 099/300 Read block 4414 len 40960
4207 100/300 Flush
4208 101/300 Read block 3028 len 20480
4209 102/300 Read block 2277 len 36864
4210 103/300 Read block 108 len 24576
4211 104/300 Write block 1333 len 12288 data: 3 3 3
4212 105/300 Write block 2808 len 12288 data: 2 2 4
4213 106/300 Write block 1439 len 4096 data: 2
4214 107/300 Read block 3321 len 16384
4215 108/300 Write block 4088 len 24576 data: 3 3 4 4 4 3
4216 109/300 Write block 288 len 24576 data: 3 3 3 3 3 3
4217 110/300 Read block 4217 len 8192
4218 111/300 Write block 3245 len 20480 data: 2 2 2 2 2
4219 112/300 Flush
4220 113/300 Write block 1959 len 16384 data: 3 3 3 3
4221 114/300 Write block 945 len 16384 data: 2 2 2 2
4222 115/300 Write block 448 len 40960 data: 2 3 2 2 2 2 2 2 2 2
4223 116/300 Write block 2194 len 8192 data: 5 4
4224 117/300 Write block 3160 len 8192 data: 2 2
4225 118/300 Read block 4144 len 12288
4226 119/300 Write block 4777 len 16384 data: 2 2 2 2
4227 120/300 Read block 3923 len 28672
4228 121/300 Read block 3396 len 12288
4229 122/300 Write block 2778 len 20480 data: 2 2 2 2 2
4230 123/300 Read block 2151 len 24576
4231 124/300 Write block 455 len 12288 data: 3 3 3
4232 125/300 Write block 1816 len 20480 data: 3 3 3 3 3
4233 126/300 Write block 3446 len 24576 data: 3 2 2 2 3 3
4234 127/300 Write block 1890 len 24576 data: 5 6 6 6 5 4
4235 128/300 Flush
4236 129/300 Write block 27 len 24576 data: 3 3 3 2 2 2
4237 130/300 Read block 3582 len 12288
4238 131/300 Write block 4931 len 32768 data: 2 2 2 2 2 2 2 2
4239 132/300 Read block 4814 len 8192
4240 133/300 Write block 2199 len 12288 data: 3 2 2
4241 134/300 Read block 634 len 16384
4242 135/300 Read block 2509 len 8192
4243 136/300 Write block 4853 len 28672 data: 2 2 2 2 2 2 2
4244 137/300 Read block 869 len 20480
4245 138/300 Write block 244 len 12288 data: 2 2 2
4246 139/300 Write block 2109 len 20480 data: 3 3 3 3 3
4247 140/300 Read block 2042 len 12288
4248 141/300 Write block 2381 len 20480 data: 2 2 2 2 2
4249 142/300 Flush
4250 143/300 Flush
4251 144/300 Write block 3679 len 40960 data: 2 2 2 3 3 3 2 2 2 2
4252 145/300 Read block 612 len 36864
4253 146/300 Read block 816 len 24576
4254 147/300 Write block 3631 len 40960 data: 2 2 4 4 4 4 3 3 3 2
4255 148/300 Write block 536 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4256 149/300 Write block 2259 len 20480 data: 2 2 2 2 2
4257 150/300 Write block 3148 len 20480 data: 3 2 2 2 2
4258 151/300 Flush
4259 152/300 Write block 4237 len 24576 data: 2 2 2 2 2 3
4260 153/300 Read block 631 len 4096
4261 154/300 Read block 2164 len 12288
4262 155/300 Read block 4061 len 8192
4263 156/300 Write block 2374 len 4096 data: 2
4264 157/300 Write block 1700 len 12288 data: 3 3 3
4265 158/300 Read block 2995 len 28672
4266 159/300 Write block 326 len 40960 data: 5 4 4 3 2 2 2 2 3 3
4267 160/300 Write block 1542 len 36864 data: 2 2 2 2 2 2 2 2 2
4268 161/300 Read block 2073 len 28672
4269 162/300 Write block 205 len 12288 data: 4 4 5
4270 163/300 Write block 1300 len 28672 data: 3 2 2 2 3 3 2
4271 164/300 Read block 3807 len 28672
4272 165/300 Read block 1773 len 4096
4273 166/300 Flush
4274 167/300 Write block 3205 len 28672 data: 3 3 4 4 4 3 3
4275 168/300 Write block 814 len 36864 data: 2 2 2 2 2 3 3 2 3
4276 169/300 Flush
4277 170/300 Write block 4324 len 20480 data: 3 3 3 3 3
4278 171/300 Flush
4279 172/300 Read block 1586 len 12288
4280 173/300 Write block 2475 len 24576 data: 3 3 3 3 3 3
4281 174/300 Read block 1649 len 20480
4282 175/300 Read block 2686 len 32768
4283 176/300 Read block 997 len 12288
4284 177/300 Read block 2554 len 36864
4285 178/300 Flush
4286 179/300 Flush
4287 180/300 Write block 487 len 20480 data: 2 2 2 2 2
4288 181/300 Write block 3142 len 36864 data: 3 3 3 3 3 3 4 3 3
4289 182/300 Read block 3889 len 12288
4290 183/300 Write block 1639 len 12288 data: 2 2 2
4291 184/300 Read block 3624 len 20480
4292 185/300 Write block 4898 len 16384 data: 2 2 2 2
4293 186/300 Read block 2699 len 36864
4294 187/300 Read block 4395 len 36864
4295 188/300 Read block 4223 len 16384
4296 189/300 Read block 356 len 16384
4297 190/300 Read block 1283 len 36864
4298 191/300 Read block 545 len 28672
4299 192/300 Read block 4148 len 12288
4300 193/300 Flush
4301 194/300 Read block 2590 len 12288
4302 195/300 Write block 4846 len 36864 data: 5 4 4 4 4 2 2 3 3
4303 196/300 Write block 1874 len 4096 data: 2
4304 197/300 Read block 2766 len 32768
4305 198/300 Read block 2635 len 4096
4306 199/300 Read block 4645 len 16384
4307 200/300 Write block 1999 len 4096 data: 2
4308 201/300 Flush
4309 202/300 Read block 4617 len 4096
4310 203/300 Read block 1483 len 4096
4311 204/300 Read block 2770 len 24576
4312 205/300 Write block 1506 len 8192 data: 2 2
4313 206/300 Read block 446 len 28672
4314 207/300 Read block 1469 len 16384
4315 208/300 Read block 247 len 4096
4316 209/300 Read block 2511 len 16384
4317 210/300 Write block 3819 len 32768 data: 3 3 2 3 3 3 3 3
4318 211/300 Read block 439 len 28672
4319 212/300 Read block 1935 len 20480
4320 213/300 Read block 3191 len 8192
4321 214/300 Read block 4201 len 24576
4322 215/300 Write block 369 len 32768 data: 2 2 2 2 2 2 2 3
4323 216/300 Read block 3617 len 40960
4324 217/300 Read block 2643 len 12288
4325 218/300 Read block 350 len 40960
4326 219/300 Read block 4696 len 32768
4327 220/300 Read block 1554 len 32768
4328 221/300 Flush
4329 222/300 Flush
4330 223/300 Read block 1141 len 36864
4331 224/300 Read block 2711 len 36864
4332 225/300 Read block 758 len 20480
4333 226/300 Write block 420 len 28672 data: 2 2 2 2 2 2 2
4334 227/300 Flush
4335 228/300 Read block 4680 len 40960
4336 229/300 Read block 4360 len 12288
4337 230/300 Read block 1526 len 28672
4338 231/300 Read block 490 len 32768
4339 232/300 Write block 799 len 4096 data: 4
4340 233/300 Read block 983 len 12288
4341 234/300 Read block 3198 len 36864
4342 235/300 Read block 920 len 20480
4343 236/300 Read block 3447 len 36864
4344 237/300 Write block 1852 len 4096 data: 3
4345 238/300 Read block 3309 len 8192
4346 239/300 Read block 2728 len 36864
4347 240/300 Read block 883 len 32768
4348 241/300 Read block 4169 len 20480
4349 242/300 Read block 2527 len 16384
4350 243/300 Write block 1699 len 36864 data: 4 4 4 4 2 2 2 2 2
4351 244/300 Read block 258 len 16384
4352 245/300 Write block 3915 len 16384 data: 3 3 2 2
4353 246/300 Read block 2043 len 28672
4354 247/300 Write block 1373 len 40960 data: 2 2 2 2 2 2 3 3 3 3
4355 248/300 Read block 3358 len 40960
4356 249/300 Write block 47 len 28672 data: 2 2 2 2 2 2 2
4357 250/300 Read block 4850 len 8192
4358 251/300 Read block 1197 len 28672
4359 252/300 Read block 4741 len 4096
4360 253/300 Read block 1079 len 20480
4361 254/300 Write block 4943 len 24576 data: 3 3 3 2 2 2
4362 255/300 Write block 1254 len 32768 data: 2 2 2 2 2 2 2 2
4363 256/300 Read block 563 len 24576
4364 257/300 Read block 1275 len 36864
4365 258/300 Write block 3806 len 28672 data: 3 3 3 3 3 3 3
4366 259/300 Write block 745 len 36864 data: 2 2 2 2 3 3 3 3 3
4367 260/300 Read block 3132 len 4096
4368 261/300 Write block 3794 len 24576 data: 2 2 2 2 3 3
4369 262/300 Read block 3833 len 32768
4370 263/300 Read block 4136 len 32768
4371 264/300 Flush
4372 265/300 Flush
4373 266/300 Read block 797 len 32768
4374 267/300 Read block 1367 len 12288
4375 268/300 Read block 1864 len 36864
4376 269/300 Flush
4377 270/300 Read block 3286 len 32768
4378 271/300 Write block 2408 len 40960 data: 2 2 2 2 2 2 2 3 4 4
4379 272/300 Write block 4516 len 28672 data: 3 3 3 2 2 2 2
4380 273/300 Write block 4634 len 20480 data: 3 2 2 2 2
4381 274/300 Read block 3391 len 16384
4382 275/300 Read block 501 len 36864
4383 276/300 Read block 3139 len 20480
4384 277/300 Read block 4714 len 28672
4385 278/300 Write block 1530 len 40960 data: 3 3 3 3 3 3 2 2 2 2
4386 279/300 Read block 2889 len 28672
4387 280/300 Read block 126 len 8192
4388 281/300 Read block 2826 len 28672
4389 282/300 Read block 1228 len 24576
4390 283/300 Read block 3849 len 4096
4391 284/300 Write block 3319 len 4096 data: 2
4392 285/300 Write block 885 len 36864 data: 4 4 4 4 4 3 3 4 3
4393 286/300 Write block 1895 len 4096 data: 5
4394 287/300 Read block 3750 len 16384
4395 288/300 Write block 143 len 32768 data: 2 2 2 2 2 2 2 2
4396 289/300 Read block 3086 len 40960
4397 290/300 Write block 2238 len 28672 data: 2 2 2 2 2 3 3
4398 291/300 Write block 4299 len 4096 data: 2
4399 292/300 Write block 3771 len 24576 data: 3 3 3 3 3 3
4400 293/300 Flush
4401 294/300 Write block 1948 len 16384 data: 3 3 3 3
4402 295/300 Write block 652 len 32768 data: 2 2 2 2 2 2 2 2
4403 296/300 Write block 1001 len 28672 data: 3 2 2 2 2 3 3
4404 297/300 Write block 4168 len 4096 data: 2
4405 298/300 Read block 3037 len 40960
4406 299/300 Read block 909 len 8192
4407 300/300 Read block 902 len 36864
4408 301/300 Flush
4409 Main task has work to do, go find it
4410 got dsc Start(1)
4411 start 1
4412 Jan 14 16:11:33.145 INFO request completed, latency_us: 190, response_code: 204, uri: /start/cid/1, method: POST, req_id: ef439117-753d-4d9a-9e22-55d2d4113e2e, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
4413 Replay: started 1, returned:Ok(())
4414 ----------------------------------------------------------------
4415 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
4416 States: Active Offline Active
4417 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4418 2207 Acked Read 9 Done Sent Done false
4419 2208 Acked Read 6 Done Sent Done false
4420 2209 Acked Flush 0 Done Sent Done false
4421 2210 Acked Read 8 Done Sent Done false
4422 2211 Acked Read 1 Done Sent Done false
4423 2212 Acked Read 1 Done Sent Done false
4424 2213 Acked Write 5 Done Sent Done false
4425 2214 Acked Write 3 Done Sent Done false
4426 2215 Acked Read 10 Done Sent Done false
4427 2216 Acked Flush 0 Done Sent Done false
4428 2217 Acked Read 5 Done Sent Done false
4429 2218 Acked Read 2 Done Sent Done false
4430 2219 Acked Read 9 Done Sent Done false
4431 2220 Acked Read 2 Done Sent Done false
4432 2221 Acked Write 3 Done Sent Done false
4433 2222 Acked Read 8 Done Sent Done false
4434 2223 Acked Read 2 Done Sent Done false
4435 2224 Acked Read 3 Done Sent Done false
4436 2225 Acked Read 1 Done Sent Done false
4437 2226 Acked Write 9 Done Sent Done false
4438 2227 Acked Read 1 Done Sent Done false
4439 2228 Acked Read 10 Done Sent Done false
4440 2229 Acked Read 9 Done Sent Done false
4441 2230 Acked Write 3 Done Sent Done false
4442 2231 Acked Read 7 Done Sent Done false
4443 2232 Acked Write 10 Done Sent Done false
4444 2233 Acked Read 9 Done Sent Done false
4445 2234 Acked Write 5 Done Sent Done false
4446 2235 Acked Write 4 Done Sent Done false
4447 2236 Acked Read 8 Done Sent Done false
4448 2237 Acked Read 9 Done Sent Done false
4449 2238 Acked Read 2 Done Sent Done false
4450 2239 Acked Write 8 Done Sent Done false
4451 2240 Acked Read 9 Done Sent Done false
4452 2241 Acked Read 10 Done Sent Done false
4453 2242 Acked Write 10 Done Sent Done false
4454 2243 Acked Read 10 Done Sent Done false
4455 2244 Acked Flush 0 Done Sent Done false
4456 2245 Acked Read 1 Done Sent Done false
4457 2246 Acked Read 4 Done Sent Done false
4458 2247 Acked Write 5 Done Sent Done false
4459 2248 Acked Read 9 Done Sent Done false
4460 2249 Acked Flush 0 Done Sent Done false
4461 2250 Acked Read 8 Done Sent Done false
4462 2251 Acked Read 5 Done Sent Done false
4463 2252 Acked Write 6 Done Sent Done false
4464 2253 Acked Read 10 Done Sent Done false
4465 2254 Acked Read 1 Done Sent Done false
4466 2255 Acked Write 3 Done Sent Done false
4467 2256 Acked Read 6 Done Sent Done false
4468 2257 Acked Write 10 Done Sent Done false
4469 2258 Acked Read 10 Done Sent Done false
4470 2259 Acked Read 2 Done Sent Done false
4471 2260 Acked Write 1 Done Sent Done false
4472 2261 Acked Read 5 Done Sent Done false
4473 2262 Acked Write 2 Done Sent Done false
4474 2263 Acked Write 1 Done Sent Done false
4475 2264 Acked Write 8 Done Sent Done false
4476 2265 Acked Read 2 Done Sent Done false
4477 2266 Acked Write 5 Done Sent Done false
4478 2267 Acked Write 4 Done Sent Done false
4479 2268 Acked Read 1 Done Sent Done false
4480 2269 Acked Read 3 Done Sent Done false
4481 2270 Acked Read 4 Done Sent Done false
4482 2271 Acked Read 9 Done Sent Done false
4483 2272 Acked Read 10 Done Sent Done false
4484 2273 Acked Write 8 Done Sent Done false
4485 2274 Acked Write 8 Done Sent Done false
4486 2275 Acked Read 3 Done Sent Done false
4487 2276 Acked Read 5 Done Sent Done false
4488 2277 Acked Write 6 Done Sent Done false
4489 2278 Acked Read 4 Done Sent Done false
4490 2279 Acked Read 7 Done Sent Done false
4491 2280 Acked Read 1 Done Sent Done false
4492 2281 Acked Read 8 Done Sent Done false
4493 2282 Acked Write 9 Done Sent Done false
4494 2283 Acked Write 4 Done Sent Done false
4495 2284 Acked Write 2 Done Sent Done false
4496 2285 Acked Read 10 Done Sent Done false
4497 2286 Acked Write 6 Done Sent Done false
4498 2287 Acked Read 3 Done Sent Done false
4499 2288 Acked Flush 0 Done Sent Done false
4500 2289 Acked Write 1 Done Sent Done false
4501 2290 Acked Read 8 Done Sent Done false
4502 2291 Acked Write 3 Done Sent Done false
4503 2292 Acked Read 1 Done Sent Done false
4504 2293 Acked Flush 0 Done Sent Done false
4505 2294 Acked Write 2 Done Sent Done false
4506 2295 Acked Read 4 Done Sent Done false
4507 2296 Acked Read 1 Done Sent Done false
4508 2297 Acked Read 6 Done Sent Done false
4509 2298 Acked Write 8 Done Sent Done false
4510 2299 Acked Read 10 Done Sent Done false
4511 2300 Acked Write 3 Done Sent Done false
4512 2301 Acked Write 8 Done Sent Done false
4513 2302 Acked Flush 0 Done Sent Done false
4514 2303 Acked Write 3 Done Sent Done false
4515 2304 Acked Write 5 Done Sent Done false
4516 2305 Acked Read 10 Done Sent Done false
4517 2306 Acked Flush 0 Done Sent Done false
4518 2307 Acked Read 5 Done Sent Done false
4519 2308 Acked Read 9 Done Sent Done false
4520 2309 Acked Read 6 Done Sent Done false
4521 2310 Acked Write 3 Done Sent Done false
4522 2311 Acked Write 3 Done Sent Done false
4523 2312 Acked Write 1 Done Sent Done false
4524 2313 Acked Read 4 Done Sent Done false
4525 2314 Acked Write 6 Done Sent Done false
4526 2315 Acked Write 6 Done Sent Done false
4527 2316 Acked Read 2 Done Sent Done false
4528 2317 Acked Write 5 Done Sent Done false
4529 2318 Acked Flush 0 Done Sent Done false
4530 2319 Acked Write 4 Done Sent Done false
4531 2320 Acked Write 4 Done Sent Done false
4532 2321 Acked Write 10 Done Sent Done false
4533 2322 Acked Write 2 Done Sent Done false
4534 2323 Acked Write 2 Done Sent Done false
4535 2324 Acked Read 3 Done Sent Done false
4536 2325 Acked Write 4 Done Sent Done false
4537 2326 Acked Read 7 Done Sent Done false
4538 2327 Acked Read 3 Done Sent Done false
4539 2328 Acked Write 5 Done Sent Done false
4540 2329 Acked Read 6 Done Sent Done false
4541 2330 Acked Write 3 Done Sent Done false
4542 2331 Acked Write 5 Done Sent Done false
4543 2332 Acked Write 6 Done Sent Done false
4544 2333 Acked Write 6 Done Sent Done false
4545 2334 Acked Flush 0 Done Sent Done false
4546 2335 Acked Write 6 Done Sent Done false
4547 2336 Acked Read 3 Done Sent Done false
4548 2337 Acked Write 8 Done Sent Done false
4549 2338 Acked Read 2 Done Sent Done false
4550 2339 Acked Write 3 Done Sent Done false
4551 2340 Acked Read 4 Done Sent Done false
4552 2341 Acked Read 2 Done Sent Done false
4553 2342 Acked Write 7 Done Sent Done false
4554 2343 Acked Read 5 Done Sent Done false
4555 2344 Acked Write 3 Done Sent Done false
4556 2345 Acked Write 5 Done Sent Done false
4557 2346 Acked Read 3 Done Sent Done false
4558 2347 Acked Write 5 Done Sent Done false
4559 2348 Acked Flush 0 Done Sent Done false
4560 2349 Acked Flush 0 Done Sent Done false
4561 2350 Acked Write 10 Done Sent Done false
4562 2351 Acked Read 9 Done Sent Done false
4563 2352 Acked Read 6 Done Sent Done false
4564 2353 Acked Write 10 Done Sent Done false
4565 2354 Acked Write 10 Done Sent Done false
4566 2355 Acked Write 5 Done Sent Done false
4567 2356 Acked Write 5 Done Sent Done false
4568 2357 Acked Flush 0 Done Sent Done false
4569 2358 Acked Write 6 Done Sent Done false
4570 2359 Acked Read 1 Done Sent Done false
4571 2360 Acked Read 3 Done Sent Done false
4572 2361 Acked Read 2 Done Sent Done false
4573 2362 Acked Write 1 Done Sent Done false
4574 2363 Acked Write 3 Done Sent Done false
4575 2364 Acked Read 7 Done Sent Done false
4576 2365 Acked Write 10 Done Sent Done false
4577 2366 Acked Write 9 Done Sent Done false
4578 2367 Acked Read 7 Done Sent Done false
4579 2368 Acked Write 3 Done Sent Done false
4580 2369 Acked Write 7 Done Sent Done false
4581 2370 Acked Read 7 Done Sent Done false
4582 2371 Acked Read 1 Done Sent Done false
4583 2372 Acked Flush 0 Done Sent Done false
4584 2373 Acked Write 7 Done Sent Done false
4585 2374 Acked Write 9 Done Sent Done false
4586 2375 Acked Flush 0 Done Sent Done false
4587 2376 Acked Write 5 Done Sent Done false
4588 2377 Acked Flush 0 Done Sent Done false
4589 2378 Acked Read 3 Done Sent Done false
4590 2379 Acked Write 6 Done Sent Done false
4591 2380 Acked Read 5 Done Sent Done false
4592 2381 Acked Read 8 Done Sent Done false
4593 2382 Acked Read 3 Done Sent Done false
4594 2383 Acked Read 9 Done Sent Done false
4595 2384 Acked Flush 0 Done Sent Done false
4596 2385 Acked Flush 0 Done Sent Done false
4597 2386 Acked Write 5 Done Sent Done false
4598 2387 Acked Write 9 Done Sent Done false
4599 2388 Acked Read 3 Done Sent Done false
4600 2389 Acked Write 3 Done Sent Done false
4601 2390 Acked Read 5 Done Sent Done false
4602 2391 Acked Write 4 Done Sent Done false
4603 2392 Acked Read 9 Done Sent Done false
4604 2393 Acked Read 9 Done Sent Done false
4605 2394 Acked Read 4 Done Sent Done false
4606 2395 Acked Read 4 Done Sent Done false
4607 2396 Acked Read 9 Done Sent Done false
4608 2397 Acked Read 7 Done Sent Done false
4609 2398 Acked Read 3 Done Sent Done false
4610 2399 Acked Flush 0 Done Sent Done false
4611 2400 Acked Read 3 Done Sent Done false
4612 2401 Acked Write 9 Done Sent Done false
4613 2402 Acked Write 1 Done Sent Done false
4614 2403 Acked Read 8 Done Sent Done false
4615 2404 Acked Read 1 Done Sent Done false
4616 2405 Acked Read 4 Done Sent Done false
4617 2406 Acked Write 1 Done Sent Done false
4618 2407 Acked Flush 0 Done Sent Done false
4619 2408 Acked Read 1 Done Sent Done false
4620 2409 Acked Read 1 Done Sent Done false
4621 2410 Acked Read 6 Done Sent Done false
4622 2411 Acked Write 2 Done Sent Done false
4623 2412 Acked Read 7 Done Sent Done false
4624 2413 Acked Read 4 Done Sent Done false
4625 2414 Acked Read 1 Done Sent Done false
4626 2415 Acked Read 4 Done Sent Done false
4627 2416 Acked Write 8 Done Sent Done false
4628 2417 Acked Read 7 Done Sent Done false
4629 2418 Acked Read 5 Done Sent Done false
4630 2419 Acked Read 2 Done Sent Done false
4631 2420 Acked Read 6 Done Sent Done false
4632 2421 Acked Write 8 Done Sent Done false
4633 2422 Acked Read 10 Done Sent Done false
4634 2423 Acked Read 3 Done Sent Done false
4635 2424 Acked Read 10 Done Sent Done false
4636 2425 Acked Read 8 Done Sent Done false
4637 2426 Acked Read 8 Done Sent Done false
4638 2427 Acked Flush 0 Done Sent Done false
4639 2428 Acked Flush 0 Done Sent Done false
4640 2429 Acked Read 9 Done Sent Done false
4641 2430 Acked Read 9 Done Sent Done false
4642 2431 Acked Read 5 Done Sent Done false
4643 2432 Acked Write 7 Done Sent Done false
4644 2433 Acked Flush 0 Done Sent Done false
4645 2434 Acked Read 10 Done Sent Done false
4646 2435 Acked Read 3 Done Sent Done false
4647 2436 Acked Read 7 Done Sent Done false
4648 2437 Acked Read 8 Done Sent Done false
4649 2438 Acked Write 1 Done Sent Done false
4650 2439 Acked Read 3 Done Sent Done false
4651 2440 Acked Read 9 Done Sent Done false
4652 2441 Acked Read 5 Done Sent Done false
4653 2442 Acked Read 9 Done Sent Done false
4654 2443 Acked Write 1 Done Sent Done false
4655 2444 Acked Read 2 Done Sent Done false
4656 2445 Acked Read 9 Done Sent Done false
4657 2446 Acked Read 8 Done Sent Done false
4658 2447 Acked Read 5 Done Sent Done false
4659 2448 Acked Read 4 Done Sent Done false
4660 2449 Acked Write 9 Done Sent Done false
4661 2450 Acked Read 4 Done Sent Done false
4662 2451 Acked Write 4 Done Sent Done false
4663 2452 Acked Read 7 Done Sent Done false
4664 2453 Acked Write 10 Done Sent Done false
4665 2454 Acked Read 10 Done Sent Done false
4666 2455 Acked Write 7 Done Sent Done false
4667 2456 Acked Read 2 Done Sent Done false
4668 2457 Acked Read 7 Done Sent Done false
4669 2458 Acked Read 1 Done Sent Done false
4670 2459 Acked Read 5 Done Sent Done false
4671 2460 Acked Write 6 Done Sent Done false
4672 2461 Acked Write 8 Done Sent Done false
4673 2462 Acked Read 6 Done Sent Done false
4674 2463 Acked Read 9 Done Sent Done false
4675 2464 Acked Write 7 Done Sent Done false
4676 2465 Acked Write 9 Done Sent Done false
4677 2466 Acked Read 1 Done Sent Done false
4678 2467 Acked Write 6 Done Sent Done false
4679 2468 Acked Read 8 Done Sent Done false
4680 2469 Acked Read 8 Done Sent Done false
4681 2470 Acked Flush 0 Done Sent Done false
4682 2471 Acked Flush 0 Done Sent Done false
4683 2472 Acked Read 8 Done Sent Done false
4684 2473 Acked Read 3 Done Sent Done false
4685 2474 Acked Read 9 Done Sent Done false
4686 2475 Acked Flush 0 Done Sent Done false
4687 2476 Acked Read 8 Done Sent Done false
4688 2477 Acked Write 10 Done Sent Done false
4689 2478 Acked Write 7 Done Sent Done false
4690 2479 Acked Write 5 Done Sent Done false
4691 2480 Acked Read 4 Done Sent Done false
4692 2481 Acked Read 9 Done Sent Done false
4693 2482 Acked Read 5 Done Sent Done false
4694 2483 Acked Read 7 Done Sent Done false
4695 2484 Acked Write 10 Done Sent Done false
4696 2485 Acked Read 7 Done Sent Done false
4697 2486 Acked Read 2 Done Sent Done false
4698 2487 Acked Read 7 Done Sent Done false
4699 2488 Acked Read 6 Done Sent Done false
4700 2489 Acked Read 1 Done Sent Done false
4701 2490 Acked Write 1 Done Sent Done false
4702 2491 Acked Write 9 Done Sent Done false
4703 2492 Acked Write 1 Done Sent Done false
4704 2493 Acked Read 4 Done Sent Done false
4705 2494 Acked Write 8 Done Sent Done false
4706 2495 Acked Read 10 Done Sent Done false
4707 2496 Acked Write 7 Done Sent Done false
4708 2497 Acked Write 1 Done Sent Done false
4709 2498 Acked Write 6 Done Sent Done false
4710 2499 Acked Flush 0 Done Sent Done false
4711 2500 Acked Write 4 Done Sent Done false
4712 2501 Acked Write 8 Done Sent Done false
4713 2502 Acked Write 7 Done Sent Done false
4714 2503 Acked Write 1 Done Sent Done false
4715 2504 Acked Read 10 Done Sent Done false
4716 2505 Acked Read 2 Done Sent Done false
4717 2506 Acked Read 9 Done Sent Done false
4718 2507 Acked Flush 0 Done Sent Done false
4719 STATES DS:0 DS:1 DS:2 TOTAL
4720 Sent 0 301 0 301
4721 Done 301 0 301 602
4722 Skipped 0 0 0 0
4723 Error 0 0 0 0
4724 Last Flush: 2507 2206 2507
4725 Downstairs last five completed: 2206 2205 2204 2203 2202
4726 Upstairs last five completed: 2507 2506 2505 2504 2503
4727 CLIENT: Up:0 ds:301 act:2
4728 [8820] got start action
4729 [8820] I am going to restart
4730 Starting downstairs at port 8820
4731 Make output file at "/tmp/dsc/downstairs-8820.txt"
4732 [8820][1] reports Starting
4733 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1207)
4734 [8820][1] reports Running
4735 ----------------------------------------------------------------
4736 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
4737 States: Active Offline Active
4738 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4739 2207 Acked Read 9 Done Sent Done false
4740 2208 Acked Read 6 Done Sent Done false
4741 2209 Acked Flush 0 Done Sent Done false
4742 2210 Acked Read 8 Done Sent Done false
4743 2211 Acked Read 1 Done Sent Done false
4744 2212 Acked Read 1 Done Sent Done false
4745 2213 Acked Write 5 Done Sent Done false
4746 2214 Acked Write 3 Done Sent Done false
4747 2215 Acked Read 10 Done Sent Done false
4748 2216 Acked Flush 0 Done Sent Done false
4749 2217 Acked Read 5 Done Sent Done false
4750 2218 Acked Read 2 Done Sent Done false
4751 2219 Acked Read 9 Done Sent Done false
4752 2220 Acked Read 2 Done Sent Done false
4753 2221 Acked Write 3 Done Sent Done false
4754 2222 Acked Read 8 Done Sent Done false
4755 2223 Acked Read 2 Done Sent Done false
4756 2224 Acked Read 3 Done Sent Done false
4757 2225 Acked Read 1 Done Sent Done false
4758 2226 Acked Write 9 Done Sent Done false
4759 2227 Acked Read 1 Done Sent Done false
4760 2228 Acked Read 10 Done Sent Done false
4761 2229 Acked Read 9 Done Sent Done false
4762 2230 Acked Write 3 Done Sent Done false
4763 2231 Acked Read 7 Done Sent Done false
4764 2232 Acked Write 10 Done Sent Done false
4765 2233 Acked Read 9 Done Sent Done false
4766 2234 Acked Write 5 Done Sent Done false
4767 2235 Acked Write 4 Done Sent Done false
4768 2236 Acked Read 8 Done Sent Done false
4769 2237 Acked Read 9 Done Sent Done false
4770 2238 Acked Read 2 Done Sent Done false
4771 2239 Acked Write 8 Done Sent Done false
4772 2240 Acked Read 9 Done Sent Done false
4773 2241 Acked Read 10 Done Sent Done false
4774 2242 Acked Write 10 Done Sent Done false
4775 2243 Acked Read 10 Done Sent Done false
4776 2244 Acked Flush 0 Done Sent Done false
4777 2245 Acked Read 1 Done Sent Done false
4778 2246 Acked Read 4 Done Sent Done false
4779 2247 Acked Write 5 Done Sent Done false
4780 2248 Acked Read 9 Done Sent Done false
4781 2249 Acked Flush 0 Done Sent Done false
4782 2250 Acked Read 8 Done Sent Done false
4783 2251 Acked Read 5 Done Sent Done false
4784 2252 Acked Write 6 Done Sent Done false
4785 2253 Acked Read 10 Done Sent Done false
4786 2254 Acked Read 1 Done Sent Done false
4787 2255 Acked Write 3 Done Sent Done false
4788 2256 Acked Read 6 Done Sent Done false
4789 2257 Acked Write 10 Done Sent Done false
4790 2258 Acked Read 10 Done Sent Done false
4791 2259 Acked Read 2 Done Sent Done false
4792 2260 Acked Write 1 Done Sent Done false
4793 2261 Acked Read 5 Done Sent Done false
4794 2262 Acked Write 2 Done Sent Done false
4795 2263 Acked Write 1 Done Sent Done false
4796 2264 Acked Write 8 Done Sent Done false
4797 2265 Acked Read 2 Done Sent Done false
4798 2266 Acked Write 5 Done Sent Done false
4799 2267 Acked Write 4 Done Sent Done false
4800 2268 Acked Read 1 Done Sent Done false
4801 2269 Acked Read 3 Done Sent Done false
4802 2270 Acked Read 4 Done Sent Done false
4803 2271 Acked Read 9 Done Sent Done false
4804 2272 Acked Read 10 Done Sent Done false
4805 2273 Acked Write 8 Done Sent Done false
4806 2274 Acked Write 8 Done Sent Done false
4807 2275 Acked Read 3 Done Sent Done false
4808 2276 Acked Read 5 Done Sent Done false
4809 2277 Acked Write 6 Done Sent Done false
4810 2278 Acked Read 4 Done Sent Done false
4811 2279 Acked Read 7 Done Sent Done false
4812 2280 Acked Read 1 Done Sent Done false
4813 2281 Acked Read 8 Done Sent Done false
4814 2282 Acked Write 9 Done Sent Done false
4815 2283 Acked Write 4 Done Sent Done false
4816 2284 Acked Write 2 Done Sent Done false
4817 2285 Acked Read 10 Done Sent Done false
4818 2286 Acked Write 6 Done Sent Done false
4819 2287 Acked Read 3 Done Sent Done false
4820 2288 Acked Flush 0 Done Sent Done false
4821 2289 Acked Write 1 Done Sent Done false
4822 2290 Acked Read 8 Done Sent Done false
4823 2291 Acked Write 3 Done Sent Done false
4824 2292 Acked Read 1 Done Sent Done false
4825 2293 Acked Flush 0 Done Sent Done false
4826 2294 Acked Write 2 Done Sent Done false
4827 2295 Acked Read 4 Done Sent Done false
4828 2296 Acked Read 1 Done Sent Done false
4829 2297 Acked Read 6 Done Sent Done false
4830 2298 Acked Write 8 Done Sent Done false
4831 2299 Acked Read 10 Done Sent Done false
4832 2300 Acked Write 3 Done Sent Done false
4833 2301 Acked Write 8 Done Sent Done false
4834 2302 Acked Flush 0 Done Sent Done false
4835 2303 Acked Write 3 Done Sent Done false
4836 2304 Acked Write 5 Done Sent Done false
4837 2305 Acked Read 10 Done Sent Done false
4838 2306 Acked Flush 0 Done Sent Done false
4839 2307 Acked Read 5 Done Sent Done false
4840 2308 Acked Read 9 Done Sent Done false
4841 2309 Acked Read 6 Done Sent Done false
4842 2310 Acked Write 3 Done Sent Done false
4843 2311 Acked Write 3 Done Sent Done false
4844 2312 Acked Write 1 Done Sent Done false
4845 2313 Acked Read 4 Done Sent Done false
4846 2314 Acked Write 6 Done Sent Done false
4847 2315 Acked Write 6 Done Sent Done false
4848 2316 Acked Read 2 Done Sent Done false
4849 2317 Acked Write 5 Done Sent Done false
4850 2318 Acked Flush 0 Done Sent Done false
4851 2319 Acked Write 4 Done Sent Done false
4852 2320 Acked Write 4 Done Sent Done false
4853 2321 Acked Write 10 Done Sent Done false
4854 2322 Acked Write 2 Done Sent Done false
4855 2323 Acked Write 2 Done Sent Done false
4856 2324 Acked Read 3 Done Sent Done false
4857 2325 Acked Write 4 Done Sent Done false
4858 2326 Acked Read 7 Done Sent Done false
4859 2327 Acked Read 3 Done Sent Done false
4860 2328 Acked Write 5 Done Sent Done false
4861 2329 Acked Read 6 Done Sent Done false
4862 2330 Acked Write 3 Done Sent Done false
4863 2331 Acked Write 5 Done Sent Done false
4864 2332 Acked Write 6 Done Sent Done false
4865 2333 Acked Write 6 Done Sent Done false
4866 2334 Acked Flush 0 Done Sent Done false
4867 2335 Acked Write 6 Done Sent Done false
4868 2336 Acked Read 3 Done Sent Done false
4869 2337 Acked Write 8 Done Sent Done false
4870 2338 Acked Read 2 Done Sent Done false
4871 2339 Acked Write 3 Done Sent Done false
4872 2340 Acked Read 4 Done Sent Done false
4873 2341 Acked Read 2 Done Sent Done false
4874 2342 Acked Write 7 Done Sent Done false
4875 2343 Acked Read 5 Done Sent Done false
4876 2344 Acked Write 3 Done Sent Done false
4877 2345 Acked Write 5 Done Sent Done false
4878 2346 Acked Read 3 Done Sent Done false
4879 2347 Acked Write 5 Done Sent Done false
4880 2348 Acked Flush 0 Done Sent Done false
4881 2349 Acked Flush 0 Done Sent Done false
4882 2350 Acked Write 10 Done Sent Done false
4883 2351 Acked Read 9 Done Sent Done false
4884 2352 Acked Read 6 Done Sent Done false
4885 2353 Acked Write 10 Done Sent Done false
4886 2354 Acked Write 10 Done Sent Done false
4887 2355 Acked Write 5 Done Sent Done false
4888 2356 Acked Write 5 Done Sent Done false
4889 2357 Acked Flush 0 Done Sent Done false
4890 2358 Acked Write 6 Done Sent Done false
4891 2359 Acked Read 1 Done Sent Done false
4892 2360 Acked Read 3 Done Sent Done false
4893 2361 Acked Read 2 Done Sent Done false
4894 2362 Acked Write 1 Done Sent Done false
4895 2363 Acked Write 3 Done Sent Done false
4896 2364 Acked Read 7 Done Sent Done false
4897 2365 Acked Write 10 Done Sent Done false
4898 2366 Acked Write 9 Done Sent Done false
4899 2367 Acked Read 7 Done Sent Done false
4900 2368 Acked Write 3 Done Sent Done false
4901 2369 Acked Write 7 Done Sent Done false
4902 2370 Acked Read 7 Done Sent Done false
4903 2371 Acked Read 1 Done Sent Done false
4904 2372 Acked Flush 0 Done Sent Done false
4905 2373 Acked Write 7 Done Sent Done false
4906 2374 Acked Write 9 Done Sent Done false
4907 2375 Acked Flush 0 Done Sent Done false
4908 2376 Acked Write 5 Done Sent Done false
4909 2377 Acked Flush 0 Done Sent Done false
4910 2378 Acked Read 3 Done Sent Done false
4911 2379 Acked Write 6 Done Sent Done false
4912 2380 Acked Read 5 Done Sent Done false
4913 2381 Acked Read 8 Done Sent Done false
4914 2382 Acked Read 3 Done Sent Done false
4915 2383 Acked Read 9 Done Sent Done false
4916 2384 Acked Flush 0 Done Sent Done false
4917 2385 Acked Flush 0 Done Sent Done false
4918 2386 Acked Write 5 Done Sent Done false
4919 2387 Acked Write 9 Done Sent Done false
4920 2388 Acked Read 3 Done Sent Done false
4921 2389 Acked Write 3 Done Sent Done false
4922 2390 Acked Read 5 Done Sent Done false
4923 2391 Acked Write 4 Done Sent Done false
4924 2392 Acked Read 9 Done Sent Done false
4925 2393 Acked Read 9 Done Sent Done false
4926 2394 Acked Read 4 Done Sent Done false
4927 2395 Acked Read 4 Done Sent Done false
4928 2396 Acked Read 9 Done Sent Done false
4929 2397 Acked Read 7 Done Sent Done false
4930 2398 Acked Read 3 Done Sent Done false
4931 2399 Acked Flush 0 Done Sent Done false
4932 2400 Acked Read 3 Done Sent Done false
4933 2401 Acked Write 9 Done Sent Done false
4934 2402 Acked Write 1 Done Sent Done false
4935 2403 Acked Read 8 Done Sent Done false
4936 2404 Acked Read 1 Done Sent Done false
4937 2405 Acked Read 4 Done Sent Done false
4938 2406 Acked Write 1 Done Sent Done false
4939 2407 Acked Flush 0 Done Sent Done false
4940 2408 Acked Read 1 Done Sent Done false
4941 2409 Acked Read 1 Done Sent Done false
4942 2410 Acked Read 6 Done Sent Done false
4943 2411 Acked Write 2 Done Sent Done false
4944 2412 Acked Read 7 Done Sent Done false
4945 2413 Acked Read 4 Done Sent Done false
4946 2414 Acked Read 1 Done Sent Done false
4947 2415 Acked Read 4 Done Sent Done false
4948 2416 Acked Write 8 Done Sent Done false
4949 2417 Acked Read 7 Done Sent Done false
4950 2418 Acked Read 5 Done Sent Done false
4951 2419 Acked Read 2 Done Sent Done false
4952 2420 Acked Read 6 Done Sent Done false
4953 2421 Acked Write 8 Done Sent Done false
4954 2422 Acked Read 10 Done Sent Done false
4955 2423 Acked Read 3 Done Sent Done false
4956 2424 Acked Read 10 Done Sent Done false
4957 2425 Acked Read 8 Done Sent Done false
4958 2426 Acked Read 8 Done Sent Done false
4959 2427 Acked Flush 0 Done Sent Done false
4960 2428 Acked Flush 0 Done Sent Done false
4961 2429 Acked Read 9 Done Sent Done false
4962 2430 Acked Read 9 Done Sent Done false
4963 2431 Acked Read 5 Done Sent Done false
4964 2432 Acked Write 7 Done Sent Done false
4965 2433 Acked Flush 0 Done Sent Done false
4966 2434 Acked Read 10 Done Sent Done false
4967 2435 Acked Read 3 Done Sent Done false
4968 2436 Acked Read 7 Done Sent Done false
4969 2437 Acked Read 8 Done Sent Done false
4970 2438 Acked Write 1 Done Sent Done false
4971 2439 Acked Read 3 Done Sent Done false
4972 2440 Acked Read 9 Done Sent Done false
4973 2441 Acked Read 5 Done Sent Done false
4974 2442 Acked Read 9 Done Sent Done false
4975 2443 Acked Write 1 Done Sent Done false
4976 2444 Acked Read 2 Done Sent Done false
4977 2445 Acked Read 9 Done Sent Done false
4978 2446 Acked Read 8 Done Sent Done false
4979 2447 Acked Read 5 Done Sent Done false
4980 2448 Acked Read 4 Done Sent Done false
4981 2449 Acked Write 9 Done Sent Done false
4982 2450 Acked Read 4 Done Sent Done false
4983 2451 Acked Write 4 Done Sent Done false
4984 2452 Acked Read 7 Done Sent Done false
4985 2453 Acked Write 10 Done Sent Done false
4986 2454 Acked Read 10 Done Sent Done false
4987 2455 Acked Write 7 Done Sent Done false
4988 2456 Acked Read 2 Done Sent Done false
4989 2457 Acked Read 7 Done Sent Done false
4990 2458 Acked Read 1 Done Sent Done false
4991 2459 Acked Read 5 Done Sent Done false
4992 2460 Acked Write 6 Done Sent Done false
4993 2461 Acked Write 8 Done Sent Done false
4994 2462 Acked Read 6 Done Sent Done false
4995 2463 Acked Read 9 Done Sent Done false
4996 2464 Acked Write 7 Done Sent Done false
4997 2465 Acked Write 9 Done Sent Done false
4998 2466 Acked Read 1 Done Sent Done false
4999 2467 Acked Write 6 Done Sent Done false
5000 2468 Acked Read 8 Done Sent Done false
5001 2469 Acked Read 8 Done Sent Done false
5002 2470 Acked Flush 0 Done Sent Done false
5003 2471 Acked Flush 0 Done Sent Done false
5004 2472 Acked Read 8 Done Sent Done false
5005 2473 Acked Read 3 Done Sent Done false
5006 2474 Acked Read 9 Done Sent Done false
5007 2475 Acked Flush 0 Done Sent Done false
5008 2476 Acked Read 8 Done Sent Done false
5009 2477 Acked Write 10 Done Sent Done false
5010 2478 Acked Write 7 Done Sent Done false
5011 2479 Acked Write 5 Done Sent Done false
5012 2480 Acked Read 4 Done Sent Done false
5013 2481 Acked Read 9 Done Sent Done false
5014 2482 Acked Read 5 Done Sent Done false
5015 2483 Acked Read 7 Done Sent Done false
5016 2484 Acked Write 10 Done Sent Done false
5017 2485 Acked Read 7 Done Sent Done false
5018 2486 Acked Read 2 Done Sent Done false
5019 2487 Acked Read 7 Done Sent Done false
5020 2488 Acked Read 6 Done Sent Done false
5021 2489 Acked Read 1 Done Sent Done false
5022 2490 Acked Write 1 Done Sent Done false
5023 2491 Acked Write 9 Done Sent Done false
5024 2492 Acked Write 1 Done Sent Done false
5025 2493 Acked Read 4 Done Sent Done false
5026 2494 Acked Write 8 Done Sent Done false
5027 2495 Acked Read 10 Done Sent Done false
5028 2496 Acked Write 7 Done Sent Done false
5029 2497 Acked Write 1 Done Sent Done false
5030 2498 Acked Write 6 Done Sent Done false
5031 2499 Acked Flush 0 Done Sent Done false
5032 2500 Acked Write 4 Done Sent Done false
5033 2501 Acked Write 8 Done Sent Done false
5034 2502 Acked Write 7 Done Sent Done false
5035 2503 Acked Write 1 Done Sent Done false
5036 2504 Acked Read 10 Done Sent Done false
5037 2505 Acked Read 2 Done Sent Done false
5038 2506 Acked Read 9 Done Sent Done false
5039 2507 Acked Flush 0 Done Sent Done false
5040 STATES DS:0 DS:1 DS:2 TOTAL
5041 Sent 0 301 0 301
5042 Done 301 0 301 602
5043 Skipped 0 0 0 0
5044 Error 0 0 0 0
5045 Last Flush: 2507 2206 2507
5046 Downstairs last five completed: 2206 2205 2204 2203 2202
5047 Upstairs last five completed: 2507 2506 2505 2504 2503
5048 CLIENT: Up:0 ds:301 act:2
50492025-01-14T16:11:38.902ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
50502025-01-14T16:11:38.902ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:63056) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
50512025-01-14T16:11:38.903ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
50522025-01-14T16:11:38.904ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
50532025-01-14T16:11:38.904ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
50542025-01-14T16:11:38.904ZINFOcrucible: send last flush ID to this DS: 2206 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
50552025-01-14T16:11:38.904ZINFOcrucible: Replied this last flush ID: 2206 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
50562025-01-14T16:11:38.904ZINFOcrucible: [1] client re-new 301 jobs since flush 2206 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
50572025-01-14T16:11:38.904ZINFOcrucible: [1] Replayed 301 jobs since flush: 2206 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
5058 ----------------------------------------------------------------
5059 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
5060 Downstairs last five completed: 2507 2506 2505 2504 2503
5061 Upstairs last five completed: 2507 2506 2505 2504 2503
5062 CLIENT: Up:0 ds:0 act:3
5063 Replay: All jobs finished
5064 Main task has work to do, go find it
5065 got dsc Stop(2)
5066 stop 2
5067 Jan 14 16:11:41.150 INFO request completed, latency_us: 300, response_code: 204, uri: [8830] Got stop action so:false kr:false
5068 /stop/cid/2, method: POST, req_id: be180502-3fc1-4629-b2db-ee7e0285258f, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
50692025-01-14T16:11:41.150ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
50702025-01-14T16:11:41.150ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
50712025-01-14T16:11:41.150ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
5072 Jan 14 16:11:41.151 INFO request completed, latency_us: 185, response_code: 200, uri: /state/cid/2, method: GET, req_id: 148fc67f-cc82-4ae5-9a08-8b915b4f82f1, remote_addr: 127.0.0.1:61942{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-14T16:11:41.15113104Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
5073 , local_addr: 127.0.0.1:9998
5074 [8830] Exited with: ExitStatus(unix_wait_status(9))
5075 [8830][2] reports Exit
5076 Jan 14 16:11:45.153 INFO request completed, latency_us: 193, response_code: 200, uri: /state/cid/2, method: GET, req_id: 29923448-a5e2-47bf-9b80-50f6077a864e, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
5077 001/300 Read block 2880 len 4096
5078 002/300 Write block 363 len 4096 data: 2
5079 003/300 Write block 2959 len 8192 data: 3 3
5080 004/300 Read block 1880 len 4096
5081 005/300 Write block 4199 len 28672 data: 2 2 2 2 2 2 2
5082 006/300 Read block 1330 len 36864
5083 007/300 Flush
5084 008/300 Read block 411 len 32768
5085 009/300 Write block 1782 len 40960 data: 3 3 2 2 2 2 2 2 2 2
5086 010/300 Write block 4770 len 16384 data: 3 3 3 3
5087 011/300 Write block 79 len 40960 data: 3 3 2 2 2 2 2 2 2 2
5088 012/300 Flush
5089 013/300 Flush
5090 014/300 Write block 2618 len 40960 data: 2 2 2 2 3 3 4 4 4 2
5091 015/300 Write block 3895 len 24576 data: 2 4 3 3 3 3
5092 016/300 Read block 659 len 40960
5093 017/300 Write block 546 len 32768 data: 2 2 2 2 2 2 2 2
5094 018/300 Read block 613 len 20480
5095 019/300 Write block 1486 len 32768 data: 2 2 2 2 3 3 3 3
5096 020/300 Read block 4873 len 36864
5097 021/300 Write block 2371 len 4096 data: 3
5098 022/300 Write block 4122 len 32768 data: 2 2 3 3 3 4 4 5
5099 023/300 Write block 1681 len 20480 data: 2 2 3 3 3
5100 024/300 Write block 1608 len 4096 data: 2
5101 025/300 Read block 233 len 32768
5102 026/300 Read block 2002 len 20480
5103 027/300 Write block 4883 len 36864 data: 2 2 2 2 2 3 3 3 3
5104 028/300 Read block 330 len 8192
5105 029/300 Read block 4818 len 12288
5106 030/300 Read block 2194 len 20480
5107 031/300 Read block 522 len 32768
5108 032/300 Write block 4898 len 32768 data: 3 3 3 3 2 2 2 2
5109 033/300 Read block 3121 len 28672
5110 034/300 Write block 1905 len 36864 data: 2 2 2 2 2 2 2 2 2
5111 035/300 Read block 4868 len 32768
5112 036/300 Read block 4032 len 4096
5113 037/300 Write block 413 len 4096 data: 3
5114 038/300 Read block 1677 len 24576
5115 039/300 Write block 131 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5116 040/300 Write block 2949 len 12288 data: 2 2 2
5117 041/300 Read block 911 len 8192
5118 042/300 Write block 4971 len 32768 data: 2 2 2 2 2 2 2 2
5119 043/300 Write block 1497 len 36864 data: 3 3 2 2 2 2 2 2 2
5120 044/300 Write block 4847 len 20480 data: 5 5 5 5 3
5121 045/300 Write block 4967 len 4096 data: 2
5122 046/300 Write block 3298 len 16384 data: 2 2 2 2
5123 047/300 Write block 2159 len 24576 data: 2 2 3 3 3 3
5124 048/300 Read block 4891 len 4096
5125 049/300 Write block 807 len 40960 data: 2 2 2 2 2 2 2 3 3 3
5126 050/300 Write block 4703 len 4096 data: 2
5127 051/300 Write block 3420 len 32768 data: 4 4 3 3 3 2 2 2
5128 052/300 Read block 558 len 40960
5129 053/300 Read block 610 len 36864
5130 054/300 Read block 279 len 8192
5131 055/300 Flush
5132 056/300 Read block 803 len 32768
5133 057/300 Flush
5134 058/300 Read block 1729 len 24576
5135 059/300 Read block 848 len 40960
5136 060/300 Read block 1496 len 12288
5137 061/300 Flush
5138 062/300 Read block 426 len 40960
5139 063/300 Flush
5140 064/300 Read block 4576 len 28672
5141 065/300 Write block 2613 len 32768 data: 2 2 2 2 2 3 3 3
5142 066/300 Write block 2431 len 16384 data: 2 3 3 3
5143 067/300 Read block 4339 len 36864
5144 068/300 Write block 1206 len 28672 data: 3 3 2 2 2 2 2
5145 069/300 Write block 3688 len 8192 data: 3 2
5146 070/300 Read block 3965 len 36864
5147 071/300 Read block 19 len 16384
5148 072/300 Read block 3013 len 8192
5149 073/300 Read block 4803 len 12288
5150 074/300 Read block 94 len 32768
5151 075/300 Read block 2321 len 24576
5152 076/300 Read block 635 len 36864
5153 077/300 Flush
5154 078/300 Write block 4758 len 8192 data: 2 2
5155 079/300 Write block 3820 len 40960 data: 4 3 4 4 4 4 4 2 2 2
5156 080/300 Read block 3744 len 28672
5157 081/300 Read block 308 len 36864
5158 082/300 Read block 1816 len 16384
5159 083/300 Read block 3549 len 40960
5160 084/300 Flush
5161 085/300 Write block 1737 len 8192 data: 3 3
5162 086/300 Read block 660 len 36864
5163 087/300 Read block 172 len 16384
5164 088/300 Read block 3533 len 40960
5165 089/300 Write block 3788 len 12288 data: 2 2 2
5166 090/300 Write block 680 len 40960 data: 2 2 3 3 3 3 3 3 3 3
5167 091/300 Write block 3349 len 28672 data: 3 2 2 3 3 4 4
5168 092/300 Read block 4084 len 24576
5169 093/300 Flush
5170 094/300 Read block 4756 len 24576
5171 095/300 Read block 143 len 12288
5172 096/300 Read block 3507 len 12288
5173 097/300 Flush
5174 098/300 Write block 4713 len 8192 data: 2 2
5175 099/300 Read block 798 len 12288
5176 100/300 Read block 975 len 24576
5177 101/300 Write block 1337 len 40960 data: 3 3 3 3 3 2 2 2 2 2
5178 102/300 Read block 2202 len 28672
5179 103/300 Write block 416 len 36864 data: 3 3 3 2 3 3 3 3 3
5180 104/300 Flush
5181 105/300 Write block 167 len 28672 data: 2 3 3 3 3 3 4
5182 106/300 Read block 4132 len 12288
5183 107/300 Write block 4540 len 16384 data: 3 3 3 5
5184 108/300 Read block 2824 len 16384
5185 109/300 Read block 3010 len 36864
5186 110/300 Flush
5187 111/300 Write block 4562 len 32768 data: 2 2 2 2 2 2 2 2
5188 112/300 Write block 1446 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5189 113/300 Write block 817 len 4096 data: 3
5190 114/300 Write block 3836 len 40960 data: 2 2 2 2 2 2 2 2 3 3
5191 115/300 Read block 628 len 8192
5192 116/300 Read block 1714 len 16384
5193 117/300 Read block 3236 len 8192
5194 118/300 Write block 1236 len 20480 data: 2 2 2 2 2
5195 119/300 Flush
5196 120/300 Write block 1126 len 24576 data: 2 2 2 2 2 2
5197 121/300 Read block 562 len 32768
5198 122/300 Read block 4202 len 8192
5199 123/300 Read block 3046 len 28672
5200 124/300 Write block 797 len 12288 data: 3 3 5
5201 125/300 Write block 4883 len 20480 data: 3 3 3 3 3
5202 126/300 Write block 2113 len 36864 data: 4 3 3 3 2 2 2 2 2
5203 127/300 Read block 4678 len 16384
5204 128/300 Read block 50 len 36864
5205 129/300 Flush
5206 130/300 Write block 276 len 20480 data: 2 2 2 2 2
5207 131/300 Write block 214 len 28672 data: 4 4 3 2 3 3 3
5208 132/300 Read block 753 len 20480
5209 133/300 Read block 4107 len 4096
5210 134/300 Read block 3475 len 36864
5211 135/300 Read block 113 len 24576
5212 136/300 Read block 2573 len 36864
5213 137/300 Read block 4790 len 12288
5214 138/300 Write block 1465 len 12288 data: 4 4 4
5215 139/300 Write block 4170 len 36864 data: 2 2 2 2 2 2 3 3 3
5216 140/300 Write block 3688 len 24576 data: 4 3 2 2 2 2
5217 141/300 Read block 1395 len 8192
5218 142/300 Write block 1182 len 20480 data: 3 3 2 2 2
5219 143/300 Write block 3227 len 12288 data: 2 3 3
5220 144/300 Read block 4459 len 12288
5221 145/300 Read block 897 len 28672
5222 146/300 Read block 910 len 8192
5223 147/300 Write block 2613 len 28672 data: 3 3 3 3 3 4 4
5224 148/300 Read block 990 len 40960
5225 149/300 Read block 4329 len 12288
5226 150/300 Write block 3621 len 20480 data: 2 2 3 2 2
5227 151/300 Write block 2106 len 12288 data: 2 3 3
5228 152/300 Read block 2217 len 8192
5229 153/300 Read block 2786 len 16384
5230 154/300 Flush
5231 155/300 Write block 1231 len 40960 data: 2 2 2 2 2 3 3 3 3 3
5232 156/300 Read block 4680 len 40960
5233 157/300 Read block 918 len 16384
5234 158/300 Write block 614 len 20480 data: 2 2 2 2 4
5235 159/300 Read block 1266 len 24576
5236 160/300 Write block 790 len 40960 data: 2 2 2 2 2 2 3 4 4 6
5237 161/300 Read block 2075 len 28672
5238 162/300 Read block 1097 len 20480
5239 163/300 Write block 2458 len 28672 data: 2 2 2 2 3 3 3
5240 164/300 Flush
5241 165/300 Flush
5242 166/300 Read block 3469 len 4096
5243 167/300 Read block 977 len 32768
5244 168/300 Read block 1063 len 12288
5245 169/300 Write block 476 len 32768 data: 2 2 2 3 4 4 4 4
5246 170/300 Read block 2429 len 32768
5247 171/300 Read block 16 len 4096
5248 172/300 Read block 3496 len 28672
5249 173/300 Write block 343 len 32768 data: 4 4 3 3 3 4 4 4
5250 174/300 Read block 80 len 16384
5251 175/300 Read block 841 len 20480
5252 176/300 Write block 2556 len 16384 data: 2 2 3 4
5253 177/300 Write block 1482 len 36864 data: 2 2 2 2 3 3 3 3 4
5254 178/300 Write block 3910 len 36864 data: 3 3 3 4 3 4 4 3 3
5255 179/300 Read block 1938 len 32768
5256 180/300 Flush
5257 181/300 Read block 3097 len 24576
5258 182/300 Read block 825 len 16384
5259 183/300 Read block 4200 len 8192
5260 184/300 Flush
5261 185/300 Read block 4987 len 8192
5262 186/300 Write block 4732 len 28672 data: 2 2 2 2 2 2 2
5263 187/300 Flush
5264 188/300 Flush
5265 189/300 Read block 3213 len 12288
5266 190/300 Read block 2360 len 28672
5267 191/300 Write block 2936 len 24576 data: 2 2 3 3 3 3
5268 192/300 Read block 1215 len 20480
5269 193/300 Read block 3769 len 28672
5270 194/300 Write block 2222 len 4096 data: 2
5271 195/300 Flush
5272 196/300 Write block 925 len 24576 data: 3 3 3 3 3 3
5273 197/300 Read block 3910 len 20480
5274 198/300 Read block 3503 len 28672
5275 199/300 Read block 2262 len 20480
5276 200/300 Read block 1179 len 24576
5277 201/300 Write block 934 len 24576 data: 2 2 2 2 2 2
5278 202/300 Write block 3818 len 8192 data: 3 4
5279 203/300 Write block 4944 len 4096 data: 4
5280 204/300 Flush
5281 205/300 Read block 2926 len 12288
5282 206/300 Write block 2152 len 36864 data: 2 2 2 2 2 2 2 3 3
5283 207/300 Read block 3638 len 28672
5284 208/300 Write block 4349 len 32768 data: 2 2 2 2 2 2 2 2
5285 209/300 Read block 755 len 28672
5286 210/300 Flush
5287 211/300 Read block 3181 len 24576
5288 212/300 Read block 178 len 40960
5289 213/300 Write block 2812 len 12288 data: 3 5 5
5290 214/300 Flush
5291 215/300 Write block 924 len 28672 data: 2 4 4 4 4 4 4
5292 216/300 Write block 2567 len 4096 data: 5
5293 217/300 Read block 2437 len 24576
5294 218/300 Read block 556 len 36864
5295 219/300 Write block 2624 len 16384 data: 5 5 5 3
5296 220/300 Read block 3289 len 28672
5297 221/300 Write block 4415 len 20480 data: 2 2 2 2 2
5298 222/300 Read block 466 len 16384
5299 223/300 Read block 3987 len 4096
5300 224/300 Write block 449 len 32768 data: 4 3 3 3 3 3 4 4
5301 225/300 Write block 4026 len 20480 data: 2 2 2 2 2
5302 226/300 Read block 3283 len 20480
5303 227/300 Read block 3548 len 8192
5304 228/300 Write block 2989 len 40960 data: 2 2 2 2 2 3 3 2 2 3
5305 229/300 Read block 4828 len 16384
5306 230/300 Flush
5307 231/300 Read block 3018 len 4096
5308 232/300 Read block 909 len 20480
5309 233/300 Write block 3883 len 36864 data: 2 2 2 2 2 2 2 2 2
5310 234/300 Read block 1185 len 16384
5311 235/300 Read block 4038 len 24576
5312 236/300 Write block 3987 len 28672 data: 2 2 2 2 3 4 4
5313 237/300 Read block 2213 len 32768
5314 238/300 Read block 3800 len 20480
5315 239/300 Read block 3039 len 12288
5316 240/300 Flush
5317 241/300 Write block 2072 len 12288 data: 2 2 2
5318 242/300 Write block 4770 len 32768 data: 4 4 4 4 3 3 2 3
5319 243/300 Read block 4446 len 8192
5320 244/300 Write block 1124 len 36864 data: 3 3 3 3 3 3 3 3 2
5321 245/300 Read block 4222 len 20480
5322 246/300 Read block 992 len 20480
5323 247/300 Read block 647 len 24576
5324 248/300 Write block 3958 len 20480 data: 3 4 5 4 4
5325 249/300 Write block 3682 len 16384 data: 4 4 4 3
5326 250/300 Write block 1352 len 24576 data: 3 4 3 4 4 4
5327 251/300 Read block 1111 len 16384
5328 252/300 Write block 65 len 32768 data: 2 2 2 2 2 2 2 2
5329 253/300 Read block 2626 len 36864
5330 254/300 Flush
5331 255/300 Read block 4797 len 24576
5332 256/300 Flush
5333 257/300 Write block 4157 len 40960 data: 3 3 4 3 3 3 3 3 2 2
5334 258/300 Flush
5335 259/300 Write block 3414 len 40960 data: 2 2 3 3 3 4 5 5 4 4
5336 260/300 Write block 114 len 4096 data: 3
5337 261/300 Write block 552 len 36864 data: 3 3 2 2 2 2 2 2 2
5338 262/300 Write block 919 len 24576 data: 2 2 2 2 2 3
5339 263/300 Read block 2666 len 20480
5340 264/300 Write block 2542 len 40960 data: 3 2 2 2 2 2 2 2 2 2
5341 265/300 Read block 2758 len 20480
5342 266/300 Read block 4564 len 12288
5343 267/300 Write block 50 len 28672 data: 3 3 3 3 2 2 2
5344 268/300 Read block 1104 len 12288
5345 269/300 Read block 4773 len 20480
5346 270/300 Read block 4228 len 12288
5347 271/300 Read block 4234 len 4096
5348 272/300 Write block 3424 len 32768 data: 4 3 3 3 2 2 2 2
5349 273/300 Write block 3637 len 4096 data: 4
5350 274/300 Write block 1896 len 20480 data: 4 4 4 3 4
5351 275/300 Write block 1552 len 28672 data: 2 2 2 2 2 2 2
5352 276/300 Read block 4131 len 32768
5353 277/300 Write block 697 len 40960 data: 4 4 4 3 4 3 3 4 4 3
5354 278/300 Read block 2447 len 24576
5355 279/300 Write block 1925 len 16384 data: 2 2 2 2
5356 280/300 Read block 3830 len 32768
5357 281/300 Write block 3190 len 20480 data: 3 3 2 3 3
5358 282/300 Read block 464 len 28672
5359 283/300 Read block 1917 len 16384
5360 284/300 Read block 4693 len 20480
5361 285/300 Read block 1630 len 36864
5362 286/300 Read block 4316 len 32768
5363 287/300 Read block 2122 len 40960
5364 288/300 Read block 1880 len 36864
5365 289/300 Read block 1422 len 40960
5366 290/300 Flush
5367 291/300 Write block 2547 len 12288 data: 3 3 3
5368 292/300 Flush
5369 293/300 Read block 931 len 32768
5370 294/300 Read block 3487 len 4096
5371 295/300 Read block 3021 len 8192
5372 296/300 Write block 302 len 28672 data: 2 2 2 2 2 3 3
5373 297/300 Read block 3830 len 24576
5374 298/300 Write block 1081 len 24576 data: 2 2 2 2 2 2
5375 299/300 Write block 2076 len 32768 data: 2 2 2 2 2 2 2 2
5376 300/300 Write block 2548 len 36864 data: 4 4 3 3 2 2 2 2 3
5377 301/300 Read block 3688 len 28672
5378 Main task has work to do, go find it
5379 got dsc Start(2)
5380 start 2
5381 Jan 14 16:11:45.391 INFO request completed, latency_us: 267, response_code: 204, uri: /start/cid/2, method: POST, req_id: ccb4b2a9-bc00-4fca-a72d-d4042a3ba1ca, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
5382 Replay: started 2, returned:Ok(())
5383 ----------------------------------------------------------------
5384 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
5385 States: Active Active Offline
5386 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5387 2508 Acked Read 1 Done Done Sent false
5388 2509 Acked Write 1 Done Done Sent false
5389 2510 Acked Write 2 Done Done Sent false
5390 2511 Acked Read 1 Done Done Sent false
5391 2512 Acked Write 7 Done Done Sent false
5392 2513 Acked Read 9 Done Done Sent false
5393 2514 Acked Flush 0 Done Done Sent false
5394 2515 Acked Read 8 Done Done Sent false
5395 2516 Acked Write 10 Done Done Sent false
5396 2517 Acked Write 4 Done Done Sent false
5397 2518 Acked Write 10 Done Done Sent false
5398 2519 Acked Flush 0 Done Done Sent false
5399 2520 Acked Flush 0 Done Done Sent false
5400 2521 Acked Write 10 Done Done Sent false
5401 2522 Acked Write 6 Done Done Sent false
5402 2523 Acked Read 10 Done Done Sent false
5403 2524 Acked Write 8 Done Done Sent false
5404 2525 Acked Read 5 Done Done Sent false
5405 2526 Acked Write 8 Done Done Sent false
5406 2527 Acked Read 9 Done Done Sent false
5407 2528 Acked Write 1 Done Done Sent false
5408 2529 Acked Write 8 Done Done Sent false
5409 2530 Acked Write 5 Done Done Sent false
5410 2531 Acked Write 1 Done Done Sent false
5411 2532 Acked Read 8 Done Done Sent false
5412 2533 Acked Read 5 Done Done Sent false
5413 2534 Acked Write 9 Done Done Sent false
5414 2535 Acked Read 2 Done Done Sent false
5415 2536 Acked Read 3 Done Done Sent false
5416 2537 Acked Read 5 Done Done Sent false
5417 2538 Acked Read 8 Done Done Sent false
5418 2539 Acked Write 8 Done Done Sent false
5419 2540 Acked Read 7 Done Done Sent false
5420 2541 Acked Write 9 Done Done Sent false
5421 2542 Acked Read 8 Done Done Sent false
5422 2543 Acked Read 1 Done Done Sent false
5423 2544 Acked Write 1 Done Done Sent false
5424 2545 Acked Read 6 Done Done Sent false
5425 2546 Acked Write 10 Done Done Sent false
5426 2547 Acked Write 3 Done Done Sent false
5427 2548 Acked Read 2 Done Done Sent false
5428 2549 Acked Write 8 Done Done Sent false
5429 2550 Acked Write 9 Done Done Sent false
5430 2551 Acked Write 5 Done Done Sent false
5431 2552 Acked Write 1 Done Done Sent false
5432 2553 Acked Write 4 Done Done Sent false
5433 2554 Acked Write 6 Done Done Sent false
5434 2555 Acked Read 1 Done Done Sent false
5435 2556 Acked Write 10 Done Done Sent false
5436 2557 Acked Write 1 Done Done Sent false
5437 2558 Acked Write 8 Done Done Sent false
5438 2559 Acked Read 10 Done Done Sent false
5439 2560 Acked Read 9 Done Done Sent false
5440 2561 Acked Read 2 Done Done Sent false
5441 2562 Acked Flush 0 Done Done Sent false
5442 2563 Acked Read 8 Done Done Sent false
5443 2564 Acked Flush 0 Done Done Sent false
5444 2565 Acked Read 6 Done Done Sent false
5445 2566 Acked Read 10 Done Done Sent false
5446 2567 Acked Read 3 Done Done Sent false
5447 2568 Acked Flush 0 Done Done Sent false
5448 2569 Acked Read 10 Done Done Sent false
5449 2570 Acked Flush 0 Done Done Sent false
5450 2571 Acked Read 7 Done Done Sent false
5451 2572 Acked Write 8 Done Done Sent false
5452 2573 Acked Write 4 Done Done Sent false
5453 2574 Acked Read 9 Done Done Sent false
5454 2575 Acked Write 7 Done Done Sent false
5455 2576 Acked Write 2 Done Done Sent false
5456 2577 Acked Read 9 Done Done Sent false
5457 2578 Acked Read 4 Done Done Sent false
5458 2579 Acked Read 2 Done Done Sent false
5459 2580 Acked Read 3 Done Done Sent false
5460 2581 Acked Read 8 Done Done Sent false
5461 2582 Acked Read 6 Done Done Sent false
5462 2583 Acked Read 9 Done Done Sent false
5463 2584 Acked Flush 0 Done Done Sent false
5464 2585 Acked Write 2 Done Done Sent false
5465 2586 Acked Write 10 Done Done Sent false
5466 2587 Acked Read 7 Done Done Sent false
5467 2588 Acked Read 9 Done Done Sent false
5468 2589 Acked Read 4 Done Done Sent false
5469 2590 Acked Read 10 Done Done Sent false
5470 2591 Acked Flush 0 Done Done Sent false
5471 2592 Acked Write 2 Done Done Sent false
5472 2593 Acked Read 9 Done Done Sent false
5473 2594 Acked Read 4 Done Done Sent false
5474 2595 Acked Read 10 Done Done Sent false
5475 2596 Acked Write 3 Done Done Sent false
5476 2597 Acked Write 10 Done Done Sent false
5477 2598 Acked Write 7 Done Done Sent false
5478 2599 Acked Read 6 Done Done Sent false
5479 2600 Acked Flush 0 Done Done Sent false
5480 2601 Acked Read 6 Done Done Sent false
5481 2602 Acked Read 3 Done Done Sent false
5482 2603 Acked Read 3 Done Done Sent false
5483 2604 Acked Flush 0 Done Done Sent false
5484 2605 Acked Write 2 Done Done Sent false
5485 2606 Acked Read 3 Done Done Sent false
5486 2607 Acked Read 6 Done Done Sent false
5487 2608 Acked Write 10 Done Done Sent false
5488 2609 Acked Read 7 Done Done Sent false
5489 2610 Acked Write 9 Done Done Sent false
5490 2611 Acked Flush 0 Done Done Sent false
5491 2612 Acked Write 7 Done Done Sent false
5492 2613 Acked Read 3 Done Done Sent false
5493 2614 Acked Write 4 Done Done Sent false
5494 2615 Acked Read 4 Done Done Sent false
5495 2616 Acked Read 9 Done Done Sent false
5496 2617 Acked Flush 0 Done Done Sent false
5497 2618 Acked Write 8 Done Done Sent false
5498 2619 Acked Write 10 Done Done Sent false
5499 2620 Acked Write 1 Done Done Sent false
5500 2621 Acked Write 10 Done Done Sent false
5501 2622 Acked Read 2 Done Done Sent false
5502 2623 Acked Read 4 Done Done Sent false
5503 2624 Acked Read 2 Done Done Sent false
5504 2625 Acked Write 5 Done Done Sent false
5505 2626 Acked Flush 0 Done Done Sent false
5506 2627 Acked Write 6 Done Done Sent false
5507 2628 Acked Read 8 Done Done Sent false
5508 2629 Acked Read 2 Done Done Sent false
5509 2630 Acked Read 7 Done Done Sent false
5510 2631 Acked Write 3 Done Done Sent false
5511 2632 Acked Write 5 Done Done Sent false
5512 2633 Acked Write 9 Done Done Sent false
5513 2634 Acked Read 4 Done Done Sent false
5514 2635 Acked Read 9 Done Done Sent false
5515 2636 Acked Flush 0 Done Done Sent false
5516 2637 Acked Write 5 Done Done Sent false
5517 2638 Acked Write 7 Done Done Sent false
5518 2639 Acked Read 5 Done Done Sent false
5519 2640 Acked Read 1 Done Done Sent false
5520 2641 Acked Read 9 Done Done Sent false
5521 2642 Acked Read 6 Done Done Sent false
5522 2643 Acked Read 9 Done Done Sent false
5523 2644 Acked Read 3 Done Done Sent false
5524 2645 Acked Write 3 Done Done Sent false
5525 2646 Acked Write 9 Done Done Sent false
5526 2647 Acked Write 6 Done Done Sent false
5527 2648 Acked Read 2 Done Done Sent false
5528 2649 Acked Write 5 Done Done Sent false
5529 2650 Acked Write 3 Done Done Sent false
5530 2651 Acked Read 3 Done Done Sent false
5531 2652 Acked Read 7 Done Done Sent false
5532 2653 Acked Read 2 Done Done Sent false
5533 2654 Acked Write 7 Done Done Sent false
5534 2655 Acked Read 10 Done Done Sent false
5535 2656 Acked Read 3 Done Done Sent false
5536 2657 Acked Write 5 Done Done Sent false
5537 2658 Acked Write 3 Done Done Sent false
5538 2659 Acked Read 2 Done Done Sent false
5539 2660 Acked Read 4 Done Done Sent false
5540 2661 Acked Flush 0 Done Done Sent false
5541 2662 Acked Write 10 Done Done Sent false
5542 2663 Acked Read 10 Done Done Sent false
5543 2664 Acked Read 4 Done Done Sent false
5544 2665 Acked Write 5 Done Done Sent false
5545 2666 Acked Read 6 Done Done Sent false
5546 2667 Acked Write 10 Done Done Sent false
5547 2668 Acked Read 7 Done Done Sent false
5548 2669 Acked Read 5 Done Done Sent false
5549 2670 Acked Write 7 Done Done Sent false
5550 2671 Acked Flush 0 Done Done Sent false
5551 2672 Acked Flush 0 Done Done Sent false
5552 2673 Acked Read 1 Done Done Sent false
5553 2674 Acked Read 8 Done Done Sent false
5554 2675 Acked Read 3 Done Done Sent false
5555 2676 Acked Write 8 Done Done Sent false
5556 2677 Acked Read 8 Done Done Sent false
5557 2678 Acked Read 1 Done Done Sent false
5558 2679 Acked Read 7 Done Done Sent false
5559 2680 Acked Write 8 Done Done Sent false
5560 2681 Acked Read 4 Done Done Sent false
5561 2682 Acked Read 5 Done Done Sent false
5562 2683 Acked Write 4 Done Done Sent false
5563 2684 Acked Write 9 Done Done Sent false
5564 2685 Acked Write 9 Done Done Sent false
5565 2686 Acked Read 8 Done Done Sent false
5566 2687 Acked Flush 0 Done Done Sent false
5567 2688 Acked Read 6 Done Done Sent false
5568 2689 Acked Read 4 Done Done Sent false
5569 2690 Acked Read 2 Done Done Sent false
5570 2691 Acked Flush 0 Done Done Sent false
5571 2692 Acked Read 2 Done Done Sent false
5572 2693 Acked Write 7 Done Done Sent false
5573 2694 Acked Flush 0 Done Done Sent false
5574 2695 Acked Flush 0 Done Done Sent false
5575 2696 Acked Read 3 Done Done Sent false
5576 2697 Acked Read 7 Done Done Sent false
5577 2698 Acked Write 6 Done Done Sent false
5578 2699 Acked Read 5 Done Done Sent false
5579 2700 Acked Read 7 Done Done Sent false
5580 2701 Acked Write 1 Done Done Sent false
5581 2702 Acked Flush 0 Done Done Sent false
5582 2703 Acked Write 6 Done Done Sent false
5583 2704 Acked Read 5 Done Done Sent false
5584 2705 Acked Read 7 Done Done Sent false
5585 2706 Acked Read 5 Done Done Sent false
5586 2707 Acked Read 6 Done Done Sent false
5587 2708 Acked Write 6 Done Done Sent false
5588 2709 Acked Write 2 Done Done Sent false
5589 2710 Acked Write 1 Done Done Sent false
5590 2711 Acked Flush 0 Done Done Sent false
5591 2712 Acked Read 3 Done Done Sent false
5592 2713 Acked Write 9 Done Done Sent false
5593 2714 Acked Read 7 Done Done Sent false
5594 2715 Acked Write 8 Done Done Sent false
5595 2716 Acked Read 7 Done Done Sent false
5596 2717 Acked Flush 0 Done Done Sent false
5597 2718 Acked Read 6 Done Done Sent false
5598 2719 Acked Read 10 Done Done Sent false
5599 2720 Acked Write 3 Done Done Sent false
5600 2721 Acked Flush 0 Done Done Sent false
5601 2722 Acked Write 7 Done Done Sent false
5602 2723 Acked Write 1 Done Done Sent false
5603 2724 Acked Read 6 Done Done Sent false
5604 2725 Acked Read 9 Done Done Sent false
5605 2726 Acked Write 4 Done Done Sent false
5606 2727 Acked Read 7 Done Done Sent false
5607 2728 Acked Write 5 Done Done Sent false
5608 2729 Acked Read 4 Done Done Sent false
5609 2730 Acked Read 1 Done Done Sent false
5610 2731 Acked Write 8 Done Done Sent false
5611 2732 Acked Write 5 Done Done Sent false
5612 2733 Acked Read 5 Done Done Sent false
5613 2734 Acked Read 2 Done Done Sent false
5614 2735 Acked Write 10 Done Done Sent false
5615 2736 Acked Read 4 Done Done Sent false
5616 2737 Acked Flush 0 Done Done Sent false
5617 2738 Acked Read 1 Done Done Sent false
5618 2739 Acked Read 5 Done Done Sent false
5619 2740 Acked Write 9 Done Done Sent false
5620 2741 Acked Read 4 Done Done Sent false
5621 2742 Acked Read 6 Done Done Sent false
5622 2743 Acked Write 7 Done Done Sent false
5623 2744 Acked Read 8 Done Done Sent false
5624 2745 Acked Read 5 Done Done Sent false
5625 2746 Acked Read 3 Done Done Sent false
5626 2747 Acked Flush 0 Done Done Sent false
5627 2748 Acked Write 3 Done Done Sent false
5628 2749 Acked Write 8 Done Done Sent false
5629 2750 Acked Read 2 Done Done Sent false
5630 2751 Acked Write 9 Done Done Sent false
5631 2752 Acked Read 5 Done Done Sent false
5632 2753 Acked Read 5 Done Done Sent false
5633 2754 Acked Read 6 Done Done Sent false
5634 2755 Acked Write 5 Done Done Sent false
5635 2756 Acked Write 4 Done Done Sent false
5636 2757 Acked Write 6 Done Done Sent false
5637 2758 Acked Read 4 Done Done Sent false
5638 2759 Acked Write 8 Done Done Sent false
5639 2760 Acked Read 9 Done Done Sent false
5640 2761 Acked Flush 0 Done Done Sent false
5641 2762 Acked Read 6 Done Done Sent false
5642 2763 Acked Flush 0 Done Done Sent false
5643 2764 Acked Write 10 Done Done Sent false
5644 2765 Acked Flush 0 Done Done Sent false
5645 2766 Acked Write 10 Done Done Sent false
5646 2767 Acked Write 1 Done Done Sent false
5647 2768 Acked Write 9 Done Done Sent false
5648 2769 Acked Write 6 Done Done Sent false
5649 2770 Acked Read 5 Done Done Sent false
5650 2771 Acked Write 10 Done Done Sent false
5651 2772 Acked Read 5 Done Done Sent false
5652 2773 Acked Read 3 Done Done Sent false
5653 2774 Acked Write 7 Done Done Sent false
5654 2775 Acked Read 3 Done Done Sent false
5655 2776 Acked Read 5 Done Done Sent false
5656 2777 Acked Read 3 Done Done Sent false
5657 2778 Acked Read 1 Done Done Sent false
5658 2779 Acked Write 8 Done Done Sent false
5659 2780 Acked Write 1 Done Done Sent false
5660 2781 Acked Write 5 Done Done Sent false
5661 2782 Acked Write 7 Done Done Sent false
5662 2783 Acked Read 8 Done Done Sent false
5663 2784 Acked Write 10 Done Done Sent false
5664 2785 Acked Read 6 Done Done Sent false
5665 2786 Acked Write 4 Done Done Sent false
5666 2787 Acked Read 8 Done Done Sent false
5667 2788 Acked Write 5 Done Done Sent false
5668 2789 Acked Read 7 Done Done Sent false
5669 2790 Acked Read 4 Done Done Sent false
5670 2791 Acked Read 5 Done Done Sent false
5671 2792 Acked Read 9 Done Done Sent false
5672 2793 Acked Read 8 Done Done Sent false
5673 2794 Acked Read 10 Done Done Sent false
5674 2795 Acked Read 9 Done Done Sent false
5675 2796 Acked Read 10 Done Done Sent false
5676 2797 Acked Flush 0 Done Done Sent false
5677 2798 Acked Write 3 Done Done Sent false
5678 2799 Acked Flush 0 Done Done Sent false
5679 2800 Acked Read 8 Done Done Sent false
5680 2801 Acked Read 1 Done Done Sent false
5681 2802 Acked Read 2 Done Done Sent false
5682 2803 Acked Write 7 Done Done Sent false
5683 2804 Acked Read 6 Done Done Sent false
5684 2805 Acked Write 6 Done Done Sent false
5685 2806 Acked Write 8 Done Done Sent false
5686 2807 Acked Write 9 Done Done Sent false
5687 2808 Acked Read 7 Done Done Sent false
5688 STATES DS:0 DS:1 DS:2 TOTAL
5689 Sent 0 0 301 301
5690 Done 301 301 0 602
5691 Skipped 0 0 0 0
5692 Error 0 0 0 0
5693 Last Flush: 2799 2799 2507
5694 Downstairs last five completed: 2507 2506 2505 2504 2503
5695 Upstairs last five completed: 2808 2807 2806 2805 2804
5696 CLIENT: Up:0 ds:301 act:2
5697 [8830] got start action
5698 [8830] I am going to restart
5699 Starting downstairs at port 8830
5700 Make output file at "/tmp/dsc/downstairs-8830.txt"
5701 [8830][2] reports Starting
5702 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1208)
5703 [8830][2] reports Running
5704 ----------------------------------------------------------------
5705 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
5706 States: Active Active Offline
5707 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5708 2508 Acked Read 1 Done Done Sent false
5709 2509 Acked Write 1 Done Done Sent false
5710 2510 Acked Write 2 Done Done Sent false
5711 2511 Acked Read 1 Done Done Sent false
5712 2512 Acked Write 7 Done Done Sent false
5713 2513 Acked Read 9 Done Done Sent false
5714 2514 Acked Flush 0 Done Done Sent false
5715 2515 Acked Read 8 Done Done Sent false
5716 2516 Acked Write 10 Done Done Sent false
5717 2517 Acked Write 4 Done Done Sent false
5718 2518 Acked Write 10 Done Done Sent false
5719 2519 Acked Flush 0 Done Done Sent false
5720 2520 Acked Flush 0 Done Done Sent false
5721 2521 Acked Write 10 Done Done Sent false
5722 2522 Acked Write 6 Done Done Sent false
5723 2523 Acked Read 10 Done Done Sent false
5724 2524 Acked Write 8 Done Done Sent false
5725 2525 Acked Read 5 Done Done Sent false
5726 2526 Acked Write 8 Done Done Sent false
5727 2527 Acked Read 9 Done Done Sent false
5728 2528 Acked Write 1 Done Done Sent false
5729 2529 Acked Write 8 Done Done Sent false
5730 2530 Acked Write 5 Done Done Sent false
5731 2531 Acked Write 1 Done Done Sent false
5732 2532 Acked Read 8 Done Done Sent false
5733 2533 Acked Read 5 Done Done Sent false
5734 2534 Acked Write 9 Done Done Sent false
5735 2535 Acked Read 2 Done Done Sent false
5736 2536 Acked Read 3 Done Done Sent false
5737 2537 Acked Read 5 Done Done Sent false
5738 2538 Acked Read 8 Done Done Sent false
5739 2539 Acked Write 8 Done Done Sent false
5740 2540 Acked Read 7 Done Done Sent false
5741 2541 Acked Write 9 Done Done Sent false
5742 2542 Acked Read 8 Done Done Sent false
5743 2543 Acked Read 1 Done Done Sent false
5744 2544 Acked Write 1 Done Done Sent false
5745 2545 Acked Read 6 Done Done Sent false
5746 2546 Acked Write 10 Done Done Sent false
5747 2547 Acked Write 3 Done Done Sent false
5748 2548 Acked Read 2 Done Done Sent false
5749 2549 Acked Write 8 Done Done Sent false
5750 2550 Acked Write 9 Done Done Sent false
5751 2551 Acked Write 5 Done Done Sent false
5752 2552 Acked Write 1 Done Done Sent false
5753 2553 Acked Write 4 Done Done Sent false
5754 2554 Acked Write 6 Done Done Sent false
5755 2555 Acked Read 1 Done Done Sent false
5756 2556 Acked Write 10 Done Done Sent false
5757 2557 Acked Write 1 Done Done Sent false
5758 2558 Acked Write 8 Done Done Sent false
5759 2559 Acked Read 10 Done Done Sent false
5760 2560 Acked Read 9 Done Done Sent false
5761 2561 Acked Read 2 Done Done Sent false
5762 2562 Acked Flush 0 Done Done Sent false
5763 2563 Acked Read 8 Done Done Sent false
5764 2564 Acked Flush 0 Done Done Sent false
5765 2565 Acked Read 6 Done Done Sent false
5766 2566 Acked Read 10 Done Done Sent false
5767 2567 Acked Read 3 Done Done Sent false
5768 2568 Acked Flush 0 Done Done Sent false
5769 2569 Acked Read 10 Done Done Sent false
5770 2570 Acked Flush 0 Done Done Sent false
5771 2571 Acked Read 7 Done Done Sent false
5772 2572 Acked Write 8 Done Done Sent false
5773 2573 Acked Write 4 Done Done Sent false
5774 2574 Acked Read 9 Done Done Sent false
5775 2575 Acked Write 7 Done Done Sent false
5776 2576 Acked Write 2 Done Done Sent false
5777 2577 Acked Read 9 Done Done Sent false
5778 2578 Acked Read 4 Done Done Sent false
5779 2579 Acked Read 2 Done Done Sent false
5780 2580 Acked Read 3 Done Done Sent false
5781 2581 Acked Read 8 Done Done Sent false
5782 2582 Acked Read 6 Done Done Sent false
5783 2583 Acked Read 9 Done Done Sent false
5784 2584 Acked Flush 0 Done Done Sent false
5785 2585 Acked Write 2 Done Done Sent false
5786 2586 Acked Write 10 Done Done Sent false
5787 2587 Acked Read 7 Done Done Sent false
5788 2588 Acked Read 9 Done Done Sent false
5789 2589 Acked Read 4 Done Done Sent false
5790 2590 Acked Read 10 Done Done Sent false
5791 2591 Acked Flush 0 Done Done Sent false
5792 2592 Acked Write 2 Done Done Sent false
5793 2593 Acked Read 9 Done Done Sent false
5794 2594 Acked Read 4 Done Done Sent false
5795 2595 Acked Read 10 Done Done Sent false
5796 2596 Acked Write 3 Done Done Sent false
5797 2597 Acked Write 10 Done Done Sent false
5798 2598 Acked Write 7 Done Done Sent false
5799 2599 Acked Read 6 Done Done Sent false
5800 2600 Acked Flush 0 Done Done Sent false
5801 2601 Acked Read 6 Done Done Sent false
5802 2602 Acked Read 3 Done Done Sent false
5803 2603 Acked Read 3 Done Done Sent false
5804 2604 Acked Flush 0 Done Done Sent false
5805 2605 Acked Write 2 Done Done Sent false
5806 2606 Acked Read 3 Done Done Sent false
5807 2607 Acked Read 6 Done Done Sent false
5808 2608 Acked Write 10 Done Done Sent false
5809 2609 Acked Read 7 Done Done Sent false
5810 2610 Acked Write 9 Done Done Sent false
5811 2611 Acked Flush 0 Done Done Sent false
5812 2612 Acked Write 7 Done Done Sent false
5813 2613 Acked Read 3 Done Done Sent false
5814 2614 Acked Write 4 Done Done Sent false
5815 2615 Acked Read 4 Done Done Sent false
5816 2616 Acked Read 9 Done Done Sent false
5817 2617 Acked Flush 0 Done Done Sent false
5818 2618 Acked Write 8 Done Done Sent false
5819 2619 Acked Write 10 Done Done Sent false
5820 2620 Acked Write 1 Done Done Sent false
5821 2621 Acked Write 10 Done Done Sent false
5822 2622 Acked Read 2 Done Done Sent false
5823 2623 Acked Read 4 Done Done Sent false
5824 2624 Acked Read 2 Done Done Sent false
5825 2625 Acked Write 5 Done Done Sent false
5826 2626 Acked Flush 0 Done Done Sent false
5827 2627 Acked Write 6 Done Done Sent false
5828 2628 Acked Read 8 Done Done Sent false
5829 2629 Acked Read 2 Done Done Sent false
5830 2630 Acked Read 7 Done Done Sent false
5831 2631 Acked Write 3 Done Done Sent false
5832 2632 Acked Write 5 Done Done Sent false
5833 2633 Acked Write 9 Done Done Sent false
5834 2634 Acked Read 4 Done Done Sent false
5835 2635 Acked Read 9 Done Done Sent false
5836 2636 Acked Flush 0 Done Done Sent false
5837 2637 Acked Write 5 Done Done Sent false
5838 2638 Acked Write 7 Done Done Sent false
5839 2639 Acked Read 5 Done Done Sent false
5840 2640 Acked Read 1 Done Done Sent false
5841 2641 Acked Read 9 Done Done Sent false
5842 2642 Acked Read 6 Done Done Sent false
5843 2643 Acked Read 9 Done Done Sent false
5844 2644 Acked Read 3 Done Done Sent false
5845 2645 Acked Write 3 Done Done Sent false
5846 2646 Acked Write 9 Done Done Sent false
5847 2647 Acked Write 6 Done Done Sent false
5848 2648 Acked Read 2 Done Done Sent false
5849 2649 Acked Write 5 Done Done Sent false
5850 2650 Acked Write 3 Done Done Sent false
5851 2651 Acked Read 3 Done Done Sent false
5852 2652 Acked Read 7 Done Done Sent false
5853 2653 Acked Read 2 Done Done Sent false
5854 2654 Acked Write 7 Done Done Sent false
5855 2655 Acked Read 10 Done Done Sent false
5856 2656 Acked Read 3 Done Done Sent false
5857 2657 Acked Write 5 Done Done Sent false
5858 2658 Acked Write 3 Done Done Sent false
5859 2659 Acked Read 2 Done Done Sent false
5860 2660 Acked Read 4 Done Done Sent false
5861 2661 Acked Flush 0 Done Done Sent false
5862 2662 Acked Write 10 Done Done Sent false
5863 2663 Acked Read 10 Done Done Sent false
5864 2664 Acked Read 4 Done Done Sent false
5865 2665 Acked Write 5 Done Done Sent false
5866 2666 Acked Read 6 Done Done Sent false
5867 2667 Acked Write 10 Done Done Sent false
5868 2668 Acked Read 7 Done Done Sent false
5869 2669 Acked Read 5 Done Done Sent false
5870 2670 Acked Write 7 Done Done Sent false
5871 2671 Acked Flush 0 Done Done Sent false
5872 2672 Acked Flush 0 Done Done Sent false
5873 2673 Acked Read 1 Done Done Sent false
5874 2674 Acked Read 8 Done Done Sent false
5875 2675 Acked Read 3 Done Done Sent false
5876 2676 Acked Write 8 Done Done Sent false
5877 2677 Acked Read 8 Done Done Sent false
5878 2678 Acked Read 1 Done Done Sent false
5879 2679 Acked Read 7 Done Done Sent false
5880 2680 Acked Write 8 Done Done Sent false
5881 2681 Acked Read 4 Done Done Sent false
5882 2682 Acked Read 5 Done Done Sent false
5883 2683 Acked Write 4 Done Done Sent false
5884 2684 Acked Write 9 Done Done Sent false
5885 2685 Acked Write 9 Done Done Sent false
5886 2686 Acked Read 8 Done Done Sent false
5887 2687 Acked Flush 0 Done Done Sent false
5888 2688 Acked Read 6 Done Done Sent false
5889 2689 Acked Read 4 Done Done Sent false
5890 2690 Acked Read 2 Done Done Sent false
5891 2691 Acked Flush 0 Done Done Sent false
5892 2692 Acked Read 2 Done Done Sent false
5893 2693 Acked Write 7 Done Done Sent false
5894 2694 Acked Flush 0 Done Done Sent false
5895 2695 Acked Flush 0 Done Done Sent false
5896 2696 Acked Read 3 Done Done Sent false
5897 2697 Acked Read 7 Done Done Sent false
5898 2698 Acked Write 6 Done Done Sent false
5899 2699 Acked Read 5 Done Done Sent false
5900 2700 Acked Read 7 Done Done Sent false
5901 2701 Acked Write 1 Done Done Sent false
5902 2702 Acked Flush 0 Done Done Sent false
5903 2703 Acked Write 6 Done Done Sent false
5904 2704 Acked Read 5 Done Done Sent false
5905 2705 Acked Read 7 Done Done Sent false
5906 2706 Acked Read 5 Done Done Sent false
5907 2707 Acked Read 6 Done Done Sent false
5908 2708 Acked Write 6 Done Done Sent false
5909 2709 Acked Write 2 Done Done Sent false
5910 2710 Acked Write 1 Done Done Sent false
5911 2711 Acked Flush 0 Done Done Sent false
5912 2712 Acked Read 3 Done Done Sent false
5913 2713 Acked Write 9 Done Done Sent false
5914 2714 Acked Read 7 Done Done Sent false
5915 2715 Acked Write 8 Done Done Sent false
5916 2716 Acked Read 7 Done Done Sent false
5917 2717 Acked Flush 0 Done Done Sent false
5918 2718 Acked Read 6 Done Done Sent false
5919 2719 Acked Read 10 Done Done Sent false
5920 2720 Acked Write 3 Done Done Sent false
5921 2721 Acked Flush 0 Done Done Sent false
5922 2722 Acked Write 7 Done Done Sent false
5923 2723 Acked Write 1 Done Done Sent false
5924 2724 Acked Read 6 Done Done Sent false
5925 2725 Acked Read 9 Done Done Sent false
5926 2726 Acked Write 4 Done Done Sent false
5927 2727 Acked Read 7 Done Done Sent false
5928 2728 Acked Write 5 Done Done Sent false
5929 2729 Acked Read 4 Done Done Sent false
5930 2730 Acked Read 1 Done Done Sent false
5931 2731 Acked Write 8 Done Done Sent false
5932 2732 Acked Write 5 Done Done Sent false
5933 2733 Acked Read 5 Done Done Sent false
5934 2734 Acked Read 2 Done Done Sent false
5935 2735 Acked Write 10 Done Done Sent false
5936 2736 Acked Read 4 Done Done Sent false
5937 2737 Acked Flush 0 Done Done Sent false
5938 2738 Acked Read 1 Done Done Sent false
5939 2739 Acked Read 5 Done Done Sent false
5940 2740 Acked Write 9 Done Done Sent false
5941 2741 Acked Read 4 Done Done Sent false
5942 2742 Acked Read 6 Done Done Sent false
5943 2743 Acked Write 7 Done Done Sent false
5944 2744 Acked Read 8 Done Done Sent false
5945 2745 Acked Read 5 Done Done Sent false
5946 2746 Acked Read 3 Done Done Sent false
5947 2747 Acked Flush 0 Done Done Sent false
5948 2748 Acked Write 3 Done Done Sent false
5949 2749 Acked Write 8 Done Done Sent false
5950 2750 Acked Read 2 Done Done Sent false
5951 2751 Acked Write 9 Done Done Sent false
5952 2752 Acked Read 5 Done Done Sent false
5953 2753 Acked Read 5 Done Done Sent false
5954 2754 Acked Read 6 Done Done Sent false
5955 2755 Acked Write 5 Done Done Sent false
5956 2756 Acked Write 4 Done Done Sent false
5957 2757 Acked Write 6 Done Done Sent false
5958 2758 Acked Read 4 Done Done Sent false
5959 2759 Acked Write 8 Done Done Sent false
5960 2760 Acked Read 9 Done Done Sent false
5961 2761 Acked Flush 0 Done Done Sent false
5962 2762 Acked Read 6 Done Done Sent false
5963 2763 Acked Flush 0 Done Done Sent false
5964 2764 Acked Write 10 Done Done Sent false
5965 2765 Acked Flush 0 Done Done Sent false
5966 2766 Acked Write 10 Done Done Sent false
5967 2767 Acked Write 1 Done Done Sent false
5968 2768 Acked Write 9 Done Done Sent false
5969 2769 Acked Write 6 Done Done Sent false
5970 2770 Acked Read 5 Done Done Sent false
5971 2771 Acked Write 10 Done Done Sent false
5972 2772 Acked Read 5 Done Done Sent false
5973 2773 Acked Read 3 Done Done Sent false
5974 2774 Acked Write 7 Done Done Sent false
5975 2775 Acked Read 3 Done Done Sent false
5976 2776 Acked Read 5 Done Done Sent false
5977 2777 Acked Read 3 Done Done Sent false
5978 2778 Acked Read 1 Done Done Sent false
5979 2779 Acked Write 8 Done Done Sent false
5980 2780 Acked Write 1 Done Done Sent false
5981 2781 Acked Write 5 Done Done Sent false
5982 2782 Acked Write 7 Done Done Sent false
5983 2783 Acked Read 8 Done Done Sent false
5984 2784 Acked Write 10 Done Done Sent false
5985 2785 Acked Read 6 Done Done Sent false
5986 2786 Acked Write 4 Done Done Sent false
5987 2787 Acked Read 8 Done Done Sent false
5988 2788 Acked Write 5 Done Done Sent false
5989 2789 Acked Read 7 Done Done Sent false
5990 2790 Acked Read 4 Done Done Sent false
5991 2791 Acked Read 5 Done Done Sent false
5992 2792 Acked Read 9 Done Done Sent false
5993 2793 Acked Read 8 Done Done Sent false
5994 2794 Acked Read 10 Done Done Sent false
5995 2795 Acked Read 9 Done Done Sent false
5996 2796 Acked Read 10 Done Done Sent false
5997 2797 Acked Flush 0 Done Done Sent false
5998 2798 Acked Write 3 Done Done Sent false
5999 2799 Acked Flush 0 Done Done Sent false
6000 2800 Acked Read 8 Done Done Sent false
6001 2801 Acked Read 1 Done Done Sent false
6002 2802 Acked Read 2 Done Done Sent false
6003 2803 Acked Write 7 Done Done Sent false
6004 2804 Acked Read 6 Done Done Sent false
6005 2805 Acked Write 6 Done Done Sent false
6006 2806 Acked Write 8 Done Done Sent false
6007 2807 Acked Write 9 Done Done Sent false
6008 2808 Acked Read 7 Done Done Sent false
6009 2809 Acked Flush 0 Done Done Sent false
6010 STATES DS:0 DS:1 DS:2 TOTAL
6011 Sent 0 0 302 302
6012 Done 302 302 0 604
6013 Skipped 0 0 0 0
6014 Error 0 0 0 0
6015 Last Flush: 2809 2809 2507
6016 Downstairs last five completed: 2507 2506 2505 2504 2503
6017 Upstairs last five completed: 2809 2808 2807 2806 2805
6018 CLIENT: Up:0 ds:302 act:2
60192025-01-14T16:11:51.152ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
60202025-01-14T16:11:51.152ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:33698) = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
60212025-01-14T16:11:51.153ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
60222025-01-14T16:11:51.153ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
60232025-01-14T16:11:51.153ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
60242025-01-14T16:11:51.153ZINFOcrucible: send last flush ID to this DS: 2507 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
60252025-01-14T16:11:51.154ZINFOcrucible: Replied this last flush ID: 2507 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
60262025-01-14T16:11:51.154ZINFOcrucible: [2] client re-new 302 jobs since flush 2507 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
60272025-01-14T16:11:51.154ZINFOcrucible: [2] Replayed 302 jobs since flush: 2507 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
6028 ----------------------------------------------------------------
6029 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
6030 Downstairs last five completed: 2809 2808 2807 2806 2805
6031 Upstairs last five completed: 2809 2808 2807 2806 2805
6032 CLIENT: Up:0 ds:0 act:3
6033 Replay: All jobs finished
6034 Main task has work to do, go find it
6035 got dsc Stop(1)
6036 stop 1
6037 Jan 14 16:11:53.396 INFO request completed, latency_us: 297, response_code: [8820] Got stop action so:false kr:false
6038 204, uri: /stop/cid/1, method: POST, req_id: 9b4d5cd1-e698-45c6-ba45-1c45f69d20bb, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
60392025-01-14T16:11:53.397ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
60402025-01-14T16:11:53.397ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
6041 Jan 14 16:11:53.397 INFO request completed, latency_us: 109, {"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-14T16:11:53.397724599Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
6042 response_code: 200, uri: /state/cid/1, method: GET, req_id: c63c9d5b-0c66-43ec-8df0-3a34c8880ba8, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-14T16:11:53.397762985Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
6043 :9998
6044 [8820] Exited with: ExitStatus(unix_wait_status(9))
6045 [8820][1] reports Exit
6046 Jan 14 16:11:57.400 INFO request completed, latency_us: 203, response_code: 200, uri: /state/cid/1, method: GET, req_id: a67b7a0f-2f51-4ea8-805f-3d0efaf912ac, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
6047 001/300 Read block 4324 len 24576
6048 002/300 Write block 121 len 32768 data: 2 2 2 2 2 2 2 2
6049 003/300 Read block 3729 len 12288
6050 004/300 Read block 4864 len 28672
6051 005/300 Read block 4135 len 36864
6052 006/300 Read block 3943 len 40960
6053 007/300 Read block 4397 len 32768
6054 008/300 Write block 642 len 4096 data: 2
6055 009/300 Write block 4925 len 36864 data: 2 2 2 2 2 2 3 3 3
6056 010/300 Write block 4472 len 36864 data: 3 3 3 3 2 2 2 2 2
6057 011/300 Flush
6058 012/300 Read block 4746 len 20480
6059 013/300 Read block 2189 len 36864
6060 014/300 Read block 1614 len 32768
6061 015/300 Read block 3635 len 32768
6062 016/300 Write block 3537 len 24576 data: 2 2 2 2 2 2
6063 017/300 Read block 2693 len 36864
6064 018/300 Read block 3158 len 4096
6065 019/300 Read block 397 len 4096
6066 020/300 Write block 738 len 36864 data: 2 2 2 2 2 2 2 3 3
6067 021/300 Write block 2861 len 4096 data: 2
6068 022/300 Write block 2397 len 8192 data: 3 2
6069 023/300 Write block 1687 len 8192 data: 3 3
6070 024/300 Read block 2397 len 32768
6071 025/300 Write block 1447 len 8192 data: 3 3
6072 026/300 Read block 2255 len 16384
6073 027/300 Write block 3998 len 20480 data: 6 6 5 5 5
6074 028/300 Write block 531 len 4096 data: 2
6075 029/300 Read block 1944 len 32768
6076 030/300 Write block 908 len 20480 data: 4 4 4 4 3
6077 031/300 Write block 3640 len 28672 data: 3 2 2 2 2 2 2
6078 032/300 Read block 4729 len 12288
6079 033/300 Read block 4199 len 32768
6080 034/300 Read block 1721 len 4096
6081 035/300 Read block 3796 len 40960
6082 036/300 Flush
6083 037/300 Read block 4963 len 16384
6084 038/300 Read block 3780 len 32768
6085 039/300 Read block 3053 len 28672
6086 040/300 Read block 2557 len 40960
6087 041/300 Flush
6088 042/300 Write block 286 len 40960 data: 2 2 4 4 4 4 4 4 3 3
6089 043/300 Read block 3304 len 16384
6090 044/300 Read block 3811 len 24576
6091 045/300 Read block 3237 len 24576
6092 046/300 Write block 671 len 24576 data: 3 3 3 3 3 2
6093 047/300 Read block 3221 len 4096
6094 048/300 Write block 1905 len 8192 data: 3 3
6095 049/300 Read block 3407 len 12288
6096 050/300 Flush
6097 051/300 Read block 3652 len 4096
6098 052/300 Write block 2 len 16384 data: 3 3 3 3
6099 053/300 Flush
6100 054/300 Write block 2101 len 16384 data: 3 2 2 2
6101 055/300 Read block 3699 len 20480
6102 056/300 Read block 3024 len 4096
6103 057/300 Flush
6104 058/300 Write block 1523 len 40960 data: 2 3 3 3 3 3 3 4 4 4
6105 059/300 Write block 78 len 12288 data: 3 4 4
6106 060/300 Flush
6107 061/300 Read block 2718 len 24576
6108 062/300 Read block 4603 len 8192
6109 063/300 Read block 2503 len 20480
6110 064/300 Read block 357 len 8192
6111 065/300 Write block 4059 len 36864 data: 2 2 2 2 2 2 3 2 2
6112 066/300 Write block 4053 len 28672 data: 2 2 2 2 2 2 3
6113 067/300 Read block 1592 len 32768
6114 068/300 Write block 230 len 40960 data: 2 2 2 3 3 3 3 3 2 2
6115 069/300 Write block 3028 len 20480 data: 3 4 4 4 3
6116 070/300 Read block 3861 len 40960
6117 071/300 Write block 3454 len 32768 data: 3 3 3 3 3 2 2 2
6118 072/300 Write block 1589 len 36864 data: 2 2 3 3 3 3 3 3 3
6119 073/300 Read block 1917 len 24576
6120 074/300 Write block 2827 len 32768 data: 2 2 2 3 4 3 3 3
6121 075/300 Read block 4342 len 32768
6122 076/300 Write block 2049 len 36864 data: 3 3 3 2 3 3 3 3 3
6123 077/300 Flush
6124 078/300 Read block 1807 len 12288
6125 079/300 Read block 4611 len 20480
6126 080/300 Read block 4228 len 24576
6127 081/300 Read block 1414 len 36864
6128 082/300 Read block 1438 len 28672
6129 083/300 Write block 1434 len 32768 data: 3 3 2 2 2 3 2 2
6130 084/300 Read block 757 len 12288
6131 085/300 Read block 3007 len 32768
6132 086/300 Write block 4201 len 12288 data: 3 3 3
6133 087/300 Read block 1203 len 32768
6134 088/300 Flush
6135 089/300 Read block 649 len 8192
6136 090/300 Flush
6137 091/300 Read block 2661 len 24576
6138 092/300 Write block 899 len 36864 data: 3 2 2 2 2 2 4 4 4
6139 093/300 Flush
6140 094/300 Read block 4548 len 16384
6141 095/300 Write block 4295 len 36864 data: 2 2 2 2 3 2 2 2 2
6142 096/300 Write block 425 len 32768 data: 3 3 2 2 2 2 2 2
6143 097/300 Write block 4410 len 32768 data: 3 3 3 3 2 3 3 3
6144 098/300 Read block 4917 len 32768
6145 099/300 Read block 3901 len 32768
6146 100/300 Read block 3086 len 12288
6147 101/300 Read block 2050 len 28672
6148 102/300 Read block 1178 len 28672
6149 103/300 Write block 3053 len 20480 data: 2 2 2 2 2
6150 104/300 Read block 1108 len 8192
6151 105/300 Write block 2197 len 16384 data: 4 3 4 3
6152 106/300 Read block 1047 len 16384
6153 107/300 Write block 1500 len 28672 data: 3 3 3 3 3 3 3
6154 108/300 Read block 1693 len 24576
6155 109/300 Read block 1969 len 24576
6156 110/300 Read block 4513 len 8192
6157 111/300 Write block 3024 len 4096 data: 3
6158 112/300 Write block 675 len 20480 data: 4 3 2 2 2
6159 113/300 Write block 3245 len 16384 data: 3 3 3 3
6160 114/300 Read block 34 len 8192
6161 115/300 Read block 1718 len 12288
6162 116/300 Read block 3890 len 32768
6163 117/300 Read block 2480 len 20480
6164 118/300 Flush
6165 119/300 Read block 3250 len 20480
6166 120/300 Flush
6167 121/300 Read block 457 len 36864
6168 122/300 Read block 1585 len 36864
6169 123/300 Read block 4981 len 28672
6170 124/300 Read block 1776 len 4096
6171 125/300 Flush
6172 126/300 Write block 4810 len 4096 data: 2
6173 127/300 Read block 1683 len 32768
6174 128/300 Read block 2371 len 40960
6175 129/300 Write block 421 len 36864 data: 4 4 4 4 4 4 3 3 3
6176 130/300 Read block 4128 len 8192
6177 131/300 Read block 3177 len 32768
6178 132/300 Read block 1035 len 4096
6179 133/300 Write block 2085 len 12288 data: 2 2 2
6180 134/300 Flush
6181 135/300 Read block 2991 len 24576
6182 136/300 Read block 407 len 40960
6183 137/300 Write block 3773 len 4096 data: 4
6184 138/300 Write block 3377 len 24576 data: 2 2 2 2 2 4
6185 139/300 Read block 79 len 16384
6186 140/300 Read block 1697 len 36864
6187 141/300 Read block 2905 len 40960
6188 142/300 Read block 841 len 36864
6189 143/300 Read block 1600 len 4096
6190 144/300 Read block 2535 len 32768
6191 145/300 Write block 4773 len 16384 data: 5 4 4 3
6192 146/300 Read block 4906 len 24576
6193 147/300 Write block 2991 len 12288 data: 3 3 3
6194 148/300 Write block 724 len 8192 data: 2 3
6195 149/300 Write block 633 len 4096 data: 2
6196 150/300 Write block 3024 len 36864 data: 4 3 3 3 4 5 5 5 4
6197 151/300 Read block 3555 len 8192
6198 152/300 Read block 3981 len 16384
6199 153/300 Write block 1880 len 20480 data: 2 2 2 2 2
6200 154/300 Read block 2484 len 40960
6201 155/300 Write block 1753 len 20480 data: 4 4 3 3 3
6202 156/300 Read block 914 len 36864
6203 157/300 Read block 3342 len 28672
6204 158/300 Flush
6205 159/300 Read block 186 len 28672
6206 160/300 Read block 833 len 32768
6207 161/300 Read block 1768 len 28672
6208 162/300 Read block 661 len 32768
6209 163/300 Read block 137 len 40960
6210 164/300 Read block 2457 len 16384
6211 165/300 Write block 564 len 4096 data: 2
6212 166/300 Read block 4427 len 40960
6213 167/300 Read block 3150 len 16384
6214 168/300 Read block 1621 len 4096
6215 169/300 Read block 766 len 8192
6216 170/300 Write block 1954 len 8192 data: 3 4
6217 171/300 Write block 1464 len 4096 data: 3
6218 172/300 Write block 2891 len 40960 data: 3 3 3 4 4 3 2 2 2 2
6219 173/300 Write block 1020 len 28672 data: 4 4 3 2 2 2 2
6220 174/300 Write block 954 len 32768 data: 2 2 2 2 2 2 2 2
6221 175/300 Read block 4354 len 12288
6222 176/300 Write block 4496 len 28672 data: 2 2 2 2 2 2 2
6223 177/300 Write block 265 len 28672 data: 2 2 2 2 2 2 2
6224 178/300 Flush
6225 179/300 Write block 4776 len 16384 data: 4 4 3 3
6226 180/300 Read block 4258 len 8192
6227 181/300 Write block 4393 len 40960 data: 3 4 3 3 3 3 4 2 2 2
6228 182/300 Flush
6229 183/300 Read block 2695 len 28672
6230 184/300 Write block 4473 len 20480 data: 4 4 4 3 3
6231 185/300 Read block 4636 len 8192
6232 186/300 Write block 3191 len 4096 data: 4
6233 187/300 Write block 2333 len 16384 data: 3 2 2 2
6234 188/300 Read block 2508 len 32768
6235 189/300 Write block 3215 len 4096 data: 3
6236 190/300 Read block 2143 len 28672
6237 191/300 Write block 416 len 36864 data: 4 4 4 3 4 5 5 5 5
6238 192/300 Write block 3831 len 24576 data: 2 2 2 2 2 3
6239 193/300 Read block 4062 len 24576
6240 194/300 Read block 1383 len 4096
6241 195/300 Read block 54 len 20480
6242 196/300 Write block 2600 len 40960 data: 3 3 2 2 2 2 2 2 2 2
6243 197/300 Write block 2945 len 8192 data: 3 3
6244 198/300 Write block 283 len 24576 data: 2 2 2 3 3 5
6245 199/300 Write block 2100 len 32768 data: 3 4 3 3 3 2 3 4
6246 200/300 Write block 1921 len 40960 data: 2 2 2 2 3 3 3 3 2 3
6247 201/300 Read block 3411 len 4096
6248 202/300 Write block 4391 len 20480 data: 3 3 4 5 4
6249 203/300 Read block 4018 len 40960
6250 204/300 Read block 766 len 36864
6251 205/300 Write block 3460 len 40960 data: 3 3 2 2 2 3 3 3 3 3
6252 206/300 Read block 3874 len 24576
6253 207/300 Read block 1291 len 4096
6254 208/300 Read block 2808 len 28672
6255 209/300 Write block 1625 len 16384 data: 2 2 2 2
6256 210/300 Write block 2666 len 40960 data: 2 2 2 2 2 2 2 2 2 2
6257 211/300 Write block 4103 len 20480 data: 3 3 3 3 2
6258 212/300 Write block 2868 len 24576 data: 2 2 2 2 2 2
6259 213/300 Write block 2471 len 12288 data: 3 2 2
6260 214/300 Write block 2772 len 32768 data: 3 3 3 3 2 2 3 3
6261 215/300 Flush
6262 216/300 Read block 4417 len 24576
6263 217/300 Write block 3852 len 8192 data: 2 2
6264 218/300 Read block 1862 len 24576
6265 219/300 Write block 3293 len 24576 data: 2 2 2 2 2 3
6266 220/300 Write block 706 len 16384 data: 4 3 3 3
6267 221/300 Write block 4703 len 4096 data: 3
6268 222/300 Read block 4105 len 12288
6269 223/300 Write block 2693 len 20480 data: 2 2 2 2 3
6270 224/300 Write block 3760 len 12288 data: 4 4 5
6271 225/300 Write block 2138 len 16384 data: 2 2 2 2
6272 226/300 Read block 2974 len 20480
6273 227/300 Read block 4501 len 12288
6274 228/300 Write block 4762 len 28672 data: 2 2 2 2 2 2 3
6275 229/300 Read block 1914 len 36864
6276 230/300 Read block 1175 len 16384
6277 231/300 Write block 2968 len 16384 data: 2 3 3 3
6278 232/300 Flush
6279 233/300 Write block 404 len 4096 data: 4
6280 234/300 Read block 4812 len 16384
6281 235/300 Write block 4093 len 12288 data: 4 3 3
6282 236/300 Read block 4035 len 32768
6283 237/300 Read block 3622 len 24576
6284 238/300 Read block 2647 len 12288
6285 239/300 Write block 1432 len 16384 data: 4 3 4 4
6286 240/300 Write block 4056 len 20480 data: 3 3 3 4 3
6287 241/300 Read block 3190 len 12288
6288 242/300 Write block 32 len 28672 data: 3 2 2 2 2 2 2
6289 243/300 Flush
6290 244/300 Write block 2980 len 16384 data: 2 2 2 2
6291 245/300 Read block 2568 len 8192
6292 246/300 Write block 4236 len 8192 data: 2 3
6293 247/300 Read block 4736 len 8192
6294 248/300 Write block 4464 len 16384 data: 2 2 2 3
6295 249/300 Write block 1569 len 8192 data: 4 4
6296 250/300 Write block 2900 len 40960 data: 3 2 2 3 3 3 3 3 3 3
6297 251/300 Read block 2233 len 28672
6298 252/300 Write block 221 len 32768 data: 3 3 3 2 2 2 2 2
6299 253/300 Write block 4975 len 12288 data: 3 3 3
6300 254/300 Flush
6301 255/300 Read block 4396 len 24576
6302 256/300 Read block 4847 len 36864
6303 257/300 Write block 3966 len 40960 data: 3 3 3 3 2 3 3 5 5 5
6304 258/300 Write block 4091 len 20480 data: 5 5 5 4 4
6305 259/300 Write block 1038 len 12288 data: 2 2 2
6306 260/300 Write block 1296 len 12288 data: 4 4 3
6307 261/300 Write block 4481 len 28672 data: 2 2 2 2 3 3 3
6308 262/300 Flush
6309 263/300 Read block 1180 len 12288
6310 264/300 Read block 2967 len 32768
6311 265/300 Read block 2476 len 4096
6312 266/300 Read block 4193 len 16384
6313 267/300 Read block 4777 len 20480
6314 268/300 Write block 2993 len 8192 data: 4 4
6315 269/300 Read block 142 len 8192
6316 270/300 Read block 4374 len 36864
6317 271/300 Write block 815 len 4096 data: 4
6318 272/300 Write block 3344 len 40960 data: 2 2 2 2 2 4 3 3 4 4
6319 273/300 Write block 2408 len 20480 data: 3 3 3 3 3
6320 274/300 Read block 37 len 40960
6321 275/300 Write block 983 len 32768 data: 2 2 2 2 2 2 2 2
6322 276/300 Write block 1032 len 8192 data: 3 3
6323 277/300 Write block 4892 len 24576 data: 2 2 2 2 2 2
6324 278/300 Write block 4577 len 8192 data: 2 2
6325 279/300 Write block 2803 len 16384 data: 3 2 2 2
6326 280/300 Read block 4511 len 28672
6327 281/300 Read block 2430 len 32768
6328 282/300 Read block 958 len 12288
6329 283/300 Write block 3846 len 20480 data: 3 3 3 2 2
6330 284/300 Read block 4344 len 20480
6331 285/300 Read block 3186 len 40960
6332 286/300 Read block 50 len 32768
6333 287/300 Write block 1876 len 28672 data: 3 3 3 3 3 3 3
6334 288/300 Write block 1558 len 16384 data: 3 2 2 2
6335 289/300 Read block 855 len 40960
6336 290/300 Write block 494 len 24576 data: 2 2 2 3 3 3
6337 291/300 Write block 2593 len 12288 data: 2 2 2
6338 292/300 Write block 643 len 32768 data: 2 3 3 3 3 2 2 2
6339 293/300 Read block 1937 len 16384
6340 294/300 Write block 4735 len 8192 data: 3 3
6341 295/300 Write block 4063 len 12288 data: 3 3 4
6342 296/300 Read block 3667 len 4096
6343 297/300 Write block 3304 len 32768 data: 2 2 2 2 2 2 3 2
6344 298/300 Write block 3015 len 40960 data: 2 3 3 2 2 3 3 3 3 5
6345 299/300 Read block 4380 len 12288
6346 300/300 Read block 1800 len 28672
6347 301/300 Read block 38 len 24576
6348 Main task has work to do, go find it
6349 got dsc Start(1)
6350 start 1
6351 Jan 14 16:11:57.634 INFO request completed, latency_us: 197, response_code: 204, uri: /start/cid/1, method: POST, req_id: e4eee774-cdfb-4d9b-8c47-7d0d172d2e92, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
6352 Replay: started 1, returned:Ok(())
6353 ----------------------------------------------------------------
6354 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6355 States: Active Offline Active
6356 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6357 2810 Acked Read 6 Done Sent Done false
6358 2811 Acked Write 8 Done Sent Done false
6359 2812 Acked Read 3 Done Sent Done false
6360 2813 Acked Read 7 Done Sent Done false
6361 2814 Acked Read 9 Done Sent Done false
6362 2815 Acked Read 10 Done Sent Done false
6363 2816 Acked Read 8 Done Sent Done false
6364 2817 Acked Write 1 Done Sent Done false
6365 2818 Acked Write 9 Done Sent Done false
6366 2819 Acked Write 9 Done Sent Done false
6367 2820 Acked Flush 0 Done Sent Done false
6368 2821 Acked Read 5 Done Sent Done false
6369 2822 Acked Read 9 Done Sent Done false
6370 2823 Acked Read 8 Done Sent Done false
6371 2824 Acked Read 8 Done Sent Done false
6372 2825 Acked Write 6 Done Sent Done false
6373 2826 Acked Read 9 Done Sent Done false
6374 2827 Acked Read 1 Done Sent Done false
6375 2828 Acked Read 1 Done Sent Done false
6376 2829 Acked Write 9 Done Sent Done false
6377 2830 Acked Write 1 Done Sent Done false
6378 2831 Acked Write 2 Done Sent Done false
6379 2832 Acked Write 2 Done Sent Done false
6380 2833 Acked Read 8 Done Sent Done false
6381 2834 Acked Write 2 Done Sent Done false
6382 2835 Acked Read 4 Done Sent Done false
6383 2836 Acked Write 5 Done Sent Done false
6384 2837 Acked Write 1 Done Sent Done false
6385 2838 Acked Read 8 Done Sent Done false
6386 2839 Acked Write 5 Done Sent Done false
6387 2840 Acked Write 7 Done Sent Done false
6388 2841 Acked Read 3 Done Sent Done false
6389 2842 Acked Read 8 Done Sent Done false
6390 2843 Acked Read 1 Done Sent Done false
6391 2844 Acked Read 10 Done Sent Done false
6392 2845 Acked Flush 0 Done Sent Done false
6393 2846 Acked Read 4 Done Sent Done false
6394 2847 Acked Read 8 Done Sent Done false
6395 2848 Acked Read 7 Done Sent Done false
6396 2849 Acked Read 10 Done Sent Done false
6397 2850 Acked Flush 0 Done Sent Done false
6398 2851 Acked Write 10 Done Sent Done false
6399 2852 Acked Read 4 Done Sent Done false
6400 2853 Acked Read 6 Done Sent Done false
6401 2854 Acked Read 6 Done Sent Done false
6402 2855 Acked Write 6 Done Sent Done false
6403 2856 Acked Read 1 Done Sent Done false
6404 2857 Acked Write 2 Done Sent Done false
6405 2858 Acked Read 3 Done Sent Done false
6406 2859 Acked Flush 0 Done Sent Done false
6407 2860 Acked Read 1 Done Sent Done false
6408 2861 Acked Write 4 Done Sent Done false
6409 2862 Acked Flush 0 Done Sent Done false
6410 2863 Acked Write 4 Done Sent Done false
6411 2864 Acked Read 5 Done Sent Done false
6412 2865 Acked Read 1 Done Sent Done false
6413 2866 Acked Flush 0 Done Sent Done false
6414 2867 Acked Write 10 Done Sent Done false
6415 2868 Acked Write 3 Done Sent Done false
6416 2869 Acked Flush 0 Done Sent Done false
6417 2870 Acked Read 6 Done Sent Done false
6418 2871 Acked Read 2 Done Sent Done false
6419 2872 Acked Read 5 Done Sent Done false
6420 2873 Acked Read 2 Done Sent Done false
6421 2874 Acked Write 9 Done Sent Done false
6422 2875 Acked Write 7 Done Sent Done false
6423 2876 Acked Read 8 Done Sent Done false
6424 2877 Acked Write 10 Done Sent Done false
6425 2878 Acked Write 5 Done Sent Done false
6426 2879 Acked Read 10 Done Sent Done false
6427 2880 Acked Write 8 Done Sent Done false
6428 2881 Acked Write 9 Done Sent Done false
6429 2882 Acked Read 6 Done Sent Done false
6430 2883 Acked Write 8 Done Sent Done false
6431 2884 Acked Read 8 Done Sent Done false
6432 2885 Acked Write 9 Done Sent Done false
6433 2886 Acked Flush 0 Done Sent Done false
6434 2887 Acked Read 3 Done Sent Done false
6435 2888 Acked Read 5 Done Sent Done false
6436 2889 Acked Read 6 Done Sent Done false
6437 2890 Acked Read 9 Done Sent Done false
6438 2891 Acked Read 7 Done Sent Done false
6439 2892 Acked Write 8 Done Sent Done false
6440 2893 Acked Read 3 Done Sent Done false
6441 2894 Acked Read 8 Done Sent Done false
6442 2895 Acked Write 3 Done Sent Done false
6443 2896 Acked Read 8 Done Sent Done false
6444 2897 Acked Flush 0 Done Sent Done false
6445 2898 Acked Read 2 Done Sent Done false
6446 2899 Acked Flush 0 Done Sent Done false
6447 2900 Acked Read 6 Done Sent Done false
6448 2901 Acked Write 9 Done Sent Done false
6449 2902 Acked Flush 0 Done Sent Done false
6450 2903 Acked Read 4 Done Sent Done false
6451 2904 Acked Write 9 Done Sent Done false
6452 2905 Acked Write 8 Done Sent Done false
6453 2906 Acked Write 8 Done Sent Done false
6454 2907 Acked Read 8 Done Sent Done false
6455 2908 Acked Read 8 Done Sent Done false
6456 2909 Acked Read 3 Done Sent Done false
6457 2910 Acked Read 7 Done Sent Done false
6458 2911 Acked Read 7 Done Sent Done false
6459 2912 Acked Write 5 Done Sent Done false
6460 2913 Acked Read 2 Done Sent Done false
6461 2914 Acked Write 4 Done Sent Done false
6462 2915 Acked Read 4 Done Sent Done false
6463 2916 Acked Write 7 Done Sent Done false
6464 2917 Acked Read 6 Done Sent Done false
6465 2918 Acked Read 6 Done Sent Done false
6466 2919 Acked Read 2 Done Sent Done false
6467 2920 Acked Write 1 Done Sent Done false
6468 2921 Acked Write 5 Done Sent Done false
6469 2922 Acked Write 4 Done Sent Done false
6470 2923 Acked Read 2 Done Sent Done false
6471 2924 Acked Read 3 Done Sent Done false
6472 2925 Acked Read 8 Done Sent Done false
6473 2926 Acked Read 5 Done Sent Done false
6474 2927 Acked Flush 0 Done Sent Done false
6475 2928 Acked Read 5 Done Sent Done false
6476 2929 Acked Flush 0 Done Sent Done false
6477 2930 Acked Read 9 Done Sent Done false
6478 2931 Acked Read 9 Done Sent Done false
6479 2932 Acked Read 7 Done Sent Done false
6480 2933 Acked Read 1 Done Sent Done false
6481 2934 Acked Flush 0 Done Sent Done false
6482 2935 Acked Write 1 Done Sent Done false
6483 2936 Acked Read 8 Done Sent Done false
6484 2937 Acked Read 10 Done Sent Done false
6485 2938 Acked Write 9 Done Sent Done false
6486 2939 Acked Read 2 Done Sent Done false
6487 2940 Acked Read 8 Done Sent Done false
6488 2941 Acked Read 1 Done Sent Done false
6489 2942 Acked Write 3 Done Sent Done false
6490 2943 Acked Flush 0 Done Sent Done false
6491 2944 Acked Read 6 Done Sent Done false
6492 2945 Acked Read 10 Done Sent Done false
6493 2946 Acked Write 1 Done Sent Done false
6494 2947 Acked Write 6 Done Sent Done false
6495 2948 Acked Read 4 Done Sent Done false
6496 2949 Acked Read 9 Done Sent Done false
6497 2950 Acked Read 10 Done Sent Done false
6498 2951 Acked Read 9 Done Sent Done false
6499 2952 Acked Read 1 Done Sent Done false
6500 2953 Acked Read 8 Done Sent Done false
6501 2954 Acked Write 4 Done Sent Done false
6502 2955 Acked Read 6 Done Sent Done false
6503 2956 Acked Write 3 Done Sent Done false
6504 2957 Acked Write 2 Done Sent Done false
6505 2958 Acked Write 1 Done Sent Done false
6506 2959 Acked Write 9 Done Sent Done false
6507 2960 Acked Read 2 Done Sent Done false
6508 2961 Acked Read 4 Done Sent Done false
6509 2962 Acked Write 5 Done Sent Done false
6510 2963 Acked Read 10 Done Sent Done false
6511 2964 Acked Write 5 Done Sent Done false
6512 2965 Acked Read 9 Done Sent Done false
6513 2966 Acked Read 7 Done Sent Done false
6514 2967 Acked Flush 0 Done Sent Done false
6515 2968 Acked Read 7 Done Sent Done false
6516 2969 Acked Read 8 Done Sent Done false
6517 2970 Acked Read 7 Done Sent Done false
6518 2971 Acked Read 8 Done Sent Done false
6519 2972 Acked Read 10 Done Sent Done false
6520 2973 Acked Read 4 Done Sent Done false
6521 2974 Acked Write 1 Done Sent Done false
6522 2975 Acked Read 10 Done Sent Done false
6523 2976 Acked Read 4 Done Sent Done false
6524 2977 Acked Read 1 Done Sent Done false
6525 2978 Acked Read 2 Done Sent Done false
6526 2979 Acked Write 2 Done Sent Done false
6527 2980 Acked Write 1 Done Sent Done false
6528 2981 Acked Write 10 Done Sent Done false
6529 2982 Acked Write 7 Done Sent Done false
6530 2983 Acked Write 8 Done Sent Done false
6531 2984 Acked Read 3 Done Sent Done false
6532 2985 Acked Write 7 Done Sent Done false
6533 2986 Acked Write 7 Done Sent Done false
6534 2987 Acked Flush 0 Done Sent Done false
6535 2988 Acked Write 4 Done Sent Done false
6536 2989 Acked Read 2 Done Sent Done false
6537 2990 Acked Write 10 Done Sent Done false
6538 2991 Acked Flush 0 Done Sent Done false
6539 2992 Acked Read 7 Done Sent Done false
6540 2993 Acked Write 5 Done Sent Done false
6541 2994 Acked Read 2 Done Sent Done false
6542 2995 Acked Write 1 Done Sent Done false
6543 2996 Acked Write 4 Done Sent Done false
6544 2997 Acked Read 8 Done Sent Done false
6545 2998 Acked Write 1 Done Sent Done false
6546 2999 Acked Read 7 Done Sent Done false
6547 3000 Acked Write 9 Done Sent Done false
6548 3001 Acked Write 6 Done Sent Done false
6549 3002 Acked Read 6 Done Sent Done false
6550 3003 Acked Read 1 Done Sent Done false
6551 3004 Acked Read 5 Done Sent Done false
6552 3005 Acked Write 10 Done Sent Done false
6553 3006 Acked Write 2 Done Sent Done false
6554 3007 Acked Write 6 Done Sent Done false
6555 3008 Acked Write 8 Done Sent Done false
6556 3009 Acked Write 10 Done Sent Done false
6557 3010 Acked Read 1 Done Sent Done false
6558 3011 Acked Write 5 Done Sent Done false
6559 3012 Acked Read 10 Done Sent Done false
6560 3013 Acked Read 9 Done Sent Done false
6561 3014 Acked Write 10 Done Sent Done false
6562 3015 Acked Read 6 Done Sent Done false
6563 3016 Acked Read 1 Done Sent Done false
6564 3017 Acked Read 7 Done Sent Done false
6565 3018 Acked Write 4 Done Sent Done false
6566 3019 Acked Write 10 Done Sent Done false
6567 3020 Acked Write 5 Done Sent Done false
6568 3021 Acked Write 6 Done Sent Done false
6569 3022 Acked Write 3 Done Sent Done false
6570 3023 Acked Write 8 Done Sent Done false
6571 3024 Acked Flush 0 Done Sent Done false
6572 3025 Acked Read 6 Done Sent Done false
6573 3026 Acked Write 2 Done Sent Done false
6574 3027 Acked Read 6 Done Sent Done false
6575 3028 Acked Write 6 Done Sent Done false
6576 3029 Acked Write 4 Done Sent Done false
6577 3030 Acked Write 1 Done Sent Done false
6578 3031 Acked Read 3 Done Sent Done false
6579 3032 Acked Write 5 Done Sent Done false
6580 3033 Acked Write 3 Done Sent Done false
6581 3034 Acked Write 4 Done Sent Done false
6582 3035 Acked Read 5 Done Sent Done false
6583 3036 Acked Read 3 Done Sent Done false
6584 3037 Acked Write 7 Done Sent Done false
6585 3038 Acked Read 9 Done Sent Done false
6586 3039 Acked Read 4 Done Sent Done false
6587 3040 Acked Write 4 Done Sent Done false
6588 3041 Acked Flush 0 Done Sent Done false
6589 3042 Acked Write 1 Done Sent Done false
6590 3043 Acked Read 4 Done Sent Done false
6591 3044 Acked Write 3 Done Sent Done false
6592 3045 Acked Read 8 Done Sent Done false
6593 3046 Acked Read 6 Done Sent Done false
6594 3047 Acked Read 3 Done Sent Done false
6595 3048 Acked Write 4 Done Sent Done false
6596 3049 Acked Write 5 Done Sent Done false
6597 3050 Acked Read 3 Done Sent Done false
6598 3051 Acked Write 7 Done Sent Done false
6599 3052 Acked Flush 0 Done Sent Done false
6600 3053 Acked Write 4 Done Sent Done false
6601 3054 Acked Read 2 Done Sent Done false
6602 3055 Acked Write 2 Done Sent Done false
6603 3056 Acked Read 2 Done Sent Done false
6604 3057 Acked Write 4 Done Sent Done false
6605 3058 Acked Write 2 Done Sent Done false
6606 3059 Acked Write 10 Done Sent Done false
6607 3060 Acked Read 7 Done Sent Done false
6608 3061 Acked Write 8 Done Sent Done false
6609 3062 Acked Write 3 Done Sent Done false
6610 3063 Acked Flush 0 Done Sent Done false
6611 3064 Acked Read 6 Done Sent Done false
6612 3065 Acked Read 9 Done Sent Done false
6613 3066 Acked Write 10 Done Sent Done false
6614 3067 Acked Write 5 Done Sent Done false
6615 3068 Acked Write 3 Done Sent Done false
6616 3069 Acked Write 3 Done Sent Done false
6617 3070 Acked Write 7 Done Sent Done false
6618 3071 Acked Flush 0 Done Sent Done false
6619 3072 Acked Read 3 Done Sent Done false
6620 3073 Acked Read 8 Done Sent Done false
6621 3074 Acked Read 1 Done Sent Done false
6622 3075 Acked Read 4 Done Sent Done false
6623 3076 Acked Read 5 Done Sent Done false
6624 3077 Acked Write 2 Done Sent Done false
6625 3078 Acked Read 2 Done Sent Done false
6626 3079 Acked Read 9 Done Sent Done false
6627 3080 Acked Write 1 Done Sent Done false
6628 3081 Acked Write 10 Done Sent Done false
6629 3082 Acked Write 5 Done Sent Done false
6630 3083 Acked Read 10 Done Sent Done false
6631 3084 Acked Write 8 Done Sent Done false
6632 3085 Acked Write 2 Done Sent Done false
6633 3086 Acked Write 6 Done Sent Done false
6634 3087 Acked Write 2 Done Sent Done false
6635 3088 Acked Write 4 Done Sent Done false
6636 3089 Acked Read 7 Done Sent Done false
6637 3090 Acked Read 8 Done Sent Done false
6638 3091 Acked Read 3 Done Sent Done false
6639 3092 Acked Write 5 Done Sent Done false
6640 3093 Acked Read 5 Done Sent Done false
6641 3094 Acked Read 10 Done Sent Done false
6642 3095 Acked Read 8 Done Sent Done false
6643 3096 Acked Write 7 Done Sent Done false
6644 3097 Acked Write 4 Done Sent Done false
6645 3098 Acked Read 10 Done Sent Done false
6646 3099 Acked Write 6 Done Sent Done false
6647 3100 Acked Write 3 Done Sent Done false
6648 3101 Acked Write 8 Done Sent Done false
6649 3102 Acked Read 4 Done Sent Done false
6650 3103 Acked Write 2 Done Sent Done false
6651 3104 Acked Write 3 Done Sent Done false
6652 3105 Acked Read 1 Done Sent Done false
6653 3106 Acked Write 8 Done Sent Done false
6654 3107 Acked Write 10 Done Sent Done false
6655 3108 Acked Read 3 Done Sent Done false
6656 3109 Acked Read 7 Done Sent Done false
6657 3110 Acked Read 6 Done Sent Done false
6658 STATES DS:0 DS:1 DS:2 TOTAL
6659 Sent 0 301 0 301
6660 Done 301 0 301 602
6661 Skipped 0 0 0 0
6662 Error 0 0 0 0
6663 Last Flush: 3071 2809 3071
6664 Downstairs last five completed: 2809 2808 2807 2806 2805
6665 Upstairs last five completed: 3110 3109 3108 3107 3106
6666 CLIENT: Up:0 ds:301 act:2
6667 ----------------------------------------------------------------
6668 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6669 States: Active Offline Active
6670 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6671 2810 Acked Read 6 Done Sent Done false
6672 2811 Acked Write 8 Done Sent Done false
6673 2812 Acked Read 3 Done Sent Done false
6674 2813 Acked Read 7 Done Sent Done false
6675 2814 Acked Read 9 Done Sent Done false
6676 2815 Acked Read 10 Done Sent Done false
6677 2816 Acked Read 8 Done Sent Done false
6678 2817 Acked Write 1 Done Sent Done false
6679 2818 Acked Write 9 Done Sent Done false
6680 2819 Acked Write 9 Done Sent Done false
6681 2820 Acked Flush 0 Done Sent Done false
6682 2821 Acked Read 5 Done Sent Done false
6683 2822 Acked Read 9 Done Sent Done false
6684 2823 Acked Read 8 Done Sent Done false
6685 2824 Acked Read 8 Done Sent Done false
6686 2825 Acked Write 6 Done Sent Done false
6687 2826 Acked Read 9 Done Sent Done false
6688 2827 Acked Read 1 Done Sent Done false
6689 2828 Acked Read 1 Done Sent Done false
6690 2829 Acked Write 9 Done Sent Done false
6691 2830 Acked Write 1 Done Sent Done false
6692 2831 Acked Write 2 Done Sent Done false
6693 2832 Acked Write 2 Done Sent Done false
6694 2833 Acked Read 8 Done Sent Done false
6695 2834 Acked Write 2 Done Sent Done false
6696 2835 Acked Read 4 Done Sent Done false
6697 2836 Acked Write 5 Done Sent Done false
6698 2837 Acked Write 1 Done Sent Done false
6699 2838 Acked Read 8 Done Sent Done false
6700 2839 Acked Write 5 Done Sent Done false
6701 2840 Acked Write 7 Done Sent Done false
6702 2841 Acked Read 3 Done Sent Done false
6703 2842 Acked Read 8 Done Sent Done false
6704 2843 Acked Read 1 Done Sent Done false
6705 2844 Acked Read 10 Done Sent Done false
6706 2845 Acked Flush 0 Done Sent Done false
6707 2846 Acked Read 4 Done Sent Done false
6708 2847 Acked Read 8 Done Sent Done false
6709 2848 Acked Read 7 Done Sent Done false
6710 2849 Acked Read 10 Done Sent Done false
6711 2850 Acked Flush 0 Done Sent Done false
6712 2851 Acked Write 10 Done Sent Done false
6713 2852 Acked Read 4 Done Sent Done false
6714 2853 Acked Read 6 Done Sent Done false
6715 2854 Acked Read 6 Done Sent Done false
6716 2855 Acked Write 6 Done Sent Done false
6717 2856 Acked Read 1 Done Sent Done false
6718 2857 Acked Write 2 Done Sent Done false
6719 2858 Acked Read 3 Done Sent Done false
6720 2859 Acked Flush 0 Done Sent Done false
6721 2860 Acked Read 1 Done Sent Done false
6722 2861 Acked Write 4 Done Sent Done false
6723 2862 Acked Flush 0 Done Sent Done false
6724 2863 Acked Write 4 Done Sent Done false
6725 2864 Acked Read 5 Done Sent Done false
6726 2865 Acked Read 1 Done Sent Done false
6727 2866 Acked Flush 0 Done Sent Done false
6728 2867 Acked Write 10 Done Sent Done false
6729 2868 Acked Write 3 Done Sent Done false
6730 2869 Acked Flush 0 Done Sent Done false
6731 2870 Acked Read 6 Done Sent Done false
6732 2871 Acked Read 2 Done Sent Done false
6733 2872 Acked Read 5 Done Sent Done false
6734 2873 Acked Read 2 Done Sent Done false
6735 2874 Acked Write 9 Done Sent Done false
6736 2875 Acked Write 7 Done Sent Done false
6737 2876 Acked Read 8 Done Sent Done false
6738 2877 Acked Write 10 Done Sent Done false
6739 2878 Acked Write 5 Done Sent Done false
6740 2879 Acked Read 10 Done Sent Done false
6741 2880 Acked Write 8 Done Sent Done false
6742 2881 Acked Write 9 Done Sent Done false
6743 2882 Acked Read 6 Done Sent Done false
6744 2883 Acked Write 8 Done Sent Done false
6745 2884 Acked Read 8 Done Sent Done false
6746 2885 Acked Write 9 Done Sent Done false
6747 2886 Acked Flush 0 Done Sent Done false
6748 2887 Acked Read 3 Done Sent Done false
6749 2888 Acked Read 5 Done Sent Done false
6750 2889 Acked Read 6 Done Sent Done false
6751 2890 Acked Read 9 Done Sent Done false
6752 2891 Acked Read 7 Done Sent Done false
6753 2892 Acked Write 8 Done Sent Done false
6754 2893 Acked Read 3 Done Sent Done false
6755 2894 Acked Read 8 Done Sent Done false
6756 2895 Acked Write 3 Done Sent Done false
6757 2896 Acked Read 8 Done Sent Done false
6758 2897 Acked Flush 0 Done Sent Done false
6759 2898 Acked Read 2 Done Sent Done false
6760 2899 Acked Flush 0 Done Sent Done false
6761 2900 Acked Read 6 Done Sent Done false
6762 2901 Acked Write 9 Done Sent Done false
6763 2902 Acked Flush 0 Done Sent Done false
6764 2903 Acked Read 4 Done Sent Done false
6765 2904 Acked Write 9 Done Sent Done false
6766 2905 Acked Write 8 Done Sent Done false
6767 2906 Acked Write 8 Done Sent Done false
6768 2907 Acked Read 8 Done Sent Done false
6769 2908 Acked Read 8 Done Sent Done false
6770 2909 Acked Read 3 Done Sent Done false
6771 2910 Acked Read 7 Done Sent Done false
6772 2911 Acked Read 7 Done Sent Done false
6773 2912 Acked Write 5 Done Sent Done false
6774 2913 Acked Read 2 Done Sent Done false
6775 2914 Acked Write 4 Done Sent Done false
6776 2915 Acked Read 4 Done Sent Done false
6777 2916 Acked Write 7 Done Sent Done false
6778 2917 Acked Read 6 Done Sent Done false
6779 2918 Acked Read 6 Done Sent Done false
6780 2919 Acked Read 2 Done Sent Done false
6781 2920 Acked Write 1 Done Sent Done false
6782 2921 Acked Write 5 Done Sent Done false
6783 2922 Acked Write 4 Done Sent Done false
6784 2923 Acked Read 2 Done Sent Done false
6785 2924 Acked Read 3 Done Sent Done false
6786 2925 Acked Read 8 Done Sent Done false
6787 2926 Acked Read 5 Done Sent Done false
6788 2927 Acked Flush 0 Done Sent Done false
6789 2928 Acked Read 5 Done Sent Done false
6790 2929 Acked Flush 0 Done Sent Done false
6791 2930 Acked Read 9 Done Sent Done false
6792 2931 Acked Read 9 Done Sent Done false
6793 2932 Acked Read 7 Done Sent Done false
6794 2933 Acked Read 1 Done Sent Done false
6795 2934 Acked Flush 0 Done Sent Done false
6796 2935 Acked Write 1 Done Sent Done false
6797 2936 Acked Read 8 Done Sent Done false
6798 2937 Acked Read 10 Done Sent Done false
6799 2938 Acked Write 9 Done Sent Done false
6800 2939 Acked Read 2 Done Sent Done false
6801 2940 Acked Read 8 Done Sent Done false
6802 2941 Acked Read 1 Done Sent Done false
6803 2942 Acked Write 3 Done Sent Done false
6804 2943 Acked Flush 0 Done Sent Done false
6805 2944 Acked Read 6 Done Sent Done false
6806 2945 Acked Read 10 Done Sent Done false
6807 2946 Acked Write 1 Done Sent Done false
6808 2947 Acked Write 6 Done Sent Done false
6809 2948 Acked Read 4 Done Sent Done false
6810 2949 Acked Read 9 Done Sent Done false
6811 2950 Acked Read 10 Done Sent Done false
6812 2951 Acked Read 9 Done Sent Done false
6813 2952 Acked Read 1 Done Sent Done false
6814 2953 Acked Read 8 Done Sent Done false
6815 2954 Acked Write 4 Done Sent Done false
6816 2955 Acked Read 6 Done Sent Done false
6817 2956 Acked Write 3 Done Sent Done false
6818 2957 Acked Write 2 Done Sent Done false
6819 2958 Acked Write 1 Done Sent Done false
6820 2959 Acked Write 9 Done Sent Done false
6821 2960 Acked Read 2 Done Sent Done false
6822 2961 Acked Read 4 Done Sent Done false
6823 2962 Acked Write 5 Done Sent Done false
6824 2963 Acked Read 10 Done Sent Done false
6825 2964 Acked Write 5 Done Sent Done false
6826 2965 Acked Read 9 Done Sent Done false
6827 2966 Acked Read 7 Done Sent Done false
6828 2967 Acked Flush 0 Done Sent Done false
6829 2968 Acked Read 7 Done Sent Done false
6830 2969 Acked Read 8 Done Sent Done false
6831 2970 Acked Read 7 Done Sent Done false
6832 2971 Acked Read 8 Done Sent Done false
6833 2972 Acked Read 10 Done Sent Done false
6834 2973 Acked Read 4 Done Sent Done false
6835 2974 Acked Write 1 Done Sent Done false
6836 2975 Acked Read 10 Done Sent Done false
6837 2976 Acked Read 4 Done Sent Done false
6838 2977 Acked Read 1 Done Sent Done false
6839 2978 Acked Read 2 Done Sent Done false
6840 2979 Acked Write 2 Done Sent Done false
6841 2980 Acked Write 1 Done Sent Done false
6842 2981 Acked Write 10 Done Sent Done false
6843 2982 Acked Write 7 Done Sent Done false
6844 2983 Acked Write 8 Done Sent Done false
6845 2984 Acked Read 3 Done Sent Done false
6846 2985 Acked Write 7 Done Sent Done false
6847 2986 Acked Write 7 Done Sent Done false
6848 2987 Acked Flush 0 Done Sent Done false
6849 2988 Acked Write 4 Done Sent Done false
6850 2989 Acked Read 2 Done Sent Done false
6851 2990 Acked Write 10 Done Sent Done false
6852 2991 Acked Flush 0 Done Sent Done false
6853 2992 Acked Read 7 Done Sent Done false
6854 2993 Acked Write 5 Done Sent Done false
6855 2994 Acked Read 2 Done Sent Done false
6856 2995 Acked Write 1 Done Sent Done false
6857 2996 Acked Write 4 Done Sent Done false
6858 2997 Acked Read 8 Done Sent Done false
6859 2998 Acked Write 1 Done Sent Done false
6860 2999 Acked Read 7 Done Sent Done false
6861 3000 Acked Write 9 Done Sent Done false
6862 3001 Acked Write 6 Done Sent Done false
6863 3002 Acked Read 6 Done Sent Done false
6864 3003 Acked Read 1 Done Sent Done false
6865 3004 Acked Read 5 Done Sent Done false
6866 3005 Acked Write 10 Done Sent Done false
6867 3006 Acked Write 2 Done Sent Done false
6868 3007 Acked Write 6 Done Sent Done false
6869 3008 Acked Write 8 Done Sent Done false
6870 3009 Acked Write 10 Done Sent Done false
6871 3010 Acked Read 1 Done Sent Done false
6872 3011 Acked Write 5 Done Sent Done false
6873 3012 Acked Read 10 Done Sent Done false
6874 3013 Acked Read 9 Done Sent Done false
6875 3014 Acked Write 10 Done Sent Done false
6876 3015 Acked Read 6 Done Sent Done false
6877 3016 Acked Read 1 Done Sent Done false
6878 3017 Acked Read 7 Done Sent Done false
6879 3018 Acked Write 4 Done Sent Done false
6880 3019 Acked Write 10 Done Sent Done false
6881 3020 Acked Write 5 Done Sent Done false
6882 3021 Acked Write 6 Done Sent Done false
6883 3022 Acked Write 3 Done Sent Done false
6884 3023 Acked Write 8 Done Sent Done false
6885 3024 Acked Flush 0 Done Sent Done false
6886 3025 Acked Read 6 Done Sent Done false
6887 3026 Acked Write 2 Done Sent Done false
6888 3027 Acked Read 6 Done Sent Done false
6889 3028 Acked Write 6 Done Sent Done false
6890 3029 Acked Write 4 Done Sent Done false
6891 3030 Acked Write 1 Done Sent Done false
6892 3031 Acked Read 3 Done Sent Done false
6893 3032 Acked Write 5 Done Sent Done false
6894 3033 Acked Write 3 Done Sent Done false
6895 3034 Acked Write 4 Done Sent Done false
6896 3035 Acked Read 5 Done Sent Done false
6897 3036 Acked Read 3 Done Sent Done false
6898 3037 Acked Write 7 Done Sent Done false
6899 3038 Acked Read 9 Done Sent Done false
6900 3039 Acked Read 4 Done Sent Done false
6901 3040 Acked Write 4 Done Sent Done false
6902 3041 Acked Flush 0 Done Sent Done false
6903 3042 Acked Write 1 Done Sent Done false
6904 3043 Acked Read 4 Done Sent Done false
6905 3044 Acked Write 3 Done Sent Done false
6906 3045 Acked Read 8 Done Sent Done false
6907 3046 Acked Read 6 Done Sent Done false
6908 3047 Acked Read 3 Done Sent Done false
6909 3048 Acked Write 4 Done Sent Done false
6910 3049 Acked Write 5 Done Sent Done false
6911 3050 Acked Read 3 Done Sent Done false
6912 3051 Acked Write 7 Done Sent Done false
6913 3052 Acked Flush 0 Done Sent Done false
6914 3053 Acked Write 4 Done Sent Done false
6915 3054 Acked Read 2 Done Sent Done false
6916 3055 Acked Write 2 Done Sent Done false
6917 3056 Acked Read 2 Done Sent Done false
6918 3057 Acked Write 4 Done Sent Done false
6919 3058 Acked Write 2 Done Sent Done false
6920 3059 Acked Write 10 Done Sent Done false
6921 3060 Acked Read 7 Done Sent Done false
6922 3061 Acked Write 8 Done Sent Done false
6923 3062 Acked Write 3 Done Sent Done false
6924 3063 Acked Flush 0 Done Sent Done false
6925 3064 Acked Read 6 Done Sent Done false
6926 3065 Acked Read 9 Done Sent Done false
6927 3066 Acked Write 10 Done Sent Done false
6928 3067 Acked Write 5 Done Sent Done false
6929 3068 Acked Write 3 Done Sent Done false
6930 3069 Acked Write 3 Done Sent Done false
6931 3070 Acked Write 7 Done Sent Done false
6932 3071 Acked Flush 0 Done Sent Done false
6933 3072 Acked Read 3 Done Sent Done false
6934 3073 Acked Read 8 Done Sent Done false
6935 3074 Acked Read 1 Done Sent Done false
6936 3075 Acked Read 4 Done Sent Done false
6937 3076 Acked Read 5 Done Sent Done false
6938 3077 Acked Write 2 Done Sent Done false
6939 3078 Acked Read 2 Done Sent Done false
6940 3079 Acked Read 9 Done Sent Done false
6941 3080 Acked Write 1 Done Sent Done false
6942 3081 Acked Write 10 Done Sent Done false
6943 3082 Acked Write 5 Done Sent Done false
6944 3083 Acked Read 10 Done Sent Done false
6945 3084 Acked Write 8 Done Sent Done false
6946 3085 Acked Write 2 Done Sent Done false
6947 3086 Acked Write 6 Done Sent Done false
6948 3087 Acked Write 2 Done Sent Done false
6949 3088 Acked Write 4 Done Sent Done false
6950 3089 Acked Read 7 Done Sent Done false
6951 3090 Acked Read 8 Done Sent Done false
6952 3091 Acked Read 3 Done Sent Done false
6953 3092 Acked Write 5 Done Sent Done false
6954 3093 Acked Read 5 Done Sent Done false
6955 3094 Acked Read 10 Done Sent Done false
6956 3095 Acked Read 8 Done Sent Done false
6957 3096 Acked Write 7 Done Sent Done false
6958 3097 Acked Write 4 Done Sent Done false
6959 3098 Acked Read 10 Done Sent Done false
6960 3099 Acked Write 6 Done Sent Done false
6961 3100 Acked Write 3 Done Sent Done false
6962 3101 Acked Write 8 Done Sent Done false
6963 3102 Acked Read 4 Done Sent Done false
6964 3103 Acked Write 2 Done Sent Done false
6965 3104 Acked Write 3 Done Sent Done false
6966 3105 Acked Read 1 Done Sent Done false
6967 3106 Acked Write 8 Done Sent Done false
6968 3107 Acked Write 10 Done Sent Done false
6969 3108 Acked Read 3 Done Sent Done false
6970 3109 Acked Read 7 Done Sent Done false
6971 3110 Acked Read 6 Done Sent Done false
6972 3111 Acked Flush 0 Done Sent Done false
6973 STATES DS:0 DS:1 DS:2 TOTAL
6974 Sent 0 302 0 302
6975 Done 302 0 302 604
6976 Skipped 0 0 0 0
6977 Error 0 0 0 0
6978 Last Flush: 3111 2809 3111
6979 Downstairs last five completed: 2809 2808 2807 2806 2805
6980 Upstairs last five completed: 3111 3110 3109 3108 3107
6981 CLIENT: Up:0 ds:302 act:2
69822025-01-14T16:12:03.399ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
69832025-01-14T16:12:03.399ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
69842025-01-14T16:12:03.399ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
69852025-01-14T16:12:03.399ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = c2423ef0-728e-4895-997a-d9044c50948b
69862025-01-14T16:12:03.399ZINFOcrucible: client task is exiting = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
6987 [8820] got start action
6988 [8820] I am going to restart
6989 Starting downstairs at port 8820
6990 Make output file at "/tmp/dsc/downstairs-8820.txt"
6991 [8820][1] reports Starting
6992 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1209)
6993 [8820][1] reports Running
6994 ----------------------------------------------------------------
6995 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6996 States: Active Offline Active
6997 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6998 2810 Acked Read 6 Done Sent Done false
6999 2811 Acked Write 8 Done Sent Done false
7000 2812 Acked Read 3 Done Sent Done false
7001 2813 Acked Read 7 Done Sent Done false
7002 2814 Acked Read 9 Done Sent Done false
7003 2815 Acked Read 10 Done Sent Done false
7004 2816 Acked Read 8 Done Sent Done false
7005 2817 Acked Write 1 Done Sent Done false
7006 2818 Acked Write 9 Done Sent Done false
7007 2819 Acked Write 9 Done Sent Done false
7008 2820 Acked Flush 0 Done Sent Done false
7009 2821 Acked Read 5 Done Sent Done false
7010 2822 Acked Read 9 Done Sent Done false
7011 2823 Acked Read 8 Done Sent Done false
7012 2824 Acked Read 8 Done Sent Done false
7013 2825 Acked Write 6 Done Sent Done false
7014 2826 Acked Read 9 Done Sent Done false
7015 2827 Acked Read 1 Done Sent Done false
7016 2828 Acked Read 1 Done Sent Done false
7017 2829 Acked Write 9 Done Sent Done false
7018 2830 Acked Write 1 Done Sent Done false
7019 2831 Acked Write 2 Done Sent Done false
7020 2832 Acked Write 2 Done Sent Done false
7021 2833 Acked Read 8 Done Sent Done false
7022 2834 Acked Write 2 Done Sent Done false
7023 2835 Acked Read 4 Done Sent Done false
7024 2836 Acked Write 5 Done Sent Done false
7025 2837 Acked Write 1 Done Sent Done false
7026 2838 Acked Read 8 Done Sent Done false
7027 2839 Acked Write 5 Done Sent Done false
7028 2840 Acked Write 7 Done Sent Done false
7029 2841 Acked Read 3 Done Sent Done false
7030 2842 Acked Read 8 Done Sent Done false
7031 2843 Acked Read 1 Done Sent Done false
7032 2844 Acked Read 10 Done Sent Done false
7033 2845 Acked Flush 0 Done Sent Done false
7034 2846 Acked Read 4 Done Sent Done false
7035 2847 Acked Read 8 Done Sent Done false
7036 2848 Acked Read 7 Done Sent Done false
7037 2849 Acked Read 10 Done Sent Done false
7038 2850 Acked Flush 0 Done Sent Done false
7039 2851 Acked Write 10 Done Sent Done false
7040 2852 Acked Read 4 Done Sent Done false
7041 2853 Acked Read 6 Done Sent Done false
7042 2854 Acked Read 6 Done Sent Done false
7043 2855 Acked Write 6 Done Sent Done false
7044 2856 Acked Read 1 Done Sent Done false
7045 2857 Acked Write 2 Done Sent Done false
7046 2858 Acked Read 3 Done Sent Done false
7047 2859 Acked Flush 0 Done Sent Done false
7048 2860 Acked Read 1 Done Sent Done false
7049 2861 Acked Write 4 Done Sent Done false
7050 2862 Acked Flush 0 Done Sent Done false
7051 2863 Acked Write 4 Done Sent Done false
7052 2864 Acked Read 5 Done Sent Done false
7053 2865 Acked Read 1 Done Sent Done false
7054 2866 Acked Flush 0 Done Sent Done false
7055 2867 Acked Write 10 Done Sent Done false
7056 2868 Acked Write 3 Done Sent Done false
7057 2869 Acked Flush 0 Done Sent Done false
7058 2870 Acked Read 6 Done Sent Done false
7059 2871 Acked Read 2 Done Sent Done false
7060 2872 Acked Read 5 Done Sent Done false
7061 2873 Acked Read 2 Done Sent Done false
7062 2874 Acked Write 9 Done Sent Done false
7063 2875 Acked Write 7 Done Sent Done false
7064 2876 Acked Read 8 Done Sent Done false
7065 2877 Acked Write 10 Done Sent Done false
7066 2878 Acked Write 5 Done Sent Done false
7067 2879 Acked Read 10 Done Sent Done false
7068 2880 Acked Write 8 Done Sent Done false
7069 2881 Acked Write 9 Done Sent Done false
7070 2882 Acked Read 6 Done Sent Done false
7071 2883 Acked Write 8 Done Sent Done false
7072 2884 Acked Read 8 Done Sent Done false
7073 2885 Acked Write 9 Done Sent Done false
7074 2886 Acked Flush 0 Done Sent Done false
7075 2887 Acked Read 3 Done Sent Done false
7076 2888 Acked Read 5 Done Sent Done false
7077 2889 Acked Read 6 Done Sent Done false
7078 2890 Acked Read 9 Done Sent Done false
7079 2891 Acked Read 7 Done Sent Done false
7080 2892 Acked Write 8 Done Sent Done false
7081 2893 Acked Read 3 Done Sent Done false
7082 2894 Acked Read 8 Done Sent Done false
7083 2895 Acked Write 3 Done Sent Done false
7084 2896 Acked Read 8 Done Sent Done false
7085 2897 Acked Flush 0 Done Sent Done false
7086 2898 Acked Read 2 Done Sent Done false
7087 2899 Acked Flush 0 Done Sent Done false
7088 2900 Acked Read 6 Done Sent Done false
7089 2901 Acked Write 9 Done Sent Done false
7090 2902 Acked Flush 0 Done Sent Done false
7091 2903 Acked Read 4 Done Sent Done false
7092 2904 Acked Write 9 Done Sent Done false
7093 2905 Acked Write 8 Done Sent Done false
7094 2906 Acked Write 8 Done Sent Done false
7095 2907 Acked Read 8 Done Sent Done false
7096 2908 Acked Read 8 Done Sent Done false
7097 2909 Acked Read 3 Done Sent Done false
7098 2910 Acked Read 7 Done Sent Done false
7099 2911 Acked Read 7 Done Sent Done false
7100 2912 Acked Write 5 Done Sent Done false
7101 2913 Acked Read 2 Done Sent Done false
7102 2914 Acked Write 4 Done Sent Done false
7103 2915 Acked Read 4 Done Sent Done false
7104 2916 Acked Write 7 Done Sent Done false
7105 2917 Acked Read 6 Done Sent Done false
7106 2918 Acked Read 6 Done Sent Done false
7107 2919 Acked Read 2 Done Sent Done false
7108 2920 Acked Write 1 Done Sent Done false
7109 2921 Acked Write 5 Done Sent Done false
7110 2922 Acked Write 4 Done Sent Done false
7111 2923 Acked Read 2 Done Sent Done false
7112 2924 Acked Read 3 Done Sent Done false
7113 2925 Acked Read 8 Done Sent Done false
7114 2926 Acked Read 5 Done Sent Done false
7115 2927 Acked Flush 0 Done Sent Done false
7116 2928 Acked Read 5 Done Sent Done false
7117 2929 Acked Flush 0 Done Sent Done false
7118 2930 Acked Read 9 Done Sent Done false
7119 2931 Acked Read 9 Done Sent Done false
7120 2932 Acked Read 7 Done Sent Done false
7121 2933 Acked Read 1 Done Sent Done false
7122 2934 Acked Flush 0 Done Sent Done false
7123 2935 Acked Write 1 Done Sent Done false
7124 2936 Acked Read 8 Done Sent Done false
7125 2937 Acked Read 10 Done Sent Done false
7126 2938 Acked Write 9 Done Sent Done false
7127 2939 Acked Read 2 Done Sent Done false
7128 2940 Acked Read 8 Done Sent Done false
7129 2941 Acked Read 1 Done Sent Done false
7130 2942 Acked Write 3 Done Sent Done false
7131 2943 Acked Flush 0 Done Sent Done false
7132 2944 Acked Read 6 Done Sent Done false
7133 2945 Acked Read 10 Done Sent Done false
7134 2946 Acked Write 1 Done Sent Done false
7135 2947 Acked Write 6 Done Sent Done false
7136 2948 Acked Read 4 Done Sent Done false
7137 2949 Acked Read 9 Done Sent Done false
7138 2950 Acked Read 10 Done Sent Done false
7139 2951 Acked Read 9 Done Sent Done false
7140 2952 Acked Read 1 Done Sent Done false
7141 2953 Acked Read 8 Done Sent Done false
7142 2954 Acked Write 4 Done Sent Done false
7143 2955 Acked Read 6 Done Sent Done false
7144 2956 Acked Write 3 Done Sent Done false
7145 2957 Acked Write 2 Done Sent Done false
7146 2958 Acked Write 1 Done Sent Done false
7147 2959 Acked Write 9 Done Sent Done false
7148 2960 Acked Read 2 Done Sent Done false
7149 2961 Acked Read 4 Done Sent Done false
7150 2962 Acked Write 5 Done Sent Done false
7151 2963 Acked Read 10 Done Sent Done false
7152 2964 Acked Write 5 Done Sent Done false
7153 2965 Acked Read 9 Done Sent Done false
7154 2966 Acked Read 7 Done Sent Done false
7155 2967 Acked Flush 0 Done Sent Done false
7156 2968 Acked Read 7 Done Sent Done false
7157 2969 Acked Read 8 Done Sent Done false
7158 2970 Acked Read 7 Done Sent Done false
7159 2971 Acked Read 8 Done Sent Done false
7160 2972 Acked Read 10 Done Sent Done false
7161 2973 Acked Read 4 Done Sent Done false
7162 2974 Acked Write 1 Done Sent Done false
7163 2975 Acked Read 10 Done Sent Done false
7164 2976 Acked Read 4 Done Sent Done false
7165 2977 Acked Read 1 Done Sent Done false
7166 2978 Acked Read 2 Done Sent Done false
7167 2979 Acked Write 2 Done Sent Done false
7168 2980 Acked Write 1 Done Sent Done false
7169 2981 Acked Write 10 Done Sent Done false
7170 2982 Acked Write 7 Done Sent Done false
7171 2983 Acked Write 8 Done Sent Done false
7172 2984 Acked Read 3 Done Sent Done false
7173 2985 Acked Write 7 Done Sent Done false
7174 2986 Acked Write 7 Done Sent Done false
7175 2987 Acked Flush 0 Done Sent Done false
7176 2988 Acked Write 4 Done Sent Done false
7177 2989 Acked Read 2 Done Sent Done false
7178 2990 Acked Write 10 Done Sent Done false
7179 2991 Acked Flush 0 Done Sent Done false
7180 2992 Acked Read 7 Done Sent Done false
7181 2993 Acked Write 5 Done Sent Done false
7182 2994 Acked Read 2 Done Sent Done false
7183 2995 Acked Write 1 Done Sent Done false
7184 2996 Acked Write 4 Done Sent Done false
7185 2997 Acked Read 8 Done Sent Done false
7186 2998 Acked Write 1 Done Sent Done false
7187 2999 Acked Read 7 Done Sent Done false
7188 3000 Acked Write 9 Done Sent Done false
7189 3001 Acked Write 6 Done Sent Done false
7190 3002 Acked Read 6 Done Sent Done false
7191 3003 Acked Read 1 Done Sent Done false
7192 3004 Acked Read 5 Done Sent Done false
7193 3005 Acked Write 10 Done Sent Done false
7194 3006 Acked Write 2 Done Sent Done false
7195 3007 Acked Write 6 Done Sent Done false
7196 3008 Acked Write 8 Done Sent Done false
7197 3009 Acked Write 10 Done Sent Done false
7198 3010 Acked Read 1 Done Sent Done false
7199 3011 Acked Write 5 Done Sent Done false
7200 3012 Acked Read 10 Done Sent Done false
7201 3013 Acked Read 9 Done Sent Done false
7202 3014 Acked Write 10 Done Sent Done false
7203 3015 Acked Read 6 Done Sent Done false
7204 3016 Acked Read 1 Done Sent Done false
7205 3017 Acked Read 7 Done Sent Done false
7206 3018 Acked Write 4 Done Sent Done false
7207 3019 Acked Write 10 Done Sent Done false
7208 3020 Acked Write 5 Done Sent Done false
7209 3021 Acked Write 6 Done Sent Done false
7210 3022 Acked Write 3 Done Sent Done false
7211 3023 Acked Write 8 Done Sent Done false
7212 3024 Acked Flush 0 Done Sent Done false
7213 3025 Acked Read 6 Done Sent Done false
7214 3026 Acked Write 2 Done Sent Done false
7215 3027 Acked Read 6 Done Sent Done false
7216 3028 Acked Write 6 Done Sent Done false
7217 3029 Acked Write 4 Done Sent Done false
7218 3030 Acked Write 1 Done Sent Done false
7219 3031 Acked Read 3 Done Sent Done false
7220 3032 Acked Write 5 Done Sent Done false
7221 3033 Acked Write 3 Done Sent Done false
7222 3034 Acked Write 4 Done Sent Done false
7223 3035 Acked Read 5 Done Sent Done false
7224 3036 Acked Read 3 Done Sent Done false
7225 3037 Acked Write 7 Done Sent Done false
7226 3038 Acked Read 9 Done Sent Done false
7227 3039 Acked Read 4 Done Sent Done false
7228 3040 Acked Write 4 Done Sent Done false
7229 3041 Acked Flush 0 Done Sent Done false
7230 3042 Acked Write 1 Done Sent Done false
7231 3043 Acked Read 4 Done Sent Done false
7232 3044 Acked Write 3 Done Sent Done false
7233 3045 Acked Read 8 Done Sent Done false
7234 3046 Acked Read 6 Done Sent Done false
7235 3047 Acked Read 3 Done Sent Done false
7236 3048 Acked Write 4 Done Sent Done false
7237 3049 Acked Write 5 Done Sent Done false
7238 3050 Acked Read 3 Done Sent Done false
7239 3051 Acked Write 7 Done Sent Done false
7240 3052 Acked Flush 0 Done Sent Done false
7241 3053 Acked Write 4 Done Sent Done false
7242 3054 Acked Read 2 Done Sent Done false
7243 3055 Acked Write 2 Done Sent Done false
7244 3056 Acked Read 2 Done Sent Done false
7245 3057 Acked Write 4 Done Sent Done false
7246 3058 Acked Write 2 Done Sent Done false
7247 3059 Acked Write 10 Done Sent Done false
7248 3060 Acked Read 7 Done Sent Done false
7249 3061 Acked Write 8 Done Sent Done false
7250 3062 Acked Write 3 Done Sent Done false
7251 3063 Acked Flush 0 Done Sent Done false
7252 3064 Acked Read 6 Done Sent Done false
7253 3065 Acked Read 9 Done Sent Done false
7254 3066 Acked Write 10 Done Sent Done false
7255 3067 Acked Write 5 Done Sent Done false
7256 3068 Acked Write 3 Done Sent Done false
7257 3069 Acked Write 3 Done Sent Done false
7258 3070 Acked Write 7 Done Sent Done false
7259 3071 Acked Flush 0 Done Sent Done false
7260 3072 Acked Read 3 Done Sent Done false
7261 3073 Acked Read 8 Done Sent Done false
7262 3074 Acked Read 1 Done Sent Done false
7263 3075 Acked Read 4 Done Sent Done false
7264 3076 Acked Read 5 Done Sent Done false
7265 3077 Acked Write 2 Done Sent Done false
7266 3078 Acked Read 2 Done Sent Done false
7267 3079 Acked Read 9 Done Sent Done false
7268 3080 Acked Write 1 Done Sent Done false
7269 3081 Acked Write 10 Done Sent Done false
7270 3082 Acked Write 5 Done Sent Done false
7271 3083 Acked Read 10 Done Sent Done false
7272 3084 Acked Write 8 Done Sent Done false
7273 3085 Acked Write 2 Done Sent Done false
7274 3086 Acked Write 6 Done Sent Done false
7275 3087 Acked Write 2 Done Sent Done false
7276 3088 Acked Write 4 Done Sent Done false
7277 3089 Acked Read 7 Done Sent Done false
7278 3090 Acked Read 8 Done Sent Done false
7279 3091 Acked Read 3 Done Sent Done false
7280 3092 Acked Write 5 Done Sent Done false
7281 3093 Acked Read 5 Done Sent Done false
7282 3094 Acked Read 10 Done Sent Done false
7283 3095 Acked Read 8 Done Sent Done false
7284 3096 Acked Write 7 Done Sent Done false
7285 3097 Acked Write 4 Done Sent Done false
7286 3098 Acked Read 10 Done Sent Done false
7287 3099 Acked Write 6 Done Sent Done false
7288 3100 Acked Write 3 Done Sent Done false
7289 3101 Acked Write 8 Done Sent Done false
7290 3102 Acked Read 4 Done Sent Done false
7291 3103 Acked Write 2 Done Sent Done false
7292 3104 Acked Write 3 Done Sent Done false
7293 3105 Acked Read 1 Done Sent Done false
7294 3106 Acked Write 8 Done Sent Done false
7295 3107 Acked Write 10 Done Sent Done false
7296 3108 Acked Read 3 Done Sent Done false
7297 3109 Acked Read 7 Done Sent Done false
7298 3110 Acked Read 6 Done Sent Done false
7299 3111 Acked Flush 0 Done Sent Done false
7300 STATES DS:0 DS:1 DS:2 TOTAL
7301 Sent 0 302 0 302
7302 Done 302 0 302 604
7303 Skipped 0 0 0 0
7304 Error 0 0 0 0
7305 Last Flush: 3111 2809 3111
7306 Downstairs last five completed: 2809 2808 2807 2806 2805
7307 Upstairs last five completed: 3111 3110 3109 3108 3107
7308 CLIENT: Up:0 ds:302 act:2
7309 ----------------------------------------------------------------
7310 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7311 States: Active Offline Active
7312 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7313 2810 Acked Read 6 Done Sent Done false
7314 2811 Acked Write 8 Done Sent Done false
7315 2812 Acked Read 3 Done Sent Done false
7316 2813 Acked Read 7 Done Sent Done false
7317 2814 Acked Read 9 Done Sent Done false
7318 2815 Acked Read 10 Done Sent Done false
7319 2816 Acked Read 8 Done Sent Done false
7320 2817 Acked Write 1 Done Sent Done false
7321 2818 Acked Write 9 Done Sent Done false
7322 2819 Acked Write 9 Done Sent Done false
7323 2820 Acked Flush 0 Done Sent Done false
7324 2821 Acked Read 5 Done Sent Done false
7325 2822 Acked Read 9 Done Sent Done false
7326 2823 Acked Read 8 Done Sent Done false
7327 2824 Acked Read 8 Done Sent Done false
7328 2825 Acked Write 6 Done Sent Done false
7329 2826 Acked Read 9 Done Sent Done false
7330 2827 Acked Read 1 Done Sent Done false
7331 2828 Acked Read 1 Done Sent Done false
7332 2829 Acked Write 9 Done Sent Done false
7333 2830 Acked Write 1 Done Sent Done false
7334 2831 Acked Write 2 Done Sent Done false
7335 2832 Acked Write 2 Done Sent Done false
7336 2833 Acked Read 8 Done Sent Done false
7337 2834 Acked Write 2 Done Sent Done false
7338 2835 Acked Read 4 Done Sent Done false
7339 2836 Acked Write 5 Done Sent Done false
7340 2837 Acked Write 1 Done Sent Done false
7341 2838 Acked Read 8 Done Sent Done false
7342 2839 Acked Write 5 Done Sent Done false
7343 2840 Acked Write 7 Done Sent Done false
7344 2841 Acked Read 3 Done Sent Done false
7345 2842 Acked Read 8 Done Sent Done false
7346 2843 Acked Read 1 Done Sent Done false
7347 2844 Acked Read 10 Done Sent Done false
7348 2845 Acked Flush 0 Done Sent Done false
7349 2846 Acked Read 4 Done Sent Done false
7350 2847 Acked Read 8 Done Sent Done false
7351 2848 Acked Read 7 Done Sent Done false
7352 2849 Acked Read 10 Done Sent Done false
7353 2850 Acked Flush 0 Done Sent Done false
7354 2851 Acked Write 10 Done Sent Done false
7355 2852 Acked Read 4 Done Sent Done false
7356 2853 Acked Read 6 Done Sent Done false
7357 2854 Acked Read 6 Done Sent Done false
7358 2855 Acked Write 6 Done Sent Done false
7359 2856 Acked Read 1 Done Sent Done false
7360 2857 Acked Write 2 Done Sent Done false
7361 2858 Acked Read 3 Done Sent Done false
7362 2859 Acked Flush 0 Done Sent Done false
7363 2860 Acked Read 1 Done Sent Done false
7364 2861 Acked Write 4 Done Sent Done false
7365 2862 Acked Flush 0 Done Sent Done false
7366 2863 Acked Write 4 Done Sent Done false
7367 2864 Acked Read 5 Done Sent Done false
7368 2865 Acked Read 1 Done Sent Done false
7369 2866 Acked Flush 0 Done Sent Done false
7370 2867 Acked Write 10 Done Sent Done false
7371 2868 Acked Write 3 Done Sent Done false
7372 2869 Acked Flush 0 Done Sent Done false
7373 2870 Acked Read 6 Done Sent Done false
7374 2871 Acked Read 2 Done Sent Done false
7375 2872 Acked Read 5 Done Sent Done false
7376 2873 Acked Read 2 Done Sent Done false
7377 2874 Acked Write 9 Done Sent Done false
7378 2875 Acked Write 7 Done Sent Done false
7379 2876 Acked Read 8 Done Sent Done false
7380 2877 Acked Write 10 Done Sent Done false
7381 2878 Acked Write 5 Done Sent Done false
7382 2879 Acked Read 10 Done Sent Done false
7383 2880 Acked Write 8 Done Sent Done false
7384 2881 Acked Write 9 Done Sent Done false
7385 2882 Acked Read 6 Done Sent Done false
7386 2883 Acked Write 8 Done Sent Done false
7387 2884 Acked Read 8 Done Sent Done false
7388 2885 Acked Write 9 Done Sent Done false
7389 2886 Acked Flush 0 Done Sent Done false
7390 2887 Acked Read 3 Done Sent Done false
7391 2888 Acked Read 5 Done Sent Done false
7392 2889 Acked Read 6 Done Sent Done false
7393 2890 Acked Read 9 Done Sent Done false
7394 2891 Acked Read 7 Done Sent Done false
7395 2892 Acked Write 8 Done Sent Done false
7396 2893 Acked Read 3 Done Sent Done false
7397 2894 Acked Read 8 Done Sent Done false
7398 2895 Acked Write 3 Done Sent Done false
7399 2896 Acked Read 8 Done Sent Done false
7400 2897 Acked Flush 0 Done Sent Done false
7401 2898 Acked Read 2 Done Sent Done false
7402 2899 Acked Flush 0 Done Sent Done false
7403 2900 Acked Read 6 Done Sent Done false
7404 2901 Acked Write 9 Done Sent Done false
7405 2902 Acked Flush 0 Done Sent Done false
7406 2903 Acked Read 4 Done Sent Done false
7407 2904 Acked Write 9 Done Sent Done false
7408 2905 Acked Write 8 Done Sent Done false
7409 2906 Acked Write 8 Done Sent Done false
7410 2907 Acked Read 8 Done Sent Done false
7411 2908 Acked Read 8 Done Sent Done false
7412 2909 Acked Read 3 Done Sent Done false
7413 2910 Acked Read 7 Done Sent Done false
7414 2911 Acked Read 7 Done Sent Done false
7415 2912 Acked Write 5 Done Sent Done false
7416 2913 Acked Read 2 Done Sent Done false
7417 2914 Acked Write 4 Done Sent Done false
7418 2915 Acked Read 4 Done Sent Done false
7419 2916 Acked Write 7 Done Sent Done false
7420 2917 Acked Read 6 Done Sent Done false
7421 2918 Acked Read 6 Done Sent Done false
7422 2919 Acked Read 2 Done Sent Done false
7423 2920 Acked Write 1 Done Sent Done false
7424 2921 Acked Write 5 Done Sent Done false
7425 2922 Acked Write 4 Done Sent Done false
7426 2923 Acked Read 2 Done Sent Done false
7427 2924 Acked Read 3 Done Sent Done false
7428 2925 Acked Read 8 Done Sent Done false
7429 2926 Acked Read 5 Done Sent Done false
7430 2927 Acked Flush 0 Done Sent Done false
7431 2928 Acked Read 5 Done Sent Done false
7432 2929 Acked Flush 0 Done Sent Done false
7433 2930 Acked Read 9 Done Sent Done false
7434 2931 Acked Read 9 Done Sent Done false
7435 2932 Acked Read 7 Done Sent Done false
7436 2933 Acked Read 1 Done Sent Done false
7437 2934 Acked Flush 0 Done Sent Done false
7438 2935 Acked Write 1 Done Sent Done false
7439 2936 Acked Read 8 Done Sent Done false
7440 2937 Acked Read 10 Done Sent Done false
7441 2938 Acked Write 9 Done Sent Done false
7442 2939 Acked Read 2 Done Sent Done false
7443 2940 Acked Read 8 Done Sent Done false
7444 2941 Acked Read 1 Done Sent Done false
7445 2942 Acked Write 3 Done Sent Done false
7446 2943 Acked Flush 0 Done Sent Done false
7447 2944 Acked Read 6 Done Sent Done false
7448 2945 Acked Read 10 Done Sent Done false
7449 2946 Acked Write 1 Done Sent Done false
7450 2947 Acked Write 6 Done Sent Done false
7451 2948 Acked Read 4 Done Sent Done false
7452 2949 Acked Read 9 Done Sent Done false
7453 2950 Acked Read 10 Done Sent Done false
7454 2951 Acked Read 9 Done Sent Done false
7455 2952 Acked Read 1 Done Sent Done false
7456 2953 Acked Read 8 Done Sent Done false
7457 2954 Acked Write 4 Done Sent Done false
7458 2955 Acked Read 6 Done Sent Done false
7459 2956 Acked Write 3 Done Sent Done false
7460 2957 Acked Write 2 Done Sent Done false
7461 2958 Acked Write 1 Done Sent Done false
7462 2959 Acked Write 9 Done Sent Done false
7463 2960 Acked Read 2 Done Sent Done false
7464 2961 Acked Read 4 Done Sent Done false
7465 2962 Acked Write 5 Done Sent Done false
7466 2963 Acked Read 10 Done Sent Done false
7467 2964 Acked Write 5 Done Sent Done false
7468 2965 Acked Read 9 Done Sent Done false
7469 2966 Acked Read 7 Done Sent Done false
7470 2967 Acked Flush 0 Done Sent Done false
7471 2968 Acked Read 7 Done Sent Done false
7472 2969 Acked Read 8 Done Sent Done false
7473 2970 Acked Read 7 Done Sent Done false
7474 2971 Acked Read 8 Done Sent Done false
7475 2972 Acked Read 10 Done Sent Done false
7476 2973 Acked Read 4 Done Sent Done false
7477 2974 Acked Write 1 Done Sent Done false
7478 2975 Acked Read 10 Done Sent Done false
7479 2976 Acked Read 4 Done Sent Done false
7480 2977 Acked Read 1 Done Sent Done false
7481 2978 Acked Read 2 Done Sent Done false
7482 2979 Acked Write 2 Done Sent Done false
7483 2980 Acked Write 1 Done Sent Done false
7484 2981 Acked Write 10 Done Sent Done false
7485 2982 Acked Write 7 Done Sent Done false
7486 2983 Acked Write 8 Done Sent Done false
7487 2984 Acked Read 3 Done Sent Done false
7488 2985 Acked Write 7 Done Sent Done false
7489 2986 Acked Write 7 Done Sent Done false
7490 2987 Acked Flush 0 Done Sent Done false
7491 2988 Acked Write 4 Done Sent Done false
7492 2989 Acked Read 2 Done Sent Done false
7493 2990 Acked Write 10 Done Sent Done false
7494 2991 Acked Flush 0 Done Sent Done false
7495 2992 Acked Read 7 Done Sent Done false
7496 2993 Acked Write 5 Done Sent Done false
7497 2994 Acked Read 2 Done Sent Done false
7498 2995 Acked Write 1 Done Sent Done false
7499 2996 Acked Write 4 Done Sent Done false
7500 2997 Acked Read 8 Done Sent Done false
7501 2998 Acked Write 1 Done Sent Done false
7502 2999 Acked Read 7 Done Sent Done false
7503 3000 Acked Write 9 Done Sent Done false
7504 3001 Acked Write 6 Done Sent Done false
7505 3002 Acked Read 6 Done Sent Done false
7506 3003 Acked Read 1 Done Sent Done false
7507 3004 Acked Read 5 Done Sent Done false
7508 3005 Acked Write 10 Done Sent Done false
7509 3006 Acked Write 2 Done Sent Done false
7510 3007 Acked Write 6 Done Sent Done false
7511 3008 Acked Write 8 Done Sent Done false
7512 3009 Acked Write 10 Done Sent Done false
7513 3010 Acked Read 1 Done Sent Done false
7514 3011 Acked Write 5 Done Sent Done false
7515 3012 Acked Read 10 Done Sent Done false
7516 3013 Acked Read 9 Done Sent Done false
7517 3014 Acked Write 10 Done Sent Done false
7518 3015 Acked Read 6 Done Sent Done false
7519 3016 Acked Read 1 Done Sent Done false
7520 3017 Acked Read 7 Done Sent Done false
7521 3018 Acked Write 4 Done Sent Done false
7522 3019 Acked Write 10 Done Sent Done false
7523 3020 Acked Write 5 Done Sent Done false
7524 3021 Acked Write 6 Done Sent Done false
7525 3022 Acked Write 3 Done Sent Done false
7526 3023 Acked Write 8 Done Sent Done false
7527 3024 Acked Flush 0 Done Sent Done false
7528 3025 Acked Read 6 Done Sent Done false
7529 3026 Acked Write 2 Done Sent Done false
7530 3027 Acked Read 6 Done Sent Done false
7531 3028 Acked Write 6 Done Sent Done false
7532 3029 Acked Write 4 Done Sent Done false
7533 3030 Acked Write 1 Done Sent Done false
7534 3031 Acked Read 3 Done Sent Done false
7535 3032 Acked Write 5 Done Sent Done false
7536 3033 Acked Write 3 Done Sent Done false
7537 3034 Acked Write 4 Done Sent Done false
7538 3035 Acked Read 5 Done Sent Done false
7539 3036 Acked Read 3 Done Sent Done false
7540 3037 Acked Write 7 Done Sent Done false
7541 3038 Acked Read 9 Done Sent Done false
7542 3039 Acked Read 4 Done Sent Done false
7543 3040 Acked Write 4 Done Sent Done false
7544 3041 Acked Flush 0 Done Sent Done false
7545 3042 Acked Write 1 Done Sent Done false
7546 3043 Acked Read 4 Done Sent Done false
7547 3044 Acked Write 3 Done Sent Done false
7548 3045 Acked Read 8 Done Sent Done false
7549 3046 Acked Read 6 Done Sent Done false
7550 3047 Acked Read 3 Done Sent Done false
7551 3048 Acked Write 4 Done Sent Done false
7552 3049 Acked Write 5 Done Sent Done false
7553 3050 Acked Read 3 Done Sent Done false
7554 3051 Acked Write 7 Done Sent Done false
7555 3052 Acked Flush 0 Done Sent Done false
7556 3053 Acked Write 4 Done Sent Done false
7557 3054 Acked Read 2 Done Sent Done false
7558 3055 Acked Write 2 Done Sent Done false
7559 3056 Acked Read 2 Done Sent Done false
7560 3057 Acked Write 4 Done Sent Done false
7561 3058 Acked Write 2 Done Sent Done false
7562 3059 Acked Write 10 Done Sent Done false
7563 3060 Acked Read 7 Done Sent Done false
7564 3061 Acked Write 8 Done Sent Done false
7565 3062 Acked Write 3 Done Sent Done false
7566 3063 Acked Flush 0 Done Sent Done false
7567 3064 Acked Read 6 Done Sent Done false
7568 3065 Acked Read 9 Done Sent Done false
7569 3066 Acked Write 10 Done Sent Done false
7570 3067 Acked Write 5 Done Sent Done false
7571 3068 Acked Write 3 Done Sent Done false
7572 3069 Acked Write 3 Done Sent Done false
7573 3070 Acked Write 7 Done Sent Done false
7574 3071 Acked Flush 0 Done Sent Done false
7575 3072 Acked Read 3 Done Sent Done false
7576 3073 Acked Read 8 Done Sent Done false
7577 3074 Acked Read 1 Done Sent Done false
7578 3075 Acked Read 4 Done Sent Done false
7579 3076 Acked Read 5 Done Sent Done false
7580 3077 Acked Write 2 Done Sent Done false
7581 3078 Acked Read 2 Done Sent Done false
7582 3079 Acked Read 9 Done Sent Done false
7583 3080 Acked Write 1 Done Sent Done false
7584 3081 Acked Write 10 Done Sent Done false
7585 3082 Acked Write 5 Done Sent Done false
7586 3083 Acked Read 10 Done Sent Done false
7587 3084 Acked Write 8 Done Sent Done false
7588 3085 Acked Write 2 Done Sent Done false
7589 3086 Acked Write 6 Done Sent Done false
7590 3087 Acked Write 2 Done Sent Done false
7591 3088 Acked Write 4 Done Sent Done false
7592 3089 Acked Read 7 Done Sent Done false
7593 3090 Acked Read 8 Done Sent Done false
7594 3091 Acked Read 3 Done Sent Done false
7595 3092 Acked Write 5 Done Sent Done false
7596 3093 Acked Read 5 Done Sent Done false
7597 3094 Acked Read 10 Done Sent Done false
7598 3095 Acked Read 8 Done Sent Done false
7599 3096 Acked Write 7 Done Sent Done false
7600 3097 Acked Write 4 Done Sent Done false
7601 3098 Acked Read 10 Done Sent Done false
7602 3099 Acked Write 6 Done Sent Done false
7603 3100 Acked Write 3 Done Sent Done false
7604 3101 Acked Write 8 Done Sent Done false
7605 3102 Acked Read 4 Done Sent Done false
7606 3103 Acked Write 2 Done Sent Done false
7607 3104 Acked Write 3 Done Sent Done false
7608 3105 Acked Read 1 Done Sent Done false
7609 3106 Acked Write 8 Done Sent Done false
7610 3107 Acked Write 10 Done Sent Done false
7611 3108 Acked Read 3 Done Sent Done false
7612 3109 Acked Read 7 Done Sent Done false
7613 3110 Acked Read 6 Done Sent Done false
7614 3111 Acked Flush 0 Done Sent Done false
7615 STATES DS:0 DS:1 DS:2 TOTAL
7616 Sent 0 302 0 302
7617 Done 302 0 302 604
7618 Skipped 0 0 0 0
7619 Error 0 0 0 0
7620 Last Flush: 3111 2809 3111
7621 Downstairs last five completed: 2809 2808 2807 2806 2805
7622 Upstairs last five completed: 3111 3110 3109 3108 3107
7623 CLIENT: Up:0 ds:302 act:2
76242025-01-14T16:12:13.401ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
76252025-01-14T16:12:13.401ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62391) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
76262025-01-14T16:12:13.402ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
76272025-01-14T16:12:13.403ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
76282025-01-14T16:12:13.403ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
76292025-01-14T16:12:13.403ZINFOcrucible: send last flush ID to this DS: 2809 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
76302025-01-14T16:12:13.403ZINFOcrucible: Replied this last flush ID: 2809 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
76312025-01-14T16:12:13.403ZINFOcrucible: [1] client re-new 302 jobs since flush 2809 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
76322025-01-14T16:12:13.403ZINFOcrucible: [1] Replayed 302 jobs since flush: 2809 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
7633 ----------------------------------------------------------------
7634 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
7635 Downstairs last five completed: 3111 3110 3109 3108 3107
7636 Upstairs last five completed: 3111 3110 3109 3108 3107
7637 CLIENT: Up:0 ds:0 act:3
7638 Replay: All jobs finished
7639 Main task has work to do, go find it
7640 got dsc Stop(1)
7641 stop 1
7642 [8820] Got stop action so:false kr:false
7643 Jan 14 16:12:13.644 INFO request completed, latency_us: 305, response_code: 204, uri: /stop/cid/1, method: POST, req_id: c6e5e429-c899-4be7-b66b-7e0d7846b1a8, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
76442025-01-14T16:12:13.645ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
76452025-01-14T16:12:13.645ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
76462025-01-14T16:12:13.645ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
76472025-01-14T16:12:13.645ZINFOcrucible: client task is exiting = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
7648 Jan 14 16:12:13.645 INFO request completed, latency_us: 135, response_code: 200, uri: /state/cid/1, method: GET, req_id: 1d4ea8ff-7866-437d-b5f7-c4a1bdb77e90, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
7649 [8820] Exited with: ExitStatus(unix_wait_status(9))
7650 [8820][1] reports Exit
7651 Jan 14 16:12:17.648 INFO request completed, latency_us: 197, response_code: 200, uri: /state/cid/1, method: GET, req_id: bc9ec554-074b-4cfe-b6a0-6fc960d08221, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
7652 001/300 Read block 738 len 24576
7653 002/300 Read block 4070 len 32768
7654 003/300 Write block 491 len 4096 data: 3
7655 004/300 Read block 1040 len 12288
7656 005/300 Write block 620 len 24576 data: 3 3 3 3 3 3
7657 006/300 Read block 1789 len 16384
7658 007/300 Read block 4544 len 8192
7659 008/300 Read block 3240 len 8192
7660 009/300 Read block 1947 len 36864
7661 010/300 Flush
7662 011/300 Write block 738 len 28672 data: 3 3 3 3 3 3 3
7663 012/300 Read block 3347 len 36864
7664 013/300 Read block 1913 len 16384
7665 014/300 Read block 1840 len 8192
7666 015/300 Write block 3608 len 32768 data: 3 3 3 3 3 3 3 2
7667 016/300 Flush
7668 017/300 Read block 1494 len 8192
7669 018/300 Flush
7670 019/300 Read block 984 len 36864
7671 020/300 Write block 422 len 4096 data: 6
7672 021/300 Read block 4388 len 12288
7673 022/300 Write block 2089 len 28672 data: 2 2 2 2 2 2 2
7674 023/300 Write block 3192 len 8192 data: 3 4
7675 024/300 Write block 2303 len 16384 data: 2 2 2 2
7676 025/300 Read block 3131 len 4096
7677 026/300 Read block 2945 len 12288
7678 027/300 Read block 375 len 36864
7679 028/300 Write block 3658 len 4096 data: 3
7680 029/300 Write block 645 len 4096 data: 4
7681 030/300 Read block 3487 len 40960
7682 031/300 Read block 3231 len 36864
7683 032/300 Write block 2605 len 8192 data: 3 3
7684 033/300 Write block 374 len 12288 data: 3 3 4
7685 034/300 Write block 348 len 32768 data: 5 5 5 2 2 2 2 2
7686 035/300 Write block 2174 len 8192 data: 2 2
7687 036/300 Read block 3905 len 16384
7688 037/300 Flush
7689 038/300 Read block 3006 len 28672
7690 039/300 Write block 2352 len 12288 data: 2 2 3
7691 040/300 Read block 428 len 36864
7692 041/300 Write block 4014 len 12288 data: 3 3 3
7693 042/300 Read block 2539 len 40960
7694 043/300 Read block 3733 len 24576
7695 044/300 Flush
7696 045/300 Flush
7697 046/300 Write block 1302 len 28672 data: 3 3 4 4 3 2 2
7698 047/300 Read block 4077 len 16384
7699 048/300 Read block 3496 len 16384
7700 049/300 Write block 4128 len 4096 data: 5
7701 050/300 Read block 2742 len 40960
7702 051/300 Read block 4082 len 20480
7703 052/300 Read block 3614 len 4096
7704 053/300 Write block 3669 len 32768 data: 2 2 2 2 2 2 2 2
7705 054/300 Read block 2512 len 20480
7706 055/300 Write block 2878 len 28672 data: 2 2 2 2 2 2 3
7707 056/300 Write block 1646 len 8192 data: 5 5
7708 057/300 Write block 3175 len 16384 data: 2 2 2 2
7709 058/300 Flush
7710 059/300 Write block 4861 len 12288 data: 3 4 4
7711 060/300 Write block 2156 len 24576 data: 3 3 3 4 4 4
7712 061/300 Write block 1063 len 24576 data: 2 2 2 2 2 2
7713 062/300 Write block 4520 len 16384 data: 3 3 3 2
7714 063/300 Read block 1871 len 8192
7715 064/300 Read block 4553 len 24576
7716 065/300 Read block 832 len 40960
7717 066/300 Read block 4503 len 20480
7718 067/300 Write block 1826 len 40960 data: 4 4 3 3 3 4 4 5 4 4
7719 068/300 Write block 560 len 24576 data: 3 2 2 2 3 2
7720 069/300 Read block 2677 len 8192
7721 070/300 Read block 1009 len 8192
7722 071/300 Write block 3642 len 4096 data: 3
7723 072/300 Write block 2828 len 32768 data: 3 3 4 5 4 4 4 3
7724 073/300 Read block 4992 len 24576
7725 074/300 Read block 4634 len 28672
7726 075/300 Read block 3296 len 36864
7727 076/300 Read block 944 len 4096
7728 077/300 Write block 1746 len 40960 data: 3 3 3 3 3 3 4 5 5 4
7729 078/300 Write block 2186 len 28672 data: 3 3 3 3 4 4 5
7730 079/300 Read block 1356 len 32768
7731 080/300 Write block 792 len 36864 data: 3 3 3 3 4 5 5 7 4
7732 081/300 Read block 1148 len 16384
7733 082/300 Write block 2338 len 28672 data: 2 2 2 2 2 2 2
7734 083/300 Write block 2313 len 20480 data: 3 3 3 4 4
7735 084/300 Write block 1607 len 8192 data: 2 3
7736 085/300 Read block 4878 len 40960
7737 086/300 Read block 2057 len 20480
7738 087/300 Write block 1741 len 32768 data: 3 2 2 2 3 4 4 4
7739 088/300 Read block 4977 len 8192
7740 089/300 Write block 3025 len 20480 data: 4 4 4 5 6
7741 090/300 Write block 3643 len 28672 data: 3 3 3 3 2 2 2
7742 091/300 Read block 3228 len 28672
7743 092/300 Read block 1525 len 4096
7744 093/300 Write block 106 len 24576 data: 2 3 3 3 3 3
7745 094/300 Read block 4472 len 24576
7746 095/300 Read block 576 len 36864
7747 096/300 Write block 253 len 32768 data: 4 4 4 4 3 2 2 3
7748 097/300 Write block 2715 len 8192 data: 4 4
7749 098/300 Write block 236 len 36864 data: 4 4 3 3 3 3 3 3 3
7750 099/300 Read block 4403 len 36864
7751 100/300 Read block 2758 len 40960
7752 101/300 Write block 3227 len 16384 data: 3 4 4 3
7753 102/300 Write block 3920 len 16384 data: 2 2 2 2
7754 103/300 Read block 1715 len 32768
7755 104/300 Write block 1381 len 16384 data: 4 4 3 4
7756 105/300 Write block 142 len 40960 data: 2 3 3 3 3 3 3 3 3 2
7757 106/300 Write block 4364 len 24576 data: 2 3 3 3 3 3
7758 107/300 Write block 547 len 12288 data: 3 3 3
7759 108/300 Read block 2070 len 28672
7760 109/300 Write block 905 len 24576 data: 5 5 5 5 5 5
7761 110/300 Read block 4165 len 4096
7762 111/300 Write block 1961 len 16384 data: 4 4 2 2
7763 112/300 Write block 4855 len 24576 data: 3 3 3 3 3 3
7764 113/300 Read block 2309 len 24576
7765 114/300 Read block 2689 len 28672
7766 115/300 Read block 2771 len 32768
7767 116/300 Read block 2765 len 8192
7768 117/300 Read block 2174 len 40960
7769 118/300 Read block 4840 len 20480
7770 119/300 Write block 3921 len 40960 data: 3 3 3 2 2 2 2 2 2 2
7771 120/300 Read block 2213 len 36864
7772 121/300 Read block 4179 len 16384
7773 122/300 Read block 733 len 28672
7774 123/300 Read block 15 len 20480
7775 124/300 Read block 1161 len 4096
7776 125/300 Read block 2129 len 4096
7777 126/300 Flush
7778 127/300 Write block 2361 len 16384 data: 2 2 2 2
7779 128/300 Read block 552 len 28672
7780 129/300 Read block 2790 len 32768
7781 130/300 Write block 3238 len 24576 data: 3 3 3 3 3 3
7782 131/300 Read block 3133 len 8192
7783 132/300 Flush
7784 133/300 Read block 4548 len 24576
7785 134/300 Write block 1512 len 16384 data: 2 2 2 2
7786 135/300 Flush
7787 136/300 Read block 417 len 20480
7788 137/300 Flush
7789 138/300 Read block 110 len 4096
7790 139/300 Read block 1199 len 16384
7791 140/300 Read block 4302 len 36864
7792 141/300 Write block 4675 len 36864 data: 3 3 3 3 3 3 2 2 2
7793 142/300 Write block 872 len 8192 data: 3 2
7794 143/300 Flush
7795 144/300 Write block 4775 len 12288 data: 5 5 5
7796 145/300 Read block 2563 len 36864
7797 146/300 Read block 4489 len 32768
7798 147/300 Read block 4844 len 28672
7799 148/300 Read block 1436 len 20480
7800 149/300 Write block 4026 len 4096 data: 3
7801 150/300 Write block 514 len 32768 data: 3 3 3 3 3 3 3 3
7802 151/300 Read block 4187 len 8192
7803 152/300 Read block 3864 len 40960
7804 153/300 Read block 2740 len 40960
7805 154/300 Write block 3138 len 24576 data: 3 3 2 3 4 4
7806 155/300 Write block 3794 len 16384 data: 3 3 3 3
7807 156/300 Write block 3087 len 20480 data: 2 2 3 3 3
7808 157/300 Read block 1165 len 4096
7809 158/300 Write block 4046 len 4096 data: 2
7810 159/300 Read block 3861 len 24576
7811 160/300 Read block 1731 len 40960
7812 161/300 Read block 2641 len 36864
7813 162/300 Read block 4097 len 16384
7814 163/300 Read block 1225 len 36864
7815 164/300 Read block 4661 len 16384
7816 165/300 Write block 374 len 12288 data: 4 4 5
7817 166/300 Read block 3483 len 12288
7818 167/300 Write block 1560 len 36864 data: 3 3 2 3 3 3 3 4 4
7819 168/300 Read block 4501 len 28672
7820 169/300 Read block 1435 len 20480
7821 170/300 Write block 335 len 24576 data: 4 3 3 3 2 2
7822 171/300 Write block 2234 len 40960 data: 2 2 2 2 3 3 3 3 3 4
7823 172/300 Read block 1920 len 28672
7824 173/300 Read block 1709 len 32768
7825 174/300 Write block 2702 len 8192 data: 4 5
7826 175/300 Write block 2288 len 24576 data: 3 3 3 3 2 2
7827 176/300 Write block 2661 len 32768 data: 2 2 2 2 2 3 3 3
7828 177/300 Write block 1438 len 36864 data: 3 4 3 3 2 2 2 2 3
7829 178/300 Read block 3583 len 28672
7830 179/300 Read block 727 len 40960
7831 180/300 Read block 1966 len 28672
7832 181/300 Flush
7833 182/300 Write block 1993 len 24576 data: 4 2 3 3 3 3
7834 183/300 Write block 3706 len 20480 data: 3 3 3 3 4
7835 184/300 Write block 4245 len 36864 data: 3 4 4 4 3 3 3 2 2
7836 185/300 Read block 1183 len 16384
7837 186/300 Read block 1100 len 36864
7838 187/300 Read block 768 len 40960
7839 188/300 Write block 1813 len 40960 data: 3 3 3 4 4 4 4 4 3 3
7840 189/300 Read block 3890 len 20480
7841 190/300 Flush
7842 191/300 Read block 3903 len 32768
7843 192/300 Read block 2176 len 32768
7844 193/300 Read block 3410 len 36864
7845 194/300 Write block 461 len 12288 data: 4 4 4
7846 195/300 Read block 3430 len 36864
7847 196/300 Read block 530 len 16384
7848 197/300 Read block 2136 len 40960
7849 198/300 Read block 4132 len 28672
7850 199/300 Write block 1401 len 36864 data: 3 2 3 3 3 2 2 3 3
7851 200/300 Write block 3954 len 20480 data: 2 2 3 3 4
7852 201/300 Write block 2946 len 4096 data: 4
7853 202/300 Write block 3780 len 4096 data: 2
7854 203/300 Write block 2565 len 4096 data: 4
7855 204/300 Write block 160 len 32768 data: 3 3 3 3 3 3 3 3
7856 205/300 Read block 1923 len 40960
7857 206/300 Write block 3239 len 4096 data: 4
7858 207/300 Read block 3856 len 32768
7859 208/300 Read block 2015 len 24576
7860 209/300 Read block 3884 len 40960
7861 210/300 Read block 2906 len 40960
7862 211/300 Write block 711 len 8192 data: 3 3
7863 212/300 Write block 1060 len 36864 data: 2 2 2 3 3 3 3 3 3
7864 213/300 Write block 3727 len 20480 data: 2 2 2 2 2
7865 214/300 Write block 4665 len 12288 data: 5 5 4
7866 215/300 Read block 4 len 12288
7867 216/300 Read block 2377 len 20480
7868 217/300 Write block 4434 len 28672 data: 2 2 2 2 2 2 2
7869 218/300 Flush
7870 219/300 Write block 2839 len 16384 data: 3 2 2 2
7871 220/300 Write block 878 len 40960 data: 2 2 2 3 3 4 4 5 5 5
7872 221/300 Write block 373 len 8192 data: 3 5
7873 222/300 Write block 2312 len 36864 data: 3 4 4 4 5 5 4 4 3
7874 223/300 Flush
7875 224/300 Write block 4867 len 16384 data: 3 3 2 2
7876 225/300 Read block 2418 len 36864
7877 226/300 Read block 848 len 4096
7878 227/300 Write block 2268 len 36864 data: 3 2 2 2 2 2 2 2 2
7879 228/300 Write block 2639 len 12288 data: 3 3 3
7880 229/300 Flush
7881 230/300 Read block 151 len 32768
7882 231/300 Write block 2169 len 36864 data: 2 2 2 2 2 3 3 2 2
7883 232/300 Write block 2912 len 24576 data: 2 2 2 2 2 2
7884 233/300 Flush
7885 234/300 Read block 3518 len 28672
7886 235/300 Read block 4405 len 4096
7887 236/300 Write block 1592 len 20480 data: 4 4 4 4 4
7888 237/300 Read block 3834 len 12288
7889 238/300 Read block 2579 len 8192
7890 239/300 Write block 673 len 20480 data: 4 4 5 4 3
7891 240/300 Read block 4607 len 24576
7892 241/300 Write block 3504 len 12288 data: 2 2 2
7893 242/300 Write block 1741 len 12288 data: 4 3 3
7894 243/300 Read block 3468 len 40960
7895 244/300 Write block 235 len 16384 data: 4 5 5 4
7896 245/300 Write block 4007 len 36864 data: 4 3 2 2 2 2 2 4 4
7897 246/300 Read block 2942 len 32768
7898 247/300 Write block 1463 len 20480 data: 3 4 5 5 5
7899 248/300 Flush
7900 249/300 Write block 851 len 24576 data: 3 3 3 3 3 2
7901 250/300 Write block 1167 len 32768 data: 2 2 2 2 3 3 3 3
7902 251/300 Read block 1001 len 24576
7903 252/300 Write block 1595 len 16384 data: 5 5 4 3
7904 253/300 Write block 187 len 8192 data: 4 4
7905 254/300 Write block 2446 len 4096 data: 2
7906 255/300 Read block 4424 len 20480
7907 256/300 Read block 1264 len 40960
7908 257/300 Read block 3950 len 16384
7909 258/300 Write block 3230 len 32768 data: 4 3 3 2 3 3 3 3
7910 259/300 Read block 2531 len 20480
7911 260/300 Write block 609 len 28672 data: 2 2 2 2 2 3 3
7912 261/300 Read block 2513 len 8192
7913 262/300 Write block 3275 len 32768 data: 2 2 2 2 2 3 4 5
7914 263/300 Write block 3174 len 36864 data: 2 3 3 3 3 2 2 2 2
7915 264/300 Read block 4624 len 36864
7916 265/300 Write block 1297 len 24576 data: 5 4 3 4 3 4
7917 266/300 Read block 3494 len 16384
7918 267/300 Flush
7919 268/300 Write block 4664 len 24576 data: 5 6 6 5 3 2
7920 269/300 Read block 470 len 36864
7921 270/300 Write block 2950 len 8192 data: 3 3
7922 271/300 Write block 731 len 4096 data: 3
7923 272/300 Read block 217 len 12288
7924 273/300 Read block 2133 len 36864
7925 274/300 Flush
7926 275/300 Read block 2250 len 32768
7927 276/300 Write block 4342 len 40960 data: 2 2 2 2 2 2 2 3 3 3
7928 277/300 Write block 998 len 12288 data: 3 3 3
7929 278/300 Write block 794 len 8192 data: 4 4
7930 279/300 Flush
7931 280/300 Write block 3840 len 40960 data: 3 3 3 3 4 4 4 4 4 3
7932 281/300 Write block 2275 len 36864 data: 3 3 2 2 3 4 4 4 5
7933 282/300 Read block 3099 len 40960
7934 283/300 Write block 329 len 40960 data: 4 3 3 3 3 4 5 4 4 4
7935 284/300 Read block 3177 len 36864
7936 285/300 Write block 4097 len 20480 data: 3 3 3 2 3
7937 286/300 Read block 517 len 32768
7938 287/300 Write block 28 len 4096 data: 4
7939 288/300 Write block 4741 len 12288 data: 2 3 3
7940 289/300 Write block 4953 len 32768 data: 4 3 3 4 4 4 4 3
7941 290/300 Write block 3885 len 32768 data: 3 3 3 3 3 3 3 2
7942 291/300 Flush
7943 292/300 Read block 3590 len 20480
7944 293/300 Write block 1748 len 12288 data: 5 4 4
7945 294/300 Write block 4822 len 36864 data: 2 2 2 2 2 2 2 2 3
7946 295/300 Read block 3275 len 24576
7947 296/300 Read block 1264 len 28672
7948 297/300 Read block 2823 len 24576
7949 298/300 Write block 2561 len 28672 data: 5 6 6 6 5 5 6
7950 299/300 Write block 640 len 4096 data: 2
7951 300/300 Read block 3132 len 36864
7952 301/300 Write block 415 len 4096 data: 3
7953 Main task has work to do, go find it
7954 got dsc Start(1)
7955 start 1
7956 Jan 14 16:12:17.883 INFO request completed, latency_us: 203, response_code: 204, uri: /start/cid/1, method: POST, req_id: 48b257ba-718f-4d63-bb33-a5e73c082be0, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
7957 Replay: started 1, returned:Ok(())
7958 ----------------------------------------------------------------
7959 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
7960 States: Active Offline Active
7961 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7962 3112 Acked Read 6 Done Sent Done false
7963 3113 Acked Read 8 Done Sent Done false
7964 3114 Acked Write 1 Done Sent Done false
7965 3115 Acked Read 3 Done Sent Done false
7966 3116 Acked Write 6 Done Sent Done false
7967 3117 Acked Read 4 Done Sent Done false
7968 3118 Acked Read 2 Done Sent Done false
7969 3119 Acked Read 2 Done Sent Done false
7970 3120 Acked Read 9 Done Sent Done false
7971 3121 Acked Flush 0 Done Sent Done false
7972 3122 Acked Write 7 Done Sent Done false
7973 3123 Acked Read 9 Done Sent Done false
7974 3124 Acked Read 4 Done Sent Done false
7975 3125 Acked Read 2 Done Sent Done false
7976 3126 Acked Write 8 Done Sent Done false
7977 3127 Acked Flush 0 Done Sent Done false
7978 3128 Acked Read 2 Done Sent Done false
7979 3129 Acked Flush 0 Done Sent Done false
7980 3130 Acked Read 9 Done Sent Done false
7981 3131 Acked Write 1 Done Sent Done false
7982 3132 Acked Read 3 Done Sent Done false
7983 3133 Acked Write 7 Done Sent Done false
7984 3134 Acked Write 2 Done Sent Done false
7985 3135 Acked Write 4 Done Sent Done false
7986 3136 Acked Read 1 Done Sent Done false
7987 3137 Acked Read 3 Done Sent Done false
7988 3138 Acked Read 9 Done Sent Done false
7989 3139 Acked Write 1 Done Sent Done false
7990 3140 Acked Write 1 Done Sent Done false
7991 3141 Acked Read 10 Done Sent Done false
7992 3142 Acked Read 9 Done Sent Done false
7993 3143 Acked Write 2 Done Sent Done false
7994 3144 Acked Write 3 Done Sent Done false
7995 3145 Acked Write 8 Done Sent Done false
7996 3146 Acked Write 2 Done Sent Done false
7997 3147 Acked Read 4 Done Sent Done false
7998 3148 Acked Flush 0 Done Sent Done false
7999 3149 Acked Read 7 Done Sent Done false
8000 3150 Acked Write 3 Done Sent Done false
8001 3151 Acked Read 9 Done Sent Done false
8002 3152 Acked Write 3 Done Sent Done false
8003 3153 Acked Read 10 Done Sent Done false
8004 3154 Acked Read 6 Done Sent Done false
8005 3155 Acked Flush 0 Done Sent Done false
8006 3156 Acked Flush 0 Done Sent Done false
8007 3157 Acked Write 7 Done Sent Done false
8008 3158 Acked Read 4 Done Sent Done false
8009 3159 Acked Read 4 Done Sent Done false
8010 3160 Acked Write 1 Done Sent Done false
8011 3161 Acked Read 10 Done Sent Done false
8012 3162 Acked Read 5 Done Sent Done false
8013 3163 Acked Read 1 Done Sent Done false
8014 3164 Acked Write 8 Done Sent Done false
8015 3165 Acked Read 5 Done Sent Done false
8016 3166 Acked Write 7 Done Sent Done false
8017 3167 Acked Write 2 Done Sent Done false
8018 3168 Acked Write 4 Done Sent Done false
8019 3169 Acked Flush 0 Done Sent Done false
8020 3170 Acked Write 3 Done Sent Done false
8021 3171 Acked Write 6 Done Sent Done false
8022 3172 Acked Write 6 Done Sent Done false
8023 3173 Acked Write 4 Done Sent Done false
8024 3174 Acked Read 2 Done Sent Done false
8025 3175 Acked Read 6 Done Sent Done false
8026 3176 Acked Read 10 Done Sent Done false
8027 3177 Acked Read 5 Done Sent Done false
8028 3178 Acked Write 10 Done Sent Done false
8029 3179 Acked Write 6 Done Sent Done false
8030 3180 Acked Read 2 Done Sent Done false
8031 3181 Acked Read 2 Done Sent Done false
8032 3182 Acked Write 1 Done Sent Done false
8033 3183 Acked Write 8 Done Sent Done false
8034 3184 Acked Read 6 Done Sent Done false
8035 3185 Acked Read 7 Done Sent Done false
8036 3186 Acked Read 9 Done Sent Done false
8037 3187 Acked Read 1 Done Sent Done false
8038 3188 Acked Write 10 Done Sent Done false
8039 3189 Acked Write 7 Done Sent Done false
8040 3190 Acked Read 8 Done Sent Done false
8041 3191 Acked Write 9 Done Sent Done false
8042 3192 Acked Read 4 Done Sent Done false
8043 3193 Acked Write 7 Done Sent Done false
8044 3194 Acked Write 5 Done Sent Done false
8045 3195 Acked Write 2 Done Sent Done false
8046 3196 Acked Read 10 Done Sent Done false
8047 3197 Acked Read 5 Done Sent Done false
8048 3198 Acked Write 8 Done Sent Done false
8049 3199 Acked Read 2 Done Sent Done false
8050 3200 Acked Write 5 Done Sent Done false
8051 3201 Acked Write 7 Done Sent Done false
8052 3202 Acked Read 7 Done Sent Done false
8053 3203 Acked Read 1 Done Sent Done false
8054 3204 Acked Write 6 Done Sent Done false
8055 3205 Acked Read 6 Done Sent Done false
8056 3206 Acked Read 9 Done Sent Done false
8057 3207 Acked Write 8 Done Sent Done false
8058 3208 Acked Write 2 Done Sent Done false
8059 3209 Acked Write 9 Done Sent Done false
8060 3210 Acked Read 9 Done Sent Done false
8061 3211 Acked Read 10 Done Sent Done false
8062 3212 Acked Write 4 Done Sent Done false
8063 3213 Acked Write 4 Done Sent Done false
8064 3214 Acked Read 8 Done Sent Done false
8065 3215 Acked Write 4 Done Sent Done false
8066 3216 Acked Write 10 Done Sent Done false
8067 3217 Acked Write 6 Done Sent Done false
8068 3218 Acked Write 3 Done Sent Done false
8069 3219 Acked Read 7 Done Sent Done false
8070 3220 Acked Write 6 Done Sent Done false
8071 3221 Acked Read 1 Done Sent Done false
8072 3222 Acked Write 4 Done Sent Done false
8073 3223 Acked Write 6 Done Sent Done false
8074 3224 Acked Read 6 Done Sent Done false
8075 3225 Acked Read 7 Done Sent Done false
8076 3226 Acked Read 8 Done Sent Done false
8077 3227 Acked Read 2 Done Sent Done false
8078 3228 Acked Read 10 Done Sent Done false
8079 3229 Acked Read 5 Done Sent Done false
8080 3230 Acked Write 10 Done Sent Done false
8081 3231 Acked Read 9 Done Sent Done false
8082 3232 Acked Read 4 Done Sent Done false
8083 3233 Acked Read 7 Done Sent Done false
8084 3234 Acked Read 5 Done Sent Done false
8085 3235 Acked Read 1 Done Sent Done false
8086 3236 Acked Read 1 Done Sent Done false
8087 3237 Acked Flush 0 Done Sent Done false
8088 3238 Acked Write 4 Done Sent Done false
8089 3239 Acked Read 7 Done Sent Done false
8090 3240 Acked Read 8 Done Sent Done false
8091 3241 Acked Write 6 Done Sent Done false
8092 3242 Acked Read 2 Done Sent Done false
8093 3243 Acked Flush 0 Done Sent Done false
8094 3244 Acked Read 6 Done Sent Done false
8095 3245 Acked Write 4 Done Sent Done false
8096 3246 Acked Flush 0 Done Sent Done false
8097 3247 Acked Read 5 Done Sent Done false
8098 3248 Acked Flush 0 Done Sent Done false
8099 3249 Acked Read 1 Done Sent Done false
8100 3250 Acked Read 4 Done Sent Done false
8101 3251 Acked Read 9 Done Sent Done false
8102 3252 Acked Write 9 Done Sent Done false
8103 3253 Acked Write 2 Done Sent Done false
8104 3254 Acked Flush 0 Done Sent Done false
8105 3255 Acked Write 3 Done Sent Done false
8106 3256 Acked Read 9 Done Sent Done false
8107 3257 Acked Read 8 Done Sent Done false
8108 3258 Acked Read 7 Done Sent Done false
8109 3259 Acked Read 5 Done Sent Done false
8110 3260 Acked Write 1 Done Sent Done false
8111 3261 Acked Write 8 Done Sent Done false
8112 3262 Acked Read 2 Done Sent Done false
8113 3263 Acked Read 10 Done Sent Done false
8114 3264 Acked Read 10 Done Sent Done false
8115 3265 Acked Write 6 Done Sent Done false
8116 3266 Acked Write 4 Done Sent Done false
8117 3267 Acked Write 5 Done Sent Done false
8118 3268 Acked Read 1 Done Sent Done false
8119 3269 Acked Write 1 Done Sent Done false
8120 3270 Acked Read 6 Done Sent Done false
8121 3271 Acked Read 10 Done Sent Done false
8122 3272 Acked Read 9 Done Sent Done false
8123 3273 Acked Read 4 Done Sent Done false
8124 3274 Acked Read 9 Done Sent Done false
8125 3275 Acked Read 4 Done Sent Done false
8126 3276 Acked Write 3 Done Sent Done false
8127 3277 Acked Read 3 Done Sent Done false
8128 3278 Acked Write 9 Done Sent Done false
8129 3279 Acked Read 7 Done Sent Done false
8130 3280 Acked Read 5 Done Sent Done false
8131 3281 Acked Write 6 Done Sent Done false
8132 3282 Acked Write 10 Done Sent Done false
8133 3283 Acked Read 7 Done Sent Done false
8134 3284 Acked Read 8 Done Sent Done false
8135 3285 Acked Write 2 Done Sent Done false
8136 3286 Acked Write 6 Done Sent Done false
8137 3287 Acked Write 8 Done Sent Done false
8138 3288 Acked Write 9 Done Sent Done false
8139 3289 Acked Read 7 Done Sent Done false
8140 3290 Acked Read 10 Done Sent Done false
8141 3291 Acked Read 7 Done Sent Done false
8142 3292 Acked Flush 0 Done Sent Done false
8143 3293 Acked Write 6 Done Sent Done false
8144 3294 Acked Write 5 Done Sent Done false
8145 3295 Acked Write 9 Done Sent Done false
8146 3296 Acked Read 4 Done Sent Done false
8147 3297 Acked Read 9 Done Sent Done false
8148 3298 Acked Read 10 Done Sent Done false
8149 3299 Acked Write 10 Done Sent Done false
8150 3300 Acked Read 5 Done Sent Done false
8151 3301 Acked Flush 0 Done Sent Done false
8152 3302 Acked Read 8 Done Sent Done false
8153 3303 Acked Read 8 Done Sent Done false
8154 3304 Acked Read 9 Done Sent Done false
8155 3305 Acked Write 3 Done Sent Done false
8156 3306 Acked Read 9 Done Sent Done false
8157 3307 Acked Read 4 Done Sent Done false
8158 3308 Acked Read 10 Done Sent Done false
8159 3309 Acked Read 7 Done Sent Done false
8160 3310 Acked Write 9 Done Sent Done false
8161 3311 Acked Write 5 Done Sent Done false
8162 3312 Acked Write 1 Done Sent Done false
8163 3313 Acked Write 1 Done Sent Done false
8164 3314 Acked Write 1 Done Sent Done false
8165 3315 Acked Write 8 Done Sent Done false
8166 3316 Acked Read 10 Done Sent Done false
8167 3317 Acked Write 1 Done Sent Done false
8168 3318 Acked Read 8 Done Sent Done false
8169 3319 Acked Read 6 Done Sent Done false
8170 3320 Acked Read 10 Done Sent Done false
8171 3321 Acked Read 10 Done Sent Done false
8172 3322 Acked Write 2 Done Sent Done false
8173 3323 Acked Write 9 Done Sent Done false
8174 3324 Acked Write 5 Done Sent Done false
8175 3325 Acked Write 3 Done Sent Done false
8176 3326 Acked Read 3 Done Sent Done false
8177 3327 Acked Read 5 Done Sent Done false
8178 3328 Acked Write 7 Done Sent Done false
8179 3329 Acked Flush 0 Done Sent Done false
8180 3330 Acked Write 4 Done Sent Done false
8181 3331 Acked Write 10 Done Sent Done false
8182 3332 Acked Write 2 Done Sent Done false
8183 3333 Acked Write 9 Done Sent Done false
8184 3334 Acked Flush 0 Done Sent Done false
8185 3335 Acked Write 4 Done Sent Done false
8186 3336 Acked Read 9 Done Sent Done false
8187 3337 Acked Read 1 Done Sent Done false
8188 3338 Acked Write 9 Done Sent Done false
8189 3339 Acked Write 3 Done Sent Done false
8190 3340 Acked Flush 0 Done Sent Done false
8191 3341 Acked Read 8 Done Sent Done false
8192 3342 Acked Write 9 Done Sent Done false
8193 3343 Acked Write 6 Done Sent Done false
8194 3344 Acked Flush 0 Done Sent Done false
8195 3345 Acked Read 7 Done Sent Done false
8196 3346 Acked Read 1 Done Sent Done false
8197 3347 Acked Write 5 Done Sent Done false
8198 3348 Acked Read 3 Done Sent Done false
8199 3349 Acked Read 2 Done Sent Done false
8200 3350 Acked Write 5 Done Sent Done false
8201 3351 Acked Read 6 Done Sent Done false
8202 3352 Acked Write 3 Done Sent Done false
8203 3353 Acked Write 3 Done Sent Done false
8204 3354 Acked Read 10 Done Sent Done false
8205 3355 Acked Write 4 Done Sent Done false
8206 3356 Acked Write 9 Done Sent Done false
8207 3357 Acked Read 8 Done Sent Done false
8208 3358 Acked Write 5 Done Sent Done false
8209 3359 Acked Flush 0 Done Sent Done false
8210 3360 Acked Write 6 Done Sent Done false
8211 3361 Acked Write 8 Done Sent Done false
8212 3362 Acked Read 6 Done Sent Done false
8213 3363 Acked Write 4 Done Sent Done false
8214 3364 Acked Write 2 Done Sent Done false
8215 3365 Acked Write 1 Done Sent Done false
8216 3366 Acked Read 5 Done Sent Done false
8217 3367 Acked Read 10 Done Sent Done false
8218 3368 Acked Read 4 Done Sent Done false
8219 3369 Acked Write 8 Done Sent Done false
8220 3370 Acked Read 5 Done Sent Done false
8221 3371 Acked Write 7 Done Sent Done false
8222 3372 Acked Read 2 Done Sent Done false
8223 3373 Acked Write 8 Done Sent Done false
8224 3374 Acked Write 9 Done Sent Done false
8225 3375 Acked Read 9 Done Sent Done false
8226 3376 Acked Write 6 Done Sent Done false
8227 3377 Acked Read 4 Done Sent Done false
8228 3378 Acked Flush 0 Done Sent Done false
8229 3379 Acked Write 6 Done Sent Done false
8230 3380 Acked Read 9 Done Sent Done false
8231 3381 Acked Write 2 Done Sent Done false
8232 3382 Acked Write 1 Done Sent Done false
8233 3383 Acked Read 3 Done Sent Done false
8234 3384 Acked Read 9 Done Sent Done false
8235 3385 Acked Flush 0 Done Sent Done false
8236 3386 Acked Read 8 Done Sent Done false
8237 3387 Acked Write 10 Done Sent Done false
8238 3388 Acked Write 3 Done Sent Done false
8239 3389 Acked Write 2 Done Sent Done false
8240 3390 Acked Flush 0 Done Sent Done false
8241 3391 Acked Write 10 Done Sent Done false
8242 3392 Acked Write 9 Done Sent Done false
8243 3393 Acked Read 10 Done Sent Done false
8244 3394 Acked Write 10 Done Sent Done false
8245 3395 Acked Read 9 Done Sent Done false
8246 3396 Acked Write 5 Done Sent Done false
8247 3397 Acked Read 8 Done Sent Done false
8248 3398 Acked Write 1 Done Sent Done false
8249 3399 Acked Write 3 Done Sent Done false
8250 3400 Acked Write 8 Done Sent Done false
8251 3401 Acked Write 8 Done Sent Done false
8252 3402 Acked Flush 0 Done Sent Done false
8253 3403 Acked Read 5 Done Sent Done false
8254 3404 Acked Write 3 Done Sent Done false
8255 3405 Acked Write 9 Done Sent Done false
8256 3406 Acked Read 6 Done Sent Done false
8257 3407 Acked Read 7 Done Sent Done false
8258 3408 Acked Read 6 Done Sent Done false
8259 3409 Acked Write 7 Done Sent Done false
8260 3410 Acked Write 1 Done Sent Done false
8261 3411 Acked Read 9 Done Sent Done false
8262 3412 Acked Write 1 Done Sent Done false
8263 STATES DS:0 DS:1 DS:2 TOTAL
8264 Sent 0 301 0 301
8265 Done 301 0 301 602
8266 Skipped 0 0 0 0
8267 Error 0 0 0 0
8268 Last Flush: 3402 3111 3402
8269 Downstairs last five completed: 3111 3110 3109 3108 3107
8270 Upstairs last five completed: 3412 3411 3410 3409 3408
8271 CLIENT: Up:0 ds:301 act:2
8272 [8820] got start action
8273 [8820] I am going to restart
8274 Starting downstairs at port 8820
8275 Make output file at "/tmp/dsc/downstairs-8820.txt"
8276 [8820][1] reports Starting
8277 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1210)
8278 [8820][1] reports Running
8279 ----------------------------------------------------------------
8280 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8281 States: Active Offline Active
8282 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8283 3112 Acked Read 6 Done Sent Done false
8284 3113 Acked Read 8 Done Sent Done false
8285 3114 Acked Write 1 Done Sent Done false
8286 3115 Acked Read 3 Done Sent Done false
8287 3116 Acked Write 6 Done Sent Done false
8288 3117 Acked Read 4 Done Sent Done false
8289 3118 Acked Read 2 Done Sent Done false
8290 3119 Acked Read 2 Done Sent Done false
8291 3120 Acked Read 9 Done Sent Done false
8292 3121 Acked Flush 0 Done Sent Done false
8293 3122 Acked Write 7 Done Sent Done false
8294 3123 Acked Read 9 Done Sent Done false
8295 3124 Acked Read 4 Done Sent Done false
8296 3125 Acked Read 2 Done Sent Done false
8297 3126 Acked Write 8 Done Sent Done false
8298 3127 Acked Flush 0 Done Sent Done false
8299 3128 Acked Read 2 Done Sent Done false
8300 3129 Acked Flush 0 Done Sent Done false
8301 3130 Acked Read 9 Done Sent Done false
8302 3131 Acked Write 1 Done Sent Done false
8303 3132 Acked Read 3 Done Sent Done false
8304 3133 Acked Write 7 Done Sent Done false
8305 3134 Acked Write 2 Done Sent Done false
8306 3135 Acked Write 4 Done Sent Done false
8307 3136 Acked Read 1 Done Sent Done false
8308 3137 Acked Read 3 Done Sent Done false
8309 3138 Acked Read 9 Done Sent Done false
8310 3139 Acked Write 1 Done Sent Done false
8311 3140 Acked Write 1 Done Sent Done false
8312 3141 Acked Read 10 Done Sent Done false
8313 3142 Acked Read 9 Done Sent Done false
8314 3143 Acked Write 2 Done Sent Done false
8315 3144 Acked Write 3 Done Sent Done false
8316 3145 Acked Write 8 Done Sent Done false
8317 3146 Acked Write 2 Done Sent Done false
8318 3147 Acked Read 4 Done Sent Done false
8319 3148 Acked Flush 0 Done Sent Done false
8320 3149 Acked Read 7 Done Sent Done false
8321 3150 Acked Write 3 Done Sent Done false
8322 3151 Acked Read 9 Done Sent Done false
8323 3152 Acked Write 3 Done Sent Done false
8324 3153 Acked Read 10 Done Sent Done false
8325 3154 Acked Read 6 Done Sent Done false
8326 3155 Acked Flush 0 Done Sent Done false
8327 3156 Acked Flush 0 Done Sent Done false
8328 3157 Acked Write 7 Done Sent Done false
8329 3158 Acked Read 4 Done Sent Done false
8330 3159 Acked Read 4 Done Sent Done false
8331 3160 Acked Write 1 Done Sent Done false
8332 3161 Acked Read 10 Done Sent Done false
8333 3162 Acked Read 5 Done Sent Done false
8334 3163 Acked Read 1 Done Sent Done false
8335 3164 Acked Write 8 Done Sent Done false
8336 3165 Acked Read 5 Done Sent Done false
8337 3166 Acked Write 7 Done Sent Done false
8338 3167 Acked Write 2 Done Sent Done false
8339 3168 Acked Write 4 Done Sent Done false
8340 3169 Acked Flush 0 Done Sent Done false
8341 3170 Acked Write 3 Done Sent Done false
8342 3171 Acked Write 6 Done Sent Done false
8343 3172 Acked Write 6 Done Sent Done false
8344 3173 Acked Write 4 Done Sent Done false
8345 3174 Acked Read 2 Done Sent Done false
8346 3175 Acked Read 6 Done Sent Done false
8347 3176 Acked Read 10 Done Sent Done false
8348 3177 Acked Read 5 Done Sent Done false
8349 3178 Acked Write 10 Done Sent Done false
8350 3179 Acked Write 6 Done Sent Done false
8351 3180 Acked Read 2 Done Sent Done false
8352 3181 Acked Read 2 Done Sent Done false
8353 3182 Acked Write 1 Done Sent Done false
8354 3183 Acked Write 8 Done Sent Done false
8355 3184 Acked Read 6 Done Sent Done false
8356 3185 Acked Read 7 Done Sent Done false
8357 3186 Acked Read 9 Done Sent Done false
8358 3187 Acked Read 1 Done Sent Done false
8359 3188 Acked Write 10 Done Sent Done false
8360 3189 Acked Write 7 Done Sent Done false
8361 3190 Acked Read 8 Done Sent Done false
8362 3191 Acked Write 9 Done Sent Done false
8363 3192 Acked Read 4 Done Sent Done false
8364 3193 Acked Write 7 Done Sent Done false
8365 3194 Acked Write 5 Done Sent Done false
8366 3195 Acked Write 2 Done Sent Done false
8367 3196 Acked Read 10 Done Sent Done false
8368 3197 Acked Read 5 Done Sent Done false
8369 3198 Acked Write 8 Done Sent Done false
8370 3199 Acked Read 2 Done Sent Done false
8371 3200 Acked Write 5 Done Sent Done false
8372 3201 Acked Write 7 Done Sent Done false
8373 3202 Acked Read 7 Done Sent Done false
8374 3203 Acked Read 1 Done Sent Done false
8375 3204 Acked Write 6 Done Sent Done false
8376 3205 Acked Read 6 Done Sent Done false
8377 3206 Acked Read 9 Done Sent Done false
8378 3207 Acked Write 8 Done Sent Done false
8379 3208 Acked Write 2 Done Sent Done false
8380 3209 Acked Write 9 Done Sent Done false
8381 3210 Acked Read 9 Done Sent Done false
8382 3211 Acked Read 10 Done Sent Done false
8383 3212 Acked Write 4 Done Sent Done false
8384 3213 Acked Write 4 Done Sent Done false
8385 3214 Acked Read 8 Done Sent Done false
8386 3215 Acked Write 4 Done Sent Done false
8387 3216 Acked Write 10 Done Sent Done false
8388 3217 Acked Write 6 Done Sent Done false
8389 3218 Acked Write 3 Done Sent Done false
8390 3219 Acked Read 7 Done Sent Done false
8391 3220 Acked Write 6 Done Sent Done false
8392 3221 Acked Read 1 Done Sent Done false
8393 3222 Acked Write 4 Done Sent Done false
8394 3223 Acked Write 6 Done Sent Done false
8395 3224 Acked Read 6 Done Sent Done false
8396 3225 Acked Read 7 Done Sent Done false
8397 3226 Acked Read 8 Done Sent Done false
8398 3227 Acked Read 2 Done Sent Done false
8399 3228 Acked Read 10 Done Sent Done false
8400 3229 Acked Read 5 Done Sent Done false
8401 3230 Acked Write 10 Done Sent Done false
8402 3231 Acked Read 9 Done Sent Done false
8403 3232 Acked Read 4 Done Sent Done false
8404 3233 Acked Read 7 Done Sent Done false
8405 3234 Acked Read 5 Done Sent Done false
8406 3235 Acked Read 1 Done Sent Done false
8407 3236 Acked Read 1 Done Sent Done false
8408 3237 Acked Flush 0 Done Sent Done false
8409 3238 Acked Write 4 Done Sent Done false
8410 3239 Acked Read 7 Done Sent Done false
8411 3240 Acked Read 8 Done Sent Done false
8412 3241 Acked Write 6 Done Sent Done false
8413 3242 Acked Read 2 Done Sent Done false
8414 3243 Acked Flush 0 Done Sent Done false
8415 3244 Acked Read 6 Done Sent Done false
8416 3245 Acked Write 4 Done Sent Done false
8417 3246 Acked Flush 0 Done Sent Done false
8418 3247 Acked Read 5 Done Sent Done false
8419 3248 Acked Flush 0 Done Sent Done false
8420 3249 Acked Read 1 Done Sent Done false
8421 3250 Acked Read 4 Done Sent Done false
8422 3251 Acked Read 9 Done Sent Done false
8423 3252 Acked Write 9 Done Sent Done false
8424 3253 Acked Write 2 Done Sent Done false
8425 3254 Acked Flush 0 Done Sent Done false
8426 3255 Acked Write 3 Done Sent Done false
8427 3256 Acked Read 9 Done Sent Done false
8428 3257 Acked Read 8 Done Sent Done false
8429 3258 Acked Read 7 Done Sent Done false
8430 3259 Acked Read 5 Done Sent Done false
8431 3260 Acked Write 1 Done Sent Done false
8432 3261 Acked Write 8 Done Sent Done false
8433 3262 Acked Read 2 Done Sent Done false
8434 3263 Acked Read 10 Done Sent Done false
8435 3264 Acked Read 10 Done Sent Done false
8436 3265 Acked Write 6 Done Sent Done false
8437 3266 Acked Write 4 Done Sent Done false
8438 3267 Acked Write 5 Done Sent Done false
8439 3268 Acked Read 1 Done Sent Done false
8440 3269 Acked Write 1 Done Sent Done false
8441 3270 Acked Read 6 Done Sent Done false
8442 3271 Acked Read 10 Done Sent Done false
8443 3272 Acked Read 9 Done Sent Done false
8444 3273 Acked Read 4 Done Sent Done false
8445 3274 Acked Read 9 Done Sent Done false
8446 3275 Acked Read 4 Done Sent Done false
8447 3276 Acked Write 3 Done Sent Done false
8448 3277 Acked Read 3 Done Sent Done false
8449 3278 Acked Write 9 Done Sent Done false
8450 3279 Acked Read 7 Done Sent Done false
8451 3280 Acked Read 5 Done Sent Done false
8452 3281 Acked Write 6 Done Sent Done false
8453 3282 Acked Write 10 Done Sent Done false
8454 3283 Acked Read 7 Done Sent Done false
8455 3284 Acked Read 8 Done Sent Done false
8456 3285 Acked Write 2 Done Sent Done false
8457 3286 Acked Write 6 Done Sent Done false
8458 3287 Acked Write 8 Done Sent Done false
8459 3288 Acked Write 9 Done Sent Done false
8460 3289 Acked Read 7 Done Sent Done false
8461 3290 Acked Read 10 Done Sent Done false
8462 3291 Acked Read 7 Done Sent Done false
8463 3292 Acked Flush 0 Done Sent Done false
8464 3293 Acked Write 6 Done Sent Done false
8465 3294 Acked Write 5 Done Sent Done false
8466 3295 Acked Write 9 Done Sent Done false
8467 3296 Acked Read 4 Done Sent Done false
8468 3297 Acked Read 9 Done Sent Done false
8469 3298 Acked Read 10 Done Sent Done false
8470 3299 Acked Write 10 Done Sent Done false
8471 3300 Acked Read 5 Done Sent Done false
8472 3301 Acked Flush 0 Done Sent Done false
8473 3302 Acked Read 8 Done Sent Done false
8474 3303 Acked Read 8 Done Sent Done false
8475 3304 Acked Read 9 Done Sent Done false
8476 3305 Acked Write 3 Done Sent Done false
8477 3306 Acked Read 9 Done Sent Done false
8478 3307 Acked Read 4 Done Sent Done false
8479 3308 Acked Read 10 Done Sent Done false
8480 3309 Acked Read 7 Done Sent Done false
8481 3310 Acked Write 9 Done Sent Done false
8482 3311 Acked Write 5 Done Sent Done false
8483 3312 Acked Write 1 Done Sent Done false
8484 3313 Acked Write 1 Done Sent Done false
8485 3314 Acked Write 1 Done Sent Done false
8486 3315 Acked Write 8 Done Sent Done false
8487 3316 Acked Read 10 Done Sent Done false
8488 3317 Acked Write 1 Done Sent Done false
8489 3318 Acked Read 8 Done Sent Done false
8490 3319 Acked Read 6 Done Sent Done false
8491 3320 Acked Read 10 Done Sent Done false
8492 3321 Acked Read 10 Done Sent Done false
8493 3322 Acked Write 2 Done Sent Done false
8494 3323 Acked Write 9 Done Sent Done false
8495 3324 Acked Write 5 Done Sent Done false
8496 3325 Acked Write 3 Done Sent Done false
8497 3326 Acked Read 3 Done Sent Done false
8498 3327 Acked Read 5 Done Sent Done false
8499 3328 Acked Write 7 Done Sent Done false
8500 3329 Acked Flush 0 Done Sent Done false
8501 3330 Acked Write 4 Done Sent Done false
8502 3331 Acked Write 10 Done Sent Done false
8503 3332 Acked Write 2 Done Sent Done false
8504 3333 Acked Write 9 Done Sent Done false
8505 3334 Acked Flush 0 Done Sent Done false
8506 3335 Acked Write 4 Done Sent Done false
8507 3336 Acked Read 9 Done Sent Done false
8508 3337 Acked Read 1 Done Sent Done false
8509 3338 Acked Write 9 Done Sent Done false
8510 3339 Acked Write 3 Done Sent Done false
8511 3340 Acked Flush 0 Done Sent Done false
8512 3341 Acked Read 8 Done Sent Done false
8513 3342 Acked Write 9 Done Sent Done false
8514 3343 Acked Write 6 Done Sent Done false
8515 3344 Acked Flush 0 Done Sent Done false
8516 3345 Acked Read 7 Done Sent Done false
8517 3346 Acked Read 1 Done Sent Done false
8518 3347 Acked Write 5 Done Sent Done false
8519 3348 Acked Read 3 Done Sent Done false
8520 3349 Acked Read 2 Done Sent Done false
8521 3350 Acked Write 5 Done Sent Done false
8522 3351 Acked Read 6 Done Sent Done false
8523 3352 Acked Write 3 Done Sent Done false
8524 3353 Acked Write 3 Done Sent Done false
8525 3354 Acked Read 10 Done Sent Done false
8526 3355 Acked Write 4 Done Sent Done false
8527 3356 Acked Write 9 Done Sent Done false
8528 3357 Acked Read 8 Done Sent Done false
8529 3358 Acked Write 5 Done Sent Done false
8530 3359 Acked Flush 0 Done Sent Done false
8531 3360 Acked Write 6 Done Sent Done false
8532 3361 Acked Write 8 Done Sent Done false
8533 3362 Acked Read 6 Done Sent Done false
8534 3363 Acked Write 4 Done Sent Done false
8535 3364 Acked Write 2 Done Sent Done false
8536 3365 Acked Write 1 Done Sent Done false
8537 3366 Acked Read 5 Done Sent Done false
8538 3367 Acked Read 10 Done Sent Done false
8539 3368 Acked Read 4 Done Sent Done false
8540 3369 Acked Write 8 Done Sent Done false
8541 3370 Acked Read 5 Done Sent Done false
8542 3371 Acked Write 7 Done Sent Done false
8543 3372 Acked Read 2 Done Sent Done false
8544 3373 Acked Write 8 Done Sent Done false
8545 3374 Acked Write 9 Done Sent Done false
8546 3375 Acked Read 9 Done Sent Done false
8547 3376 Acked Write 6 Done Sent Done false
8548 3377 Acked Read 4 Done Sent Done false
8549 3378 Acked Flush 0 Done Sent Done false
8550 3379 Acked Write 6 Done Sent Done false
8551 3380 Acked Read 9 Done Sent Done false
8552 3381 Acked Write 2 Done Sent Done false
8553 3382 Acked Write 1 Done Sent Done false
8554 3383 Acked Read 3 Done Sent Done false
8555 3384 Acked Read 9 Done Sent Done false
8556 3385 Acked Flush 0 Done Sent Done false
8557 3386 Acked Read 8 Done Sent Done false
8558 3387 Acked Write 10 Done Sent Done false
8559 3388 Acked Write 3 Done Sent Done false
8560 3389 Acked Write 2 Done Sent Done false
8561 3390 Acked Flush 0 Done Sent Done false
8562 3391 Acked Write 10 Done Sent Done false
8563 3392 Acked Write 9 Done Sent Done false
8564 3393 Acked Read 10 Done Sent Done false
8565 3394 Acked Write 10 Done Sent Done false
8566 3395 Acked Read 9 Done Sent Done false
8567 3396 Acked Write 5 Done Sent Done false
8568 3397 Acked Read 8 Done Sent Done false
8569 3398 Acked Write 1 Done Sent Done false
8570 3399 Acked Write 3 Done Sent Done false
8571 3400 Acked Write 8 Done Sent Done false
8572 3401 Acked Write 8 Done Sent Done false
8573 3402 Acked Flush 0 Done Sent Done false
8574 3403 Acked Read 5 Done Sent Done false
8575 3404 Acked Write 3 Done Sent Done false
8576 3405 Acked Write 9 Done Sent Done false
8577 3406 Acked Read 6 Done Sent Done false
8578 3407 Acked Read 7 Done Sent Done false
8579 3408 Acked Read 6 Done Sent Done false
8580 3409 Acked Write 7 Done Sent Done false
8581 3410 Acked Write 1 Done Sent Done false
8582 3411 Acked Read 9 Done Sent Done false
8583 3412 Acked Write 1 Done Sent Done false
8584 3413 Acked Flush 0 Done Sent Done false
8585 STATES DS:0 DS:1 DS:2 TOTAL
8586 Sent 0 302 0 302
8587 Done 302 0 302 604
8588 Skipped 0 0 0 0
8589 Error 0 0 0 0
8590 Last Flush: 3413 3111 3413
8591 Downstairs last five completed: 3111 3110 3109 3108 3107
8592 Upstairs last five completed: 3413 3412 3411 3410 3409
8593 CLIENT: Up:0 ds:302 act:2
85942025-01-14T16:12:23.646ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
85952025-01-14T16:12:23.646ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:63242) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
85962025-01-14T16:12:23.647ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
85972025-01-14T16:12:23.648ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
85982025-01-14T16:12:23.648ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
85992025-01-14T16:12:23.648ZINFOcrucible: send last flush ID to this DS: 3111 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
86002025-01-14T16:12:23.648ZINFOcrucible: Replied this last flush ID: 3111 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
86012025-01-14T16:12:23.648ZINFOcrucible: [1] client re-new 302 jobs since flush 3111 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
86022025-01-14T16:12:23.649ZINFOcrucible: [1] Replayed 302 jobs since flush: 3111 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
8603 ----------------------------------------------------------------
8604 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
8605 Downstairs last five completed: 3413 3412 3411 3410 3409
8606 Upstairs last five completed: 3413 3412 3411 3410 3409
8607 CLIENT: Up:0 ds:0 act:3
8608 Replay: All jobs finished
8609 Main task has work to do, go find it
8610 got dsc Stop(2)
8611 stop 2
8612 Jan 14 16:12:25.888 INFO request completed[8830] Got stop action so:false kr:false
8613 , latency_us: 372, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 1e4209b5-65c2-436c-a4bc-3f3550ee5bda, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
86142025-01-14T16:12:25.889ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
86152025-01-14T16:12:25.889ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
86162025-01-14T16:12:25.889ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
8617 Jan 14 16:12:25.889 INFO request completed, latency_us: 173, response_code: 200, uri: /state/cid/2, method:{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-14T16:12:25.889501304Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
8618 GET, req_id: bff141d2-7c5b-46bb-b15d-ca3fbc239010, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
8619 [8830] Exited with: ExitStatus(unix_wait_status(9))
8620 [8830][2] reports Exit
8621 Jan 14 16:12:29.891 INFO request completed, latency_us: 197, response_code: 200, uri: /state/cid/2, method: GET, req_id: 2c9633c1-2537-45c0-881a-da8c01e3de8a, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
8622 001/300 Read block 3007 len 40960
8623 002/300 Read block 3296 len 20480
8624 003/300 Read block 2755 len 32768
8625 004/300 Write block 4623 len 16384 data: 3 3 2 2
8626 005/300 Flush
8627 006/300 Write block 1658 len 16384 data: 3 3 2 2
8628 007/300 Read block 131 len 20480
8629 008/300 Read block 3641 len 20480
8630 009/300 Read block 1418 len 24576
8631 010/300 Flush
8632 011/300 Read block 807 len 16384
8633 012/300 Write block 1996 len 40960 data: 4 4 4 3 3 3 2 2 3 3
8634 013/300 Read block 4723 len 32768
8635 014/300 Read block 2948 len 40960
8636 015/300 Write block 1096 len 16384 data: 2 2 2 2
8637 016/300 Read block 1302 len 12288
8638 017/300 Flush
8639 018/300 Write block 1074 len 24576 data: 3 3 2 2 2 2
8640 019/300 Read block 945 len 20480
8641 020/300 Read block 3160 len 8192
8642 021/300 Read block 3456 len 40960
8643 022/300 Read block 2845 len 16384
8644 023/300 Read block 4883 len 8192
8645 024/300 Write block 2002 len 12288 data: 3 3 4
8646 025/300 Read block 3485 len 36864
8647 026/300 Write block 2389 len 40960 data: 2 3 3 3 3 3 3 3 4 3
8648 027/300 Flush
8649 028/300 Write block 2792 len 8192 data: 2 2
8650 029/300 Read block 4282 len 32768
8651 030/300 Read block 3980 len 20480
8652 031/300 Read block 3479 len 36864
8653 032/300 Write block 1566 len 28672 data: 4 5 5 5 5 3 3
8654 033/300 Read block 3148 len 12288
8655 034/300 Read block 1884 len 24576
8656 035/300 Read block 408 len 16384
8657 036/300 Write block 2526 len 16384 data: 2 2 2 2
8658 037/300 Write block 4993 len 20480 data: 2 2 2 2 2
8659 038/300 Read block 4055 len 8192
8660 039/300 Read block 640 len 32768
8661 040/300 Write block 982 len 8192 data: 2 3
8662 041/300 Write block 1244 len 20480 data: 2 2 2 2 2
8663 042/300 Read block 1119 len 12288
8664 043/300 Read block 1605 len 4096
8665 044/300 Read block 561 len 8192
8666 045/300 Read block 3978 len 36864
8667 046/300 Read block 657 len 16384
8668 047/300 Write block 3533 len 36864 data: 2 2 2 2 3 3 3 3 3
8669 048/300 Write block 2870 len 40960 data: 3 3 3 3 2 2 2 2 3 3
8670 049/300 Write block 3148 len 32768 data: 5 4 4 3 3 2 2 3
8671 050/300 Write block 4261 len 40960 data: 2 2 2 2 2 2 2 2 2 2
8672 051/300 Read block 822 len 16384
8673 052/300 Write block 4585 len 28672 data: 2 2 2 2 3 3 3
8674 053/300 Read block 1592 len 16384
8675 054/300 Read block 2845 len 12288
8676 055/300 Read block 3379 len 36864
8677 056/300 Write block 3129 len 12288 data: 2 2 2
8678 057/300 Read block 1430 len 16384
8679 058/300 Write block 2458 len 12288 data: 3 3 3
8680 059/300 Write block 256 len 32768 data: 5 4 3 3 4 3 3 3
8681 060/300 Write block 615 len 8192 data: 4 3
8682 061/300 Flush
8683 062/300 Write block 4680 len 28672 data: 4 3 3 3 2 2 2
8684 063/300 Write block 4914 len 24576 data: 2 2 2 2 2 2
8685 064/300 Write block 4365 len 8192 data: 4 4
8686 065/300 Read block 1220 len 12288
8687 066/300 Write block 113 len 20480 data: 3 4 2 2 2
8688 067/300 Read block 968 len 36864
8689 068/300 Write block 4801 len 36864 data: 2 2 2 3 3 3 3 3 2
8690 069/300 Write block 1205 len 12288 data: 3 4 4
8691 070/300 Write block 4350 len 36864 data: 4 4 3 3 3 3 3 2 2
8692 071/300 Read block 750 len 36864
8693 072/300 Flush
8694 073/300 Read block 627 len 40960
8695 074/300 Write block 3773 len 8192 data: 5 4
8696 075/300 Read block 1050 len 20480
8697 076/300 Write block 904 len 4096 data: 3
8698 077/300 Read block 2088 len 40960
8699 078/300 Write block 207 len 32768 data: 6 5 5 4 4 4 4 5
8700 079/300 Read block 449 len 24576
8701 080/300 Write block 1698 len 4096 data: 4
8702 081/300 Write block 3608 len 16384 data: 4 4 4 4
8703 082/300 Flush
8704 083/300 Read block 3202 len 28672
8705 084/300 Read block 3685 len 24576
8706 085/300 Read block 4022 len 8192
8707 086/300 Read block 2349 len 40960
8708 087/300 Read block 2568 len 4096
8709 088/300 Write block 1455 len 28672 data: 3 2 2 2 2 2 2
8710 089/300 Read block 2610 len 4096
8711 090/300 Write block 1632 len 28672 data: 2 2 2 2 2 2 2
8712 091/300 Read block 1338 len 8192
8713 092/300 Write block 2478 len 36864 data: 4 4 4 4 4 3 3 3 3
8714 093/300 Read block 4250 len 8192
8715 094/300 Flush
8716 095/300 Read block 1326 len 8192
8717 096/300 Read block 4640 len 28672
8718 097/300 Read block 2338 len 4096
8719 098/300 Read block 135 len 24576
8720 099/300 Read block 1907 len 16384
8721 100/300 Read block 2893 len 36864
8722 101/300 Read block 1297 len 8192
8723 102/300 Read block 2932 len 36864
8724 103/300 Read block 4336 len 20480
8725 104/300 Write block 4717 len 4096 data: 3
8726 105/300 Read block 1650 len 12288
8727 106/300 Write block 775 len 12288 data: 5 5 3
8728 107/300 Read block 3855 len 8192
8729 108/300 Read block 1677 len 32768
8730 109/300 Flush
8731 110/300 Write block 1538 len 24576 data: 3 3 2 2 3 3
8732 111/300 Read block 3970 len 28672
8733 112/300 Read block 863 len 12288
8734 113/300 Read block 125 len 12288
8735 114/300 Read block 2760 len 4096
8736 115/300 Read block 4758 len 16384
8737 116/300 Flush
8738 117/300 Write block 551 len 40960 data: 3 4 4 3 3 3 3 3 3 4
8739 118/300 Write block 2855 len 12288 data: 2 2 2
8740 119/300 Read block 3137 len 20480
8741 120/300 Flush
8742 121/300 Read block 1119 len 8192
8743 122/300 Read block 4000 len 12288
8744 123/300 Read block 2686 len 28672
8745 124/300 Read block 4671 len 20480
8746 125/300 Read block 2345 len 40960
8747 126/300 Write block 3106 len 16384 data: 2 2 2 2
8748 127/300 Write block 1617 len 40960 data: 3 3 3 3 3 3 2 2 3 3
8749 128/300 Write block 4744 len 4096 data: 3
8750 129/300 Write block 3277 len 32768 data: 3 3 3 4 5 6 5 5
8751 130/300 Write block 4556 len 28672 data: 2 2 2 2 2 2 3
8752 131/300 Write block 3634 len 24576 data: 5 5 5 5 4 4
8753 132/300 Flush
8754 133/300 Write block 1000 len 4096 data: 4
8755 134/300 Read block 205 len 28672
8756 135/300 Read block 2573 len 28672
8757 136/300 Read block 1128 len 4096
8758 137/300 Write block 4733 len 36864 data: 3 3 4 4 3 3 2 2 3
8759 138/300 Read block 3969 len 24576
8760 139/300 Read block 3614 len 20480
8761 140/300 Write block 4686 len 36864 data: 3 3 3 3 3 3 4 3 2
8762 141/300 Read block 4489 len 4096
8763 142/300 Read block 2286 len 28672
8764 143/300 Write block 1328 len 32768 data: 2 2 2 3 3 4 4 4
8765 144/300 Flush
8766 145/300 Write block 2707 len 4096 data: 3
8767 146/300 Read block 2978 len 24576
8768 147/300 Write block 311 len 4096 data: 3
8769 148/300 Write block 3391 len 40960 data: 2 2 2 2 2 3 3 3 3 3
8770 149/300 Read block 3002 len 16384
8771 150/300 Read block 3048 len 8192
8772 151/300 Write block 131 len 40960 data: 3 3 3 3 3 3 3 3 3 3
8773 152/300 Write block 453 len 4096 data: 4
8774 153/300 Write block 4222 len 16384 data: 3 2 2 2
8775 154/300 Read block 4784 len 8192
8776 155/300 Write block 4028 len 32768 data: 3 3 3 2 2 2 2 2
8777 156/300 Read block 3439 len 20480
8778 157/300 Read block 4574 len 20480
8779 158/300 Read block 210 len 12288
8780 159/300 Read block 3263 len 36864
8781 160/300 Write block 3753 len 20480 data: 2 2 3 3 4
8782 161/300 Write block 1013 len 24576 data: 4 4 4 4 4 4
8783 162/300 Read block 4381 len 4096
8784 163/300 Write block 1164 len 12288 data: 3 2 2
8785 164/300 Write block 4010 len 28672 data: 3 3 3 3 5 5 4
8786 165/300 Read block 4695 len 32768
8787 166/300 Read block 1345 len 36864
8788 167/300 Write block 1707 len 32768 data: 3 2 2 2 2 2 3 3
8789 168/300 Write block 3161 len 40960 data: 3 2 2 2 3 3 3 3 3 2
8790 169/300 Read block 3876 len 20480
8791 170/300 Write block 3115 len 8192 data: 2 2
8792 171/300 Write block 78 len 20480 data: 4 5 5 3 3
8793 172/300 Write block 3809 len 16384 data: 4 4 4 4
8794 173/300 Read block 219 len 28672
8795 174/300 Read block 4091 len 4096
8796 175/300 Read block 3896 len 20480
8797 176/300 Read block 2277 len 16384
8798 177/300 Read block 4840 len 28672
8799 178/300 Write block 1712 len 28672 data: 3 4 4 3 3 2 2
8800 179/300 Read block 1421 len 20480
8801 180/300 Read block 2208 len 4096
8802 181/300 Read block 734 len 36864
8803 182/300 Write block 2870 len 40960 data: 4 4 4 4 3 3 3 3 4 4
8804 183/300 Read block 4788 len 40960
8805 184/300 Flush
8806 185/300 Read block 4568 len 36864
8807 186/300 Flush
8808 187/300 Flush
8809 188/300 Read block 1559 len 12288
8810 189/300 Read block 1082 len 4096
8811 190/300 Read block 2181 len 40960
8812 191/300 Read block 1108 len 8192
8813 192/300 Write block 1720 len 20480 data: 2 2 2 2 3
8814 193/300 Write block 1451 len 40960 data: 3 3 3 3 4 3 3 3 3 3
8815 194/300 Read block 4276 len 16384
8816 195/300 Write block 2639 len 32768 data: 4 4 4 4 4 4 4 3
8817 196/300 Read block 523 len 36864
8818 197/300 Flush
8819 198/300 Read block 3816 len 16384
8820 199/300 Write block 2803 len 24576 data: 4 3 3 3 2 3
8821 200/300 Write block 3352 len 12288 data: 5 5 5
8822 201/300 Write block 893 len 4096 data: 4
8823 202/300 Write block 1693 len 16384 data: 3 3 4 5
8824 203/300 Write block 2694 len 32768 data: 3 3 3 4 3 3 3 4
8825 204/300 Write block 143 len 32768 data: 4 4 4 4 4 4 4 4
8826 205/300 Write block 3941 len 12288 data: 3 3 4
8827 206/300 Write block 2392 len 32768 data: 4 4 4 4 4 5 4 3
8828 207/300 Read block 282 len 8192
8829 208/300 Read block 2482 len 20480
8830 209/300 Read block 4878 len 32768
8831 210/300 Read block 2789 len 24576
8832 211/300 Flush
8833 212/300 Read block 2868 len 24576
8834 213/300 Read block 4709 len 32768
8835 214/300 Write block 2725 len 20480 data: 3 4 4 4 3
8836 215/300 Read block 4674 len 36864
8837 216/300 Write block 1764 len 8192 data: 2 2
8838 217/300 Read block 2936 len 40960
8839 218/300 Write block 1130 len 20480 data: 4 4 3 2 2
8840 219/300 Write block 3793 len 32768 data: 2 4 4 4 4 4 4 3
8841 220/300 Read block 163 len 20480
8842 221/300 Write block 14 len 36864 data: 3 3 3 3 3 2 2 3 3
8843 222/300 Read block 215 len 32768
8844 223/300 Read block 700 len 4096
8845 224/300 Write block 4271 len 28672 data: 2 2 2 2 2 2 3
8846 225/300 Read block 4178 len 20480
8847 226/300 Read block 3427 len 20480
8848 227/300 Write block 3533 len 36864 data: 3 3 3 3 4 4 4 4 4
8849 228/300 Write block 3764 len 16384 data: 4 4 4 5
8850 229/300 Read block 4451 len 36864
8851 230/300 Write block 1065 len 28672 data: 4 4 4 4 3 3 4
8852 231/300 Read block 1858 len 40960
8853 232/300 Read block 3067 len 40960
8854 233/300 Read block 2288 len 12288
8855 234/300 Read block 2579 len 24576
8856 235/300 Flush
8857 236/300 Read block 3955 len 20480
8858 237/300 Write block 977 len 12288 data: 2 2 2
8859 238/300 Read block 670 len 32768
8860 239/300 Read block 3524 len 24576
8861 240/300 Read block 4970 len 40960
8862 241/300 Read block 3815 len 12288
8863 242/300 Read block 3526 len 28672
8864 243/300 Read block 3951 len 4096
8865 244/300 Read block 4027 len 36864
8866 245/300 Read block 1557 len 24576
8867 246/300 Write block 185 len 8192 data: 3 3
8868 247/300 Read block 4370 len 36864
8869 248/300 Write block 2660 len 20480 data: 2 3 3 3 3
8870 249/300 Read block 1329 len 24576
8871 250/300 Write block 1745 len 32768 data: 4 5 5 6 5 5 4 5
8872 251/300 Write block 3061 len 40960 data: 2 2 3 3 3 2 2 2 2 2
8873 252/300 Write block 165 len 28672 data: 4 4 4 4 4 4 4
8874 253/300 Write block 1900 len 4096 data: 5
8875 254/300 Write block 4949 len 8192 data: 2 2
8876 255/300 Write block 970 len 8192 data: 4 3
8877 256/300 Write block 2310 len 40960 data: 2 2 4 5 5 5 6 6 5 5
8878 257/300 Write block 2999 len 36864 data: 4 4 4 4 4 4 3 3 3
8879 258/300 Read block 3053 len 24576
8880 259/300 Read block 3303 len 20480
8881 260/300 Read block 1176 len 32768
8882 261/300 Write block 2605 len 36864 data: 4 4 3 3 3 2 2 2 4
8883 262/300 Write block 596 len 20480 data: 2 2 2 2 2
8884 263/300 Write block 19 len 8192 data: 3 3
8885 264/300 Write block 998 len 16384 data: 4 4 5 4
8886 265/300 Read block 1697 len 28672
8887 266/300 Read block 3493 len 36864
8888 267/300 Write block 501 len 8192 data: 3 2
8889 268/300 Read block 2188 len 20480
8890 269/300 Write block 553 len 16384 data: 5 4 4 4
8891 270/300 Write block 1505 len 28672 data: 4 4 3 3 3 3 2
8892 271/300 Write block 4740 len 36864 data: 3 4 4 4 4 3 3 2 2
8893 272/300 Read block 3382 len 20480
8894 273/300 Read block 71 len 20480
8895 274/300 Read block 2215 len 4096
8896 275/300 Read block 1623 len 20480
8897 276/300 Read block 714 len 40960
8898 277/300 Read block 402 len 12288
8899 278/300 Flush
8900 279/300 Read block 2695 len 8192
8901 280/300 Write block 1002 len 16384 data: 3 3 3 3
8902 281/300 Read block 1834 len 32768
8903 282/300 Read block 77 len 28672
8904 283/300 Write block 2784 len 32768 data: 2 2 2 2 2 2 2 2
8905 284/300 Read block 4718 len 20480
8906 285/300 Read block 1997 len 24576
8907 286/300 Flush
8908 287/300 Read block 852 len 12288
8909 288/300 Write block 1922 len 24576 data: 3 3 3 4 4 4
8910 289/300 Write block 271 len 36864 data: 3 2 2 2 3 3 3 3 3
8911 290/300 Write block 4659 len 16384 data: 2 2 3 3
8912 291/300 Read block 598 len 28672
8913 292/300 Read block 2867 len 20480
8914 293/300 Read block 4541 len 32768
8915 294/300 Write block 2398 len 20480 data: 5 4 4 4 4
8916 295/300 Read block 1449 len 16384
8917 296/300 Read block 4659 len 28672
8918 297/300 Write block 3787 len 16384 data: 2 3 3 3
8919 298/300 Read block 4346 len 12288
8920 299/300 Read block 1801 len 20480
8921 300/300 Read block 3879 len 32768
8922 301/300 Write block 3948 len 24576 data: 2 3 3 3 3 2
8923 Main task has work to do, go find it
8924 got dsc Start(2)
8925 start 2
8926 Jan 14 16:12:30.134 INFO request completed, latency_us: 223, response_code: 204, uri: /start/cid/2, method: POST, req_id: 5a9046fb-1138-4a71-bf7e-8361ebd45a4c, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
8927 Replay: started 2, returned:Ok(())
8928 ----------------------------------------------------------------
8929 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
8930 States: Active Active Offline
8931 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8932 3414 Acked Read 10 Done Done Sent false
8933 3415 Acked Read 5 Done Done Sent false
8934 3416 Acked Read 8 Done Done Sent false
8935 3417 Acked Write 4 Done Done Sent false
8936 3418 Acked Flush 0 Done Done Sent false
8937 3419 Acked Write 4 Done Done Sent false
8938 3420 Acked Read 5 Done Done Sent false
8939 3421 Acked Read 5 Done Done Sent false
8940 3422 Acked Read 6 Done Done Sent false
8941 3423 Acked Flush 0 Done Done Sent false
8942 3424 Acked Read 4 Done Done Sent false
8943 3425 Acked Write 10 Done Done Sent false
8944 3426 Acked Read 8 Done Done Sent false
8945 3427 Acked Read 10 Done Done Sent false
8946 3428 Acked Write 4 Done Done Sent false
8947 3429 Acked Read 3 Done Done Sent false
8948 3430 Acked Flush 0 Done Done Sent false
8949 3431 Acked Write 6 Done Done Sent false
8950 3432 Acked Read 5 Done Done Sent false
8951 3433 Acked Read 2 Done Done Sent false
8952 3434 Acked Read 10 Done Done Sent false
8953 3435 Acked Read 4 Done Done Sent false
8954 3436 Acked Read 2 Done Done Sent false
8955 3437 Acked Write 3 Done Done Sent false
8956 3438 Acked Read 9 Done Done Sent false
8957 3439 Acked Write 10 Done Done Sent false
8958 3440 Acked Flush 0 Done Done Sent false
8959 3441 Acked Write 2 Done Done Sent false
8960 3442 Acked Read 8 Done Done Sent false
8961 3443 Acked Read 5 Done Done Sent false
8962 3444 Acked Read 9 Done Done Sent false
8963 3445 Acked Write 7 Done Done Sent false
8964 3446 Acked Read 3 Done Done Sent false
8965 3447 Acked Read 6 Done Done Sent false
8966 3448 Acked Read 4 Done Done Sent false
8967 3449 Acked Write 4 Done Done Sent false
8968 3450 Acked Write 5 Done Done Sent false
8969 3451 Acked Read 2 Done Done Sent false
8970 3452 Acked Read 8 Done Done Sent false
8971 3453 Acked Write 2 Done Done Sent false
8972 3454 Acked Write 5 Done Done Sent false
8973 3455 Acked Read 3 Done Done Sent false
8974 3456 Acked Read 1 Done Done Sent false
8975 3457 Acked Read 2 Done Done Sent false
8976 3458 Acked Read 9 Done Done Sent false
8977 3459 Acked Read 4 Done Done Sent false
8978 3460 Acked Write 9 Done Done Sent false
8979 3461 Acked Write 10 Done Done Sent false
8980 3462 Acked Write 8 Done Done Sent false
8981 3463 Acked Write 10 Done Done Sent false
8982 3464 Acked Read 4 Done Done Sent false
8983 3465 Acked Write 7 Done Done Sent false
8984 3466 Acked Read 4 Done Done Sent false
8985 3467 Acked Read 3 Done Done Sent false
8986 3468 Acked Read 9 Done Done Sent false
8987 3469 Acked Write 3 Done Done Sent false
8988 3470 Acked Read 4 Done Done Sent false
8989 3471 Acked Write 3 Done Done Sent false
8990 3472 Acked Write 8 Done Done Sent false
8991 3473 Acked Write 2 Done Done Sent false
8992 3474 Acked Flush 0 Done Done Sent false
8993 3475 Acked Write 7 Done Done Sent false
8994 3476 Acked Write 6 Done Done Sent false
8995 3477 Acked Write 2 Done Done Sent false
8996 3478 Acked Read 3 Done Done Sent false
8997 3479 Acked Write 5 Done Done Sent false
8998 3480 Acked Read 9 Done Done Sent false
8999 3481 Acked Write 9 Done Done Sent false
9000 3482 Acked Write 3 Done Done Sent false
9001 3483 Acked Write 9 Done Done Sent false
9002 3484 Acked Read 9 Done Done Sent false
9003 3485 Acked Flush 0 Done Done Sent false
9004 3486 Acked Read 10 Done Done Sent false
9005 3487 Acked Write 2 Done Done Sent false
9006 3488 Acked Read 5 Done Done Sent false
9007 3489 Acked Write 1 Done Done Sent false
9008 3490 Acked Read 10 Done Done Sent false
9009 3491 Acked Write 8 Done Done Sent false
9010 3492 Acked Read 6 Done Done Sent false
9011 3493 Acked Write 1 Done Done Sent false
9012 3494 Acked Write 4 Done Done Sent false
9013 3495 Acked Flush 0 Done Done Sent false
9014 3496 Acked Read 7 Done Done Sent false
9015 3497 Acked Read 6 Done Done Sent false
9016 3498 Acked Read 2 Done Done Sent false
9017 3499 Acked Read 10 Done Done Sent false
9018 3500 Acked Read 1 Done Done Sent false
9019 3501 Acked Write 7 Done Done Sent false
9020 3502 Acked Read 1 Done Done Sent false
9021 3503 Acked Write 7 Done Done Sent false
9022 3504 Acked Read 2 Done Done Sent false
9023 3505 Acked Write 9 Done Done Sent false
9024 3506 Acked Read 2 Done Done Sent false
9025 3507 Acked Flush 0 Done Done Sent false
9026 3508 Acked Read 2 Done Done Sent false
9027 3509 Acked Read 7 Done Done Sent false
9028 3510 Acked Read 1 Done Done Sent false
9029 3511 Acked Read 6 Done Done Sent false
9030 3512 Acked Read 4 Done Done Sent false
9031 3513 Acked Read 9 Done Done Sent false
9032 3514 Acked Read 2 Done Done Sent false
9033 3515 Acked Read 9 Done Done Sent false
9034 3516 Acked Read 5 Done Done Sent false
9035 3517 Acked Write 1 Done Done Sent false
9036 3518 Acked Read 3 Done Done Sent false
9037 3519 Acked Write 3 Done Done Sent false
9038 3520 Acked Read 2 Done Done Sent false
9039 3521 Acked Read 8 Done Done Sent false
9040 3522 Acked Flush 0 Done Done Sent false
9041 3523 Acked Write 6 Done Done Sent false
9042 3524 Acked Read 7 Done Done Sent false
9043 3525 Acked Read 3 Done Done Sent false
9044 3526 Acked Read 3 Done Done Sent false
9045 3527 Acked Read 1 Done Done Sent false
9046 3528 Acked Read 4 Done Done Sent false
9047 3529 Acked Flush 0 Done Done Sent false
9048 3530 Acked Write 10 Done Done Sent false
9049 3531 Acked Write 3 Done Done Sent false
9050 3532 Acked Read 5 Done Done Sent false
9051 3533 Acked Flush 0 Done Done Sent false
9052 3534 Acked Read 2 Done Done Sent false
9053 3535 Acked Read 3 Done Done Sent false
9054 3536 Acked Read 7 Done Done Sent false
9055 3537 Acked Read 5 Done Done Sent false
9056 3538 Acked Read 10 Done Done Sent false
9057 3539 Acked Write 4 Done Done Sent false
9058 3540 Acked Write 10 Done Done Sent false
9059 3541 Acked Write 1 Done Done Sent false
9060 3542 Acked Write 8 Done Done Sent false
9061 3543 Acked Write 7 Done Done Sent false
9062 3544 Acked Write 6 Done Done Sent false
9063 3545 Acked Flush 0 Done Done Sent false
9064 3546 Acked Write 1 Done Done Sent false
9065 3547 Acked Read 7 Done Done Sent false
9066 3548 Acked Read 7 Done Done Sent false
9067 3549 Acked Read 1 Done Done Sent false
9068 3550 Acked Write 9 Done Done Sent false
9069 3551 Acked Read 6 Done Done Sent false
9070 3552 Acked Read 5 Done Done Sent false
9071 3553 Acked Write 9 Done Done Sent false
9072 3554 Acked Read 1 Done Done Sent false
9073 3555 Acked Read 7 Done Done Sent false
9074 3556 Acked Write 8 Done Done Sent false
9075 3557 Acked Flush 0 Done Done Sent false
9076 3558 Acked Write 1 Done Done Sent false
9077 3559 Acked Read 6 Done Done Sent false
9078 3560 Acked Write 1 Done Done Sent false
9079 3561 Acked Write 10 Done Done Sent false
9080 3562 Acked Read 4 Done Done Sent false
9081 3563 Acked Read 2 Done Done Sent false
9082 3564 Acked Write 10 Done Done Sent false
9083 3565 Acked Write 1 Done Done Sent false
9084 3566 Acked Write 4 Done Done Sent false
9085 3567 Acked Read 2 Done Done Sent false
9086 3568 Acked Write 8 Done Done Sent false
9087 3569 Acked Read 5 Done Done Sent false
9088 3570 Acked Read 5 Done Done Sent false
9089 3571 Acked Read 3 Done Done Sent false
9090 3572 Acked Read 9 Done Done Sent false
9091 3573 Acked Write 5 Done Done Sent false
9092 3574 Acked Write 6 Done Done Sent false
9093 3575 Acked Read 1 Done Done Sent false
9094 3576 Acked Write 3 Done Done Sent false
9095 3577 Acked Write 7 Done Done Sent false
9096 3578 Acked Read 8 Done Done Sent false
9097 3579 Acked Read 9 Done Done Sent false
9098 3580 Acked Write 8 Done Done Sent false
9099 3581 Acked Write 10 Done Done Sent false
9100 3582 Acked Read 5 Done Done Sent false
9101 3583 Acked Write 2 Done Done Sent false
9102 3584 Acked Write 5 Done Done Sent false
9103 3585 Acked Write 4 Done Done Sent false
9104 3586 Acked Read 7 Done Done Sent false
9105 3587 Acked Read 1 Done Done Sent false
9106 3588 Acked Read 5 Done Done Sent false
9107 3589 Acked Read 4 Done Done Sent false
9108 3590 Acked Read 7 Done Done Sent false
9109 3591 Acked Write 7 Done Done Sent false
9110 3592 Acked Read 5 Done Done Sent false
9111 3593 Acked Read 1 Done Done Sent false
9112 3594 Acked Read 9 Done Done Sent false
9113 3595 Acked Write 10 Done Done Sent false
9114 3596 Acked Read 10 Done Done Sent false
9115 3597 Acked Flush 0 Done Done Sent false
9116 3598 Acked Read 9 Done Done Sent false
9117 3599 Acked Flush 0 Done Done Sent false
9118 3600 Acked Flush 0 Done Done Sent false
9119 3601 Acked Read 3 Done Done Sent false
9120 3602 Acked Read 1 Done Done Sent false
9121 3603 Acked Read 10 Done Done Sent false
9122 3604 Acked Read 2 Done Done Sent false
9123 3605 Acked Write 5 Done Done Sent false
9124 3606 Acked Write 10 Done Done Sent false
9125 3607 Acked Read 4 Done Done Sent false
9126 3608 Acked Write 8 Done Done Sent false
9127 3609 Acked Read 9 Done Done Sent false
9128 3610 Acked Flush 0 Done Done Sent false
9129 3611 Acked Read 4 Done Done Sent false
9130 3612 Acked Write 6 Done Done Sent false
9131 3613 Acked Write 3 Done Done Sent false
9132 3614 Acked Write 1 Done Done Sent false
9133 3615 Acked Write 4 Done Done Sent false
9134 3616 Acked Write 8 Done Done Sent false
9135 3617 Acked Write 8 Done Done Sent false
9136 3618 Acked Write 3 Done Done Sent false
9137 3619 Acked Write 8 Done Done Sent false
9138 3620 Acked Read 2 Done Done Sent false
9139 3621 Acked Read 5 Done Done Sent false
9140 3622 Acked Read 8 Done Done Sent false
9141 3623 Acked Read 6 Done Done Sent false
9142 3624 Acked Flush 0 Done Done Sent false
9143 3625 Acked Read 6 Done Done Sent false
9144 3626 Acked Read 8 Done Done Sent false
9145 3627 Acked Write 5 Done Done Sent false
9146 3628 Acked Read 9 Done Done Sent false
9147 3629 Acked Write 2 Done Done Sent false
9148 3630 Acked Read 10 Done Done Sent false
9149 3631 Acked Write 5 Done Done Sent false
9150 3632 Acked Write 8 Done Done Sent false
9151 3633 Acked Read 5 Done Done Sent false
9152 3634 Acked Write 9 Done Done Sent false
9153 3635 Acked Read 8 Done Done Sent false
9154 3636 Acked Read 1 Done Done Sent false
9155 3637 Acked Write 7 Done Done Sent false
9156 3638 Acked Read 5 Done Done Sent false
9157 3639 Acked Read 5 Done Done Sent false
9158 3640 Acked Write 9 Done Done Sent false
9159 3641 Acked Write 4 Done Done Sent false
9160 3642 Acked Read 9 Done Done Sent false
9161 3643 Acked Write 7 Done Done Sent false
9162 3644 Acked Read 10 Done Done Sent false
9163 3645 Acked Read 10 Done Done Sent false
9164 3646 Acked Read 3 Done Done Sent false
9165 3647 Acked Read 6 Done Done Sent false
9166 3648 Acked Flush 0 Done Done Sent false
9167 3649 Acked Read 5 Done Done Sent false
9168 3650 Acked Write 3 Done Done Sent false
9169 3651 Acked Read 8 Done Done Sent false
9170 3652 Acked Read 6 Done Done Sent false
9171 3653 Acked Read 10 Done Done Sent false
9172 3654 Acked Read 3 Done Done Sent false
9173 3655 Acked Read 7 Done Done Sent false
9174 3656 Acked Read 1 Done Done Sent false
9175 3657 Acked Read 9 Done Done Sent false
9176 3658 Acked Read 6 Done Done Sent false
9177 3659 Acked Write 2 Done Done Sent false
9178 3660 Acked Read 9 Done Done Sent false
9179 3661 Acked Write 5 Done Done Sent false
9180 3662 Acked Read 6 Done Done Sent false
9181 3663 Acked Write 8 Done Done Sent false
9182 3664 Acked Write 10 Done Done Sent false
9183 3665 Acked Write 7 Done Done Sent false
9184 3666 Acked Write 1 Done Done Sent false
9185 3667 Acked Write 2 Done Done Sent false
9186 3668 Acked Write 2 Done Done Sent false
9187 3669 Acked Write 10 Done Done Sent false
9188 3670 Acked Write 9 Done Done Sent false
9189 3671 Acked Read 6 Done Done Sent false
9190 3672 Acked Read 5 Done Done Sent false
9191 3673 Acked Read 8 Done Done Sent false
9192 3674 Acked Write 9 Done Done Sent false
9193 3675 Acked Write 5 Done Done Sent false
9194 3676 Acked Write 2 Done Done Sent false
9195 3677 Acked Write 4 Done Done Sent false
9196 3678 Acked Read 7 Done Done Sent false
9197 3679 Acked Read 9 Done Done Sent false
9198 3680 Acked Write 2 Done Done Sent false
9199 3681 Acked Read 5 Done Done Sent false
9200 3682 Acked Write 4 Done Done Sent false
9201 3683 Acked Write 7 Done Done Sent false
9202 3684 Acked Write 9 Done Done Sent false
9203 3685 Acked Read 5 Done Done Sent false
9204 3686 Acked Read 5 Done Done Sent false
9205 3687 Acked Read 1 Done Done Sent false
9206 3688 Acked Read 5 Done Done Sent false
9207 3689 Acked Read 10 Done Done Sent false
9208 3690 Acked Read 3 Done Done Sent false
9209 3691 Acked Flush 0 Done Done Sent false
9210 3692 Acked Read 2 Done Done Sent false
9211 3693 Acked Write 4 Done Done Sent false
9212 3694 Acked Read 8 Done Done Sent false
9213 3695 Acked Read 7 Done Done Sent false
9214 3696 Acked Write 8 Done Done Sent false
9215 3697 Acked Read 5 Done Done Sent false
9216 3698 Acked Read 6 Done Done Sent false
9217 3699 Acked Flush 0 Done Done Sent false
9218 3700 Acked Read 3 Done Done Sent false
9219 3701 Acked Write 6 Done Done Sent false
9220 3702 Acked Write 9 Done Done Sent false
9221 3703 Acked Write 4 Done Done Sent false
9222 3704 Acked Read 7 Done Done Sent false
9223 3705 Acked Read 5 Done Done Sent false
9224 3706 Acked Read 8 Done Done Sent false
9225 3707 Acked Write 5 Done Done Sent false
9226 3708 Acked Read 4 Done Done Sent false
9227 3709 Acked Read 7 Done Done Sent false
9228 3710 Acked Write 4 Done Done Sent false
9229 3711 Acked Read 3 Done Done Sent false
9230 3712 Acked Read 5 Done Done Sent false
9231 3713 Acked Read 8 Done Done Sent false
9232 3714 Acked Write 6 Done Done Sent false
9233 STATES DS:0 DS:1 DS:2 TOTAL
9234 Sent 0 0 301 301
9235 Done 301 301 0 602
9236 Skipped 0 0 0 0
9237 Error 0 0 0 0
9238 Last Flush: 3699 3699 3413
9239 Downstairs last five completed: 3413 3412 3411 3410 3409
9240 Upstairs last five completed: 3714 3713 3712 3711 3710
9241 CLIENT: Up:0 ds:301 act:2
9242 [8830] got start action
9243 [8830] I am going to restart
9244 Starting downstairs at port 8830
9245 Make output file at "/tmp/dsc/downstairs-8830.txt"
9246 [8830][2] reports Starting
9247 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1211)
9248 [8830][2] reports Running
9249 ----------------------------------------------------------------
9250 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
9251 States: Active Active Offline
9252 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9253 3414 Acked Read 10 Done Done Sent false
9254 3415 Acked Read 5 Done Done Sent false
9255 3416 Acked Read 8 Done Done Sent false
9256 3417 Acked Write 4 Done Done Sent false
9257 3418 Acked Flush 0 Done Done Sent false
9258 3419 Acked Write 4 Done Done Sent false
9259 3420 Acked Read 5 Done Done Sent false
9260 3421 Acked Read 5 Done Done Sent false
9261 3422 Acked Read 6 Done Done Sent false
9262 3423 Acked Flush 0 Done Done Sent false
9263 3424 Acked Read 4 Done Done Sent false
9264 3425 Acked Write 10 Done Done Sent false
9265 3426 Acked Read 8 Done Done Sent false
9266 3427 Acked Read 10 Done Done Sent false
9267 3428 Acked Write 4 Done Done Sent false
9268 3429 Acked Read 3 Done Done Sent false
9269 3430 Acked Flush 0 Done Done Sent false
9270 3431 Acked Write 6 Done Done Sent false
9271 3432 Acked Read 5 Done Done Sent false
9272 3433 Acked Read 2 Done Done Sent false
9273 3434 Acked Read 10 Done Done Sent false
9274 3435 Acked Read 4 Done Done Sent false
9275 3436 Acked Read 2 Done Done Sent false
9276 3437 Acked Write 3 Done Done Sent false
9277 3438 Acked Read 9 Done Done Sent false
9278 3439 Acked Write 10 Done Done Sent false
9279 3440 Acked Flush 0 Done Done Sent false
9280 3441 Acked Write 2 Done Done Sent false
9281 3442 Acked Read 8 Done Done Sent false
9282 3443 Acked Read 5 Done Done Sent false
9283 3444 Acked Read 9 Done Done Sent false
9284 3445 Acked Write 7 Done Done Sent false
9285 3446 Acked Read 3 Done Done Sent false
9286 3447 Acked Read 6 Done Done Sent false
9287 3448 Acked Read 4 Done Done Sent false
9288 3449 Acked Write 4 Done Done Sent false
9289 3450 Acked Write 5 Done Done Sent false
9290 3451 Acked Read 2 Done Done Sent false
9291 3452 Acked Read 8 Done Done Sent false
9292 3453 Acked Write 2 Done Done Sent false
9293 3454 Acked Write 5 Done Done Sent false
9294 3455 Acked Read 3 Done Done Sent false
9295 3456 Acked Read 1 Done Done Sent false
9296 3457 Acked Read 2 Done Done Sent false
9297 3458 Acked Read 9 Done Done Sent false
9298 3459 Acked Read 4 Done Done Sent false
9299 3460 Acked Write 9 Done Done Sent false
9300 3461 Acked Write 10 Done Done Sent false
9301 3462 Acked Write 8 Done Done Sent false
9302 3463 Acked Write 10 Done Done Sent false
9303 3464 Acked Read 4 Done Done Sent false
9304 3465 Acked Write 7 Done Done Sent false
9305 3466 Acked Read 4 Done Done Sent false
9306 3467 Acked Read 3 Done Done Sent false
9307 3468 Acked Read 9 Done Done Sent false
9308 3469 Acked Write 3 Done Done Sent false
9309 3470 Acked Read 4 Done Done Sent false
9310 3471 Acked Write 3 Done Done Sent false
9311 3472 Acked Write 8 Done Done Sent false
9312 3473 Acked Write 2 Done Done Sent false
9313 3474 Acked Flush 0 Done Done Sent false
9314 3475 Acked Write 7 Done Done Sent false
9315 3476 Acked Write 6 Done Done Sent false
9316 3477 Acked Write 2 Done Done Sent false
9317 3478 Acked Read 3 Done Done Sent false
9318 3479 Acked Write 5 Done Done Sent false
9319 3480 Acked Read 9 Done Done Sent false
9320 3481 Acked Write 9 Done Done Sent false
9321 3482 Acked Write 3 Done Done Sent false
9322 3483 Acked Write 9 Done Done Sent false
9323 3484 Acked Read 9 Done Done Sent false
9324 3485 Acked Flush 0 Done Done Sent false
9325 3486 Acked Read 10 Done Done Sent false
9326 3487 Acked Write 2 Done Done Sent false
9327 3488 Acked Read 5 Done Done Sent false
9328 3489 Acked Write 1 Done Done Sent false
9329 3490 Acked Read 10 Done Done Sent false
9330 3491 Acked Write 8 Done Done Sent false
9331 3492 Acked Read 6 Done Done Sent false
9332 3493 Acked Write 1 Done Done Sent false
9333 3494 Acked Write 4 Done Done Sent false
9334 3495 Acked Flush 0 Done Done Sent false
9335 3496 Acked Read 7 Done Done Sent false
9336 3497 Acked Read 6 Done Done Sent false
9337 3498 Acked Read 2 Done Done Sent false
9338 3499 Acked Read 10 Done Done Sent false
9339 3500 Acked Read 1 Done Done Sent false
9340 3501 Acked Write 7 Done Done Sent false
9341 3502 Acked Read 1 Done Done Sent false
9342 3503 Acked Write 7 Done Done Sent false
9343 3504 Acked Read 2 Done Done Sent false
9344 3505 Acked Write 9 Done Done Sent false
9345 3506 Acked Read 2 Done Done Sent false
9346 3507 Acked Flush 0 Done Done Sent false
9347 3508 Acked Read 2 Done Done Sent false
9348 3509 Acked Read 7 Done Done Sent false
9349 3510 Acked Read 1 Done Done Sent false
9350 3511 Acked Read 6 Done Done Sent false
9351 3512 Acked Read 4 Done Done Sent false
9352 3513 Acked Read 9 Done Done Sent false
9353 3514 Acked Read 2 Done Done Sent false
9354 3515 Acked Read 9 Done Done Sent false
9355 3516 Acked Read 5 Done Done Sent false
9356 3517 Acked Write 1 Done Done Sent false
9357 3518 Acked Read 3 Done Done Sent false
9358 3519 Acked Write 3 Done Done Sent false
9359 3520 Acked Read 2 Done Done Sent false
9360 3521 Acked Read 8 Done Done Sent false
9361 3522 Acked Flush 0 Done Done Sent false
9362 3523 Acked Write 6 Done Done Sent false
9363 3524 Acked Read 7 Done Done Sent false
9364 3525 Acked Read 3 Done Done Sent false
9365 3526 Acked Read 3 Done Done Sent false
9366 3527 Acked Read 1 Done Done Sent false
9367 3528 Acked Read 4 Done Done Sent false
9368 3529 Acked Flush 0 Done Done Sent false
9369 3530 Acked Write 10 Done Done Sent false
9370 3531 Acked Write 3 Done Done Sent false
9371 3532 Acked Read 5 Done Done Sent false
9372 3533 Acked Flush 0 Done Done Sent false
9373 3534 Acked Read 2 Done Done Sent false
9374 3535 Acked Read 3 Done Done Sent false
9375 3536 Acked Read 7 Done Done Sent false
9376 3537 Acked Read 5 Done Done Sent false
9377 3538 Acked Read 10 Done Done Sent false
9378 3539 Acked Write 4 Done Done Sent false
9379 3540 Acked Write 10 Done Done Sent false
9380 3541 Acked Write 1 Done Done Sent false
9381 3542 Acked Write 8 Done Done Sent false
9382 3543 Acked Write 7 Done Done Sent false
9383 3544 Acked Write 6 Done Done Sent false
9384 3545 Acked Flush 0 Done Done Sent false
9385 3546 Acked Write 1 Done Done Sent false
9386 3547 Acked Read 7 Done Done Sent false
9387 3548 Acked Read 7 Done Done Sent false
9388 3549 Acked Read 1 Done Done Sent false
9389 3550 Acked Write 9 Done Done Sent false
9390 3551 Acked Read 6 Done Done Sent false
9391 3552 Acked Read 5 Done Done Sent false
9392 3553 Acked Write 9 Done Done Sent false
9393 3554 Acked Read 1 Done Done Sent false
9394 3555 Acked Read 7 Done Done Sent false
9395 3556 Acked Write 8 Done Done Sent false
9396 3557 Acked Flush 0 Done Done Sent false
9397 3558 Acked Write 1 Done Done Sent false
9398 3559 Acked Read 6 Done Done Sent false
9399 3560 Acked Write 1 Done Done Sent false
9400 3561 Acked Write 10 Done Done Sent false
9401 3562 Acked Read 4 Done Done Sent false
9402 3563 Acked Read 2 Done Done Sent false
9403 3564 Acked Write 10 Done Done Sent false
9404 3565 Acked Write 1 Done Done Sent false
9405 3566 Acked Write 4 Done Done Sent false
9406 3567 Acked Read 2 Done Done Sent false
9407 3568 Acked Write 8 Done Done Sent false
9408 3569 Acked Read 5 Done Done Sent false
9409 3570 Acked Read 5 Done Done Sent false
9410 3571 Acked Read 3 Done Done Sent false
9411 3572 Acked Read 9 Done Done Sent false
9412 3573 Acked Write 5 Done Done Sent false
9413 3574 Acked Write 6 Done Done Sent false
9414 3575 Acked Read 1 Done Done Sent false
9415 3576 Acked Write 3 Done Done Sent false
9416 3577 Acked Write 7 Done Done Sent false
9417 3578 Acked Read 8 Done Done Sent false
9418 3579 Acked Read 9 Done Done Sent false
9419 3580 Acked Write 8 Done Done Sent false
9420 3581 Acked Write 10 Done Done Sent false
9421 3582 Acked Read 5 Done Done Sent false
9422 3583 Acked Write 2 Done Done Sent false
9423 3584 Acked Write 5 Done Done Sent false
9424 3585 Acked Write 4 Done Done Sent false
9425 3586 Acked Read 7 Done Done Sent false
9426 3587 Acked Read 1 Done Done Sent false
9427 3588 Acked Read 5 Done Done Sent false
9428 3589 Acked Read 4 Done Done Sent false
9429 3590 Acked Read 7 Done Done Sent false
9430 3591 Acked Write 7 Done Done Sent false
9431 3592 Acked Read 5 Done Done Sent false
9432 3593 Acked Read 1 Done Done Sent false
9433 3594 Acked Read 9 Done Done Sent false
9434 3595 Acked Write 10 Done Done Sent false
9435 3596 Acked Read 10 Done Done Sent false
9436 3597 Acked Flush 0 Done Done Sent false
9437 3598 Acked Read 9 Done Done Sent false
9438 3599 Acked Flush 0 Done Done Sent false
9439 3600 Acked Flush 0 Done Done Sent false
9440 3601 Acked Read 3 Done Done Sent false
9441 3602 Acked Read 1 Done Done Sent false
9442 3603 Acked Read 10 Done Done Sent false
9443 3604 Acked Read 2 Done Done Sent false
9444 3605 Acked Write 5 Done Done Sent false
9445 3606 Acked Write 10 Done Done Sent false
9446 3607 Acked Read 4 Done Done Sent false
9447 3608 Acked Write 8 Done Done Sent false
9448 3609 Acked Read 9 Done Done Sent false
9449 3610 Acked Flush 0 Done Done Sent false
9450 3611 Acked Read 4 Done Done Sent false
9451 3612 Acked Write 6 Done Done Sent false
9452 3613 Acked Write 3 Done Done Sent false
9453 3614 Acked Write 1 Done Done Sent false
9454 3615 Acked Write 4 Done Done Sent false
9455 3616 Acked Write 8 Done Done Sent false
9456 3617 Acked Write 8 Done Done Sent false
9457 3618 Acked Write 3 Done Done Sent false
9458 3619 Acked Write 8 Done Done Sent false
9459 3620 Acked Read 2 Done Done Sent false
9460 3621 Acked Read 5 Done Done Sent false
9461 3622 Acked Read 8 Done Done Sent false
9462 3623 Acked Read 6 Done Done Sent false
9463 3624 Acked Flush 0 Done Done Sent false
9464 3625 Acked Read 6 Done Done Sent false
9465 3626 Acked Read 8 Done Done Sent false
9466 3627 Acked Write 5 Done Done Sent false
9467 3628 Acked Read 9 Done Done Sent false
9468 3629 Acked Write 2 Done Done Sent false
9469 3630 Acked Read 10 Done Done Sent false
9470 3631 Acked Write 5 Done Done Sent false
9471 3632 Acked Write 8 Done Done Sent false
9472 3633 Acked Read 5 Done Done Sent false
9473 3634 Acked Write 9 Done Done Sent false
9474 3635 Acked Read 8 Done Done Sent false
9475 3636 Acked Read 1 Done Done Sent false
9476 3637 Acked Write 7 Done Done Sent false
9477 3638 Acked Read 5 Done Done Sent false
9478 3639 Acked Read 5 Done Done Sent false
9479 3640 Acked Write 9 Done Done Sent false
9480 3641 Acked Write 4 Done Done Sent false
9481 3642 Acked Read 9 Done Done Sent false
9482 3643 Acked Write 7 Done Done Sent false
9483 3644 Acked Read 10 Done Done Sent false
9484 3645 Acked Read 10 Done Done Sent false
9485 3646 Acked Read 3 Done Done Sent false
9486 3647 Acked Read 6 Done Done Sent false
9487 3648 Acked Flush 0 Done Done Sent false
9488 3649 Acked Read 5 Done Done Sent false
9489 3650 Acked Write 3 Done Done Sent false
9490 3651 Acked Read 8 Done Done Sent false
9491 3652 Acked Read 6 Done Done Sent false
9492 3653 Acked Read 10 Done Done Sent false
9493 3654 Acked Read 3 Done Done Sent false
9494 3655 Acked Read 7 Done Done Sent false
9495 3656 Acked Read 1 Done Done Sent false
9496 3657 Acked Read 9 Done Done Sent false
9497 3658 Acked Read 6 Done Done Sent false
9498 3659 Acked Write 2 Done Done Sent false
9499 3660 Acked Read 9 Done Done Sent false
9500 3661 Acked Write 5 Done Done Sent false
9501 3662 Acked Read 6 Done Done Sent false
9502 3663 Acked Write 8 Done Done Sent false
9503 3664 Acked Write 10 Done Done Sent false
9504 3665 Acked Write 7 Done Done Sent false
9505 3666 Acked Write 1 Done Done Sent false
9506 3667 Acked Write 2 Done Done Sent false
9507 3668 Acked Write 2 Done Done Sent false
9508 3669 Acked Write 10 Done Done Sent false
9509 3670 Acked Write 9 Done Done Sent false
9510 3671 Acked Read 6 Done Done Sent false
9511 3672 Acked Read 5 Done Done Sent false
9512 3673 Acked Read 8 Done Done Sent false
9513 3674 Acked Write 9 Done Done Sent false
9514 3675 Acked Write 5 Done Done Sent false
9515 3676 Acked Write 2 Done Done Sent false
9516 3677 Acked Write 4 Done Done Sent false
9517 3678 Acked Read 7 Done Done Sent false
9518 3679 Acked Read 9 Done Done Sent false
9519 3680 Acked Write 2 Done Done Sent false
9520 3681 Acked Read 5 Done Done Sent false
9521 3682 Acked Write 4 Done Done Sent false
9522 3683 Acked Write 7 Done Done Sent false
9523 3684 Acked Write 9 Done Done Sent false
9524 3685 Acked Read 5 Done Done Sent false
9525 3686 Acked Read 5 Done Done Sent false
9526 3687 Acked Read 1 Done Done Sent false
9527 3688 Acked Read 5 Done Done Sent false
9528 3689 Acked Read 10 Done Done Sent false
9529 3690 Acked Read 3 Done Done Sent false
9530 3691 Acked Flush 0 Done Done Sent false
9531 3692 Acked Read 2 Done Done Sent false
9532 3693 Acked Write 4 Done Done Sent false
9533 3694 Acked Read 8 Done Done Sent false
9534 3695 Acked Read 7 Done Done Sent false
9535 3696 Acked Write 8 Done Done Sent false
9536 3697 Acked Read 5 Done Done Sent false
9537 3698 Acked Read 6 Done Done Sent false
9538 3699 Acked Flush 0 Done Done Sent false
9539 3700 Acked Read 3 Done Done Sent false
9540 3701 Acked Write 6 Done Done Sent false
9541 3702 Acked Write 9 Done Done Sent false
9542 3703 Acked Write 4 Done Done Sent false
9543 3704 Acked Read 7 Done Done Sent false
9544 3705 Acked Read 5 Done Done Sent false
9545 3706 Acked Read 8 Done Done Sent false
9546 3707 Acked Write 5 Done Done Sent false
9547 3708 Acked Read 4 Done Done Sent false
9548 3709 Acked Read 7 Done Done Sent false
9549 3710 Acked Write 4 Done Done Sent false
9550 3711 Acked Read 3 Done Done Sent false
9551 3712 Acked Read 5 Done Done Sent false
9552 3713 Acked Read 8 Done Done Sent false
9553 3714 Acked Write 6 Done Done Sent false
9554 3715 Acked Flush 0 Done Done Sent false
9555 STATES DS:0 DS:1 DS:2 TOTAL
9556 Sent 0 0 302 302
9557 Done 302 302 0 604
9558 Skipped 0 0 0 0
9559 Error 0 0 0 0
9560 Last Flush: 3715 3715 3413
9561 Downstairs last five completed: 3413 3412 3411 3410 3409
9562 Upstairs last five completed: 3715 3714 3713 3712 3711
9563 CLIENT: Up:0 ds:302 act:2
95642025-01-14T16:12:35.890ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
95652025-01-14T16:12:35.890ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:34806) = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
95662025-01-14T16:12:35.891ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
95672025-01-14T16:12:35.892ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
95682025-01-14T16:12:35.892ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
95692025-01-14T16:12:35.892ZINFOcrucible: send last flush ID to this DS: 3413 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
95702025-01-14T16:12:35.892ZINFOcrucible: Replied this last flush ID: 3413 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
95712025-01-14T16:12:35.892ZINFOcrucible: [2] client re-new 302 jobs since flush 3413 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
95722025-01-14T16:12:35.892ZINFOcrucible: [2] Replayed 302 jobs since flush: 3413 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
9573 ----------------------------------------------------------------
9574 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
9575 Downstairs last five completed: 3715 3714 3713 3712 3711
9576 Upstairs last five completed: 3715 3714 3713 3712 3711
9577 CLIENT: Up:0 ds:0 act:3
9578 Replay: All jobs finished
9579 Main task has work to do, go find it
9580 got dsc Stop(1)
9581 stop 1
9582 Jan 14 16:12:38.140 [8820] Got stop action so:false kr:false
9583 INFO request completed, latency_us: 339, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 6b08295a-f645-43e5-843f-9a7f90a0d96d, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
9584 Jan 14 16:12:38.141 INFO request completed, latency_us: 104, response_code: 200, uri: /state/cid/1, method: GET, req_id: 26f80b32-681a-4eb7-8b70-22e3985157f0, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
95852025-01-14T16:12:38.141ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
95862025-01-14T16:12:38.141ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
95872025-01-14T16:12:38.141ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
95882025-01-14T16:12:38.141ZINFOcrucible: client task is exiting = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
9589 [8820] Exited with: ExitStatus(unix_wait_status(9))
9590 [8820][1] reports Exit
9591 Jan 14 16:12:42.142 INFO request completed, latency_us: 201, response_code: 200, uri: /state/cid/1, method: GET, req_id: b8a9c8dd-961c-4ce1-b5e8-a898345c5da6, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
9592 001/300 Write block 94 len 4096 data: 2
9593 002/300 Write block 4389 len 20480 data: 2 3 4 4 5
9594 003/300 Write block 937 len 24576 data: 3 3 3 2 2 2
9595 004/300 Write block 1894 len 12288 data: 6 6 5
9596 005/300 Read block 2648 len 4096
9597 006/300 Write block 4107 len 16384 data: 3 2 2 2
9598 007/300 Write block 1234 len 32768 data: 3 3 4 4 4 4 4 2
9599 008/300 Read block 1895 len 32768
9600 009/300 Write block 2348 len 4096 data: 2
9601 010/300 Write block 4172 len 24576 data: 3 3 3 3 4 4
9602 011/300 Write block 688 len 24576 data: 4 4 3 3 3 3
9603 012/300 Flush
9604 013/300 Read block 767 len 8192
9605 014/300 Read block 2553 len 36864
9606 015/300 Write block 653 len 12288 data: 3 3 3
9607 016/300 Flush
9608 017/300 Flush
9609 018/300 Read block 3320 len 16384
9610 019/300 Write block 837 len 36864 data: 2 2 2 2 2 2 2 2 2
9611 020/300 Read block 1168 len 32768
9612 021/300 Write block 3865 len 24576 data: 2 2 3 3 3 3
9613 022/300 Read block 711 len 36864
9614 023/300 Read block 2159 len 36864
9615 024/300 Read block 4557 len 40960
9616 025/300 Write block 911 len 24576 data: 5 4 3 2 2 2
9617 026/300 Flush
9618 027/300 Read block 3783 len 24576
9619 028/300 Read block 3432 len 4096
9620 029/300 Read block 3771 len 24576
9621 030/300 Flush
9622 031/300 Read block 2852 len 40960
9623 032/300 Flush
9624 033/300 Write block 322 len 40960 data: 5 4 5 5 6 5 5 5 4 4
9625 034/300 Write block 3888 len 8192 data: 4 4
9626 035/300 Write block 140 len 36864 data: 4 2 3 5 5 5 5 5 5
9627 036/300 Write block 1766 len 28672 data: 2 2 2 2 3 3 3
9628 037/300 Write block 152 len 16384 data: 2 3 2 2
9629 038/300 Write block 837 len 32768 data: 3 3 3 3 3 3 3 3
9630 039/300 Read block 1720 len 4096
9631 040/300 Write block 443 len 28672 data: 3 3 3 2 2 3 5
9632 041/300 Read block 3446 len 40960
9633 042/300 Read block 2020 len 4096
9634 043/300 Read block 4016 len 36864
9635 044/300 Flush
9636 045/300 Read block 378 len 8192
9637 046/300 Read block 3641 len 16384
9638 047/300 Read block 3289 len 4096
9639 048/300 Read block 486 len 4096
9640 049/300 Read block 3571 len 12288
9641 050/300 Write block 122 len 36864 data: 3 3 3 3 3 3 3 2 2
9642 051/300 Read block 4102 len 16384
9643 052/300 Flush
9644 053/300 Read block 2162 len 40960
9645 054/300 Read block 527 len 40960
9646 055/300 Write block 1366 len 12288 data: 3 3 2
9647 056/300 Read block 1368 len 4096
9648 057/300 Write block 1532 len 8192 data: 5 4
9649 058/300 Read block 2431 len 28672
9650 059/300 Read block 962 len 16384
9651 060/300 Read block 708 len 32768
9652 061/300 Read block 4623 len 16384
9653 062/300 Flush
9654 063/300 Read block 862 len 40960
9655 064/300 Read block 964 len 36864
9656 065/300 Write block 317 len 16384 data: 4 4 5 5
9657 066/300 Read block 3360 len 24576
9658 067/300 Read block 4476 len 12288
9659 068/300 Read block 4365 len 12288
9660 069/300 Write block 4819 len 40960 data: 2 2 2 3 3 3 3 3 3 3
9661 070/300 Flush
9662 071/300 Write block 4625 len 4096 data: 3
9663 072/300 Write block 1669 len 8192 data: 2 3
9664 073/300 Read block 1129 len 28672
9665 074/300 Read block 4281 len 8192
9666 075/300 Read block 4758 len 8192
9667 076/300 Flush
9668 077/300 Read block 4262 len 16384
9669 078/300 Write block 2262 len 16384 data: 3 3 2 2
9670 079/300 Write block 543 len 36864 data: 3 3 3 3 4 4 4 3 4
9671 080/300 Write block 2389 len 12288 data: 3 4 4
9672 081/300 Flush
9673 082/300 Write block 3830 len 32768 data: 2 3 3 3 3 3 4 3
9674 083/300 Write block 1185 len 28672 data: 3 3 2 2 2 3 3
9675 084/300 Flush
9676 085/300 Read block 2968 len 12288
9677 086/300 Read block 4456 len 28672
9678 087/300 Write block 1600 len 32768 data: 2 2 2 2 2 2 2 3
9679 088/300 Flush
9680 089/300 Write block 576 len 16384 data: 2 2 2 2
9681 090/300 Read block 2569 len 16384
9682 091/300 Read block 3845 len 40960
9683 092/300 Read block 4529 len 24576
9684 093/300 Read block 449 len 12288
9685 094/300 Read block 4337 len 32768
9686 095/300 Write block 1799 len 28672 data: 3 3 3 3 3 2 3
9687 096/300 Write block 1574 len 28672 data: 2 2 2 3 3 2 2
9688 097/300 Read block 3763 len 40960
9689 098/300 Flush
9690 099/300 Write block 2333 len 20480 data: 4 3 3 3 2
9691 100/300 Write block 4813 len 8192 data: 2 2
9692 101/300 Read block 3360 len 16384
9693 102/300 Read block 672 len 36864
9694 103/300 Write block 3746 len 20480 data: 2 2 2 2 2
9695 104/300 Read block 103 len 32768
9696 105/300 Write block 4005 len 36864 data: 4 4 5 4 3 4 4 4 4
9697 106/300 Flush
9698 107/300 Read block 611 len 4096
9699 108/300 Write block 3397 len 20480 data: 4 4 4 4 3
9700 109/300 Read block 1689 len 16384
9701 110/300 Read block 3778 len 40960
9702 111/300 Flush
9703 112/300 Read block 3216 len 24576
9704 113/300 Write block 3522 len 36864 data: 3 3 3 3 2 2 2 2 2
9705 114/300 Write block 2064 len 24576 data: 2 2 2 2 2 2
9706 115/300 Write block 887 len 4096 data: 6
9707 116/300 Flush
9708 117/300 Flush
9709 118/300 Write block 1398 len 8192 data: 3 3
9710 119/300 Read block 4876 len 40960
9711 120/300 Write block 4573 len 40960 data: 2 2 2 2 3 3 2 2 2 2
9712 121/300 Read block 3903 len 32768
9713 122/300 Read block 123 len 28672
9714 123/300 Read block 1189 len 20480
9715 124/300 Read block 2509 len 40960
9716 125/300 Read block 1358 len 4096
9717 126/300 Read block 3758 len 20480
9718 127/300 Write block 1325 len 24576 data: 3 3 2 3 3 3
9719 128/300 Read block 435 len 12288
9720 129/300 Write block 418 len 12288 data: 5 4 5
9721 130/300 Read block 1352 len 36864
9722 131/300 Write block 3612 len 32768 data: 4 4 4 3 2 2 2 2
9723 132/300 Read block 3844 len 4096
9724 133/300 Read block 4540 len 28672
9725 134/300 Read block 2547 len 12288
9726 135/300 Read block 4458 len 32768
9727 136/300 Write block 4302 len 16384 data: 3 3 2 3
9728 137/300 Read block 3813 len 8192
9729 138/300 Write block 4816 len 4096 data: 2
9730 139/300 Write block 374 len 40960 data: 6 5 6 3 2 2 2 2 2 2
9731 140/300 Read block 2902 len 36864
9732 141/300 Read block 2712 len 20480
9733 142/300 Write block 4079 len 16384 data: 3 3 3 3
9734 143/300 Read block 312 len 8192
9735 144/300 Write block 2359 len 16384 data: 2 2 3 3
9736 145/300 Read block 2692 len 36864
9737 146/300 Write block 383 len 12288 data: 3 2 2
9738 147/300 Write block 4136 len 16384 data: 3 3 5 4
9739 148/300 Read block 4124 len 20480
9740 149/300 Write block 3128 len 40960 data: 2 3 3 3 3 3 3 3 3 3
9741 150/300 Read block 3422 len 32768
9742 151/300 Read block 1193 len 16384
9743 152/300 Write block 3420 len 28672 data: 6 6 5 5 5 4 4
9744 153/300 Write block 4838 len 28672 data: 2 2 2 2 3 3 4
9745 154/300 Read block 3255 len 16384
9746 155/300 Write block 404 len 8192 data: 5 3
9747 156/300 Read block 4005 len 20480
9748 157/300 Read block 565 len 16384
9749 158/300 Read block 237 len 8192
9750 159/300 Flush
9751 160/300 Write block 1255 len 16384 data: 3 3 3 3
9752 161/300 Write block 1938 len 24576 data: 3 3 3 3 3 3
9753 162/300 Write block 1965 len 32768 data: 2 2 2 2 2 3 3 2
9754 163/300 Write block 4335 len 12288 data: 2 2 2
9755 164/300 Read block 3514 len 12288
9756 165/300 Read block 363 len 40960
9757 166/300 Write block 2170 len 8192 data: 3 3
9758 167/300 Read block 705 len 36864
9759 168/300 Flush
9760 169/300 Read block 2918 len 12288
9761 170/300 Write block 2413 len 8192 data: 3 3
9762 171/300 Flush
9763 172/300 Read block 4831 len 32768
9764 173/300 Write block 542 len 4096 data: 3
9765 174/300 Write block 4492 len 4096 data: 2
9766 175/300 Flush
9767 176/300 Read block 957 len 20480
9768 177/300 Read block 1702 len 20480
9769 178/300 Read block 2163 len 28672
9770 179/300 Read block 86 len 32768
9771 180/300 Write block 786 len 16384 data: 2 2 2 2
9772 181/300 Write block 2851 len 4096 data: 2
9773 182/300 Read block 3414 len 16384
9774 183/300 Read block 1983 len 40960
9775 184/300 Read block 1574 len 8192
9776 185/300 Flush
9777 186/300 Read block 4381 len 40960
9778 187/300 Write block 4386 len 28672 data: 2 2 2 3 4 5 5
9779 188/300 Read block 3958 len 32768
9780 189/300 Read block 3110 len 24576
9781 190/300 Write block 3691 len 12288 data: 3 3 3
9782 191/300 Read block 847 len 32768
9783 192/300 Flush
9784 193/300 Write block 4994 len 20480 data: 3 3 3 3 2
9785 194/300 Read block 3075 len 32768
9786 195/300 Read block 3510 len 28672
9787 196/300 Read block 1239 len 12288
9788 197/300 Write block 4278 len 32768 data: 3 3 4 4 4 4 4 3
9789 198/300 Write block 1039 len 12288 data: 3 3 2
9790 199/300 Read block 4004 len 16384
9791 200/300 Flush
9792 201/300 Read block 819 len 28672
9793 202/300 Read block 2626 len 32768
9794 203/300 Read block 3194 len 12288
9795 204/300 Read block 337 len 4096
9796 205/300 Read block 769 len 24576
9797 206/300 Read block 360 len 40960
9798 207/300 Flush
9799 208/300 Read block 4469 len 32768
9800 209/300 Write block 1536 len 32768 data: 3 3 4 4 3 3 4 4
9801 210/300 Write block 1440 len 32768 data: 4 4 3 3 3 3 4 4
9802 211/300 Write block 1768 len 8192 data: 3 3
9803 212/300 Write block 4634 len 16384 data: 4 3 3 3
9804 213/300 Flush
9805 214/300 Read block 4518 len 28672
9806 215/300 Write block 381 len 4096 data: 3
9807 216/300 Write block 1003 len 28672 data: 4 4 4 4 4 3 3
9808 217/300 Write block 4838 len 32768 data: 3 3 3 3 4 4 5 4
9809 218/300 Read block 636 len 8192
9810 219/300 Flush
9811 220/300 Read block 4100 len 4096
9812 221/300 Read block 175 len 4096
9813 222/300 Read block 1748 len 20480
9814 223/300 Flush
9815 224/300 Read block 866 len 16384
9816 225/300 Read block 3630 len 32768
9817 226/300 Read block 877 len 24576
9818 227/300 Read block 3575 len 16384
9819 228/300 Write block 3231 len 8192 data: 4 4
9820 229/300 Write block 4368 len 24576 data: 4 4 3 4 5 4
9821 230/300 Read block 4099 len 40960
9822 231/300 Write block 1265 len 20480 data: 2 2 2 2 2
9823 232/300 Write block 22 len 40960 data: 4 4 5 4 4 4 5 4 3 3
9824 233/300 Read block 3443 len 12288
9825 234/300 Write block 3364 len 24576 data: 2 3 3 3 3 3
9826 235/300 Write block 4982 len 12288 data: 3 3 3
9827 236/300 Write block 1450 len 28672 data: 3 4 4 4 4 5 4
9828 237/300 Write block 3827 len 28672 data: 3 3 3 3 4 4 4
9829 238/300 Write block 3390 len 12288 data: 2 3 3
9830 239/300 Write block 4895 len 16384 data: 3 3 3 4
9831 240/300 Flush
9832 241/300 Read block 230 len 16384
9833 242/300 Write block 3815 len 12288 data: 3 2 3
9834 243/300 Read block 2237 len 12288
9835 244/300 Read block 68 len 28672
9836 245/300 Write block 2118 len 12288 data: 3 3 3
9837 246/300 Flush
9838 247/300 Read block 1000 len 8192
9839 248/300 Read block 3329 len 24576
9840 249/300 Write block 1727 len 32768 data: 3 3 3 3 3 3 3 2
9841 250/300 Read block 2799 len 12288
9842 251/300 Flush
9843 252/300 Read block 1191 len 20480
9844 253/300 Read block 2462 len 32768
9845 254/300 Write block 3631 len 36864 data: 3 3 5 6 6 6 6 5 5
9846 255/300 Write block 59 len 40960 data: 3 3 2 2 2 2 3 3 3 3
9847 256/300 Write block 1474 len 40960 data: 3 3 3 3 3 3 3 3 3 3
9848 257/300 Read block 2814 len 36864
9849 258/300 Read block 2895 len 32768
9850 259/300 Write block 963 len 4096 data: 3
9851 260/300 Read block 1329 len 32768
9852 261/300 Write block 2795 len 28672 data: 2 2 2 2 2 2 3
9853 262/300 Read block 159 len 16384
9854 263/300 Read block 1980 len 24576
9855 264/300 Read block 1580 len 20480
9856 265/300 Read block 743 len 28672
9857 266/300 Read block 1566 len 20480
9858 267/300 Read block 2426 len 32768
9859 268/300 Read block 1167 len 8192
9860 269/300 Read block 1615 len 16384
9861 270/300 Write block 24 len 12288 data: 6 5 5
9862 271/300 Read block 2913 len 28672
9863 272/300 Read block 2878 len 12288
9864 273/300 Write block 804 len 36864 data: 3 3 3 3 3 3 3 3 3
9865 274/300 Read block 69 len 8192
9866 275/300 Flush
9867 276/300 Read block 3575 len 4096
9868 277/300 Write block 3954 len 28672 data: 3 3 4 4 5 5 6
9869 278/300 Read block 44 len 4096
9870 279/300 Flush
9871 280/300 Write block 4089 len 4096 data: 4
9872 281/300 Write block 2525 len 36864 data: 2 3 3 3 3 2 2 2 2
9873 282/300 Read block 840 len 8192
9874 283/300 Read block 1800 len 8192
9875 284/300 Read block 3523 len 24576
9876 285/300 Read block 4200 len 20480
9877 286/300 Write block 4860 len 12288 data: 4 4 5
9878 287/300 Write block 2040 len 12288 data: 4 4 3
9879 288/300 Read block 50 len 16384
9880 289/300 Write block 79 len 36864 data: 6 6 4 4 3 3 3 3 3
9881 290/300 Read block 819 len 28672
9882 291/300 Read block 2917 len 28672
9883 292/300 Flush
9884 293/300 Write block 2456 len 20480 data: 2 2 4 4 4
9885 294/300 Read block 1347 len 20480
9886 295/300 Write block 1591 len 32768 data: 4 5 5 5 6 6 5 4
9887 296/300 Write block 836 len 36864 data: 2 4 4 4 4 4 4 4 4
9888 297/300 Flush
9889 298/300 Read block 4376 len 8192
9890 299/300 Read block 2036 len 20480
9891 300/300 Read block 1658 len 20480
9892 301/300 Write block 3050 len 8192 data: 3 3
9893 Main task has work to do, go find it
9894 got dsc Start(1)
9895 start 1
9896 Jan 14 16:12:42.376 INFO request completed, latency_us: 195, response_code: 204, uri: /start/cid/1, method: POST, req_id: b9885994-5c32-47d1-aa96-f509e0d25356, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
9897 Replay: started 1, returned:Ok(())
9898 ----------------------------------------------------------------
9899 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
9900 States: Active Offline Active
9901 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9902 3716 Acked Write 1 Done Sent Done false
9903 3717 Acked Write 5 Done Sent Done false
9904 3718 Acked Write 6 Done Sent Done false
9905 3719 Acked Write 3 Done Sent Done false
9906 3720 Acked Read 1 Done Sent Done false
9907 3721 Acked Write 4 Done Sent Done false
9908 3722 Acked Write 8 Done Sent Done false
9909 3723 Acked Read 8 Done Sent Done false
9910 3724 Acked Write 1 Done Sent Done false
9911 3725 Acked Write 6 Done Sent Done false
9912 3726 Acked Write 6 Done Sent Done false
9913 3727 Acked Flush 0 Done Sent Done false
9914 3728 Acked Read 2 Done Sent Done false
9915 3729 Acked Read 9 Done Sent Done false
9916 3730 Acked Write 3 Done Sent Done false
9917 3731 Acked Flush 0 Done Sent Done false
9918 3732 Acked Flush 0 Done Sent Done false
9919 3733 Acked Read 4 Done Sent Done false
9920 3734 Acked Write 9 Done Sent Done false
9921 3735 Acked Read 8 Done Sent Done false
9922 3736 Acked Write 6 Done Sent Done false
9923 3737 Acked Read 9 Done Sent Done false
9924 3738 Acked Read 9 Done Sent Done false
9925 3739 Acked Read 10 Done Sent Done false
9926 3740 Acked Write 6 Done Sent Done false
9927 3741 Acked Flush 0 Done Sent Done false
9928 3742 Acked Read 6 Done Sent Done false
9929 3743 Acked Read 1 Done Sent Done false
9930 3744 Acked Read 6 Done Sent Done false
9931 3745 Acked Flush 0 Done Sent Done false
9932 3746 Acked Read 10 Done Sent Done false
9933 3747 Acked Flush 0 Done Sent Done false
9934 3748 Acked Write 10 Done Sent Done false
9935 3749 Acked Write 2 Done Sent Done false
9936 3750 Acked Write 9 Done Sent Done false
9937 3751 Acked Write 7 Done Sent Done false
9938 3752 Acked Write 4 Done Sent Done false
9939 3753 Acked Write 8 Done Sent Done false
9940 3754 Acked Read 1 Done Sent Done false
9941 3755 Acked Write 7 Done Sent Done false
9942 3756 Acked Read 10 Done Sent Done false
9943 3757 Acked Read 1 Done Sent Done false
9944 3758 Acked Read 9 Done Sent Done false
9945 3759 Acked Flush 0 Done Sent Done false
9946 3760 Acked Read 2 Done Sent Done false
9947 3761 Acked Read 4 Done Sent Done false
9948 3762 Acked Read 1 Done Sent Done false
9949 3763 Acked Read 1 Done Sent Done false
9950 3764 Acked Read 3 Done Sent Done false
9951 3765 Acked Write 9 Done Sent Done false
9952 3766 Acked Read 4 Done Sent Done false
9953 3767 Acked Flush 0 Done Sent Done false
9954 3768 Acked Read 10 Done Sent Done false
9955 3769 Acked Read 10 Done Sent Done false
9956 3770 Acked Write 3 Done Sent Done false
9957 3771 Acked Read 1 Done Sent Done false
9958 3772 Acked Write 2 Done Sent Done false
9959 3773 Acked Read 7 Done Sent Done false
9960 3774 Acked Read 4 Done Sent Done false
9961 3775 Acked Read 8 Done Sent Done false
9962 3776 Acked Read 4 Done Sent Done false
9963 3777 Acked Flush 0 Done Sent Done false
9964 3778 Acked Read 10 Done Sent Done false
9965 3779 Acked Read 9 Done Sent Done false
9966 3780 Acked Write 4 Done Sent Done false
9967 3781 Acked Read 6 Done Sent Done false
9968 3782 Acked Read 3 Done Sent Done false
9969 3783 Acked Read 3 Done Sent Done false
9970 3784 Acked Write 10 Done Sent Done false
9971 3785 Acked Flush 0 Done Sent Done false
9972 3786 Acked Write 1 Done Sent Done false
9973 3787 Acked Write 2 Done Sent Done false
9974 3788 Acked Read 7 Done Sent Done false
9975 3789 Acked Read 2 Done Sent Done false
9976 3790 Acked Read 2 Done Sent Done false
9977 3791 Acked Flush 0 Done Sent Done false
9978 3792 Acked Read 4 Done Sent Done false
9979 3793 Acked Write 4 Done Sent Done false
9980 3794 Acked Write 9 Done Sent Done false
9981 3795 Acked Write 3 Done Sent Done false
9982 3796 Acked Flush 0 Done Sent Done false
9983 3797 Acked Write 8 Done Sent Done false
9984 3798 Acked Write 7 Done Sent Done false
9985 3799 Acked Flush 0 Done Sent Done false
9986 3800 Acked Read 3 Done Sent Done false
9987 3801 Acked Read 7 Done Sent Done false
9988 3802 Acked Write 8 Done Sent Done false
9989 3803 Acked Flush 0 Done Sent Done false
9990 3804 Acked Write 4 Done Sent Done false
9991 3805 Acked Read 4 Done Sent Done false
9992 3806 Acked Read 10 Done Sent Done false
9993 3807 Acked Read 6 Done Sent Done false
9994 3808 Acked Read 3 Done Sent Done false
9995 3809 Acked Read 8 Done Sent Done false
9996 3810 Acked Write 7 Done Sent Done false
9997 3811 Acked Write 7 Done Sent Done false
9998 3812 Acked Read 10 Done Sent Done false
9999 3813 Acked Flush 0 Done Sent Done false
10000 3814 Acked Write 5 Done Sent Done false
10001 3815 Acked Write 2 Done Sent Done false
10002 3816 Acked Read 4 Done Sent Done false
10003 3817 Acked Read 9 Done Sent Done false
10004 3818 Acked Write 5 Done Sent Done false
10005 3819 Acked Read 8 Done Sent Done false
10006 3820 Acked Write 9 Done Sent Done false
10007 3821 Acked Flush 0 Done Sent Done false
10008 3822 Acked Read 1 Done Sent Done false
10009 3823 Acked Write 5 Done Sent Done false
10010 3824 Acked Read 4 Done Sent Done false
10011 3825 Acked Read 10 Done Sent Done false
10012 3826 Acked Flush 0 Done Sent Done false
10013 3827 Acked Read 6 Done Sent Done false
10014 3828 Acked Write 9 Done Sent Done false
10015 3829 Acked Write 6 Done Sent Done false
10016 3830 Acked Write 1 Done Sent Done false
10017 3831 Acked Flush 0 Done Sent Done false
10018 3832 Acked Flush 0 Done Sent Done false
10019 3833 Acked Write 2 Done Sent Done false
10020 3834 Acked Read 10 Done Sent Done false
10021 3835 Acked Write 10 Done Sent Done false
10022 3836 Acked Read 8 Done Sent Done false
10023 3837 Acked Read 7 Done Sent Done false
10024 3838 Acked Read 5 Done Sent Done false
10025 3839 Acked Read 10 Done Sent Done false
10026 3840 Acked Read 1 Done Sent Done false
10027 3841 Acked Read 5 Done Sent Done false
10028 3842 Acked Write 6 Done Sent Done false
10029 3843 Acked Read 3 Done Sent Done false
10030 3844 Acked Write 3 Done Sent Done false
10031 3845 Acked Read 9 Done Sent Done false
10032 3846 Acked Write 8 Done Sent Done false
10033 3847 Acked Read 1 Done Sent Done false
10034 3848 Acked Read 7 Done Sent Done false
10035 3849 Acked Read 3 Done Sent Done false
10036 3850 Acked Read 8 Done Sent Done false
10037 3851 Acked Write 4 Done Sent Done false
10038 3852 Acked Read 2 Done Sent Done false
10039 3853 Acked Write 1 Done Sent Done false
10040 3854 Acked Write 10 Done Sent Done false
10041 3855 Acked Read 9 Done Sent Done false
10042 3856 Acked Read 5 Done Sent Done false
10043 3857 Acked Write 4 Done Sent Done false
10044 3858 Acked Read 2 Done Sent Done false
10045 3859 Acked Write 4 Done Sent Done false
10046 3860 Acked Read 9 Done Sent Done false
10047 3861 Acked Write 3 Done Sent Done false
10048 3862 Acked Write 4 Done Sent Done false
10049 3863 Acked Read 5 Done Sent Done false
10050 3864 Acked Write 10 Done Sent Done false
10051 3865 Acked Read 8 Done Sent Done false
10052 3866 Acked Read 4 Done Sent Done false
10053 3867 Acked Write 7 Done Sent Done false
10054 3868 Acked Write 7 Done Sent Done false
10055 3869 Acked Read 4 Done Sent Done false
10056 3870 Acked Write 2 Done Sent Done false
10057 3871 Acked Read 5 Done Sent Done false
10058 3872 Acked Read 4 Done Sent Done false
10059 3873 Acked Read 2 Done Sent Done false
10060 3874 Acked Flush 0 Done Sent Done false
10061 3875 Acked Write 4 Done Sent Done false
10062 3876 Acked Write 6 Done Sent Done false
10063 3877 Acked Write 8 Done Sent Done false
10064 3878 Acked Write 3 Done Sent Done false
10065 3879 Acked Read 3 Done Sent Done false
10066 3880 Acked Read 10 Done Sent Done false
10067 3881 Acked Write 2 Done Sent Done false
10068 3882 Acked Read 9 Done Sent Done false
10069 3883 Acked Flush 0 Done Sent Done false
10070 3884 Acked Read 3 Done Sent Done false
10071 3885 Acked Write 2 Done Sent Done false
10072 3886 Acked Flush 0 Done Sent Done false
10073 3887 Acked Read 8 Done Sent Done false
10074 3888 Acked Write 1 Done Sent Done false
10075 3889 Acked Write 1 Done Sent Done false
10076 3890 Acked Flush 0 Done Sent Done false
10077 3891 Acked Read 5 Done Sent Done false
10078 3892 Acked Read 5 Done Sent Done false
10079 3893 Acked Read 7 Done Sent Done false
10080 3894 Acked Read 8 Done Sent Done false
10081 3895 Acked Write 4 Done Sent Done false
10082 3896 Acked Write 1 Done Sent Done false
10083 3897 Acked Read 4 Done Sent Done false
10084 3898 Acked Read 10 Done Sent Done false
10085 3899 Acked Read 2 Done Sent Done false
10086 3900 Acked Flush 0 Done Sent Done false
10087 3901 Acked Read 10 Done Sent Done false
10088 3902 Acked Write 7 Done Sent Done false
10089 3903 Acked Read 8 Done Sent Done false
10090 3904 Acked Read 6 Done Sent Done false
10091 3905 Acked Write 3 Done Sent Done false
10092 3906 Acked Read 8 Done Sent Done false
10093 3907 Acked Flush 0 Done Sent Done false
10094 3908 Acked Write 5 Done Sent Done false
10095 3909 Acked Read 8 Done Sent Done false
10096 3910 Acked Read 7 Done Sent Done false
10097 3911 Acked Read 3 Done Sent Done false
10098 3912 Acked Write 8 Done Sent Done false
10099 3913 Acked Write 3 Done Sent Done false
10100 3914 Acked Read 4 Done Sent Done false
10101 3915 Acked Flush 0 Done Sent Done false
10102 3916 Acked Read 7 Done Sent Done false
10103 3917 Acked Read 8 Done Sent Done false
10104 3918 Acked Read 3 Done Sent Done false
10105 3919 Acked Read 1 Done Sent Done false
10106 3920 Acked Read 6 Done Sent Done false
10107 3921 Acked Read 10 Done Sent Done false
10108 3922 Acked Flush 0 Done Sent Done false
10109 3923 Acked Read 8 Done Sent Done false
10110 3924 Acked Write 8 Done Sent Done false
10111 3925 Acked Write 8 Done Sent Done false
10112 3926 Acked Write 2 Done Sent Done false
10113 3927 Acked Write 4 Done Sent Done false
10114 3928 Acked Flush 0 Done Sent Done false
10115 3929 Acked Read 7 Done Sent Done false
10116 3930 Acked Write 1 Done Sent Done false
10117 3931 Acked Write 7 Done Sent Done false
10118 3932 Acked Write 8 Done Sent Done false
10119 3933 Acked Read 2 Done Sent Done false
10120 3934 Acked Flush 0 Done Sent Done false
10121 3935 Acked Read 1 Done Sent Done false
10122 3936 Acked Read 1 Done Sent Done false
10123 3937 Acked Read 5 Done Sent Done false
10124 3938 Acked Flush 0 Done Sent Done false
10125 3939 Acked Read 4 Done Sent Done false
10126 3940 Acked Read 8 Done Sent Done false
10127 3941 Acked Read 6 Done Sent Done false
10128 3942 Acked Read 4 Done Sent Done false
10129 3943 Acked Write 2 Done Sent Done false
10130 3944 Acked Write 6 Done Sent Done false
10131 3945 Acked Read 10 Done Sent Done false
10132 3946 Acked Write 5 Done Sent Done false
10133 3947 Acked Write 10 Done Sent Done false
10134 3948 Acked Read 3 Done Sent Done false
10135 3949 Acked Write 6 Done Sent Done false
10136 3950 Acked Write 3 Done Sent Done false
10137 3951 Acked Write 7 Done Sent Done false
10138 3952 Acked Write 7 Done Sent Done false
10139 3953 Acked Write 3 Done Sent Done false
10140 3954 Acked Write 4 Done Sent Done false
10141 3955 Acked Flush 0 Done Sent Done false
10142 3956 Acked Read 4 Done Sent Done false
10143 3957 Acked Write 3 Done Sent Done false
10144 3958 Acked Read 3 Done Sent Done false
10145 3959 Acked Read 7 Done Sent Done false
10146 3960 Acked Write 3 Done Sent Done false
10147 3961 Acked Flush 0 Done Sent Done false
10148 3962 Acked Read 2 Done Sent Done false
10149 3963 Acked Read 6 Done Sent Done false
10150 3964 Acked Write 8 Done Sent Done false
10151 3965 Acked Read 3 Done Sent Done false
10152 3966 Acked Flush 0 Done Sent Done false
10153 3967 Acked Read 5 Done Sent Done false
10154 3968 Acked Read 8 Done Sent Done false
10155 3969 Acked Write 9 Done Sent Done false
10156 3970 Acked Write 10 Done Sent Done false
10157 3971 Acked Write 10 Done Sent Done false
10158 3972 Acked Read 9 Done Sent Done false
10159 3973 Acked Read 8 Done Sent Done false
10160 3974 Acked Write 1 Done Sent Done false
10161 3975 Acked Read 8 Done Sent Done false
10162 3976 Acked Write 7 Done Sent Done false
10163 3977 Acked Read 4 Done Sent Done false
10164 3978 Acked Read 6 Done Sent Done false
10165 3979 Acked Read 5 Done Sent Done false
10166 3980 Acked Read 7 Done Sent Done false
10167 3981 Acked Read 5 Done Sent Done false
10168 3982 Acked Read 8 Done Sent Done false
10169 3983 Acked Read 2 Done Sent Done false
10170 3984 Acked Read 4 Done Sent Done false
10171 3985 Acked Write 3 Done Sent Done false
10172 3986 Acked Read 7 Done Sent Done false
10173 3987 Acked Read 3 Done Sent Done false
10174 3988 Acked Write 9 Done Sent Done false
10175 3989 Acked Read 2 Done Sent Done false
10176 3990 Acked Flush 0 Done Sent Done false
10177 3991 Acked Read 1 Done Sent Done false
10178 3992 Acked Write 7 Done Sent Done false
10179 3993 Acked Read 1 Done Sent Done false
10180 3994 Acked Flush 0 Done Sent Done false
10181 3995 Acked Write 1 Done Sent Done false
10182 3996 Acked Write 9 Done Sent Done false
10183 3997 Acked Read 2 Done Sent Done false
10184 3998 Acked Read 2 Done Sent Done false
10185 3999 Acked Read 6 Done Sent Done false
10186 4000 Acked Read 5 Done Sent Done false
10187 4001 Acked Write 3 Done Sent Done false
10188 4002 Acked Write 3 Done Sent Done false
10189 4003 Acked Read 4 Done Sent Done false
10190 4004 Acked Write 9 Done Sent Done false
10191 4005 Acked Read 7 Done Sent Done false
10192 4006 Acked Read 7 Done Sent Done false
10193 4007 Acked Flush 0 Done Sent Done false
10194 4008 Acked Write 5 Done Sent Done false
10195 4009 Acked Read 5 Done Sent Done false
10196 4010 Acked Write 8 Done Sent Done false
10197 4011 Acked Write 9 Done Sent Done false
10198 4012 Acked Flush 0 Done Sent Done false
10199 4013 Acked Read 2 Done Sent Done false
10200 4014 Acked Read 5 Done Sent Done false
10201 4015 Acked Read 5 Done Sent Done false
10202 4016 Acked Write 2 Done Sent Done false
10203 STATES DS:0 DS:1 DS:2 TOTAL
10204 Sent 0 301 0 301
10205 Done 301 0 301 602
10206 Skipped 0 0 0 0
10207 Error 0 0 0 0
10208 Last Flush: 4012 3715 4012
10209 Downstairs last five completed: 3715 3714 3713 3712 3711
10210 Upstairs last five completed: 4016 4015 4014 4013 4012
10211 CLIENT: Up:0 ds:301 act:2
10212 [8820] got start action
10213 [8820] I am going to restart
10214 Starting downstairs at port 8820
10215 Make output file at "/tmp/dsc/downstairs-8820.txt"
10216 [8820][1] reports Starting
10217 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1212)
10218 [8820][1] reports Running
10219 ----------------------------------------------------------------
10220 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
10221 States: Active Offline Active
10222 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10223 3716 Acked Write 1 Done Sent Done false
10224 3717 Acked Write 5 Done Sent Done false
10225 3718 Acked Write 6 Done Sent Done false
10226 3719 Acked Write 3 Done Sent Done false
10227 3720 Acked Read 1 Done Sent Done false
10228 3721 Acked Write 4 Done Sent Done false
10229 3722 Acked Write 8 Done Sent Done false
10230 3723 Acked Read 8 Done Sent Done false
10231 3724 Acked Write 1 Done Sent Done false
10232 3725 Acked Write 6 Done Sent Done false
10233 3726 Acked Write 6 Done Sent Done false
10234 3727 Acked Flush 0 Done Sent Done false
10235 3728 Acked Read 2 Done Sent Done false
10236 3729 Acked Read 9 Done Sent Done false
10237 3730 Acked Write 3 Done Sent Done false
10238 3731 Acked Flush 0 Done Sent Done false
10239 3732 Acked Flush 0 Done Sent Done false
10240 3733 Acked Read 4 Done Sent Done false
10241 3734 Acked Write 9 Done Sent Done false
10242 3735 Acked Read 8 Done Sent Done false
10243 3736 Acked Write 6 Done Sent Done false
10244 3737 Acked Read 9 Done Sent Done false
10245 3738 Acked Read 9 Done Sent Done false
10246 3739 Acked Read 10 Done Sent Done false
10247 3740 Acked Write 6 Done Sent Done false
10248 3741 Acked Flush 0 Done Sent Done false
10249 3742 Acked Read 6 Done Sent Done false
10250 3743 Acked Read 1 Done Sent Done false
10251 3744 Acked Read 6 Done Sent Done false
10252 3745 Acked Flush 0 Done Sent Done false
10253 3746 Acked Read 10 Done Sent Done false
10254 3747 Acked Flush 0 Done Sent Done false
10255 3748 Acked Write 10 Done Sent Done false
10256 3749 Acked Write 2 Done Sent Done false
10257 3750 Acked Write 9 Done Sent Done false
10258 3751 Acked Write 7 Done Sent Done false
10259 3752 Acked Write 4 Done Sent Done false
10260 3753 Acked Write 8 Done Sent Done false
10261 3754 Acked Read 1 Done Sent Done false
10262 3755 Acked Write 7 Done Sent Done false
10263 3756 Acked Read 10 Done Sent Done false
10264 3757 Acked Read 1 Done Sent Done false
10265 3758 Acked Read 9 Done Sent Done false
10266 3759 Acked Flush 0 Done Sent Done false
10267 3760 Acked Read 2 Done Sent Done false
10268 3761 Acked Read 4 Done Sent Done false
10269 3762 Acked Read 1 Done Sent Done false
10270 3763 Acked Read 1 Done Sent Done false
10271 3764 Acked Read 3 Done Sent Done false
10272 3765 Acked Write 9 Done Sent Done false
10273 3766 Acked Read 4 Done Sent Done false
10274 3767 Acked Flush 0 Done Sent Done false
10275 3768 Acked Read 10 Done Sent Done false
10276 3769 Acked Read 10 Done Sent Done false
10277 3770 Acked Write 3 Done Sent Done false
10278 3771 Acked Read 1 Done Sent Done false
10279 3772 Acked Write 2 Done Sent Done false
10280 3773 Acked Read 7 Done Sent Done false
10281 3774 Acked Read 4 Done Sent Done false
10282 3775 Acked Read 8 Done Sent Done false
10283 3776 Acked Read 4 Done Sent Done false
10284 3777 Acked Flush 0 Done Sent Done false
10285 3778 Acked Read 10 Done Sent Done false
10286 3779 Acked Read 9 Done Sent Done false
10287 3780 Acked Write 4 Done Sent Done false
10288 3781 Acked Read 6 Done Sent Done false
10289 3782 Acked Read 3 Done Sent Done false
10290 3783 Acked Read 3 Done Sent Done false
10291 3784 Acked Write 10 Done Sent Done false
10292 3785 Acked Flush 0 Done Sent Done false
10293 3786 Acked Write 1 Done Sent Done false
10294 3787 Acked Write 2 Done Sent Done false
10295 3788 Acked Read 7 Done Sent Done false
10296 3789 Acked Read 2 Done Sent Done false
10297 3790 Acked Read 2 Done Sent Done false
10298 3791 Acked Flush 0 Done Sent Done false
10299 3792 Acked Read 4 Done Sent Done false
10300 3793 Acked Write 4 Done Sent Done false
10301 3794 Acked Write 9 Done Sent Done false
10302 3795 Acked Write 3 Done Sent Done false
10303 3796 Acked Flush 0 Done Sent Done false
10304 3797 Acked Write 8 Done Sent Done false
10305 3798 Acked Write 7 Done Sent Done false
10306 3799 Acked Flush 0 Done Sent Done false
10307 3800 Acked Read 3 Done Sent Done false
10308 3801 Acked Read 7 Done Sent Done false
10309 3802 Acked Write 8 Done Sent Done false
10310 3803 Acked Flush 0 Done Sent Done false
10311 3804 Acked Write 4 Done Sent Done false
10312 3805 Acked Read 4 Done Sent Done false
10313 3806 Acked Read 10 Done Sent Done false
10314 3807 Acked Read 6 Done Sent Done false
10315 3808 Acked Read 3 Done Sent Done false
10316 3809 Acked Read 8 Done Sent Done false
10317 3810 Acked Write 7 Done Sent Done false
10318 3811 Acked Write 7 Done Sent Done false
10319 3812 Acked Read 10 Done Sent Done false
10320 3813 Acked Flush 0 Done Sent Done false
10321 3814 Acked Write 5 Done Sent Done false
10322 3815 Acked Write 2 Done Sent Done false
10323 3816 Acked Read 4 Done Sent Done false
10324 3817 Acked Read 9 Done Sent Done false
10325 3818 Acked Write 5 Done Sent Done false
10326 3819 Acked Read 8 Done Sent Done false
10327 3820 Acked Write 9 Done Sent Done false
10328 3821 Acked Flush 0 Done Sent Done false
10329 3822 Acked Read 1 Done Sent Done false
10330 3823 Acked Write 5 Done Sent Done false
10331 3824 Acked Read 4 Done Sent Done false
10332 3825 Acked Read 10 Done Sent Done false
10333 3826 Acked Flush 0 Done Sent Done false
10334 3827 Acked Read 6 Done Sent Done false
10335 3828 Acked Write 9 Done Sent Done false
10336 3829 Acked Write 6 Done Sent Done false
10337 3830 Acked Write 1 Done Sent Done false
10338 3831 Acked Flush 0 Done Sent Done false
10339 3832 Acked Flush 0 Done Sent Done false
10340 3833 Acked Write 2 Done Sent Done false
10341 3834 Acked Read 10 Done Sent Done false
10342 3835 Acked Write 10 Done Sent Done false
10343 3836 Acked Read 8 Done Sent Done false
10344 3837 Acked Read 7 Done Sent Done false
10345 3838 Acked Read 5 Done Sent Done false
10346 3839 Acked Read 10 Done Sent Done false
10347 3840 Acked Read 1 Done Sent Done false
10348 3841 Acked Read 5 Done Sent Done false
10349 3842 Acked Write 6 Done Sent Done false
10350 3843 Acked Read 3 Done Sent Done false
10351 3844 Acked Write 3 Done Sent Done false
10352 3845 Acked Read 9 Done Sent Done false
10353 3846 Acked Write 8 Done Sent Done false
10354 3847 Acked Read 1 Done Sent Done false
10355 3848 Acked Read 7 Done Sent Done false
10356 3849 Acked Read 3 Done Sent Done false
10357 3850 Acked Read 8 Done Sent Done false
10358 3851 Acked Write 4 Done Sent Done false
10359 3852 Acked Read 2 Done Sent Done false
10360 3853 Acked Write 1 Done Sent Done false
10361 3854 Acked Write 10 Done Sent Done false
10362 3855 Acked Read 9 Done Sent Done false
10363 3856 Acked Read 5 Done Sent Done false
10364 3857 Acked Write 4 Done Sent Done false
10365 3858 Acked Read 2 Done Sent Done false
10366 3859 Acked Write 4 Done Sent Done false
10367 3860 Acked Read 9 Done Sent Done false
10368 3861 Acked Write 3 Done Sent Done false
10369 3862 Acked Write 4 Done Sent Done false
10370 3863 Acked Read 5 Done Sent Done false
10371 3864 Acked Write 10 Done Sent Done false
10372 3865 Acked Read 8 Done Sent Done false
10373 3866 Acked Read 4 Done Sent Done false
10374 3867 Acked Write 7 Done Sent Done false
10375 3868 Acked Write 7 Done Sent Done false
10376 3869 Acked Read 4 Done Sent Done false
10377 3870 Acked Write 2 Done Sent Done false
10378 3871 Acked Read 5 Done Sent Done false
10379 3872 Acked Read 4 Done Sent Done false
10380 3873 Acked Read 2 Done Sent Done false
10381 3874 Acked Flush 0 Done Sent Done false
10382 3875 Acked Write 4 Done Sent Done false
10383 3876 Acked Write 6 Done Sent Done false
10384 3877 Acked Write 8 Done Sent Done false
10385 3878 Acked Write 3 Done Sent Done false
10386 3879 Acked Read 3 Done Sent Done false
10387 3880 Acked Read 10 Done Sent Done false
10388 3881 Acked Write 2 Done Sent Done false
10389 3882 Acked Read 9 Done Sent Done false
10390 3883 Acked Flush 0 Done Sent Done false
10391 3884 Acked Read 3 Done Sent Done false
10392 3885 Acked Write 2 Done Sent Done false
10393 3886 Acked Flush 0 Done Sent Done false
10394 3887 Acked Read 8 Done Sent Done false
10395 3888 Acked Write 1 Done Sent Done false
10396 3889 Acked Write 1 Done Sent Done false
10397 3890 Acked Flush 0 Done Sent Done false
10398 3891 Acked Read 5 Done Sent Done false
10399 3892 Acked Read 5 Done Sent Done false
10400 3893 Acked Read 7 Done Sent Done false
10401 3894 Acked Read 8 Done Sent Done false
10402 3895 Acked Write 4 Done Sent Done false
10403 3896 Acked Write 1 Done Sent Done false
10404 3897 Acked Read 4 Done Sent Done false
10405 3898 Acked Read 10 Done Sent Done false
10406 3899 Acked Read 2 Done Sent Done false
10407 3900 Acked Flush 0 Done Sent Done false
10408 3901 Acked Read 10 Done Sent Done false
10409 3902 Acked Write 7 Done Sent Done false
10410 3903 Acked Read 8 Done Sent Done false
10411 3904 Acked Read 6 Done Sent Done false
10412 3905 Acked Write 3 Done Sent Done false
10413 3906 Acked Read 8 Done Sent Done false
10414 3907 Acked Flush 0 Done Sent Done false
10415 3908 Acked Write 5 Done Sent Done false
10416 3909 Acked Read 8 Done Sent Done false
10417 3910 Acked Read 7 Done Sent Done false
10418 3911 Acked Read 3 Done Sent Done false
10419 3912 Acked Write 8 Done Sent Done false
10420 3913 Acked Write 3 Done Sent Done false
10421 3914 Acked Read 4 Done Sent Done false
10422 3915 Acked Flush 0 Done Sent Done false
10423 3916 Acked Read 7 Done Sent Done false
10424 3917 Acked Read 8 Done Sent Done false
10425 3918 Acked Read 3 Done Sent Done false
10426 3919 Acked Read 1 Done Sent Done false
10427 3920 Acked Read 6 Done Sent Done false
10428 3921 Acked Read 10 Done Sent Done false
10429 3922 Acked Flush 0 Done Sent Done false
10430 3923 Acked Read 8 Done Sent Done false
10431 3924 Acked Write 8 Done Sent Done false
10432 3925 Acked Write 8 Done Sent Done false
10433 3926 Acked Write 2 Done Sent Done false
10434 3927 Acked Write 4 Done Sent Done false
10435 3928 Acked Flush 0 Done Sent Done false
10436 3929 Acked Read 7 Done Sent Done false
10437 3930 Acked Write 1 Done Sent Done false
10438 3931 Acked Write 7 Done Sent Done false
10439 3932 Acked Write 8 Done Sent Done false
10440 3933 Acked Read 2 Done Sent Done false
10441 3934 Acked Flush 0 Done Sent Done false
10442 3935 Acked Read 1 Done Sent Done false
10443 3936 Acked Read 1 Done Sent Done false
10444 3937 Acked Read 5 Done Sent Done false
10445 3938 Acked Flush 0 Done Sent Done false
10446 3939 Acked Read 4 Done Sent Done false
10447 3940 Acked Read 8 Done Sent Done false
10448 3941 Acked Read 6 Done Sent Done false
10449 3942 Acked Read 4 Done Sent Done false
10450 3943 Acked Write 2 Done Sent Done false
10451 3944 Acked Write 6 Done Sent Done false
10452 3945 Acked Read 10 Done Sent Done false
10453 3946 Acked Write 5 Done Sent Done false
10454 3947 Acked Write 10 Done Sent Done false
10455 3948 Acked Read 3 Done Sent Done false
10456 3949 Acked Write 6 Done Sent Done false
10457 3950 Acked Write 3 Done Sent Done false
10458 3951 Acked Write 7 Done Sent Done false
10459 3952 Acked Write 7 Done Sent Done false
10460 3953 Acked Write 3 Done Sent Done false
10461 3954 Acked Write 4 Done Sent Done false
10462 3955 Acked Flush 0 Done Sent Done false
10463 3956 Acked Read 4 Done Sent Done false
10464 3957 Acked Write 3 Done Sent Done false
10465 3958 Acked Read 3 Done Sent Done false
10466 3959 Acked Read 7 Done Sent Done false
10467 3960 Acked Write 3 Done Sent Done false
10468 3961 Acked Flush 0 Done Sent Done false
10469 3962 Acked Read 2 Done Sent Done false
10470 3963 Acked Read 6 Done Sent Done false
10471 3964 Acked Write 8 Done Sent Done false
10472 3965 Acked Read 3 Done Sent Done false
10473 3966 Acked Flush 0 Done Sent Done false
10474 3967 Acked Read 5 Done Sent Done false
10475 3968 Acked Read 8 Done Sent Done false
10476 3969 Acked Write 9 Done Sent Done false
10477 3970 Acked Write 10 Done Sent Done false
10478 3971 Acked Write 10 Done Sent Done false
10479 3972 Acked Read 9 Done Sent Done false
10480 3973 Acked Read 8 Done Sent Done false
10481 3974 Acked Write 1 Done Sent Done false
10482 3975 Acked Read 8 Done Sent Done false
10483 3976 Acked Write 7 Done Sent Done false
10484 3977 Acked Read 4 Done Sent Done false
10485 3978 Acked Read 6 Done Sent Done false
10486 3979 Acked Read 5 Done Sent Done false
10487 3980 Acked Read 7 Done Sent Done false
10488 3981 Acked Read 5 Done Sent Done false
10489 3982 Acked Read 8 Done Sent Done false
10490 3983 Acked Read 2 Done Sent Done false
10491 3984 Acked Read 4 Done Sent Done false
10492 3985 Acked Write 3 Done Sent Done false
10493 3986 Acked Read 7 Done Sent Done false
10494 3987 Acked Read 3 Done Sent Done false
10495 3988 Acked Write 9 Done Sent Done false
10496 3989 Acked Read 2 Done Sent Done false
10497 3990 Acked Flush 0 Done Sent Done false
10498 3991 Acked Read 1 Done Sent Done false
10499 3992 Acked Write 7 Done Sent Done false
10500 3993 Acked Read 1 Done Sent Done false
10501 3994 Acked Flush 0 Done Sent Done false
10502 3995 Acked Write 1 Done Sent Done false
10503 3996 Acked Write 9 Done Sent Done false
10504 3997 Acked Read 2 Done Sent Done false
10505 3998 Acked Read 2 Done Sent Done false
10506 3999 Acked Read 6 Done Sent Done false
10507 4000 Acked Read 5 Done Sent Done false
10508 4001 Acked Write 3 Done Sent Done false
10509 4002 Acked Write 3 Done Sent Done false
10510 4003 Acked Read 4 Done Sent Done false
10511 4004 Acked Write 9 Done Sent Done false
10512 4005 Acked Read 7 Done Sent Done false
10513 4006 Acked Read 7 Done Sent Done false
10514 4007 Acked Flush 0 Done Sent Done false
10515 4008 Acked Write 5 Done Sent Done false
10516 4009 Acked Read 5 Done Sent Done false
10517 4010 Acked Write 8 Done Sent Done false
10518 4011 Acked Write 9 Done Sent Done false
10519 4012 Acked Flush 0 Done Sent Done false
10520 4013 Acked Read 2 Done Sent Done false
10521 4014 Acked Read 5 Done Sent Done false
10522 4015 Acked Read 5 Done Sent Done false
10523 4016 Acked Write 2 Done Sent Done false
10524 4017 Acked Flush 0 Done Sent Done false
10525 STATES DS:0 DS:1 DS:2 TOTAL
10526 Sent 0 302 0 302
10527 Done 302 0 302 604
10528 Skipped 0 0 0 0
10529 Error 0 0 0 0
10530 Last Flush: 4017 3715 4017
10531 Downstairs last five completed: 3715 3714 3713 3712 3711
10532 Upstairs last five completed: 4017 4016 4015 4014 4013
10533 CLIENT: Up:0 ds:302 act:2
105342025-01-14T16:12:48.142ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
105352025-01-14T16:12:48.142ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49204) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
105362025-01-14T16:12:48.143ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
105372025-01-14T16:12:48.144ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
105382025-01-14T16:12:48.144ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
105392025-01-14T16:12:48.144ZINFOcrucible: send last flush ID to this DS: 3715 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
105402025-01-14T16:12:48.144ZINFOcrucible: Replied this last flush ID: 3715 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
105412025-01-14T16:12:48.144ZINFOcrucible: [1] client re-new 302 jobs since flush 3715 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
105422025-01-14T16:12:48.144ZINFOcrucible: [1] Replayed 302 jobs since flush: 3715 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
10543 ----------------------------------------------------------------
10544 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
10545 Downstairs last five completed: 4017 4016 4015 4014 4013
10546 Upstairs last five completed: 4017 4016 4015 4014 4013
10547 CLIENT: Up:0 ds:0 act:3
10548 Replay: All jobs finished
10549 Main task has work to do, go find it
10550 got dsc Stop(2)
10551 Jan 14 16:12:50.380stop 2
10552 INFO request completed, latency_us: 323, response_code: 204, uri: /stop/cid/2, method: POST, req_id: b8064fac-d21c-48ba-88b3-273351f2c722, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
10553 [8830] Got stop action so:false kr:false
105542025-01-14T16:12:50.381ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
105552025-01-14T16:12:50.381ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
10556 Jan 14 16:12:50.381 INFO request completed, latency_us: 265, response_code: 200, uri: /state/cid/2, method: GET, req_id{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-14T16:12:50.381181149Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
10557 : cd79edc3-0b8b-48ce-b1c1-6e7788d3cc5e, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
105582025-01-14T16:12:50.381ZINFOcrucible: client task is exiting = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
10559 [8830] Exited with: ExitStatus(unix_wait_status(9))
10560 [8830][2] reports Exit
10561 Jan 14 16:12:54.382 INFO request completed, latency_us: 217, response_code: 200, uri: /state/cid/2, method: GET, req_id: 08039b52-4fb9-4db9-bfeb-381cf4937159, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
10562 001/300 Flush
10563 002/300 Write block 3439 len 4096 data: 3
10564 003/300 Write block 4353 len 8192 data: 4 4
10565 004/300 Read block 3747 len 20480
10566 005/300 Write block 3104 len 28672 data: 3 2 3 3 3 3 2
10567 006/300 Flush
10568 007/300 Write block 2237 len 24576 data: 3 4 4 4 4 4
10569 008/300 Write block 3740 len 28672 data: 3 3 3 2 2 2 3
10570 009/300 Write block 2920 len 40960 data: 3 4 4 3 3 4 4 5 4 4
10571 010/300 Write block 2307 len 36864 data: 2 2 2 3 3 5 6 6 6
10572 011/300 Write block 1393 len 4096 data: 3
10573 012/300 Flush
10574 013/300 Write block 4173 len 32768 data: 4 4 4 5 5 4 3 3
10575 014/300 Write block 768 len 4096 data: 3
10576 015/300 Read block 2105 len 40960
10577 016/300 Read block 2948 len 20480
10578 017/300 Read block 626 len 12288
10579 018/300 Read block 3553 len 28672
10580 019/300 Read block 2604 len 40960
10581 020/300 Read block 3390 len 12288
10582 021/300 Write block 82 len 12288 data: 5 4 4
10583 022/300 Write block 124 len 20480 data: 4 4 4 4 4
10584 023/300 Write block 3115 len 36864 data: 3 3 2 2 2 2 2 2 2
10585 024/300 Read block 1296 len 36864
10586 025/300 Read block 86 len 8192
10587 026/300 Read block 4569 len 8192
10588 027/300 Write block 3056 len 20480 data: 3 3 2 2 2
10589 028/300 Flush
10590 029/300 Read block 2940 len 8192
10591 030/300 Read block 2291 len 12288
10592 031/300 Flush
10593 032/300 Flush
10594 033/300 Read block 2749 len 40960
10595 034/300 Read block 1714 len 4096
10596 035/300 Read block 2664 len 32768
10597 036/300 Read block 1984 len 16384
10598 037/300 Read block 2392 len 20480
10599 038/300 Write block 1989 len 12288 data: 4 4 4
10600 039/300 Write block 1633 len 8192 data: 3 3
10601 040/300 Write block 3894 len 8192 data: 2 3
10602 041/300 Write block 1659 len 4096 data: 4
10603 042/300 Write block 4825 len 28672 data: 4 4 4 4 3 4 3
10604 043/300 Write block 1178 len 32768 data: 5 4 4 3 4 4 3 4
10605 044/300 Read block 1395 len 24576
10606 045/300 Read block 4096 len 16384
10607 046/300 Write block 54 len 12288 data: 3 3 3
10608 047/300 Read block 2030 len 32768
10609 048/300 Read block 1485 len 16384
10610 049/300 Flush
10611 050/300 Read block 1330 len 8192
10612 051/300 Read block 600 len 4096
10613 052/300 Read block 75 len 24576
10614 053/300 Read block 152 len 8192
10615 054/300 Write block 1029 len 40960 data: 2 3 3 4 4 3 3 3 3 3
10616 055/300 Read block 2984 len 20480
10617 056/300 Read block 2962 len 36864
10618 057/300 Read block 1232 len 12288
10619 058/300 Flush
10620 059/300 Write block 1991 len 36864 data: 5 4 5 3 4 5 5 5 4
10621 060/300 Write block 123 len 36864 data: 4 5 5 5 5 5 3 3 4
10622 061/300 Read block 3912 len 28672
10623 062/300 Flush
10624 063/300 Read block 2503 len 16384
10625 064/300 Read block 1634 len 4096
10626 065/300 Write block 805 len 32768 data: 4 4 4 4 4 4 4 4
10627 066/300 Read block 769 len 20480
10628 067/300 Write block 2899 len 40960 data: 3 4 3 3 4 4 4 4 4 4
10629 068/300 Read block 3528 len 20480
10630 069/300 Write block 3037 len 12288 data: 2 2 2
10631 070/300 Read block 926 len 32768
10632 071/300 Read block 3758 len 16384
10633 072/300 Read block 3173 len 16384
10634 073/300 Flush
10635 074/300 Read block 3203 len 8192
10636 075/300 Read block 1271 len 4096
10637 076/300 Read block 4242 len 4096
10638 077/300 Read block 4944 len 20480
10639 078/300 Write block 460 len 20480 data: 3 5 5 5 4
10640 079/300 Flush
10641 080/300 Read block 2733 len 12288
10642 081/300 Flush
10643 082/300 Flush
10644 083/300 Read block 1881 len 24576
10645 084/300 Write block 4329 len 12288 data: 3 3 3
10646 085/300 Flush
10647 086/300 Read block 1289 len 12288
10648 087/300 Write block 3699 len 36864 data: 2 2 2 2 2 3 3 4 4
10649 088/300 Read block 3425 len 12288
10650 089/300 Read block 2451 len 36864
10651 090/300 Write block 1002 len 8192 data: 4 5
10652 091/300 Write block 2775 len 36864 data: 4 3 3 4 4 3 3 3 2
10653 092/300 Flush
10654 093/300 Read block 1171 len 40960
10655 094/300 Write block 1064 len 12288 data: 4 5 5
10656 095/300 Flush
10657 096/300 Read block 2687 len 40960
10658 097/300 Read block 1841 len 16384
10659 098/300 Write block 807 len 20480 data: 5 5 5 5 5
10660 099/300 Read block 985 len 12288
10661 100/300 Write block 4178 len 8192 data: 5 4
10662 101/300 Read block 1487 len 28672
10663 102/300 Read block 1117 len 24576
10664 103/300 Read block 921 len 28672
10665 104/300 Read block 1130 len 40960
10666 105/300 Read block 4540 len 8192
10667 106/300 Read block 4651 len 28672
10668 107/300 Write block 2185 len 16384 data: 4 4 4 4
10669 108/300 Write block 1722 len 28672 data: 3 3 4 3 3 4 4
10670 109/300 Write block 3427 len 24576 data: 4 3 3 3 3 2
10671 110/300 Write block 2995 len 20480 data: 4 3 3 4 5
10672 111/300 Write block 117 len 36864 data: 3 2 2 2 3 4 5 6 6
10673 112/300 Flush
10674 113/300 Write block 1331 len 36864 data: 4 4 5 5 5 3 4 4 4
10675 114/300 Read block 2630 len 12288
10676 115/300 Read block 1003 len 40960
10677 116/300 Write block 3799 len 8192 data: 5 4
10678 117/300 Read block 4131 len 36864
10679 118/300 Read block 322 len 40960
10680 119/300 Write block 72 len 16384 data: 3 2 2 3
10681 120/300 Write block 193 len 32768 data: 2 2 2 3 3 3 3 3
10682 121/300 Write block 1441 len 28672 data: 5 4 4 4 4 5 5
10683 122/300 Read block 4284 len 28672
10684 123/300 Write block 3343 len 20480 data: 2 3 3 3 3
10685 124/300 Read block 2404 len 24576
10686 125/300 Write block 1345 len 4096 data: 3
10687 126/300 Write block 2885 len 12288 data: 3 3 4
10688 127/300 Flush
10689 128/300 Read block 2302 len 40960
10690 129/300 Read block 1634 len 24576
10691 130/300 Read block 2739 len 4096
10692 131/300 Flush
10693 132/300 Write block 3551 len 28672 data: 2 2 2 2 2 2 2
10694 133/300 Write block 4421 len 28672 data: 2 2 2 2 2 2 2
10695 134/300 Read block 3083 len 32768
10696 135/300 Write block 129 len 40960 data: 4 4 5 4 4 4 4 4 4 4
10697 136/300 Write block 3504 len 8192 data: 3 3
10698 137/300 Read block 1899 len 40960
10699 138/300 Write block 4044 len 28672 data: 2 2 3 2 2 2 3
10700 139/300 Read block 3637 len 16384
10701 140/300 Read block 2558 len 32768
10702 141/300 Read block 1742 len 28672
10703 142/300 Flush
10704 143/300 Read block 816 len 40960
10705 144/300 Write block 2481 len 16384 data: 5 5 4 4
10706 145/300 Write block 1951 len 28672 data: 4 3 3 4 5 4 5
10707 146/300 Read block 356 len 24576
10708 147/300 Read block 4241 len 20480
10709 148/300 Read block 1565 len 8192
10710 149/300 Read block 2483 len 4096
10711 150/300 Write block 4628 len 12288 data: 2 2 2
10712 151/300 Write block 4210 len 20480 data: 4 4 4 4 3
10713 152/300 Flush
10714 153/300 Write block 3990 len 28672 data: 3 4 5 5 5 5 5
10715 154/300 Write block 3986 len 12288 data: 2 3 3
10716 155/300 Flush
10717 156/300 Read block 2951 len 4096
10718 157/300 Flush
10719 158/300 Write block 452 len 36864 data: 4 5 4 5 5 4 2 3 4
10720 159/300 Read block 3663 len 24576
10721 160/300 Read block 2051 len 36864
10722 161/300 Write block 2365 len 4096 data: 2
10723 162/300 Read block 1132 len 24576
10724 163/300 Write block 1465 len 12288 data: 6 6 6
10725 164/300 Read block 409 len 40960
10726 165/300 Read block 4846 len 32768
10727 166/300 Read block 2358 len 40960
10728 167/300 Write block 1701 len 16384 data: 5 5 3 3
10729 168/300 Write block 529 len 32768 data: 2 2 3 2 2 2 2 3
10730 169/300 Read block 3954 len 36864
10731 170/300 Write block 178 len 36864 data: 2 3 3 3 3 3 3 4 4
10732 171/300 Read block 3509 len 24576
10733 172/300 Read block 2815 len 16384
10734 173/300 Read block 3028 len 20480
10735 174/300 Write block 2666 len 12288 data: 4 4 4
10736 175/300 Write block 2753 len 12288 data: 3 3 3
10737 176/300 Write block 2850 len 32768 data: 2 3 2 2 2 3 3 3
10738 177/300 Write block 2024 len 32768 data: 2 2 2 2 2 3 4 4
10739 178/300 Write block 2900 len 20480 data: 5 4 4 5 5
10740 179/300 Write block 4254 len 24576 data: 2 2 2 2 2 2
10741 180/300 Write block 4055 len 24576 data: 3 4 4 4 5 4
10742 181/300 Read block 17 len 4096
10743 182/300 Write block 4641 len 24576 data: 2 2 2 2 2 2
10744 183/300 Read block 4480 len 12288
10745 184/300 Write block 4963 len 28672 data: 3 2 2 2 3 2 2
10746 185/300 Write block 2508 len 4096 data: 2
10747 186/300 Read block 3518 len 8192
10748 187/300 Write block 1315 len 36864 data: 2 2 2 2 2 2 2 2 3
10749 188/300 Flush
10750 189/300 Write block 324 len 24576 data: 6 6 7 6 6 6
10751 190/300 Read block 926 len 20480
10752 191/300 Write block 1447 len 24576 data: 6 4 3 4 5 5
10753 192/300 Flush
10754 193/300 Flush
10755 194/300 Read block 1431 len 4096
10756 195/300 Flush
10757 196/300 Write block 2367 len 20480 data: 3 3 3 3 4
10758 197/300 Write block 383 len 36864 data: 4 3 3 2 3 2 2 2 2
10759 198/300 Flush
10760 199/300 Write block 4373 len 20480 data: 5 4 3 3 3
10761 200/300 Read block 1747 len 24576
10762 201/300 Flush
10763 202/300 Write block 1556 len 20480 data: 3 3 4 3 4
10764 203/300 Read block 1471 len 32768
10765 204/300 Write block 3849 len 12288 data: 4 3 2
10766 205/300 Read block 3092 len 4096
10767 206/300 Read block 4130 len 40960
10768 207/300 Read block 2120 len 8192
10769 208/300 Read block 1089 len 12288
10770 209/300 Write block 1754 len 12288 data: 6 5 4
10771 210/300 Read block 4483 len 36864
10772 211/300 Read block 368 len 24576
10773 212/300 Read block 583 len 20480
10774 213/300 Read block 2743 len 36864
10775 214/300 Read block 2623 len 12288
10776 215/300 Flush
10777 216/300 Read block 1548 len 36864
10778 217/300 Read block 58 len 28672
10779 218/300 Read block 4809 len 32768
10780 219/300 Write block 589 len 12288 data: 4 3 2
10781 220/300 Read block 3702 len 8192
10782 221/300 Read block 412 len 4096
10783 222/300 Write block 1565 len 32768 data: 4 5 6 6 6 6 4 4
10784 223/300 Read block 4504 len 20480
10785 224/300 Write block 1381 len 4096 data: 5
10786 225/300 Write block 2694 len 40960 data: 4 4 4 5 4 4 4 5 5 6
10787 226/300 Write block 66 len 4096 data: 4
10788 227/300 Flush
10789 228/300 Read block 2324 len 40960
10790 229/300 Read block 2630 len 16384
10791 230/300 Read block 3691 len 4096
10792 231/300 Write block 4974 len 28672 data: 3 4 4 4 3 2 3
10793 232/300 Write block 4662 len 36864 data: 4 5 6 7 7 6 4 3 2
10794 233/300 Write block 2793 len 36864 data: 3 2 3 3 3 3 3 3 4
10795 234/300 Write block 1700 len 8192 data: 5 6
10796 235/300 Read block 2916 len 28672
10797 236/300 Read block 1628 len 4096
10798 237/300 Read block 2964 len 28672
10799 238/300 Read block 4870 len 24576
10800 239/300 Read block 4284 len 40960
10801 240/300 Read block 3288 len 4096
10802 241/300 Write block 3197 len 12288 data: 4 4 3
10803 242/300 Read block 2972 len 4096
10804 243/300 Read block 2756 len 20480
10805 244/300 Write block 3806 len 28672 data: 4 4 4 5 5 5 5
10806 245/300 Read block 4232 len 20480
10807 246/300 Read block 178 len 24576
10808 247/300 Read block 3363 len 40960
10809 248/300 Write block 4793 len 32768 data: 3 3 3 3 3 3 2 2
10810 249/300 Flush
10811 250/300 Read block 63 len 36864
10812 251/300 Read block 3264 len 8192
10813 252/300 Write block 509 len 40960 data: 2 2 2 2 3 4 4 4 4 4
10814 253/300 Write block 2263 len 40960 data: 4 3 3 2 2 4 3 3 3 3
10815 254/300 Write block 4564 len 24576 data: 3 3 3 3 3 3
10816 255/300 Write block 1755 len 24576 data: 6 5 4 2 2 2
10817 256/300 Write block 3282 len 20480 data: 7 6 6 5 5
10818 257/300 Flush
10819 258/300 Read block 1088 len 8192
10820 259/300 Write block 665 len 40960 data: 2 2 2 2 3 3 4 4 5 5
10821 260/300 Read block 857 len 16384
10822 261/300 Flush
10823 262/300 Write block 4385 len 8192 data: 2 3
10824 263/300 Write block 131 len 12288 data: 6 5 5
10825 264/300 Read block 4324 len 16384
10826 265/300 Write block 355 len 4096 data: 3
10827 266/300 Read block 3995 len 36864
10828 267/300 Read block 4751 len 28672
10829 268/300 Read block 3244 len 20480
10830 269/300 Read block 2765 len 16384
10831 270/300 Write block 4024 len 20480 data: 2 2 4 3 4
10832 271/300 Read block 4025 len 36864
10833 272/300 Read block 4450 len 32768
10834 273/300 Read block 3638 len 8192
10835 274/300 Flush
10836 275/300 Flush
10837 276/300 Read block 93 len 4096
10838 277/300 Write block 1322 len 28672 data: 3 4 3 4 4 3 4
10839 278/300 Read block 1563 len 32768
10840 279/300 Read block 899 len 28672
10841 280/300 Read block 2102 len 4096
10842 281/300 Read block 3366 len 16384
10843 282/300 Write block 775 len 36864 data: 6 6 4 3 2 2 2 2 2
10844 283/300 Write block 306 len 32768 data: 3 4 4 3 3 4 3 4
10845 284/300 Read block 2860 len 8192
10846 285/300 Write block 3335 len 20480 data: 2 2 2 2 2
10847 286/300 Read block 945 len 40960
10848 287/300 Read block 1911 len 16384
10849 288/300 Write block 1838 len 12288 data: 4 3 2
10850 289/300 Read block 2503 len 12288
10851 290/300 Flush
10852 291/300 Write block 3932 len 40960 data: 2 2 2 2 2 3 3 3 3 4
10853 292/300 Read block 312 len 24576
10854 293/300 Write block 2136 len 12288 data: 2 2 3
10855 294/300 Write block 4439 len 24576 data: 3 3 2 2 2 2
10856 295/300 Write block 725 len 8192 data: 4 3
10857 296/300 Read block 3109 len 32768
10858 297/300 Read block 2201 len 40960
10859 298/300 Read block 3714 len 16384
10860 299/300 Read block 602 len 32768
10861 300/300 Write block 2129 len 28672 data: 2 2 2 2 2 2 2
10862 301/300 Write block 2774 len 28672 data: 4 5 4 4 5 5 4
10863 Main task has work to do, go find it
10864 got dsc Start(2)
10865 start 2
10866 Jan 14 16:12:54.616 INFO request completed, latency_us: 248, response_code: 204, uri: /start/cid/2, method: POST, req_id: 9d9fefd4-ed3e-4f8d-ba5d-bb009f31ca00, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
10867 Replay: started 2, returned:Ok(())
10868 ----------------------------------------------------------------
10869 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
10870 States: Active Active Offline
10871 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10872 4018 Acked Flush 0 Done Done Sent false
10873 4019 Acked Write 1 Done Done Sent false
10874 4020 Acked Write 2 Done Done Sent false
10875 4021 Acked Read 5 Done Done Sent false
10876 4022 Acked Write 7 Done Done Sent false
10877 4023 Acked Flush 0 Done Done Sent false
10878 4024 Acked Write 6 Done Done Sent false
10879 4025 Acked Write 7 Done Done Sent false
10880 4026 Acked Write 10 Done Done Sent false
10881 4027 Acked Write 9 Done Done Sent false
10882 4028 Acked Write 1 Done Done Sent false
10883 4029 Acked Flush 0 Done Done Sent false
10884 4030 Acked Write 8 Done Done Sent false
10885 4031 Acked Write 1 Done Done Sent false
10886 4032 Acked Read 10 Done Done Sent false
10887 4033 Acked Read 5 Done Done Sent false
10888 4034 Acked Read 3 Done Done Sent false
10889 4035 Acked Read 7 Done Done Sent false
10890 4036 Acked Read 10 Done Done Sent false
10891 4037 Acked Read 3 Done Done Sent false
10892 4038 Acked Write 3 Done Done Sent false
10893 4039 Acked Write 5 Done Done Sent false
10894 4040 Acked Write 9 Done Done Sent false
10895 4041 Acked Read 9 Done Done Sent false
10896 4042 Acked Read 2 Done Done Sent false
10897 4043 Acked Read 2 Done Done Sent false
10898 4044 Acked Write 5 Done Done Sent false
10899 4045 Acked Flush 0 Done Done Sent false
10900 4046 Acked Read 2 Done Done Sent false
10901 4047 Acked Read 3 Done Done Sent false
10902 4048 Acked Flush 0 Done Done Sent false
10903 4049 Acked Flush 0 Done Done Sent false
10904 4050 Acked Read 10 Done Done Sent false
10905 4051 Acked Read 1 Done Done Sent false
10906 4052 Acked Read 8 Done Done Sent false
10907 4053 Acked Read 4 Done Done Sent false
10908 4054 Acked Read 5 Done Done Sent false
10909 4055 Acked Write 3 Done Done Sent false
10910 4056 Acked Write 2 Done Done Sent false
10911 4057 Acked Write 2 Done Done Sent false
10912 4058 Acked Write 1 Done Done Sent false
10913 4059 Acked Write 7 Done Done Sent false
10914 4060 Acked Write 8 Done Done Sent false
10915 4061 Acked Read 6 Done Done Sent false
10916 4062 Acked Read 4 Done Done Sent false
10917 4063 Acked Write 3 Done Done Sent false
10918 4064 Acked Read 8 Done Done Sent false
10919 4065 Acked Read 4 Done Done Sent false
10920 4066 Acked Flush 0 Done Done Sent false
10921 4067 Acked Read 2 Done Done Sent false
10922 4068 Acked Read 1 Done Done Sent false
10923 4069 Acked Read 6 Done Done Sent false
10924 4070 Acked Read 2 Done Done Sent false
10925 4071 Acked Write 10 Done Done Sent false
10926 4072 Acked Read 5 Done Done Sent false
10927 4073 Acked Read 9 Done Done Sent false
10928 4074 Acked Read 3 Done Done Sent false
10929 4075 Acked Flush 0 Done Done Sent false
10930 4076 Acked Write 9 Done Done Sent false
10931 4077 Acked Write 9 Done Done Sent false
10932 4078 Acked Read 7 Done Done Sent false
10933 4079 Acked Flush 0 Done Done Sent false
10934 4080 Acked Read 4 Done Done Sent false
10935 4081 Acked Read 1 Done Done Sent false
10936 4082 Acked Write 8 Done Done Sent false
10937 4083 Acked Read 5 Done Done Sent false
10938 4084 Acked Write 10 Done Done Sent false
10939 4085 Acked Read 5 Done Done Sent false
10940 4086 Acked Write 3 Done Done Sent false
10941 4087 Acked Read 8 Done Done Sent false
10942 4088 Acked Read 4 Done Done Sent false
10943 4089 Acked Read 4 Done Done Sent false
10944 4090 Acked Flush 0 Done Done Sent false
10945 4091 Acked Read 2 Done Done Sent false
10946 4092 Acked Read 1 Done Done Sent false
10947 4093 Acked Read 1 Done Done Sent false
10948 4094 Acked Read 5 Done Done Sent false
10949 4095 Acked Write 5 Done Done Sent false
10950 4096 Acked Flush 0 Done Done Sent false
10951 4097 Acked Read 3 Done Done Sent false
10952 4098 Acked Flush 0 Done Done Sent false
10953 4099 Acked Flush 0 Done Done Sent false
10954 4100 Acked Read 6 Done Done Sent false
10955 4101 Acked Write 3 Done Done Sent false
10956 4102 Acked Flush 0 Done Done Sent false
10957 4103 Acked Read 3 Done Done Sent false
10958 4104 Acked Write 9 Done Done Sent false
10959 4105 Acked Read 3 Done Done Sent false
10960 4106 Acked Read 9 Done Done Sent false
10961 4107 Acked Write 2 Done Done Sent false
10962 4108 Acked Write 9 Done Done Sent false
10963 4109 Acked Flush 0 Done Done Sent false
10964 4110 Acked Read 10 Done Done Sent false
10965 4111 Acked Write 3 Done Done Sent false
10966 4112 Acked Flush 0 Done Done Sent false
10967 4113 Acked Read 10 Done Done Sent false
10968 4114 Acked Read 4 Done Done Sent false
10969 4115 Acked Write 5 Done Done Sent false
10970 4116 Acked Read 3 Done Done Sent false
10971 4117 Acked Write 2 Done Done Sent false
10972 4118 Acked Read 7 Done Done Sent false
10973 4119 Acked Read 6 Done Done Sent false
10974 4120 Acked Read 7 Done Done Sent false
10975 4121 Acked Read 10 Done Done Sent false
10976 4122 Acked Read 2 Done Done Sent false
10977 4123 Acked Read 7 Done Done Sent false
10978 4124 Acked Write 4 Done Done Sent false
10979 4125 Acked Write 7 Done Done Sent false
10980 4126 Acked Write 6 Done Done Sent false
10981 4127 Acked Write 5 Done Done Sent false
10982 4128 Acked Write 9 Done Done Sent false
10983 4129 Acked Flush 0 Done Done Sent false
10984 4130 Acked Write 9 Done Done Sent false
10985 4131 Acked Read 3 Done Done Sent false
10986 4132 Acked Read 10 Done Done Sent false
10987 4133 Acked Write 2 Done Done Sent false
10988 4134 Acked Read 9 Done Done Sent false
10989 4135 Acked Read 10 Done Done Sent false
10990 4136 Acked Write 4 Done Done Sent false
10991 4137 Acked Write 8 Done Done Sent false
10992 4138 Acked Write 7 Done Done Sent false
10993 4139 Acked Read 7 Done Done Sent false
10994 4140 Acked Write 5 Done Done Sent false
10995 4141 Acked Read 6 Done Done Sent false
10996 4142 Acked Write 1 Done Done Sent false
10997 4143 Acked Write 3 Done Done Sent false
10998 4144 Acked Flush 0 Done Done Sent false
10999 4145 Acked Read 10 Done Done Sent false
11000 4146 Acked Read 6 Done Done Sent false
11001 4147 Acked Read 1 Done Done Sent false
11002 4148 Acked Flush 0 Done Done Sent false
11003 4149 Acked Write 7 Done Done Sent false
11004 4150 Acked Write 7 Done Done Sent false
11005 4151 Acked Read 8 Done Done Sent false
11006 4152 Acked Write 10 Done Done Sent false
11007 4153 Acked Write 2 Done Done Sent false
11008 4154 Acked Read 10 Done Done Sent false
11009 4155 Acked Write 7 Done Done Sent false
11010 4156 Acked Read 4 Done Done Sent false
11011 4157 Acked Read 8 Done Done Sent false
11012 4158 Acked Read 7 Done Done Sent false
11013 4159 Acked Flush 0 Done Done Sent false
11014 4160 Acked Read 10 Done Done Sent false
11015 4161 Acked Write 4 Done Done Sent false
11016 4162 Acked Write 7 Done Done Sent false
11017 4163 Acked Read 6 Done Done Sent false
11018 4164 Acked Read 5 Done Done Sent false
11019 4165 Acked Read 2 Done Done Sent false
11020 4166 Acked Read 1 Done Done Sent false
11021 4167 Acked Write 3 Done Done Sent false
11022 4168 Acked Write 5 Done Done Sent false
11023 4169 Acked Flush 0 Done Done Sent false
11024 4170 Acked Write 7 Done Done Sent false
11025 4171 Acked Write 3 Done Done Sent false
11026 4172 Acked Flush 0 Done Done Sent false
11027 4173 Acked Read 1 Done Done Sent false
11028 4174 Acked Flush 0 Done Done Sent false
11029 4175 Acked Write 9 Done Done Sent false
11030 4176 Acked Read 6 Done Done Sent false
11031 4177 Acked Read 9 Done Done Sent false
11032 4178 Acked Write 1 Done Done Sent false
11033 4179 Acked Read 6 Done Done Sent false
11034 4180 Acked Write 3 Done Done Sent false
11035 4181 Acked Read 10 Done Done Sent false
11036 4182 Acked Read 8 Done Done Sent false
11037 4183 Acked Read 10 Done Done Sent false
11038 4184 Acked Write 4 Done Done Sent false
11039 4185 Acked Write 8 Done Done Sent false
11040 4186 Acked Read 9 Done Done Sent false
11041 4187 Acked Write 9 Done Done Sent false
11042 4188 Acked Read 6 Done Done Sent false
11043 4189 Acked Read 4 Done Done Sent false
11044 4190 Acked Read 5 Done Done Sent false
11045 4191 Acked Write 3 Done Done Sent false
11046 4192 Acked Write 3 Done Done Sent false
11047 4193 Acked Write 8 Done Done Sent false
11048 4194 Acked Write 8 Done Done Sent false
11049 4195 Acked Write 5 Done Done Sent false
11050 4196 Acked Write 6 Done Done Sent false
11051 4197 Acked Write 6 Done Done Sent false
11052 4198 Acked Read 1 Done Done Sent false
11053 4199 Acked Write 6 Done Done Sent false
11054 4200 Acked Read 3 Done Done Sent false
11055 4201 Acked Write 7 Done Done Sent false
11056 4202 Acked Write 1 Done Done Sent false
11057 4203 Acked Read 2 Done Done Sent false
11058 4204 Acked Write 9 Done Done Sent false
11059 4205 Acked Flush 0 Done Done Sent false
11060 4206 Acked Write 6 Done Done Sent false
11061 4207 Acked Read 5 Done Done Sent false
11062 4208 Acked Write 6 Done Done Sent false
11063 4209 Acked Flush 0 Done Done Sent false
11064 4210 Acked Flush 0 Done Done Sent false
11065 4211 Acked Read 1 Done Done Sent false
11066 4212 Acked Flush 0 Done Done Sent false
11067 4213 Acked Write 5 Done Done Sent false
11068 4214 Acked Write 9 Done Done Sent false
11069 4215 Acked Flush 0 Done Done Sent false
11070 4216 Acked Write 5 Done Done Sent false
11071 4217 Acked Read 6 Done Done Sent false
11072 4218 Acked Flush 0 Done Done Sent false
11073 4219 Acked Write 5 Done Done Sent false
11074 4220 Acked Read 8 Done Done Sent false
11075 4221 Acked Write 3 Done Done Sent false
11076 4222 Acked Read 1 Done Done Sent false
11077 4223 Acked Read 10 Done Done Sent false
11078 4224 Acked Read 2 Done Done Sent false
11079 4225 Acked Read 3 Done Done Sent false
11080 4226 Acked Write 3 Done Done Sent false
11081 4227 Acked Read 9 Done Done Sent false
11082 4228 Acked Read 6 Done Done Sent false
11083 4229 Acked Read 5 Done Done Sent false
11084 4230 Acked Read 9 Done Done Sent false
11085 4231 Acked Read 3 Done Done Sent false
11086 4232 Acked Flush 0 Done Done Sent false
11087 4233 Acked Read 9 Done Done Sent false
11088 4234 Acked Read 7 Done Done Sent false
11089 4235 Acked Read 8 Done Done Sent false
11090 4236 Acked Write 3 Done Done Sent false
11091 4237 Acked Read 2 Done Done Sent false
11092 4238 Acked Read 1 Done Done Sent false
11093 4239 Acked Write 8 Done Done Sent false
11094 4240 Acked Read 5 Done Done Sent false
11095 4241 Acked Write 1 Done Done Sent false
11096 4242 Acked Write 10 Done Done Sent false
11097 4243 Acked Write 1 Done Done Sent false
11098 4244 Acked Flush 0 Done Done Sent false
11099 4245 Acked Read 10 Done Done Sent false
11100 4246 Acked Read 4 Done Done Sent false
11101 4247 Acked Read 1 Done Done Sent false
11102 4248 Acked Write 7 Done Done Sent false
11103 4249 Acked Write 9 Done Done Sent false
11104 4250 Acked Write 9 Done Done Sent false
11105 4251 Acked Write 2 Done Done Sent false
11106 4252 Acked Read 7 Done Done Sent false
11107 4253 Acked Read 1 Done Done Sent false
11108 4254 Acked Read 7 Done Done Sent false
11109 4255 Acked Read 6 Done Done Sent false
11110 4256 Acked Read 10 Done Done Sent false
11111 4257 Acked Read 1 Done Done Sent false
11112 4258 Acked Write 3 Done Done Sent false
11113 4259 Acked Read 1 Done Done Sent false
11114 4260 Acked Read 5 Done Done Sent false
11115 4261 Acked Write 7 Done Done Sent false
11116 4262 Acked Read 5 Done Done Sent false
11117 4263 Acked Read 6 Done Done Sent false
11118 4264 Acked Read 10 Done Done Sent false
11119 4265 Acked Write 8 Done Done Sent false
11120 4266 Acked Flush 0 Done Done Sent false
11121 4267 Acked Read 9 Done Done Sent false
11122 4268 Acked Read 2 Done Done Sent false
11123 4269 Acked Write 10 Done Done Sent false
11124 4270 Acked Write 10 Done Done Sent false
11125 4271 Acked Write 6 Done Done Sent false
11126 4272 Acked Write 6 Done Done Sent false
11127 4273 Acked Write 5 Done Done Sent false
11128 4274 Acked Flush 0 Done Done Sent false
11129 4275 Acked Read 2 Done Done Sent false
11130 4276 Acked Write 10 Done Done Sent false
11131 4277 Acked Read 4 Done Done Sent false
11132 4278 Acked Flush 0 Done Done Sent false
11133 4279 Acked Write 2 Done Done Sent false
11134 4280 Acked Write 3 Done Done Sent false
11135 4281 Acked Read 4 Done Done Sent false
11136 4282 Acked Write 1 Done Done Sent false
11137 4283 Acked Read 9 Done Done Sent false
11138 4284 Acked Read 7 Done Done Sent false
11139 4285 Acked Read 5 Done Done Sent false
11140 4286 Acked Read 4 Done Done Sent false
11141 4287 Acked Write 5 Done Done Sent false
11142 4288 Acked Read 9 Done Done Sent false
11143 4289 Acked Read 8 Done Done Sent false
11144 4290 Acked Read 2 Done Done Sent false
11145 4291 Acked Flush 0 Done Done Sent false
11146 4292 Acked Flush 0 Done Done Sent false
11147 4293 Acked Read 1 Done Done Sent false
11148 4294 Acked Write 7 Done Done Sent false
11149 4295 Acked Read 8 Done Done Sent false
11150 4296 Acked Read 7 Done Done Sent false
11151 4297 Acked Read 1 Done Done Sent false
11152 4298 Acked Read 4 Done Done Sent false
11153 4299 Acked Write 9 Done Done Sent false
11154 4300 Acked Write 8 Done Done Sent false
11155 4301 Acked Read 2 Done Done Sent false
11156 4302 Acked Write 5 Done Done Sent false
11157 4303 Acked Read 10 Done Done Sent false
11158 4304 Acked Read 4 Done Done Sent false
11159 4305 Acked Write 3 Done Done Sent false
11160 4306 Acked Read 3 Done Done Sent false
11161 4307 Acked Flush 0 Done Done Sent false
11162 4308 Acked Write 10 Done Done Sent false
11163 4309 Acked Read 6 Done Done Sent false
11164 4310 Acked Write 3 Done Done Sent false
11165 4311 Acked Write 6 Done Done Sent false
11166 4312 Acked Write 2 Done Done Sent false
11167 4313 Acked Read 8 Done Done Sent false
11168 4314 Acked Read 10 Done Done Sent false
11169 4315 Acked Read 4 Done Done Sent false
11170 4316 Acked Read 8 Done Done Sent false
11171 4317 Acked Write 7 Done Done Sent false
11172 4318 Acked Write 7 Done Done Sent false
11173 STATES DS:0 DS:1 DS:2 TOTAL
11174 Sent 0 0 301 301
11175 Done 301 301 0 602
11176 Skipped 0 0 0 0
11177 Error 0 0 0 0
11178 Last Flush: 4307 4307 4017
11179 Downstairs last five completed: 4017 4016 4015 4014 4013
11180 Upstairs last five completed: 4318 4317 4316 4315 4314
11181 CLIENT: Up:0 ds:301 act:2
11182 [8830] got start action
11183 [8830] I am going to restart
11184 Starting downstairs at port 8830
11185 Make output file at "/tmp/dsc/downstairs-8830.txt"
11186 [8830][2] reports Starting
11187 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1213)
11188 [8830][2] reports Running
11189 ----------------------------------------------------------------
11190 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11191 States: Active Active Offline
11192 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11193 4018 Acked Flush 0 Done Done Sent false
11194 4019 Acked Write 1 Done Done Sent false
11195 4020 Acked Write 2 Done Done Sent false
11196 4021 Acked Read 5 Done Done Sent false
11197 4022 Acked Write 7 Done Done Sent false
11198 4023 Acked Flush 0 Done Done Sent false
11199 4024 Acked Write 6 Done Done Sent false
11200 4025 Acked Write 7 Done Done Sent false
11201 4026 Acked Write 10 Done Done Sent false
11202 4027 Acked Write 9 Done Done Sent false
11203 4028 Acked Write 1 Done Done Sent false
11204 4029 Acked Flush 0 Done Done Sent false
11205 4030 Acked Write 8 Done Done Sent false
11206 4031 Acked Write 1 Done Done Sent false
11207 4032 Acked Read 10 Done Done Sent false
11208 4033 Acked Read 5 Done Done Sent false
11209 4034 Acked Read 3 Done Done Sent false
11210 4035 Acked Read 7 Done Done Sent false
11211 4036 Acked Read 10 Done Done Sent false
11212 4037 Acked Read 3 Done Done Sent false
11213 4038 Acked Write 3 Done Done Sent false
11214 4039 Acked Write 5 Done Done Sent false
11215 4040 Acked Write 9 Done Done Sent false
11216 4041 Acked Read 9 Done Done Sent false
11217 4042 Acked Read 2 Done Done Sent false
11218 4043 Acked Read 2 Done Done Sent false
11219 4044 Acked Write 5 Done Done Sent false
11220 4045 Acked Flush 0 Done Done Sent false
11221 4046 Acked Read 2 Done Done Sent false
11222 4047 Acked Read 3 Done Done Sent false
11223 4048 Acked Flush 0 Done Done Sent false
11224 4049 Acked Flush 0 Done Done Sent false
11225 4050 Acked Read 10 Done Done Sent false
11226 4051 Acked Read 1 Done Done Sent false
11227 4052 Acked Read 8 Done Done Sent false
11228 4053 Acked Read 4 Done Done Sent false
11229 4054 Acked Read 5 Done Done Sent false
11230 4055 Acked Write 3 Done Done Sent false
11231 4056 Acked Write 2 Done Done Sent false
11232 4057 Acked Write 2 Done Done Sent false
11233 4058 Acked Write 1 Done Done Sent false
11234 4059 Acked Write 7 Done Done Sent false
11235 4060 Acked Write 8 Done Done Sent false
11236 4061 Acked Read 6 Done Done Sent false
11237 4062 Acked Read 4 Done Done Sent false
11238 4063 Acked Write 3 Done Done Sent false
11239 4064 Acked Read 8 Done Done Sent false
11240 4065 Acked Read 4 Done Done Sent false
11241 4066 Acked Flush 0 Done Done Sent false
11242 4067 Acked Read 2 Done Done Sent false
11243 4068 Acked Read 1 Done Done Sent false
11244 4069 Acked Read 6 Done Done Sent false
11245 4070 Acked Read 2 Done Done Sent false
11246 4071 Acked Write 10 Done Done Sent false
11247 4072 Acked Read 5 Done Done Sent false
11248 4073 Acked Read 9 Done Done Sent false
11249 4074 Acked Read 3 Done Done Sent false
11250 4075 Acked Flush 0 Done Done Sent false
11251 4076 Acked Write 9 Done Done Sent false
11252 4077 Acked Write 9 Done Done Sent false
11253 4078 Acked Read 7 Done Done Sent false
11254 4079 Acked Flush 0 Done Done Sent false
11255 4080 Acked Read 4 Done Done Sent false
11256 4081 Acked Read 1 Done Done Sent false
11257 4082 Acked Write 8 Done Done Sent false
11258 4083 Acked Read 5 Done Done Sent false
11259 4084 Acked Write 10 Done Done Sent false
11260 4085 Acked Read 5 Done Done Sent false
11261 4086 Acked Write 3 Done Done Sent false
11262 4087 Acked Read 8 Done Done Sent false
11263 4088 Acked Read 4 Done Done Sent false
11264 4089 Acked Read 4 Done Done Sent false
11265 4090 Acked Flush 0 Done Done Sent false
11266 4091 Acked Read 2 Done Done Sent false
11267 4092 Acked Read 1 Done Done Sent false
11268 4093 Acked Read 1 Done Done Sent false
11269 4094 Acked Read 5 Done Done Sent false
11270 4095 Acked Write 5 Done Done Sent false
11271 4096 Acked Flush 0 Done Done Sent false
11272 4097 Acked Read 3 Done Done Sent false
11273 4098 Acked Flush 0 Done Done Sent false
11274 4099 Acked Flush 0 Done Done Sent false
11275 4100 Acked Read 6 Done Done Sent false
11276 4101 Acked Write 3 Done Done Sent false
11277 4102 Acked Flush 0 Done Done Sent false
11278 4103 Acked Read 3 Done Done Sent false
11279 4104 Acked Write 9 Done Done Sent false
11280 4105 Acked Read 3 Done Done Sent false
11281 4106 Acked Read 9 Done Done Sent false
11282 4107 Acked Write 2 Done Done Sent false
11283 4108 Acked Write 9 Done Done Sent false
11284 4109 Acked Flush 0 Done Done Sent false
11285 4110 Acked Read 10 Done Done Sent false
11286 4111 Acked Write 3 Done Done Sent false
11287 4112 Acked Flush 0 Done Done Sent false
11288 4113 Acked Read 10 Done Done Sent false
11289 4114 Acked Read 4 Done Done Sent false
11290 4115 Acked Write 5 Done Done Sent false
11291 4116 Acked Read 3 Done Done Sent false
11292 4117 Acked Write 2 Done Done Sent false
11293 4118 Acked Read 7 Done Done Sent false
11294 4119 Acked Read 6 Done Done Sent false
11295 4120 Acked Read 7 Done Done Sent false
11296 4121 Acked Read 10 Done Done Sent false
11297 4122 Acked Read 2 Done Done Sent false
11298 4123 Acked Read 7 Done Done Sent false
11299 4124 Acked Write 4 Done Done Sent false
11300 4125 Acked Write 7 Done Done Sent false
11301 4126 Acked Write 6 Done Done Sent false
11302 4127 Acked Write 5 Done Done Sent false
11303 4128 Acked Write 9 Done Done Sent false
11304 4129 Acked Flush 0 Done Done Sent false
11305 4130 Acked Write 9 Done Done Sent false
11306 4131 Acked Read 3 Done Done Sent false
11307 4132 Acked Read 10 Done Done Sent false
11308 4133 Acked Write 2 Done Done Sent false
11309 4134 Acked Read 9 Done Done Sent false
11310 4135 Acked Read 10 Done Done Sent false
11311 4136 Acked Write 4 Done Done Sent false
11312 4137 Acked Write 8 Done Done Sent false
11313 4138 Acked Write 7 Done Done Sent false
11314 4139 Acked Read 7 Done Done Sent false
11315 4140 Acked Write 5 Done Done Sent false
11316 4141 Acked Read 6 Done Done Sent false
11317 4142 Acked Write 1 Done Done Sent false
11318 4143 Acked Write 3 Done Done Sent false
11319 4144 Acked Flush 0 Done Done Sent false
11320 4145 Acked Read 10 Done Done Sent false
11321 4146 Acked Read 6 Done Done Sent false
11322 4147 Acked Read 1 Done Done Sent false
11323 4148 Acked Flush 0 Done Done Sent false
11324 4149 Acked Write 7 Done Done Sent false
11325 4150 Acked Write 7 Done Done Sent false
11326 4151 Acked Read 8 Done Done Sent false
11327 4152 Acked Write 10 Done Done Sent false
11328 4153 Acked Write 2 Done Done Sent false
11329 4154 Acked Read 10 Done Done Sent false
11330 4155 Acked Write 7 Done Done Sent false
11331 4156 Acked Read 4 Done Done Sent false
11332 4157 Acked Read 8 Done Done Sent false
11333 4158 Acked Read 7 Done Done Sent false
11334 4159 Acked Flush 0 Done Done Sent false
11335 4160 Acked Read 10 Done Done Sent false
11336 4161 Acked Write 4 Done Done Sent false
11337 4162 Acked Write 7 Done Done Sent false
11338 4163 Acked Read 6 Done Done Sent false
11339 4164 Acked Read 5 Done Done Sent false
11340 4165 Acked Read 2 Done Done Sent false
11341 4166 Acked Read 1 Done Done Sent false
11342 4167 Acked Write 3 Done Done Sent false
11343 4168 Acked Write 5 Done Done Sent false
11344 4169 Acked Flush 0 Done Done Sent false
11345 4170 Acked Write 7 Done Done Sent false
11346 4171 Acked Write 3 Done Done Sent false
11347 4172 Acked Flush 0 Done Done Sent false
11348 4173 Acked Read 1 Done Done Sent false
11349 4174 Acked Flush 0 Done Done Sent false
11350 4175 Acked Write 9 Done Done Sent false
11351 4176 Acked Read 6 Done Done Sent false
11352 4177 Acked Read 9 Done Done Sent false
11353 4178 Acked Write 1 Done Done Sent false
11354 4179 Acked Read 6 Done Done Sent false
11355 4180 Acked Write 3 Done Done Sent false
11356 4181 Acked Read 10 Done Done Sent false
11357 4182 Acked Read 8 Done Done Sent false
11358 4183 Acked Read 10 Done Done Sent false
11359 4184 Acked Write 4 Done Done Sent false
11360 4185 Acked Write 8 Done Done Sent false
11361 4186 Acked Read 9 Done Done Sent false
11362 4187 Acked Write 9 Done Done Sent false
11363 4188 Acked Read 6 Done Done Sent false
11364 4189 Acked Read 4 Done Done Sent false
11365 4190 Acked Read 5 Done Done Sent false
11366 4191 Acked Write 3 Done Done Sent false
11367 4192 Acked Write 3 Done Done Sent false
11368 4193 Acked Write 8 Done Done Sent false
11369 4194 Acked Write 8 Done Done Sent false
11370 4195 Acked Write 5 Done Done Sent false
11371 4196 Acked Write 6 Done Done Sent false
11372 4197 Acked Write 6 Done Done Sent false
11373 4198 Acked Read 1 Done Done Sent false
11374 4199 Acked Write 6 Done Done Sent false
11375 4200 Acked Read 3 Done Done Sent false
11376 4201 Acked Write 7 Done Done Sent false
11377 4202 Acked Write 1 Done Done Sent false
11378 4203 Acked Read 2 Done Done Sent false
11379 4204 Acked Write 9 Done Done Sent false
11380 4205 Acked Flush 0 Done Done Sent false
11381 4206 Acked Write 6 Done Done Sent false
11382 4207 Acked Read 5 Done Done Sent false
11383 4208 Acked Write 6 Done Done Sent false
11384 4209 Acked Flush 0 Done Done Sent false
11385 4210 Acked Flush 0 Done Done Sent false
11386 4211 Acked Read 1 Done Done Sent false
11387 4212 Acked Flush 0 Done Done Sent false
11388 4213 Acked Write 5 Done Done Sent false
11389 4214 Acked Write 9 Done Done Sent false
11390 4215 Acked Flush 0 Done Done Sent false
11391 4216 Acked Write 5 Done Done Sent false
11392 4217 Acked Read 6 Done Done Sent false
11393 4218 Acked Flush 0 Done Done Sent false
11394 4219 Acked Write 5 Done Done Sent false
11395 4220 Acked Read 8 Done Done Sent false
11396 4221 Acked Write 3 Done Done Sent false
11397 4222 Acked Read 1 Done Done Sent false
11398 4223 Acked Read 10 Done Done Sent false
11399 4224 Acked Read 2 Done Done Sent false
11400 4225 Acked Read 3 Done Done Sent false
11401 4226 Acked Write 3 Done Done Sent false
11402 4227 Acked Read 9 Done Done Sent false
11403 4228 Acked Read 6 Done Done Sent false
11404 4229 Acked Read 5 Done Done Sent false
11405 4230 Acked Read 9 Done Done Sent false
11406 4231 Acked Read 3 Done Done Sent false
11407 4232 Acked Flush 0 Done Done Sent false
11408 4233 Acked Read 9 Done Done Sent false
11409 4234 Acked Read 7 Done Done Sent false
11410 4235 Acked Read 8 Done Done Sent false
11411 4236 Acked Write 3 Done Done Sent false
11412 4237 Acked Read 2 Done Done Sent false
11413 4238 Acked Read 1 Done Done Sent false
11414 4239 Acked Write 8 Done Done Sent false
11415 4240 Acked Read 5 Done Done Sent false
11416 4241 Acked Write 1 Done Done Sent false
11417 4242 Acked Write 10 Done Done Sent false
11418 4243 Acked Write 1 Done Done Sent false
11419 4244 Acked Flush 0 Done Done Sent false
11420 4245 Acked Read 10 Done Done Sent false
11421 4246 Acked Read 4 Done Done Sent false
11422 4247 Acked Read 1 Done Done Sent false
11423 4248 Acked Write 7 Done Done Sent false
11424 4249 Acked Write 9 Done Done Sent false
11425 4250 Acked Write 9 Done Done Sent false
11426 4251 Acked Write 2 Done Done Sent false
11427 4252 Acked Read 7 Done Done Sent false
11428 4253 Acked Read 1 Done Done Sent false
11429 4254 Acked Read 7 Done Done Sent false
11430 4255 Acked Read 6 Done Done Sent false
11431 4256 Acked Read 10 Done Done Sent false
11432 4257 Acked Read 1 Done Done Sent false
11433 4258 Acked Write 3 Done Done Sent false
11434 4259 Acked Read 1 Done Done Sent false
11435 4260 Acked Read 5 Done Done Sent false
11436 4261 Acked Write 7 Done Done Sent false
11437 4262 Acked Read 5 Done Done Sent false
11438 4263 Acked Read 6 Done Done Sent false
11439 4264 Acked Read 10 Done Done Sent false
11440 4265 Acked Write 8 Done Done Sent false
11441 4266 Acked Flush 0 Done Done Sent false
11442 4267 Acked Read 9 Done Done Sent false
11443 4268 Acked Read 2 Done Done Sent false
11444 4269 Acked Write 10 Done Done Sent false
11445 4270 Acked Write 10 Done Done Sent false
11446 4271 Acked Write 6 Done Done Sent false
11447 4272 Acked Write 6 Done Done Sent false
11448 4273 Acked Write 5 Done Done Sent false
11449 4274 Acked Flush 0 Done Done Sent false
11450 4275 Acked Read 2 Done Done Sent false
11451 4276 Acked Write 10 Done Done Sent false
11452 4277 Acked Read 4 Done Done Sent false
11453 4278 Acked Flush 0 Done Done Sent false
11454 4279 Acked Write 2 Done Done Sent false
11455 4280 Acked Write 3 Done Done Sent false
11456 4281 Acked Read 4 Done Done Sent false
11457 4282 Acked Write 1 Done Done Sent false
11458 4283 Acked Read 9 Done Done Sent false
11459 4284 Acked Read 7 Done Done Sent false
11460 4285 Acked Read 5 Done Done Sent false
11461 4286 Acked Read 4 Done Done Sent false
11462 4287 Acked Write 5 Done Done Sent false
11463 4288 Acked Read 9 Done Done Sent false
11464 4289 Acked Read 8 Done Done Sent false
11465 4290 Acked Read 2 Done Done Sent false
11466 4291 Acked Flush 0 Done Done Sent false
11467 4292 Acked Flush 0 Done Done Sent false
11468 4293 Acked Read 1 Done Done Sent false
11469 4294 Acked Write 7 Done Done Sent false
11470 4295 Acked Read 8 Done Done Sent false
11471 4296 Acked Read 7 Done Done Sent false
11472 4297 Acked Read 1 Done Done Sent false
11473 4298 Acked Read 4 Done Done Sent false
11474 4299 Acked Write 9 Done Done Sent false
11475 4300 Acked Write 8 Done Done Sent false
11476 4301 Acked Read 2 Done Done Sent false
11477 4302 Acked Write 5 Done Done Sent false
11478 4303 Acked Read 10 Done Done Sent false
11479 4304 Acked Read 4 Done Done Sent false
11480 4305 Acked Write 3 Done Done Sent false
11481 4306 Acked Read 3 Done Done Sent false
11482 4307 Acked Flush 0 Done Done Sent false
11483 4308 Acked Write 10 Done Done Sent false
11484 4309 Acked Read 6 Done Done Sent false
11485 4310 Acked Write 3 Done Done Sent false
11486 4311 Acked Write 6 Done Done Sent false
11487 4312 Acked Write 2 Done Done Sent false
11488 4313 Acked Read 8 Done Done Sent false
11489 4314 Acked Read 10 Done Done Sent false
11490 4315 Acked Read 4 Done Done Sent false
11491 4316 Acked Read 8 Done Done Sent false
11492 4317 Acked Write 7 Done Done Sent false
11493 4318 Acked Write 7 Done Done Sent false
11494 4319 Acked Flush 0 Done Done Sent false
11495 STATES DS:0 DS:1 DS:2 TOTAL
11496 Sent 0 0 302 302
11497 Done 302 302 0 604
11498 Skipped 0 0 0 0
11499 Error 0 0 0 0
11500 Last Flush: 4319 4319 4017
11501 Downstairs last five completed: 4017 4016 4015 4014 4013
11502 Upstairs last five completed: 4319 4318 4317 4316 4315
11503 CLIENT: Up:0 ds:302 act:2
115042025-01-14T16:13:00.381ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
115052025-01-14T16:13:00.381ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36651) = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
115062025-01-14T16:13:00.383ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
115072025-01-14T16:13:00.383ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
115082025-01-14T16:13:00.383ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
115092025-01-14T16:13:00.383ZINFOcrucible: send last flush ID to this DS: 4017 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
115102025-01-14T16:13:00.384ZINFOcrucible: Replied this last flush ID: 4017 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
115112025-01-14T16:13:00.384ZINFOcrucible: [2] client re-new 302 jobs since flush 4017 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
115122025-01-14T16:13:00.384ZINFOcrucible: [2] Replayed 302 jobs since flush: 4017 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
11513 ----------------------------------------------------------------
11514 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
11515 Downstairs last five completed: 4319 4318 4317 4316 4315
11516 Upstairs last five completed: 4319 4318 4317 4316 4315
11517 CLIENT: Up:0 ds:0 act:3
11518 Replay: All jobs finished
11519 Main task has work to do, go find it
11520 got dsc Stop(1)
11521 stop 1
11522 Jan 14 16:13:02.620 INFO request completed, latency_us: 319, response_code: 204, uri: /stop/cid/1, method: [8820] Got stop action so:false kr:false
11523 POST, req_id: c1d09bd8-b0b8-480c-9509-01a1761a6e23, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
115242025-01-14T16:13:02.621ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
115252025-01-14T16:13:02.621ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
115262025-01-14T16:13:02.621ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
11527 Jan 14 16:13:02.621 INFO request completed, latency_us: 112, response_code: 200, uri: /state/cid/1, method: GET, req_id: 279a8281-36fb-4e5c-982f-8eda40577d2b, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
115282025-01-14T16:13:02.621ZINFOcrucible: client task is exiting = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
11529 [8820] Exited with: ExitStatus(unix_wait_status(9))
11530 [8820][1] reports Exit
11531 Jan 14 16:13:06.623 INFO request completed, latency_us: 232, response_code: 200, uri: /state/cid/1, method: GET, req_id: 5e062e79-fbea-4c0c-8a92-d17bcb68443a, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
11532 001/300 Write block 2337 len 4096 data: 3
11533 002/300 Write block 2617 len 36864 data: 4 5 5 4 3 4 4 6 6
11534 003/300 Write block 4788 len 16384 data: 3 3 4 3
11535 004/300 Write block 4158 len 36864 data: 4 5 4 4 4 4 4 3 3
11536 005/300 Read block 4892 len 32768
11537 006/300 Read block 3687 len 24576
11538 007/300 Read block 744 len 8192
11539 008/300 Read block 1222 len 24576
11540 009/300 Write block 3296 len 32768 data: 3 3 4 3 3 3 2 2
11541 010/300 Write block 4712 len 8192 data: 2 3
11542 011/300 Read block 4010 len 36864
11543 012/300 Write block 2602 len 20480 data: 3 3 3 5 5
11544 013/300 Read block 3835 len 24576
11545 014/300 Read block 38 len 20480
11546 015/300 Read block 255 len 4096
11547 016/300 Read block 4951 len 40960
11548 017/300 Write block 921 len 40960 data: 3 3 3 4 5 5 5 5 5 5
11549 018/300 Read block 1669 len 40960
11550 019/300 Read block 2583 len 12288
11551 020/300 Write block 1224 len 8192 data: 4 2
11552 021/300 Flush
11553 022/300 Read block 1523 len 32768
11554 023/300 Read block 4984 len 40960
11555 024/300 Read block 3420 len 28672
11556 025/300 Write block 2921 len 20480 data: 5 5 4 4 5
11557 026/300 Read block 3229 len 24576
11558 027/300 Write block 1275 len 12288 data: 3 3 3
11559 028/300 Read block 1384 len 8192
11560 029/300 Write block 3 len 12288 data: 4 4 4
11561 030/300 Write block 2534 len 36864 data: 2 2 3 4 3 3 3 3 4
11562 031/300 Write block 58 len 4096 data: 3
11563 032/300 Read block 256 len 12288
11564 033/300 Flush
11565 034/300 Read block 2651 len 24576
11566 035/300 Read block 3540 len 20480
11567 036/300 Read block 2430 len 4096
11568 037/300 Read block 1935 len 28672
11569 038/300 Write block 4320 len 32768 data: 3 3 2 2 4 4 4 4
11570 039/300 Write block 3289 len 40960 data: 3 3 3 2 3 3 3 4 4 5
11571 040/300 Write block 241 len 32768 data: 4 4 4 4 3 3 2 2
11572 041/300 Read block 4599 len 12288
11573 042/300 Read block 3940 len 16384
11574 043/300 Write block 3689 len 36864 data: 4 3 4 4 4 2 2 2 2
11575 044/300 Write block 1080 len 32768 data: 2 3 3 3 3 3 3 2
11576 045/300 Read block 4723 len 12288
11577 046/300 Read block 2926 len 20480
11578 047/300 Read block 3945 len 40960
11579 048/300 Read block 3524 len 28672
11580 049/300 Read block 4824 len 28672
11581 050/300 Read block 847 len 20480
11582 051/300 Read block 1874 len 36864
11583 052/300 Flush
11584 053/300 Write block 4332 len 20480 data: 3 2 2 3 3
11585 054/300 Write block 4173 len 4096 data: 5
11586 055/300 Read block 2379 len 28672
11587 056/300 Read block 1875 len 16384
11588 057/300 Read block 2803 len 32768
11589 058/300 Write block 4001 len 40960 data: 6 6 5 4 5 5 6 5 4 5
11590 059/300 Write block 1813 len 32768 data: 4 4 4 5 5 5 5 5
11591 060/300 Write block 4156 len 36864 data: 3 4 5 6 5 5 5 5 5
11592 061/300 Read block 3357 len 4096
11593 062/300 Write block 33 len 4096 data: 3
11594 063/300 Read block 1563 len 20480
11595 064/300 Flush
11596 065/300 Read block 4632 len 20480
11597 066/300 Flush
11598 067/300 Read block 1545 len 36864
11599 068/300 Write block 3244 len 16384 data: 2 4 4 4
11600 069/300 Read block 1098 len 8192
11601 070/300 Read block 1943 len 24576
11602 071/300 Write block 2129 len 24576 data: 3 3 3 3 3 3
11603 072/300 Write block 3217 len 20480 data: 2 2 2 2 2
11604 073/300 Flush
11605 074/300 Read block 2585 len 32768
11606 075/300 Write block 659 len 20480 data: 3 2 2 2 2
11607 076/300 Read block 4772 len 4096
11608 077/300 Write block 3312 len 8192 data: 2 2
11609 078/300 Write block 721 len 28672 data: 2 2 2 3 5 4 3
11610 079/300 Read block 15 len 12288
11611 080/300 Write block 2630 len 4096 data: 3
11612 081/300 Read block 3455 len 36864
11613 082/300 Write block 4112 len 4096 data: 2
11614 083/300 Read block 3805 len 32768
11615 084/300 Write block 851 len 32768 data: 4 4 4 4 4 3 2 2
11616 085/300 Write block 643 len 32768 data: 3 4 5 4 4 3 3 3
11617 086/300 Read block 1747 len 24576
11618 087/300 Read block 890 len 32768
11619 088/300 Write block 403 len 16384 data: 4 6 4 3
11620 089/300 Write block 3996 len 28672 data: 6 5 7 7 6 7 7
11621 090/300 Read block 1211 len 36864
11622 091/300 Read block 1715 len 28672
11623 092/300 Write block 4898 len 4096 data: 5
11624 093/300 Flush
11625 094/300 Write block 1992 len 36864 data: 5 6 4 5 6 6 6 5 4
11626 095/300 Write block 1417 len 16384 data: 3 3 3 4
11627 096/300 Read block 1216 len 36864
11628 097/300 Read block 3916 len 16384
11629 098/300 Read block 4775 len 4096
11630 099/300 Read block 3767 len 24576
11631 100/300 Read block 1658 len 8192
11632 101/300 Read block 4133 len 28672
11633 102/300 Read block 3909 len 16384
11634 103/300 Write block 3015 len 12288 data: 3 4 4
11635 104/300 Read block 192 len 32768
11636 105/300 Flush
11637 106/300 Read block 1560 len 20480
11638 107/300 Flush
11639 108/300 Write block 1703 len 20480 data: 4 4 3 3 4
11640 109/300 Read block 4170 len 40960
11641 110/300 Write block 4403 len 24576 data: 2 2 3 3 3 3
11642 111/300 Flush
11643 112/300 Flush
11644 113/300 Read block 761 len 12288
11645 114/300 Read block 1172 len 12288
11646 115/300 Flush
11647 116/300 Read block 2166 len 20480
11648 117/300 Write block 3294 len 24576 data: 4 4 5 5 6 4
11649 118/300 Read block 3585 len 16384
11650 119/300 Write block 3688 len 20480 data: 5 5 4 5 5
11651 120/300 Write block 334 len 8192 data: 5 6
11652 121/300 Write block 2073 len 8192 data: 3 3
11653 122/300 Write block 1531 len 16384 data: 5 6 5 4
11654 123/300 Read block 4231 len 24576
11655 124/300 Write block 4589 len 12288 data: 4 4 4
11656 125/300 Read block 1848 len 24576
11657 126/300 Write block 345 len 32768 data: 4 4 4 6 6 6 3 3
11658 127/300 Flush
11659 128/300 Write block 3792 len 28672 data: 3 3 5 5 5 5 5
11660 129/300 Write block 2916 len 36864 data: 3 3 2 2 4 6 6 5 5
11661 130/300 Write block 2106 len 36864 data: 4 5 4 4 4 4 4 5 4
11662 131/300 Write block 3714 len 20480 data: 4 4 4 4 4
11663 132/300 Flush
11664 133/300 Read block 116 len 8192
11665 134/300 Flush
11666 135/300 Write block 1099 len 4096 data: 3
11667 136/300 Write block 581 len 24576 data: 3 3 3 4 5 5
11668 137/300 Read block 2185 len 16384
11669 138/300 Flush
11670 139/300 Read block 2579 len 40960
11671 140/300 Read block 1085 len 8192
11672 141/300 Read block 4083 len 40960
11673 142/300 Flush
11674 143/300 Write block 4118 len 32768 data: 2 2 2 2 3 3 4 4
11675 144/300 Read block 3031 len 8192
11676 145/300 Read block 2907 len 20480
11677 146/300 Read block 4135 len 32768
11678 147/300 Read block 3930 len 32768
11679 148/300 Flush
11680 149/300 Write block 816 len 28672 data: 4 4 3 4 4 3 4
11681 150/300 Write block 4905 len 12288 data: 3 2 2
11682 151/300 Write block 295 len 32768 data: 4 2 2 2 2 2 2 3
11683 152/300 Read block 1505 len 24576
11684 153/300 Write block 568 len 8192 data: 2 2
11685 154/300 Read block 4411 len 32768
11686 155/300 Write block 4320 len 28672 data: 4 4 3 3 5 5 5
11687 156/300 Read block 4373 len 40960
11688 157/300 Read block 3998 len 32768
11689 158/300 Read block 4636 len 4096
11690 159/300 Read block 4177 len 20480
11691 160/300 Write block 462 len 16384 data: 6 6 5 4
11692 161/300 Flush
11693 162/300 Read block 3288 len 36864
11694 163/300 Read block 3606 len 28672
11695 164/300 Read block 1585 len 8192
11696 165/300 Flush
11697 166/300 Read block 4674 len 4096
11698 167/300 Write block 1188 len 24576 data: 3 3 4 4 3 3
11699 168/300 Write block 573 len 24576 data: 2 2 2 3 3 3
11700 169/300 Read block 4063 len 8192
11701 170/300 Write block 142 len 40960 data: 4 6 6 6 6 6 6 5 5 3
11702 171/300 Read block 208 len 16384
11703 172/300 Read block 4324 len 32768
11704 173/300 Write block 3944 len 12288 data: 3 2 2
11705 174/300 Read block 3683 len 28672
11706 175/300 Flush
11707 176/300 Read block 2871 len 20480
11708 177/300 Read block 2100 len 12288
11709 178/300 Read block 4541 len 8192
11710 179/300 Write block 25 len 36864 data: 6 6 5 6 5 4 4 4 4
11711 180/300 Write block 1058 len 40960 data: 2 2 3 3 3 4 5 6 6 5
11712 181/300 Write block 2474 len 24576 data: 3 4 4 4 5 5
11713 182/300 Write block 534 len 16384 data: 3 3 4 3
11714 183/300 Read block 798 len 16384
11715 184/300 Write block 2519 len 28672 data: 2 2 2 2 2 2 3
11716 185/300 Write block 1074 len 32768 data: 4 4 3 3 3 3 3 4
11717 186/300 Read block 773 len 32768
11718 187/300 Write block 4643 len 24576 data: 3 3 3 3 2 2
11719 188/300 Write block 590 len 16384 data: 4 3 2 2
11720 189/300 Write block 3874 len 36864 data: 3 2 2 2 2 2 2 2 2
11721 190/300 Read block 1856 len 24576
11722 191/300 Read block 3931 len 20480
11723 192/300 Write block 805 len 24576 data: 5 5 6 6 6 6
11724 193/300 Read block 1124 len 24576
11725 194/300 Read block 2252 len 28672
11726 195/300 Read block 269 len 8192
11727 196/300 Flush
11728 197/300 Read block 2411 len 40960
11729 198/300 Read block 4222 len 16384
11730 199/300 Write block 4049 len 4096 data: 3
11731 200/300 Read block 3181 len 28672
11732 201/300 Read block 1993 len 40960
11733 202/300 Flush
11734 203/300 Write block 4156 len 36864 data: 4 5 6 7 6 6 6 6 6
11735 204/300 Read block 2733 len 28672
11736 205/300 Read block 1334 len 40960
11737 206/300 Write block 2947 len 20480 data: 3 2 3 4 4
11738 207/300 Write block 263 len 16384 data: 4 3 3 3
11739 208/300 Read block 714 len 4096
11740 209/300 Write block 1269 len 12288 data: 3 2 3
11741 210/300 Read block 3199 len 32768
11742 211/300 Read block 2914 len 32768
11743 212/300 Read block 3212 len 12288
11744 213/300 Write block 1136 len 12288 data: 2 2 2
11745 214/300 Read block 1640 len 36864
11746 215/300 Flush
11747 216/300 Write block 2341 len 16384 data: 3 3 3 3
11748 217/300 Write block 4716 len 16384 data: 3 4 2 2
11749 218/300 Flush
11750 219/300 Read block 2408 len 4096
11751 220/300 Flush
11752 221/300 Write block 509 len 20480 data: 3 3 3 3 4
11753 222/300 Write block 3254 len 24576 data: 5 5 5 3 2 2
11754 223/300 Flush
11755 224/300 Write block 2858 len 28672 data: 2 2 2 3 2 2 2
11756 225/300 Write block 877 len 36864 data: 2 3 3 3 4 4 5 5 6
11757 226/300 Write block 1667 len 28672 data: 2 2 3 4 3 2 2
11758 227/300 Write block 3416 len 20480 data: 4 4 4 5 7
11759 228/300 Write block 4392 len 28672 data: 6 6 6 5 4 4 4
11760 229/300 Write block 2312 len 12288 data: 6 7 7
11761 230/300 Write block 4500 len 12288 data: 3 3 3
11762 231/300 Read block 1536 len 32768
11763 232/300 Write block 928 len 40960 data: 6 6 6 3 3 2 3 3 3 4
11764 233/300 Write block 4584 len 8192 data: 2 3
11765 234/300 Flush
11766 235/300 Flush
11767 236/300 Read block 3140 len 12288
11768 237/300 Read block 1336 len 8192
11769 238/300 Read block 3213 len 36864
11770 239/300 Read block 2404 len 36864
11771 240/300 Write block 2386 len 32768 data: 2 2 2 4 5 5 5 5
11772 241/300 Read block 1185 len 12288
11773 242/300 Read block 2496 len 8192
11774 243/300 Write block 2547 len 8192 data: 4 5
11775 244/300 Write block 4432 len 20480 data: 2 2 3 3 3
11776 245/300 Read block 1524 len 8192
11777 246/300 Write block 852 len 36864 data: 5 5 5 5 4 3 3 2 2
11778 247/300 Flush
11779 248/300 Flush
11780 249/300 Read block 1320 len 16384
11781 250/300 Read block 2723 len 16384
11782 251/300 Read block 1464 len 36864
11783 252/300 Read block 4779 len 16384
11784 253/300 Read block 1179 len 20480
11785 254/300 Read block 1802 len 4096
11786 255/300 Read block 2730 len 32768
11787 256/300 Write block 861 len 20480 data: 2 2 2 2 2
11788 257/300 Write block 3739 len 24576 data: 3 4 4 4 3 3
11789 258/300 Write block 3931 len 40960 data: 2 3 3 3 3 3 4 4 4 4
11790 259/300 Read block 2440 len 16384
11791 260/300 Read block 2964 len 4096
11792 261/300 Write block 716 len 16384 data: 2 2 2 2
11793 262/300 Read block 318 len 20480
11794 263/300 Read block 3408 len 28672
11795 264/300 Write block 2951 len 40960 data: 5 2 2 2 2 2 3 3 4 4
11796 265/300 Write block 2648 len 24576 data: 3 3 3 2 2 2
11797 266/300 Write block 58 len 32768 data: 4 4 4 3 3 3 3 4
11798 267/300 Read block 3384 len 12288
11799 268/300 Write block 3388 len 36864 data: 3 3 3 4 4 3 3 3 4
11800 269/300 Read block 540 len 32768
11801 270/300 Write block 1722 len 20480 data: 4 4 5 4 4
11802 271/300 Read block 2095 len 24576
11803 272/300 Write block 425 len 20480 data: 5 5 4 4 4
11804 273/300 Write block 3787 len 4096 data: 3
11805 274/300 Read block 3013 len 20480
11806 275/300 Read block 1119 len 32768
11807 276/300 Read block 3985 len 24576
11808 277/300 Write block 4168 len 16384 data: 3 2 3 3
11809 278/300 Read block 3511 len 40960
11810 279/300 Read block 474 len 20480
11811 280/300 Flush
11812 281/300 Read block 621 len 4096
11813 282/300 Flush
11814 283/300 Flush
11815 284/300 Read block 1635 len 36864
11816 285/300 Read block 2591 len 32768
11817 286/300 Read block 2664 len 12288
11818 287/300 Flush
11819 288/300 Write block 835 len 24576 data: 2 3 5 5 5 5
11820 289/300 Write block 3227 len 40960 data: 4 5 5 5 5 5 3 4 4 4
11821 290/300 Write block 3580 len 36864 data: 3 3 3 3 3 2 2 2 2
11822 291/300 Read block 3887 len 4096
11823 292/300 Read block 4152 len 8192
11824 293/300 Read block 561 len 8192
11825 294/300 Write block 4690 len 24576 data: 4 4 5 4 3 2
11826 295/300 Read block 2699 len 32768
11827 296/300 Read block 663 len 32768
11828 297/300 Read block 4768 len 32768
11829 298/300 Write block 469 len 28672 data: 2 2 2 2 2 2 2
11830 299/300 Write block 1489 len 28672 data: 4 5 4 4 4 3 3
11831 300/300 Write block 2436 len 16384 data: 2 2 2 2
11832 301/300 Read block 4219 len 12288
11833 Main task has work to do, go find it
11834 got dsc Start(1)
11835 start 1
11836 Jan 14 16:13:06.854 INFO request completed, latency_us: 198, response_code: 204, uri: /start/cid/1, method: POST, req_id: bb20d2f9-2868-423b-84c3-1e03cd7d37b4, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
11837 Replay: started 1, returned:Ok(())
11838 ----------------------------------------------------------------
11839 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
11840 States: Active Offline Active
11841 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11842 4320 Acked Write 1 Done Sent Done false
11843 4321 Acked Write 9 Done Sent Done false
11844 4322 Acked Write 4 Done Sent Done false
11845 4323 Acked Write 9 Done Sent Done false
11846 4324 Acked Read 8 Done Sent Done false
11847 4325 Acked Read 6 Done Sent Done false
11848 4326 Acked Read 2 Done Sent Done false
11849 4327 Acked Read 6 Done Sent Done false
11850 4328 Acked Write 8 Done Sent Done false
11851 4329 Acked Write 2 Done Sent Done false
11852 4330 Acked Read 9 Done Sent Done false
11853 4331 Acked Write 5 Done Sent Done false
11854 4332 Acked Read 6 Done Sent Done false
11855 4333 Acked Read 5 Done Sent Done false
11856 4334 Acked Read 1 Done Sent Done false
11857 4335 Acked Read 10 Done Sent Done false
11858 4336 Acked Write 10 Done Sent Done false
11859 4337 Acked Read 10 Done Sent Done false
11860 4338 Acked Read 3 Done Sent Done false
11861 4339 Acked Write 2 Done Sent Done false
11862 4340 Acked Flush 0 Done Sent Done false
11863 4341 Acked Read 8 Done Sent Done false
11864 4342 Acked Read 10 Done Sent Done false
11865 4343 Acked Read 7 Done Sent Done false
11866 4344 Acked Write 5 Done Sent Done false
11867 4345 Acked Read 6 Done Sent Done false
11868 4346 Acked Write 3 Done Sent Done false
11869 4347 Acked Read 2 Done Sent Done false
11870 4348 Acked Write 3 Done Sent Done false
11871 4349 Acked Write 9 Done Sent Done false
11872 4350 Acked Write 1 Done Sent Done false
11873 4351 Acked Read 3 Done Sent Done false
11874 4352 Acked Flush 0 Done Sent Done false
11875 4353 Acked Read 6 Done Sent Done false
11876 4354 Acked Read 5 Done Sent Done false
11877 4355 Acked Read 1 Done Sent Done false
11878 4356 Acked Read 7 Done Sent Done false
11879 4357 Acked Write 8 Done Sent Done false
11880 4358 Acked Write 10 Done Sent Done false
11881 4359 Acked Write 8 Done Sent Done false
11882 4360 Acked Read 3 Done Sent Done false
11883 4361 Acked Read 4 Done Sent Done false
11884 4362 Acked Write 9 Done Sent Done false
11885 4363 Acked Write 8 Done Sent Done false
11886 4364 Acked Read 3 Done Sent Done false
11887 4365 Acked Read 5 Done Sent Done false
11888 4366 Acked Read 10 Done Sent Done false
11889 4367 Acked Read 7 Done Sent Done false
11890 4368 Acked Read 7 Done Sent Done false
11891 4369 Acked Read 5 Done Sent Done false
11892 4370 Acked Read 9 Done Sent Done false
11893 4371 Acked Flush 0 Done Sent Done false
11894 4372 Acked Write 5 Done Sent Done false
11895 4373 Acked Write 1 Done Sent Done false
11896 4374 Acked Read 7 Done Sent Done false
11897 4375 Acked Read 4 Done Sent Done false
11898 4376 Acked Read 8 Done Sent Done false
11899 4377 Acked Write 10 Done Sent Done false
11900 4378 Acked Write 8 Done Sent Done false
11901 4379 Acked Write 9 Done Sent Done false
11902 4380 Acked Read 1 Done Sent Done false
11903 4381 Acked Write 1 Done Sent Done false
11904 4382 Acked Read 5 Done Sent Done false
11905 4383 Acked Flush 0 Done Sent Done false
11906 4384 Acked Read 5 Done Sent Done false
11907 4385 Acked Flush 0 Done Sent Done false
11908 4386 Acked Read 9 Done Sent Done false
11909 4387 Acked Write 4 Done Sent Done false
11910 4388 Acked Read 2 Done Sent Done false
11911 4389 Acked Read 6 Done Sent Done false
11912 4390 Acked Write 6 Done Sent Done false
11913 4391 Acked Write 5 Done Sent Done false
11914 4392 Acked Flush 0 Done Sent Done false
11915 4393 Acked Read 8 Done Sent Done false
11916 4394 Acked Write 5 Done Sent Done false
11917 4395 Acked Read 1 Done Sent Done false
11918 4396 Acked Write 2 Done Sent Done false
11919 4397 Acked Write 7 Done Sent Done false
11920 4398 Acked Read 3 Done Sent Done false
11921 4399 Acked Write 1 Done Sent Done false
11922 4400 Acked Read 9 Done Sent Done false
11923 4401 Acked Write 1 Done Sent Done false
11924 4402 Acked Read 8 Done Sent Done false
11925 4403 Acked Write 8 Done Sent Done false
11926 4404 Acked Write 8 Done Sent Done false
11927 4405 Acked Read 6 Done Sent Done false
11928 4406 Acked Read 8 Done Sent Done false
11929 4407 Acked Write 4 Done Sent Done false
11930 4408 Acked Write 7 Done Sent Done false
11931 4409 Acked Read 9 Done Sent Done false
11932 4410 Acked Read 7 Done Sent Done false
11933 4411 Acked Write 1 Done Sent Done false
11934 4412 Acked Flush 0 Done Sent Done false
11935 4413 Acked Write 9 Done Sent Done false
11936 4414 Acked Write 4 Done Sent Done false
11937 4415 Acked Read 9 Done Sent Done false
11938 4416 Acked Read 4 Done Sent Done false
11939 4417 Acked Read 1 Done Sent Done false
11940 4418 Acked Read 6 Done Sent Done false
11941 4419 Acked Read 2 Done Sent Done false
11942 4420 Acked Read 7 Done Sent Done false
11943 4421 Acked Read 4 Done Sent Done false
11944 4422 Acked Write 3 Done Sent Done false
11945 4423 Acked Read 8 Done Sent Done false
11946 4424 Acked Flush 0 Done Sent Done false
11947 4425 Acked Read 5 Done Sent Done false
11948 4426 Acked Flush 0 Done Sent Done false
11949 4427 Acked Write 5 Done Sent Done false
11950 4428 Acked Read 10 Done Sent Done false
11951 4429 Acked Write 6 Done Sent Done false
11952 4430 Acked Flush 0 Done Sent Done false
11953 4431 Acked Flush 0 Done Sent Done false
11954 4432 Acked Read 3 Done Sent Done false
11955 4433 Acked Read 3 Done Sent Done false
11956 4434 Acked Flush 0 Done Sent Done false
11957 4435 Acked Read 5 Done Sent Done false
11958 4436 Acked Write 6 Done Sent Done false
11959 4437 Acked Read 4 Done Sent Done false
11960 4438 Acked Write 5 Done Sent Done false
11961 4439 Acked Write 2 Done Sent Done false
11962 4440 Acked Write 2 Done Sent Done false
11963 4441 Acked Write 4 Done Sent Done false
11964 4442 Acked Read 6 Done Sent Done false
11965 4443 Acked Write 3 Done Sent Done false
11966 4444 Acked Read 6 Done Sent Done false
11967 4445 Acked Write 8 Done Sent Done false
11968 4446 Acked Flush 0 Done Sent Done false
11969 4447 Acked Write 7 Done Sent Done false
11970 4448 Acked Write 9 Done Sent Done false
11971 4449 Acked Write 9 Done Sent Done false
11972 4450 Acked Write 5 Done Sent Done false
11973 4451 Acked Flush 0 Done Sent Done false
11974 4452 Acked Read 2 Done Sent Done false
11975 4453 Acked Flush 0 Done Sent Done false
11976 4454 Acked Write 1 Done Sent Done false
11977 4455 Acked Write 6 Done Sent Done false
11978 4456 Acked Read 4 Done Sent Done false
11979 4457 Acked Flush 0 Done Sent Done false
11980 4458 Acked Read 10 Done Sent Done false
11981 4459 Acked Read 2 Done Sent Done false
11982 4460 Acked Read 10 Done Sent Done false
11983 4461 Acked Flush 0 Done Sent Done false
11984 4462 Acked Write 8 Done Sent Done false
11985 4463 Acked Read 2 Done Sent Done false
11986 4464 Acked Read 5 Done Sent Done false
11987 4465 Acked Read 8 Done Sent Done false
11988 4466 Acked Read 8 Done Sent Done false
11989 4467 Acked Flush 0 Done Sent Done false
11990 4468 Acked Write 7 Done Sent Done false
11991 4469 Acked Write 3 Done Sent Done false
11992 4470 Acked Write 8 Done Sent Done false
11993 4471 Acked Read 6 Done Sent Done false
11994 4472 Acked Write 2 Done Sent Done false
11995 4473 Acked Read 8 Done Sent Done false
11996 4474 Acked Write 7 Done Sent Done false
11997 4475 Acked Read 10 Done Sent Done false
11998 4476 Acked Read 8 Done Sent Done false
11999 4477 Acked Read 1 Done Sent Done false
12000 4478 Acked Read 5 Done Sent Done false
12001 4479 Acked Write 4 Done Sent Done false
12002 4480 Acked Flush 0 Done Sent Done false
12003 4481 Acked Read 9 Done Sent Done false
12004 4482 Acked Read 7 Done Sent Done false
12005 4483 Acked Read 2 Done Sent Done false
12006 4484 Acked Flush 0 Done Sent Done false
12007 4485 Acked Read 1 Done Sent Done false
12008 4486 Acked Write 6 Done Sent Done false
12009 4487 Acked Write 6 Done Sent Done false
12010 4488 Acked Read 2 Done Sent Done false
12011 4489 Acked Write 10 Done Sent Done false
12012 4490 Acked Read 4 Done Sent Done false
12013 4491 Acked Read 8 Done Sent Done false
12014 4492 Acked Write 3 Done Sent Done false
12015 4493 Acked Read 7 Done Sent Done false
12016 4494 Acked Flush 0 Done Sent Done false
12017 4495 Acked Read 5 Done Sent Done false
12018 4496 Acked Read 3 Done Sent Done false
12019 4497 Acked Read 2 Done Sent Done false
12020 4498 Acked Write 9 Done Sent Done false
12021 4499 Acked Write 10 Done Sent Done false
12022 4500 Acked Write 6 Done Sent Done false
12023 4501 Acked Write 4 Done Sent Done false
12024 4502 Acked Read 4 Done Sent Done false
12025 4503 Acked Write 7 Done Sent Done false
12026 4504 Acked Write 8 Done Sent Done false
12027 4505 Acked Read 8 Done Sent Done false
12028 4506 Acked Write 6 Done Sent Done false
12029 4507 Acked Write 4 Done Sent Done false
12030 4508 Acked Write 9 Done Sent Done false
12031 4509 Acked Read 6 Done Sent Done false
12032 4510 Acked Read 5 Done Sent Done false
12033 4511 Acked Write 6 Done Sent Done false
12034 4512 Acked Read 6 Done Sent Done false
12035 4513 Acked Read 7 Done Sent Done false
12036 4514 Acked Read 2 Done Sent Done false
12037 4515 Acked Flush 0 Done Sent Done false
12038 4516 Acked Read 10 Done Sent Done false
12039 4517 Acked Read 4 Done Sent Done false
12040 4518 Acked Write 1 Done Sent Done false
12041 4519 Acked Read 7 Done Sent Done false
12042 4520 Acked Read 10 Done Sent Done false
12043 4521 Acked Flush 0 Done Sent Done false
12044 4522 Acked Write 9 Done Sent Done false
12045 4523 Acked Read 7 Done Sent Done false
12046 4524 Acked Read 10 Done Sent Done false
12047 4525 Acked Write 5 Done Sent Done false
12048 4526 Acked Write 4 Done Sent Done false
12049 4527 Acked Read 1 Done Sent Done false
12050 4528 Acked Write 3 Done Sent Done false
12051 4529 Acked Read 8 Done Sent Done false
12052 4530 Acked Read 8 Done Sent Done false
12053 4531 Acked Read 3 Done Sent Done false
12054 4532 Acked Write 3 Done Sent Done false
12055 4533 Acked Read 9 Done Sent Done false
12056 4534 Acked Flush 0 Done Sent Done false
12057 4535 Acked Write 4 Done Sent Done false
12058 4536 Acked Write 4 Done Sent Done false
12059 4537 Acked Flush 0 Done Sent Done false
12060 4538 Acked Read 1 Done Sent Done false
12061 4539 Acked Flush 0 Done Sent Done false
12062 4540 Acked Write 5 Done Sent Done false
12063 4541 Acked Write 6 Done Sent Done false
12064 4542 Acked Flush 0 Done Sent Done false
12065 4543 Acked Write 7 Done Sent Done false
12066 4544 Acked Write 9 Done Sent Done false
12067 4545 Acked Write 7 Done Sent Done false
12068 4546 Acked Write 5 Done Sent Done false
12069 4547 Acked Write 7 Done Sent Done false
12070 4548 Acked Write 3 Done Sent Done false
12071 4549 Acked Write 3 Done Sent Done false
12072 4550 Acked Read 8 Done Sent Done false
12073 4551 Acked Write 10 Done Sent Done false
12074 4552 Acked Write 2 Done Sent Done false
12075 4553 Acked Flush 0 Done Sent Done false
12076 4554 Acked Flush 0 Done Sent Done false
12077 4555 Acked Read 3 Done Sent Done false
12078 4556 Acked Read 2 Done Sent Done false
12079 4557 Acked Read 9 Done Sent Done false
12080 4558 Acked Read 9 Done Sent Done false
12081 4559 Acked Write 8 Done Sent Done false
12082 4560 Acked Read 3 Done Sent Done false
12083 4561 Acked Read 2 Done Sent Done false
12084 4562 Acked Write 2 Done Sent Done false
12085 4563 Acked Write 5 Done Sent Done false
12086 4564 Acked Read 2 Done Sent Done false
12087 4565 Acked Write 9 Done Sent Done false
12088 4566 Acked Flush 0 Done Sent Done false
12089 4567 Acked Flush 0 Done Sent Done false
12090 4568 Acked Read 4 Done Sent Done false
12091 4569 Acked Read 4 Done Sent Done false
12092 4570 Acked Read 9 Done Sent Done false
12093 4571 Acked Read 4 Done Sent Done false
12094 4572 Acked Read 5 Done Sent Done false
12095 4573 Acked Read 1 Done Sent Done false
12096 4574 Acked Read 8 Done Sent Done false
12097 4575 Acked Write 5 Done Sent Done false
12098 4576 Acked Write 6 Done Sent Done false
12099 4577 Acked Write 10 Done Sent Done false
12100 4578 Acked Read 4 Done Sent Done false
12101 4579 Acked Read 1 Done Sent Done false
12102 4580 Acked Write 4 Done Sent Done false
12103 4581 Acked Read 5 Done Sent Done false
12104 4582 Acked Read 7 Done Sent Done false
12105 4583 Acked Write 10 Done Sent Done false
12106 4584 Acked Write 6 Done Sent Done false
12107 4585 Acked Write 8 Done Sent Done false
12108 4586 Acked Read 3 Done Sent Done false
12109 4587 Acked Write 9 Done Sent Done false
12110 4588 Acked Read 8 Done Sent Done false
12111 4589 Acked Write 5 Done Sent Done false
12112 4590 Acked Read 6 Done Sent Done false
12113 4591 Acked Write 5 Done Sent Done false
12114 4592 Acked Write 1 Done Sent Done false
12115 4593 Acked Read 5 Done Sent Done false
12116 4594 Acked Read 8 Done Sent Done false
12117 4595 Acked Read 6 Done Sent Done false
12118 4596 Acked Write 4 Done Sent Done false
12119 4597 Acked Read 10 Done Sent Done false
12120 4598 Acked Read 5 Done Sent Done false
12121 4599 Acked Flush 0 Done Sent Done false
12122 4600 Acked Read 1 Done Sent Done false
12123 4601 Acked Flush 0 Done Sent Done false
12124 4602 Acked Flush 0 Done Sent Done false
12125 4603 Acked Read 9 Done Sent Done false
12126 4604 Acked Read 8 Done Sent Done false
12127 4605 Acked Read 3 Done Sent Done false
12128 4606 Acked Flush 0 Done Sent Done false
12129 4607 Acked Write 6 Done Sent Done false
12130 4608 Acked Write 10 Done Sent Done false
12131 4609 Acked Write 9 Done Sent Done false
12132 4610 Acked Read 1 Done Sent Done false
12133 4611 Acked Read 2 Done Sent Done false
12134 4612 Acked Read 2 Done Sent Done false
12135 4613 Acked Write 6 Done Sent Done false
12136 4614 Acked Read 8 Done Sent Done false
12137 4615 Acked Read 8 Done Sent Done false
12138 4616 Acked Read 8 Done Sent Done false
12139 4617 Acked Write 7 Done Sent Done false
12140 4618 Acked Write 7 Done Sent Done false
12141 4619 Acked Write 4 Done Sent Done false
12142 4620 Acked Read 3 Done Sent Done false
12143 STATES DS:0 DS:1 DS:2 TOTAL
12144 Sent 0 301 0 301
12145 Done 301 0 301 602
12146 Skipped 0 0 0 0
12147 Error 0 0 0 0
12148 Last Flush: 4606 4319 4606
12149 Downstairs last five completed: 4319 4318 4317 4316 4315
12150 Upstairs last five completed: 4620 4619 4618 4617 4616
12151 CLIENT: Up:0 ds:301 act:2
12152 [8820] got start action
12153 [8820] I am going to restart
12154 Starting downstairs at port 8820
12155 Make output file at "/tmp/dsc/downstairs-8820.txt"
12156 [8820][1] reports Starting
12157 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1214)
12158 [8820][1] reports Running
12159 ----------------------------------------------------------------
12160 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
12161 States: Active Offline Active
12162 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12163 4320 Acked Write 1 Done Sent Done false
12164 4321 Acked Write 9 Done Sent Done false
12165 4322 Acked Write 4 Done Sent Done false
12166 4323 Acked Write 9 Done Sent Done false
12167 4324 Acked Read 8 Done Sent Done false
12168 4325 Acked Read 6 Done Sent Done false
12169 4326 Acked Read 2 Done Sent Done false
12170 4327 Acked Read 6 Done Sent Done false
12171 4328 Acked Write 8 Done Sent Done false
12172 4329 Acked Write 2 Done Sent Done false
12173 4330 Acked Read 9 Done Sent Done false
12174 4331 Acked Write 5 Done Sent Done false
12175 4332 Acked Read 6 Done Sent Done false
12176 4333 Acked Read 5 Done Sent Done false
12177 4334 Acked Read 1 Done Sent Done false
12178 4335 Acked Read 10 Done Sent Done false
12179 4336 Acked Write 10 Done Sent Done false
12180 4337 Acked Read 10 Done Sent Done false
12181 4338 Acked Read 3 Done Sent Done false
12182 4339 Acked Write 2 Done Sent Done false
12183 4340 Acked Flush 0 Done Sent Done false
12184 4341 Acked Read 8 Done Sent Done false
12185 4342 Acked Read 10 Done Sent Done false
12186 4343 Acked Read 7 Done Sent Done false
12187 4344 Acked Write 5 Done Sent Done false
12188 4345 Acked Read 6 Done Sent Done false
12189 4346 Acked Write 3 Done Sent Done false
12190 4347 Acked Read 2 Done Sent Done false
12191 4348 Acked Write 3 Done Sent Done false
12192 4349 Acked Write 9 Done Sent Done false
12193 4350 Acked Write 1 Done Sent Done false
12194 4351 Acked Read 3 Done Sent Done false
12195 4352 Acked Flush 0 Done Sent Done false
12196 4353 Acked Read 6 Done Sent Done false
12197 4354 Acked Read 5 Done Sent Done false
12198 4355 Acked Read 1 Done Sent Done false
12199 4356 Acked Read 7 Done Sent Done false
12200 4357 Acked Write 8 Done Sent Done false
12201 4358 Acked Write 10 Done Sent Done false
12202 4359 Acked Write 8 Done Sent Done false
12203 4360 Acked Read 3 Done Sent Done false
12204 4361 Acked Read 4 Done Sent Done false
12205 4362 Acked Write 9 Done Sent Done false
12206 4363 Acked Write 8 Done Sent Done false
12207 4364 Acked Read 3 Done Sent Done false
12208 4365 Acked Read 5 Done Sent Done false
12209 4366 Acked Read 10 Done Sent Done false
12210 4367 Acked Read 7 Done Sent Done false
12211 4368 Acked Read 7 Done Sent Done false
12212 4369 Acked Read 5 Done Sent Done false
12213 4370 Acked Read 9 Done Sent Done false
12214 4371 Acked Flush 0 Done Sent Done false
12215 4372 Acked Write 5 Done Sent Done false
12216 4373 Acked Write 1 Done Sent Done false
12217 4374 Acked Read 7 Done Sent Done false
12218 4375 Acked Read 4 Done Sent Done false
12219 4376 Acked Read 8 Done Sent Done false
12220 4377 Acked Write 10 Done Sent Done false
12221 4378 Acked Write 8 Done Sent Done false
12222 4379 Acked Write 9 Done Sent Done false
12223 4380 Acked Read 1 Done Sent Done false
12224 4381 Acked Write 1 Done Sent Done false
12225 4382 Acked Read 5 Done Sent Done false
12226 4383 Acked Flush 0 Done Sent Done false
12227 4384 Acked Read 5 Done Sent Done false
12228 4385 Acked Flush 0 Done Sent Done false
12229 4386 Acked Read 9 Done Sent Done false
12230 4387 Acked Write 4 Done Sent Done false
12231 4388 Acked Read 2 Done Sent Done false
12232 4389 Acked Read 6 Done Sent Done false
12233 4390 Acked Write 6 Done Sent Done false
12234 4391 Acked Write 5 Done Sent Done false
12235 4392 Acked Flush 0 Done Sent Done false
12236 4393 Acked Read 8 Done Sent Done false
12237 4394 Acked Write 5 Done Sent Done false
12238 4395 Acked Read 1 Done Sent Done false
12239 4396 Acked Write 2 Done Sent Done false
12240 4397 Acked Write 7 Done Sent Done false
12241 4398 Acked Read 3 Done Sent Done false
12242 4399 Acked Write 1 Done Sent Done false
12243 4400 Acked Read 9 Done Sent Done false
12244 4401 Acked Write 1 Done Sent Done false
12245 4402 Acked Read 8 Done Sent Done false
12246 4403 Acked Write 8 Done Sent Done false
12247 4404 Acked Write 8 Done Sent Done false
12248 4405 Acked Read 6 Done Sent Done false
12249 4406 Acked Read 8 Done Sent Done false
12250 4407 Acked Write 4 Done Sent Done false
12251 4408 Acked Write 7 Done Sent Done false
12252 4409 Acked Read 9 Done Sent Done false
12253 4410 Acked Read 7 Done Sent Done false
12254 4411 Acked Write 1 Done Sent Done false
12255 4412 Acked Flush 0 Done Sent Done false
12256 4413 Acked Write 9 Done Sent Done false
12257 4414 Acked Write 4 Done Sent Done false
12258 4415 Acked Read 9 Done Sent Done false
12259 4416 Acked Read 4 Done Sent Done false
12260 4417 Acked Read 1 Done Sent Done false
12261 4418 Acked Read 6 Done Sent Done false
12262 4419 Acked Read 2 Done Sent Done false
12263 4420 Acked Read 7 Done Sent Done false
12264 4421 Acked Read 4 Done Sent Done false
12265 4422 Acked Write 3 Done Sent Done false
12266 4423 Acked Read 8 Done Sent Done false
12267 4424 Acked Flush 0 Done Sent Done false
12268 4425 Acked Read 5 Done Sent Done false
12269 4426 Acked Flush 0 Done Sent Done false
12270 4427 Acked Write 5 Done Sent Done false
12271 4428 Acked Read 10 Done Sent Done false
12272 4429 Acked Write 6 Done Sent Done false
12273 4430 Acked Flush 0 Done Sent Done false
12274 4431 Acked Flush 0 Done Sent Done false
12275 4432 Acked Read 3 Done Sent Done false
12276 4433 Acked Read 3 Done Sent Done false
12277 4434 Acked Flush 0 Done Sent Done false
12278 4435 Acked Read 5 Done Sent Done false
12279 4436 Acked Write 6 Done Sent Done false
12280 4437 Acked Read 4 Done Sent Done false
12281 4438 Acked Write 5 Done Sent Done false
12282 4439 Acked Write 2 Done Sent Done false
12283 4440 Acked Write 2 Done Sent Done false
12284 4441 Acked Write 4 Done Sent Done false
12285 4442 Acked Read 6 Done Sent Done false
12286 4443 Acked Write 3 Done Sent Done false
12287 4444 Acked Read 6 Done Sent Done false
12288 4445 Acked Write 8 Done Sent Done false
12289 4446 Acked Flush 0 Done Sent Done false
12290 4447 Acked Write 7 Done Sent Done false
12291 4448 Acked Write 9 Done Sent Done false
12292 4449 Acked Write 9 Done Sent Done false
12293 4450 Acked Write 5 Done Sent Done false
12294 4451 Acked Flush 0 Done Sent Done false
12295 4452 Acked Read 2 Done Sent Done false
12296 4453 Acked Flush 0 Done Sent Done false
12297 4454 Acked Write 1 Done Sent Done false
12298 4455 Acked Write 6 Done Sent Done false
12299 4456 Acked Read 4 Done Sent Done false
12300 4457 Acked Flush 0 Done Sent Done false
12301 4458 Acked Read 10 Done Sent Done false
12302 4459 Acked Read 2 Done Sent Done false
12303 4460 Acked Read 10 Done Sent Done false
12304 4461 Acked Flush 0 Done Sent Done false
12305 4462 Acked Write 8 Done Sent Done false
12306 4463 Acked Read 2 Done Sent Done false
12307 4464 Acked Read 5 Done Sent Done false
12308 4465 Acked Read 8 Done Sent Done false
12309 4466 Acked Read 8 Done Sent Done false
12310 4467 Acked Flush 0 Done Sent Done false
12311 4468 Acked Write 7 Done Sent Done false
12312 4469 Acked Write 3 Done Sent Done false
12313 4470 Acked Write 8 Done Sent Done false
12314 4471 Acked Read 6 Done Sent Done false
12315 4472 Acked Write 2 Done Sent Done false
12316 4473 Acked Read 8 Done Sent Done false
12317 4474 Acked Write 7 Done Sent Done false
12318 4475 Acked Read 10 Done Sent Done false
12319 4476 Acked Read 8 Done Sent Done false
12320 4477 Acked Read 1 Done Sent Done false
12321 4478 Acked Read 5 Done Sent Done false
12322 4479 Acked Write 4 Done Sent Done false
12323 4480 Acked Flush 0 Done Sent Done false
12324 4481 Acked Read 9 Done Sent Done false
12325 4482 Acked Read 7 Done Sent Done false
12326 4483 Acked Read 2 Done Sent Done false
12327 4484 Acked Flush 0 Done Sent Done false
12328 4485 Acked Read 1 Done Sent Done false
12329 4486 Acked Write 6 Done Sent Done false
12330 4487 Acked Write 6 Done Sent Done false
12331 4488 Acked Read 2 Done Sent Done false
12332 4489 Acked Write 10 Done Sent Done false
12333 4490 Acked Read 4 Done Sent Done false
12334 4491 Acked Read 8 Done Sent Done false
12335 4492 Acked Write 3 Done Sent Done false
12336 4493 Acked Read 7 Done Sent Done false
12337 4494 Acked Flush 0 Done Sent Done false
12338 4495 Acked Read 5 Done Sent Done false
12339 4496 Acked Read 3 Done Sent Done false
12340 4497 Acked Read 2 Done Sent Done false
12341 4498 Acked Write 9 Done Sent Done false
12342 4499 Acked Write 10 Done Sent Done false
12343 4500 Acked Write 6 Done Sent Done false
12344 4501 Acked Write 4 Done Sent Done false
12345 4502 Acked Read 4 Done Sent Done false
12346 4503 Acked Write 7 Done Sent Done false
12347 4504 Acked Write 8 Done Sent Done false
12348 4505 Acked Read 8 Done Sent Done false
12349 4506 Acked Write 6 Done Sent Done false
12350 4507 Acked Write 4 Done Sent Done false
12351 4508 Acked Write 9 Done Sent Done false
12352 4509 Acked Read 6 Done Sent Done false
12353 4510 Acked Read 5 Done Sent Done false
12354 4511 Acked Write 6 Done Sent Done false
12355 4512 Acked Read 6 Done Sent Done false
12356 4513 Acked Read 7 Done Sent Done false
12357 4514 Acked Read 2 Done Sent Done false
12358 4515 Acked Flush 0 Done Sent Done false
12359 4516 Acked Read 10 Done Sent Done false
12360 4517 Acked Read 4 Done Sent Done false
12361 4518 Acked Write 1 Done Sent Done false
12362 4519 Acked Read 7 Done Sent Done false
12363 4520 Acked Read 10 Done Sent Done false
12364 4521 Acked Flush 0 Done Sent Done false
12365 4522 Acked Write 9 Done Sent Done false
12366 4523 Acked Read 7 Done Sent Done false
12367 4524 Acked Read 10 Done Sent Done false
12368 4525 Acked Write 5 Done Sent Done false
12369 4526 Acked Write 4 Done Sent Done false
12370 4527 Acked Read 1 Done Sent Done false
12371 4528 Acked Write 3 Done Sent Done false
12372 4529 Acked Read 8 Done Sent Done false
12373 4530 Acked Read 8 Done Sent Done false
12374 4531 Acked Read 3 Done Sent Done false
12375 4532 Acked Write 3 Done Sent Done false
12376 4533 Acked Read 9 Done Sent Done false
12377 4534 Acked Flush 0 Done Sent Done false
12378 4535 Acked Write 4 Done Sent Done false
12379 4536 Acked Write 4 Done Sent Done false
12380 4537 Acked Flush 0 Done Sent Done false
12381 4538 Acked Read 1 Done Sent Done false
12382 4539 Acked Flush 0 Done Sent Done false
12383 4540 Acked Write 5 Done Sent Done false
12384 4541 Acked Write 6 Done Sent Done false
12385 4542 Acked Flush 0 Done Sent Done false
12386 4543 Acked Write 7 Done Sent Done false
12387 4544 Acked Write 9 Done Sent Done false
12388 4545 Acked Write 7 Done Sent Done false
12389 4546 Acked Write 5 Done Sent Done false
12390 4547 Acked Write 7 Done Sent Done false
12391 4548 Acked Write 3 Done Sent Done false
12392 4549 Acked Write 3 Done Sent Done false
12393 4550 Acked Read 8 Done Sent Done false
12394 4551 Acked Write 10 Done Sent Done false
12395 4552 Acked Write 2 Done Sent Done false
12396 4553 Acked Flush 0 Done Sent Done false
12397 4554 Acked Flush 0 Done Sent Done false
12398 4555 Acked Read 3 Done Sent Done false
12399 4556 Acked Read 2 Done Sent Done false
12400 4557 Acked Read 9 Done Sent Done false
12401 4558 Acked Read 9 Done Sent Done false
12402 4559 Acked Write 8 Done Sent Done false
12403 4560 Acked Read 3 Done Sent Done false
12404 4561 Acked Read 2 Done Sent Done false
12405 4562 Acked Write 2 Done Sent Done false
12406 4563 Acked Write 5 Done Sent Done false
12407 4564 Acked Read 2 Done Sent Done false
12408 4565 Acked Write 9 Done Sent Done false
12409 4566 Acked Flush 0 Done Sent Done false
12410 4567 Acked Flush 0 Done Sent Done false
12411 4568 Acked Read 4 Done Sent Done false
12412 4569 Acked Read 4 Done Sent Done false
12413 4570 Acked Read 9 Done Sent Done false
12414 4571 Acked Read 4 Done Sent Done false
12415 4572 Acked Read 5 Done Sent Done false
12416 4573 Acked Read 1 Done Sent Done false
12417 4574 Acked Read 8 Done Sent Done false
12418 4575 Acked Write 5 Done Sent Done false
12419 4576 Acked Write 6 Done Sent Done false
12420 4577 Acked Write 10 Done Sent Done false
12421 4578 Acked Read 4 Done Sent Done false
12422 4579 Acked Read 1 Done Sent Done false
12423 4580 Acked Write 4 Done Sent Done false
12424 4581 Acked Read 5 Done Sent Done false
12425 4582 Acked Read 7 Done Sent Done false
12426 4583 Acked Write 10 Done Sent Done false
12427 4584 Acked Write 6 Done Sent Done false
12428 4585 Acked Write 8 Done Sent Done false
12429 4586 Acked Read 3 Done Sent Done false
12430 4587 Acked Write 9 Done Sent Done false
12431 4588 Acked Read 8 Done Sent Done false
12432 4589 Acked Write 5 Done Sent Done false
12433 4590 Acked Read 6 Done Sent Done false
12434 4591 Acked Write 5 Done Sent Done false
12435 4592 Acked Write 1 Done Sent Done false
12436 4593 Acked Read 5 Done Sent Done false
12437 4594 Acked Read 8 Done Sent Done false
12438 4595 Acked Read 6 Done Sent Done false
12439 4596 Acked Write 4 Done Sent Done false
12440 4597 Acked Read 10 Done Sent Done false
12441 4598 Acked Read 5 Done Sent Done false
12442 4599 Acked Flush 0 Done Sent Done false
12443 4600 Acked Read 1 Done Sent Done false
12444 4601 Acked Flush 0 Done Sent Done false
12445 4602 Acked Flush 0 Done Sent Done false
12446 4603 Acked Read 9 Done Sent Done false
12447 4604 Acked Read 8 Done Sent Done false
12448 4605 Acked Read 3 Done Sent Done false
12449 4606 Acked Flush 0 Done Sent Done false
12450 4607 Acked Write 6 Done Sent Done false
12451 4608 Acked Write 10 Done Sent Done false
12452 4609 Acked Write 9 Done Sent Done false
12453 4610 Acked Read 1 Done Sent Done false
12454 4611 Acked Read 2 Done Sent Done false
12455 4612 Acked Read 2 Done Sent Done false
12456 4613 Acked Write 6 Done Sent Done false
12457 4614 Acked Read 8 Done Sent Done false
12458 4615 Acked Read 8 Done Sent Done false
12459 4616 Acked Read 8 Done Sent Done false
12460 4617 Acked Write 7 Done Sent Done false
12461 4618 Acked Write 7 Done Sent Done false
12462 4619 Acked Write 4 Done Sent Done false
12463 4620 Acked Read 3 Done Sent Done false
12464 4621 Acked Flush 0 Done Sent Done false
12465 STATES DS:0 DS:1 DS:2 TOTAL
12466 Sent 0 302 0 302
12467 Done 302 0 302 604
12468 Skipped 0 0 0 0
12469 Error 0 0 0 0
12470 Last Flush: 4621 4319 4621
12471 Downstairs last five completed: 4319 4318 4317 4316 4315
12472 Upstairs last five completed: 4621 4620 4619 4618 4617
12473 CLIENT: Up:0 ds:302 act:2
124742025-01-14T16:13:12.623ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
124752025-01-14T16:13:12.623ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36354) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
124762025-01-14T16:13:12.624ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
124772025-01-14T16:13:12.625ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
124782025-01-14T16:13:12.625ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
124792025-01-14T16:13:12.625ZINFOcrucible: send last flush ID to this DS: 4319 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
124802025-01-14T16:13:12.625ZINFOcrucible: Replied this last flush ID: 4319 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
124812025-01-14T16:13:12.625ZINFOcrucible: [1] client re-new 302 jobs since flush 4319 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
124822025-01-14T16:13:12.625ZINFOcrucible: [1] Replayed 302 jobs since flush: 4319 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
12483 ----------------------------------------------------------------
12484 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
12485 Downstairs last five completed: 4621 4620 4619 4618 4617
12486 Upstairs last five completed: 4621 4620 4619 4618 4617
12487 CLIENT: Up:0 ds:0 act:3
12488 Replay: All jobs finished
12489 Main task has work to do, go find it
12490 got dsc Stop(1)
12491 stop 1
12492 Jan 14 16:13:14.858[8820] Got stop action so:false kr:false
12493 INFO request completed, latency_us: 332, response_code: 204, uri: /stop/cid/1, method: POST, req_id: c37fa51f-fe1c-4839-8985-a954e4c98d0d, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
124942025-01-14T16:13:14.859ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
124952025-01-14T16:13:14.859ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
124962025-01-14T16:13:14.859ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
12497 Jan 14 16:13:14.859 INFO request completed, latency_us: 150, response_code: 200, uri: /state/cid/1, method: GET, req_id: 886f5c2a-2545-4340-9349-4f0d8317f0b5, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
124982025-01-14T16:13:14.859ZINFOcrucible: client task is exiting = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
12499 [8820] Exited with: ExitStatus(unix_wait_status(9))
12500 [8820][1] reports Exit
12501 Jan 14 16:13:18.861 INFO request completed, latency_us: 196, response_code: 200, uri: /state/cid/1, method: GET, req_id: 74c2c665-9909-4054-b0e6-ecc0c63bda4f, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
12502 001/300 Read block 1138 len 28672
12503 002/300 Read block 4711 len 24576
12504 003/300 Read block 3787 len 28672
12505 004/300 Write block 3545 len 24576 data: 2 2 2 2 2 2
12506 005/300 Write block 227 len 12288 data: 3 3 2
12507 006/300 Write block 2716 len 24576 data: 5 4 4 4 5 4
12508 007/300 Flush
12509 008/300 Read block 18 len 28672
12510 009/300 Write block 1149 len 16384 data: 3 3 3 3
12511 010/300 Write block 4804 len 16384 data: 4 4 4 4
12512 011/300 Read block 4513 len 24576
12513 012/300 Read block 2219 len 24576
12514 013/300 Read block 2342 len 8192
12515 014/300 Write block 4546 len 8192 data: 3 3
12516 015/300 Read block 1414 len 24576
12517 016/300 Write block 917 len 16384 data: 2 2 3 3
12518 017/300 Read block 2248 len 20480
12519 018/300 Write block 4971 len 40960 data: 3 3 3 4 5 5 5 4 3 4
12520 019/300 Read block 4289 len 4096
12521 020/300 Read block 3480 len 24576
12522 021/300 Write block 1772 len 12288 data: 4 3 3
12523 022/300 Write block 2126 len 36864 data: 2 2 2 4 4 4 4 4 4
12524 023/300 Write block 4426 len 12288 data: 3 3 2
12525 024/300 Write block 2123 len 8192 data: 2 2
12526 025/300 Write block 3424 len 36864 data: 6 5 5 5 4 4 4 4 3
12527 026/300 Read block 4644 len 12288
12528 027/300 Write block 4348 len 24576 data: 3 4 5 5 4 5
12529 028/300 Read block 1604 len 16384
12530 029/300 Flush
12531 030/300 Read block 2970 len 12288
12532 031/300 Write block 1147 len 40960 data: 2 3 4 4 4 4 2 2 2 3
12533 032/300 Flush
12534 033/300 Read block 4644 len 32768
12535 034/300 Read block 4532 len 28672
12536 035/300 Flush
12537 036/300 Write block 622 len 40960 data: 4 4 4 4 3 2 2 2 2 2
12538 037/300 Read block 3016 len 4096
12539 038/300 Read block 611 len 24576
12540 039/300 Read block 1733 len 12288
12541 040/300 Write block 3971 len 8192 data: 4 4
12542 041/300 Write block 4364 len 24576 data: 3 5 5 4 5 5
12543 042/300 Read block 2222 len 36864
12544 043/300 Flush
12545 044/300 Write block 679 len 36864 data: 3 3 3 4 4 4 4 4 4
12546 045/300 Read block 534 len 20480
12547 046/300 Read block 542 len 32768
12548 047/300 Read block 1819 len 8192
12549 048/300 Flush
12550 049/300 Write block 2870 len 36864 data: 5 5 5 5 4 4 4 4 5
12551 050/300 Write block 2847 len 28672 data: 2 2 2 3 4 3 3
12552 051/300 Write block 433 len 32768 data: 2 2 3 3 4 4 3 3
12553 052/300 Read block 727 len 12288
12554 053/300 Write block 4624 len 12288 data: 4 4 3
12555 054/300 Read block 911 len 20480
12556 055/300 Read block 3305 len 8192
12557 056/300 Read block 1947 len 32768
12558 057/300 Write block 1607 len 28672 data: 4 4 2 3 3 3 3
12559 058/300 Read block 1007 len 28672
12560 059/300 Read block 1455 len 12288
12561 060/300 Write block 2575 len 8192 data: 3 2
12562 061/300 Write block 1396 len 40960 data: 3 3 4 4 3 4 3 4 4 4
12563 062/300 Write block 1190 len 36864 data: 5 5 4 4 3 3 2 2 3
12564 063/300 Write block 3257 len 40960 data: 4 3 3 2 3 3 4 4 4 4
12565 064/300 Read block 3660 len 40960
12566 065/300 Read block 4686 len 32768
12567 066/300 Write block 167 len 16384 data: 5 5 5 5
12568 067/300 Write block 741 len 8192 data: 4 4
12569 068/300 Write block 1889 len 24576 data: 5 6 7 7 7 7
12570 069/300 Write block 4327 len 20480 data: 5 4 4 4 4
12571 070/300 Write block 4379 len 4096 data: 3
12572 071/300 Write block 3480 len 32768 data: 2 2 2 2 2 2 2 2
12573 072/300 Write block 870 len 16384 data: 3 3 4 3
12574 073/300 Flush
12575 074/300 Flush
12576 075/300 Write block 1468 len 28672 data: 4 4 4 4 4 4 4
12577 076/300 Read block 369 len 36864
12578 077/300 Write block 4769 len 28672 data: 3 5 5 5 6 5 6
12579 078/300 Write block 3846 len 28672 data: 5 5 5 5 4 3 3
12580 079/300 Read block 4833 len 40960
12581 080/300 Flush
12582 081/300 Read block 4677 len 12288
12583 082/300 Write block 4031 len 12288 data: 3 3 3
12584 083/300 Write block 4460 len 4096 data: 2
12585 084/300 Read block 864 len 36864
12586 085/300 Read block 4198 len 16384
12587 086/300 Flush
12588 087/300 Write block 3864 len 4096 data: 2
12589 088/300 Flush
12590 089/300 Read block 4184 len 28672
12591 090/300 Read block 3382 len 20480
12592 091/300 Flush
12593 092/300 Write block 4510 len 20480 data: 3 3 3 4 3
12594 093/300 Flush
12595 094/300 Write block 610 len 40960 data: 3 3 3 3 4 5 4 3 5 4
12596 095/300 Read block 3737 len 28672
12597 096/300 Read block 4498 len 4096
12598 097/300 Read block 3521 len 8192
12599 098/300 Read block 2728 len 8192
12600 099/300 Write block 2693 len 24576 data: 3 5 5 5 6 5
12601 100/300 Read block 3726 len 32768
12602 101/300 Read block 3884 len 8192
12603 102/300 Read block 4292 len 8192
12604 103/300 Read block 13 len 16384
12605 104/300 Read block 4452 len 40960
12606 105/300 Write block 555 len 28672 data: 5 5 4 4 4 5 3
12607 106/300 Read block 517 len 36864
12608 107/300 Read block 4546 len 4096
12609 108/300 Write block 3440 len 16384 data: 3 3 3 3
12610 109/300 Read block 3528 len 36864
12611 110/300 Flush
12612 111/300 Read block 2335 len 20480
12613 112/300 Write block 4420 len 32768 data: 2 3 3 3 3 3 4 4
12614 113/300 Read block 1385 len 24576
12615 114/300 Read block 1627 len 8192
12616 115/300 Read block 1678 len 32768
12617 116/300 Write block 4388 len 32768 data: 3 4 5 6 7 7 7 6
12618 117/300 Read block 4410 len 20480
12619 118/300 Flush
12620 119/300 Write block 156 len 12288 data: 3 3 2
12621 120/300 Write block 1793 len 8192 data: 2 2
12622 121/300 Flush
12623 122/300 Read block 337 len 16384
12624 123/300 Flush
12625 124/300 Write block 3389 len 24576 data: 4 4 5 5 4 4
12626 125/300 Write block 4293 len 36864 data: 2 2 3 3 3 3 4 3 3
12627 126/300 Read block 1438 len 4096
12628 127/300 Write block 504 len 24576 data: 2 2 2 2 2 4
12629 128/300 Read block 1761 len 40960
12630 129/300 Write block 115 len 20480 data: 3 3 4 3 3
12631 130/300 Write block 4007 len 20480 data: 7 6 5 6 5
12632 131/300 Flush
12633 132/300 Read block 129 len 12288
12634 133/300 Write block 1533 len 20480 data: 6 5 4 4 4
12635 134/300 Read block 3876 len 36864
12636 135/300 Write block 95 len 32768 data: 3 2 2 2 2 2 2 2
12637 136/300 Write block 2453 len 12288 data: 3 4 2
12638 137/300 Read block 2041 len 12288
12639 138/300 Read block 1573 len 16384
12640 139/300 Write block 559 len 24576 data: 5 6 4 3 3 4
12641 140/300 Write block 3187 len 16384 data: 3 3 3 4
12642 141/300 Read block 3954 len 8192
12643 142/300 Write block 2977 len 40960 data: 2 2 2 3 3 3 3 2 2 2
12644 143/300 Read block 2318 len 16384
12645 144/300 Read block 2885 len 20480
12646 145/300 Read block 2101 len 32768
12647 146/300 Write block 4311 len 40960 data: 2 2 2 2 3 3 3 3 3 5
12648 147/300 Read block 3478 len 16384
12649 148/300 Read block 2831 len 32768
12650 149/300 Read block 2424 len 4096
12651 150/300 Read block 2596 len 8192
12652 151/300 Read block 1003 len 12288
12653 152/300 Write block 1152 len 36864 data: 5 3 3 3 4 3 4 3 3
12654 153/300 Read block 3444 len 4096
12655 154/300 Write block 3193 len 28672 data: 5 4 3 4 5 5 4
12656 155/300 Read block 4949 len 16384
12657 156/300 Flush
12658 157/300 Write block 4699 len 16384 data: 2 2 2 2
12659 158/300 Read block 2175 len 40960
12660 159/300 Write block 4761 len 12288 data: 2 3 3
12661 160/300 Read block 4210 len 8192
12662 161/300 Write block 3165 len 16384 data: 4 4 4 4
12663 162/300 Write block 4204 len 16384 data: 3 3 3 3
12664 163/300 Write block 470 len 8192 data: 3 3
12665 164/300 Read block 3760 len 20480
12666 165/300 Write block 3999 len 4096 data: 8
12667 166/300 Read block 4648 len 20480
12668 167/300 Read block 1737 len 16384
12669 168/300 Read block 1238 len 16384
12670 169/300 Write block 3774 len 40960 data: 5 4 4 3 3 2 3 2 2 2
12671 170/300 Read block 3972 len 32768
12672 171/300 Read block 1093 len 16384
12673 172/300 Flush
12674 173/300 Write block 1716 len 28672 data: 4 3 3 2 3 3 5
12675 174/300 Flush
12676 175/300 Read block 172 len 40960
12677 176/300 Flush
12678 177/300 Read block 1483 len 32768
12679 178/300 Read block 1960 len 4096
12680 179/300 Read block 3650 len 40960
12681 180/300 Write block 4768 len 40960 data: 4 4 6 6 6 7 6 7 6 6
12682 181/300 Write block 1025 len 20480 data: 3 3 2 2 3
12683 182/300 Write block 4228 len 8192 data: 2 2
12684 183/300 Write block 4295 len 20480 data: 4 4 4 4 5
12685 184/300 Flush
12686 185/300 Write block 2696 len 16384 data: 6 7 6 5
12687 186/300 Flush
12688 187/300 Read block 2729 len 40960
12689 188/300 Read block 4706 len 20480
12690 189/300 Write block 2034 len 20480 data: 3 2 2 3 3
12691 190/300 Read block 3655 len 8192
12692 191/300 Write block 1741 len 20480 data: 5 4 4 3 5
12693 192/300 Read block 705 len 40960
12694 193/300 Write block 1058 len 16384 data: 3 3 4 4
12695 194/300 Write block 2818 len 4096 data: 3
12696 195/300 Read block 389 len 36864
12697 196/300 Read block 4964 len 28672
12698 197/300 Write block 2404 len 8192 data: 4 4
12699 198/300 Read block 2743 len 4096
12700 199/300 Write block 1277 len 40960 data: 4 3 3 3 2 2 2 2 2 2
12701 200/300 Read block 4183 len 12288
12702 201/300 Read block 3753 len 16384
12703 202/300 Read block 4032 len 40960
12704 203/300 Read block 4157 len 40960
12705 204/300 Write block 4837 len 40960 data: 2 4 4 4 4 5 5 6 5 6
12706 205/300 Read block 4407 len 12288
12707 206/300 Flush
12708 207/300 Read block 4327 len 4096
12709 208/300 Write block 4758 len 4096 data: 3
12710 209/300 Read block 939 len 20480
12711 210/300 Write block 4976 len 40960 data: 6 6 5 4 5 3 4 4 4 3
12712 211/300 Read block 3126 len 40960
12713 212/300 Write block 1288 len 20480 data: 2 2 3 3 3
12714 213/300 Read block 3558 len 32768
12715 214/300 Write block 4066 len 8192 data: 3 3
12716 215/300 Read block 4307 len 24576
12717 216/300 Flush
12718 217/300 Flush
12719 218/300 Read block 2828 len 4096
12720 219/300 Read block 4841 len 36864
12721 220/300 Write block 1545 len 20480 data: 3 3 3 3 3
12722 221/300 Write block 113 len 4096 data: 4
12723 222/300 Write block 3246 len 12288 data: 5 5 4
12724 223/300 Read block 1891 len 16384
12725 224/300 Write block 2021 len 24576 data: 2 2 2 3 3 3
12726 225/300 Read block 4789 len 8192
12727 226/300 Write block 4272 len 36864 data: 3 3 3 3 3 4 4 4 5
12728 227/300 Flush
12729 228/300 Write block 1549 len 40960 data: 4 3 2 3 3 3 3 4 4 5
12730 229/300 Write block 4083 len 40960 data: 3 4 4 4 3 4 5 5 6 6
12731 230/300 Write block 4417 len 16384 data: 4 3 3 3
12732 231/300 Read block 3654 len 40960
12733 232/300 Read block 4037 len 40960
12734 233/300 Write block 651 len 8192 data: 2 3
12735 234/300 Read block 1394 len 40960
12736 235/300 Write block 3074 len 32768 data: 3 3 3 3 3 3 3 3
12737 236/300 Write block 197 len 8192 data: 4 4
12738 237/300 Write block 4291 len 24576 data: 2 2 3 3 5 5
12739 238/300 Read block 4674 len 4096
12740 239/300 Write block 1375 len 32768 data: 3 3 3 3 4 4 6 5
12741 240/300 Read block 3676 len 40960
12742 241/300 Flush
12743 242/300 Write block 307 len 24576 data: 5 5 4 4 5 4
12744 243/300 Write block 2796 len 4096 data: 4
12745 244/300 Read block 4531 len 28672
12746 245/300 Read block 2162 len 28672
12747 246/300 Read block 4180 len 16384
12748 247/300 Write block 3333 len 24576 data: 3 3 3 3 3 3
12749 248/300 Write block 2611 len 32768 data: 3 3 5 4 4 4 5 6
12750 249/300 Read block 1056 len 20480
12751 250/300 Read block 3247 len 32768
12752 251/300 Write block 3616 len 40960 data: 3 3 3 3 2 3 3 4 3 3
12753 252/300 Read block 163 len 4096
12754 253/300 Read block 3218 len 24576
12755 254/300 Read block 1578 len 12288
12756 255/300 Write block 1846 len 12288 data: 2 2 2
12757 256/300 Write block 3347 len 28672 data: 4 3 5 4 4 6 6
12758 257/300 Write block 2020 len 36864 data: 2 3 3 3 4 4 4 3 3
12759 258/300 Read block 78 len 28672
12760 259/300 Write block 3782 len 40960 data: 3 3 2 2 2 4 4 4 4 3
12761 260/300 Read block 2561 len 4096
12762 261/300 Read block 2500 len 12288
12763 262/300 Write block 418 len 32768 data: 6 5 6 6 7 6 6 6
12764 263/300 Read block 2255 len 16384
12765 264/300 Read block 1352 len 8192
12766 265/300 Read block 3338 len 12288
12767 266/300 Write block 2721 len 4096 data: 5
12768 267/300 Read block 1001 len 40960
12769 268/300 Read block 3905 len 24576
12770 269/300 Read block 3394 len 40960
12771 270/300 Read block 4054 len 16384
12772 271/300 Write block 1074 len 32768 data: 5 5 4 4 4 4 4 5
12773 272/300 Read block 4774 len 12288
12774 273/300 Write block 39 len 20480 data: 3 3 3 3 3
12775 274/300 Write block 1219 len 4096 data: 4
12776 275/300 Read block 4479 len 28672
12777 276/300 Read block 3979 len 12288
12778 277/300 Write block 4218 len 20480 data: 3 3 3 3 4
12779 278/300 Write block 75 len 4096 data: 4
12780 279/300 Write block 4423 len 32768 data: 4 4 4 5 5 3 2 2
12781 280/300 Write block 937 len 20480 data: 5 4 4 3 3
12782 281/300 Write block 3621 len 36864 data: 4 4 5 4 4 2 2 2 2
12783 282/300 Read block 4230 len 4096
12784 283/300 Read block 1013 len 32768
12785 284/300 Read block 4258 len 40960
12786 285/300 Read block 106 len 40960
12787 286/300 Read block 999 len 24576
12788 287/300 Flush
12789 288/300 Read block 2826 len 12288
12790 289/300 Write block 1871 len 24576 data: 2 2 2 3 2 4
12791 290/300 Write block 2379 len 12288 data: 2 2 3
12792 291/300 Write block 527 len 24576 data: 3 2 3 3 4 3
12793 292/300 Read block 222 len 28672
12794 293/300 Read block 646 len 28672
12795 294/300 Write block 3800 len 20480 data: 5 3 2 2 2
12796 295/300 Write block 3153 len 20480 data: 3 3 4 3 3
12797 296/300 Flush
12798 297/300 Read block 2492 len 40960
12799 298/300 Write block 2643 len 20480 data: 5 5 5 4 3
12800 299/300 Write block 2863 len 24576 data: 3 3 2 2 2 3
12801 300/300 Write block 4844 len 8192 data: 7 6
12802 301/300 Read block 3119 len 32768
12803 Main task has work to do, go find it
12804 got dsc Start(1)
12805 start 1
12806 Jan 14 16:13:19.094 INFO request completed, latency_us: 197, response_code: 204, uri: /start/cid/1, method: POST, req_id: 360c69a3-ffd2-44de-8542-33d3e2e68437, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
12807 Replay: started 1, returned:Ok(())
12808 ----------------------------------------------------------------
12809 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
12810 States: Active Offline Active
12811 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12812 4622 Acked Read 7 Done Sent Done false
12813 4623 Acked Read 6 Done Sent Done false
12814 4624 Acked Read 7 Done Sent Done false
12815 4625 Acked Write 6 Done Sent Done false
12816 4626 Acked Write 3 Done Sent Done false
12817 4627 Acked Write 6 Done Sent Done false
12818 4628 Acked Flush 0 Done Sent Done false
12819 4629 Acked Read 7 Done Sent Done false
12820 4630 Acked Write 4 Done Sent Done false
12821 4631 Acked Write 4 Done Sent Done false
12822 4632 Acked Read 6 Done Sent Done false
12823 4633 Acked Read 6 Done Sent Done false
12824 4634 Acked Read 2 Done Sent Done false
12825 4635 Acked Write 2 Done Sent Done false
12826 4636 Acked Read 6 Done Sent Done false
12827 4637 Acked Write 4 Done Sent Done false
12828 4638 Acked Read 5 Done Sent Done false
12829 4639 Acked Write 10 Done Sent Done false
12830 4640 Acked Read 1 Done Sent Done false
12831 4641 Acked Read 6 Done Sent Done false
12832 4642 Acked Write 3 Done Sent Done false
12833 4643 Acked Write 9 Done Sent Done false
12834 4644 Acked Write 3 Done Sent Done false
12835 4645 Acked Write 2 Done Sent Done false
12836 4646 Acked Write 9 Done Sent Done false
12837 4647 Acked Read 3 Done Sent Done false
12838 4648 Acked Write 6 Done Sent Done false
12839 4649 Acked Read 4 Done Sent Done false
12840 4650 Acked Flush 0 Done Sent Done false
12841 4651 Acked Read 3 Done Sent Done false
12842 4652 Acked Write 10 Done Sent Done false
12843 4653 Acked Flush 0 Done Sent Done false
12844 4654 Acked Read 8 Done Sent Done false
12845 4655 Acked Read 7 Done Sent Done false
12846 4656 Acked Flush 0 Done Sent Done false
12847 4657 Acked Write 10 Done Sent Done false
12848 4658 Acked Read 1 Done Sent Done false
12849 4659 Acked Read 6 Done Sent Done false
12850 4660 Acked Read 3 Done Sent Done false
12851 4661 Acked Write 2 Done Sent Done false
12852 4662 Acked Write 6 Done Sent Done false
12853 4663 Acked Read 9 Done Sent Done false
12854 4664 Acked Flush 0 Done Sent Done false
12855 4665 Acked Write 9 Done Sent Done false
12856 4666 Acked Read 5 Done Sent Done false
12857 4667 Acked Read 8 Done Sent Done false
12858 4668 Acked Read 2 Done Sent Done false
12859 4669 Acked Flush 0 Done Sent Done false
12860 4670 Acked Write 9 Done Sent Done false
12861 4671 Acked Write 7 Done Sent Done false
12862 4672 Acked Write 8 Done Sent Done false
12863 4673 Acked Read 3 Done Sent Done false
12864 4674 Acked Write 3 Done Sent Done false
12865 4675 Acked Read 5 Done Sent Done false
12866 4676 Acked Read 2 Done Sent Done false
12867 4677 Acked Read 8 Done Sent Done false
12868 4678 Acked Write 7 Done Sent Done false
12869 4679 Acked Read 7 Done Sent Done false
12870 4680 Acked Read 3 Done Sent Done false
12871 4681 Acked Write 2 Done Sent Done false
12872 4682 Acked Write 10 Done Sent Done false
12873 4683 Acked Write 9 Done Sent Done false
12874 4684 Acked Write 10 Done Sent Done false
12875 4685 Acked Read 10 Done Sent Done false
12876 4686 Acked Read 8 Done Sent Done false
12877 4687 Acked Write 4 Done Sent Done false
12878 4688 Acked Write 2 Done Sent Done false
12879 4689 Acked Write 6 Done Sent Done false
12880 4690 Acked Write 5 Done Sent Done false
12881 4691 Acked Write 1 Done Sent Done false
12882 4692 Acked Write 8 Done Sent Done false
12883 4693 Acked Write 4 Done Sent Done false
12884 4694 Acked Flush 0 Done Sent Done false
12885 4695 Acked Flush 0 Done Sent Done false
12886 4696 Acked Write 7 Done Sent Done false
12887 4697 Acked Read 9 Done Sent Done false
12888 4698 Acked Write 7 Done Sent Done false
12889 4699 Acked Write 7 Done Sent Done false
12890 4700 Acked Read 10 Done Sent Done false
12891 4701 Acked Flush 0 Done Sent Done false
12892 4702 Acked Read 3 Done Sent Done false
12893 4703 Acked Write 3 Done Sent Done false
12894 4704 Acked Write 1 Done Sent Done false
12895 4705 Acked Read 9 Done Sent Done false
12896 4706 Acked Read 4 Done Sent Done false
12897 4707 Acked Flush 0 Done Sent Done false
12898 4708 Acked Write 1 Done Sent Done false
12899 4709 Acked Flush 0 Done Sent Done false
12900 4710 Acked Read 7 Done Sent Done false
12901 4711 Acked Read 5 Done Sent Done false
12902 4712 Acked Flush 0 Done Sent Done false
12903 4713 Acked Write 5 Done Sent Done false
12904 4714 Acked Flush 0 Done Sent Done false
12905 4715 Acked Write 10 Done Sent Done false
12906 4716 Acked Read 7 Done Sent Done false
12907 4717 Acked Read 1 Done Sent Done false
12908 4718 Acked Read 2 Done Sent Done false
12909 4719 Acked Read 2 Done Sent Done false
12910 4720 Acked Write 6 Done Sent Done false
12911 4721 Acked Read 8 Done Sent Done false
12912 4722 Acked Read 2 Done Sent Done false
12913 4723 Acked Read 2 Done Sent Done false
12914 4724 Acked Read 4 Done Sent Done false
12915 4725 Acked Read 10 Done Sent Done false
12916 4726 Acked Write 7 Done Sent Done false
12917 4727 Acked Read 9 Done Sent Done false
12918 4728 Acked Read 1 Done Sent Done false
12919 4729 Acked Write 4 Done Sent Done false
12920 4730 Acked Read 9 Done Sent Done false
12921 4731 Acked Flush 0 Done Sent Done false
12922 4732 Acked Read 5 Done Sent Done false
12923 4733 Acked Write 8 Done Sent Done false
12924 4734 Acked Read 6 Done Sent Done false
12925 4735 Acked Read 2 Done Sent Done false
12926 4736 Acked Read 8 Done Sent Done false
12927 4737 Acked Write 8 Done Sent Done false
12928 4738 Acked Read 5 Done Sent Done false
12929 4739 Acked Flush 0 Done Sent Done false
12930 4740 Acked Write 3 Done Sent Done false
12931 4741 Acked Write 2 Done Sent Done false
12932 4742 Acked Flush 0 Done Sent Done false
12933 4743 Acked Read 4 Done Sent Done false
12934 4744 Acked Flush 0 Done Sent Done false
12935 4745 Acked Write 6 Done Sent Done false
12936 4746 Acked Write 9 Done Sent Done false
12937 4747 Acked Read 1 Done Sent Done false
12938 4748 Acked Write 6 Done Sent Done false
12939 4749 Acked Read 10 Done Sent Done false
12940 4750 Acked Write 5 Done Sent Done false
12941 4751 Acked Write 5 Done Sent Done false
12942 4752 Acked Flush 0 Done Sent Done false
12943 4753 Acked Read 3 Done Sent Done false
12944 4754 Acked Write 5 Done Sent Done false
12945 4755 Acked Read 9 Done Sent Done false
12946 4756 Acked Write 8 Done Sent Done false
12947 4757 Acked Write 3 Done Sent Done false
12948 4758 Acked Read 3 Done Sent Done false
12949 4759 Acked Read 4 Done Sent Done false
12950 4760 Acked Write 6 Done Sent Done false
12951 4761 Acked Write 4 Done Sent Done false
12952 4762 Acked Read 2 Done Sent Done false
12953 4763 Acked Write 10 Done Sent Done false
12954 4764 Acked Read 4 Done Sent Done false
12955 4765 Acked Read 5 Done Sent Done false
12956 4766 Acked Read 8 Done Sent Done false
12957 4767 Acked Write 10 Done Sent Done false
12958 4768 Acked Read 4 Done Sent Done false
12959 4769 Acked Read 8 Done Sent Done false
12960 4770 Acked Read 1 Done Sent Done false
12961 4771 Acked Read 2 Done Sent Done false
12962 4772 Acked Read 3 Done Sent Done false
12963 4773 Acked Write 9 Done Sent Done false
12964 4774 Acked Read 1 Done Sent Done false
12965 4775 Acked Write 7 Done Sent Done false
12966 4776 Acked Read 4 Done Sent Done false
12967 4777 Acked Flush 0 Done Sent Done false
12968 4778 Acked Write 4 Done Sent Done false
12969 4779 Acked Read 10 Done Sent Done false
12970 4780 Acked Write 3 Done Sent Done false
12971 4781 Acked Read 2 Done Sent Done false
12972 4782 Acked Write 4 Done Sent Done false
12973 4783 Acked Write 4 Done Sent Done false
12974 4784 Acked Write 2 Done Sent Done false
12975 4785 Acked Read 5 Done Sent Done false
12976 4786 Acked Write 1 Done Sent Done false
12977 4787 Acked Read 5 Done Sent Done false
12978 4788 Acked Read 4 Done Sent Done false
12979 4789 Acked Read 4 Done Sent Done false
12980 4790 Acked Write 10 Done Sent Done false
12981 4791 Acked Read 8 Done Sent Done false
12982 4792 Acked Read 4 Done Sent Done false
12983 4793 Acked Flush 0 Done Sent Done false
12984 4794 Acked Write 7 Done Sent Done false
12985 4795 Acked Flush 0 Done Sent Done false
12986 4796 Acked Read 10 Done Sent Done false
12987 4797 Acked Flush 0 Done Sent Done false
12988 4798 Acked Read 8 Done Sent Done false
12989 4799 Acked Read 1 Done Sent Done false
12990 4800 Acked Read 10 Done Sent Done false
12991 4801 Acked Write 10 Done Sent Done false
12992 4802 Acked Write 5 Done Sent Done false
12993 4803 Acked Write 2 Done Sent Done false
12994 4804 Acked Write 5 Done Sent Done false
12995 4805 Acked Flush 0 Done Sent Done false
12996 4806 Acked Write 4 Done Sent Done false
12997 4807 Acked Flush 0 Done Sent Done false
12998 4808 Acked Read 10 Done Sent Done false
12999 4809 Acked Read 5 Done Sent Done false
13000 4810 Acked Write 5 Done Sent Done false
13001 4811 Acked Read 2 Done Sent Done false
13002 4812 Acked Write 5 Done Sent Done false
13003 4813 Acked Read 10 Done Sent Done false
13004 4814 Acked Write 4 Done Sent Done false
13005 4815 Acked Write 1 Done Sent Done false
13006 4816 Acked Read 9 Done Sent Done false
13007 4817 Acked Read 7 Done Sent Done false
13008 4818 Acked Write 2 Done Sent Done false
13009 4819 Acked Read 1 Done Sent Done false
13010 4820 Acked Write 10 Done Sent Done false
13011 4821 Acked Read 3 Done Sent Done false
13012 4822 Acked Read 4 Done Sent Done false
13013 4823 Acked Read 10 Done Sent Done false
13014 4824 Acked Read 10 Done Sent Done false
13015 4825 Acked Write 10 Done Sent Done false
13016 4826 Acked Read 3 Done Sent Done false
13017 4827 Acked Flush 0 Done Sent Done false
13018 4828 Acked Read 1 Done Sent Done false
13019 4829 Acked Write 1 Done Sent Done false
13020 4830 Acked Read 5 Done Sent Done false
13021 4831 Acked Write 10 Done Sent Done false
13022 4832 Acked Read 10 Done Sent Done false
13023 4833 Acked Write 5 Done Sent Done false
13024 4834 Acked Read 8 Done Sent Done false
13025 4835 Acked Write 2 Done Sent Done false
13026 4836 Acked Read 6 Done Sent Done false
13027 4837 Acked Flush 0 Done Sent Done false
13028 4838 Acked Flush 0 Done Sent Done false
13029 4839 Acked Read 1 Done Sent Done false
13030 4840 Acked Read 9 Done Sent Done false
13031 4841 Acked Write 5 Done Sent Done false
13032 4842 Acked Write 1 Done Sent Done false
13033 4843 Acked Write 3 Done Sent Done false
13034 4844 Acked Read 4 Done Sent Done false
13035 4845 Acked Write 6 Done Sent Done false
13036 4846 Acked Read 2 Done Sent Done false
13037 4847 Acked Write 9 Done Sent Done false
13038 4848 Acked Flush 0 Done Sent Done false
13039 4849 Acked Write 10 Done Sent Done false
13040 4850 Acked Write 10 Done Sent Done false
13041 4851 Acked Write 4 Done Sent Done false
13042 4852 Acked Read 10 Done Sent Done false
13043 4853 Acked Read 10 Done Sent Done false
13044 4854 Acked Write 2 Done Sent Done false
13045 4855 Acked Read 10 Done Sent Done false
13046 4856 Acked Write 8 Done Sent Done false
13047 4857 Acked Write 2 Done Sent Done false
13048 4858 Acked Write 6 Done Sent Done false
13049 4859 Acked Read 1 Done Sent Done false
13050 4860 Acked Write 8 Done Sent Done false
13051 4861 Acked Read 10 Done Sent Done false
13052 4862 Acked Flush 0 Done Sent Done false
13053 4863 Acked Write 6 Done Sent Done false
13054 4864 Acked Write 1 Done Sent Done false
13055 4865 Acked Read 7 Done Sent Done false
13056 4866 Acked Read 7 Done Sent Done false
13057 4867 Acked Read 4 Done Sent Done false
13058 4868 Acked Write 6 Done Sent Done false
13059 4869 Acked Write 8 Done Sent Done false
13060 4870 Acked Read 5 Done Sent Done false
13061 4871 Acked Read 8 Done Sent Done false
13062 4872 Acked Write 10 Done Sent Done false
13063 4873 Acked Read 1 Done Sent Done false
13064 4874 Acked Read 6 Done Sent Done false
13065 4875 Acked Read 3 Done Sent Done false
13066 4876 Acked Write 3 Done Sent Done false
13067 4877 Acked Write 7 Done Sent Done false
13068 4878 Acked Write 9 Done Sent Done false
13069 4879 Acked Read 7 Done Sent Done false
13070 4880 Acked Write 10 Done Sent Done false
13071 4881 Acked Read 1 Done Sent Done false
13072 4882 Acked Read 3 Done Sent Done false
13073 4883 Acked Write 8 Done Sent Done false
13074 4884 Acked Read 4 Done Sent Done false
13075 4885 Acked Read 2 Done Sent Done false
13076 4886 Acked Read 3 Done Sent Done false
13077 4887 Acked Write 1 Done Sent Done false
13078 4888 Acked Read 10 Done Sent Done false
13079 4889 Acked Read 6 Done Sent Done false
13080 4890 Acked Read 10 Done Sent Done false
13081 4891 Acked Read 4 Done Sent Done false
13082 4892 Acked Write 8 Done Sent Done false
13083 4893 Acked Read 3 Done Sent Done false
13084 4894 Acked Write 5 Done Sent Done false
13085 4895 Acked Write 1 Done Sent Done false
13086 4896 Acked Read 7 Done Sent Done false
13087 4897 Acked Read 3 Done Sent Done false
13088 4898 Acked Write 5 Done Sent Done false
13089 4899 Acked Write 1 Done Sent Done false
13090 4900 Acked Write 8 Done Sent Done false
13091 4901 Acked Write 5 Done Sent Done false
13092 4902 Acked Write 9 Done Sent Done false
13093 4903 Acked Read 1 Done Sent Done false
13094 4904 Acked Read 8 Done Sent Done false
13095 4905 Acked Read 10 Done Sent Done false
13096 4906 Acked Read 10 Done Sent Done false
13097 4907 Acked Read 6 Done Sent Done false
13098 4908 Acked Flush 0 Done Sent Done false
13099 4909 Acked Read 3 Done Sent Done false
13100 4910 Acked Write 6 Done Sent Done false
13101 4911 Acked Write 3 Done Sent Done false
13102 4912 Acked Write 6 Done Sent Done false
13103 4913 Acked Read 7 Done Sent Done false
13104 4914 Acked Read 7 Done Sent Done false
13105 4915 Acked Write 5 Done Sent Done false
13106 4916 Acked Write 5 Done Sent Done false
13107 4917 Acked Flush 0 Done Sent Done false
13108 4918 Acked Read 10 Done Sent Done false
13109 4919 Acked Write 5 Done Sent Done false
13110 4920 Acked Write 6 Done Sent Done false
13111 4921 Acked Write 2 Done Sent Done false
13112 4922 Acked Read 8 Done Sent Done false
13113 STATES DS:0 DS:1 DS:2 TOTAL
13114 Sent 0 301 0 301
13115 Done 301 0 301 602
13116 Skipped 0 0 0 0
13117 Error 0 0 0 0
13118 Last Flush: 4917 4621 4917
13119 Downstairs last five completed: 4621 4620 4619 4618 4617
13120 Upstairs last five completed: 4922 4921 4920 4919 4918
13121 CLIENT: Up:0 ds:301 act:2
13122 [8820] got start action
13123 [8820] I am going to restart
13124 Starting downstairs at port 8820
13125 Make output file at "/tmp/dsc/downstairs-8820.txt"
13126 [8820][1] reports Starting
13127 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1215)
13128 [8820][1] reports Running
13129 ----------------------------------------------------------------
13130 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13131 States: Active Offline Active
13132 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13133 4622 Acked Read 7 Done Sent Done false
13134 4623 Acked Read 6 Done Sent Done false
13135 4624 Acked Read 7 Done Sent Done false
13136 4625 Acked Write 6 Done Sent Done false
13137 4626 Acked Write 3 Done Sent Done false
13138 4627 Acked Write 6 Done Sent Done false
13139 4628 Acked Flush 0 Done Sent Done false
13140 4629 Acked Read 7 Done Sent Done false
13141 4630 Acked Write 4 Done Sent Done false
13142 4631 Acked Write 4 Done Sent Done false
13143 4632 Acked Read 6 Done Sent Done false
13144 4633 Acked Read 6 Done Sent Done false
13145 4634 Acked Read 2 Done Sent Done false
13146 4635 Acked Write 2 Done Sent Done false
13147 4636 Acked Read 6 Done Sent Done false
13148 4637 Acked Write 4 Done Sent Done false
13149 4638 Acked Read 5 Done Sent Done false
13150 4639 Acked Write 10 Done Sent Done false
13151 4640 Acked Read 1 Done Sent Done false
13152 4641 Acked Read 6 Done Sent Done false
13153 4642 Acked Write 3 Done Sent Done false
13154 4643 Acked Write 9 Done Sent Done false
13155 4644 Acked Write 3 Done Sent Done false
13156 4645 Acked Write 2 Done Sent Done false
13157 4646 Acked Write 9 Done Sent Done false
13158 4647 Acked Read 3 Done Sent Done false
13159 4648 Acked Write 6 Done Sent Done false
13160 4649 Acked Read 4 Done Sent Done false
13161 4650 Acked Flush 0 Done Sent Done false
13162 4651 Acked Read 3 Done Sent Done false
13163 4652 Acked Write 10 Done Sent Done false
13164 4653 Acked Flush 0 Done Sent Done false
13165 4654 Acked Read 8 Done Sent Done false
13166 4655 Acked Read 7 Done Sent Done false
13167 4656 Acked Flush 0 Done Sent Done false
13168 4657 Acked Write 10 Done Sent Done false
13169 4658 Acked Read 1 Done Sent Done false
13170 4659 Acked Read 6 Done Sent Done false
13171 4660 Acked Read 3 Done Sent Done false
13172 4661 Acked Write 2 Done Sent Done false
13173 4662 Acked Write 6 Done Sent Done false
13174 4663 Acked Read 9 Done Sent Done false
13175 4664 Acked Flush 0 Done Sent Done false
13176 4665 Acked Write 9 Done Sent Done false
13177 4666 Acked Read 5 Done Sent Done false
13178 4667 Acked Read 8 Done Sent Done false
13179 4668 Acked Read 2 Done Sent Done false
13180 4669 Acked Flush 0 Done Sent Done false
13181 4670 Acked Write 9 Done Sent Done false
13182 4671 Acked Write 7 Done Sent Done false
13183 4672 Acked Write 8 Done Sent Done false
13184 4673 Acked Read 3 Done Sent Done false
13185 4674 Acked Write 3 Done Sent Done false
13186 4675 Acked Read 5 Done Sent Done false
13187 4676 Acked Read 2 Done Sent Done false
13188 4677 Acked Read 8 Done Sent Done false
13189 4678 Acked Write 7 Done Sent Done false
13190 4679 Acked Read 7 Done Sent Done false
13191 4680 Acked Read 3 Done Sent Done false
13192 4681 Acked Write 2 Done Sent Done false
13193 4682 Acked Write 10 Done Sent Done false
13194 4683 Acked Write 9 Done Sent Done false
13195 4684 Acked Write 10 Done Sent Done false
13196 4685 Acked Read 10 Done Sent Done false
13197 4686 Acked Read 8 Done Sent Done false
13198 4687 Acked Write 4 Done Sent Done false
13199 4688 Acked Write 2 Done Sent Done false
13200 4689 Acked Write 6 Done Sent Done false
13201 4690 Acked Write 5 Done Sent Done false
13202 4691 Acked Write 1 Done Sent Done false
13203 4692 Acked Write 8 Done Sent Done false
13204 4693 Acked Write 4 Done Sent Done false
13205 4694 Acked Flush 0 Done Sent Done false
13206 4695 Acked Flush 0 Done Sent Done false
13207 4696 Acked Write 7 Done Sent Done false
13208 4697 Acked Read 9 Done Sent Done false
13209 4698 Acked Write 7 Done Sent Done false
13210 4699 Acked Write 7 Done Sent Done false
13211 4700 Acked Read 10 Done Sent Done false
13212 4701 Acked Flush 0 Done Sent Done false
13213 4702 Acked Read 3 Done Sent Done false
13214 4703 Acked Write 3 Done Sent Done false
13215 4704 Acked Write 1 Done Sent Done false
13216 4705 Acked Read 9 Done Sent Done false
13217 4706 Acked Read 4 Done Sent Done false
13218 4707 Acked Flush 0 Done Sent Done false
13219 4708 Acked Write 1 Done Sent Done false
13220 4709 Acked Flush 0 Done Sent Done false
13221 4710 Acked Read 7 Done Sent Done false
13222 4711 Acked Read 5 Done Sent Done false
13223 4712 Acked Flush 0 Done Sent Done false
13224 4713 Acked Write 5 Done Sent Done false
13225 4714 Acked Flush 0 Done Sent Done false
13226 4715 Acked Write 10 Done Sent Done false
13227 4716 Acked Read 7 Done Sent Done false
13228 4717 Acked Read 1 Done Sent Done false
13229 4718 Acked Read 2 Done Sent Done false
13230 4719 Acked Read 2 Done Sent Done false
13231 4720 Acked Write 6 Done Sent Done false
13232 4721 Acked Read 8 Done Sent Done false
13233 4722 Acked Read 2 Done Sent Done false
13234 4723 Acked Read 2 Done Sent Done false
13235 4724 Acked Read 4 Done Sent Done false
13236 4725 Acked Read 10 Done Sent Done false
13237 4726 Acked Write 7 Done Sent Done false
13238 4727 Acked Read 9 Done Sent Done false
13239 4728 Acked Read 1 Done Sent Done false
13240 4729 Acked Write 4 Done Sent Done false
13241 4730 Acked Read 9 Done Sent Done false
13242 4731 Acked Flush 0 Done Sent Done false
13243 4732 Acked Read 5 Done Sent Done false
13244 4733 Acked Write 8 Done Sent Done false
13245 4734 Acked Read 6 Done Sent Done false
13246 4735 Acked Read 2 Done Sent Done false
13247 4736 Acked Read 8 Done Sent Done false
13248 4737 Acked Write 8 Done Sent Done false
13249 4738 Acked Read 5 Done Sent Done false
13250 4739 Acked Flush 0 Done Sent Done false
13251 4740 Acked Write 3 Done Sent Done false
13252 4741 Acked Write 2 Done Sent Done false
13253 4742 Acked Flush 0 Done Sent Done false
13254 4743 Acked Read 4 Done Sent Done false
13255 4744 Acked Flush 0 Done Sent Done false
13256 4745 Acked Write 6 Done Sent Done false
13257 4746 Acked Write 9 Done Sent Done false
13258 4747 Acked Read 1 Done Sent Done false
13259 4748 Acked Write 6 Done Sent Done false
13260 4749 Acked Read 10 Done Sent Done false
13261 4750 Acked Write 5 Done Sent Done false
13262 4751 Acked Write 5 Done Sent Done false
13263 4752 Acked Flush 0 Done Sent Done false
13264 4753 Acked Read 3 Done Sent Done false
13265 4754 Acked Write 5 Done Sent Done false
13266 4755 Acked Read 9 Done Sent Done false
13267 4756 Acked Write 8 Done Sent Done false
13268 4757 Acked Write 3 Done Sent Done false
13269 4758 Acked Read 3 Done Sent Done false
13270 4759 Acked Read 4 Done Sent Done false
13271 4760 Acked Write 6 Done Sent Done false
13272 4761 Acked Write 4 Done Sent Done false
13273 4762 Acked Read 2 Done Sent Done false
13274 4763 Acked Write 10 Done Sent Done false
13275 4764 Acked Read 4 Done Sent Done false
13276 4765 Acked Read 5 Done Sent Done false
13277 4766 Acked Read 8 Done Sent Done false
13278 4767 Acked Write 10 Done Sent Done false
13279 4768 Acked Read 4 Done Sent Done false
13280 4769 Acked Read 8 Done Sent Done false
13281 4770 Acked Read 1 Done Sent Done false
13282 4771 Acked Read 2 Done Sent Done false
13283 4772 Acked Read 3 Done Sent Done false
13284 4773 Acked Write 9 Done Sent Done false
13285 4774 Acked Read 1 Done Sent Done false
13286 4775 Acked Write 7 Done Sent Done false
13287 4776 Acked Read 4 Done Sent Done false
13288 4777 Acked Flush 0 Done Sent Done false
13289 4778 Acked Write 4 Done Sent Done false
13290 4779 Acked Read 10 Done Sent Done false
13291 4780 Acked Write 3 Done Sent Done false
13292 4781 Acked Read 2 Done Sent Done false
13293 4782 Acked Write 4 Done Sent Done false
13294 4783 Acked Write 4 Done Sent Done false
13295 4784 Acked Write 2 Done Sent Done false
13296 4785 Acked Read 5 Done Sent Done false
13297 4786 Acked Write 1 Done Sent Done false
13298 4787 Acked Read 5 Done Sent Done false
13299 4788 Acked Read 4 Done Sent Done false
13300 4789 Acked Read 4 Done Sent Done false
13301 4790 Acked Write 10 Done Sent Done false
13302 4791 Acked Read 8 Done Sent Done false
13303 4792 Acked Read 4 Done Sent Done false
13304 4793 Acked Flush 0 Done Sent Done false
13305 4794 Acked Write 7 Done Sent Done false
13306 4795 Acked Flush 0 Done Sent Done false
13307 4796 Acked Read 10 Done Sent Done false
13308 4797 Acked Flush 0 Done Sent Done false
13309 4798 Acked Read 8 Done Sent Done false
13310 4799 Acked Read 1 Done Sent Done false
13311 4800 Acked Read 10 Done Sent Done false
13312 4801 Acked Write 10 Done Sent Done false
13313 4802 Acked Write 5 Done Sent Done false
13314 4803 Acked Write 2 Done Sent Done false
13315 4804 Acked Write 5 Done Sent Done false
13316 4805 Acked Flush 0 Done Sent Done false
13317 4806 Acked Write 4 Done Sent Done false
13318 4807 Acked Flush 0 Done Sent Done false
13319 4808 Acked Read 10 Done Sent Done false
13320 4809 Acked Read 5 Done Sent Done false
13321 4810 Acked Write 5 Done Sent Done false
13322 4811 Acked Read 2 Done Sent Done false
13323 4812 Acked Write 5 Done Sent Done false
13324 4813 Acked Read 10 Done Sent Done false
13325 4814 Acked Write 4 Done Sent Done false
13326 4815 Acked Write 1 Done Sent Done false
13327 4816 Acked Read 9 Done Sent Done false
13328 4817 Acked Read 7 Done Sent Done false
13329 4818 Acked Write 2 Done Sent Done false
13330 4819 Acked Read 1 Done Sent Done false
13331 4820 Acked Write 10 Done Sent Done false
13332 4821 Acked Read 3 Done Sent Done false
13333 4822 Acked Read 4 Done Sent Done false
13334 4823 Acked Read 10 Done Sent Done false
13335 4824 Acked Read 10 Done Sent Done false
13336 4825 Acked Write 10 Done Sent Done false
13337 4826 Acked Read 3 Done Sent Done false
13338 4827 Acked Flush 0 Done Sent Done false
13339 4828 Acked Read 1 Done Sent Done false
13340 4829 Acked Write 1 Done Sent Done false
13341 4830 Acked Read 5 Done Sent Done false
13342 4831 Acked Write 10 Done Sent Done false
13343 4832 Acked Read 10 Done Sent Done false
13344 4833 Acked Write 5 Done Sent Done false
13345 4834 Acked Read 8 Done Sent Done false
13346 4835 Acked Write 2 Done Sent Done false
13347 4836 Acked Read 6 Done Sent Done false
13348 4837 Acked Flush 0 Done Sent Done false
13349 4838 Acked Flush 0 Done Sent Done false
13350 4839 Acked Read 1 Done Sent Done false
13351 4840 Acked Read 9 Done Sent Done false
13352 4841 Acked Write 5 Done Sent Done false
13353 4842 Acked Write 1 Done Sent Done false
13354 4843 Acked Write 3 Done Sent Done false
13355 4844 Acked Read 4 Done Sent Done false
13356 4845 Acked Write 6 Done Sent Done false
13357 4846 Acked Read 2 Done Sent Done false
13358 4847 Acked Write 9 Done Sent Done false
13359 4848 Acked Flush 0 Done Sent Done false
13360 4849 Acked Write 10 Done Sent Done false
13361 4850 Acked Write 10 Done Sent Done false
13362 4851 Acked Write 4 Done Sent Done false
13363 4852 Acked Read 10 Done Sent Done false
13364 4853 Acked Read 10 Done Sent Done false
13365 4854 Acked Write 2 Done Sent Done false
13366 4855 Acked Read 10 Done Sent Done false
13367 4856 Acked Write 8 Done Sent Done false
13368 4857 Acked Write 2 Done Sent Done false
13369 4858 Acked Write 6 Done Sent Done false
13370 4859 Acked Read 1 Done Sent Done false
13371 4860 Acked Write 8 Done Sent Done false
13372 4861 Acked Read 10 Done Sent Done false
13373 4862 Acked Flush 0 Done Sent Done false
13374 4863 Acked Write 6 Done Sent Done false
13375 4864 Acked Write 1 Done Sent Done false
13376 4865 Acked Read 7 Done Sent Done false
13377 4866 Acked Read 7 Done Sent Done false
13378 4867 Acked Read 4 Done Sent Done false
13379 4868 Acked Write 6 Done Sent Done false
13380 4869 Acked Write 8 Done Sent Done false
13381 4870 Acked Read 5 Done Sent Done false
13382 4871 Acked Read 8 Done Sent Done false
13383 4872 Acked Write 10 Done Sent Done false
13384 4873 Acked Read 1 Done Sent Done false
13385 4874 Acked Read 6 Done Sent Done false
13386 4875 Acked Read 3 Done Sent Done false
13387 4876 Acked Write 3 Done Sent Done false
13388 4877 Acked Write 7 Done Sent Done false
13389 4878 Acked Write 9 Done Sent Done false
13390 4879 Acked Read 7 Done Sent Done false
13391 4880 Acked Write 10 Done Sent Done false
13392 4881 Acked Read 1 Done Sent Done false
13393 4882 Acked Read 3 Done Sent Done false
13394 4883 Acked Write 8 Done Sent Done false
13395 4884 Acked Read 4 Done Sent Done false
13396 4885 Acked Read 2 Done Sent Done false
13397 4886 Acked Read 3 Done Sent Done false
13398 4887 Acked Write 1 Done Sent Done false
13399 4888 Acked Read 10 Done Sent Done false
13400 4889 Acked Read 6 Done Sent Done false
13401 4890 Acked Read 10 Done Sent Done false
13402 4891 Acked Read 4 Done Sent Done false
13403 4892 Acked Write 8 Done Sent Done false
13404 4893 Acked Read 3 Done Sent Done false
13405 4894 Acked Write 5 Done Sent Done false
13406 4895 Acked Write 1 Done Sent Done false
13407 4896 Acked Read 7 Done Sent Done false
13408 4897 Acked Read 3 Done Sent Done false
13409 4898 Acked Write 5 Done Sent Done false
13410 4899 Acked Write 1 Done Sent Done false
13411 4900 Acked Write 8 Done Sent Done false
13412 4901 Acked Write 5 Done Sent Done false
13413 4902 Acked Write 9 Done Sent Done false
13414 4903 Acked Read 1 Done Sent Done false
13415 4904 Acked Read 8 Done Sent Done false
13416 4905 Acked Read 10 Done Sent Done false
13417 4906 Acked Read 10 Done Sent Done false
13418 4907 Acked Read 6 Done Sent Done false
13419 4908 Acked Flush 0 Done Sent Done false
13420 4909 Acked Read 3 Done Sent Done false
13421 4910 Acked Write 6 Done Sent Done false
13422 4911 Acked Write 3 Done Sent Done false
13423 4912 Acked Write 6 Done Sent Done false
13424 4913 Acked Read 7 Done Sent Done false
13425 4914 Acked Read 7 Done Sent Done false
13426 4915 Acked Write 5 Done Sent Done false
13427 4916 Acked Write 5 Done Sent Done false
13428 4917 Acked Flush 0 Done Sent Done false
13429 4918 Acked Read 10 Done Sent Done false
13430 4919 Acked Write 5 Done Sent Done false
13431 4920 Acked Write 6 Done Sent Done false
13432 4921 Acked Write 2 Done Sent Done false
13433 4922 Acked Read 8 Done Sent Done false
13434 4923 Acked Flush 0 Done Sent Done false
13435 STATES DS:0 DS:1 DS:2 TOTAL
13436 Sent 0 302 0 302
13437 Done 302 0 302 604
13438 Skipped 0 0 0 0
13439 Error 0 0 0 0
13440 Last Flush: 4923 4621 4923
13441 Downstairs last five completed: 4621 4620 4619 4618 4617
13442 Upstairs last five completed: 4923 4922 4921 4920 4919
13443 CLIENT: Up:0 ds:302 act:2
134442025-01-14T16:13:24.860ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
134452025-01-14T16:13:24.860ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:44323) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
134462025-01-14T16:13:24.861ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
134472025-01-14T16:13:24.862ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
134482025-01-14T16:13:24.862ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
134492025-01-14T16:13:24.862ZINFOcrucible: send last flush ID to this DS: 4621 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
134502025-01-14T16:13:24.862ZINFOcrucible: Replied this last flush ID: 4621 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
134512025-01-14T16:13:24.862ZINFOcrucible: [1] client re-new 302 jobs since flush 4621 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
134522025-01-14T16:13:24.862ZINFOcrucible: [1] Replayed 302 jobs since flush: 4621 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
13453 ----------------------------------------------------------------
13454 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
13455 Downstairs last five completed: 4923 4922 4921 4920 4919
13456 Upstairs last five completed: 4923 4922 4921 4920 4919
13457 CLIENT: Up:0 ds:0 act:3
13458 Replay: All jobs finished
13459 Main task has work to do, go find it
13460 got dsc Stop(1)
13461 stop 1
13462 Jan 14 16:13:27.099 INFO request completed, latency_us: [8820] Got stop action so:false kr:false
13463 356, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 87856c27-e18b-41ef-848d-bf08cad4f990, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
13464 Jan 14 16:13:27.100 {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-14T16:13:27.100730548Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
13465 INFO request completed, latency_us: 163, response_code: 200, uri: /state/cid/1, method: GET, req_id: 764963a2-6ab4-4a79-82ea-bce7b62b2008, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-14T16:13:27.100835446Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
13466 1:9998
134672025-01-14T16:13:27.100ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
134682025-01-14T16:13:27.100ZINFOcrucible: client task is exiting = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
13469 [8820] Exited with: ExitStatus(unix_wait_status(9))
13470 [8820][1] reports Exit
13471 Jan 14 16:13:31.103 INFO request completed, latency_us: 196, response_code: 200, uri: /state/cid/1, method: GET, req_id: 73937e17-cf87-408f-90f7-653a13dc01eb, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
13472 001/300 Read block 2256 len 24576
13473 002/300 Read block 1486 len 4096
13474 003/300 Write block 1439 len 32768 data: 5 5 6 5 5 5 5 6
13475 004/300 Write block 3759 len 32768 data: 3 5 5 6 5 5 5 5
13476 005/300 Read block 848 len 4096
13477 006/300 Read block 2098 len 40960
13478 007/300 Read block 1735 len 8192
13479 008/300 Write block 1666 len 36864 data: 2 3 3 4 5 4 3 3 3
13480 009/300 Read block 1569 len 12288
13481 010/300 Write block 4206 len 8192 data: 4 4
13482 011/300 Write block 4207 len 20480 data: 5 3 3 5 5
13483 012/300 Flush
13484 013/300 Read block 3324 len 32768
13485 014/300 Read block 3306 len 8192
13486 015/300 Write block 2703 len 12288 data: 7 4 4
13487 016/300 Flush
13488 017/300 Write block 1847 len 20480 data: 3 3 2 2 2
13489 018/300 Read block 2869 len 20480
13490 019/300 Write block 159 len 32768 data: 3 4 4 4 4 4 5 5
13491 020/300 Write block 512 len 4096 data: 4
13492 021/300 Read block 736 len 12288
13493 022/300 Write block 1495 len 8192 data: 4 3
13494 023/300 Write block 926 len 40960 data: 6 6 7 7 7 4 4 3 4 4
13495 024/300 Flush
13496 025/300 Read block 3452 len 8192
13497 026/300 Read block 1056 len 36864
13498 027/300 Read block 4298 len 24576
13499 028/300 Write block 2814 len 20480 data: 6 5 5 4 4
13500 029/300 Read block 769 len 12288
13501 030/300 Write block 3064 len 20480 data: 4 4 3 3 3
13502 031/300 Read block 3043 len 4096
13503 032/300 Write block 685 len 40960 data: 5 5 5 5 5 4 4 4 4 3
13504 033/300 Write block 4043 len 40960 data: 2 3 3 4 3 3 4 4 2 2
13505 034/300 Read block 2381 len 24576
13506 035/300 Write block 1378 len 24576 data: 4 5 5 7 6 4
13507 036/300 Read block 1560 len 4096
13508 037/300 Read block 3046 len 32768
13509 038/300 Read block 812 len 32768
13510 039/300 Read block 4956 len 16384
13511 040/300 Read block 55 len 8192
13512 041/300 Read block 669 len 40960
13513 042/300 Write block 2076 len 40960 data: 3 3 3 3 3 3 3 3 2 3
13514 043/300 Write block 4355 len 20480 data: 4 4 3 3 2
13515 044/300 Write block 4880 len 32768 data: 2 2 2 4 4 4 4 4
13516 045/300 Flush
13517 046/300 Read block 1870 len 16384
13518 047/300 Write block 3602 len 12288 data: 3 3 3
13519 048/300 Write block 1407 len 20480 data: 3 4 4 3 3
13520 049/300 Write block 2089 len 16384 data: 3 3 3 3
13521 050/300 Write block 148 len 4096 data: 7
13522 051/300 Flush
13523 052/300 Write block 3574 len 28672 data: 3 3 2 2 3 3 4
13524 053/300 Write block 859 len 16384 data: 3 3 3 3
13525 054/300 Flush
13526 055/300 Flush
13527 056/300 Write block 1438 len 20480 data: 4 6 6 7 6
13528 057/300 Flush
13529 058/300 Read block 3905 len 40960
13530 059/300 Write block 3584 len 8192 data: 4 3
13531 060/300 Write block 2199 len 40960 data: 5 4 3 2 2 2 2 2 2 2
13532 061/300 Write block 3364 len 28672 data: 3 4 4 4 4 4 3
13533 062/300 Read block 3297 len 36864
13534 063/300 Read block 3464 len 32768
13535 064/300 Read block 4539 len 32768
13536 065/300 Read block 2003 len 36864
13537 066/300 Read block 2804 len 16384
13538 067/300 Write block 1429 len 40960 data: 2 3 3 5 4 5 5 3 3 5
13539 068/300 Flush
13540 069/300 Read block 818 len 36864
13541 070/300 Read block 26 len 8192
13542 071/300 Write block 4149 len 4096 data: 2
13543 072/300 Write block 68 len 20480 data: 4 3 3 3 4
13544 073/300 Write block 3898 len 40960 data: 4 4 4 2 2 2 2 2 3 3
13545 074/300 Read block 3054 len 28672
13546 075/300 Read block 3373 len 16384
13547 076/300 Write block 195 len 4096 data: 3
13548 077/300 Read block 2728 len 24576
13549 078/300 Flush
13550 079/300 Write block 3870 len 8192 data: 4 3
13551 080/300 Read block 104 len 12288
13552 081/300 Write block 4666 len 16384 data: 8 7 5 4
13553 082/300 Read block 332 len 40960
13554 083/300 Read block 213 len 24576
13555 084/300 Read block 247 len 36864
13556 085/300 Write block 916 len 8192 data: 3 3
13557 086/300 Read block 2061 len 40960
13558 087/300 Flush
13559 088/300 Write block 4592 len 12288 data: 4 4 4
13560 089/300 Write block 4358 len 4096 data: 4
13561 090/300 Write block 95 len 8192 data: 4 3
13562 091/300 Write block 993 len 4096 data: 3
13563 092/300 Read block 2036 len 4096
13564 093/300 Write block 1215 len 40960 data: 2 2 3 4 5 4 4 4 4 5
13565 094/300 Write block 4138 len 40960 data: 6 5 4 4 4 4 3 3 3 3
13566 095/300 Flush
13567 096/300 Write block 1245 len 8192 data: 3 3
13568 097/300 Write block 4214 len 40960 data: 4 3 2 2 4 4 4 4 5 3
13569 098/300 Flush
13570 099/300 Read block 1795 len 40960
13571 100/300 Read block 2454 len 16384
13572 101/300 Read block 4944 len 28672
13573 102/300 Flush
13574 103/300 Read block 4439 len 24576
13575 104/300 Read block 2142 len 8192
13576 105/300 Read block 2515 len 4096
13577 106/300 Read block 2494 len 24576
13578 107/300 Write block 1229 len 20480 data: 2 2 3 3 3
13579 108/300 Read block 2803 len 28672
13580 109/300 Read block 3218 len 32768
13581 110/300 Write block 2006 len 8192 data: 3 3
13582 111/300 Write block 2854 len 36864 data: 3 4 4 4 3 3 3 4 3
13583 112/300 Flush
13584 113/300 Write block 2772 len 24576 data: 4 4 5 6 5 5
13585 114/300 Read block 4840 len 28672
13586 115/300 Read block 1777 len 36864
13587 116/300 Write block 4755 len 28672 data: 3 3 3 4 3 2 3
13588 117/300 Read block 662 len 16384
13589 118/300 Write block 1840 len 36864 data: 3 2 2 2 2 2 3 4 4
13590 119/300 Write block 4970 len 4096 data: 2
13591 120/300 Write block 4518 len 12288 data: 4 3 4
13592 121/300 Read block 232 len 24576
13593 122/300 Write block 4014 len 12288 data: 6 6 5
13594 123/300 Write block 1643 len 24576 data: 2 2 3 6 6 5
13595 124/300 Write block 440 len 12288 data: 4 4 3
13596 125/300 Read block 3889 len 16384
13597 126/300 Write block 2821 len 12288 data: 2 2 2
13598 127/300 Write block 4098 len 36864 data: 4 4 3 4 3 4 4 4 4
13599 128/300 Write block 2502 len 12288 data: 2 2 2
13600 129/300 Read block 2458 len 4096
13601 130/300 Flush
13602 131/300 Flush
13603 132/300 Read block 2423 len 16384
13604 133/300 Read block 1894 len 20480
13605 134/300 Write block 4610 len 24576 data: 2 3 3 2 2 2
13606 135/300 Write block 1033 len 28672 data: 5 4 4 4 4 4 4
13607 136/300 Write block 1543 len 24576 data: 5 3 4 4 4 4
13608 137/300 Write block 2667 len 36864 data: 5 5 3 3 3 3 3 3 3
13609 138/300 Read block 1134 len 16384
13610 139/300 Read block 952 len 20480
13611 140/300 Write block 35 len 32768 data: 3 3 3 3 4 4 4 4
13612 141/300 Read block 4670 len 20480
13613 142/300 Write block 472 len 40960 data: 3 3 3 3 3 3 3 4 5 5
13614 143/300 Read block 331 len 36864
13615 144/300 Write block 2977 len 16384 data: 3 3 3 4
13616 145/300 Read block 898 len 8192
13617 146/300 Read block 3908 len 28672
13618 147/300 Flush
13619 148/300 Read block 3967 len 16384
13620 149/300 Read block 1973 len 36864
13621 150/300 Read block 2795 len 4096
13622 151/300 Read block 1211 len 4096
13623 152/300 Read block 515 len 4096
13624 153/300 Flush
13625 154/300 Read block 1781 len 12288
13626 155/300 Read block 3432 len 16384
13627 156/300 Write block 1425 len 4096 data: 4
13628 157/300 Read block 4455 len 20480
13629 158/300 Write block 2663 len 4096 data: 4
13630 159/300 Write block 3140 len 16384 data: 3 4 5 5
13631 160/300 Write block 3305 len 8192 data: 3 3
13632 161/300 Write block 3384 len 36864 data: 4 4 4 3 4 5 5 6 6
13633 162/300 Read block 4981 len 24576
13634 163/300 Read block 2750 len 16384
13635 164/300 Write block 2332 len 36864 data: 3 5 4 4 4 4 3 3 3
13636 165/300 Write block 3613 len 36864 data: 5 5 4 4 4 4 4 3 5
13637 166/300 Write block 3068 len 28672 data: 4 3 3 2 2 3 4
13638 167/300 Read block 3446 len 8192
13639 168/300 Read block 1071 len 36864
13640 169/300 Read block 3759 len 40960
13641 170/300 Read block 63 len 36864
13642 171/300 Write block 618 len 20480 data: 6 5 4 4 5
13643 172/300 Flush
13644 173/300 Write block 2211 len 8192 data: 3 3
13645 174/300 Write block 3736 len 40960 data: 3 3 3 4 5 5 5 4 4 3
13646 175/300 Read block 3689 len 16384
13647 176/300 Read block 2654 len 28672
13648 177/300 Read block 504 len 4096
13649 178/300 Read block 2854 len 32768
13650 179/300 Flush
13651 180/300 Write block 1871 len 24576 data: 3 3 3 4 3 5
13652 181/300 Flush
13653 182/300 Read block 1486 len 36864
13654 183/300 Write block 3626 len 40960 data: 3 3 3 3 2 4 4 6 7 7
13655 184/300 Read block 2640 len 36864
13656 185/300 Read block 396 len 8192
13657 186/300 Read block 4495 len 20480
13658 187/300 Read block 3319 len 24576
13659 188/300 Read block 3388 len 40960
13660 189/300 Flush
13661 190/300 Read block 172 len 8192
13662 191/300 Read block 3712 len 12288
13663 192/300 Write block 2978 len 28672 data: 4 4 5 4 4 4 3
13664 193/300 Read block 2443 len 28672
13665 194/300 Read block 2286 len 4096
13666 195/300 Read block 1223 len 12288
13667 196/300 Read block 1407 len 36864
13668 197/300 Read block 203 len 36864
13669 198/300 Write block 2399 len 32768 data: 5 5 5 5 4 5 5 3
13670 199/300 Read block 1285 len 8192
13671 200/300 Read block 2845 len 32768
13672 201/300 Flush
13673 202/300 Write block 3074 len 16384 data: 5 4 4 4
13674 203/300 Flush
13675 204/300 Write block 616 len 8192 data: 5 4
13676 205/300 Write block 1449 len 16384 data: 4 5 6 6
13677 206/300 Write block 984 len 16384 data: 3 3 3 3
13678 207/300 Write block 2869 len 24576 data: 3 6 6 6 6 5
13679 208/300 Read block 2336 len 24576
13680 209/300 Read block 2857 len 36864
13681 210/300 Write block 3416 len 32768 data: 5 5 5 6 8 7 6 6
13682 211/300 Read block 2639 len 40960
13683 212/300 Write block 3635 len 12288 data: 8 7 7
13684 213/300 Flush
13685 214/300 Flush
13686 215/300 Read block 3761 len 24576
13687 216/300 Write block 2592 len 28672 data: 2 3 3 3 2 2 2
13688 217/300 Flush
13689 218/300 Read block 2875 len 40960
13690 219/300 Write block 4247 len 20480 data: 5 5 4 4 4
13691 220/300 Write block 3896 len 32768 data: 5 4 5 5 5 3 3 3
13692 221/300 Read block 2453 len 16384
13693 222/300 Read block 2485 len 24576
13694 223/300 Read block 2187 len 16384
13695 224/300 Read block 4902 len 40960
13696 225/300 Read block 4413 len 20480
13697 226/300 Write block 1261 len 4096 data: 3
13698 227/300 Flush
13699 228/300 Read block 1725 len 24576
13700 229/300 Read block 812 len 8192
13701 230/300 Flush
13702 231/300 Read block 3875 len 12288
13703 232/300 Read block 3047 len 8192
13704 233/300 Flush
13705 234/300 Write block 3720 len 40960 data: 3 3 3 2 2 2 2 3 3 3
13706 235/300 Flush
13707 236/300 Write block 4313 len 20480 data: 3 3 4 4 4
13708 237/300 Write block 3002 len 36864 data: 5 5 5 4 4 4 2 2 2
13709 238/300 Write block 2451 len 16384 data: 3 3 4 5
13710 239/300 Read block 1927 len 4096
13711 240/300 Read block 486 len 36864
13712 241/300 Read block 3050 len 8192
13713 242/300 Write block 3780 len 8192 data: 4 3
13714 243/300 Flush
13715 244/300 Read block 3110 len 20480
13716 245/300 Write block 57 len 8192 data: 3 5
13717 246/300 Read block 4512 len 32768
13718 247/300 Read block 257 len 8192
13719 248/300 Flush
13720 249/300 Read block 2560 len 28672
13721 250/300 Write block 2136 len 16384 data: 3 3 4 3
13722 251/300 Flush
13723 252/300 Read block 4811 len 12288
13724 253/300 Flush
13725 254/300 Write block 1836 len 12288 data: 4 4 5
13726 255/300 Read block 441 len 8192
13727 256/300 Write block 4807 len 28672 data: 5 4 3 3 2 2 3
13728 257/300 Read block 3522 len 28672
13729 258/300 Write block 1521 len 20480 data: 2 2 3 4 4
13730 259/300 Write block 4568 len 4096 data: 4
13731 260/300 Read block 1086 len 8192
13732 261/300 Read block 563 len 28672
13733 262/300 Write block 878 len 36864 data: 4 4 4 5 5 6 6 7 6
13734 263/300 Write block 4429 len 32768 data: 3 3 2 3 3 4 4 4
13735 264/300 Read block 3519 len 36864
13736 265/300 Write block 4926 len 32768 data: 3 3 3 3 3 4 4 4
13737 266/300 Flush
13738 267/300 Read block 3949 len 40960
13739 268/300 Write block 4845 len 24576 data: 7 7 6 6 6 6
13740 269/300 Read block 2269 len 36864
13741 270/300 Read block 591 len 32768
13742 271/300 Write block 510 len 32768 data: 4 4 5 5 5 5 5 5
13743 272/300 Read block 3480 len 32768
13744 273/300 Flush
13745 274/300 Write block 3633 len 32768 data: 7 8 9 8 8 6 6 4
13746 275/300 Write block 1895 len 4096 data: 7
13747 276/300 Read block 4379 len 12288
13748 277/300 Read block 2689 len 24576
13749 278/300 Read block 3337 len 28672
13750 279/300 Read block 331 len 16384
13751 280/300 Write block 3603 len 36864 data: 4 4 4 4 3 5 5 5 5
13752 281/300 Read block 425 len 20480
13753 282/300 Write block 3515 len 24576 data: 4 5 6 5 4 4
13754 283/300 Read block 2395 len 24576
13755 284/300 Write block 1431 len 12288 data: 4 6 5
13756 285/300 Write block 2330 len 24576 data: 3 3 4 6 5 5
13757 286/300 Read block 1714 len 20480
13758 287/300 Write block 3203 len 4096 data: 3
13759 288/300 Write block 4664 len 24576 data: 7 8 9 8 6 5
13760 289/300 Read block 2808 len 28672
13761 290/300 Write block 3930 len 24576 data: 3 3 4 4 4 4
13762 291/300 Write block 1246 len 36864 data: 4 3 3 2 2 2 2 2 3
13763 292/300 Flush
13764 293/300 Write block 1172 len 4096 data: 4
13765 294/300 Write block 1748 len 4096 data: 7
13766 295/300 Write block 41 len 24576 data: 5 5 4 3 3 2
13767 296/300 Write block 1994 len 12288 data: 5 6 7
13768 297/300 Write block 4255 len 8192 data: 3 3
13769 298/300 Read block 3501 len 32768
13770 299/300 Read block 1231 len 40960
13771 300/300 Write block 1768 len 8192 data: 4 4
13772 301/300 Read block 4750 len 32768
13773 Main task has work to do, go find it
13774 got dsc Start(1)
13775 start 1
13776 Jan 14 16:13:31.330 INFO request completed, latency_us: 220, response_code: 204, uri: /start/cid/1, method: POST, req_id: 6aafb9e0-e8ab-4620-81f7-10ae35469054, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
13777 Replay: started 1, returned:Ok(())
13778 ----------------------------------------------------------------
13779 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
13780 States: Active Offline Active
13781 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13782 4924 Acked Read 6 Done Sent Done false
13783 4925 Acked Read 1 Done Sent Done false
13784 4926 Acked Write 8 Done Sent Done false
13785 4927 Acked Write 8 Done Sent Done false
13786 4928 Acked Read 1 Done Sent Done false
13787 4929 Acked Read 10 Done Sent Done false
13788 4930 Acked Read 2 Done Sent Done false
13789 4931 Acked Write 9 Done Sent Done false
13790 4932 Acked Read 3 Done Sent Done false
13791 4933 Acked Write 2 Done Sent Done false
13792 4934 Acked Write 5 Done Sent Done false
13793 4935 Acked Flush 0 Done Sent Done false
13794 4936 Acked Read 8 Done Sent Done false
13795 4937 Acked Read 2 Done Sent Done false
13796 4938 Acked Write 3 Done Sent Done false
13797 4939 Acked Flush 0 Done Sent Done false
13798 4940 Acked Write 5 Done Sent Done false
13799 4941 Acked Read 5 Done Sent Done false
13800 4942 Acked Write 8 Done Sent Done false
13801 4943 Acked Write 1 Done Sent Done false
13802 4944 Acked Read 3 Done Sent Done false
13803 4945 Acked Write 2 Done Sent Done false
13804 4946 Acked Write 10 Done Sent Done false
13805 4947 Acked Flush 0 Done Sent Done false
13806 4948 Acked Read 2 Done Sent Done false
13807 4949 Acked Read 9 Done Sent Done false
13808 4950 Acked Read 6 Done Sent Done false
13809 4951 Acked Write 5 Done Sent Done false
13810 4952 Acked Read 3 Done Sent Done false
13811 4953 Acked Write 5 Done Sent Done false
13812 4954 Acked Read 1 Done Sent Done false
13813 4955 Acked Write 10 Done Sent Done false
13814 4956 Acked Write 10 Done Sent Done false
13815 4957 Acked Read 6 Done Sent Done false
13816 4958 Acked Write 6 Done Sent Done false
13817 4959 Acked Read 1 Done Sent Done false
13818 4960 Acked Read 8 Done Sent Done false
13819 4961 Acked Read 8 Done Sent Done false
13820 4962 Acked Read 4 Done Sent Done false
13821 4963 Acked Read 2 Done Sent Done false
13822 4964 Acked Read 10 Done Sent Done false
13823 4965 Acked Write 10 Done Sent Done false
13824 4966 Acked Write 5 Done Sent Done false
13825 4967 Acked Write 8 Done Sent Done false
13826 4968 Acked Flush 0 Done Sent Done false
13827 4969 Acked Read 4 Done Sent Done false
13828 4970 Acked Write 3 Done Sent Done false
13829 4971 Acked Write 5 Done Sent Done false
13830 4972 Acked Write 4 Done Sent Done false
13831 4973 Acked Write 1 Done Sent Done false
13832 4974 Acked Flush 0 Done Sent Done false
13833 4975 Acked Write 7 Done Sent Done false
13834 4976 Acked Write 4 Done Sent Done false
13835 4977 Acked Flush 0 Done Sent Done false
13836 4978 Acked Flush 0 Done Sent Done false
13837 4979 Acked Write 5 Done Sent Done false
13838 4980 Acked Flush 0 Done Sent Done false
13839 4981 Acked Read 10 Done Sent Done false
13840 4982 Acked Write 2 Done Sent Done false
13841 4983 Acked Write 10 Done Sent Done false
13842 4984 Acked Write 7 Done Sent Done false
13843 4985 Acked Read 9 Done Sent Done false
13844 4986 Acked Read 8 Done Sent Done false
13845 4987 Acked Read 8 Done Sent Done false
13846 4988 Acked Read 9 Done Sent Done false
13847 4989 Acked Read 4 Done Sent Done false
13848 4990 Acked Write 10 Done Sent Done false
13849 4991 Acked Flush 0 Done Sent Done false
13850 4992 Acked Read 9 Done Sent Done false
13851 4993 Acked Read 2 Done Sent Done false
13852 4994 Acked Write 1 Done Sent Done false
13853 4995 Acked Write 5 Done Sent Done false
13854 4996 Acked Write 10 Done Sent Done false
13855 4997 Acked Read 7 Done Sent Done false
13856 4998 Acked Read 4 Done Sent Done false
13857 4999 Acked Write 1 Done Sent Done false
13858 5000 Acked Read 6 Done Sent Done false
13859 5001 Acked Flush 0 Done Sent Done false
13860 5002 Acked Write 2 Done Sent Done false
13861 5003 Acked Read 3 Done Sent Done false
13862 5004 Acked Write 4 Done Sent Done false
13863 5005 Acked Read 10 Done Sent Done false
13864 5006 Acked Read 6 Done Sent Done false
13865 5007 Acked Read 9 Done Sent Done false
13866 5008 Acked Write 2 Done Sent Done false
13867 5009 Acked Read 10 Done Sent Done false
13868 5010 Acked Flush 0 Done Sent Done false
13869 5011 Acked Write 3 Done Sent Done false
13870 5012 Acked Write 1 Done Sent Done false
13871 5013 Acked Write 2 Done Sent Done false
13872 5014 Acked Write 1 Done Sent Done false
13873 5015 Acked Read 1 Done Sent Done false
13874 5016 Acked Write 10 Done Sent Done false
13875 5017 Acked Write 10 Done Sent Done false
13876 5018 Acked Flush 0 Done Sent Done false
13877 5019 Acked Write 2 Done Sent Done false
13878 5020 Acked Write 10 Done Sent Done false
13879 5021 Acked Flush 0 Done Sent Done false
13880 5022 Acked Read 10 Done Sent Done false
13881 5023 Acked Read 4 Done Sent Done false
13882 5024 Acked Read 7 Done Sent Done false
13883 5025 Acked Flush 0 Done Sent Done false
13884 5026 Acked Read 6 Done Sent Done false
13885 5027 Acked Read 2 Done Sent Done false
13886 5028 Acked Read 1 Done Sent Done false
13887 5029 Acked Read 6 Done Sent Done false
13888 5030 Acked Write 5 Done Sent Done false
13889 5031 Acked Read 7 Done Sent Done false
13890 5032 Acked Read 8 Done Sent Done false
13891 5033 Acked Write 2 Done Sent Done false
13892 5034 Acked Write 9 Done Sent Done false
13893 5035 Acked Flush 0 Done Sent Done false
13894 5036 Acked Write 6 Done Sent Done false
13895 5037 Acked Read 7 Done Sent Done false
13896 5038 Acked Read 9 Done Sent Done false
13897 5039 Acked Write 7 Done Sent Done false
13898 5040 Acked Read 4 Done Sent Done false
13899 5041 Acked Write 9 Done Sent Done false
13900 5042 Acked Write 1 Done Sent Done false
13901 5043 Acked Write 3 Done Sent Done false
13902 5044 Acked Read 6 Done Sent Done false
13903 5045 Acked Write 3 Done Sent Done false
13904 5046 Acked Write 6 Done Sent Done false
13905 5047 Acked Write 3 Done Sent Done false
13906 5048 Acked Read 4 Done Sent Done false
13907 5049 Acked Write 3 Done Sent Done false
13908 5050 Acked Write 9 Done Sent Done false
13909 5051 Acked Write 3 Done Sent Done false
13910 5052 Acked Read 1 Done Sent Done false
13911 5053 Acked Flush 0 Done Sent Done false
13912 5054 Acked Flush 0 Done Sent Done false
13913 5055 Acked Read 4 Done Sent Done false
13914 5056 Acked Read 5 Done Sent Done false
13915 5057 Acked Write 6 Done Sent Done false
13916 5058 Acked Write 7 Done Sent Done false
13917 5059 Acked Write 6 Done Sent Done false
13918 5060 Acked Write 9 Done Sent Done false
13919 5061 Acked Read 4 Done Sent Done false
13920 5062 Acked Read 5 Done Sent Done false
13921 5063 Acked Write 8 Done Sent Done false
13922 5064 Acked Read 5 Done Sent Done false
13923 5065 Acked Write 10 Done Sent Done false
13924 5066 Acked Read 9 Done Sent Done false
13925 5067 Acked Write 4 Done Sent Done false
13926 5068 Acked Read 2 Done Sent Done false
13927 5069 Acked Read 7 Done Sent Done false
13928 5070 Acked Flush 0 Done Sent Done false
13929 5071 Acked Read 4 Done Sent Done false
13930 5072 Acked Read 9 Done Sent Done false
13931 5073 Acked Read 1 Done Sent Done false
13932 5074 Acked Read 1 Done Sent Done false
13933 5075 Acked Read 1 Done Sent Done false
13934 5076 Acked Flush 0 Done Sent Done false
13935 5077 Acked Read 3 Done Sent Done false
13936 5078 Acked Read 4 Done Sent Done false
13937 5079 Acked Write 1 Done Sent Done false
13938 5080 Acked Read 5 Done Sent Done false
13939 5081 Acked Write 1 Done Sent Done false
13940 5082 Acked Write 4 Done Sent Done false
13941 5083 Acked Write 2 Done Sent Done false
13942 5084 Acked Write 9 Done Sent Done false
13943 5085 Acked Read 6 Done Sent Done false
13944 5086 Acked Read 4 Done Sent Done false
13945 5087 Acked Write 9 Done Sent Done false
13946 5088 Acked Write 9 Done Sent Done false
13947 5089 Acked Write 7 Done Sent Done false
13948 5090 Acked Read 2 Done Sent Done false
13949 5091 Acked Read 9 Done Sent Done false
13950 5092 Acked Read 10 Done Sent Done false
13951 5093 Acked Read 9 Done Sent Done false
13952 5094 Acked Write 5 Done Sent Done false
13953 5095 Acked Flush 0 Done Sent Done false
13954 5096 Acked Write 2 Done Sent Done false
13955 5097 Acked Write 10 Done Sent Done false
13956 5098 Acked Read 4 Done Sent Done false
13957 5099 Acked Read 7 Done Sent Done false
13958 5100 Acked Read 1 Done Sent Done false
13959 5101 Acked Read 8 Done Sent Done false
13960 5102 Acked Flush 0 Done Sent Done false
13961 5103 Acked Write 6 Done Sent Done false
13962 5104 Acked Flush 0 Done Sent Done false
13963 5105 Acked Read 9 Done Sent Done false
13964 5106 Acked Write 10 Done Sent Done false
13965 5107 Acked Read 9 Done Sent Done false
13966 5108 Acked Read 2 Done Sent Done false
13967 5109 Acked Read 5 Done Sent Done false
13968 5110 Acked Read 6 Done Sent Done false
13969 5111 Acked Read 10 Done Sent Done false
13970 5112 Acked Flush 0 Done Sent Done false
13971 5113 Acked Read 2 Done Sent Done false
13972 5114 Acked Read 3 Done Sent Done false
13973 5115 Acked Write 7 Done Sent Done false
13974 5116 Acked Read 7 Done Sent Done false
13975 5117 Acked Read 1 Done Sent Done false
13976 5118 Acked Read 3 Done Sent Done false
13977 5119 Acked Read 9 Done Sent Done false
13978 5120 Acked Read 9 Done Sent Done false
13979 5121 Acked Write 8 Done Sent Done false
13980 5122 Acked Read 2 Done Sent Done false
13981 5123 Acked Read 8 Done Sent Done false
13982 5124 Acked Flush 0 Done Sent Done false
13983 5125 Acked Write 4 Done Sent Done false
13984 5126 Acked Flush 0 Done Sent Done false
13985 5127 Acked Write 2 Done Sent Done false
13986 5128 Acked Write 4 Done Sent Done false
13987 5129 Acked Write 4 Done Sent Done false
13988 5130 Acked Write 6 Done Sent Done false
13989 5131 Acked Read 6 Done Sent Done false
13990 5132 Acked Read 9 Done Sent Done false
13991 5133 Acked Write 8 Done Sent Done false
13992 5134 Acked Read 10 Done Sent Done false
13993 5135 Acked Write 3 Done Sent Done false
13994 5136 Acked Flush 0 Done Sent Done false
13995 5137 Acked Flush 0 Done Sent Done false
13996 5138 Acked Read 6 Done Sent Done false
13997 5139 Acked Write 7 Done Sent Done false
13998 5140 Acked Flush 0 Done Sent Done false
13999 5141 Acked Read 10 Done Sent Done false
14000 5142 Acked Write 5 Done Sent Done false
14001 5143 Acked Write 8 Done Sent Done false
14002 5144 Acked Read 4 Done Sent Done false
14003 5145 Acked Read 6 Done Sent Done false
14004 5146 Acked Read 4 Done Sent Done false
14005 5147 Acked Read 10 Done Sent Done false
14006 5148 Acked Read 5 Done Sent Done false
14007 5149 Acked Write 1 Done Sent Done false
14008 5150 Acked Flush 0 Done Sent Done false
14009 5151 Acked Read 6 Done Sent Done false
14010 5152 Acked Read 2 Done Sent Done false
14011 5153 Acked Flush 0 Done Sent Done false
14012 5154 Acked Read 3 Done Sent Done false
14013 5155 Acked Read 2 Done Sent Done false
14014 5156 Acked Flush 0 Done Sent Done false
14015 5157 Acked Write 10 Done Sent Done false
14016 5158 Acked Flush 0 Done Sent Done false
14017 5159 Acked Write 5 Done Sent Done false
14018 5160 Acked Write 9 Done Sent Done false
14019 5161 Acked Write 4 Done Sent Done false
14020 5162 Acked Read 1 Done Sent Done false
14021 5163 Acked Read 9 Done Sent Done false
14022 5164 Acked Read 2 Done Sent Done false
14023 5165 Acked Write 2 Done Sent Done false
14024 5166 Acked Flush 0 Done Sent Done false
14025 5167 Acked Read 5 Done Sent Done false
14026 5168 Acked Write 2 Done Sent Done false
14027 5169 Acked Read 8 Done Sent Done false
14028 5170 Acked Read 2 Done Sent Done false
14029 5171 Acked Flush 0 Done Sent Done false
14030 5172 Acked Read 7 Done Sent Done false
14031 5173 Acked Write 4 Done Sent Done false
14032 5174 Acked Flush 0 Done Sent Done false
14033 5175 Acked Read 3 Done Sent Done false
14034 5176 Acked Flush 0 Done Sent Done false
14035 5177 Acked Write 3 Done Sent Done false
14036 5178 Acked Read 2 Done Sent Done false
14037 5179 Acked Write 7 Done Sent Done false
14038 5180 Acked Read 7 Done Sent Done false
14039 5181 Acked Write 5 Done Sent Done false
14040 5182 Acked Write 1 Done Sent Done false
14041 5183 Acked Read 2 Done Sent Done false
14042 5184 Acked Read 7 Done Sent Done false
14043 5185 Acked Write 9 Done Sent Done false
14044 5186 Acked Write 8 Done Sent Done false
14045 5187 Acked Read 9 Done Sent Done false
14046 5188 Acked Write 8 Done Sent Done false
14047 5189 Acked Flush 0 Done Sent Done false
14048 5190 Acked Read 10 Done Sent Done false
14049 5191 Acked Write 6 Done Sent Done false
14050 5192 Acked Read 9 Done Sent Done false
14051 5193 Acked Read 8 Done Sent Done false
14052 5194 Acked Write 8 Done Sent Done false
14053 5195 Acked Read 8 Done Sent Done false
14054 5196 Acked Flush 0 Done Sent Done false
14055 5197 Acked Write 8 Done Sent Done false
14056 5198 Acked Write 1 Done Sent Done false
14057 5199 Acked Read 3 Done Sent Done false
14058 5200 Acked Read 6 Done Sent Done false
14059 5201 Acked Read 7 Done Sent Done false
14060 5202 Acked Read 4 Done Sent Done false
14061 5203 Acked Write 9 Done Sent Done false
14062 5204 Acked Read 5 Done Sent Done false
14063 5205 Acked Write 6 Done Sent Done false
14064 5206 Acked Read 6 Done Sent Done false
14065 5207 Acked Write 3 Done Sent Done false
14066 5208 Acked Write 6 Done Sent Done false
14067 5209 Acked Read 5 Done Sent Done false
14068 5210 Acked Write 1 Done Sent Done false
14069 5211 Acked Write 6 Done Sent Done false
14070 5212 Acked Read 7 Done Sent Done false
14071 5213 Acked Write 6 Done Sent Done false
14072 5214 Acked Write 9 Done Sent Done false
14073 5215 Acked Flush 0 Done Sent Done false
14074 5216 Acked Write 1 Done Sent Done false
14075 5217 Acked Write 1 Done Sent Done false
14076 5218 Acked Write 6 Done Sent Done false
14077 5219 Acked Write 3 Done Sent Done false
14078 5220 Acked Write 2 Done Sent Done false
14079 5221 Acked Read 8 Done Sent Done false
14080 5222 Acked Read 10 Done Sent Done false
14081 5223 Acked Write 2 Done Sent Done false
14082 5224 Acked Read 8 Done Sent Done false
14083 STATES DS:0 DS:1 DS:2 TOTAL
14084 Sent 0 301 0 301
14085 Done 301 0 301 602
14086 Skipped 0 0 0 0
14087 Error 0 0 0 0
14088 Last Flush: 5215 4923 5215
14089 Downstairs last five completed: 4923 4922 4921 4920 4919
14090 Upstairs last five completed: 5224 5223 5222 5221 5220
14091 CLIENT: Up:0 ds:301 act:2
14092 [8820] got start action
14093 [8820] I am going to restart
14094 Starting downstairs at port 8820
14095 Make output file at "/tmp/dsc/downstairs-8820.txt"
14096 [8820][1] reports Starting
14097 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1216)
14098 [8820][1] reports Running
14099 ----------------------------------------------------------------
14100 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14101 States: Active Offline Active
14102 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14103 4924 Acked Read 6 Done Sent Done false
14104 4925 Acked Read 1 Done Sent Done false
14105 4926 Acked Write 8 Done Sent Done false
14106 4927 Acked Write 8 Done Sent Done false
14107 4928 Acked Read 1 Done Sent Done false
14108 4929 Acked Read 10 Done Sent Done false
14109 4930 Acked Read 2 Done Sent Done false
14110 4931 Acked Write 9 Done Sent Done false
14111 4932 Acked Read 3 Done Sent Done false
14112 4933 Acked Write 2 Done Sent Done false
14113 4934 Acked Write 5 Done Sent Done false
14114 4935 Acked Flush 0 Done Sent Done false
14115 4936 Acked Read 8 Done Sent Done false
14116 4937 Acked Read 2 Done Sent Done false
14117 4938 Acked Write 3 Done Sent Done false
14118 4939 Acked Flush 0 Done Sent Done false
14119 4940 Acked Write 5 Done Sent Done false
14120 4941 Acked Read 5 Done Sent Done false
14121 4942 Acked Write 8 Done Sent Done false
14122 4943 Acked Write 1 Done Sent Done false
14123 4944 Acked Read 3 Done Sent Done false
14124 4945 Acked Write 2 Done Sent Done false
14125 4946 Acked Write 10 Done Sent Done false
14126 4947 Acked Flush 0 Done Sent Done false
14127 4948 Acked Read 2 Done Sent Done false
14128 4949 Acked Read 9 Done Sent Done false
14129 4950 Acked Read 6 Done Sent Done false
14130 4951 Acked Write 5 Done Sent Done false
14131 4952 Acked Read 3 Done Sent Done false
14132 4953 Acked Write 5 Done Sent Done false
14133 4954 Acked Read 1 Done Sent Done false
14134 4955 Acked Write 10 Done Sent Done false
14135 4956 Acked Write 10 Done Sent Done false
14136 4957 Acked Read 6 Done Sent Done false
14137 4958 Acked Write 6 Done Sent Done false
14138 4959 Acked Read 1 Done Sent Done false
14139 4960 Acked Read 8 Done Sent Done false
14140 4961 Acked Read 8 Done Sent Done false
14141 4962 Acked Read 4 Done Sent Done false
14142 4963 Acked Read 2 Done Sent Done false
14143 4964 Acked Read 10 Done Sent Done false
14144 4965 Acked Write 10 Done Sent Done false
14145 4966 Acked Write 5 Done Sent Done false
14146 4967 Acked Write 8 Done Sent Done false
14147 4968 Acked Flush 0 Done Sent Done false
14148 4969 Acked Read 4 Done Sent Done false
14149 4970 Acked Write 3 Done Sent Done false
14150 4971 Acked Write 5 Done Sent Done false
14151 4972 Acked Write 4 Done Sent Done false
14152 4973 Acked Write 1 Done Sent Done false
14153 4974 Acked Flush 0 Done Sent Done false
14154 4975 Acked Write 7 Done Sent Done false
14155 4976 Acked Write 4 Done Sent Done false
14156 4977 Acked Flush 0 Done Sent Done false
14157 4978 Acked Flush 0 Done Sent Done false
14158 4979 Acked Write 5 Done Sent Done false
14159 4980 Acked Flush 0 Done Sent Done false
14160 4981 Acked Read 10 Done Sent Done false
14161 4982 Acked Write 2 Done Sent Done false
14162 4983 Acked Write 10 Done Sent Done false
14163 4984 Acked Write 7 Done Sent Done false
14164 4985 Acked Read 9 Done Sent Done false
14165 4986 Acked Read 8 Done Sent Done false
14166 4987 Acked Read 8 Done Sent Done false
14167 4988 Acked Read 9 Done Sent Done false
14168 4989 Acked Read 4 Done Sent Done false
14169 4990 Acked Write 10 Done Sent Done false
14170 4991 Acked Flush 0 Done Sent Done false
14171 4992 Acked Read 9 Done Sent Done false
14172 4993 Acked Read 2 Done Sent Done false
14173 4994 Acked Write 1 Done Sent Done false
14174 4995 Acked Write 5 Done Sent Done false
14175 4996 Acked Write 10 Done Sent Done false
14176 4997 Acked Read 7 Done Sent Done false
14177 4998 Acked Read 4 Done Sent Done false
14178 4999 Acked Write 1 Done Sent Done false
14179 5000 Acked Read 6 Done Sent Done false
14180 5001 Acked Flush 0 Done Sent Done false
14181 5002 Acked Write 2 Done Sent Done false
14182 5003 Acked Read 3 Done Sent Done false
14183 5004 Acked Write 4 Done Sent Done false
14184 5005 Acked Read 10 Done Sent Done false
14185 5006 Acked Read 6 Done Sent Done false
14186 5007 Acked Read 9 Done Sent Done false
14187 5008 Acked Write 2 Done Sent Done false
14188 5009 Acked Read 10 Done Sent Done false
14189 5010 Acked Flush 0 Done Sent Done false
14190 5011 Acked Write 3 Done Sent Done false
14191 5012 Acked Write 1 Done Sent Done false
14192 5013 Acked Write 2 Done Sent Done false
14193 5014 Acked Write 1 Done Sent Done false
14194 5015 Acked Read 1 Done Sent Done false
14195 5016 Acked Write 10 Done Sent Done false
14196 5017 Acked Write 10 Done Sent Done false
14197 5018 Acked Flush 0 Done Sent Done false
14198 5019 Acked Write 2 Done Sent Done false
14199 5020 Acked Write 10 Done Sent Done false
14200 5021 Acked Flush 0 Done Sent Done false
14201 5022 Acked Read 10 Done Sent Done false
14202 5023 Acked Read 4 Done Sent Done false
14203 5024 Acked Read 7 Done Sent Done false
14204 5025 Acked Flush 0 Done Sent Done false
14205 5026 Acked Read 6 Done Sent Done false
14206 5027 Acked Read 2 Done Sent Done false
14207 5028 Acked Read 1 Done Sent Done false
14208 5029 Acked Read 6 Done Sent Done false
14209 5030 Acked Write 5 Done Sent Done false
14210 5031 Acked Read 7 Done Sent Done false
14211 5032 Acked Read 8 Done Sent Done false
14212 5033 Acked Write 2 Done Sent Done false
14213 5034 Acked Write 9 Done Sent Done false
14214 5035 Acked Flush 0 Done Sent Done false
14215 5036 Acked Write 6 Done Sent Done false
14216 5037 Acked Read 7 Done Sent Done false
14217 5038 Acked Read 9 Done Sent Done false
14218 5039 Acked Write 7 Done Sent Done false
14219 5040 Acked Read 4 Done Sent Done false
14220 5041 Acked Write 9 Done Sent Done false
14221 5042 Acked Write 1 Done Sent Done false
14222 5043 Acked Write 3 Done Sent Done false
14223 5044 Acked Read 6 Done Sent Done false
14224 5045 Acked Write 3 Done Sent Done false
14225 5046 Acked Write 6 Done Sent Done false
14226 5047 Acked Write 3 Done Sent Done false
14227 5048 Acked Read 4 Done Sent Done false
14228 5049 Acked Write 3 Done Sent Done false
14229 5050 Acked Write 9 Done Sent Done false
14230 5051 Acked Write 3 Done Sent Done false
14231 5052 Acked Read 1 Done Sent Done false
14232 5053 Acked Flush 0 Done Sent Done false
14233 5054 Acked Flush 0 Done Sent Done false
14234 5055 Acked Read 4 Done Sent Done false
14235 5056 Acked Read 5 Done Sent Done false
14236 5057 Acked Write 6 Done Sent Done false
14237 5058 Acked Write 7 Done Sent Done false
14238 5059 Acked Write 6 Done Sent Done false
14239 5060 Acked Write 9 Done Sent Done false
14240 5061 Acked Read 4 Done Sent Done false
14241 5062 Acked Read 5 Done Sent Done false
14242 5063 Acked Write 8 Done Sent Done false
14243 5064 Acked Read 5 Done Sent Done false
14244 5065 Acked Write 10 Done Sent Done false
14245 5066 Acked Read 9 Done Sent Done false
14246 5067 Acked Write 4 Done Sent Done false
14247 5068 Acked Read 2 Done Sent Done false
14248 5069 Acked Read 7 Done Sent Done false
14249 5070 Acked Flush 0 Done Sent Done false
14250 5071 Acked Read 4 Done Sent Done false
14251 5072 Acked Read 9 Done Sent Done false
14252 5073 Acked Read 1 Done Sent Done false
14253 5074 Acked Read 1 Done Sent Done false
14254 5075 Acked Read 1 Done Sent Done false
14255 5076 Acked Flush 0 Done Sent Done false
14256 5077 Acked Read 3 Done Sent Done false
14257 5078 Acked Read 4 Done Sent Done false
14258 5079 Acked Write 1 Done Sent Done false
14259 5080 Acked Read 5 Done Sent Done false
14260 5081 Acked Write 1 Done Sent Done false
14261 5082 Acked Write 4 Done Sent Done false
14262 5083 Acked Write 2 Done Sent Done false
14263 5084 Acked Write 9 Done Sent Done false
14264 5085 Acked Read 6 Done Sent Done false
14265 5086 Acked Read 4 Done Sent Done false
14266 5087 Acked Write 9 Done Sent Done false
14267 5088 Acked Write 9 Done Sent Done false
14268 5089 Acked Write 7 Done Sent Done false
14269 5090 Acked Read 2 Done Sent Done false
14270 5091 Acked Read 9 Done Sent Done false
14271 5092 Acked Read 10 Done Sent Done false
14272 5093 Acked Read 9 Done Sent Done false
14273 5094 Acked Write 5 Done Sent Done false
14274 5095 Acked Flush 0 Done Sent Done false
14275 5096 Acked Write 2 Done Sent Done false
14276 5097 Acked Write 10 Done Sent Done false
14277 5098 Acked Read 4 Done Sent Done false
14278 5099 Acked Read 7 Done Sent Done false
14279 5100 Acked Read 1 Done Sent Done false
14280 5101 Acked Read 8 Done Sent Done false
14281 5102 Acked Flush 0 Done Sent Done false
14282 5103 Acked Write 6 Done Sent Done false
14283 5104 Acked Flush 0 Done Sent Done false
14284 5105 Acked Read 9 Done Sent Done false
14285 5106 Acked Write 10 Done Sent Done false
14286 5107 Acked Read 9 Done Sent Done false
14287 5108 Acked Read 2 Done Sent Done false
14288 5109 Acked Read 5 Done Sent Done false
14289 5110 Acked Read 6 Done Sent Done false
14290 5111 Acked Read 10 Done Sent Done false
14291 5112 Acked Flush 0 Done Sent Done false
14292 5113 Acked Read 2 Done Sent Done false
14293 5114 Acked Read 3 Done Sent Done false
14294 5115 Acked Write 7 Done Sent Done false
14295 5116 Acked Read 7 Done Sent Done false
14296 5117 Acked Read 1 Done Sent Done false
14297 5118 Acked Read 3 Done Sent Done false
14298 5119 Acked Read 9 Done Sent Done false
14299 5120 Acked Read 9 Done Sent Done false
14300 5121 Acked Write 8 Done Sent Done false
14301 5122 Acked Read 2 Done Sent Done false
14302 5123 Acked Read 8 Done Sent Done false
14303 5124 Acked Flush 0 Done Sent Done false
14304 5125 Acked Write 4 Done Sent Done false
14305 5126 Acked Flush 0 Done Sent Done false
14306 5127 Acked Write 2 Done Sent Done false
14307 5128 Acked Write 4 Done Sent Done false
14308 5129 Acked Write 4 Done Sent Done false
14309 5130 Acked Write 6 Done Sent Done false
14310 5131 Acked Read 6 Done Sent Done false
14311 5132 Acked Read 9 Done Sent Done false
14312 5133 Acked Write 8 Done Sent Done false
14313 5134 Acked Read 10 Done Sent Done false
14314 5135 Acked Write 3 Done Sent Done false
14315 5136 Acked Flush 0 Done Sent Done false
14316 5137 Acked Flush 0 Done Sent Done false
14317 5138 Acked Read 6 Done Sent Done false
14318 5139 Acked Write 7 Done Sent Done false
14319 5140 Acked Flush 0 Done Sent Done false
14320 5141 Acked Read 10 Done Sent Done false
14321 5142 Acked Write 5 Done Sent Done false
14322 5143 Acked Write 8 Done Sent Done false
14323 5144 Acked Read 4 Done Sent Done false
14324 5145 Acked Read 6 Done Sent Done false
14325 5146 Acked Read 4 Done Sent Done false
14326 5147 Acked Read 10 Done Sent Done false
14327 5148 Acked Read 5 Done Sent Done false
14328 5149 Acked Write 1 Done Sent Done false
14329 5150 Acked Flush 0 Done Sent Done false
14330 5151 Acked Read 6 Done Sent Done false
14331 5152 Acked Read 2 Done Sent Done false
14332 5153 Acked Flush 0 Done Sent Done false
14333 5154 Acked Read 3 Done Sent Done false
14334 5155 Acked Read 2 Done Sent Done false
14335 5156 Acked Flush 0 Done Sent Done false
14336 5157 Acked Write 10 Done Sent Done false
14337 5158 Acked Flush 0 Done Sent Done false
14338 5159 Acked Write 5 Done Sent Done false
14339 5160 Acked Write 9 Done Sent Done false
14340 5161 Acked Write 4 Done Sent Done false
14341 5162 Acked Read 1 Done Sent Done false
14342 5163 Acked Read 9 Done Sent Done false
14343 5164 Acked Read 2 Done Sent Done false
14344 5165 Acked Write 2 Done Sent Done false
14345 5166 Acked Flush 0 Done Sent Done false
14346 5167 Acked Read 5 Done Sent Done false
14347 5168 Acked Write 2 Done Sent Done false
14348 5169 Acked Read 8 Done Sent Done false
14349 5170 Acked Read 2 Done Sent Done false
14350 5171 Acked Flush 0 Done Sent Done false
14351 5172 Acked Read 7 Done Sent Done false
14352 5173 Acked Write 4 Done Sent Done false
14353 5174 Acked Flush 0 Done Sent Done false
14354 5175 Acked Read 3 Done Sent Done false
14355 5176 Acked Flush 0 Done Sent Done false
14356 5177 Acked Write 3 Done Sent Done false
14357 5178 Acked Read 2 Done Sent Done false
14358 5179 Acked Write 7 Done Sent Done false
14359 5180 Acked Read 7 Done Sent Done false
14360 5181 Acked Write 5 Done Sent Done false
14361 5182 Acked Write 1 Done Sent Done false
14362 5183 Acked Read 2 Done Sent Done false
14363 5184 Acked Read 7 Done Sent Done false
14364 5185 Acked Write 9 Done Sent Done false
14365 5186 Acked Write 8 Done Sent Done false
14366 5187 Acked Read 9 Done Sent Done false
14367 5188 Acked Write 8 Done Sent Done false
14368 5189 Acked Flush 0 Done Sent Done false
14369 5190 Acked Read 10 Done Sent Done false
14370 5191 Acked Write 6 Done Sent Done false
14371 5192 Acked Read 9 Done Sent Done false
14372 5193 Acked Read 8 Done Sent Done false
14373 5194 Acked Write 8 Done Sent Done false
14374 5195 Acked Read 8 Done Sent Done false
14375 5196 Acked Flush 0 Done Sent Done false
14376 5197 Acked Write 8 Done Sent Done false
14377 5198 Acked Write 1 Done Sent Done false
14378 5199 Acked Read 3 Done Sent Done false
14379 5200 Acked Read 6 Done Sent Done false
14380 5201 Acked Read 7 Done Sent Done false
14381 5202 Acked Read 4 Done Sent Done false
14382 5203 Acked Write 9 Done Sent Done false
14383 5204 Acked Read 5 Done Sent Done false
14384 5205 Acked Write 6 Done Sent Done false
14385 5206 Acked Read 6 Done Sent Done false
14386 5207 Acked Write 3 Done Sent Done false
14387 5208 Acked Write 6 Done Sent Done false
14388 5209 Acked Read 5 Done Sent Done false
14389 5210 Acked Write 1 Done Sent Done false
14390 5211 Acked Write 6 Done Sent Done false
14391 5212 Acked Read 7 Done Sent Done false
14392 5213 Acked Write 6 Done Sent Done false
14393 5214 Acked Write 9 Done Sent Done false
14394 5215 Acked Flush 0 Done Sent Done false
14395 5216 Acked Write 1 Done Sent Done false
14396 5217 Acked Write 1 Done Sent Done false
14397 5218 Acked Write 6 Done Sent Done false
14398 5219 Acked Write 3 Done Sent Done false
14399 5220 Acked Write 2 Done Sent Done false
14400 5221 Acked Read 8 Done Sent Done false
14401 5222 Acked Read 10 Done Sent Done false
14402 5223 Acked Write 2 Done Sent Done false
14403 5224 Acked Read 8 Done Sent Done false
14404 5225 Acked Flush 0 Done Sent Done false
14405 STATES DS:0 DS:1 DS:2 TOTAL
14406 Sent 0 302 0 302
14407 Done 302 0 302 604
14408 Skipped 0 0 0 0
14409 Error 0 0 0 0
14410 Last Flush: 5225 4923 5225
14411 Downstairs last five completed: 4923 4922 4921 4920 4919
14412 Upstairs last five completed: 5225 5224 5223 5222 5221
14413 CLIENT: Up:0 ds:302 act:2
144142025-01-14T16:13:37.102ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
144152025-01-14T16:13:37.102ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:63917) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
144162025-01-14T16:13:37.103ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
144172025-01-14T16:13:37.104ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
144182025-01-14T16:13:37.104ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
144192025-01-14T16:13:37.104ZINFOcrucible: send last flush ID to this DS: 4923 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
144202025-01-14T16:13:37.104ZINFOcrucible: Replied this last flush ID: 4923 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
144212025-01-14T16:13:37.104ZINFOcrucible: [1] client re-new 302 jobs since flush 4923 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
144222025-01-14T16:13:37.104ZINFOcrucible: [1] Replayed 302 jobs since flush: 4923 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
14423 ----------------------------------------------------------------
14424 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
14425 Downstairs last five completed: 5225 5224 5223 5222 5221
14426 Upstairs last five completed: 5225 5224 5223 5222 5221
14427 CLIENT: Up:0 ds:0 act:3
14428 Replay: All jobs finished
14429 Main task has work to do, go find it
14430 got dsc Stop(0)
14431 stop 0
14432 Jan 14 16:13:39.335 INFO request completed, latency_us: 321, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 36652fd4-be94-4394-80b3-1f5097dc2546, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
14433 [8810] Got stop action so:false kr:false
14434 Jan 14 16:13:39.336 INFO request completed, latency_us: 211, response_code: 200, uri: /state/cid/0, method: GET, req_id: 16082ce3-465e-4710-a50c-75c47500dd30, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
144352025-01-14T16:13:39.336ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
144362025-01-14T16:13:39.336ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
144372025-01-14T16:13:39.336ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
144382025-01-14T16:13:39.336ZINFOcrucible: client task is exiting = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
14439 [8810] Exited with: ExitStatus(unix_wait_status(9))
14440 [8810][0] reports Exit
14441 Jan 14 16:13:43.339 INFO request completed, latency_us: 197, response_code: 200, uri: /state/cid/0, method: GET, req_id: d07289de-e358-423d-973a-03c146c01ae7, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
14442 001/300 Write block 675 len 20480 data: 6 5 4 3 4
14443 002/300 Write block 1268 len 8192 data: 3 4
14444 003/300 Write block 2612 len 24576 data: 4 6 5 5 5 6
14445 004/300 Flush
14446 005/300 Read block 528 len 16384
14447 006/300 Read block 4878 len 24576
14448 007/300 Read block 2394 len 12288
14449 008/300 Read block 3789 len 24576
14450 009/300 Flush
14451 010/300 Read block 4696 len 32768
14452 011/300 Read block 4907 len 12288
14453 012/300 Write block 294 len 32768 data: 4 5 3 3 3 3 3 3
14454 013/300 Write block 2426 len 32768 data: 3 3 3 3 2 3 4 4
14455 014/300 Write block 1624 len 12288 data: 3 4 4
14456 015/300 Write block 3113 len 8192 data: 2 2
14457 016/300 Write block 1001 len 32768 data: 5 5 6 5 5 5 5 4
14458 017/300 Write block 303 len 24576 data: 3 3 3 4 6 6
14459 018/300 Read block 1628 len 16384
14460 019/300 Read block 116 len 16384
14461 020/300 Read block 2341 len 4096
14462 021/300 Read block 2763 len 24576
14463 022/300 Write block 1152 len 16384 data: 6 4 4 4
14464 023/300 Read block 193 len 12288
14465 024/300 Read block 2318 len 40960
14466 025/300 Read block 289 len 16384
14467 026/300 Write block 4231 len 8192 data: 2 2
14468 027/300 Write block 3685 len 12288 data: 4 3 3
14469 028/300 Read block 2602 len 40960
14470 029/300 Flush
14471 030/300 Read block 4173 len 16384
14472 031/300 Read block 4600 len 16384
14473 032/300 Read block 702 len 12288
14474 033/300 Flush
14475 034/300 Write block 1600 len 12288 data: 3 3 3
14476 035/300 Read block 281 len 20480
14477 036/300 Write block 859 len 12288 data: 4 4 4
14478 037/300 Read block 3358 len 12288
14479 038/300 Read block 2899 len 8192
14480 039/300 Read block 999 len 28672
14481 040/300 Read block 2529 len 24576
14482 041/300 Read block 85 len 12288
14483 042/300 Flush
14484 043/300 Write block 1528 len 28672 data: 4 4 5 6 7 7 6
14485 044/300 Read block 4455 len 28672
14486 045/300 Write block 4842 len 24576 data: 6 6 8 8 8 7
14487 046/300 Read block 1730 len 16384
14488 047/300 Flush
14489 048/300 Write block 4943 len 32768 data: 4 5 4 3 3 3 3 3
14490 049/300 Read block 1004 len 28672
14491 050/300 Write block 3549 len 20480 data: 3 3 3 3 3
14492 051/300 Write block 701 len 40960 data: 5 4 4 5 5 5 4 4 4 3
14493 052/300 Write block 1835 len 28672 data: 5 5 5 6 4 4 3
14494 053/300 Write block 1973 len 40960 data: 2 2 2 2 2 3 3 3 3 3
14495 054/300 Read block 3378 len 12288
14496 055/300 Write block 3061 len 4096 data: 3
14497 056/300 Read block 2444 len 20480
14498 057/300 Read block 1417 len 16384
14499 058/300 Read block 950 len 28672
14500 059/300 Read block 1793 len 12288
14501 060/300 Read block 1192 len 16384
14502 061/300 Write block 132 len 8192 data: 6 6
14503 062/300 Write block 442 len 32768 data: 4 4 4 4 3 3 4 6
14504 063/300 Flush
14505 064/300 Write block 2257 len 24576 data: 2 2 3 3 3 4
14506 065/300 Read block 3492 len 24576
14507 066/300 Read block 2204 len 24576
14508 067/300 Flush
14509 068/300 Write block 1841 len 36864 data: 4 3 3 3 3 4 5 5 3
14510 069/300 Read block 2890 len 32768
14511 070/300 Read block 1761 len 28672
14512 071/300 Read block 3201 len 24576
14513 072/300 Write block 470 len 20480 data: 4 4 4 4 4
14514 073/300 Write block 660 len 36864 data: 3 3 3 3 2 3 3 3 3
14515 074/300 Read block 367 len 12288
14516 075/300 Read block 376 len 24576
14517 076/300 Read block 2989 len 36864
14518 077/300 Write block 13 len 16384 data: 2 4 4 4
14519 078/300 Read block 2087 len 36864
14520 079/300 Write block 1558 len 4096 data: 6
14521 080/300 Read block 1725 len 32768
14522 081/300 Read block 2356 len 32768
14523 082/300 Write block 4916 len 24576 data: 3 3 3 3 2 2
14524 083/300 Read block 1879 len 36864
14525 084/300 Write block 1717 len 12288 data: 4 4 3
14526 085/300 Read block 1205 len 28672
14527 086/300 Write block 3025 len 20480 data: 5 5 5 6 7
14528 087/300 Read block 458 len 16384
14529 088/300 Write block 3423 len 16384 data: 7 7 6 6
14530 089/300 Read block 4045 len 8192
14531 090/300 Read block 3584 len 36864
14532 091/300 Flush
14533 092/300 Write block 1375 len 36864 data: 4 4 4 5 6 6 8 7 5
14534 093/300 Flush
14535 094/300 Write block 4225 len 8192 data: 3 2
14536 095/300 Write block 4335 len 4096 data: 4
14537 096/300 Flush
14538 097/300 Read block 4013 len 32768
14539 098/300 Read block 3175 len 12288
14540 099/300 Read block 2652 len 32768
14541 100/300 Read block 348 len 20480
14542 101/300 Flush
14543 102/300 Read block 4654 len 4096
14544 103/300 Read block 1008 len 28672
14545 104/300 Read block 4885 len 8192
14546 105/300 Read block 1566 len 24576
14547 106/300 Write block 2671 len 36864 data: 4 4 4 4 4 2 2 3 4
14548 107/300 Write block 3118 len 24576 data: 3 3 3 3 3 3
14549 108/300 Read block 3418 len 16384
14550 109/300 Read block 3220 len 24576
14551 110/300 Write block 117 len 8192 data: 5 4
14552 111/300 Read block 293 len 20480
14553 112/300 Flush
14554 113/300 Read block 2973 len 12288
14555 114/300 Write block 572 len 32768 data: 2 3 3 3 4 4 4 3
14556 115/300 Write block 3902 len 32768 data: 4 4 3 3 4 4 3 3
14557 116/300 Write block 1859 len 16384 data: 4 3 3 2
14558 117/300 Read block 3106 len 24576
14559 118/300 Flush
14560 119/300 Write block 2235 len 8192 data: 3 3
14561 120/300 Read block 2857 len 28672
14562 121/300 Write block 70 len 20480 data: 4 4 5 3 3
14563 122/300 Write block 4766 len 28672 data: 3 3 5 5 7 7 7
14564 123/300 Read block 1501 len 32768
14565 124/300 Read block 4459 len 32768
14566 125/300 Write block 348 len 40960 data: 7 7 7 4 4 3 3 4 2 2
14567 126/300 Write block 4120 len 40960 data: 3 3 4 4 5 5 4 5 6 6
14568 127/300 Write block 2652 len 16384 data: 3 3 2 2
14569 128/300 Read block 4074 len 20480
14570 129/300 Write block 1848 len 28672 data: 6 4 3 3 4 3 3
14571 130/300 Flush
14572 131/300 Write block 2966 len 16384 data: 2 2 3 4
14573 132/300 Flush
14574 133/300 Flush
14575 134/300 Read block 2821 len 12288
14576 135/300 Write block 1496 len 4096 data: 4
14577 136/300 Read block 1182 len 36864
14578 137/300 Write block 61 len 32768 data: 4 4 4 4 5 5 4 5
14579 138/300 Read block 4507 len 28672
14580 139/300 Write block 2724 len 4096 data: 3
14581 140/300 Read block 2434 len 24576
14582 141/300 Write block 1129 len 8192 data: 4 5
14583 142/300 Write block 1593 len 32768 data: 6 6 7 7 6 5 2 4
14584 143/300 Read block 783 len 28672
14585 144/300 Flush
14586 145/300 Flush
14587 146/300 Flush
14588 147/300 Write block 3551 len 20480 data: 4 4 4 3 3
14589 148/300 Read block 1433 len 24576
14590 149/300 Read block 3330 len 4096
14591 150/300 Write block 3031 len 24576 data: 6 5 2 3 3 2
14592 151/300 Write block 2904 len 8192 data: 6 5
14593 152/300 Write block 2217 len 40960 data: 4 3 3 2 2 3 2 2 2 2
14594 153/300 Read block 4145 len 8192
14595 154/300 Read block 1792 len 8192
14596 155/300 Read block 1328 len 24576
14597 156/300 Write block 1521 len 36864 data: 3 3 4 5 5 4 4 5 5
14598 157/300 Write block 4300 len 16384 data: 4 4 4 4
14599 158/300 Read block 1952 len 32768
14600 159/300 Write block 76 len 32768 data: 3 3 5 7 7 5 6 5
14601 160/300 Write block 1714 len 24576 data: 5 4 5 5 5 4
14602 161/300 Write block 2086 len 40960 data: 3 3 2 4 4 4 4 3 3 3
14603 162/300 Write block 2591 len 12288 data: 2 3 4
14604 163/300 Read block 4785 len 36864
14605 164/300 Write block 785 len 36864 data: 2 3 3 3 3 3 3 4 4
14606 165/300 Flush
14607 166/300 Read block 2752 len 8192
14608 167/300 Write block 2203 len 28672 data: 3 3 3 3 3 3 3
14609 168/300 Read block 336 len 20480
14610 169/300 Read block 2496 len 12288
14611 170/300 Read block 4022 len 8192
14612 171/300 Write block 212 len 16384 data: 5 5 6 5
14613 172/300 Read block 1754 len 16384
14614 173/300 Flush
14615 174/300 Flush
14616 175/300 Read block 2756 len 16384
14617 176/300 Write block 3401 len 4096 data: 4
14618 177/300 Write block 2236 len 24576 data: 4 4 5 5 5 5
14619 178/300 Read block 1490 len 20480
14620 179/300 Write block 1183 len 40960 data: 5 4 5 4 3 4 4 6 6 5
14621 180/300 Write block 2193 len 32768 data: 5 6 5 4 5 4 6 5
14622 181/300 Flush
14623 182/300 Read block 2257 len 28672
14624 183/300 Flush
14625 184/300 Read block 870 len 24576
14626 185/300 Read block 1007 len 28672
14627 186/300 Read block 1485 len 24576
14628 187/300 Write block 679 len 24576 data: 5 4 4 5 5 5
14629 188/300 Read block 1693 len 16384
14630 189/300 Write block 291 len 16384 data: 5 5 5 5
14631 190/300 Read block 78 len 4096
14632 191/300 Write block 2340 len 8192 data: 4 4
14633 192/300 Read block 1632 len 12288
14634 193/300 Read block 4369 len 28672
14635 194/300 Write block 2343 len 12288 data: 4 4 2
14636 195/300 Read block 4732 len 4096
14637 196/300 Read block 1260 len 8192
14638 197/300 Read block 890 len 20480
14639 198/300 Write block 4392 len 28672 data: 8 8 8 7 5 5 5
14640 199/300 Read block 1841 len 16384
14641 200/300 Write block 778 len 32768 data: 4 3 3 3 3 3 2 3
14642 201/300 Read block 2088 len 16384
14643 202/300 Write block 4638 len 40960 data: 3 2 2 3 3 4 4 4 4 3
14644 203/300 Read block 4979 len 4096
14645 204/300 Read block 1271 len 16384
14646 205/300 Write block 1275 len 32768 data: 4 4 5 4 4 4 3 3
14647 206/300 Read block 2864 len 20480
14648 207/300 Read block 342 len 40960
14649 208/300 Read block 2826 len 40960
14650 209/300 Write block 2231 len 28672 data: 2 2 2 3 4 5 5
14651 210/300 Read block 1522 len 28672
14652 211/300 Read block 2955 len 24576
14653 212/300 Read block 2098 len 36864
14654 213/300 Read block 2157 len 24576
14655 214/300 Flush
14656 215/300 Flush
14657 216/300 Read block 2347 len 24576
14658 217/300 Read block 218 len 36864
14659 218/300 Flush
14660 219/300 Read block 3282 len 4096
14661 220/300 Write block 3923 len 12288 data: 4 3 3
14662 221/300 Write block 3569 len 36864 data: 3 3 3 3 3 4 4 3 3
14663 222/300 Read block 3939 len 4096
14664 223/300 Read block 3750 len 4096
14665 224/300 Read block 4318 len 12288
14666 225/300 Read block 4093 len 32768
14667 226/300 Read block 1144 len 16384
14668 227/300 Read block 1465 len 40960
14669 228/300 Read block 4666 len 4096
14670 229/300 Write block 2907 len 16384 data: 5 5 4 3
14671 230/300 Write block 710 len 8192 data: 4 4
14672 231/300 Read block 1325 len 16384
14673 232/300 Read block 4713 len 24576
14674 233/300 Read block 4346 len 20480
14675 234/300 Read block 3784 len 28672
14676 235/300 Write block 1020 len 20480 data: 5 5 4 3 3
14677 236/300 Write block 956 len 28672 data: 3 3 3 3 3 3 2
14678 237/300 Read block 3692 len 36864
14679 238/300 Read block 2603 len 28672
14680 239/300 Write block 1359 len 4096 data: 3
14681 240/300 Write block 1632 len 4096 data: 3
14682 241/300 Write block 2794 len 16384 data: 3 4 5 4
14683 242/300 Write block 4623 len 4096 data: 4
14684 243/300 Read block 349 len 24576
14685 244/300 Read block 454 len 40960
14686 245/300 Read block 1313 len 24576
14687 246/300 Write block 1661 len 28672 data: 3 2 2 2 2 3 4
14688 247/300 Flush
14689 248/300 Write block 3875 len 40960 data: 3 3 3 3 3 3 3 3 3 3
14690 249/300 Write block 687 len 36864 data: 6 6 6 5 5 5 5 4 3
14691 250/300 Write block 2221 len 4096 data: 3
14692 251/300 Write block 4416 len 40960 data: 4 5 4 4 4 4 4 5 5 5
14693 252/300 Read block 237 len 28672
14694 253/300 Read block 153 len 28672
14695 254/300 Read block 1052 len 24576
14696 255/300 Write block 3769 len 40960 data: 4 4 4 4 6 6 5 5 4 4
14697 256/300 Write block 3532 len 4096 data: 2
14698 257/300 Read block 3912 len 12288
14699 258/300 Write block 2019 len 24576 data: 3 3 4 4 4 5
14700 259/300 Write block 2687 len 36864 data: 2 2 2 2 2 2 4 6 6
14701 260/300 Write block 895 len 40960 data: 3 3 3 3 4 3 3 3 3 4
14702 261/300 Read block 3371 len 4096
14703 262/300 Read block 888 len 40960
14704 263/300 Write block 2858 len 32768 data: 4 4 4 5 4 4 4 3
14705 264/300 Read block 3704 len 4096
14706 265/300 Write block 1271 len 36864 data: 4 3 3 3 5 5 6 5 5
14707 266/300 Read block 4948 len 16384
14708 267/300 Flush
14709 268/300 Write block 1817 len 40960 data: 6 6 6 6 4 4 3 3 3 5
14710 269/300 Read block 4384 len 36864
14711 270/300 Write block 4875 len 12288 data: 4 4 4
14712 271/300 Write block 4166 len 4096 data: 4
14713 272/300 Read block 2315 len 8192
14714 273/300 Write block 3049 len 20480 data: 2 4 4 3 3
14715 274/300 Write block 2830 len 12288 data: 5 6 5
14716 275/300 Write block 2581 len 8192 data: 3 5
14717 276/300 Read block 1713 len 16384
14718 277/300 Read block 604 len 28672
14719 278/300 Read block 3170 len 16384
14720 279/300 Read block 501 len 16384
14721 280/300 Write block 592 len 20480 data: 3 3 2 2 3
14722 281/300 Write block 4197 len 24576 data: 3 2 3 3 4 4
14723 282/300 Write block 3149 len 16384 data: 5 5 4 4
14724 283/300 Flush
14725 284/300 Read block 1922 len 36864
14726 285/300 Read block 747 len 8192
14727 286/300 Read block 2133 len 32768
14728 287/300 Read block 3663 len 24576
14729 288/300 Read block 4742 len 20480
14730 289/300 Read block 3829 len 20480
14731 290/300 Flush
14732 291/300 Read block 3768 len 4096
14733 292/300 Read block 2370 len 4096
14734 293/300 Read block 478 len 40960
14735 294/300 Read block 1341 len 40960
14736 295/300 Read block 4484 len 36864
14737 296/300 Flush
14738 297/300 Write block 323 len 24576 data: 5 7 7 8 7 7
14739 298/300 Read block 3295 len 36864
14740 299/300 Read block 2985 len 12288
14741 300/300 Flush
14742 301/300 Read block 2484 len 8192
14743 Main task has work to do, go find it
14744 got dsc Start(0)
14745 start 0
14746 Jan 14 16:13:43.577 INFO request completed, latency_us: 198, response_code: 204, uri: /start/cid/0, method: POST, req_id: 75f0c2a3-1a31-4c7f-99c7-78134f355d58, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
14747 Replay: started 0, returned:Ok(())
14748 ----------------------------------------------------------------
14749 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
14750 States: Offline Active Active
14751 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14752 5226 Acked Write 5 Sent Done Done false
14753 5227 Acked Write 2 Sent Done Done false
14754 5228 Acked Write 6 Sent Done Done false
14755 5229 Acked Flush 0 Sent Done Done false
14756 5230 Acked Read 4 Sent Done Done false
14757 5231 Acked Read 6 Sent Done Done false
14758 5232 Acked Read 3 Sent Done Done false
14759 5233 Acked Read 6 Sent Done Done false
14760 5234 Acked Flush 0 Sent Done Done false
14761 5235 Acked Read 8 Sent Done Done false
14762 5236 Acked Read 3 Sent Done Done false
14763 5237 Acked Write 8 Sent Done Done false
14764 5238 Acked Write 8 Sent Done Done false
14765 5239 Acked Write 3 Sent Done Done false
14766 5240 Acked Write 2 Sent Done Done false
14767 5241 Acked Write 8 Sent Done Done false
14768 5242 Acked Write 6 Sent Done Done false
14769 5243 Acked Read 4 Sent Done Done false
14770 5244 Acked Read 4 Sent Done Done false
14771 5245 Acked Read 1 Sent Done Done false
14772 5246 Acked Read 6 Sent Done Done false
14773 5247 Acked Write 4 Sent Done Done false
14774 5248 Acked Read 3 Sent Done Done false
14775 5249 Acked Read 10 Sent Done Done false
14776 5250 Acked Read 4 Sent Done Done false
14777 5251 Acked Write 2 Sent Done Done false
14778 5252 Acked Write 3 Sent Done Done false
14779 5253 Acked Read 10 Sent Done Done false
14780 5254 Acked Flush 0 Sent Done Done false
14781 5255 Acked Read 4 Sent Done Done false
14782 5256 Acked Read 4 Sent Done Done false
14783 5257 Acked Read 3 Sent Done Done false
14784 5258 Acked Flush 0 Sent Done Done false
14785 5259 Acked Write 3 Sent Done Done false
14786 5260 Acked Read 5 Sent Done Done false
14787 5261 Acked Write 3 Sent Done Done false
14788 5262 Acked Read 3 Sent Done Done false
14789 5263 Acked Read 2 Sent Done Done false
14790 5264 Acked Read 7 Sent Done Done false
14791 5265 Acked Read 6 Sent Done Done false
14792 5266 Acked Read 3 Sent Done Done false
14793 5267 Acked Flush 0 Sent Done Done false
14794 5268 Acked Write 7 Sent Done Done false
14795 5269 Acked Read 7 Sent Done Done false
14796 5270 Acked Write 6 Sent Done Done false
14797 5271 Acked Read 4 Sent Done Done false
14798 5272 Acked Flush 0 Sent Done Done false
14799 5273 Acked Write 8 Sent Done Done false
14800 5274 Acked Read 7 Sent Done Done false
14801 5275 Acked Write 5 Sent Done Done false
14802 5276 Acked Write 10 Sent Done Done false
14803 5277 Acked Write 7 Sent Done Done false
14804 5278 Acked Write 10 Sent Done Done false
14805 5279 Acked Read 3 Sent Done Done false
14806 5280 Acked Write 1 Sent Done Done false
14807 5281 Acked Read 5 Sent Done Done false
14808 5282 Acked Read 4 Sent Done Done false
14809 5283 Acked Read 7 Sent Done Done false
14810 5284 Acked Read 3 Sent Done Done false
14811 5285 Acked Read 4 Sent Done Done false
14812 5286 Acked Write 2 Sent Done Done false
14813 5287 Acked Write 8 Sent Done Done false
14814 5288 Acked Flush 0 Sent Done Done false
14815 5289 Acked Write 6 Sent Done Done false
14816 5290 Acked Read 6 Sent Done Done false
14817 5291 Acked Read 6 Sent Done Done false
14818 5292 Acked Flush 0 Sent Done Done false
14819 5293 Acked Write 9 Sent Done Done false
14820 5294 Acked Read 8 Sent Done Done false
14821 5295 Acked Read 7 Sent Done Done false
14822 5296 Acked Read 6 Sent Done Done false
14823 5297 Acked Write 5 Sent Done Done false
14824 5298 Acked Write 9 Sent Done Done false
14825 5299 Acked Read 3 Sent Done Done false
14826 5300 Acked Read 6 Sent Done Done false
14827 5301 Acked Read 9 Sent Done Done false
14828 5302 Acked Write 4 Sent Done Done false
14829 5303 Acked Read 9 Sent Done Done false
14830 5304 Acked Write 1 Sent Done Done false
14831 5305 Acked Read 8 Sent Done Done false
14832 5306 Acked Read 8 Sent Done Done false
14833 5307 Acked Write 6 Sent Done Done false
14834 5308 Acked Read 9 Sent Done Done false
14835 5309 Acked Write 3 Sent Done Done false
14836 5310 Acked Read 7 Sent Done Done false
14837 5311 Acked Write 5 Sent Done Done false
14838 5312 Acked Read 4 Sent Done Done false
14839 5313 Acked Write 4 Sent Done Done false
14840 5314 Acked Read 2 Sent Done Done false
14841 5315 Acked Read 9 Sent Done Done false
14842 5316 Acked Flush 0 Sent Done Done false
14843 5317 Acked Write 9 Sent Done Done false
14844 5318 Acked Flush 0 Sent Done Done false
14845 5319 Acked Write 2 Sent Done Done false
14846 5320 Acked Write 1 Sent Done Done false
14847 5321 Acked Flush 0 Sent Done Done false
14848 5322 Acked Read 8 Sent Done Done false
14849 5323 Acked Read 3 Sent Done Done false
14850 5324 Acked Read 8 Sent Done Done false
14851 5325 Acked Read 5 Sent Done Done false
14852 5326 Acked Flush 0 Sent Done Done false
14853 5327 Acked Read 1 Sent Done Done false
14854 5328 Acked Read 7 Sent Done Done false
14855 5329 Acked Read 2 Sent Done Done false
14856 5330 Acked Read 6 Sent Done Done false
14857 5331 Acked Write 9 Sent Done Done false
14858 5332 Acked Write 6 Sent Done Done false
14859 5333 Acked Read 4 Sent Done Done false
14860 5334 Acked Read 6 Sent Done Done false
14861 5335 Acked Write 2 Sent Done Done false
14862 5336 Acked Read 5 Sent Done Done false
14863 5337 Acked Flush 0 Sent Done Done false
14864 5338 Acked Read 3 Sent Done Done false
14865 5339 Acked Write 8 Sent Done Done false
14866 5340 Acked Write 8 Sent Done Done false
14867 5341 Acked Write 4 Sent Done Done false
14868 5342 Acked Read 6 Sent Done Done false
14869 5343 Acked Flush 0 Sent Done Done false
14870 5344 Acked Write 2 Sent Done Done false
14871 5345 Acked Read 7 Sent Done Done false
14872 5346 Acked Write 5 Sent Done Done false
14873 5347 Acked Write 7 Sent Done Done false
14874 5348 Acked Read 8 Sent Done Done false
14875 5349 Acked Read 8 Sent Done Done false
14876 5350 Acked Write 10 Sent Done Done false
14877 5351 Acked Write 10 Sent Done Done false
14878 5352 Acked Write 4 Sent Done Done false
14879 5353 Acked Read 5 Sent Done Done false
14880 5354 Acked Write 7 Sent Done Done false
14881 5355 Acked Flush 0 Sent Done Done false
14882 5356 Acked Write 4 Sent Done Done false
14883 5357 Acked Flush 0 Sent Done Done false
14884 5358 Acked Flush 0 Sent Done Done false
14885 5359 Acked Read 3 Sent Done Done false
14886 5360 Acked Write 1 Sent Done Done false
14887 5361 Acked Read 9 Sent Done Done false
14888 5362 Acked Write 8 Sent Done Done false
14889 5363 Acked Read 7 Sent Done Done false
14890 5364 Acked Write 1 Sent Done Done false
14891 5365 Acked Read 6 Sent Done Done false
14892 5366 Acked Write 2 Sent Done Done false
14893 5367 Acked Write 8 Sent Done Done false
14894 5368 Acked Read 7 Sent Done Done false
14895 5369 Acked Flush 0 Sent Done Done false
14896 5370 Acked Flush 0 Sent Done Done false
14897 5371 Acked Flush 0 Sent Done Done false
14898 5372 Acked Write 5 Sent Done Done false
14899 5373 Acked Read 6 Sent Done Done false
14900 5374 Acked Read 1 Sent Done Done false
14901 5375 Acked Write 6 Sent Done Done false
14902 5376 Acked Write 2 Sent Done Done false
14903 5377 Acked Write 10 Sent Done Done false
14904 5378 Acked Read 2 Sent Done Done false
14905 5379 Acked Read 2 Sent Done Done false
14906 5380 Acked Read 6 Sent Done Done false
14907 5381 Acked Write 9 Sent Done Done false
14908 5382 Acked Write 4 Sent Done Done false
14909 5383 Acked Read 8 Sent Done Done false
14910 5384 Acked Write 8 Sent Done Done false
14911 5385 Acked Write 6 Sent Done Done false
14912 5386 Acked Write 10 Sent Done Done false
14913 5387 Acked Write 3 Sent Done Done false
14914 5388 Acked Read 9 Sent Done Done false
14915 5389 Acked Write 9 Sent Done Done false
14916 5390 Acked Flush 0 Sent Done Done false
14917 5391 Acked Read 2 Sent Done Done false
14918 5392 Acked Write 7 Sent Done Done false
14919 5393 Acked Read 5 Sent Done Done false
14920 5394 Acked Read 3 Sent Done Done false
14921 5395 Acked Read 2 Sent Done Done false
14922 5396 Acked Write 4 Sent Done Done false
14923 5397 Acked Read 4 Sent Done Done false
14924 5398 Acked Flush 0 Sent Done Done false
14925 5399 Acked Flush 0 Sent Done Done false
14926 5400 Acked Read 4 Sent Done Done false
14927 5401 Acked Write 1 Sent Done Done false
14928 5402 Acked Write 6 Sent Done Done false
14929 5403 Acked Read 5 Sent Done Done false
14930 5404 Acked Write 10 Sent Done Done false
14931 5405 Acked Write 8 Sent Done Done false
14932 5406 Acked Flush 0 Sent Done Done false
14933 5407 Acked Read 7 Sent Done Done false
14934 5408 Acked Flush 0 Sent Done Done false
14935 5409 Acked Read 6 Sent Done Done false
14936 5410 Acked Read 7 Sent Done Done false
14937 5411 Acked Read 6 Sent Done Done false
14938 5412 Acked Write 6 Sent Done Done false
14939 5413 Acked Read 4 Sent Done Done false
14940 5414 Acked Write 4 Sent Done Done false
14941 5415 Acked Read 1 Sent Done Done false
14942 5416 Acked Write 2 Sent Done Done false
14943 5417 Acked Read 3 Sent Done Done false
14944 5418 Acked Read 7 Sent Done Done false
14945 5419 Acked Write 3 Sent Done Done false
14946 5420 Acked Read 1 Sent Done Done false
14947 5421 Acked Read 2 Sent Done Done false
14948 5422 Acked Read 5 Sent Done Done false
14949 5423 Acked Write 7 Sent Done Done false
14950 5424 Acked Read 4 Sent Done Done false
14951 5425 Acked Write 8 Sent Done Done false
14952 5426 Acked Read 4 Sent Done Done false
14953 5427 Acked Write 10 Sent Done Done false
14954 5428 Acked Read 1 Sent Done Done false
14955 5429 Acked Read 4 Sent Done Done false
14956 5430 Acked Write 8 Sent Done Done false
14957 5431 Acked Read 5 Sent Done Done false
14958 5432 Acked Read 10 Sent Done Done false
14959 5433 Acked Read 10 Sent Done Done false
14960 5434 Acked Write 7 Sent Done Done false
14961 5435 Acked Read 7 Sent Done Done false
14962 5436 Acked Read 6 Sent Done Done false
14963 5437 Acked Read 9 Sent Done Done false
14964 5438 Acked Read 6 Sent Done Done false
14965 5439 Acked Flush 0 Sent Done Done false
14966 5440 Acked Flush 0 Sent Done Done false
14967 5441 Acked Read 6 Sent Done Done false
14968 5442 Acked Read 9 Sent Done Done false
14969 5443 Acked Flush 0 Sent Done Done false
14970 5444 Acked Read 1 Sent Done Done false
14971 5445 Acked Write 3 Sent Done Done false
14972 5446 Acked Write 9 Sent Done Done false
14973 5447 Acked Read 1 Sent Done Done false
14974 5448 Acked Read 1 Sent Done Done false
14975 5449 Acked Read 3 Sent Done Done false
14976 5450 Acked Read 8 Sent Done Done false
14977 5451 Acked Read 4 Sent Done Done false
14978 5452 Acked Read 10 Sent Done Done false
14979 5453 Acked Read 1 Sent Done Done false
14980 5454 Acked Write 4 Sent Done Done false
14981 5455 Acked Write 2 Sent Done Done false
14982 5456 Acked Read 4 Sent Done Done false
14983 5457 Acked Read 6 Sent Done Done false
14984 5458 Acked Read 5 Sent Done Done false
14985 5459 Acked Read 7 Sent Done Done false
14986 5460 Acked Write 5 Sent Done Done false
14987 5461 Acked Write 7 Sent Done Done false
14988 5462 Acked Read 9 Sent Done Done false
14989 5463 Acked Read 7 Sent Done Done false
14990 5464 Acked Write 1 Sent Done Done false
14991 5465 Acked Write 1 Sent Done Done false
14992 5466 Acked Write 4 Sent Done Done false
14993 5467 Acked Write 1 Sent Done Done false
14994 5468 Acked Read 6 Sent Done Done false
14995 5469 Acked Read 10 Sent Done Done false
14996 5470 Acked Read 6 Sent Done Done false
14997 5471 Acked Write 7 Sent Done Done false
14998 5472 Acked Flush 0 Sent Done Done false
14999 5473 Acked Write 10 Sent Done Done false
15000 5474 Acked Write 9 Sent Done Done false
15001 5475 Acked Write 1 Sent Done Done false
15002 5476 Acked Write 10 Sent Done Done false
15003 5477 Acked Read 7 Sent Done Done false
15004 5478 Acked Read 7 Sent Done Done false
15005 5479 Acked Read 6 Sent Done Done false
15006 5480 Acked Write 10 Sent Done Done false
15007 5481 Acked Write 1 Sent Done Done false
15008 5482 Acked Read 3 Sent Done Done false
15009 5483 Acked Write 6 Sent Done Done false
15010 5484 Acked Write 9 Sent Done Done false
15011 5485 Acked Write 10 Sent Done Done false
15012 5486 Acked Read 1 Sent Done Done false
15013 5487 Acked Read 10 Sent Done Done false
15014 5488 Acked Write 8 Sent Done Done false
15015 5489 Acked Read 1 Sent Done Done false
15016 5490 Acked Write 9 Sent Done Done false
15017 5491 Acked Read 4 Sent Done Done false
15018 5492 Acked Flush 0 Sent Done Done false
15019 5493 Acked Write 10 Sent Done Done false
15020 5494 Acked Read 9 Sent Done Done false
15021 5495 Acked Write 3 Sent Done Done false
15022 5496 Acked Write 1 Sent Done Done false
15023 5497 Acked Read 2 Sent Done Done false
15024 5498 Acked Write 5 Sent Done Done false
15025 5499 Acked Write 3 Sent Done Done false
15026 5500 Acked Write 2 Sent Done Done false
15027 5501 Acked Read 4 Sent Done Done false
15028 5502 Acked Read 7 Sent Done Done false
15029 5503 Acked Read 4 Sent Done Done false
15030 5504 Acked Read 4 Sent Done Done false
15031 5505 Acked Write 5 Sent Done Done false
15032 5506 Acked Write 6 Sent Done Done false
15033 5507 Acked Write 4 Sent Done Done false
15034 5508 Acked Flush 0 Sent Done Done false
15035 5509 Acked Read 9 Sent Done Done false
15036 5510 Acked Read 2 Sent Done Done false
15037 5511 Acked Read 8 Sent Done Done false
15038 5512 Acked Read 6 Sent Done Done false
15039 5513 Acked Read 5 Sent Done Done false
15040 5514 Acked Read 5 Sent Done Done false
15041 5515 Acked Flush 0 Sent Done Done false
15042 5516 Acked Read 1 Sent Done Done false
15043 5517 Acked Read 1 Sent Done Done false
15044 5518 Acked Read 10 Sent Done Done false
15045 5519 Acked Read 10 Sent Done Done false
15046 5520 Acked Read 9 Sent Done Done false
15047 5521 Acked Flush 0 Sent Done Done false
15048 5522 Acked Write 6 Sent Done Done false
15049 5523 Acked Read 9 Sent Done Done false
15050 5524 Acked Read 3 Sent Done Done false
15051 5525 Acked Flush 0 Sent Done Done false
15052 5526 Acked Read 2 Sent Done Done false
15053 STATES DS:0 DS:1 DS:2 TOTAL
15054 Sent 301 0 0 301
15055 Done 0 301 301 602
15056 Skipped 0 0 0 0
15057 Error 0 0 0 0
15058 Last Flush: 5225 5525 5525
15059 Downstairs last five completed: 5225 5224 5223 5222 5221
15060 Upstairs last five completed: 5526 5525 5524 5523 5522
15061 CLIENT: Up:0 ds:301 act:2
15062 ----------------------------------------------------------------
15063 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15064 States: Offline Active Active
15065 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15066 5226 Acked Write 5 Sent Done Done false
15067 5227 Acked Write 2 Sent Done Done false
15068 5228 Acked Write 6 Sent Done Done false
15069 5229 Acked Flush 0 Sent Done Done false
15070 5230 Acked Read 4 Sent Done Done false
15071 5231 Acked Read 6 Sent Done Done false
15072 5232 Acked Read 3 Sent Done Done false
15073 5233 Acked Read 6 Sent Done Done false
15074 5234 Acked Flush 0 Sent Done Done false
15075 5235 Acked Read 8 Sent Done Done false
15076 5236 Acked Read 3 Sent Done Done false
15077 5237 Acked Write 8 Sent Done Done false
15078 5238 Acked Write 8 Sent Done Done false
15079 5239 Acked Write 3 Sent Done Done false
15080 5240 Acked Write 2 Sent Done Done false
15081 5241 Acked Write 8 Sent Done Done false
15082 5242 Acked Write 6 Sent Done Done false
15083 5243 Acked Read 4 Sent Done Done false
15084 5244 Acked Read 4 Sent Done Done false
15085 5245 Acked Read 1 Sent Done Done false
15086 5246 Acked Read 6 Sent Done Done false
15087 5247 Acked Write 4 Sent Done Done false
15088 5248 Acked Read 3 Sent Done Done false
15089 5249 Acked Read 10 Sent Done Done false
15090 5250 Acked Read 4 Sent Done Done false
15091 5251 Acked Write 2 Sent Done Done false
15092 5252 Acked Write 3 Sent Done Done false
15093 5253 Acked Read 10 Sent Done Done false
15094 5254 Acked Flush 0 Sent Done Done false
15095 5255 Acked Read 4 Sent Done Done false
15096 5256 Acked Read 4 Sent Done Done false
15097 5257 Acked Read 3 Sent Done Done false
15098 5258 Acked Flush 0 Sent Done Done false
15099 5259 Acked Write 3 Sent Done Done false
15100 5260 Acked Read 5 Sent Done Done false
15101 5261 Acked Write 3 Sent Done Done false
15102 5262 Acked Read 3 Sent Done Done false
15103 5263 Acked Read 2 Sent Done Done false
15104 5264 Acked Read 7 Sent Done Done false
15105 5265 Acked Read 6 Sent Done Done false
15106 5266 Acked Read 3 Sent Done Done false
15107 5267 Acked Flush 0 Sent Done Done false
15108 5268 Acked Write 7 Sent Done Done false
15109 5269 Acked Read 7 Sent Done Done false
15110 5270 Acked Write 6 Sent Done Done false
15111 5271 Acked Read 4 Sent Done Done false
15112 5272 Acked Flush 0 Sent Done Done false
15113 5273 Acked Write 8 Sent Done Done false
15114 5274 Acked Read 7 Sent Done Done false
15115 5275 Acked Write 5 Sent Done Done false
15116 5276 Acked Write 10 Sent Done Done false
15117 5277 Acked Write 7 Sent Done Done false
15118 5278 Acked Write 10 Sent Done Done false
15119 5279 Acked Read 3 Sent Done Done false
15120 5280 Acked Write 1 Sent Done Done false
15121 5281 Acked Read 5 Sent Done Done false
15122 5282 Acked Read 4 Sent Done Done false
15123 5283 Acked Read 7 Sent Done Done false
15124 5284 Acked Read 3 Sent Done Done false
15125 5285 Acked Read 4 Sent Done Done false
15126 5286 Acked Write 2 Sent Done Done false
15127 5287 Acked Write 8 Sent Done Done false
15128 5288 Acked Flush 0 Sent Done Done false
15129 5289 Acked Write 6 Sent Done Done false
15130 5290 Acked Read 6 Sent Done Done false
15131 5291 Acked Read 6 Sent Done Done false
15132 5292 Acked Flush 0 Sent Done Done false
15133 5293 Acked Write 9 Sent Done Done false
15134 5294 Acked Read 8 Sent Done Done false
15135 5295 Acked Read 7 Sent Done Done false
15136 5296 Acked Read 6 Sent Done Done false
15137 5297 Acked Write 5 Sent Done Done false
15138 5298 Acked Write 9 Sent Done Done false
15139 5299 Acked Read 3 Sent Done Done false
15140 5300 Acked Read 6 Sent Done Done false
15141 5301 Acked Read 9 Sent Done Done false
15142 5302 Acked Write 4 Sent Done Done false
15143 5303 Acked Read 9 Sent Done Done false
15144 5304 Acked Write 1 Sent Done Done false
15145 5305 Acked Read 8 Sent Done Done false
15146 5306 Acked Read 8 Sent Done Done false
15147 5307 Acked Write 6 Sent Done Done false
15148 5308 Acked Read 9 Sent Done Done false
15149 5309 Acked Write 3 Sent Done Done false
15150 5310 Acked Read 7 Sent Done Done false
15151 5311 Acked Write 5 Sent Done Done false
15152 5312 Acked Read 4 Sent Done Done false
15153 5313 Acked Write 4 Sent Done Done false
15154 5314 Acked Read 2 Sent Done Done false
15155 5315 Acked Read 9 Sent Done Done false
15156 5316 Acked Flush 0 Sent Done Done false
15157 5317 Acked Write 9 Sent Done Done false
15158 5318 Acked Flush 0 Sent Done Done false
15159 5319 Acked Write 2 Sent Done Done false
15160 5320 Acked Write 1 Sent Done Done false
15161 5321 Acked Flush 0 Sent Done Done false
15162 5322 Acked Read 8 Sent Done Done false
15163 5323 Acked Read 3 Sent Done Done false
15164 5324 Acked Read 8 Sent Done Done false
15165 5325 Acked Read 5 Sent Done Done false
15166 5326 Acked Flush 0 Sent Done Done false
15167 5327 Acked Read 1 Sent Done Done false
15168 5328 Acked Read 7 Sent Done Done false
15169 5329 Acked Read 2 Sent Done Done false
15170 5330 Acked Read 6 Sent Done Done false
15171 5331 Acked Write 9 Sent Done Done false
15172 5332 Acked Write 6 Sent Done Done false
15173 5333 Acked Read 4 Sent Done Done false
15174 5334 Acked Read 6 Sent Done Done false
15175 5335 Acked Write 2 Sent Done Done false
15176 5336 Acked Read 5 Sent Done Done false
15177 5337 Acked Flush 0 Sent Done Done false
15178 5338 Acked Read 3 Sent Done Done false
15179 5339 Acked Write 8 Sent Done Done false
15180 5340 Acked Write 8 Sent Done Done false
15181 5341 Acked Write 4 Sent Done Done false
15182 5342 Acked Read 6 Sent Done Done false
15183 5343 Acked Flush 0 Sent Done Done false
15184 5344 Acked Write 2 Sent Done Done false
15185 5345 Acked Read 7 Sent Done Done false
15186 5346 Acked Write 5 Sent Done Done false
15187 5347 Acked Write 7 Sent Done Done false
15188 5348 Acked Read 8 Sent Done Done false
15189 5349 Acked Read 8 Sent Done Done false
15190 5350 Acked Write 10 Sent Done Done false
15191 5351 Acked Write 10 Sent Done Done false
15192 5352 Acked Write 4 Sent Done Done false
15193 5353 Acked Read 5 Sent Done Done false
15194 5354 Acked Write 7 Sent Done Done false
15195 5355 Acked Flush 0 Sent Done Done false
15196 5356 Acked Write 4 Sent Done Done false
15197 5357 Acked Flush 0 Sent Done Done false
15198 5358 Acked Flush 0 Sent Done Done false
15199 5359 Acked Read 3 Sent Done Done false
15200 5360 Acked Write 1 Sent Done Done false
15201 5361 Acked Read 9 Sent Done Done false
15202 5362 Acked Write 8 Sent Done Done false
15203 5363 Acked Read 7 Sent Done Done false
15204 5364 Acked Write 1 Sent Done Done false
15205 5365 Acked Read 6 Sent Done Done false
15206 5366 Acked Write 2 Sent Done Done false
15207 5367 Acked Write 8 Sent Done Done false
15208 5368 Acked Read 7 Sent Done Done false
15209 5369 Acked Flush 0 Sent Done Done false
15210 5370 Acked Flush 0 Sent Done Done false
15211 5371 Acked Flush 0 Sent Done Done false
15212 5372 Acked Write 5 Sent Done Done false
15213 5373 Acked Read 6 Sent Done Done false
15214 5374 Acked Read 1 Sent Done Done false
15215 5375 Acked Write 6 Sent Done Done false
15216 5376 Acked Write 2 Sent Done Done false
15217 5377 Acked Write 10 Sent Done Done false
15218 5378 Acked Read 2 Sent Done Done false
15219 5379 Acked Read 2 Sent Done Done false
15220 5380 Acked Read 6 Sent Done Done false
15221 5381 Acked Write 9 Sent Done Done false
15222 5382 Acked Write 4 Sent Done Done false
15223 5383 Acked Read 8 Sent Done Done false
15224 5384 Acked Write 8 Sent Done Done false
15225 5385 Acked Write 6 Sent Done Done false
15226 5386 Acked Write 10 Sent Done Done false
15227 5387 Acked Write 3 Sent Done Done false
15228 5388 Acked Read 9 Sent Done Done false
15229 5389 Acked Write 9 Sent Done Done false
15230 5390 Acked Flush 0 Sent Done Done false
15231 5391 Acked Read 2 Sent Done Done false
15232 5392 Acked Write 7 Sent Done Done false
15233 5393 Acked Read 5 Sent Done Done false
15234 5394 Acked Read 3 Sent Done Done false
15235 5395 Acked Read 2 Sent Done Done false
15236 5396 Acked Write 4 Sent Done Done false
15237 5397 Acked Read 4 Sent Done Done false
15238 5398 Acked Flush 0 Sent Done Done false
15239 5399 Acked Flush 0 Sent Done Done false
15240 5400 Acked Read 4 Sent Done Done false
15241 5401 Acked Write 1 Sent Done Done false
15242 5402 Acked Write 6 Sent Done Done false
15243 5403 Acked Read 5 Sent Done Done false
15244 5404 Acked Write 10 Sent Done Done false
15245 5405 Acked Write 8 Sent Done Done false
15246 5406 Acked Flush 0 Sent Done Done false
15247 5407 Acked Read 7 Sent Done Done false
15248 5408 Acked Flush 0 Sent Done Done false
15249 5409 Acked Read 6 Sent Done Done false
15250 5410 Acked Read 7 Sent Done Done false
15251 5411 Acked Read 6 Sent Done Done false
15252 5412 Acked Write 6 Sent Done Done false
15253 5413 Acked Read 4 Sent Done Done false
15254 5414 Acked Write 4 Sent Done Done false
15255 5415 Acked Read 1 Sent Done Done false
15256 5416 Acked Write 2 Sent Done Done false
15257 5417 Acked Read 3 Sent Done Done false
15258 5418 Acked Read 7 Sent Done Done false
15259 5419 Acked Write 3 Sent Done Done false
15260 5420 Acked Read 1 Sent Done Done false
15261 5421 Acked Read 2 Sent Done Done false
15262 5422 Acked Read 5 Sent Done Done false
15263 5423 Acked Write 7 Sent Done Done false
15264 5424 Acked Read 4 Sent Done Done false
15265 5425 Acked Write 8 Sent Done Done false
15266 5426 Acked Read 4 Sent Done Done false
15267 5427 Acked Write 10 Sent Done Done false
15268 5428 Acked Read 1 Sent Done Done false
15269 5429 Acked Read 4 Sent Done Done false
15270 5430 Acked Write 8 Sent Done Done false
15271 5431 Acked Read 5 Sent Done Done false
15272 5432 Acked Read 10 Sent Done Done false
15273 5433 Acked Read 10 Sent Done Done false
15274 5434 Acked Write 7 Sent Done Done false
15275 5435 Acked Read 7 Sent Done Done false
15276 5436 Acked Read 6 Sent Done Done false
15277 5437 Acked Read 9 Sent Done Done false
15278 5438 Acked Read 6 Sent Done Done false
15279 5439 Acked Flush 0 Sent Done Done false
15280 5440 Acked Flush 0 Sent Done Done false
15281 5441 Acked Read 6 Sent Done Done false
15282 5442 Acked Read 9 Sent Done Done false
15283 5443 Acked Flush 0 Sent Done Done false
15284 5444 Acked Read 1 Sent Done Done false
15285 5445 Acked Write 3 Sent Done Done false
15286 5446 Acked Write 9 Sent Done Done false
15287 5447 Acked Read 1 Sent Done Done false
15288 5448 Acked Read 1 Sent Done Done false
15289 5449 Acked Read 3 Sent Done Done false
15290 5450 Acked Read 8 Sent Done Done false
15291 5451 Acked Read 4 Sent Done Done false
15292 5452 Acked Read 10 Sent Done Done false
15293 5453 Acked Read 1 Sent Done Done false
15294 5454 Acked Write 4 Sent Done Done false
15295 5455 Acked Write 2 Sent Done Done false
15296 5456 Acked Read 4 Sent Done Done false
15297 5457 Acked Read 6 Sent Done Done false
15298 5458 Acked Read 5 Sent Done Done false
15299 5459 Acked Read 7 Sent Done Done false
15300 5460 Acked Write 5 Sent Done Done false
15301 5461 Acked Write 7 Sent Done Done false
15302 5462 Acked Read 9 Sent Done Done false
15303 5463 Acked Read 7 Sent Done Done false
15304 5464 Acked Write 1 Sent Done Done false
15305 5465 Acked Write 1 Sent Done Done false
15306 5466 Acked Write 4 Sent Done Done false
15307 5467 Acked Write 1 Sent Done Done false
15308 5468 Acked Read 6 Sent Done Done false
15309 5469 Acked Read 10 Sent Done Done false
15310 5470 Acked Read 6 Sent Done Done false
15311 5471 Acked Write 7 Sent Done Done false
15312 5472 Acked Flush 0 Sent Done Done false
15313 5473 Acked Write 10 Sent Done Done false
15314 5474 Acked Write 9 Sent Done Done false
15315 5475 Acked Write 1 Sent Done Done false
15316 5476 Acked Write 10 Sent Done Done false
15317 5477 Acked Read 7 Sent Done Done false
15318 5478 Acked Read 7 Sent Done Done false
15319 5479 Acked Read 6 Sent Done Done false
15320 5480 Acked Write 10 Sent Done Done false
15321 5481 Acked Write 1 Sent Done Done false
15322 5482 Acked Read 3 Sent Done Done false
15323 5483 Acked Write 6 Sent Done Done false
15324 5484 Acked Write 9 Sent Done Done false
15325 5485 Acked Write 10 Sent Done Done false
15326 5486 Acked Read 1 Sent Done Done false
15327 5487 Acked Read 10 Sent Done Done false
15328 5488 Acked Write 8 Sent Done Done false
15329 5489 Acked Read 1 Sent Done Done false
15330 5490 Acked Write 9 Sent Done Done false
15331 5491 Acked Read 4 Sent Done Done false
15332 5492 Acked Flush 0 Sent Done Done false
15333 5493 Acked Write 10 Sent Done Done false
15334 5494 Acked Read 9 Sent Done Done false
15335 5495 Acked Write 3 Sent Done Done false
15336 5496 Acked Write 1 Sent Done Done false
15337 5497 Acked Read 2 Sent Done Done false
15338 5498 Acked Write 5 Sent Done Done false
15339 5499 Acked Write 3 Sent Done Done false
15340 5500 Acked Write 2 Sent Done Done false
15341 5501 Acked Read 4 Sent Done Done false
15342 5502 Acked Read 7 Sent Done Done false
15343 5503 Acked Read 4 Sent Done Done false
15344 5504 Acked Read 4 Sent Done Done false
15345 5505 Acked Write 5 Sent Done Done false
15346 5506 Acked Write 6 Sent Done Done false
15347 5507 Acked Write 4 Sent Done Done false
15348 5508 Acked Flush 0 Sent Done Done false
15349 5509 Acked Read 9 Sent Done Done false
15350 5510 Acked Read 2 Sent Done Done false
15351 5511 Acked Read 8 Sent Done Done false
15352 5512 Acked Read 6 Sent Done Done false
15353 5513 Acked Read 5 Sent Done Done false
15354 5514 Acked Read 5 Sent Done Done false
15355 5515 Acked Flush 0 Sent Done Done false
15356 5516 Acked Read 1 Sent Done Done false
15357 5517 Acked Read 1 Sent Done Done false
15358 5518 Acked Read 10 Sent Done Done false
15359 5519 Acked Read 10 Sent Done Done false
15360 5520 Acked Read 9 Sent Done Done false
15361 5521 Acked Flush 0 Sent Done Done false
15362 5522 Acked Write 6 Sent Done Done false
15363 5523 Acked Read 9 Sent Done Done false
15364 5524 Acked Read 3 Sent Done Done false
15365 5525 Acked Flush 0 Sent Done Done false
15366 5526 Acked Read 2 Sent Done Done false
15367 5527 Acked Flush 0 Sent Done Done false
15368 STATES DS:0 DS:1 DS:2 TOTAL
15369 Sent 302 0 0 302
15370 Done 0 302 302 604
15371 Skipped 0 0 0 0
15372 Error 0 0 0 0
15373 Last Flush: 5225 5527 5527
15374 Downstairs last five completed: 5225 5224 5223 5222 5221
15375 Upstairs last five completed: 5527 5526 5525 5524 5523
15376 CLIENT: Up:0 ds:302 act:2
153772025-01-14T16:13:49.336ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
153782025-01-14T16:13:49.337ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
153792025-01-14T16:13:49.337ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
153802025-01-14T16:13:49.337ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = c2423ef0-728e-4895-997a-d9044c50948b
153812025-01-14T16:13:49.337ZINFOcrucible: client task is exiting = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
15382 [8810] got start action
15383 [8810] I am going to restart
15384 Starting downstairs at port 8810
15385 Make output file at "/tmp/dsc/downstairs-8810.txt"
15386 [8810][0] reports Starting
15387 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1217)
15388 [8810][0] reports Running
15389 ----------------------------------------------------------------
15390 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15391 States: Offline Active Active
15392 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15393 5226 Acked Write 5 Sent Done Done false
15394 5227 Acked Write 2 Sent Done Done false
15395 5228 Acked Write 6 Sent Done Done false
15396 5229 Acked Flush 0 Sent Done Done false
15397 5230 Acked Read 4 Sent Done Done false
15398 5231 Acked Read 6 Sent Done Done false
15399 5232 Acked Read 3 Sent Done Done false
15400 5233 Acked Read 6 Sent Done Done false
15401 5234 Acked Flush 0 Sent Done Done false
15402 5235 Acked Read 8 Sent Done Done false
15403 5236 Acked Read 3 Sent Done Done false
15404 5237 Acked Write 8 Sent Done Done false
15405 5238 Acked Write 8 Sent Done Done false
15406 5239 Acked Write 3 Sent Done Done false
15407 5240 Acked Write 2 Sent Done Done false
15408 5241 Acked Write 8 Sent Done Done false
15409 5242 Acked Write 6 Sent Done Done false
15410 5243 Acked Read 4 Sent Done Done false
15411 5244 Acked Read 4 Sent Done Done false
15412 5245 Acked Read 1 Sent Done Done false
15413 5246 Acked Read 6 Sent Done Done false
15414 5247 Acked Write 4 Sent Done Done false
15415 5248 Acked Read 3 Sent Done Done false
15416 5249 Acked Read 10 Sent Done Done false
15417 5250 Acked Read 4 Sent Done Done false
15418 5251 Acked Write 2 Sent Done Done false
15419 5252 Acked Write 3 Sent Done Done false
15420 5253 Acked Read 10 Sent Done Done false
15421 5254 Acked Flush 0 Sent Done Done false
15422 5255 Acked Read 4 Sent Done Done false
15423 5256 Acked Read 4 Sent Done Done false
15424 5257 Acked Read 3 Sent Done Done false
15425 5258 Acked Flush 0 Sent Done Done false
15426 5259 Acked Write 3 Sent Done Done false
15427 5260 Acked Read 5 Sent Done Done false
15428 5261 Acked Write 3 Sent Done Done false
15429 5262 Acked Read 3 Sent Done Done false
15430 5263 Acked Read 2 Sent Done Done false
15431 5264 Acked Read 7 Sent Done Done false
15432 5265 Acked Read 6 Sent Done Done false
15433 5266 Acked Read 3 Sent Done Done false
15434 5267 Acked Flush 0 Sent Done Done false
15435 5268 Acked Write 7 Sent Done Done false
15436 5269 Acked Read 7 Sent Done Done false
15437 5270 Acked Write 6 Sent Done Done false
15438 5271 Acked Read 4 Sent Done Done false
15439 5272 Acked Flush 0 Sent Done Done false
15440 5273 Acked Write 8 Sent Done Done false
15441 5274 Acked Read 7 Sent Done Done false
15442 5275 Acked Write 5 Sent Done Done false
15443 5276 Acked Write 10 Sent Done Done false
15444 5277 Acked Write 7 Sent Done Done false
15445 5278 Acked Write 10 Sent Done Done false
15446 5279 Acked Read 3 Sent Done Done false
15447 5280 Acked Write 1 Sent Done Done false
15448 5281 Acked Read 5 Sent Done Done false
15449 5282 Acked Read 4 Sent Done Done false
15450 5283 Acked Read 7 Sent Done Done false
15451 5284 Acked Read 3 Sent Done Done false
15452 5285 Acked Read 4 Sent Done Done false
15453 5286 Acked Write 2 Sent Done Done false
15454 5287 Acked Write 8 Sent Done Done false
15455 5288 Acked Flush 0 Sent Done Done false
15456 5289 Acked Write 6 Sent Done Done false
15457 5290 Acked Read 6 Sent Done Done false
15458 5291 Acked Read 6 Sent Done Done false
15459 5292 Acked Flush 0 Sent Done Done false
15460 5293 Acked Write 9 Sent Done Done false
15461 5294 Acked Read 8 Sent Done Done false
15462 5295 Acked Read 7 Sent Done Done false
15463 5296 Acked Read 6 Sent Done Done false
15464 5297 Acked Write 5 Sent Done Done false
15465 5298 Acked Write 9 Sent Done Done false
15466 5299 Acked Read 3 Sent Done Done false
15467 5300 Acked Read 6 Sent Done Done false
15468 5301 Acked Read 9 Sent Done Done false
15469 5302 Acked Write 4 Sent Done Done false
15470 5303 Acked Read 9 Sent Done Done false
15471 5304 Acked Write 1 Sent Done Done false
15472 5305 Acked Read 8 Sent Done Done false
15473 5306 Acked Read 8 Sent Done Done false
15474 5307 Acked Write 6 Sent Done Done false
15475 5308 Acked Read 9 Sent Done Done false
15476 5309 Acked Write 3 Sent Done Done false
15477 5310 Acked Read 7 Sent Done Done false
15478 5311 Acked Write 5 Sent Done Done false
15479 5312 Acked Read 4 Sent Done Done false
15480 5313 Acked Write 4 Sent Done Done false
15481 5314 Acked Read 2 Sent Done Done false
15482 5315 Acked Read 9 Sent Done Done false
15483 5316 Acked Flush 0 Sent Done Done false
15484 5317 Acked Write 9 Sent Done Done false
15485 5318 Acked Flush 0 Sent Done Done false
15486 5319 Acked Write 2 Sent Done Done false
15487 5320 Acked Write 1 Sent Done Done false
15488 5321 Acked Flush 0 Sent Done Done false
15489 5322 Acked Read 8 Sent Done Done false
15490 5323 Acked Read 3 Sent Done Done false
15491 5324 Acked Read 8 Sent Done Done false
15492 5325 Acked Read 5 Sent Done Done false
15493 5326 Acked Flush 0 Sent Done Done false
15494 5327 Acked Read 1 Sent Done Done false
15495 5328 Acked Read 7 Sent Done Done false
15496 5329 Acked Read 2 Sent Done Done false
15497 5330 Acked Read 6 Sent Done Done false
15498 5331 Acked Write 9 Sent Done Done false
15499 5332 Acked Write 6 Sent Done Done false
15500 5333 Acked Read 4 Sent Done Done false
15501 5334 Acked Read 6 Sent Done Done false
15502 5335 Acked Write 2 Sent Done Done false
15503 5336 Acked Read 5 Sent Done Done false
15504 5337 Acked Flush 0 Sent Done Done false
15505 5338 Acked Read 3 Sent Done Done false
15506 5339 Acked Write 8 Sent Done Done false
15507 5340 Acked Write 8 Sent Done Done false
15508 5341 Acked Write 4 Sent Done Done false
15509 5342 Acked Read 6 Sent Done Done false
15510 5343 Acked Flush 0 Sent Done Done false
15511 5344 Acked Write 2 Sent Done Done false
15512 5345 Acked Read 7 Sent Done Done false
15513 5346 Acked Write 5 Sent Done Done false
15514 5347 Acked Write 7 Sent Done Done false
15515 5348 Acked Read 8 Sent Done Done false
15516 5349 Acked Read 8 Sent Done Done false
15517 5350 Acked Write 10 Sent Done Done false
15518 5351 Acked Write 10 Sent Done Done false
15519 5352 Acked Write 4 Sent Done Done false
15520 5353 Acked Read 5 Sent Done Done false
15521 5354 Acked Write 7 Sent Done Done false
15522 5355 Acked Flush 0 Sent Done Done false
15523 5356 Acked Write 4 Sent Done Done false
15524 5357 Acked Flush 0 Sent Done Done false
15525 5358 Acked Flush 0 Sent Done Done false
15526 5359 Acked Read 3 Sent Done Done false
15527 5360 Acked Write 1 Sent Done Done false
15528 5361 Acked Read 9 Sent Done Done false
15529 5362 Acked Write 8 Sent Done Done false
15530 5363 Acked Read 7 Sent Done Done false
15531 5364 Acked Write 1 Sent Done Done false
15532 5365 Acked Read 6 Sent Done Done false
15533 5366 Acked Write 2 Sent Done Done false
15534 5367 Acked Write 8 Sent Done Done false
15535 5368 Acked Read 7 Sent Done Done false
15536 5369 Acked Flush 0 Sent Done Done false
15537 5370 Acked Flush 0 Sent Done Done false
15538 5371 Acked Flush 0 Sent Done Done false
15539 5372 Acked Write 5 Sent Done Done false
15540 5373 Acked Read 6 Sent Done Done false
15541 5374 Acked Read 1 Sent Done Done false
15542 5375 Acked Write 6 Sent Done Done false
15543 5376 Acked Write 2 Sent Done Done false
15544 5377 Acked Write 10 Sent Done Done false
15545 5378 Acked Read 2 Sent Done Done false
15546 5379 Acked Read 2 Sent Done Done false
15547 5380 Acked Read 6 Sent Done Done false
15548 5381 Acked Write 9 Sent Done Done false
15549 5382 Acked Write 4 Sent Done Done false
15550 5383 Acked Read 8 Sent Done Done false
15551 5384 Acked Write 8 Sent Done Done false
15552 5385 Acked Write 6 Sent Done Done false
15553 5386 Acked Write 10 Sent Done Done false
15554 5387 Acked Write 3 Sent Done Done false
15555 5388 Acked Read 9 Sent Done Done false
15556 5389 Acked Write 9 Sent Done Done false
15557 5390 Acked Flush 0 Sent Done Done false
15558 5391 Acked Read 2 Sent Done Done false
15559 5392 Acked Write 7 Sent Done Done false
15560 5393 Acked Read 5 Sent Done Done false
15561 5394 Acked Read 3 Sent Done Done false
15562 5395 Acked Read 2 Sent Done Done false
15563 5396 Acked Write 4 Sent Done Done false
15564 5397 Acked Read 4 Sent Done Done false
15565 5398 Acked Flush 0 Sent Done Done false
15566 5399 Acked Flush 0 Sent Done Done false
15567 5400 Acked Read 4 Sent Done Done false
15568 5401 Acked Write 1 Sent Done Done false
15569 5402 Acked Write 6 Sent Done Done false
15570 5403 Acked Read 5 Sent Done Done false
15571 5404 Acked Write 10 Sent Done Done false
15572 5405 Acked Write 8 Sent Done Done false
15573 5406 Acked Flush 0 Sent Done Done false
15574 5407 Acked Read 7 Sent Done Done false
15575 5408 Acked Flush 0 Sent Done Done false
15576 5409 Acked Read 6 Sent Done Done false
15577 5410 Acked Read 7 Sent Done Done false
15578 5411 Acked Read 6 Sent Done Done false
15579 5412 Acked Write 6 Sent Done Done false
15580 5413 Acked Read 4 Sent Done Done false
15581 5414 Acked Write 4 Sent Done Done false
15582 5415 Acked Read 1 Sent Done Done false
15583 5416 Acked Write 2 Sent Done Done false
15584 5417 Acked Read 3 Sent Done Done false
15585 5418 Acked Read 7 Sent Done Done false
15586 5419 Acked Write 3 Sent Done Done false
15587 5420 Acked Read 1 Sent Done Done false
15588 5421 Acked Read 2 Sent Done Done false
15589 5422 Acked Read 5 Sent Done Done false
15590 5423 Acked Write 7 Sent Done Done false
15591 5424 Acked Read 4 Sent Done Done false
15592 5425 Acked Write 8 Sent Done Done false
15593 5426 Acked Read 4 Sent Done Done false
15594 5427 Acked Write 10 Sent Done Done false
15595 5428 Acked Read 1 Sent Done Done false
15596 5429 Acked Read 4 Sent Done Done false
15597 5430 Acked Write 8 Sent Done Done false
15598 5431 Acked Read 5 Sent Done Done false
15599 5432 Acked Read 10 Sent Done Done false
15600 5433 Acked Read 10 Sent Done Done false
15601 5434 Acked Write 7 Sent Done Done false
15602 5435 Acked Read 7 Sent Done Done false
15603 5436 Acked Read 6 Sent Done Done false
15604 5437 Acked Read 9 Sent Done Done false
15605 5438 Acked Read 6 Sent Done Done false
15606 5439 Acked Flush 0 Sent Done Done false
15607 5440 Acked Flush 0 Sent Done Done false
15608 5441 Acked Read 6 Sent Done Done false
15609 5442 Acked Read 9 Sent Done Done false
15610 5443 Acked Flush 0 Sent Done Done false
15611 5444 Acked Read 1 Sent Done Done false
15612 5445 Acked Write 3 Sent Done Done false
15613 5446 Acked Write 9 Sent Done Done false
15614 5447 Acked Read 1 Sent Done Done false
15615 5448 Acked Read 1 Sent Done Done false
15616 5449 Acked Read 3 Sent Done Done false
15617 5450 Acked Read 8 Sent Done Done false
15618 5451 Acked Read 4 Sent Done Done false
15619 5452 Acked Read 10 Sent Done Done false
15620 5453 Acked Read 1 Sent Done Done false
15621 5454 Acked Write 4 Sent Done Done false
15622 5455 Acked Write 2 Sent Done Done false
15623 5456 Acked Read 4 Sent Done Done false
15624 5457 Acked Read 6 Sent Done Done false
15625 5458 Acked Read 5 Sent Done Done false
15626 5459 Acked Read 7 Sent Done Done false
15627 5460 Acked Write 5 Sent Done Done false
15628 5461 Acked Write 7 Sent Done Done false
15629 5462 Acked Read 9 Sent Done Done false
15630 5463 Acked Read 7 Sent Done Done false
15631 5464 Acked Write 1 Sent Done Done false
15632 5465 Acked Write 1 Sent Done Done false
15633 5466 Acked Write 4 Sent Done Done false
15634 5467 Acked Write 1 Sent Done Done false
15635 5468 Acked Read 6 Sent Done Done false
15636 5469 Acked Read 10 Sent Done Done false
15637 5470 Acked Read 6 Sent Done Done false
15638 5471 Acked Write 7 Sent Done Done false
15639 5472 Acked Flush 0 Sent Done Done false
15640 5473 Acked Write 10 Sent Done Done false
15641 5474 Acked Write 9 Sent Done Done false
15642 5475 Acked Write 1 Sent Done Done false
15643 5476 Acked Write 10 Sent Done Done false
15644 5477 Acked Read 7 Sent Done Done false
15645 5478 Acked Read 7 Sent Done Done false
15646 5479 Acked Read 6 Sent Done Done false
15647 5480 Acked Write 10 Sent Done Done false
15648 5481 Acked Write 1 Sent Done Done false
15649 5482 Acked Read 3 Sent Done Done false
15650 5483 Acked Write 6 Sent Done Done false
15651 5484 Acked Write 9 Sent Done Done false
15652 5485 Acked Write 10 Sent Done Done false
15653 5486 Acked Read 1 Sent Done Done false
15654 5487 Acked Read 10 Sent Done Done false
15655 5488 Acked Write 8 Sent Done Done false
15656 5489 Acked Read 1 Sent Done Done false
15657 5490 Acked Write 9 Sent Done Done false
15658 5491 Acked Read 4 Sent Done Done false
15659 5492 Acked Flush 0 Sent Done Done false
15660 5493 Acked Write 10 Sent Done Done false
15661 5494 Acked Read 9 Sent Done Done false
15662 5495 Acked Write 3 Sent Done Done false
15663 5496 Acked Write 1 Sent Done Done false
15664 5497 Acked Read 2 Sent Done Done false
15665 5498 Acked Write 5 Sent Done Done false
15666 5499 Acked Write 3 Sent Done Done false
15667 5500 Acked Write 2 Sent Done Done false
15668 5501 Acked Read 4 Sent Done Done false
15669 5502 Acked Read 7 Sent Done Done false
15670 5503 Acked Read 4 Sent Done Done false
15671 5504 Acked Read 4 Sent Done Done false
15672 5505 Acked Write 5 Sent Done Done false
15673 5506 Acked Write 6 Sent Done Done false
15674 5507 Acked Write 4 Sent Done Done false
15675 5508 Acked Flush 0 Sent Done Done false
15676 5509 Acked Read 9 Sent Done Done false
15677 5510 Acked Read 2 Sent Done Done false
15678 5511 Acked Read 8 Sent Done Done false
15679 5512 Acked Read 6 Sent Done Done false
15680 5513 Acked Read 5 Sent Done Done false
15681 5514 Acked Read 5 Sent Done Done false
15682 5515 Acked Flush 0 Sent Done Done false
15683 5516 Acked Read 1 Sent Done Done false
15684 5517 Acked Read 1 Sent Done Done false
15685 5518 Acked Read 10 Sent Done Done false
15686 5519 Acked Read 10 Sent Done Done false
15687 5520 Acked Read 9 Sent Done Done false
15688 5521 Acked Flush 0 Sent Done Done false
15689 5522 Acked Write 6 Sent Done Done false
15690 5523 Acked Read 9 Sent Done Done false
15691 5524 Acked Read 3 Sent Done Done false
15692 5525 Acked Flush 0 Sent Done Done false
15693 5526 Acked Read 2 Sent Done Done false
15694 5527 Acked Flush 0 Sent Done Done false
15695 STATES DS:0 DS:1 DS:2 TOTAL
15696 Sent 302 0 0 302
15697 Done 0 302 302 604
15698 Skipped 0 0 0 0
15699 Error 0 0 0 0
15700 Last Flush: 5225 5527 5527
15701 Downstairs last five completed: 5225 5224 5223 5222 5221
15702 Upstairs last five completed: 5527 5526 5525 5524 5523
15703 CLIENT: Up:0 ds:302 act:2
15704 ----------------------------------------------------------------
15705 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15706 States: Offline Active Active
15707 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15708 5226 Acked Write 5 Sent Done Done false
15709 5227 Acked Write 2 Sent Done Done false
15710 5228 Acked Write 6 Sent Done Done false
15711 5229 Acked Flush 0 Sent Done Done false
15712 5230 Acked Read 4 Sent Done Done false
15713 5231 Acked Read 6 Sent Done Done false
15714 5232 Acked Read 3 Sent Done Done false
15715 5233 Acked Read 6 Sent Done Done false
15716 5234 Acked Flush 0 Sent Done Done false
15717 5235 Acked Read 8 Sent Done Done false
15718 5236 Acked Read 3 Sent Done Done false
15719 5237 Acked Write 8 Sent Done Done false
15720 5238 Acked Write 8 Sent Done Done false
15721 5239 Acked Write 3 Sent Done Done false
15722 5240 Acked Write 2 Sent Done Done false
15723 5241 Acked Write 8 Sent Done Done false
15724 5242 Acked Write 6 Sent Done Done false
15725 5243 Acked Read 4 Sent Done Done false
15726 5244 Acked Read 4 Sent Done Done false
15727 5245 Acked Read 1 Sent Done Done false
15728 5246 Acked Read 6 Sent Done Done false
15729 5247 Acked Write 4 Sent Done Done false
15730 5248 Acked Read 3 Sent Done Done false
15731 5249 Acked Read 10 Sent Done Done false
15732 5250 Acked Read 4 Sent Done Done false
15733 5251 Acked Write 2 Sent Done Done false
15734 5252 Acked Write 3 Sent Done Done false
15735 5253 Acked Read 10 Sent Done Done false
15736 5254 Acked Flush 0 Sent Done Done false
15737 5255 Acked Read 4 Sent Done Done false
15738 5256 Acked Read 4 Sent Done Done false
15739 5257 Acked Read 3 Sent Done Done false
15740 5258 Acked Flush 0 Sent Done Done false
15741 5259 Acked Write 3 Sent Done Done false
15742 5260 Acked Read 5 Sent Done Done false
15743 5261 Acked Write 3 Sent Done Done false
15744 5262 Acked Read 3 Sent Done Done false
15745 5263 Acked Read 2 Sent Done Done false
15746 5264 Acked Read 7 Sent Done Done false
15747 5265 Acked Read 6 Sent Done Done false
15748 5266 Acked Read 3 Sent Done Done false
15749 5267 Acked Flush 0 Sent Done Done false
15750 5268 Acked Write 7 Sent Done Done false
15751 5269 Acked Read 7 Sent Done Done false
15752 5270 Acked Write 6 Sent Done Done false
15753 5271 Acked Read 4 Sent Done Done false
15754 5272 Acked Flush 0 Sent Done Done false
15755 5273 Acked Write 8 Sent Done Done false
15756 5274 Acked Read 7 Sent Done Done false
15757 5275 Acked Write 5 Sent Done Done false
15758 5276 Acked Write 10 Sent Done Done false
15759 5277 Acked Write 7 Sent Done Done false
15760 5278 Acked Write 10 Sent Done Done false
15761 5279 Acked Read 3 Sent Done Done false
15762 5280 Acked Write 1 Sent Done Done false
15763 5281 Acked Read 5 Sent Done Done false
15764 5282 Acked Read 4 Sent Done Done false
15765 5283 Acked Read 7 Sent Done Done false
15766 5284 Acked Read 3 Sent Done Done false
15767 5285 Acked Read 4 Sent Done Done false
15768 5286 Acked Write 2 Sent Done Done false
15769 5287 Acked Write 8 Sent Done Done false
15770 5288 Acked Flush 0 Sent Done Done false
15771 5289 Acked Write 6 Sent Done Done false
15772 5290 Acked Read 6 Sent Done Done false
15773 5291 Acked Read 6 Sent Done Done false
15774 5292 Acked Flush 0 Sent Done Done false
15775 5293 Acked Write 9 Sent Done Done false
15776 5294 Acked Read 8 Sent Done Done false
15777 5295 Acked Read 7 Sent Done Done false
15778 5296 Acked Read 6 Sent Done Done false
15779 5297 Acked Write 5 Sent Done Done false
15780 5298 Acked Write 9 Sent Done Done false
15781 5299 Acked Read 3 Sent Done Done false
15782 5300 Acked Read 6 Sent Done Done false
15783 5301 Acked Read 9 Sent Done Done false
15784 5302 Acked Write 4 Sent Done Done false
15785 5303 Acked Read 9 Sent Done Done false
15786 5304 Acked Write 1 Sent Done Done false
15787 5305 Acked Read 8 Sent Done Done false
15788 5306 Acked Read 8 Sent Done Done false
15789 5307 Acked Write 6 Sent Done Done false
15790 5308 Acked Read 9 Sent Done Done false
15791 5309 Acked Write 3 Sent Done Done false
15792 5310 Acked Read 7 Sent Done Done false
15793 5311 Acked Write 5 Sent Done Done false
15794 5312 Acked Read 4 Sent Done Done false
15795 5313 Acked Write 4 Sent Done Done false
15796 5314 Acked Read 2 Sent Done Done false
15797 5315 Acked Read 9 Sent Done Done false
15798 5316 Acked Flush 0 Sent Done Done false
15799 5317 Acked Write 9 Sent Done Done false
15800 5318 Acked Flush 0 Sent Done Done false
15801 5319 Acked Write 2 Sent Done Done false
15802 5320 Acked Write 1 Sent Done Done false
15803 5321 Acked Flush 0 Sent Done Done false
15804 5322 Acked Read 8 Sent Done Done false
15805 5323 Acked Read 3 Sent Done Done false
15806 5324 Acked Read 8 Sent Done Done false
15807 5325 Acked Read 5 Sent Done Done false
15808 5326 Acked Flush 0 Sent Done Done false
15809 5327 Acked Read 1 Sent Done Done false
15810 5328 Acked Read 7 Sent Done Done false
15811 5329 Acked Read 2 Sent Done Done false
15812 5330 Acked Read 6 Sent Done Done false
15813 5331 Acked Write 9 Sent Done Done false
15814 5332 Acked Write 6 Sent Done Done false
15815 5333 Acked Read 4 Sent Done Done false
15816 5334 Acked Read 6 Sent Done Done false
15817 5335 Acked Write 2 Sent Done Done false
15818 5336 Acked Read 5 Sent Done Done false
15819 5337 Acked Flush 0 Sent Done Done false
15820 5338 Acked Read 3 Sent Done Done false
15821 5339 Acked Write 8 Sent Done Done false
15822 5340 Acked Write 8 Sent Done Done false
15823 5341 Acked Write 4 Sent Done Done false
15824 5342 Acked Read 6 Sent Done Done false
15825 5343 Acked Flush 0 Sent Done Done false
15826 5344 Acked Write 2 Sent Done Done false
15827 5345 Acked Read 7 Sent Done Done false
15828 5346 Acked Write 5 Sent Done Done false
15829 5347 Acked Write 7 Sent Done Done false
15830 5348 Acked Read 8 Sent Done Done false
15831 5349 Acked Read 8 Sent Done Done false
15832 5350 Acked Write 10 Sent Done Done false
15833 5351 Acked Write 10 Sent Done Done false
15834 5352 Acked Write 4 Sent Done Done false
15835 5353 Acked Read 5 Sent Done Done false
15836 5354 Acked Write 7 Sent Done Done false
15837 5355 Acked Flush 0 Sent Done Done false
15838 5356 Acked Write 4 Sent Done Done false
15839 5357 Acked Flush 0 Sent Done Done false
15840 5358 Acked Flush 0 Sent Done Done false
15841 5359 Acked Read 3 Sent Done Done false
15842 5360 Acked Write 1 Sent Done Done false
15843 5361 Acked Read 9 Sent Done Done false
15844 5362 Acked Write 8 Sent Done Done false
15845 5363 Acked Read 7 Sent Done Done false
15846 5364 Acked Write 1 Sent Done Done false
15847 5365 Acked Read 6 Sent Done Done false
15848 5366 Acked Write 2 Sent Done Done false
15849 5367 Acked Write 8 Sent Done Done false
15850 5368 Acked Read 7 Sent Done Done false
15851 5369 Acked Flush 0 Sent Done Done false
15852 5370 Acked Flush 0 Sent Done Done false
15853 5371 Acked Flush 0 Sent Done Done false
15854 5372 Acked Write 5 Sent Done Done false
15855 5373 Acked Read 6 Sent Done Done false
15856 5374 Acked Read 1 Sent Done Done false
15857 5375 Acked Write 6 Sent Done Done false
15858 5376 Acked Write 2 Sent Done Done false
15859 5377 Acked Write 10 Sent Done Done false
15860 5378 Acked Read 2 Sent Done Done false
15861 5379 Acked Read 2 Sent Done Done false
15862 5380 Acked Read 6 Sent Done Done false
15863 5381 Acked Write 9 Sent Done Done false
15864 5382 Acked Write 4 Sent Done Done false
15865 5383 Acked Read 8 Sent Done Done false
15866 5384 Acked Write 8 Sent Done Done false
15867 5385 Acked Write 6 Sent Done Done false
15868 5386 Acked Write 10 Sent Done Done false
15869 5387 Acked Write 3 Sent Done Done false
15870 5388 Acked Read 9 Sent Done Done false
15871 5389 Acked Write 9 Sent Done Done false
15872 5390 Acked Flush 0 Sent Done Done false
15873 5391 Acked Read 2 Sent Done Done false
15874 5392 Acked Write 7 Sent Done Done false
15875 5393 Acked Read 5 Sent Done Done false
15876 5394 Acked Read 3 Sent Done Done false
15877 5395 Acked Read 2 Sent Done Done false
15878 5396 Acked Write 4 Sent Done Done false
15879 5397 Acked Read 4 Sent Done Done false
15880 5398 Acked Flush 0 Sent Done Done false
15881 5399 Acked Flush 0 Sent Done Done false
15882 5400 Acked Read 4 Sent Done Done false
15883 5401 Acked Write 1 Sent Done Done false
15884 5402 Acked Write 6 Sent Done Done false
15885 5403 Acked Read 5 Sent Done Done false
15886 5404 Acked Write 10 Sent Done Done false
15887 5405 Acked Write 8 Sent Done Done false
15888 5406 Acked Flush 0 Sent Done Done false
15889 5407 Acked Read 7 Sent Done Done false
15890 5408 Acked Flush 0 Sent Done Done false
15891 5409 Acked Read 6 Sent Done Done false
15892 5410 Acked Read 7 Sent Done Done false
15893 5411 Acked Read 6 Sent Done Done false
15894 5412 Acked Write 6 Sent Done Done false
15895 5413 Acked Read 4 Sent Done Done false
15896 5414 Acked Write 4 Sent Done Done false
15897 5415 Acked Read 1 Sent Done Done false
15898 5416 Acked Write 2 Sent Done Done false
15899 5417 Acked Read 3 Sent Done Done false
15900 5418 Acked Read 7 Sent Done Done false
15901 5419 Acked Write 3 Sent Done Done false
15902 5420 Acked Read 1 Sent Done Done false
15903 5421 Acked Read 2 Sent Done Done false
15904 5422 Acked Read 5 Sent Done Done false
15905 5423 Acked Write 7 Sent Done Done false
15906 5424 Acked Read 4 Sent Done Done false
15907 5425 Acked Write 8 Sent Done Done false
15908 5426 Acked Read 4 Sent Done Done false
15909 5427 Acked Write 10 Sent Done Done false
15910 5428 Acked Read 1 Sent Done Done false
15911 5429 Acked Read 4 Sent Done Done false
15912 5430 Acked Write 8 Sent Done Done false
15913 5431 Acked Read 5 Sent Done Done false
15914 5432 Acked Read 10 Sent Done Done false
15915 5433 Acked Read 10 Sent Done Done false
15916 5434 Acked Write 7 Sent Done Done false
15917 5435 Acked Read 7 Sent Done Done false
15918 5436 Acked Read 6 Sent Done Done false
15919 5437 Acked Read 9 Sent Done Done false
15920 5438 Acked Read 6 Sent Done Done false
15921 5439 Acked Flush 0 Sent Done Done false
15922 5440 Acked Flush 0 Sent Done Done false
15923 5441 Acked Read 6 Sent Done Done false
15924 5442 Acked Read 9 Sent Done Done false
15925 5443 Acked Flush 0 Sent Done Done false
15926 5444 Acked Read 1 Sent Done Done false
15927 5445 Acked Write 3 Sent Done Done false
15928 5446 Acked Write 9 Sent Done Done false
15929 5447 Acked Read 1 Sent Done Done false
15930 5448 Acked Read 1 Sent Done Done false
15931 5449 Acked Read 3 Sent Done Done false
15932 5450 Acked Read 8 Sent Done Done false
15933 5451 Acked Read 4 Sent Done Done false
15934 5452 Acked Read 10 Sent Done Done false
15935 5453 Acked Read 1 Sent Done Done false
15936 5454 Acked Write 4 Sent Done Done false
15937 5455 Acked Write 2 Sent Done Done false
15938 5456 Acked Read 4 Sent Done Done false
15939 5457 Acked Read 6 Sent Done Done false
15940 5458 Acked Read 5 Sent Done Done false
15941 5459 Acked Read 7 Sent Done Done false
15942 5460 Acked Write 5 Sent Done Done false
15943 5461 Acked Write 7 Sent Done Done false
15944 5462 Acked Read 9 Sent Done Done false
15945 5463 Acked Read 7 Sent Done Done false
15946 5464 Acked Write 1 Sent Done Done false
15947 5465 Acked Write 1 Sent Done Done false
15948 5466 Acked Write 4 Sent Done Done false
15949 5467 Acked Write 1 Sent Done Done false
15950 5468 Acked Read 6 Sent Done Done false
15951 5469 Acked Read 10 Sent Done Done false
15952 5470 Acked Read 6 Sent Done Done false
15953 5471 Acked Write 7 Sent Done Done false
15954 5472 Acked Flush 0 Sent Done Done false
15955 5473 Acked Write 10 Sent Done Done false
15956 5474 Acked Write 9 Sent Done Done false
15957 5475 Acked Write 1 Sent Done Done false
15958 5476 Acked Write 10 Sent Done Done false
15959 5477 Acked Read 7 Sent Done Done false
15960 5478 Acked Read 7 Sent Done Done false
15961 5479 Acked Read 6 Sent Done Done false
15962 5480 Acked Write 10 Sent Done Done false
15963 5481 Acked Write 1 Sent Done Done false
15964 5482 Acked Read 3 Sent Done Done false
15965 5483 Acked Write 6 Sent Done Done false
15966 5484 Acked Write 9 Sent Done Done false
15967 5485 Acked Write 10 Sent Done Done false
15968 5486 Acked Read 1 Sent Done Done false
15969 5487 Acked Read 10 Sent Done Done false
15970 5488 Acked Write 8 Sent Done Done false
15971 5489 Acked Read 1 Sent Done Done false
15972 5490 Acked Write 9 Sent Done Done false
15973 5491 Acked Read 4 Sent Done Done false
15974 5492 Acked Flush 0 Sent Done Done false
15975 5493 Acked Write 10 Sent Done Done false
15976 5494 Acked Read 9 Sent Done Done false
15977 5495 Acked Write 3 Sent Done Done false
15978 5496 Acked Write 1 Sent Done Done false
15979 5497 Acked Read 2 Sent Done Done false
15980 5498 Acked Write 5 Sent Done Done false
15981 5499 Acked Write 3 Sent Done Done false
15982 5500 Acked Write 2 Sent Done Done false
15983 5501 Acked Read 4 Sent Done Done false
15984 5502 Acked Read 7 Sent Done Done false
15985 5503 Acked Read 4 Sent Done Done false
15986 5504 Acked Read 4 Sent Done Done false
15987 5505 Acked Write 5 Sent Done Done false
15988 5506 Acked Write 6 Sent Done Done false
15989 5507 Acked Write 4 Sent Done Done false
15990 5508 Acked Flush 0 Sent Done Done false
15991 5509 Acked Read 9 Sent Done Done false
15992 5510 Acked Read 2 Sent Done Done false
15993 5511 Acked Read 8 Sent Done Done false
15994 5512 Acked Read 6 Sent Done Done false
15995 5513 Acked Read 5 Sent Done Done false
15996 5514 Acked Read 5 Sent Done Done false
15997 5515 Acked Flush 0 Sent Done Done false
15998 5516 Acked Read 1 Sent Done Done false
15999 5517 Acked Read 1 Sent Done Done false
16000 5518 Acked Read 10 Sent Done Done false
16001 5519 Acked Read 10 Sent Done Done false
16002 5520 Acked Read 9 Sent Done Done false
16003 5521 Acked Flush 0 Sent Done Done false
16004 5522 Acked Write 6 Sent Done Done false
16005 5523 Acked Read 9 Sent Done Done false
16006 5524 Acked Read 3 Sent Done Done false
16007 5525 Acked Flush 0 Sent Done Done false
16008 5526 Acked Read 2 Sent Done Done false
16009 5527 Acked Flush 0 Sent Done Done false
16010 STATES DS:0 DS:1 DS:2 TOTAL
16011 Sent 302 0 0 302
16012 Done 0 302 302 604
16013 Skipped 0 0 0 0
16014 Error 0 0 0 0
16015 Last Flush: 5225 5527 5527
16016 Downstairs last five completed: 5225 5224 5223 5222 5221
16017 Upstairs last five completed: 5527 5526 5525 5524 5523
16018 CLIENT: Up:0 ds:302 act:2
160192025-01-14T16:13:59.338ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
160202025-01-14T16:13:59.338ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61424) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
160212025-01-14T16:13:59.339ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
160222025-01-14T16:13:59.340ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
160232025-01-14T16:13:59.340ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
160242025-01-14T16:13:59.340ZINFOcrucible: send last flush ID to this DS: 5225 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
160252025-01-14T16:13:59.340ZINFOcrucible: Replied this last flush ID: 5225 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
160262025-01-14T16:13:59.340ZINFOcrucible: [0] client re-new 302 jobs since flush 5225 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
160272025-01-14T16:13:59.341ZINFOcrucible: [0] Replayed 302 jobs since flush: 5225 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
16028 ----------------------------------------------------------------
16029 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
16030 Downstairs last five completed: 5527 5526 5525 5524 5523
16031 Upstairs last five completed: 5527 5526 5525 5524 5523
16032 CLIENT: Up:0 ds:0 act:3
16033 Replay: All jobs finished
16034 Main task has work to do, go find it
16035 got dsc Stop(0)
16036 stop 0
16037 Jan 14 16:13:59.586 INFO request completed, latency_us: 335, response_code: 204, uri: /stop/cid/0, method: POST, req_id: b3e45595-307b-47e6-b94c-ae95184c1522, remote_addr: 127.0.0.1:61942, local_addr:[8810] Got stop action so:false kr:false
16038 127.0.0.1:9998
16039 Jan 14 16:13:59.586 INFO request completed, latency_us: 125, response_code: 200, uri: /state/cid/0, method: GET, req_id: 677b7bd9-570f-4108-973b-525fceefe57c, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
160402025-01-14T16:13:59.586ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
160412025-01-14T16:13:59.586ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
160422025-01-14T16:13:59.586ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
160432025-01-14T16:13:59.587ZINFOcrucible: client task is exiting = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
16044 [8810] Exited with: ExitStatus(unix_wait_status(9))
16045 [8810][0] reports Exit
16046 Jan 14 16:14:03.589 INFO request completed, latency_us: 199, response_code: 200, uri: /state/cid/0, method: GET, req_id: b83b27f1-43d2-4335-96c8-207f8437ed50, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
16047 001/300 Read block 2990 len 32768
16048 002/300 Read block 1061 len 4096
16049 003/300 Write block 3292 len 16384 data: 3 4 5 5
16050 004/300 Read block 2774 len 20480
16051 005/300 Read block 2913 len 32768
16052 006/300 Read block 361 len 20480
16053 007/300 Write block 3683 len 32768 data: 5 5 5 4 4 6 6 5
16054 008/300 Write block 4722 len 32768 data: 3 3 3 3 2 2 2 2
16055 009/300 Read block 1801 len 20480
16056 010/300 Read block 2441 len 40960
16057 011/300 Write block 4952 len 4096 data: 2
16058 012/300 Write block 3448 len 40960 data: 3 3 4 4 3 3 4 4 4 4
16059 013/300 Read block 1905 len 16384
16060 014/300 Write block 2303 len 20480 data: 3 3 3 3 3
16061 015/300 Read block 868 len 32768
16062 016/300 Read block 1339 len 8192
16063 017/300 Read block 4898 len 32768
16064 018/300 Read block 3460 len 20480
16065 019/300 Read block 2344 len 8192
16066 020/300 Write block 557 len 36864 data: 5 5 6 7 5 4 4 5 3
16067 021/300 Read block 561 len 20480
16068 022/300 Read block 4925 len 16384
16069 023/300 Write block 4496 len 40960 data: 3 3 3 3 4 4 4 2 2 3
16070 024/300 Write block 1862 len 8192 data: 3 2
16071 025/300 Read block 450 len 28672
16072 026/300 Read block 2628 len 4096
16073 027/300 Read block 106 len 28672
16074 028/300 Write block 4912 len 24576 data: 2 2 3 3 4 4
16075 029/300 Read block 2010 len 40960
16076 030/300 Read block 4486 len 28672
16077 031/300 Write block 4790 len 24576 data: 5 4 3 4 4 4
16078 032/300 Read block 327 len 24576
16079 033/300 Read block 3688 len 40960
16080 034/300 Read block 795 len 16384
16081 035/300 Write block 1305 len 40960 data: 5 4 3 3 2 2 2 3 3 3
16082 036/300 Read block 3303 len 20480
16083 037/300 Read block 693 len 32768
16084 038/300 Read block 414 len 36864
16085 039/300 Read block 1420 len 36864
16086 040/300 Write block 1327 len 24576 data: 4 5 4 4 5 5
16087 041/300 Write block 2051 len 8192 data: 4 3
16088 042/300 Write block 4097 len 16384 data: 4 5 5 4
16089 043/300 Read block 1416 len 32768
16090 044/300 Write block 3961 len 8192 data: 5 5
16091 045/300 Read block 2293 len 8192
16092 046/300 Write block 2647 len 16384 data: 4 4 4 4
16093 047/300 Write block 4041 len 32768 data: 2 2 3 4 4 5 4 4
16094 048/300 Write block 4386 len 4096 data: 4
16095 049/300 Write block 900 len 4096 data: 4
16096 050/300 Read block 144 len 8192
16097 051/300 Flush
16098 052/300 Flush
16099 053/300 Write block 4840 len 12288 data: 5 5 7
16100 054/300 Write block 2306 len 12288 data: 4 4 3
16101 055/300 Flush
16102 056/300 Write block 4954 len 40960 data: 4 4 5 5 5 5 4 2 2 4
16103 057/300 Write block 1760 len 32768 data: 3 2 2 2 3 3 3 3
16104 058/300 Read block 807 len 40960
16105 059/300 Read block 152 len 32768
16106 060/300 Write block 529 len 8192 data: 4 4
16107 061/300 Read block 424 len 8192
16108 062/300 Flush
16109 063/300 Read block 86 len 28672
16110 064/300 Read block 1837 len 12288
16111 065/300 Read block 3312 len 28672
16112 066/300 Read block 4607 len 4096
16113 067/300 Write block 935 len 24576 data: 5 4 6 5 5 4
16114 068/300 Flush
16115 069/300 Write block 2468 len 28672 data: 3 3 3 4 3 3 4
16116 070/300 Read block 1739 len 8192
16117 071/300 Read block 3114 len 36864
16118 072/300 Read block 920 len 40960
16119 073/300 Read block 2069 len 16384
16120 074/300 Write block 4370 len 24576 data: 4 5 6 6 5 4
16121 075/300 Read block 4279 len 28672
16122 076/300 Read block 1454 len 32768
16123 077/300 Read block 2243 len 20480
16124 078/300 Read block 4579 len 8192
16125 079/300 Read block 1556 len 16384
16126 080/300 Write block 1379 len 20480 data: 7 7 9 8 6
16127 081/300 Write block 2371 len 40960 data: 5 3 3 3 2 3 3 3 3 3
16128 082/300 Read block 2716 len 28672
16129 083/300 Flush
16130 084/300 Write block 3783 len 16384 data: 4 3 3 3
16131 085/300 Read block 343 len 32768
16132 086/300 Read block 4505 len 4096
16133 087/300 Read block 3303 len 40960
16134 088/300 Read block 384 len 4096
16135 089/300 Write block 824 len 4096 data: 3
16136 090/300 Write block 934 len 16384 data: 5 6 5 7
16137 091/300 Write block 2053 len 36864 data: 4 4 4 4 4 3 3 3 3
16138 092/300 Write block 2079 len 4096 data: 4
16139 093/300 Read block 797 len 40960
16140 094/300 Read block 1524 len 24576
16141 095/300 Read block 3955 len 24576
16142 096/300 Write block 3024 len 12288 data: 6 6 6
16143 097/300 Read block 4878 len 32768
16144 098/300 Read block 2745 len 40960
16145 099/300 Read block 4736 len 40960
16146 100/300 Write block 1045 len 4096 data: 3
16147 101/300 Read block 4795 len 36864
16148 102/300 Write block 1662 len 16384 data: 3 3 3 3
16149 103/300 Write block 4400 len 20480 data: 3 3 3 3 3
16150 104/300 Read block 3372 len 8192
16151 105/300 Write block 1385 len 40960 data: 4 4 4 3 3 3 3 3 4 3
16152 106/300 Flush
16153 107/300 Read block 2843 len 16384
16154 108/300 Write block 686 len 28672 data: 6 7 7 7 6 6 6
16155 109/300 Read block 1920 len 4096
16156 110/300 Write block 3195 len 20480 data: 4 5 6 6 5
16157 111/300 Write block 1259 len 24576 data: 3 3 4 2 2 2
16158 112/300 Write block 2725 len 28672 data: 4 5 5 5 4 3 3
16159 113/300 Flush
16160 114/300 Write block 3341 len 16384 data: 2 2 3 4
16161 115/300 Write block 4536 len 40960 data: 3 3 4 4 4 4 4 6 4 3
16162 116/300 Write block 3420 len 32768 data: 9 8 7 8 8 7 7 6
16163 117/300 Write block 1696 len 24576 data: 6 5 5 5 6 7
16164 118/300 Read block 2764 len 4096
16165 119/300 Read block 1998 len 16384
16166 120/300 Read block 132 len 4096
16167 121/300 Read block 4880 len 32768
16168 122/300 Read block 3070 len 20480
16169 123/300 Write block 512 len 40960 data: 6 6 6 6 6 6 5 4 4 4
16170 124/300 Read block 313 len 12288
16171 125/300 Flush
16172 126/300 Flush
16173 127/300 Flush
16174 128/300 Read block 2393 len 16384
16175 129/300 Flush
16176 130/300 Write block 3404 len 36864 data: 2 2 2 2 2 2 2 2 2
16177 131/300 Read block 1536 len 20480
16178 132/300 Read block 3168 len 8192
16179 133/300 Write block 2833 len 24576 data: 5 5 4 5 4 3
16180 134/300 Read block 3954 len 4096
16181 135/300 Read block 184 len 40960
16182 136/300 Read block 916 len 24576
16183 137/300 Write block 2460 len 16384 data: 5 3 4 4
16184 138/300 Write block 527 len 12288 data: 4 3 5
16185 139/300 Read block 4096 len 12288
16186 140/300 Flush
16187 141/300 Flush
16188 142/300 Write block 1375 len 20480 data: 5 5 5 6 8
16189 143/300 Read block 617 len 20480
16190 144/300 Read block 686 len 12288
16191 145/300 Write block 3160 len 8192 data: 3 4
16192 146/300 Read block 3262 len 36864
16193 147/300 Write block 3863 len 40960 data: 2 3 3 3 4 4 4 5 4 3
16194 148/300 Read block 3988 len 4096
16195 149/300 Read block 1867 len 36864
16196 150/300 Write block 632 len 28672 data: 2 3 2 2 2 2 2
16197 151/300 Write block 4281 len 24576 data: 5 5 5 5 4 2
16198 152/300 Read block 1314 len 24576
16199 153/300 Read block 4395 len 40960
16200 154/300 Read block 770 len 20480
16201 155/300 Flush
16202 156/300 Read block 1943 len 12288
16203 157/300 Read block 692 len 16384
16204 158/300 Write block 3272 len 20480 data: 2 2 2 3 3
16205 159/300 Read block 1221 len 20480
16206 160/300 Flush
16207 161/300 Read block 737 len 28672
16208 162/300 Read block 4966 len 24576
16209 163/300 Write block 2309 len 28672 data: 3 4 4 7 8 8 7
16210 164/300 Read block 920 len 16384
16211 165/300 Write block 1276 len 28672 data: 6 7 6 6 5 4 4
16212 166/300 Read block 3956 len 8192
16213 167/300 Read block 3279 len 32768
16214 168/300 Read block 4193 len 32768
16215 169/300 Write block 813 len 16384 data: 3 4 5 5
16216 170/300 Write block 4661 len 8192 data: 4 5
16217 171/300 Write block 2208 len 24576 data: 4 4 3 4 4 3
16218 172/300 Write block 1901 len 36864 data: 4 4 4 3 4 4 3 3 3
16219 173/300 Write block 3220 len 32768 data: 3 3 2 2 2 2 2 5
16220 174/300 Read block 3441 len 24576
16221 175/300 Write block 2894 len 32768 data: 5 5 4 3 3 4 6 5
16222 176/300 Read block 3701 len 28672
16223 177/300 Write block 340 len 8192 data: 3 2
16224 178/300 Read block 4702 len 24576
16225 179/300 Flush
16226 180/300 Write block 3798 len 4096 data: 6
16227 181/300 Read block 3917 len 32768
16228 182/300 Write block 4628 len 20480 data: 3 3 3 3 3
16229 183/300 Write block 4913 len 16384 data: 3 4 4 5
16230 184/300 Read block 2434 len 4096
16231 185/300 Read block 1293 len 40960
16232 186/300 Read block 3593 len 28672
16233 187/300 Read block 657 len 32768
16234 188/300 Write block 3531 len 36864 data: 2 3 4 4 4 4 5 5 5
16235 189/300 Read block 2583 len 12288
16236 190/300 Read block 4097 len 20480
16237 191/300 Write block 4420 len 40960 data: 5 5 5 6 6 6 6 6 4 4
16238 192/300 Flush
16239 193/300 Write block 2621 len 16384 data: 4 5 5 7
16240 194/300 Write block 151 len 32768 data: 4 3 4 3 3 4 4 3
16241 195/300 Flush
16242 196/300 Flush
16243 197/300 Read block 2554 len 20480
16244 198/300 Write block 4001 len 12288 data: 8 8 6
16245 199/300 Read block 2960 len 32768
16246 200/300 Write block 1901 len 4096 data: 5
16247 201/300 Write block 2004 len 4096 data: 5
16248 202/300 Write block 1625 len 32768 data: 5 5 3 3 2 2 2 4
16249 203/300 Write block 3128 len 28672 data: 3 4 4 4 4 4 4
16250 204/300 Flush
16251 205/300 Write block 954 len 4096 data: 3
16252 206/300 Flush
16253 207/300 Read block 4837 len 4096
16254 208/300 Read block 4472 len 8192
16255 209/300 Write block 3415 len 16384 data: 3 6 6 6
16256 210/300 Write block 4764 len 28672 data: 3 3 4 4 6 6 8
16257 211/300 Flush
16258 212/300 Read block 4524 len 8192
16259 213/300 Write block 730 len 12288 data: 3 4 3
16260 214/300 Write block 3973 len 12288 data: 6 6 6
16261 215/300 Read block 1732 len 32768
16262 216/300 Read block 4154 len 16384
16263 217/300 Write block 1671 len 36864 data: 5 4 4 4 3 3 3 3 3
16264 218/300 Read block 560 len 28672
16265 219/300 Read block 318 len 4096
16266 220/300 Write block 61 len 12288 data: 5 5 5
16267 221/300 Read block 4786 len 12288
16268 222/300 Read block 4825 len 28672
16269 223/300 Read block 1354 len 16384
16270 224/300 Read block 2124 len 40960
16271 225/300 Read block 3441 len 8192
16272 226/300 Read block 728 len 16384
16273 227/300 Flush
16274 228/300 Write block 2399 len 40960 data: 6 6 6 6 5 6 6 4 3 4
16275 229/300 Write block 1389 len 16384 data: 4 4 4 4
16276 230/300 Write block 3828 len 40960 data: 4 4 4 5 5 5 4 4 5 4
16277 231/300 Write block 1658 len 8192 data: 4 5
16278 232/300 Write block 3879 len 20480 data: 4 4 4 4 4
16279 233/300 Read block 4187 len 8192
16280 234/300 Write block 309 len 24576 data: 5 5 6 5 5 4
16281 235/300 Write block 2876 len 16384 data: 5 5 6 5
16282 236/300 Read block 2744 len 40960
16283 237/300 Write block 4543 len 4096 data: 7
16284 238/300 Read block 4984 len 4096
16285 239/300 Read block 3992 len 40960
16286 240/300 Write block 4125 len 40960 data: 6 5 6 7 7 5 5 3 3 3
16287 241/300 Write block 834 len 4096 data: 2
16288 242/300 Write block 1700 len 12288 data: 7 8 6
16289 243/300 Write block 3177 len 28672 data: 4 4 3 3 3 3 2
16290 244/300 Write block 2082 len 24576 data: 4 4 3 4 4 4
16291 245/300 Read block 340 len 36864
16292 246/300 Write block 890 len 16384 data: 4 4 5 5
16293 247/300 Write block 1381 len 4096 data: 10
16294 248/300 Read block 1030 len 40960
16295 249/300 Write block 2568 len 28672 data: 5 4 4 3 3 4 3
16296 250/300 Read block 2602 len 28672
16297 251/300 Write block 3149 len 36864 data: 6 6 5 5 4 4 5 4 4
16298 252/300 Read block 3437 len 40960
16299 253/300 Read block 4753 len 32768
16300 254/300 Read block 541 len 32768
16301 255/300 Write block 841 len 20480 data: 5 5 5 5 3
16302 256/300 Read block 878 len 32768
16303 257/300 Read block 4271 len 28672
16304 258/300 Read block 2673 len 32768
16305 259/300 Read block 1578 len 4096
16306 260/300 Write block 2693 len 20480 data: 5 7 7 7 8
16307 261/300 Read block 840 len 4096
16308 262/300 Write block 2172 len 20480 data: 3 3 4 4 3
16309 263/300 Write block 2394 len 32768 data: 5 5 5 6 6 7 7 7
16310 264/300 Write block 1372 len 4096 data: 2
16311 265/300 Write block 21 len 40960 data: 4 5 5 7 7 7 6 7 6 5
16312 266/300 Read block 1357 len 40960
16313 267/300 Read block 1362 len 4096
16314 268/300 Write block 4505 len 28672 data: 4 2 2 2 3 4 4
16315 269/300 Write block 2202 len 36864 data: 3 4 4 4 4 4 5 5 4
16316 270/300 Read block 593 len 20480
16317 271/300 Read block 4088 len 8192
16318 272/300 Write block 4668 len 24576 data: 7 6 3 2 2 2
16319 273/300 Read block 57 len 20480
16320 274/300 Write block 4862 len 40960 data: 6 5 4 3 3 4 4 3 3 2
16321 275/300 Read block 628 len 28672
16322 276/300 Flush
16323 277/300 Write block 2548 len 12288 data: 6 5 4
16324 278/300 Write block 41 len 12288 data: 6 6 5
16325 279/300 Flush
16326 280/300 Read block 2701 len 32768
16327 281/300 Read block 4175 len 4096
16328 282/300 Write block 4520 len 32768 data: 5 4 4 3 3 3 2 2
16329 283/300 Read block 678 len 4096
16330 284/300 Read block 1928 len 36864
16331 285/300 Write block 140 len 40960 data: 5 3 5 7 7 7 7 7 8 6
16332 286/300 Read block 2229 len 8192
16333 287/300 Read block 4547 len 12288
16334 288/300 Read block 2408 len 36864
16335 289/300 Read block 3298 len 32768
16336 290/300 Read block 1668 len 20480
16337 291/300 Read block 4352 len 8192
16338 292/300 Write block 471 len 16384 data: 5 5 5 5
16339 293/300 Write block 2926 len 36864 data: 5 6 5 5 3 3 3 3 3
16340 294/300 Read block 4002 len 16384
16341 295/300 Write block 4764 len 4096 data: 4
16342 296/300 Write block 4171 len 36864 data: 4 4 6 5 5 6 6 6 5
16343 297/300 Flush
16344 298/300 Write block 2577 len 36864 data: 3 3 3 3 4 6 3 3 3
16345 299/300 Read block 2892 len 36864
16346 300/300 Read block 4392 len 20480
16347 301/300 Read block 592 len 12288
16348 Main task has work to do, go find it
16349 got dsc Start(0)
16350 start 0
16351 Jan 14 16:14:03.823 INFO request completed, latency_us: 223, response_code: 204, uri: /start/cid/0, method: POST, req_id: 0c36a7c1-d9d1-4af8-a791-432fdc9fb6a7, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
16352 Replay: started 0, returned:Ok(())
16353 ----------------------------------------------------------------
16354 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16355 States: Offline Active Active
16356 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16357 5528 Acked Read 8 Sent Done Done false
16358 5529 Acked Read 1 Sent Done Done false
16359 5530 Acked Write 4 Sent Done Done false
16360 5531 Acked Read 5 Sent Done Done false
16361 5532 Acked Read 8 Sent Done Done false
16362 5533 Acked Read 5 Sent Done Done false
16363 5534 Acked Write 8 Sent Done Done false
16364 5535 Acked Write 8 Sent Done Done false
16365 5536 Acked Read 5 Sent Done Done false
16366 5537 Acked Read 10 Sent Done Done false
16367 5538 Acked Write 1 Sent Done Done false
16368 5539 Acked Write 10 Sent Done Done false
16369 5540 Acked Read 4 Sent Done Done false
16370 5541 Acked Write 5 Sent Done Done false
16371 5542 Acked Read 8 Sent Done Done false
16372 5543 Acked Read 2 Sent Done Done false
16373 5544 Acked Read 8 Sent Done Done false
16374 5545 Acked Read 5 Sent Done Done false
16375 5546 Acked Read 2 Sent Done Done false
16376 5547 Acked Write 9 Sent Done Done false
16377 5548 Acked Read 5 Sent Done Done false
16378 5549 Acked Read 4 Sent Done Done false
16379 5550 Acked Write 10 Sent Done Done false
16380 5551 Acked Write 2 Sent Done Done false
16381 5552 Acked Read 7 Sent Done Done false
16382 5553 Acked Read 1 Sent Done Done false
16383 5554 Acked Read 7 Sent Done Done false
16384 5555 Acked Write 6 Sent Done Done false
16385 5556 Acked Read 10 Sent Done Done false
16386 5557 Acked Read 7 Sent Done Done false
16387 5558 Acked Write 6 Sent Done Done false
16388 5559 Acked Read 6 Sent Done Done false
16389 5560 Acked Read 10 Sent Done Done false
16390 5561 Acked Read 4 Sent Done Done false
16391 5562 Acked Write 10 Sent Done Done false
16392 5563 Acked Read 5 Sent Done Done false
16393 5564 Acked Read 8 Sent Done Done false
16394 5565 Acked Read 9 Sent Done Done false
16395 5566 Acked Read 9 Sent Done Done false
16396 5567 Acked Write 6 Sent Done Done false
16397 5568 Acked Write 2 Sent Done Done false
16398 5569 Acked Write 4 Sent Done Done false
16399 5570 Acked Read 8 Sent Done Done false
16400 5571 Acked Write 2 Sent Done Done false
16401 5572 Acked Read 2 Sent Done Done false
16402 5573 Acked Write 4 Sent Done Done false
16403 5574 Acked Write 8 Sent Done Done false
16404 5575 Acked Write 1 Sent Done Done false
16405 5576 Acked Write 1 Sent Done Done false
16406 5577 Acked Read 2 Sent Done Done false
16407 5578 Acked Flush 0 Sent Done Done false
16408 5579 Acked Flush 0 Sent Done Done false
16409 5580 Acked Write 3 Sent Done Done false
16410 5581 Acked Write 3 Sent Done Done false
16411 5582 Acked Flush 0 Sent Done Done false
16412 5583 Acked Write 10 Sent Done Done false
16413 5584 Acked Write 8 Sent Done Done false
16414 5585 Acked Read 10 Sent Done Done false
16415 5586 Acked Read 8 Sent Done Done false
16416 5587 Acked Write 2 Sent Done Done false
16417 5588 Acked Read 2 Sent Done Done false
16418 5589 Acked Flush 0 Sent Done Done false
16419 5590 Acked Read 7 Sent Done Done false
16420 5591 Acked Read 3 Sent Done Done false
16421 5592 Acked Read 7 Sent Done Done false
16422 5593 Acked Read 1 Sent Done Done false
16423 5594 Acked Write 6 Sent Done Done false
16424 5595 Acked Flush 0 Sent Done Done false
16425 5596 Acked Write 7 Sent Done Done false
16426 5597 Acked Read 2 Sent Done Done false
16427 5598 Acked Read 9 Sent Done Done false
16428 5599 Acked Read 10 Sent Done Done false
16429 5600 Acked Read 4 Sent Done Done false
16430 5601 Acked Write 6 Sent Done Done false
16431 5602 Acked Read 7 Sent Done Done false
16432 5603 Acked Read 8 Sent Done Done false
16433 5604 Acked Read 5 Sent Done Done false
16434 5605 Acked Read 2 Sent Done Done false
16435 5606 Acked Read 4 Sent Done Done false
16436 5607 Acked Write 5 Sent Done Done false
16437 5608 Acked Write 10 Sent Done Done false
16438 5609 Acked Read 7 Sent Done Done false
16439 5610 Acked Flush 0 Sent Done Done false
16440 5611 Acked Write 4 Sent Done Done false
16441 5612 Acked Read 8 Sent Done Done false
16442 5613 Acked Read 1 Sent Done Done false
16443 5614 Acked Read 10 Sent Done Done false
16444 5615 Acked Read 1 Sent Done Done false
16445 5616 Acked Write 1 Sent Done Done false
16446 5617 Acked Write 4 Sent Done Done false
16447 5618 Acked Write 9 Sent Done Done false
16448 5619 Acked Write 1 Sent Done Done false
16449 5620 Acked Read 10 Sent Done Done false
16450 5621 Acked Read 6 Sent Done Done false
16451 5622 Acked Read 6 Sent Done Done false
16452 5623 Acked Write 3 Sent Done Done false
16453 5624 Acked Read 8 Sent Done Done false
16454 5625 Acked Read 10 Sent Done Done false
16455 5626 Acked Read 10 Sent Done Done false
16456 5627 Acked Write 1 Sent Done Done false
16457 5628 Acked Read 9 Sent Done Done false
16458 5629 Acked Write 4 Sent Done Done false
16459 5630 Acked Write 5 Sent Done Done false
16460 5631 Acked Read 2 Sent Done Done false
16461 5632 Acked Write 10 Sent Done Done false
16462 5633 Acked Flush 0 Sent Done Done false
16463 5634 Acked Read 4 Sent Done Done false
16464 5635 Acked Write 7 Sent Done Done false
16465 5636 Acked Read 1 Sent Done Done false
16466 5637 Acked Write 5 Sent Done Done false
16467 5638 Acked Write 6 Sent Done Done false
16468 5639 Acked Write 7 Sent Done Done false
16469 5640 Acked Flush 0 Sent Done Done false
16470 5641 Acked Write 4 Sent Done Done false
16471 5642 Acked Write 10 Sent Done Done false
16472 5643 Acked Write 8 Sent Done Done false
16473 5644 Acked Write 6 Sent Done Done false
16474 5645 Acked Read 1 Sent Done Done false
16475 5646 Acked Read 4 Sent Done Done false
16476 5647 Acked Read 1 Sent Done Done false
16477 5648 Acked Read 8 Sent Done Done false
16478 5649 Acked Read 5 Sent Done Done false
16479 5650 Acked Write 10 Sent Done Done false
16480 5651 Acked Read 3 Sent Done Done false
16481 5652 Acked Flush 0 Sent Done Done false
16482 5653 Acked Flush 0 Sent Done Done false
16483 5654 Acked Flush 0 Sent Done Done false
16484 5655 Acked Read 4 Sent Done Done false
16485 5656 Acked Flush 0 Sent Done Done false
16486 5657 Acked Write 9 Sent Done Done false
16487 5658 Acked Read 5 Sent Done Done false
16488 5659 Acked Read 2 Sent Done Done false
16489 5660 Acked Write 6 Sent Done Done false
16490 5661 Acked Read 1 Sent Done Done false
16491 5662 Acked Read 10 Sent Done Done false
16492 5663 Acked Read 6 Sent Done Done false
16493 5664 Acked Write 4 Sent Done Done false
16494 5665 Acked Write 3 Sent Done Done false
16495 5666 Acked Read 3 Sent Done Done false
16496 5667 Acked Flush 0 Sent Done Done false
16497 5668 Acked Flush 0 Sent Done Done false
16498 5669 Acked Write 5 Sent Done Done false
16499 5670 Acked Read 5 Sent Done Done false
16500 5671 Acked Read 3 Sent Done Done false
16501 5672 Acked Write 2 Sent Done Done false
16502 5673 Acked Read 9 Sent Done Done false
16503 5674 Acked Write 10 Sent Done Done false
16504 5675 Acked Read 1 Sent Done Done false
16505 5676 Acked Read 9 Sent Done Done false
16506 5677 Acked Write 7 Sent Done Done false
16507 5678 Acked Write 6 Sent Done Done false
16508 5679 Acked Read 6 Sent Done Done false
16509 5680 Acked Read 10 Sent Done Done false
16510 5681 Acked Read 5 Sent Done Done false
16511 5682 Acked Flush 0 Sent Done Done false
16512 5683 Acked Read 3 Sent Done Done false
16513 5684 Acked Read 4 Sent Done Done false
16514 5685 Acked Write 5 Sent Done Done false
16515 5686 Acked Read 5 Sent Done Done false
16516 5687 Acked Flush 0 Sent Done Done false
16517 5688 Acked Read 7 Sent Done Done false
16518 5689 Acked Read 6 Sent Done Done false
16519 5690 Acked Write 7 Sent Done Done false
16520 5691 Acked Read 4 Sent Done Done false
16521 5692 Acked Write 7 Sent Done Done false
16522 5693 Acked Read 2 Sent Done Done false
16523 5694 Acked Read 8 Sent Done Done false
16524 5695 Acked Read 8 Sent Done Done false
16525 5696 Acked Write 4 Sent Done Done false
16526 5697 Acked Write 2 Sent Done Done false
16527 5698 Acked Write 6 Sent Done Done false
16528 5699 Acked Write 9 Sent Done Done false
16529 5700 Acked Write 8 Sent Done Done false
16530 5701 Acked Read 6 Sent Done Done false
16531 5702 Acked Write 8 Sent Done Done false
16532 5703 Acked Read 7 Sent Done Done false
16533 5704 Acked Write 2 Sent Done Done false
16534 5705 Acked Read 6 Sent Done Done false
16535 5706 Acked Flush 0 Sent Done Done false
16536 5707 Acked Write 1 Sent Done Done false
16537 5708 Acked Read 8 Sent Done Done false
16538 5709 Acked Write 5 Sent Done Done false
16539 5710 Acked Write 4 Sent Done Done false
16540 5711 Acked Read 1 Sent Done Done false
16541 5712 Acked Read 10 Sent Done Done false
16542 5713 Acked Read 7 Sent Done Done false
16543 5714 Acked Read 8 Sent Done Done false
16544 5715 Acked Write 9 Sent Done Done false
16545 5716 Acked Read 3 Sent Done Done false
16546 5717 Acked Read 5 Sent Done Done false
16547 5718 Acked Write 10 Sent Done Done false
16548 5719 Acked Flush 0 Sent Done Done false
16549 5720 Acked Write 4 Sent Done Done false
16550 5721 Acked Write 8 Sent Done Done false
16551 5722 Acked Flush 0 Sent Done Done false
16552 5723 Acked Flush 0 Sent Done Done false
16553 5724 Acked Read 5 Sent Done Done false
16554 5725 Acked Write 3 Sent Done Done false
16555 5726 Acked Read 8 Sent Done Done false
16556 5727 Acked Write 1 Sent Done Done false
16557 5728 Acked Write 1 Sent Done Done false
16558 5729 Acked Write 8 Sent Done Done false
16559 5730 Acked Write 7 Sent Done Done false
16560 5731 Acked Flush 0 Sent Done Done false
16561 5732 Acked Write 1 Sent Done Done false
16562 5733 Acked Flush 0 Sent Done Done false
16563 5734 Acked Read 1 Sent Done Done false
16564 5735 Acked Read 2 Sent Done Done false
16565 5736 Acked Write 4 Sent Done Done false
16566 5737 Acked Write 7 Sent Done Done false
16567 5738 Acked Flush 0 Sent Done Done false
16568 5739 Acked Read 2 Sent Done Done false
16569 5740 Acked Write 3 Sent Done Done false
16570 5741 Acked Write 3 Sent Done Done false
16571 5742 Acked Read 8 Sent Done Done false
16572 5743 Acked Read 4 Sent Done Done false
16573 5744 Acked Write 9 Sent Done Done false
16574 5745 Acked Read 7 Sent Done Done false
16575 5746 Acked Read 1 Sent Done Done false
16576 5747 Acked Write 3 Sent Done Done false
16577 5748 Acked Read 3 Sent Done Done false
16578 5749 Acked Read 7 Sent Done Done false
16579 5750 Acked Read 4 Sent Done Done false
16580 5751 Acked Read 10 Sent Done Done false
16581 5752 Acked Read 2 Sent Done Done false
16582 5753 Acked Read 4 Sent Done Done false
16583 5754 Acked Flush 0 Sent Done Done false
16584 5755 Acked Write 10 Sent Done Done false
16585 5756 Acked Write 4 Sent Done Done false
16586 5757 Acked Write 10 Sent Done Done false
16587 5758 Acked Write 2 Sent Done Done false
16588 5759 Acked Write 5 Sent Done Done false
16589 5760 Acked Read 2 Sent Done Done false
16590 5761 Acked Write 6 Sent Done Done false
16591 5762 Acked Write 4 Sent Done Done false
16592 5763 Acked Read 10 Sent Done Done false
16593 5764 Acked Write 1 Sent Done Done false
16594 5765 Acked Read 1 Sent Done Done false
16595 5766 Acked Read 10 Sent Done Done false
16596 5767 Acked Write 10 Sent Done Done false
16597 5768 Acked Write 1 Sent Done Done false
16598 5769 Acked Write 3 Sent Done Done false
16599 5770 Acked Write 7 Sent Done Done false
16600 5771 Acked Write 6 Sent Done Done false
16601 5772 Acked Read 9 Sent Done Done false
16602 5773 Acked Write 4 Sent Done Done false
16603 5774 Acked Write 1 Sent Done Done false
16604 5775 Acked Read 10 Sent Done Done false
16605 5776 Acked Write 7 Sent Done Done false
16606 5777 Acked Read 7 Sent Done Done false
16607 5778 Acked Write 9 Sent Done Done false
16608 5779 Acked Read 10 Sent Done Done false
16609 5780 Acked Read 8 Sent Done Done false
16610 5781 Acked Read 8 Sent Done Done false
16611 5782 Acked Write 5 Sent Done Done false
16612 5783 Acked Read 8 Sent Done Done false
16613 5784 Acked Read 7 Sent Done Done false
16614 5785 Acked Read 8 Sent Done Done false
16615 5786 Acked Read 1 Sent Done Done false
16616 5787 Acked Write 5 Sent Done Done false
16617 5788 Acked Read 1 Sent Done Done false
16618 5789 Acked Write 5 Sent Done Done false
16619 5790 Acked Write 8 Sent Done Done false
16620 5791 Acked Write 1 Sent Done Done false
16621 5792 Acked Write 10 Sent Done Done false
16622 5793 Acked Read 10 Sent Done Done false
16623 5794 Acked Read 1 Sent Done Done false
16624 5795 Acked Write 7 Sent Done Done false
16625 5796 Acked Write 9 Sent Done Done false
16626 5797 Acked Read 5 Sent Done Done false
16627 5798 Acked Read 2 Sent Done Done false
16628 5799 Acked Write 6 Sent Done Done false
16629 5800 Acked Read 5 Sent Done Done false
16630 5801 Acked Write 10 Sent Done Done false
16631 5802 Acked Read 7 Sent Done Done false
16632 5803 Acked Flush 0 Sent Done Done false
16633 5804 Acked Write 3 Sent Done Done false
16634 5805 Acked Write 3 Sent Done Done false
16635 5806 Acked Flush 0 Sent Done Done false
16636 5807 Acked Read 8 Sent Done Done false
16637 5808 Acked Read 1 Sent Done Done false
16638 5809 Acked Write 8 Sent Done Done false
16639 5810 Acked Read 1 Sent Done Done false
16640 5811 Acked Read 9 Sent Done Done false
16641 5812 Acked Write 10 Sent Done Done false
16642 5813 Acked Read 2 Sent Done Done false
16643 5814 Acked Read 3 Sent Done Done false
16644 5815 Acked Read 9 Sent Done Done false
16645 5816 Acked Read 8 Sent Done Done false
16646 5817 Acked Read 5 Sent Done Done false
16647 5818 Acked Read 2 Sent Done Done false
16648 5819 Acked Write 4 Sent Done Done false
16649 5820 Acked Write 9 Sent Done Done false
16650 5821 Acked Read 4 Sent Done Done false
16651 5822 Acked Write 1 Sent Done Done false
16652 5823 Acked Write 9 Sent Done Done false
16653 5824 Acked Flush 0 Sent Done Done false
16654 5825 Acked Write 9 Sent Done Done false
16655 5826 Acked Read 9 Sent Done Done false
16656 5827 Acked Read 5 Sent Done Done false
16657 5828 Acked Read 3 Sent Done Done false
16658 STATES DS:0 DS:1 DS:2 TOTAL
16659 Sent 301 0 0 301
16660 Done 0 301 301 602
16661 Skipped 0 0 0 0
16662 Error 0 0 0 0
16663 Last Flush: 5527 5824 5824
16664 Downstairs last five completed: 5527 5526 5525 5524 5523
16665 Upstairs last five completed: 5828 5827 5826 5825 5824
16666 CLIENT: Up:0 ds:301 act:2
16667 [8810] got start action
16668 [8810] I am going to restart
16669 Starting downstairs at port 8810
16670 Make output file at "/tmp/dsc/downstairs-8810.txt"
16671 [8810][0] reports Starting
16672 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1218)
16673 [8810][0] reports Running
16674 ----------------------------------------------------------------
16675 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
16676 States: Offline Active Active
16677 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16678 5528 Acked Read 8 Sent Done Done false
16679 5529 Acked Read 1 Sent Done Done false
16680 5530 Acked Write 4 Sent Done Done false
16681 5531 Acked Read 5 Sent Done Done false
16682 5532 Acked Read 8 Sent Done Done false
16683 5533 Acked Read 5 Sent Done Done false
16684 5534 Acked Write 8 Sent Done Done false
16685 5535 Acked Write 8 Sent Done Done false
16686 5536 Acked Read 5 Sent Done Done false
16687 5537 Acked Read 10 Sent Done Done false
16688 5538 Acked Write 1 Sent Done Done false
16689 5539 Acked Write 10 Sent Done Done false
16690 5540 Acked Read 4 Sent Done Done false
16691 5541 Acked Write 5 Sent Done Done false
16692 5542 Acked Read 8 Sent Done Done false
16693 5543 Acked Read 2 Sent Done Done false
16694 5544 Acked Read 8 Sent Done Done false
16695 5545 Acked Read 5 Sent Done Done false
16696 5546 Acked Read 2 Sent Done Done false
16697 5547 Acked Write 9 Sent Done Done false
16698 5548 Acked Read 5 Sent Done Done false
16699 5549 Acked Read 4 Sent Done Done false
16700 5550 Acked Write 10 Sent Done Done false
16701 5551 Acked Write 2 Sent Done Done false
16702 5552 Acked Read 7 Sent Done Done false
16703 5553 Acked Read 1 Sent Done Done false
16704 5554 Acked Read 7 Sent Done Done false
16705 5555 Acked Write 6 Sent Done Done false
16706 5556 Acked Read 10 Sent Done Done false
16707 5557 Acked Read 7 Sent Done Done false
16708 5558 Acked Write 6 Sent Done Done false
16709 5559 Acked Read 6 Sent Done Done false
16710 5560 Acked Read 10 Sent Done Done false
16711 5561 Acked Read 4 Sent Done Done false
16712 5562 Acked Write 10 Sent Done Done false
16713 5563 Acked Read 5 Sent Done Done false
16714 5564 Acked Read 8 Sent Done Done false
16715 5565 Acked Read 9 Sent Done Done false
16716 5566 Acked Read 9 Sent Done Done false
16717 5567 Acked Write 6 Sent Done Done false
16718 5568 Acked Write 2 Sent Done Done false
16719 5569 Acked Write 4 Sent Done Done false
16720 5570 Acked Read 8 Sent Done Done false
16721 5571 Acked Write 2 Sent Done Done false
16722 5572 Acked Read 2 Sent Done Done false
16723 5573 Acked Write 4 Sent Done Done false
16724 5574 Acked Write 8 Sent Done Done false
16725 5575 Acked Write 1 Sent Done Done false
16726 5576 Acked Write 1 Sent Done Done false
16727 5577 Acked Read 2 Sent Done Done false
16728 5578 Acked Flush 0 Sent Done Done false
16729 5579 Acked Flush 0 Sent Done Done false
16730 5580 Acked Write 3 Sent Done Done false
16731 5581 Acked Write 3 Sent Done Done false
16732 5582 Acked Flush 0 Sent Done Done false
16733 5583 Acked Write 10 Sent Done Done false
16734 5584 Acked Write 8 Sent Done Done false
16735 5585 Acked Read 10 Sent Done Done false
16736 5586 Acked Read 8 Sent Done Done false
16737 5587 Acked Write 2 Sent Done Done false
16738 5588 Acked Read 2 Sent Done Done false
16739 5589 Acked Flush 0 Sent Done Done false
16740 5590 Acked Read 7 Sent Done Done false
16741 5591 Acked Read 3 Sent Done Done false
16742 5592 Acked Read 7 Sent Done Done false
16743 5593 Acked Read 1 Sent Done Done false
16744 5594 Acked Write 6 Sent Done Done false
16745 5595 Acked Flush 0 Sent Done Done false
16746 5596 Acked Write 7 Sent Done Done false
16747 5597 Acked Read 2 Sent Done Done false
16748 5598 Acked Read 9 Sent Done Done false
16749 5599 Acked Read 10 Sent Done Done false
16750 5600 Acked Read 4 Sent Done Done false
16751 5601 Acked Write 6 Sent Done Done false
16752 5602 Acked Read 7 Sent Done Done false
16753 5603 Acked Read 8 Sent Done Done false
16754 5604 Acked Read 5 Sent Done Done false
16755 5605 Acked Read 2 Sent Done Done false
16756 5606 Acked Read 4 Sent Done Done false
16757 5607 Acked Write 5 Sent Done Done false
16758 5608 Acked Write 10 Sent Done Done false
16759 5609 Acked Read 7 Sent Done Done false
16760 5610 Acked Flush 0 Sent Done Done false
16761 5611 Acked Write 4 Sent Done Done false
16762 5612 Acked Read 8 Sent Done Done false
16763 5613 Acked Read 1 Sent Done Done false
16764 5614 Acked Read 10 Sent Done Done false
16765 5615 Acked Read 1 Sent Done Done false
16766 5616 Acked Write 1 Sent Done Done false
16767 5617 Acked Write 4 Sent Done Done false
16768 5618 Acked Write 9 Sent Done Done false
16769 5619 Acked Write 1 Sent Done Done false
16770 5620 Acked Read 10 Sent Done Done false
16771 5621 Acked Read 6 Sent Done Done false
16772 5622 Acked Read 6 Sent Done Done false
16773 5623 Acked Write 3 Sent Done Done false
16774 5624 Acked Read 8 Sent Done Done false
16775 5625 Acked Read 10 Sent Done Done false
16776 5626 Acked Read 10 Sent Done Done false
16777 5627 Acked Write 1 Sent Done Done false
16778 5628 Acked Read 9 Sent Done Done false
16779 5629 Acked Write 4 Sent Done Done false
16780 5630 Acked Write 5 Sent Done Done false
16781 5631 Acked Read 2 Sent Done Done false
16782 5632 Acked Write 10 Sent Done Done false
16783 5633 Acked Flush 0 Sent Done Done false
16784 5634 Acked Read 4 Sent Done Done false
16785 5635 Acked Write 7 Sent Done Done false
16786 5636 Acked Read 1 Sent Done Done false
16787 5637 Acked Write 5 Sent Done Done false
16788 5638 Acked Write 6 Sent Done Done false
16789 5639 Acked Write 7 Sent Done Done false
16790 5640 Acked Flush 0 Sent Done Done false
16791 5641 Acked Write 4 Sent Done Done false
16792 5642 Acked Write 10 Sent Done Done false
16793 5643 Acked Write 8 Sent Done Done false
16794 5644 Acked Write 6 Sent Done Done false
16795 5645 Acked Read 1 Sent Done Done false
16796 5646 Acked Read 4 Sent Done Done false
16797 5647 Acked Read 1 Sent Done Done false
16798 5648 Acked Read 8 Sent Done Done false
16799 5649 Acked Read 5 Sent Done Done false
16800 5650 Acked Write 10 Sent Done Done false
16801 5651 Acked Read 3 Sent Done Done false
16802 5652 Acked Flush 0 Sent Done Done false
16803 5653 Acked Flush 0 Sent Done Done false
16804 5654 Acked Flush 0 Sent Done Done false
16805 5655 Acked Read 4 Sent Done Done false
16806 5656 Acked Flush 0 Sent Done Done false
16807 5657 Acked Write 9 Sent Done Done false
16808 5658 Acked Read 5 Sent Done Done false
16809 5659 Acked Read 2 Sent Done Done false
16810 5660 Acked Write 6 Sent Done Done false
16811 5661 Acked Read 1 Sent Done Done false
16812 5662 Acked Read 10 Sent Done Done false
16813 5663 Acked Read 6 Sent Done Done false
16814 5664 Acked Write 4 Sent Done Done false
16815 5665 Acked Write 3 Sent Done Done false
16816 5666 Acked Read 3 Sent Done Done false
16817 5667 Acked Flush 0 Sent Done Done false
16818 5668 Acked Flush 0 Sent Done Done false
16819 5669 Acked Write 5 Sent Done Done false
16820 5670 Acked Read 5 Sent Done Done false
16821 5671 Acked Read 3 Sent Done Done false
16822 5672 Acked Write 2 Sent Done Done false
16823 5673 Acked Read 9 Sent Done Done false
16824 5674 Acked Write 10 Sent Done Done false
16825 5675 Acked Read 1 Sent Done Done false
16826 5676 Acked Read 9 Sent Done Done false
16827 5677 Acked Write 7 Sent Done Done false
16828 5678 Acked Write 6 Sent Done Done false
16829 5679 Acked Read 6 Sent Done Done false
16830 5680 Acked Read 10 Sent Done Done false
16831 5681 Acked Read 5 Sent Done Done false
16832 5682 Acked Flush 0 Sent Done Done false
16833 5683 Acked Read 3 Sent Done Done false
16834 5684 Acked Read 4 Sent Done Done false
16835 5685 Acked Write 5 Sent Done Done false
16836 5686 Acked Read 5 Sent Done Done false
16837 5687 Acked Flush 0 Sent Done Done false
16838 5688 Acked Read 7 Sent Done Done false
16839 5689 Acked Read 6 Sent Done Done false
16840 5690 Acked Write 7 Sent Done Done false
16841 5691 Acked Read 4 Sent Done Done false
16842 5692 Acked Write 7 Sent Done Done false
16843 5693 Acked Read 2 Sent Done Done false
16844 5694 Acked Read 8 Sent Done Done false
16845 5695 Acked Read 8 Sent Done Done false
16846 5696 Acked Write 4 Sent Done Done false
16847 5697 Acked Write 2 Sent Done Done false
16848 5698 Acked Write 6 Sent Done Done false
16849 5699 Acked Write 9 Sent Done Done false
16850 5700 Acked Write 8 Sent Done Done false
16851 5701 Acked Read 6 Sent Done Done false
16852 5702 Acked Write 8 Sent Done Done false
16853 5703 Acked Read 7 Sent Done Done false
16854 5704 Acked Write 2 Sent Done Done false
16855 5705 Acked Read 6 Sent Done Done false
16856 5706 Acked Flush 0 Sent Done Done false
16857 5707 Acked Write 1 Sent Done Done false
16858 5708 Acked Read 8 Sent Done Done false
16859 5709 Acked Write 5 Sent Done Done false
16860 5710 Acked Write 4 Sent Done Done false
16861 5711 Acked Read 1 Sent Done Done false
16862 5712 Acked Read 10 Sent Done Done false
16863 5713 Acked Read 7 Sent Done Done false
16864 5714 Acked Read 8 Sent Done Done false
16865 5715 Acked Write 9 Sent Done Done false
16866 5716 Acked Read 3 Sent Done Done false
16867 5717 Acked Read 5 Sent Done Done false
16868 5718 Acked Write 10 Sent Done Done false
16869 5719 Acked Flush 0 Sent Done Done false
16870 5720 Acked Write 4 Sent Done Done false
16871 5721 Acked Write 8 Sent Done Done false
16872 5722 Acked Flush 0 Sent Done Done false
16873 5723 Acked Flush 0 Sent Done Done false
16874 5724 Acked Read 5 Sent Done Done false
16875 5725 Acked Write 3 Sent Done Done false
16876 5726 Acked Read 8 Sent Done Done false
16877 5727 Acked Write 1 Sent Done Done false
16878 5728 Acked Write 1 Sent Done Done false
16879 5729 Acked Write 8 Sent Done Done false
16880 5730 Acked Write 7 Sent Done Done false
16881 5731 Acked Flush 0 Sent Done Done false
16882 5732 Acked Write 1 Sent Done Done false
16883 5733 Acked Flush 0 Sent Done Done false
16884 5734 Acked Read 1 Sent Done Done false
16885 5735 Acked Read 2 Sent Done Done false
16886 5736 Acked Write 4 Sent Done Done false
16887 5737 Acked Write 7 Sent Done Done false
16888 5738 Acked Flush 0 Sent Done Done false
16889 5739 Acked Read 2 Sent Done Done false
16890 5740 Acked Write 3 Sent Done Done false
16891 5741 Acked Write 3 Sent Done Done false
16892 5742 Acked Read 8 Sent Done Done false
16893 5743 Acked Read 4 Sent Done Done false
16894 5744 Acked Write 9 Sent Done Done false
16895 5745 Acked Read 7 Sent Done Done false
16896 5746 Acked Read 1 Sent Done Done false
16897 5747 Acked Write 3 Sent Done Done false
16898 5748 Acked Read 3 Sent Done Done false
16899 5749 Acked Read 7 Sent Done Done false
16900 5750 Acked Read 4 Sent Done Done false
16901 5751 Acked Read 10 Sent Done Done false
16902 5752 Acked Read 2 Sent Done Done false
16903 5753 Acked Read 4 Sent Done Done false
16904 5754 Acked Flush 0 Sent Done Done false
16905 5755 Acked Write 10 Sent Done Done false
16906 5756 Acked Write 4 Sent Done Done false
16907 5757 Acked Write 10 Sent Done Done false
16908 5758 Acked Write 2 Sent Done Done false
16909 5759 Acked Write 5 Sent Done Done false
16910 5760 Acked Read 2 Sent Done Done false
16911 5761 Acked Write 6 Sent Done Done false
16912 5762 Acked Write 4 Sent Done Done false
16913 5763 Acked Read 10 Sent Done Done false
16914 5764 Acked Write 1 Sent Done Done false
16915 5765 Acked Read 1 Sent Done Done false
16916 5766 Acked Read 10 Sent Done Done false
16917 5767 Acked Write 10 Sent Done Done false
16918 5768 Acked Write 1 Sent Done Done false
16919 5769 Acked Write 3 Sent Done Done false
16920 5770 Acked Write 7 Sent Done Done false
16921 5771 Acked Write 6 Sent Done Done false
16922 5772 Acked Read 9 Sent Done Done false
16923 5773 Acked Write 4 Sent Done Done false
16924 5774 Acked Write 1 Sent Done Done false
16925 5775 Acked Read 10 Sent Done Done false
16926 5776 Acked Write 7 Sent Done Done false
16927 5777 Acked Read 7 Sent Done Done false
16928 5778 Acked Write 9 Sent Done Done false
16929 5779 Acked Read 10 Sent Done Done false
16930 5780 Acked Read 8 Sent Done Done false
16931 5781 Acked Read 8 Sent Done Done false
16932 5782 Acked Write 5 Sent Done Done false
16933 5783 Acked Read 8 Sent Done Done false
16934 5784 Acked Read 7 Sent Done Done false
16935 5785 Acked Read 8 Sent Done Done false
16936 5786 Acked Read 1 Sent Done Done false
16937 5787 Acked Write 5 Sent Done Done false
16938 5788 Acked Read 1 Sent Done Done false
16939 5789 Acked Write 5 Sent Done Done false
16940 5790 Acked Write 8 Sent Done Done false
16941 5791 Acked Write 1 Sent Done Done false
16942 5792 Acked Write 10 Sent Done Done false
16943 5793 Acked Read 10 Sent Done Done false
16944 5794 Acked Read 1 Sent Done Done false
16945 5795 Acked Write 7 Sent Done Done false
16946 5796 Acked Write 9 Sent Done Done false
16947 5797 Acked Read 5 Sent Done Done false
16948 5798 Acked Read 2 Sent Done Done false
16949 5799 Acked Write 6 Sent Done Done false
16950 5800 Acked Read 5 Sent Done Done false
16951 5801 Acked Write 10 Sent Done Done false
16952 5802 Acked Read 7 Sent Done Done false
16953 5803 Acked Flush 0 Sent Done Done false
16954 5804 Acked Write 3 Sent Done Done false
16955 5805 Acked Write 3 Sent Done Done false
16956 5806 Acked Flush 0 Sent Done Done false
16957 5807 Acked Read 8 Sent Done Done false
16958 5808 Acked Read 1 Sent Done Done false
16959 5809 Acked Write 8 Sent Done Done false
16960 5810 Acked Read 1 Sent Done Done false
16961 5811 Acked Read 9 Sent Done Done false
16962 5812 Acked Write 10 Sent Done Done false
16963 5813 Acked Read 2 Sent Done Done false
16964 5814 Acked Read 3 Sent Done Done false
16965 5815 Acked Read 9 Sent Done Done false
16966 5816 Acked Read 8 Sent Done Done false
16967 5817 Acked Read 5 Sent Done Done false
16968 5818 Acked Read 2 Sent Done Done false
16969 5819 Acked Write 4 Sent Done Done false
16970 5820 Acked Write 9 Sent Done Done false
16971 5821 Acked Read 4 Sent Done Done false
16972 5822 Acked Write 1 Sent Done Done false
16973 5823 Acked Write 9 Sent Done Done false
16974 5824 Acked Flush 0 Sent Done Done false
16975 5825 Acked Write 9 Sent Done Done false
16976 5826 Acked Read 9 Sent Done Done false
16977 5827 Acked Read 5 Sent Done Done false
16978 5828 Acked Read 3 Sent Done Done false
16979 5829 Acked Flush 0 Sent Done Done false
16980 STATES DS:0 DS:1 DS:2 TOTAL
16981 Sent 302 0 0 302
16982 Done 0 302 302 604
16983 Skipped 0 0 0 0
16984 Error 0 0 0 0
16985 Last Flush: 5527 5829 5829
16986 Downstairs last five completed: 5527 5526 5525 5524 5523
16987 Upstairs last five completed: 5829 5828 5827 5826 5825
16988 CLIENT: Up:0 ds:302 act:2
169892025-01-14T16:14:09.587ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
169902025-01-14T16:14:09.588ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:34516) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
169912025-01-14T16:14:09.589ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
169922025-01-14T16:14:09.589ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
169932025-01-14T16:14:09.589ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
169942025-01-14T16:14:09.589ZINFOcrucible: send last flush ID to this DS: 5527 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
169952025-01-14T16:14:09.589ZINFOcrucible: Replied this last flush ID: 5527 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
169962025-01-14T16:14:09.589ZINFOcrucible: [0] client re-new 302 jobs since flush 5527 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
169972025-01-14T16:14:09.590ZINFOcrucible: [0] Replayed 302 jobs since flush: 5527 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
16998 ----------------------------------------------------------------
16999 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
17000 Downstairs last five completed: 5829 5828 5827 5826 5825
17001 Upstairs last five completed: 5829 5828 5827 5826 5825
17002 CLIENT: Up:0 ds:0 act:3
17003 Replay: All jobs finished
17004 Main task has work to do, go find it
17005 got dsc Stop(0)
17006 stop 0
17007 [8810] Got stop action so:false kr:false
17008 Jan 14 16:14:11.829 INFO request completed, latency_us: 294, response_code: 204, uri: /stop/cid/0, method: POST, req_id: ccc93082-425b-4c58-9af2-93b11035413a, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
17009 Jan 14 16:14:11.830 INFO request completed, latency_us: {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-14T16:14:11.829990557Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
17010 105, response_code: 200, uri: /state/cid/0, method: GET, req_id: fb637d31-890d-4cee-9bc5-3e9c81e25a92, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
170112025-01-14T16:14:11.830ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
170122025-01-14T16:14:11.830ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
170132025-01-14T16:14:11.830ZINFOcrucible: client task is exiting = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
17014 [8810] Exited with: ExitStatus(unix_wait_status(9))
17015 [8810][0] reports Exit
17016 Jan 14 16:14:15.831 INFO request completed, latency_us: 233, response_code: 200, uri: /state/cid/0, method: GET, req_id: 065bb0e4-acdd-440f-98da-8030e8d9640d, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
17017 001/300 Read block 4538 len 40960
17018 002/300 Write block 605 len 36864 data: 2 2 2 2 3 4 4 4 4
17019 003/300 Flush
17020 004/300 Write block 3427 len 8192 data: 7 5
17021 005/300 Read block 2483 len 20480
17022 006/300 Write block 3558 len 8192 data: 2 2
17023 007/300 Read block 3182 len 28672
17024 008/300 Read block 3264 len 12288
17025 009/300 Read block 1581 len 4096
17026 010/300 Write block 2375 len 16384 data: 3 4 4 4
17027 011/300 Write block 967 len 4096 data: 4
17028 012/300 Write block 2212 len 32768 data: 5 4 4 3 4 5 4 4
17029 013/300 Read block 4772 len 16384
17030 014/300 Read block 1605 len 32768
17031 015/300 Write block 477 len 32768 data: 4 4 5 6 6 5 5 4
17032 016/300 Write block 3044 len 36864 data: 2 2 2 2 2 3 5 5 4
17033 017/300 Read block 2800 len 8192
17034 018/300 Read block 358 len 20480
17035 019/300 Read block 1511 len 24576
17036 020/300 Read block 1163 len 28672
17037 021/300 Read block 2425 len 16384
17038 022/300 Read block 3969 len 36864
17039 023/300 Write block 3024 len 8192 data: 7 7
17040 024/300 Read block 2185 len 40960
17041 025/300 Read block 4637 len 28672
17042 026/300 Read block 785 len 4096
17043 027/300 Read block 3693 len 8192
17044 028/300 Read block 21 len 4096
17045 029/300 Read block 3123 len 24576
17046 030/300 Write block 3946 len 40960 data: 3 2 3 4 4 4 4 3 4 4
17047 031/300 Write block 4775 len 32768 data: 8 7 7 4 4 3 2 2
17048 032/300 Read block 1887 len 4096
17049 033/300 Read block 4988 len 28672
17050 034/300 Read block 4660 len 12288
17051 035/300 Read block 3661 len 16384
17052 036/300 Write block 315 len 12288 data: 4 4 5
17053 037/300 Write block 1551 len 12288 data: 3 4 4
17054 038/300 Write block 2506 len 28672 data: 2 2 3 3 3 2 3
17055 039/300 Flush
17056 040/300 Write block 3263 len 40960 data: 5 5 5 5 4 4 3 3 2 3
17057 041/300 Read block 3923 len 16384
17058 042/300 Write block 4933 len 24576 data: 5 3 3 3 3 3
17059 043/300 Write block 186 len 20480 data: 5 5 5 2 2
17060 044/300 Read block 4011 len 28672
17061 045/300 Read block 3045 len 12288
17062 046/300 Write block 3180 len 32768 data: 4 4 4 3 2 2 3 4
17063 047/300 Read block 4910 len 28672
17064 048/300 Write block 2657 len 36864 data: 2 2 2 3 4 4 5 4 3
17065 049/300 Read block 422 len 12288
17066 050/300 Read block 2059 len 4096
17067 051/300 Read block 2073 len 4096
17068 052/300 Read block 2318 len 20480
17069 053/300 Write block 1073 len 16384 data: 3 6 6 5
17070 054/300 Read block 3611 len 40960
17071 055/300 Flush
17072 056/300 Write block 2823 len 24576 data: 3 2 2 2 3 4
17073 057/300 Read block 1608 len 28672
17074 058/300 Read block 2917 len 24576
17075 059/300 Write block 2618 len 12288 data: 7 6 5
17076 060/300 Read block 545 len 36864
17077 061/300 Write block 3894 len 32768 data: 3 4 6 5 6 6 6 4
17078 062/300 Write block 791 len 36864 data: 4 5 5 5 5 5 6 6 8
17079 063/300 Write block 948 len 20480 data: 3 2 2 2 2
17080 064/300 Write block 1991 len 12288 data: 6 6 7
17081 065/300 Read block 1951 len 8192
17082 066/300 Flush
17083 067/300 Write block 4880 len 24576 data: 3 3 3 5 5 5
17084 068/300 Write block 2550 len 28672 data: 5 4 3 3 3 3 4
17085 069/300 Read block 407 len 8192
17086 070/300 Read block 3446 len 28672
17087 071/300 Read block 538 len 36864
17088 072/300 Write block 1001 len 4096 data: 6
17089 073/300 Write block 3424 len 24576 data: 9 8 8 8 6 5
17090 074/300 Write block 692 len 36864 data: 7 6 5 4 3 5 5 5 4
17091 075/300 Write block 3401 len 12288 data: 5 3 3
17092 076/300 Read block 1197 len 12288
17093 077/300 Read block 2197 len 8192
17094 078/300 Write block 2230 len 32768 data: 2 3 3 3 4 5 6 6
17095 079/300 Read block 4442 len 20480
17096 080/300 Write block 4560 len 32768 data: 3 3 4 3 4 4 4 4
17097 081/300 Write block 3043 len 32768 data: 2 3 3 3 3 3 4 6
17098 082/300 Read block 3387 len 28672
17099 083/300 Write block 1918 len 12288 data: 2 2 2
17100 084/300 Read block 323 len 16384
17101 085/300 Read block 4438 len 32768
17102 086/300 Write block 767 len 28672 data: 2 4 3 3 4 4 4
17103 087/300 Write block 2500 len 28672 data: 2 2 3 3 3 2 3
17104 088/300 Read block 3365 len 8192
17105 089/300 Read block 4003 len 40960
17106 090/300 Read block 4954 len 24576
17107 091/300 Write block 3959 len 20480 data: 6 7 6 6 3
17108 092/300 Write block 873 len 16384 data: 4 2 2 2
17109 093/300 Read block 2708 len 20480
17110 094/300 Write block 2842 len 20480 data: 3 2 2 2 2
17111 095/300 Read block 4787 len 20480
17112 096/300 Write block 4749 len 24576 data: 3 4 4 4 4 3
17113 097/300 Write block 4263 len 28672 data: 3 3 3 3 3 3 3
17114 098/300 Read block 996 len 24576
17115 099/300 Flush
17116 100/300 Read block 3955 len 20480
17117 101/300 Write block 3703 len 36864 data: 3 4 4 5 5 4 4 5 4
17118 102/300 Read block 4629 len 24576
17119 103/300 Write block 4661 len 40960 data: 5 6 6 8 9 10 9 8 7 4
17120 104/300 Write block 2772 len 8192 data: 5 5
17121 105/300 Write block 3500 len 12288 data: 4 2 2
17122 106/300 Read block 4466 len 36864
17123 107/300 Read block 4030 len 28672
17124 108/300 Write block 2966 len 32768 data: 3 3 4 5 4 4 3 3
17125 109/300 Write block 2908 len 4096 data: 6
17126 110/300 Write block 3322 len 12288 data: 3 3 3
17127 111/300 Read block 4372 len 40960
17128 112/300 Read block 1122 len 40960
17129 113/300 Read block 1498 len 20480
17130 114/300 Write block 4671 len 4096 data: 3
17131 115/300 Read block 258 len 40960
17132 116/300 Read block 4630 len 12288
17133 117/300 Write block 4374 len 16384 data: 6 5 4 4
17134 118/300 Write block 272 len 32768 data: 3 3 3 4 4 4 4 4
17135 119/300 Write block 540 len 16384 data: 3 3 4 4
17136 120/300 Write block 3368 len 24576 data: 5 5 4 3 3 3
17137 121/300 Write block 587 len 24576 data: 5 4 5 5 4 4
17138 122/300 Write block 1024 len 4096 data: 4
17139 123/300 Flush
17140 124/300 Flush
17141 125/300 Read block 3415 len 16384
17142 126/300 Read block 4899 len 4096
17143 127/300 Write block 3082 len 20480 data: 2 2 2 2 2
17144 128/300 Write block 4307 len 24576 data: 3 2 2 2 3 3
17145 129/300 Write block 186 len 8192 data: 6 6
17146 130/300 Read block 2174 len 16384
17147 131/300 Read block 4285 len 12288
17148 132/300 Read block 2884 len 12288
17149 133/300 Write block 4152 len 4096 data: 2
17150 134/300 Read block 3173 len 4096
17151 135/300 Write block 1751 len 32768 data: 5 6 6 7 7 6 5 3
17152 136/300 Write block 4069 len 8192 data: 3 3
17153 137/300 Write block 2401 len 24576 data: 8 7 6 7 7 5
17154 138/300 Write block 1672 len 36864 data: 5 5 5 4 4 4 4 4 3
17155 139/300 Write block 2803 len 4096 data: 5
17156 140/300 Read block 4661 len 36864
17157 141/300 Write block 447 len 20480 data: 4 5 7 4 4
17158 142/300 Write block 927 len 4096 data: 7
17159 143/300 Read block 3001 len 28672
17160 144/300 Write block 1851 len 32768 data: 4 5 4 4 3 3 3 3
17161 145/300 Read block 4518 len 24576
17162 146/300 Write block 2279 len 40960 data: 4 5 5 5 6 5 5 4 3 4
17163 147/300 Write block 2611 len 40960 data: 4 5 7 6 6 6 7 8 7 6
17164 148/300 Read block 420 len 12288
17165 149/300 Read block 923 len 36864
17166 150/300 Read block 2762 len 20480
17167 151/300 Read block 2990 len 4096
17168 152/300 Read block 2108 len 36864
17169 153/300 Write block 4086 len 36864 data: 5 4 5 6 6 7 7 6 5
17170 154/300 Read block 1300 len 12288
17171 155/300 Write block 3642 len 24576 data: 4 4 4 4 4 3
17172 156/300 Flush
17173 157/300 Read block 1237 len 4096
17174 158/300 Read block 1712 len 8192
17175 159/300 Flush
17176 160/300 Write block 1940 len 16384 data: 4 4 4 4
17177 161/300 Write block 277 len 4096 data: 5
17178 162/300 Write block 2650 len 8192 data: 5 3
17179 163/300 Flush
17180 164/300 Write block 4844 len 12288 data: 9 9 9
17181 165/300 Read block 4188 len 20480
17182 166/300 Read block 1904 len 4096
17183 167/300 Write block 911 len 8192 data: 6 5
17184 168/300 Write block 3090 len 12288 data: 4 4 3
17185 169/300 Read block 1741 len 4096
17186 170/300 Write block 4993 len 12288 data: 3 4 4
17187 171/300 Write block 2618 len 28672 data: 9 8 7 5 6 6 8
17188 172/300 Write block 1830 len 4096 data: 4
17189 173/300 Read block 3000 len 8192
17190 174/300 Write block 2978 len 40960 data: 5 5 6 5 5 5 4 3 3 2
17191 175/300 Write block 119 len 32768 data: 4 3 4 5 6 7 7 6
17192 176/300 Read block 2920 len 16384
17193 177/300 Read block 342 len 36864
17194 178/300 Write block 1602 len 16384 data: 4 3 3 3
17195 179/300 Read block 2866 len 16384
17196 180/300 Write block 4398 len 8192 data: 6 5
17197 181/300 Flush
17198 182/300 Read block 434 len 40960
17199 183/300 Read block 4399 len 32768
17200 184/300 Write block 1052 len 24576 data: 2 2 2 2 2 2
17201 185/300 Write block 2722 len 8192 data: 4 3
17202 186/300 Read block 4887 len 8192
17203 187/300 Write block 769 len 16384 data: 4 4 5 5
17204 188/300 Read block 4441 len 12288
17205 189/300 Read block 1305 len 32768
17206 190/300 Write block 503 len 24576 data: 2 3 3 3 3 3
17207 191/300 Read block 1335 len 16384
17208 192/300 Read block 3484 len 32768
17209 193/300 Read block 1255 len 28672
17210 194/300 Read block 1617 len 8192
17211 195/300 Flush
17212 196/300 Read block 1199 len 36864
17213 197/300 Flush
17214 198/300 Write block 3311 len 24576 data: 3 3 3 2 2 2
17215 199/300 Write block 3848 len 4096 data: 6
17216 200/300 Read block 3591 len 32768
17217 201/300 Flush
17218 202/300 Read block 708 len 12288
17219 203/300 Write block 4275 len 8192 data: 4 4
17220 204/300 Write block 4514 len 32768 data: 4 3 4 4 5 4 6 5
17221 205/300 Read block 1139 len 40960
17222 206/300 Write block 3018 len 16384 data: 3 3 4 4
17223 207/300 Write block 2681 len 32768 data: 4 4 4 4 3 2 3 3
17224 208/300 Write block 2465 len 16384 data: 2 3 3 4
17225 209/300 Read block 4937 len 8192
17226 210/300 Read block 3792 len 20480
17227 211/300 Read block 688 len 12288
17228 212/300 Flush
17229 213/300 Read block 614 len 36864
17230 214/300 Write block 2168 len 28672 data: 3 3 4 4 4 4 5
17231 215/300 Write block 4523 len 8192 data: 4 4
17232 216/300 Write block 1844 len 4096 data: 4
17233 217/300 Read block 2246 len 20480
17234 218/300 Write block 3934 len 8192 data: 5 5
17235 219/300 Write block 4484 len 36864 data: 3 4 4 4 3 3 3 3 3
17236 220/300 Flush
17237 221/300 Write block 3156 len 16384 data: 5 5 3 2
17238 222/300 Read block 4822 len 40960
17239 223/300 Read block 2453 len 32768
17240 224/300 Write block 4085 len 36864 data: 5 6 5 6 7 7 8 8 7
17241 225/300 Write block 1617 len 20480 data: 4 4 4 4 4
17242 226/300 Read block 517 len 20480
17243 227/300 Flush
17244 228/300 Flush
17245 229/300 Read block 32 len 36864
17246 230/300 Write block 2316 len 24576 data: 7 7 6 6 4 3
17247 231/300 Read block 705 len 28672
17248 232/300 Flush
17249 233/300 Read block 291 len 36864
17250 234/300 Write block 2290 len 16384 data: 4 4 3 3
17251 235/300 Read block 1346 len 32768
17252 236/300 Flush
17253 237/300 Read block 277 len 20480
17254 238/300 Read block 3342 len 40960
17255 239/300 Write block 2094 len 12288 data: 4 4 2
17256 240/300 Write block 2711 len 32768 data: 3 3 3 3 5 6 5 5
17257 241/300 Write block 2905 len 12288 data: 6 5 6
17258 242/300 Write block 4555 len 32768 data: 2 3 3 3 3 4 4 5
17259 243/300 Write block 1853 len 32768 data: 5 5 4 4 4 4 5 4
17260 244/300 Write block 1581 len 28672 data: 2 2 2 2 2 2 2
17261 245/300 Write block 2324 len 32768 data: 4 4 2 2 2 2 4 4
17262 246/300 Read block 3868 len 4096
17263 247/300 Read block 4222 len 4096
17264 248/300 Flush
17265 249/300 Write block 2110 len 36864 data: 5 5 5 6 5 4 4 3 4
17266 250/300 Read block 4203 len 40960
17267 251/300 Write block 219 len 28672 data: 4 4 4 4 4 3 3
17268 252/300 Write block 2910 len 24576 data: 4 2 3 3 3 3
17269 253/300 Read block 3815 len 20480
17270 254/300 Flush
17271 255/300 Read block 3762 len 36864
17272 256/300 Read block 809 len 36864
17273 257/300 Write block 1356 len 16384 data: 5 5 4 4
17274 258/300 Read block 1919 len 24576
17275 259/300 Write block 1871 len 28672 data: 4 4 4 5 4 6 4
17276 260/300 Read block 3620 len 28672
17277 261/300 Write block 4459 len 12288 data: 2 3 2
17278 262/300 Read block 4987 len 20480
17279 263/300 Read block 3856 len 16384
17280 264/300 Read block 1984 len 16384
17281 265/300 Write block 4665 len 28672 data: 10 11 10 9 8 5 4
17282 266/300 Flush
17283 267/300 Write block 3110 len 12288 data: 3 2 2
17284 268/300 Flush
17285 269/300 Write block 1530 len 4096 data: 6
17286 270/300 Read block 4729 len 20480
17287 271/300 Read block 983 len 8192
17288 272/300 Write block 4159 len 24576 data: 8 7 7 7 7 7
17289 273/300 Read block 4437 len 24576
17290 274/300 Read block 2464 len 12288
17291 275/300 Write block 56 len 4096 data: 4
17292 276/300 Read block 415 len 20480
17293 277/300 Read block 14 len 12288
17294 278/300 Read block 3209 len 12288
17295 279/300 Write block 344 len 32768 data: 5 5 5 5 8 8 8 5
17296 280/300 Write block 1781 len 8192 data: 3 4
17297 281/300 Flush
17298 282/300 Write block 3451 len 40960 data: 5 4 4 5 5 5 5 4 3 4
17299 283/300 Read block 3341 len 12288
17300 284/300 Read block 4376 len 24576
17301 285/300 Read block 1002 len 12288
17302 286/300 Flush
17303 287/300 Write block 163 len 20480 data: 5 5 6 6 6
17304 288/300 Read block 4723 len 8192
17305 289/300 Read block 3527 len 16384
17306 290/300 Read block 34 len 16384
17307 291/300 Read block 2655 len 16384
17308 292/300 Write block 2868 len 32768 data: 4 4 7 7 7 7 6 5
17309 293/300 Read block 4021 len 20480
17310 294/300 Write block 707 len 4096 data: 5
17311 295/300 Read block 3347 len 40960
17312 296/300 Read block 3762 len 16384
17313 297/300 Read block 2819 len 24576
17314 298/300 Read block 2168 len 12288
17315 299/300 Write block 4436 len 40960 data: 5 3 3 4 4 3 3 3 3 2
17316 300/300 Write block 3816 len 40960 data: 3 4 4 5 5 4 5 5 5 5
17317 301/300 Flush
17318 Main task has work to do, go find it
17319 got dsc Start(0)
17320 start 0
17321 Jan 14 16:14:16.060 INFO request completed, latency_us: 229, response_code: 204, uri: /start/cid/0, method: POST, req_id: f067552f-a9c0-4a9c-a3de-46f0eb3f74b0, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
17322 Replay: started 0, returned:Ok(())
17323 ----------------------------------------------------------------
17324 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
17325 States: Offline Active Active
17326 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17327 5830 Acked Read 10 Sent Done Done false
17328 5831 Acked Write 9 Sent Done Done false
17329 5832 Acked Flush 0 Sent Done Done false
17330 5833 Acked Write 2 Sent Done Done false
17331 5834 Acked Read 5 Sent Done Done false
17332 5835 Acked Write 2 Sent Done Done false
17333 5836 Acked Read 7 Sent Done Done false
17334 5837 Acked Read 3 Sent Done Done false
17335 5838 Acked Read 1 Sent Done Done false
17336 5839 Acked Write 4 Sent Done Done false
17337 5840 Acked Write 1 Sent Done Done false
17338 5841 Acked Write 8 Sent Done Done false
17339 5842 Acked Read 4 Sent Done Done false
17340 5843 Acked Read 8 Sent Done Done false
17341 5844 Acked Write 8 Sent Done Done false
17342 5845 Acked Write 9 Sent Done Done false
17343 5846 Acked Read 2 Sent Done Done false
17344 5847 Acked Read 5 Sent Done Done false
17345 5848 Acked Read 6 Sent Done Done false
17346 5849 Acked Read 7 Sent Done Done false
17347 5850 Acked Read 4 Sent Done Done false
17348 5851 Acked Read 9 Sent Done Done false
17349 5852 Acked Write 2 Sent Done Done false
17350 5853 Acked Read 10 Sent Done Done false
17351 5854 Acked Read 7 Sent Done Done false
17352 5855 Acked Read 1 Sent Done Done false
17353 5856 Acked Read 2 Sent Done Done false
17354 5857 Acked Read 1 Sent Done Done false
17355 5858 Acked Read 6 Sent Done Done false
17356 5859 Acked Write 10 Sent Done Done false
17357 5860 Acked Write 8 Sent Done Done false
17358 5861 Acked Read 1 Sent Done Done false
17359 5862 Acked Read 7 Sent Done Done false
17360 5863 Acked Read 3 Sent Done Done false
17361 5864 Acked Read 4 Sent Done Done false
17362 5865 Acked Write 3 Sent Done Done false
17363 5866 Acked Write 3 Sent Done Done false
17364 5867 Acked Write 7 Sent Done Done false
17365 5868 Acked Flush 0 Sent Done Done false
17366 5869 Acked Write 10 Sent Done Done false
17367 5870 Acked Read 4 Sent Done Done false
17368 5871 Acked Write 6 Sent Done Done false
17369 5872 Acked Write 5 Sent Done Done false
17370 5873 Acked Read 7 Sent Done Done false
17371 5874 Acked Read 3 Sent Done Done false
17372 5875 Acked Write 8 Sent Done Done false
17373 5876 Acked Read 7 Sent Done Done false
17374 5877 Acked Write 9 Sent Done Done false
17375 5878 Acked Read 3 Sent Done Done false
17376 5879 Acked Read 1 Sent Done Done false
17377 5880 Acked Read 1 Sent Done Done false
17378 5881 Acked Read 5 Sent Done Done false
17379 5882 Acked Write 4 Sent Done Done false
17380 5883 Acked Read 10 Sent Done Done false
17381 5884 Acked Flush 0 Sent Done Done false
17382 5885 Acked Write 6 Sent Done Done false
17383 5886 Acked Read 7 Sent Done Done false
17384 5887 Acked Read 6 Sent Done Done false
17385 5888 Acked Write 3 Sent Done Done false
17386 5889 Acked Read 9 Sent Done Done false
17387 5890 Acked Write 8 Sent Done Done false
17388 5891 Acked Write 9 Sent Done Done false
17389 5892 Acked Write 5 Sent Done Done false
17390 5893 Acked Write 3 Sent Done Done false
17391 5894 Acked Read 2 Sent Done Done false
17392 5895 Acked Flush 0 Sent Done Done false
17393 5896 Acked Write 6 Sent Done Done false
17394 5897 Acked Write 7 Sent Done Done false
17395 5898 Acked Read 2 Sent Done Done false
17396 5899 Acked Read 7 Sent Done Done false
17397 5900 Acked Read 9 Sent Done Done false
17398 5901 Acked Write 1 Sent Done Done false
17399 5902 Acked Write 6 Sent Done Done false
17400 5903 Acked Write 9 Sent Done Done false
17401 5904 Acked Write 3 Sent Done Done false
17402 5905 Acked Read 3 Sent Done Done false
17403 5906 Acked Read 2 Sent Done Done false
17404 5907 Acked Write 8 Sent Done Done false
17405 5908 Acked Read 5 Sent Done Done false
17406 5909 Acked Write 8 Sent Done Done false
17407 5910 Acked Write 8 Sent Done Done false
17408 5911 Acked Read 7 Sent Done Done false
17409 5912 Acked Write 3 Sent Done Done false
17410 5913 Acked Read 4 Sent Done Done false
17411 5914 Acked Read 8 Sent Done Done false
17412 5915 Acked Write 7 Sent Done Done false
17413 5916 Acked Write 7 Sent Done Done false
17414 5917 Acked Read 2 Sent Done Done false
17415 5918 Acked Read 10 Sent Done Done false
17416 5919 Acked Read 6 Sent Done Done false
17417 5920 Acked Write 5 Sent Done Done false
17418 5921 Acked Write 4 Sent Done Done false
17419 5922 Acked Read 5 Sent Done Done false
17420 5923 Acked Write 5 Sent Done Done false
17421 5924 Acked Read 5 Sent Done Done false
17422 5925 Acked Write 6 Sent Done Done false
17423 5926 Acked Write 7 Sent Done Done false
17424 5927 Acked Read 6 Sent Done Done false
17425 5928 Acked Flush 0 Sent Done Done false
17426 5929 Acked Read 5 Sent Done Done false
17427 5930 Acked Write 9 Sent Done Done false
17428 5931 Acked Read 6 Sent Done Done false
17429 5932 Acked Write 10 Sent Done Done false
17430 5933 Acked Write 2 Sent Done Done false
17431 5934 Acked Write 3 Sent Done Done false
17432 5935 Acked Read 9 Sent Done Done false
17433 5936 Acked Read 7 Sent Done Done false
17434 5937 Acked Write 8 Sent Done Done false
17435 5938 Acked Write 1 Sent Done Done false
17436 5939 Acked Write 3 Sent Done Done false
17437 5940 Acked Read 10 Sent Done Done false
17438 5941 Acked Read 10 Sent Done Done false
17439 5942 Acked Read 5 Sent Done Done false
17440 5943 Acked Write 1 Sent Done Done false
17441 5944 Acked Read 10 Sent Done Done false
17442 5945 Acked Read 3 Sent Done Done false
17443 5946 Acked Write 4 Sent Done Done false
17444 5947 Acked Write 8 Sent Done Done false
17445 5948 Acked Write 4 Sent Done Done false
17446 5949 Acked Write 6 Sent Done Done false
17447 5950 Acked Write 6 Sent Done Done false
17448 5951 Acked Write 1 Sent Done Done false
17449 5952 Acked Flush 0 Sent Done Done false
17450 5953 Acked Flush 0 Sent Done Done false
17451 5954 Acked Read 4 Sent Done Done false
17452 5955 Acked Read 1 Sent Done Done false
17453 5956 Acked Write 5 Sent Done Done false
17454 5957 Acked Write 6 Sent Done Done false
17455 5958 Acked Write 2 Sent Done Done false
17456 5959 Acked Read 4 Sent Done Done false
17457 5960 Acked Read 3 Sent Done Done false
17458 5961 Acked Read 3 Sent Done Done false
17459 5962 Acked Write 1 Sent Done Done false
17460 5963 Acked Read 1 Sent Done Done false
17461 5964 Acked Write 8 Sent Done Done false
17462 5965 Acked Write 2 Sent Done Done false
17463 5966 Acked Write 6 Sent Done Done false
17464 5967 Acked Write 9 Sent Done Done false
17465 5968 Acked Write 1 Sent Done Done false
17466 5969 Acked Read 9 Sent Done Done false
17467 5970 Acked Write 5 Sent Done Done false
17468 5971 Acked Write 1 Sent Done Done false
17469 5972 Acked Read 7 Sent Done Done false
17470 5973 Acked Write 8 Sent Done Done false
17471 5974 Acked Read 6 Sent Done Done false
17472 5975 Acked Write 10 Sent Done Done false
17473 5976 Acked Write 10 Sent Done Done false
17474 5977 Acked Read 3 Sent Done Done false
17475 5978 Acked Read 9 Sent Done Done false
17476 5979 Acked Read 5 Sent Done Done false
17477 5980 Acked Read 1 Sent Done Done false
17478 5981 Acked Read 9 Sent Done Done false
17479 5982 Acked Write 9 Sent Done Done false
17480 5983 Acked Read 3 Sent Done Done false
17481 5984 Acked Write 6 Sent Done Done false
17482 5985 Acked Flush 0 Sent Done Done false
17483 5986 Acked Read 1 Sent Done Done false
17484 5987 Acked Read 2 Sent Done Done false
17485 5988 Acked Flush 0 Sent Done Done false
17486 5989 Acked Write 4 Sent Done Done false
17487 5990 Acked Write 1 Sent Done Done false
17488 5991 Acked Write 2 Sent Done Done false
17489 5992 Acked Flush 0 Sent Done Done false
17490 5993 Acked Write 3 Sent Done Done false
17491 5994 Acked Read 5 Sent Done Done false
17492 5995 Acked Read 1 Sent Done Done false
17493 5996 Acked Write 2 Sent Done Done false
17494 5997 Acked Write 3 Sent Done Done false
17495 5998 Acked Read 1 Sent Done Done false
17496 5999 Acked Write 3 Sent Done Done false
17497 6000 Acked Write 7 Sent Done Done false
17498 6001 Acked Write 1 Sent Done Done false
17499 6002 Acked Read 2 Sent Done Done false
17500 6003 Acked Write 10 Sent Done Done false
17501 6004 Acked Write 8 Sent Done Done false
17502 6005 Acked Read 4 Sent Done Done false
17503 6006 Acked Read 9 Sent Done Done false
17504 6007 Acked Write 4 Sent Done Done false
17505 6008 Acked Read 4 Sent Done Done false
17506 6009 Acked Write 2 Sent Done Done false
17507 6010 Acked Flush 0 Sent Done Done false
17508 6011 Acked Read 10 Sent Done Done false
17509 6012 Acked Read 8 Sent Done Done false
17510 6013 Acked Write 6 Sent Done Done false
17511 6014 Acked Write 2 Sent Done Done false
17512 6015 Acked Read 2 Sent Done Done false
17513 6016 Acked Write 4 Sent Done Done false
17514 6017 Acked Read 3 Sent Done Done false
17515 6018 Acked Read 8 Sent Done Done false
17516 6019 Acked Write 6 Sent Done Done false
17517 6020 Acked Read 4 Sent Done Done false
17518 6021 Acked Read 8 Sent Done Done false
17519 6022 Acked Read 7 Sent Done Done false
17520 6023 Acked Read 2 Sent Done Done false
17521 6024 Acked Flush 0 Sent Done Done false
17522 6025 Acked Read 9 Sent Done Done false
17523 6026 Acked Flush 0 Sent Done Done false
17524 6027 Acked Write 6 Sent Done Done false
17525 6028 Acked Write 1 Sent Done Done false
17526 6029 Acked Read 8 Sent Done Done false
17527 6030 Acked Flush 0 Sent Done Done false
17528 6031 Acked Read 3 Sent Done Done false
17529 6032 Acked Write 2 Sent Done Done false
17530 6033 Acked Write 8 Sent Done Done false
17531 6034 Acked Read 10 Sent Done Done false
17532 6035 Acked Write 4 Sent Done Done false
17533 6036 Acked Write 8 Sent Done Done false
17534 6037 Acked Write 4 Sent Done Done false
17535 6038 Acked Read 2 Sent Done Done false
17536 6039 Acked Read 5 Sent Done Done false
17537 6040 Acked Read 3 Sent Done Done false
17538 6041 Acked Flush 0 Sent Done Done false
17539 6042 Acked Read 9 Sent Done Done false
17540 6043 Acked Write 7 Sent Done Done false
17541 6044 Acked Write 2 Sent Done Done false
17542 6045 Acked Write 1 Sent Done Done false
17543 6046 Acked Read 5 Sent Done Done false
17544 6047 Acked Write 2 Sent Done Done false
17545 6048 Acked Write 9 Sent Done Done false
17546 6049 Acked Flush 0 Sent Done Done false
17547 6050 Acked Write 4 Sent Done Done false
17548 6051 Acked Read 10 Sent Done Done false
17549 6052 Acked Read 8 Sent Done Done false
17550 6053 Acked Write 9 Sent Done Done false
17551 6054 Acked Write 5 Sent Done Done false
17552 6055 Acked Read 5 Sent Done Done false
17553 6056 Acked Flush 0 Sent Done Done false
17554 6057 Acked Flush 0 Sent Done Done false
17555 6058 Acked Read 9 Sent Done Done false
17556 6059 Acked Write 6 Sent Done Done false
17557 6060 Acked Read 7 Sent Done Done false
17558 6061 Acked Flush 0 Sent Done Done false
17559 6062 Acked Read 9 Sent Done Done false
17560 6063 Acked Write 4 Sent Done Done false
17561 6064 Acked Read 8 Sent Done Done false
17562 6065 Acked Flush 0 Sent Done Done false
17563 6066 Acked Read 5 Sent Done Done false
17564 6067 Acked Read 10 Sent Done Done false
17565 6068 Acked Write 3 Sent Done Done false
17566 6069 Acked Write 8 Sent Done Done false
17567 6070 Acked Write 3 Sent Done Done false
17568 6071 Acked Write 8 Sent Done Done false
17569 6072 Acked Write 8 Sent Done Done false
17570 6073 Acked Write 7 Sent Done Done false
17571 6074 Acked Write 8 Sent Done Done false
17572 6075 Acked Read 1 Sent Done Done false
17573 6076 Acked Read 1 Sent Done Done false
17574 6077 Acked Flush 0 Sent Done Done false
17575 6078 Acked Write 9 Sent Done Done false
17576 6079 Acked Read 10 Sent Done Done false
17577 6080 Acked Write 7 Sent Done Done false
17578 6081 Acked Write 6 Sent Done Done false
17579 6082 Acked Read 5 Sent Done Done false
17580 6083 Acked Flush 0 Sent Done Done false
17581 6084 Acked Read 9 Sent Done Done false
17582 6085 Acked Read 9 Sent Done Done false
17583 6086 Acked Write 4 Sent Done Done false
17584 6087 Acked Read 6 Sent Done Done false
17585 6088 Acked Write 7 Sent Done Done false
17586 6089 Acked Read 7 Sent Done Done false
17587 6090 Acked Write 3 Sent Done Done false
17588 6091 Acked Read 5 Sent Done Done false
17589 6092 Acked Read 4 Sent Done Done false
17590 6093 Acked Read 4 Sent Done Done false
17591 6094 Acked Write 7 Sent Done Done false
17592 6095 Acked Flush 0 Sent Done Done false
17593 6096 Acked Write 3 Sent Done Done false
17594 6097 Acked Flush 0 Sent Done Done false
17595 6098 Acked Write 1 Sent Done Done false
17596 6099 Acked Read 5 Sent Done Done false
17597 6100 Acked Read 2 Sent Done Done false
17598 6101 Acked Write 6 Sent Done Done false
17599 6102 Acked Read 6 Sent Done Done false
17600 6103 Acked Read 3 Sent Done Done false
17601 6104 Acked Write 1 Sent Done Done false
17602 6105 Acked Read 5 Sent Done Done false
17603 6106 Acked Read 3 Sent Done Done false
17604 6107 Acked Read 3 Sent Done Done false
17605 6108 Acked Write 8 Sent Done Done false
17606 6109 Acked Write 2 Sent Done Done false
17607 6110 Acked Flush 0 Sent Done Done false
17608 6111 Acked Write 10 Sent Done Done false
17609 6112 Acked Read 3 Sent Done Done false
17610 6113 Acked Read 6 Sent Done Done false
17611 6114 Acked Read 3 Sent Done Done false
17612 6115 Acked Flush 0 Sent Done Done false
17613 6116 Acked Write 5 Sent Done Done false
17614 6117 Acked Read 2 Sent Done Done false
17615 6118 Acked Read 4 Sent Done Done false
17616 6119 Acked Read 4 Sent Done Done false
17617 6120 Acked Read 4 Sent Done Done false
17618 6121 Acked Write 8 Sent Done Done false
17619 6122 Acked Read 5 Sent Done Done false
17620 6123 Acked Write 1 Sent Done Done false
17621 6124 Acked Read 10 Sent Done Done false
17622 6125 Acked Read 4 Sent Done Done false
17623 6126 Acked Read 6 Sent Done Done false
17624 6127 Acked Read 3 Sent Done Done false
17625 6128 Acked Write 10 Sent Done Done false
17626 6129 Acked Write 10 Sent Done Done false
17627 6130 Acked Flush 0 Sent Done Done false
17628 STATES DS:0 DS:1 DS:2 TOTAL
17629 Sent 301 0 0 301
17630 Done 0 301 301 602
17631 Skipped 0 0 0 0
17632 Error 0 0 0 0
17633 Last Flush: 5829 6130 6130
17634 Downstairs last five completed: 5829 5828 5827 5826 5825
17635 Upstairs last five completed: 6130 6129 6128 6127 6126
17636 CLIENT: Up:0 ds:301 act:2
17637 [8810] got start action
17638 [8810] I am going to restart
17639 Starting downstairs at port 8810
17640 Make output file at "/tmp/dsc/downstairs-8810.txt"
17641 [8810][0] reports Starting
17642 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1219)
17643 [8810][0] reports Running
17644 ----------------------------------------------------------------
17645 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
17646 States: Offline Active Active
17647 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17648 5830 Acked Read 10 Sent Done Done false
17649 5831 Acked Write 9 Sent Done Done false
17650 5832 Acked Flush 0 Sent Done Done false
17651 5833 Acked Write 2 Sent Done Done false
17652 5834 Acked Read 5 Sent Done Done false
17653 5835 Acked Write 2 Sent Done Done false
17654 5836 Acked Read 7 Sent Done Done false
17655 5837 Acked Read 3 Sent Done Done false
17656 5838 Acked Read 1 Sent Done Done false
17657 5839 Acked Write 4 Sent Done Done false
17658 5840 Acked Write 1 Sent Done Done false
17659 5841 Acked Write 8 Sent Done Done false
17660 5842 Acked Read 4 Sent Done Done false
17661 5843 Acked Read 8 Sent Done Done false
17662 5844 Acked Write 8 Sent Done Done false
17663 5845 Acked Write 9 Sent Done Done false
17664 5846 Acked Read 2 Sent Done Done false
17665 5847 Acked Read 5 Sent Done Done false
17666 5848 Acked Read 6 Sent Done Done false
17667 5849 Acked Read 7 Sent Done Done false
17668 5850 Acked Read 4 Sent Done Done false
17669 5851 Acked Read 9 Sent Done Done false
17670 5852 Acked Write 2 Sent Done Done false
17671 5853 Acked Read 10 Sent Done Done false
17672 5854 Acked Read 7 Sent Done Done false
17673 5855 Acked Read 1 Sent Done Done false
17674 5856 Acked Read 2 Sent Done Done false
17675 5857 Acked Read 1 Sent Done Done false
17676 5858 Acked Read 6 Sent Done Done false
17677 5859 Acked Write 10 Sent Done Done false
17678 5860 Acked Write 8 Sent Done Done false
17679 5861 Acked Read 1 Sent Done Done false
17680 5862 Acked Read 7 Sent Done Done false
17681 5863 Acked Read 3 Sent Done Done false
17682 5864 Acked Read 4 Sent Done Done false
17683 5865 Acked Write 3 Sent Done Done false
17684 5866 Acked Write 3 Sent Done Done false
17685 5867 Acked Write 7 Sent Done Done false
17686 5868 Acked Flush 0 Sent Done Done false
17687 5869 Acked Write 10 Sent Done Done false
17688 5870 Acked Read 4 Sent Done Done false
17689 5871 Acked Write 6 Sent Done Done false
17690 5872 Acked Write 5 Sent Done Done false
17691 5873 Acked Read 7 Sent Done Done false
17692 5874 Acked Read 3 Sent Done Done false
17693 5875 Acked Write 8 Sent Done Done false
17694 5876 Acked Read 7 Sent Done Done false
17695 5877 Acked Write 9 Sent Done Done false
17696 5878 Acked Read 3 Sent Done Done false
17697 5879 Acked Read 1 Sent Done Done false
17698 5880 Acked Read 1 Sent Done Done false
17699 5881 Acked Read 5 Sent Done Done false
17700 5882 Acked Write 4 Sent Done Done false
17701 5883 Acked Read 10 Sent Done Done false
17702 5884 Acked Flush 0 Sent Done Done false
17703 5885 Acked Write 6 Sent Done Done false
17704 5886 Acked Read 7 Sent Done Done false
17705 5887 Acked Read 6 Sent Done Done false
17706 5888 Acked Write 3 Sent Done Done false
17707 5889 Acked Read 9 Sent Done Done false
17708 5890 Acked Write 8 Sent Done Done false
17709 5891 Acked Write 9 Sent Done Done false
17710 5892 Acked Write 5 Sent Done Done false
17711 5893 Acked Write 3 Sent Done Done false
17712 5894 Acked Read 2 Sent Done Done false
17713 5895 Acked Flush 0 Sent Done Done false
17714 5896 Acked Write 6 Sent Done Done false
17715 5897 Acked Write 7 Sent Done Done false
17716 5898 Acked Read 2 Sent Done Done false
17717 5899 Acked Read 7 Sent Done Done false
17718 5900 Acked Read 9 Sent Done Done false
17719 5901 Acked Write 1 Sent Done Done false
17720 5902 Acked Write 6 Sent Done Done false
17721 5903 Acked Write 9 Sent Done Done false
17722 5904 Acked Write 3 Sent Done Done false
17723 5905 Acked Read 3 Sent Done Done false
17724 5906 Acked Read 2 Sent Done Done false
17725 5907 Acked Write 8 Sent Done Done false
17726 5908 Acked Read 5 Sent Done Done false
17727 5909 Acked Write 8 Sent Done Done false
17728 5910 Acked Write 8 Sent Done Done false
17729 5911 Acked Read 7 Sent Done Done false
17730 5912 Acked Write 3 Sent Done Done false
17731 5913 Acked Read 4 Sent Done Done false
17732 5914 Acked Read 8 Sent Done Done false
17733 5915 Acked Write 7 Sent Done Done false
17734 5916 Acked Write 7 Sent Done Done false
17735 5917 Acked Read 2 Sent Done Done false
17736 5918 Acked Read 10 Sent Done Done false
17737 5919 Acked Read 6 Sent Done Done false
17738 5920 Acked Write 5 Sent Done Done false
17739 5921 Acked Write 4 Sent Done Done false
17740 5922 Acked Read 5 Sent Done Done false
17741 5923 Acked Write 5 Sent Done Done false
17742 5924 Acked Read 5 Sent Done Done false
17743 5925 Acked Write 6 Sent Done Done false
17744 5926 Acked Write 7 Sent Done Done false
17745 5927 Acked Read 6 Sent Done Done false
17746 5928 Acked Flush 0 Sent Done Done false
17747 5929 Acked Read 5 Sent Done Done false
17748 5930 Acked Write 9 Sent Done Done false
17749 5931 Acked Read 6 Sent Done Done false
17750 5932 Acked Write 10 Sent Done Done false
17751 5933 Acked Write 2 Sent Done Done false
17752 5934 Acked Write 3 Sent Done Done false
17753 5935 Acked Read 9 Sent Done Done false
17754 5936 Acked Read 7 Sent Done Done false
17755 5937 Acked Write 8 Sent Done Done false
17756 5938 Acked Write 1 Sent Done Done false
17757 5939 Acked Write 3 Sent Done Done false
17758 5940 Acked Read 10 Sent Done Done false
17759 5941 Acked Read 10 Sent Done Done false
17760 5942 Acked Read 5 Sent Done Done false
17761 5943 Acked Write 1 Sent Done Done false
17762 5944 Acked Read 10 Sent Done Done false
17763 5945 Acked Read 3 Sent Done Done false
17764 5946 Acked Write 4 Sent Done Done false
17765 5947 Acked Write 8 Sent Done Done false
17766 5948 Acked Write 4 Sent Done Done false
17767 5949 Acked Write 6 Sent Done Done false
17768 5950 Acked Write 6 Sent Done Done false
17769 5951 Acked Write 1 Sent Done Done false
17770 5952 Acked Flush 0 Sent Done Done false
17771 5953 Acked Flush 0 Sent Done Done false
17772 5954 Acked Read 4 Sent Done Done false
17773 5955 Acked Read 1 Sent Done Done false
17774 5956 Acked Write 5 Sent Done Done false
17775 5957 Acked Write 6 Sent Done Done false
17776 5958 Acked Write 2 Sent Done Done false
17777 5959 Acked Read 4 Sent Done Done false
17778 5960 Acked Read 3 Sent Done Done false
17779 5961 Acked Read 3 Sent Done Done false
17780 5962 Acked Write 1 Sent Done Done false
17781 5963 Acked Read 1 Sent Done Done false
17782 5964 Acked Write 8 Sent Done Done false
17783 5965 Acked Write 2 Sent Done Done false
17784 5966 Acked Write 6 Sent Done Done false
17785 5967 Acked Write 9 Sent Done Done false
17786 5968 Acked Write 1 Sent Done Done false
17787 5969 Acked Read 9 Sent Done Done false
17788 5970 Acked Write 5 Sent Done Done false
17789 5971 Acked Write 1 Sent Done Done false
17790 5972 Acked Read 7 Sent Done Done false
17791 5973 Acked Write 8 Sent Done Done false
17792 5974 Acked Read 6 Sent Done Done false
17793 5975 Acked Write 10 Sent Done Done false
17794 5976 Acked Write 10 Sent Done Done false
17795 5977 Acked Read 3 Sent Done Done false
17796 5978 Acked Read 9 Sent Done Done false
17797 5979 Acked Read 5 Sent Done Done false
17798 5980 Acked Read 1 Sent Done Done false
17799 5981 Acked Read 9 Sent Done Done false
17800 5982 Acked Write 9 Sent Done Done false
17801 5983 Acked Read 3 Sent Done Done false
17802 5984 Acked Write 6 Sent Done Done false
17803 5985 Acked Flush 0 Sent Done Done false
17804 5986 Acked Read 1 Sent Done Done false
17805 5987 Acked Read 2 Sent Done Done false
17806 5988 Acked Flush 0 Sent Done Done false
17807 5989 Acked Write 4 Sent Done Done false
17808 5990 Acked Write 1 Sent Done Done false
17809 5991 Acked Write 2 Sent Done Done false
17810 5992 Acked Flush 0 Sent Done Done false
17811 5993 Acked Write 3 Sent Done Done false
17812 5994 Acked Read 5 Sent Done Done false
17813 5995 Acked Read 1 Sent Done Done false
17814 5996 Acked Write 2 Sent Done Done false
17815 5997 Acked Write 3 Sent Done Done false
17816 5998 Acked Read 1 Sent Done Done false
17817 5999 Acked Write 3 Sent Done Done false
17818 6000 Acked Write 7 Sent Done Done false
17819 6001 Acked Write 1 Sent Done Done false
17820 6002 Acked Read 2 Sent Done Done false
17821 6003 Acked Write 10 Sent Done Done false
17822 6004 Acked Write 8 Sent Done Done false
17823 6005 Acked Read 4 Sent Done Done false
17824 6006 Acked Read 9 Sent Done Done false
17825 6007 Acked Write 4 Sent Done Done false
17826 6008 Acked Read 4 Sent Done Done false
17827 6009 Acked Write 2 Sent Done Done false
17828 6010 Acked Flush 0 Sent Done Done false
17829 6011 Acked Read 10 Sent Done Done false
17830 6012 Acked Read 8 Sent Done Done false
17831 6013 Acked Write 6 Sent Done Done false
17832 6014 Acked Write 2 Sent Done Done false
17833 6015 Acked Read 2 Sent Done Done false
17834 6016 Acked Write 4 Sent Done Done false
17835 6017 Acked Read 3 Sent Done Done false
17836 6018 Acked Read 8 Sent Done Done false
17837 6019 Acked Write 6 Sent Done Done false
17838 6020 Acked Read 4 Sent Done Done false
17839 6021 Acked Read 8 Sent Done Done false
17840 6022 Acked Read 7 Sent Done Done false
17841 6023 Acked Read 2 Sent Done Done false
17842 6024 Acked Flush 0 Sent Done Done false
17843 6025 Acked Read 9 Sent Done Done false
17844 6026 Acked Flush 0 Sent Done Done false
17845 6027 Acked Write 6 Sent Done Done false
17846 6028 Acked Write 1 Sent Done Done false
17847 6029 Acked Read 8 Sent Done Done false
17848 6030 Acked Flush 0 Sent Done Done false
17849 6031 Acked Read 3 Sent Done Done false
17850 6032 Acked Write 2 Sent Done Done false
17851 6033 Acked Write 8 Sent Done Done false
17852 6034 Acked Read 10 Sent Done Done false
17853 6035 Acked Write 4 Sent Done Done false
17854 6036 Acked Write 8 Sent Done Done false
17855 6037 Acked Write 4 Sent Done Done false
17856 6038 Acked Read 2 Sent Done Done false
17857 6039 Acked Read 5 Sent Done Done false
17858 6040 Acked Read 3 Sent Done Done false
17859 6041 Acked Flush 0 Sent Done Done false
17860 6042 Acked Read 9 Sent Done Done false
17861 6043 Acked Write 7 Sent Done Done false
17862 6044 Acked Write 2 Sent Done Done false
17863 6045 Acked Write 1 Sent Done Done false
17864 6046 Acked Read 5 Sent Done Done false
17865 6047 Acked Write 2 Sent Done Done false
17866 6048 Acked Write 9 Sent Done Done false
17867 6049 Acked Flush 0 Sent Done Done false
17868 6050 Acked Write 4 Sent Done Done false
17869 6051 Acked Read 10 Sent Done Done false
17870 6052 Acked Read 8 Sent Done Done false
17871 6053 Acked Write 9 Sent Done Done false
17872 6054 Acked Write 5 Sent Done Done false
17873 6055 Acked Read 5 Sent Done Done false
17874 6056 Acked Flush 0 Sent Done Done false
17875 6057 Acked Flush 0 Sent Done Done false
17876 6058 Acked Read 9 Sent Done Done false
17877 6059 Acked Write 6 Sent Done Done false
17878 6060 Acked Read 7 Sent Done Done false
17879 6061 Acked Flush 0 Sent Done Done false
17880 6062 Acked Read 9 Sent Done Done false
17881 6063 Acked Write 4 Sent Done Done false
17882 6064 Acked Read 8 Sent Done Done false
17883 6065 Acked Flush 0 Sent Done Done false
17884 6066 Acked Read 5 Sent Done Done false
17885 6067 Acked Read 10 Sent Done Done false
17886 6068 Acked Write 3 Sent Done Done false
17887 6069 Acked Write 8 Sent Done Done false
17888 6070 Acked Write 3 Sent Done Done false
17889 6071 Acked Write 8 Sent Done Done false
17890 6072 Acked Write 8 Sent Done Done false
17891 6073 Acked Write 7 Sent Done Done false
17892 6074 Acked Write 8 Sent Done Done false
17893 6075 Acked Read 1 Sent Done Done false
17894 6076 Acked Read 1 Sent Done Done false
17895 6077 Acked Flush 0 Sent Done Done false
17896 6078 Acked Write 9 Sent Done Done false
17897 6079 Acked Read 10 Sent Done Done false
17898 6080 Acked Write 7 Sent Done Done false
17899 6081 Acked Write 6 Sent Done Done false
17900 6082 Acked Read 5 Sent Done Done false
17901 6083 Acked Flush 0 Sent Done Done false
17902 6084 Acked Read 9 Sent Done Done false
17903 6085 Acked Read 9 Sent Done Done false
17904 6086 Acked Write 4 Sent Done Done false
17905 6087 Acked Read 6 Sent Done Done false
17906 6088 Acked Write 7 Sent Done Done false
17907 6089 Acked Read 7 Sent Done Done false
17908 6090 Acked Write 3 Sent Done Done false
17909 6091 Acked Read 5 Sent Done Done false
17910 6092 Acked Read 4 Sent Done Done false
17911 6093 Acked Read 4 Sent Done Done false
17912 6094 Acked Write 7 Sent Done Done false
17913 6095 Acked Flush 0 Sent Done Done false
17914 6096 Acked Write 3 Sent Done Done false
17915 6097 Acked Flush 0 Sent Done Done false
17916 6098 Acked Write 1 Sent Done Done false
17917 6099 Acked Read 5 Sent Done Done false
17918 6100 Acked Read 2 Sent Done Done false
17919 6101 Acked Write 6 Sent Done Done false
17920 6102 Acked Read 6 Sent Done Done false
17921 6103 Acked Read 3 Sent Done Done false
17922 6104 Acked Write 1 Sent Done Done false
17923 6105 Acked Read 5 Sent Done Done false
17924 6106 Acked Read 3 Sent Done Done false
17925 6107 Acked Read 3 Sent Done Done false
17926 6108 Acked Write 8 Sent Done Done false
17927 6109 Acked Write 2 Sent Done Done false
17928 6110 Acked Flush 0 Sent Done Done false
17929 6111 Acked Write 10 Sent Done Done false
17930 6112 Acked Read 3 Sent Done Done false
17931 6113 Acked Read 6 Sent Done Done false
17932 6114 Acked Read 3 Sent Done Done false
17933 6115 Acked Flush 0 Sent Done Done false
17934 6116 Acked Write 5 Sent Done Done false
17935 6117 Acked Read 2 Sent Done Done false
17936 6118 Acked Read 4 Sent Done Done false
17937 6119 Acked Read 4 Sent Done Done false
17938 6120 Acked Read 4 Sent Done Done false
17939 6121 Acked Write 8 Sent Done Done false
17940 6122 Acked Read 5 Sent Done Done false
17941 6123 Acked Write 1 Sent Done Done false
17942 6124 Acked Read 10 Sent Done Done false
17943 6125 Acked Read 4 Sent Done Done false
17944 6126 Acked Read 6 Sent Done Done false
17945 6127 Acked Read 3 Sent Done Done false
17946 6128 Acked Write 10 Sent Done Done false
17947 6129 Acked Write 10 Sent Done Done false
17948 6130 Acked Flush 0 Sent Done Done false
17949 STATES DS:0 DS:1 DS:2 TOTAL
17950 Sent 301 0 0 301
17951 Done 0 301 301 602
17952 Skipped 0 0 0 0
17953 Error 0 0 0 0
17954 Last Flush: 5829 6130 6130
17955 Downstairs last five completed: 5829 5828 5827 5826 5825
17956 Upstairs last five completed: 6130 6129 6128 6127 6126
17957 CLIENT: Up:0 ds:301 act:2
179582025-01-14T16:14:21.831ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
179592025-01-14T16:14:21.831ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:34138) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
179602025-01-14T16:14:21.832ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
179612025-01-14T16:14:21.833ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
179622025-01-14T16:14:21.833ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
179632025-01-14T16:14:21.833ZINFOcrucible: send last flush ID to this DS: 5829 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
179642025-01-14T16:14:21.833ZINFOcrucible: Replied this last flush ID: 5829 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
179652025-01-14T16:14:21.833ZINFOcrucible: [0] client re-new 301 jobs since flush 5829 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
179662025-01-14T16:14:21.833ZINFOcrucible: [0] Replayed 301 jobs since flush: 5829 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
17967 ----------------------------------------------------------------
17968 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
17969 Downstairs last five completed: 6130 6129 6128 6127 6126
17970 Upstairs last five completed: 6130 6129 6128 6127 6126
17971 CLIENT: Up:0 ds:0 act:3
17972 Replay: All jobs finished
17973 Main task has work to do, go find it
17974 got dsc Stop(0)
17975 stop 0
17976 [8810] Got stop action so:false kr:false
17977 Jan 14 16:14:24.065 INFO request completed, latency_us: 368, response_code: 204, uri: /stop/cid/0, method: POST, req_id: ae50df5f-dfce-49b9-bb29-a99bd8664f2b, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
179782025-01-14T16:14:24.065ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
17979 Jan 14 16:14:24.065 INFO request completed, latency_us:{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-14T16:14:24.065850085Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
17980 111, response_code: 200, uri: /state/cid/0, method: GET{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-14T16:14:24.065870843Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
17981 , req_id: 1e8f169a-3460-4c1a-9500-29302b11db3e, remote_addr{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-14T16:14:24.065906269Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
17982 : 127.0.0.1:61942, local_addr: 127.0.0.1:9998
17983 [8810] Exited with: ExitStatus(unix_wait_status(9))
17984 [8810][0] reports Exit
17985 Jan 14 16:14:28.068 INFO request completed, latency_us: 196, response_code: 200, uri: /state/cid/0, method: GET, req_id: b1653292-cc07-45b0-a6c1-24121a8e31b0, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
17986 001/300 Flush
17987 002/300 Write block 451 len 24576 data: 5 5 6 5 6 6
17988 003/300 Read block 501 len 40960
17989 004/300 Read block 1791 len 28672
17990 005/300 Write block 2517 len 20480 data: 2 2 3 3 3
17991 006/300 Write block 949 len 12288 data: 3 3 3
17992 007/300 Write block 506 len 40960 data: 4 4 4 5 5 5 7 7 7 7
17993 008/300 Write block 1988 len 40960 data: 4 5 5 7 7 8 6 7 8 7
17994 009/300 Read block 13 len 12288
17995 010/300 Read block 1974 len 4096
17996 011/300 Read block 630 len 40960
17997 012/300 Flush
17998 013/300 Read block 4662 len 36864
17999 014/300 Flush
18000 015/300 Read block 2816 len 36864
18001 016/300 Write block 4917 len 40960 data: 5 4 4 3 3 2 2 2 3 4
18002 017/300 Read block 2089 len 32768
18003 018/300 Read block 729 len 16384
18004 019/300 Read block 4203 len 20480
18005 020/300 Read block 2607 len 28672
18006 021/300 Read block 4516 len 24576
18007 022/300 Flush
18008 023/300 Write block 3045 len 40960 data: 4 4 4 4 5 7 6 5 4 3
18009 024/300 Write block 2905 len 32768 data: 7 6 7 7 5 5 3 4
18010 025/300 Write block 2489 len 36864 data: 3 3 3 3 3 2 2 2 2
18011 026/300 Write block 2273 len 20480 data: 3 3 4 4 3
18012 027/300 Read block 1816 len 16384
18013 028/300 Read block 473 len 8192
18014 029/300 Read block 2691 len 24576
18015 030/300 Read block 214 len 4096
18016 031/300 Write block 3774 len 20480 data: 7 6 6 5 5
18017 032/300 Write block 4575 len 40960 data: 3 3 4 4 3 3 3 3 2 3
18018 033/300 Write block 1568 len 4096 data: 7
18019 034/300 Read block 3683 len 12288
18020 035/300 Read block 3804 len 8192
18021 036/300 Read block 4441 len 32768
18022 037/300 Read block 2095 len 12288
18023 038/300 Write block 4066 len 36864 data: 4 4 2 4 4 3 3 3 3
18024 039/300 Read block 3989 len 32768
18025 040/300 Read block 2093 len 8192
18026 041/300 Write block 1790 len 16384 data: 3 3 2 3
18027 042/300 Read block 1388 len 36864
18028 043/300 Read block 4984 len 4096
18029 044/300 Write block 3562 len 36864 data: 2 2 2 2 2 3 3 4 4
18030 045/300 Write block 4412 len 36864 data: 4 4 3 4 5 6 5 5 6
18031 046/300 Read block 218 len 8192
18032 047/300 Write block 1846 len 28672 data: 5 6 7 5 4 5 6
18033 048/300 Flush
18034 049/300 Read block 3874 len 4096
18035 050/300 Read block 1678 len 8192
18036 051/300 Read block 2506 len 24576
18037 052/300 Read block 2842 len 28672
18038 053/300 Read block 2337 len 20480
18039 054/300 Write block 4211 len 20480 data: 6 5 5 5 4
18040 055/300 Write block 2492 len 20480 data: 4 4 3 3 3
18041 056/300 Flush
18042 057/300 Read block 4504 len 20480
18043 058/300 Read block 1758 len 20480
18044 059/300 Write block 3954 len 40960 data: 5 5 5 5 6 7 8 7 7 4
18045 060/300 Flush
18046 061/300 Write block 1754 len 24576 data: 8 8 7 6 4 3
18047 062/300 Read block 3692 len 12288
18048 063/300 Flush
18049 064/300 Read block 2140 len 16384
18050 065/300 Write block 2285 len 20480 data: 6 5 4 5 4
18051 066/300 Flush
18052 067/300 Write block 1468 len 32768 data: 5 5 5 5 5 5 5 4
18053 068/300 Write block 4091 len 4096 data: 9
18054 069/300 Read block 84 len 4096
18055 070/300 Read block 2471 len 20480
18056 071/300 Write block 2203 len 20480 data: 5 5 5 5 5
18057 072/300 Write block 3519 len 8192 data: 5 5
18058 073/300 Flush
18059 074/300 Read block 4299 len 20480
18060 075/300 Write block 3726 len 8192 data: 3 4
18061 076/300 Read block 1148 len 24576
18062 077/300 Write block 4123 len 16384 data: 5 6 7 6
18063 078/300 Read block 1594 len 12288
18064 079/300 Write block 1673 len 24576 data: 6 6 5 5 5 5
18065 080/300 Read block 3601 len 24576
18066 081/300 Read block 1230 len 24576
18067 082/300 Write block 2552 len 20480 data: 4 4 4 4 5
18068 083/300 Write block 4503 len 12288 data: 3 3 5
18069 084/300 Flush
18070 085/300 Read block 2317 len 4096
18071 086/300 Read block 3515 len 24576
18072 087/300 Read block 2311 len 32768
18073 088/300 Write block 1951 len 32768 data: 5 4 4 5 6 5 6 3
18074 089/300 Write block 2730 len 16384 data: 4 4 3 3
18075 090/300 Read block 1879 len 12288
18076 091/300 Read block 4459 len 8192
18077 092/300 Read block 434 len 16384
18078 093/300 Write block 3057 len 20480 data: 4 3 3 3 4
18079 094/300 Read block 701 len 16384
18080 095/300 Flush
18081 096/300 Flush
18082 097/300 Write block 2856 len 16384 data: 5 5 5 5
18083 098/300 Write block 467 len 28672 data: 4 2 3 5 6 6 6
18084 099/300 Read block 1790 len 28672
18085 100/300 Read block 4339 len 12288
18086 101/300 Write block 4499 len 8192 data: 4 5
18087 102/300 Write block 4164 len 16384 data: 8 4 5 2
18088 103/300 Write block 2098 len 24576 data: 2 3 4 5 4 4
18089 104/300 Write block 3303 len 28672 data: 3 3 4 4 3 3 3
18090 105/300 Flush
18091 106/300 Read block 1940 len 4096
18092 107/300 Write block 560 len 24576 data: 8 6 5 5 6 4
18093 108/300 Write block 1734 len 8192 data: 3 3
18094 109/300 Read block 3816 len 28672
18095 110/300 Read block 1119 len 32768
18096 111/300 Write block 4487 len 20480 data: 5 4 4 4 4
18097 112/300 Read block 3478 len 20480
18098 113/300 Read block 3421 len 32768
18099 114/300 Read block 4850 len 32768
18100 115/300 Read block 1312 len 36864
18101 116/300 Read block 850 len 24576
18102 117/300 Write block 3802 len 28672 data: 3 3 3 2 5 5 5
18103 118/300 Write block 4988 len 40960 data: 3 3 3 3 2 4 5 5 4 4
18104 119/300 Write block 3909 len 32768 data: 4 4 4 4 5 4 5 5
18105 120/300 Read block 4151 len 32768
18106 121/300 Read block 1327 len 20480
18107 122/300 Flush
18108 123/300 Write block 810 len 36864 data: 7 6 5 4 5 6 6 5 4
18109 124/300 Read block 3907 len 32768
18110 125/300 Write block 3312 len 32768 data: 4 4 3 3 3 2 2 3
18111 126/300 Read block 4859 len 32768
18112 127/300 Write block 4835 len 12288 data: 2 2 3
18113 128/300 Write block 3604 len 16384 data: 5 5 5 4
18114 129/300 Write block 4386 len 32768 data: 5 3 4 5 6 7 9 9
18115 130/300 Read block 962 len 28672
18116 131/300 Write block 2000 len 28672 data: 5 4 4 4 6 4 4
18117 132/300 Write block 2363 len 4096 data: 3
18118 133/300 Read block 3302 len 12288
18119 134/300 Write block 4002 len 4096 data: 9
18120 135/300 Read block 1303 len 28672
18121 136/300 Read block 1810 len 8192
18122 137/300 Write block 1945 len 4096 data: 3
18123 138/300 Read block 3607 len 20480
18124 139/300 Write block 4639 len 32768 data: 3 3 4 4 5 5 5 5
18125 140/300 Read block 1735 len 4096
18126 141/300 Read block 4479 len 36864
18127 142/300 Write block 312 len 40960 data: 6 6 5 5 5 6 5 6 6 4
18128 143/300 Flush
18129 144/300 Write block 2876 len 24576 data: 6 6 7 6 3 3
18130 145/300 Read block 2182 len 8192
18131 146/300 Write block 187 len 4096 data: 7
18132 147/300 Flush
18133 148/300 Read block 2093 len 4096
18134 149/300 Read block 3033 len 32768
18135 150/300 Read block 4833 len 28672
18136 151/300 Read block 1602 len 32768
18137 152/300 Write block 1639 len 20480 data: 3 3 3 2 3
18138 153/300 Read block 3047 len 16384
18139 154/300 Write block 372 len 8192 data: 3 4
18140 155/300 Flush
18141 156/300 Write block 1808 len 4096 data: 3
18142 157/300 Read block 2029 len 28672
18143 158/300 Write block 1803 len 36864 data: 4 3 4 3 3 4 4 4 4
18144 159/300 Read block 1513 len 40960
18145 160/300 Read block 919 len 28672
18146 161/300 Read block 1624 len 28672
18147 162/300 Write block 908 len 12288 data: 6 6 6
18148 163/300 Write block 898 len 28672 data: 4 5 5 4 4 4 5
18149 164/300 Flush
18150 165/300 Write block 4216 len 28672 data: 3 3 5 5 5 5 6
18151 166/300 Write block 566 len 16384 data: 2 2 3 3
18152 167/300 Read block 3891 len 36864
18153 168/300 Write block 1510 len 36864 data: 4 3 3 3 3 3 2 2 2
18154 169/300 Write block 4949 len 24576 data: 4 4 2 3 5 5
18155 170/300 Read block 1236 len 32768
18156 171/300 Write block 4881 len 8192 data: 4 4
18157 172/300 Read block 3410 len 4096
18158 173/300 Write block 4365 len 16384 data: 6 6 5 6
18159 174/300 Read block 41 len 12288
18160 175/300 Read block 3799 len 16384
18161 176/300 Write block 3999 len 24576 data: 9 7 9 10 7 5
18162 177/300 Write block 4497 len 24576 data: 4 4 5 6 5 5
18163 178/300 Write block 2908 len 16384 data: 8 6 6 4
18164 179/300 Flush
18165 180/300 Read block 915 len 40960
18166 181/300 Flush
18167 182/300 Write block 4904 len 16384 data: 3 4 3 3
18168 183/300 Write block 4043 len 36864 data: 4 5 5 6 5 5 5 5 3
18169 184/300 Write block 590 len 32768 data: 6 5 5 4 3 3 4 3
18170 185/300 Flush
18171 186/300 Read block 954 len 24576
18172 187/300 Write block 2796 len 36864 data: 6 5 4 4 4 5 3 6 4
18173 188/300 Read block 839 len 32768
18174 189/300 Write block 2719 len 8192 data: 5 6
18175 190/300 Write block 2246 len 20480 data: 4 4 4 3 2
18176 191/300 Flush
18177 192/300 Write block 3928 len 24576 data: 3 3 4 4 5 5
18178 193/300 Read block 2620 len 12288
18179 194/300 Write block 4680 len 20480 data: 5 4 4 4 3
18180 195/300 Flush
18181 196/300 Write block 4432 len 24576 data: 4 4 5 5 6 4
18182 197/300 Read block 4913 len 20480
18183 198/300 Write block 1213 len 40960 data: 2 2 3 3 4 5 6 5 5 5
18184 199/300 Read block 1657 len 16384
18185 200/300 Read block 2348 len 8192
18186 201/300 Read block 3010 len 16384
18187 202/300 Write block 4359 len 4096 data: 3
18188 203/300 Read block 808 len 20480
18189 204/300 Flush
18190 205/300 Write block 4657 len 32768 data: 2 2 3 3 6 7 7 9
18191 206/300 Read block 91 len 28672
18192 207/300 Write block 3173 len 36864 data: 2 3 4 4 5 5 4 5 5
18193 208/300 Read block 1352 len 36864
18194 209/300 Read block 765 len 24576
18195 210/300 Read block 2694 len 16384
18196 211/300 Write block 3495 len 8192 data: 3 3
18197 212/300 Read block 3303 len 4096
18198 213/300 Write block 4406 len 4096 data: 4
18199 214/300 Read block 1661 len 32768
18200 215/300 Read block 4906 len 8192
18201 216/300 Read block 2342 len 20480
18202 217/300 Read block 1158 len 40960
18203 218/300 Read block 3358 len 40960
18204 219/300 Write block 1944 len 4096 data: 3
18205 220/300 Read block 4347 len 8192
18206 221/300 Read block 4749 len 40960
18207 222/300 Read block 2462 len 28672
18208 223/300 Write block 4139 len 24576 data: 6 5 5 5 5 4
18209 224/300 Read block 892 len 32768
18210 225/300 Read block 4319 len 28672
18211 226/300 Write block 1720 len 12288 data: 4 4 6
18212 227/300 Flush
18213 228/300 Flush
18214 229/300 Write block 1365 len 16384 data: 3 4 4 3
18215 230/300 Flush
18216 231/300 Write block 3963 len 24576 data: 5 3 3 4 4 4
18217 232/300 Read block 3143 len 8192
18218 233/300 Read block 4200 len 40960
18219 234/300 Write block 3262 len 8192 data: 4 6
18220 235/300 Read block 652 len 20480
18221 236/300 Read block 3179 len 20480
18222 237/300 Write block 4287 len 36864 data: 2 2 2 2 3 3 4 4 6
18223 238/300 Read block 210 len 4096
18224 239/300 Read block 477 len 36864
18225 240/300 Write block 4099 len 20480 data: 6 5 5 4 5
18226 241/300 Read block 93 len 8192
18227 242/300 Read block 894 len 8192
18228 243/300 Write block 3429 len 40960 data: 6 5 5 4 2 2 2 2 2 2
18229 244/300 Write block 3292 len 28672 data: 4 5 6 6 6 6 7
18230 245/300 Flush
18231 246/300 Write block 4920 len 12288 data: 4 4 3
18232 247/300 Flush
18233 248/300 Write block 3157 len 16384 data: 6 4 3 4
18234 249/300 Read block 924 len 20480
18235 250/300 Write block 2989 len 24576 data: 3 3 4 4 5 5
18236 251/300 Write block 534 len 16384 data: 4 4 5 4
18237 252/300 Read block 1680 len 4096
18238 253/300 Read block 995 len 32768
18239 254/300 Write block 1494 len 28672 data: 4 5 5 4 4 3 4
18240 255/300 Read block 2197 len 40960
18241 256/300 Read block 1821 len 24576
18242 257/300 Read block 2096 len 40960
18243 258/300 Write block 2475 len 32768 data: 5 5 5 6 6 5 6 6
18244 259/300 Write block 4188 len 24576 data: 3 3 3 3 2 2
18245 260/300 Flush
18246 261/300 Read block 4694 len 36864
18247 262/300 Read block 4673 len 40960
18248 263/300 Write block 4239 len 24576 data: 3 3 3 4 3 3
18249 264/300 Read block 4586 len 24576
18250 265/300 Read block 2036 len 36864
18251 266/300 Flush
18252 267/300 Write block 3535 len 12288 data: 5 5 6
18253 268/300 Read block 2861 len 12288
18254 269/300 Read block 3906 len 8192
18255 270/300 Write block 1986 len 32768 data: 3 3 5 6 6 8 8 9
18256 271/300 Write block 368 len 20480 data: 2 3 3 3 4
18257 272/300 Read block 3101 len 32768
18258 273/300 Write block 12 len 12288 data: 2 3 5
18259 274/300 Read block 808 len 24576
18260 275/300 Flush
18261 276/300 Read block 3168 len 36864
18262 277/300 Write block 401 len 4096 data: 4
18263 278/300 Write block 2047 len 40960 data: 3 3 4 4 5 4 5 5 5 5
18264 279/300 Read block 1668 len 32768
18265 280/300 Read block 3638 len 12288
18266 281/300 Read block 392 len 20480
18267 282/300 Write block 1550 len 28672 data: 4 4 5 5 4 4 5
18268 283/300 Read block 2242 len 4096
18269 284/300 Write block 3573 len 40960 data: 4 5 5 4 4 4 4 5 4 4
18270 285/300 Write block 1092 len 40960 data: 2 2 2 2 3 3 3 4 2 2
18271 286/300 Read block 505 len 32768
18272 287/300 Write block 495 len 16384 data: 3 3 4 4
18273 288/300 Read block 2747 len 28672
18274 289/300 Read block 3240 len 8192
18275 290/300 Write block 3698 len 28672 data: 2 3 3 3 3 4 5
18276 291/300 Write block 1152 len 36864 data: 7 5 5 5 5 4 5 4 4
18277 292/300 Read block 199 len 32768
18278 293/300 Write block 2191 len 40960 data: 5 6 6 7 6 5 6 5 7 6
18279 294/300 Read block 480 len 24576
18280 295/300 Read block 3676 len 36864
18281 296/300 Read block 259 len 32768
18282 297/300 Write block 8 len 12288 data: 3 3 3
18283 298/300 Read block 612 len 36864
18284 299/300 Read block 3598 len 36864
18285 300/300 Read block 2073 len 4096
18286 301/300 Write block 4225 len 28672 data: 4 3 2 3 3 2 3
18287 Main task has work to do, go find it
18288 got dsc Start(0)
18289 start 0
18290 Jan 14 16:14:28.308 INFO request completed, latency_us: 248, response_code: 204, uri: /start/cid/0, method: POST, req_id: 6f90fe55-0a81-40a2-a68a-ad998d4a6736, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
18291 Replay: started 0, returned:Ok(())
18292 ----------------------------------------------------------------
18293 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
18294 States: Offline Active Active
18295 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18296 6131 Acked Flush 0 Sent Done Done false
18297 6132 Acked Write 6 Sent Done Done false
18298 6133 Acked Read 10 Sent Done Done false
18299 6134 Acked Read 7 Sent Done Done false
18300 6135 Acked Write 5 Sent Done Done false
18301 6136 Acked Write 3 Sent Done Done false
18302 6137 Acked Write 10 Sent Done Done false
18303 6138 Acked Write 10 Sent Done Done false
18304 6139 Acked Read 3 Sent Done Done false
18305 6140 Acked Read 1 Sent Done Done false
18306 6141 Acked Read 10 Sent Done Done false
18307 6142 Acked Flush 0 Sent Done Done false
18308 6143 Acked Read 9 Sent Done Done false
18309 6144 Acked Flush 0 Sent Done Done false
18310 6145 Acked Read 9 Sent Done Done false
18311 6146 Acked Write 10 Sent Done Done false
18312 6147 Acked Read 8 Sent Done Done false
18313 6148 Acked Read 4 Sent Done Done false
18314 6149 Acked Read 5 Sent Done Done false
18315 6150 Acked Read 7 Sent Done Done false
18316 6151 Acked Read 6 Sent Done Done false
18317 6152 Acked Flush 0 Sent Done Done false
18318 6153 Acked Write 10 Sent Done Done false
18319 6154 Acked Write 8 Sent Done Done false
18320 6155 Acked Write 9 Sent Done Done false
18321 6156 Acked Write 5 Sent Done Done false
18322 6157 Acked Read 4 Sent Done Done false
18323 6158 Acked Read 2 Sent Done Done false
18324 6159 Acked Read 6 Sent Done Done false
18325 6160 Acked Read 1 Sent Done Done false
18326 6161 Acked Write 5 Sent Done Done false
18327 6162 Acked Write 10 Sent Done Done false
18328 6163 Acked Write 1 Sent Done Done false
18329 6164 Acked Read 3 Sent Done Done false
18330 6165 Acked Read 2 Sent Done Done false
18331 6166 Acked Read 8 Sent Done Done false
18332 6167 Acked Read 3 Sent Done Done false
18333 6168 Acked Write 9 Sent Done Done false
18334 6169 Acked Read 8 Sent Done Done false
18335 6170 Acked Read 2 Sent Done Done false
18336 6171 Acked Write 4 Sent Done Done false
18337 6172 Acked Read 9 Sent Done Done false
18338 6173 Acked Read 1 Sent Done Done false
18339 6174 Acked Write 9 Sent Done Done false
18340 6175 Acked Write 9 Sent Done Done false
18341 6176 Acked Read 2 Sent Done Done false
18342 6177 Acked Write 7 Sent Done Done false
18343 6178 Acked Flush 0 Sent Done Done false
18344 6179 Acked Read 1 Sent Done Done false
18345 6180 Acked Read 2 Sent Done Done false
18346 6181 Acked Read 6 Sent Done Done false
18347 6182 Acked Read 7 Sent Done Done false
18348 6183 Acked Read 5 Sent Done Done false
18349 6184 Acked Write 5 Sent Done Done false
18350 6185 Acked Write 5 Sent Done Done false
18351 6186 Acked Flush 0 Sent Done Done false
18352 6187 Acked Read 5 Sent Done Done false
18353 6188 Acked Read 5 Sent Done Done false
18354 6189 Acked Write 10 Sent Done Done false
18355 6190 Acked Flush 0 Sent Done Done false
18356 6191 Acked Write 6 Sent Done Done false
18357 6192 Acked Read 3 Sent Done Done false
18358 6193 Acked Flush 0 Sent Done Done false
18359 6194 Acked Read 4 Sent Done Done false
18360 6195 Acked Write 5 Sent Done Done false
18361 6196 Acked Flush 0 Sent Done Done false
18362 6197 Acked Write 8 Sent Done Done false
18363 6198 Acked Write 1 Sent Done Done false
18364 6199 Acked Read 1 Sent Done Done false
18365 6200 Acked Read 5 Sent Done Done false
18366 6201 Acked Write 5 Sent Done Done false
18367 6202 Acked Write 2 Sent Done Done false
18368 6203 Acked Flush 0 Sent Done Done false
18369 6204 Acked Read 5 Sent Done Done false
18370 6205 Acked Write 2 Sent Done Done false
18371 6206 Acked Read 6 Sent Done Done false
18372 6207 Acked Write 4 Sent Done Done false
18373 6208 Acked Read 3 Sent Done Done false
18374 6209 Acked Write 6 Sent Done Done false
18375 6210 Acked Read 6 Sent Done Done false
18376 6211 Acked Read 6 Sent Done Done false
18377 6212 Acked Write 5 Sent Done Done false
18378 6213 Acked Write 3 Sent Done Done false
18379 6214 Acked Flush 0 Sent Done Done false
18380 6215 Acked Read 1 Sent Done Done false
18381 6216 Acked Read 6 Sent Done Done false
18382 6217 Acked Read 8 Sent Done Done false
18383 6218 Acked Write 8 Sent Done Done false
18384 6219 Acked Write 4 Sent Done Done false
18385 6220 Acked Read 3 Sent Done Done false
18386 6221 Acked Read 2 Sent Done Done false
18387 6222 Acked Read 4 Sent Done Done false
18388 6223 Acked Write 5 Sent Done Done false
18389 6224 Acked Read 4 Sent Done Done false
18390 6225 Acked Flush 0 Sent Done Done false
18391 6226 Acked Flush 0 Sent Done Done false
18392 6227 Acked Write 4 Sent Done Done false
18393 6228 Acked Write 7 Sent Done Done false
18394 6229 Acked Read 7 Sent Done Done false
18395 6230 Acked Read 3 Sent Done Done false
18396 6231 Acked Write 2 Sent Done Done false
18397 6232 Acked Write 4 Sent Done Done false
18398 6233 Acked Write 6 Sent Done Done false
18399 6234 Acked Write 7 Sent Done Done false
18400 6235 Acked Flush 0 Sent Done Done false
18401 6236 Acked Read 1 Sent Done Done false
18402 6237 Acked Write 6 Sent Done Done false
18403 6238 Acked Write 2 Sent Done Done false
18404 6239 Acked Read 7 Sent Done Done false
18405 6240 Acked Read 8 Sent Done Done false
18406 6241 Acked Write 5 Sent Done Done false
18407 6242 Acked Read 5 Sent Done Done false
18408 6243 Acked Read 8 Sent Done Done false
18409 6244 Acked Read 8 Sent Done Done false
18410 6245 Acked Read 9 Sent Done Done false
18411 6246 Acked Read 6 Sent Done Done false
18412 6247 Acked Write 7 Sent Done Done false
18413 6248 Acked Write 10 Sent Done Done false
18414 6249 Acked Write 8 Sent Done Done false
18415 6250 Acked Read 8 Sent Done Done false
18416 6251 Acked Read 5 Sent Done Done false
18417 6252 Acked Flush 0 Sent Done Done false
18418 6253 Acked Write 9 Sent Done Done false
18419 6254 Acked Read 8 Sent Done Done false
18420 6255 Acked Write 8 Sent Done Done false
18421 6256 Acked Read 8 Sent Done Done false
18422 6257 Acked Write 3 Sent Done Done false
18423 6258 Acked Write 4 Sent Done Done false
18424 6259 Acked Write 8 Sent Done Done false
18425 6260 Acked Read 7 Sent Done Done false
18426 6261 Acked Write 7 Sent Done Done false
18427 6262 Acked Write 1 Sent Done Done false
18428 6263 Acked Read 3 Sent Done Done false
18429 6264 Acked Write 1 Sent Done Done false
18430 6265 Acked Read 7 Sent Done Done false
18431 6266 Acked Read 2 Sent Done Done false
18432 6267 Acked Write 1 Sent Done Done false
18433 6268 Acked Read 5 Sent Done Done false
18434 6269 Acked Write 8 Sent Done Done false
18435 6270 Acked Read 1 Sent Done Done false
18436 6271 Acked Read 9 Sent Done Done false
18437 6272 Acked Write 10 Sent Done Done false
18438 6273 Acked Flush 0 Sent Done Done false
18439 6274 Acked Write 6 Sent Done Done false
18440 6275 Acked Read 2 Sent Done Done false
18441 6276 Acked Write 1 Sent Done Done false
18442 6277 Acked Flush 0 Sent Done Done false
18443 6278 Acked Read 1 Sent Done Done false
18444 6279 Acked Read 8 Sent Done Done false
18445 6280 Acked Read 7 Sent Done Done false
18446 6281 Acked Read 8 Sent Done Done false
18447 6282 Acked Write 5 Sent Done Done false
18448 6283 Acked Read 4 Sent Done Done false
18449 6284 Acked Write 2 Sent Done Done false
18450 6285 Acked Flush 0 Sent Done Done false
18451 6286 Acked Write 1 Sent Done Done false
18452 6287 Acked Read 7 Sent Done Done false
18453 6288 Acked Write 9 Sent Done Done false
18454 6289 Acked Read 10 Sent Done Done false
18455 6290 Acked Read 7 Sent Done Done false
18456 6291 Acked Read 7 Sent Done Done false
18457 6292 Acked Write 3 Sent Done Done false
18458 6293 Acked Write 7 Sent Done Done false
18459 6294 Acked Flush 0 Sent Done Done false
18460 6295 Acked Write 7 Sent Done Done false
18461 6296 Acked Write 4 Sent Done Done false
18462 6297 Acked Read 9 Sent Done Done false
18463 6298 Acked Write 9 Sent Done Done false
18464 6299 Acked Write 6 Sent Done Done false
18465 6300 Acked Read 8 Sent Done Done false
18466 6301 Acked Write 2 Sent Done Done false
18467 6302 Acked Read 1 Sent Done Done false
18468 6303 Acked Write 4 Sent Done Done false
18469 6304 Acked Read 3 Sent Done Done false
18470 6305 Acked Read 4 Sent Done Done false
18471 6306 Acked Write 6 Sent Done Done false
18472 6307 Acked Write 6 Sent Done Done false
18473 6308 Acked Write 4 Sent Done Done false
18474 6309 Acked Flush 0 Sent Done Done false
18475 6310 Acked Read 10 Sent Done Done false
18476 6311 Acked Flush 0 Sent Done Done false
18477 6312 Acked Write 4 Sent Done Done false
18478 6313 Acked Write 9 Sent Done Done false
18479 6314 Acked Write 8 Sent Done Done false
18480 6315 Acked Flush 0 Sent Done Done false
18481 6316 Acked Read 6 Sent Done Done false
18482 6317 Acked Write 9 Sent Done Done false
18483 6318 Acked Read 8 Sent Done Done false
18484 6319 Acked Write 2 Sent Done Done false
18485 6320 Acked Write 5 Sent Done Done false
18486 6321 Acked Flush 0 Sent Done Done false
18487 6322 Acked Write 6 Sent Done Done false
18488 6323 Acked Read 3 Sent Done Done false
18489 6324 Acked Write 5 Sent Done Done false
18490 6325 Acked Flush 0 Sent Done Done false
18491 6326 Acked Write 6 Sent Done Done false
18492 6327 Acked Read 5 Sent Done Done false
18493 6328 Acked Write 10 Sent Done Done false
18494 6329 Acked Read 4 Sent Done Done false
18495 6330 Acked Read 2 Sent Done Done false
18496 6331 Acked Read 4 Sent Done Done false
18497 6332 Acked Write 1 Sent Done Done false
18498 6333 Acked Read 5 Sent Done Done false
18499 6334 Acked Flush 0 Sent Done Done false
18500 6335 Acked Write 8 Sent Done Done false
18501 6336 Acked Read 7 Sent Done Done false
18502 6337 Acked Write 9 Sent Done Done false
18503 6338 Acked Read 9 Sent Done Done false
18504 6339 Acked Read 6 Sent Done Done false
18505 6340 Acked Read 4 Sent Done Done false
18506 6341 Acked Write 2 Sent Done Done false
18507 6342 Acked Read 1 Sent Done Done false
18508 6343 Acked Write 1 Sent Done Done false
18509 6344 Acked Read 8 Sent Done Done false
18510 6345 Acked Read 2 Sent Done Done false
18511 6346 Acked Read 5 Sent Done Done false
18512 6347 Acked Read 10 Sent Done Done false
18513 6348 Acked Read 10 Sent Done Done false
18514 6349 Acked Write 1 Sent Done Done false
18515 6350 Acked Read 2 Sent Done Done false
18516 6351 Acked Read 10 Sent Done Done false
18517 6352 Acked Read 7 Sent Done Done false
18518 6353 Acked Write 6 Sent Done Done false
18519 6354 Acked Read 8 Sent Done Done false
18520 6355 Acked Read 7 Sent Done Done false
18521 6356 Acked Write 3 Sent Done Done false
18522 6357 Acked Flush 0 Sent Done Done false
18523 6358 Acked Flush 0 Sent Done Done false
18524 6359 Acked Write 4 Sent Done Done false
18525 6360 Acked Flush 0 Sent Done Done false
18526 6361 Acked Write 6 Sent Done Done false
18527 6362 Acked Read 2 Sent Done Done false
18528 6363 Acked Read 10 Sent Done Done false
18529 6364 Acked Write 2 Sent Done Done false
18530 6365 Acked Read 5 Sent Done Done false
18531 6366 Acked Read 5 Sent Done Done false
18532 6367 Acked Write 9 Sent Done Done false
18533 6368 Acked Read 1 Sent Done Done false
18534 6369 Acked Read 9 Sent Done Done false
18535 6370 Acked Write 5 Sent Done Done false
18536 6371 Acked Read 2 Sent Done Done false
18537 6372 Acked Read 2 Sent Done Done false
18538 6373 Acked Write 10 Sent Done Done false
18539 6374 Acked Write 7 Sent Done Done false
18540 6375 Acked Flush 0 Sent Done Done false
18541 6376 Acked Write 3 Sent Done Done false
18542 6377 Acked Flush 0 Sent Done Done false
18543 6378 Acked Write 4 Sent Done Done false
18544 6379 Acked Read 5 Sent Done Done false
18545 6380 Acked Write 6 Sent Done Done false
18546 6381 Acked Write 4 Sent Done Done false
18547 6382 Acked Read 1 Sent Done Done false
18548 6383 Acked Read 8 Sent Done Done false
18549 6384 Acked Write 7 Sent Done Done false
18550 6385 Acked Read 10 Sent Done Done false
18551 6386 Acked Read 6 Sent Done Done false
18552 6387 Acked Read 10 Sent Done Done false
18553 6388 Acked Write 8 Sent Done Done false
18554 6389 Acked Write 6 Sent Done Done false
18555 6390 Acked Flush 0 Sent Done Done false
18556 6391 Acked Read 9 Sent Done Done false
18557 6392 Acked Read 10 Sent Done Done false
18558 6393 Acked Write 6 Sent Done Done false
18559 6394 Acked Read 6 Sent Done Done false
18560 6395 Acked Read 9 Sent Done Done false
18561 6396 Acked Flush 0 Sent Done Done false
18562 6397 Acked Write 3 Sent Done Done false
18563 6398 Acked Read 3 Sent Done Done false
18564 6399 Acked Read 2 Sent Done Done false
18565 6400 Acked Write 8 Sent Done Done false
18566 6401 Acked Write 5 Sent Done Done false
18567 6402 Acked Read 8 Sent Done Done false
18568 6403 Acked Write 3 Sent Done Done false
18569 6404 Acked Read 6 Sent Done Done false
18570 6405 Acked Flush 0 Sent Done Done false
18571 6406 Acked Read 9 Sent Done Done false
18572 6407 Acked Write 1 Sent Done Done false
18573 6408 Acked Write 10 Sent Done Done false
18574 6409 Acked Read 8 Sent Done Done false
18575 6410 Acked Read 3 Sent Done Done false
18576 6411 Acked Read 5 Sent Done Done false
18577 6412 Acked Write 7 Sent Done Done false
18578 6413 Acked Read 1 Sent Done Done false
18579 6414 Acked Write 10 Sent Done Done false
18580 6415 Acked Write 10 Sent Done Done false
18581 6416 Acked Read 8 Sent Done Done false
18582 6417 Acked Write 4 Sent Done Done false
18583 6418 Acked Read 7 Sent Done Done false
18584 6419 Acked Read 2 Sent Done Done false
18585 6420 Acked Write 7 Sent Done Done false
18586 6421 Acked Write 9 Sent Done Done false
18587 6422 Acked Read 8 Sent Done Done false
18588 6423 Acked Write 10 Sent Done Done false
18589 6424 Acked Read 6 Sent Done Done false
18590 6425 Acked Read 9 Sent Done Done false
18591 6426 Acked Read 8 Sent Done Done false
18592 6427 Acked Write 3 Sent Done Done false
18593 6428 Acked Read 9 Sent Done Done false
18594 6429 Acked Read 9 Sent Done Done false
18595 6430 Acked Read 1 Sent Done Done false
18596 6431 Acked Write 7 Sent Done Done false
18597 STATES DS:0 DS:1 DS:2 TOTAL
18598 Sent 301 0 0 301
18599 Done 0 301 301 602
18600 Skipped 0 0 0 0
18601 Error 0 0 0 0
18602 Last Flush: 6130 6405 6405
18603 Downstairs last five completed: 6130 6129 6128 6127 6126
18604 Upstairs last five completed: 6431 6430 6429 6428 6427
18605 CLIENT: Up:0 ds:301 act:2
18606 [8810] got start action
18607 [8810] I am going to restart
18608 Starting downstairs at port 8810
18609 Make output file at "/tmp/dsc/downstairs-8810.txt"
18610 [8810][0] reports Starting
18611 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1220)
18612 [8810][0] reports Running
18613 ----------------------------------------------------------------
18614 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18615 States: Offline Active Active
18616 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18617 6131 Acked Flush 0 Sent Done Done false
18618 6132 Acked Write 6 Sent Done Done false
18619 6133 Acked Read 10 Sent Done Done false
18620 6134 Acked Read 7 Sent Done Done false
18621 6135 Acked Write 5 Sent Done Done false
18622 6136 Acked Write 3 Sent Done Done false
18623 6137 Acked Write 10 Sent Done Done false
18624 6138 Acked Write 10 Sent Done Done false
18625 6139 Acked Read 3 Sent Done Done false
18626 6140 Acked Read 1 Sent Done Done false
18627 6141 Acked Read 10 Sent Done Done false
18628 6142 Acked Flush 0 Sent Done Done false
18629 6143 Acked Read 9 Sent Done Done false
18630 6144 Acked Flush 0 Sent Done Done false
18631 6145 Acked Read 9 Sent Done Done false
18632 6146 Acked Write 10 Sent Done Done false
18633 6147 Acked Read 8 Sent Done Done false
18634 6148 Acked Read 4 Sent Done Done false
18635 6149 Acked Read 5 Sent Done Done false
18636 6150 Acked Read 7 Sent Done Done false
18637 6151 Acked Read 6 Sent Done Done false
18638 6152 Acked Flush 0 Sent Done Done false
18639 6153 Acked Write 10 Sent Done Done false
18640 6154 Acked Write 8 Sent Done Done false
18641 6155 Acked Write 9 Sent Done Done false
18642 6156 Acked Write 5 Sent Done Done false
18643 6157 Acked Read 4 Sent Done Done false
18644 6158 Acked Read 2 Sent Done Done false
18645 6159 Acked Read 6 Sent Done Done false
18646 6160 Acked Read 1 Sent Done Done false
18647 6161 Acked Write 5 Sent Done Done false
18648 6162 Acked Write 10 Sent Done Done false
18649 6163 Acked Write 1 Sent Done Done false
18650 6164 Acked Read 3 Sent Done Done false
18651 6165 Acked Read 2 Sent Done Done false
18652 6166 Acked Read 8 Sent Done Done false
18653 6167 Acked Read 3 Sent Done Done false
18654 6168 Acked Write 9 Sent Done Done false
18655 6169 Acked Read 8 Sent Done Done false
18656 6170 Acked Read 2 Sent Done Done false
18657 6171 Acked Write 4 Sent Done Done false
18658 6172 Acked Read 9 Sent Done Done false
18659 6173 Acked Read 1 Sent Done Done false
18660 6174 Acked Write 9 Sent Done Done false
18661 6175 Acked Write 9 Sent Done Done false
18662 6176 Acked Read 2 Sent Done Done false
18663 6177 Acked Write 7 Sent Done Done false
18664 6178 Acked Flush 0 Sent Done Done false
18665 6179 Acked Read 1 Sent Done Done false
18666 6180 Acked Read 2 Sent Done Done false
18667 6181 Acked Read 6 Sent Done Done false
18668 6182 Acked Read 7 Sent Done Done false
18669 6183 Acked Read 5 Sent Done Done false
18670 6184 Acked Write 5 Sent Done Done false
18671 6185 Acked Write 5 Sent Done Done false
18672 6186 Acked Flush 0 Sent Done Done false
18673 6187 Acked Read 5 Sent Done Done false
18674 6188 Acked Read 5 Sent Done Done false
18675 6189 Acked Write 10 Sent Done Done false
18676 6190 Acked Flush 0 Sent Done Done false
18677 6191 Acked Write 6 Sent Done Done false
18678 6192 Acked Read 3 Sent Done Done false
18679 6193 Acked Flush 0 Sent Done Done false
18680 6194 Acked Read 4 Sent Done Done false
18681 6195 Acked Write 5 Sent Done Done false
18682 6196 Acked Flush 0 Sent Done Done false
18683 6197 Acked Write 8 Sent Done Done false
18684 6198 Acked Write 1 Sent Done Done false
18685 6199 Acked Read 1 Sent Done Done false
18686 6200 Acked Read 5 Sent Done Done false
18687 6201 Acked Write 5 Sent Done Done false
18688 6202 Acked Write 2 Sent Done Done false
18689 6203 Acked Flush 0 Sent Done Done false
18690 6204 Acked Read 5 Sent Done Done false
18691 6205 Acked Write 2 Sent Done Done false
18692 6206 Acked Read 6 Sent Done Done false
18693 6207 Acked Write 4 Sent Done Done false
18694 6208 Acked Read 3 Sent Done Done false
18695 6209 Acked Write 6 Sent Done Done false
18696 6210 Acked Read 6 Sent Done Done false
18697 6211 Acked Read 6 Sent Done Done false
18698 6212 Acked Write 5 Sent Done Done false
18699 6213 Acked Write 3 Sent Done Done false
18700 6214 Acked Flush 0 Sent Done Done false
18701 6215 Acked Read 1 Sent Done Done false
18702 6216 Acked Read 6 Sent Done Done false
18703 6217 Acked Read 8 Sent Done Done false
18704 6218 Acked Write 8 Sent Done Done false
18705 6219 Acked Write 4 Sent Done Done false
18706 6220 Acked Read 3 Sent Done Done false
18707 6221 Acked Read 2 Sent Done Done false
18708 6222 Acked Read 4 Sent Done Done false
18709 6223 Acked Write 5 Sent Done Done false
18710 6224 Acked Read 4 Sent Done Done false
18711 6225 Acked Flush 0 Sent Done Done false
18712 6226 Acked Flush 0 Sent Done Done false
18713 6227 Acked Write 4 Sent Done Done false
18714 6228 Acked Write 7 Sent Done Done false
18715 6229 Acked Read 7 Sent Done Done false
18716 6230 Acked Read 3 Sent Done Done false
18717 6231 Acked Write 2 Sent Done Done false
18718 6232 Acked Write 4 Sent Done Done false
18719 6233 Acked Write 6 Sent Done Done false
18720 6234 Acked Write 7 Sent Done Done false
18721 6235 Acked Flush 0 Sent Done Done false
18722 6236 Acked Read 1 Sent Done Done false
18723 6237 Acked Write 6 Sent Done Done false
18724 6238 Acked Write 2 Sent Done Done false
18725 6239 Acked Read 7 Sent Done Done false
18726 6240 Acked Read 8 Sent Done Done false
18727 6241 Acked Write 5 Sent Done Done false
18728 6242 Acked Read 5 Sent Done Done false
18729 6243 Acked Read 8 Sent Done Done false
18730 6244 Acked Read 8 Sent Done Done false
18731 6245 Acked Read 9 Sent Done Done false
18732 6246 Acked Read 6 Sent Done Done false
18733 6247 Acked Write 7 Sent Done Done false
18734 6248 Acked Write 10 Sent Done Done false
18735 6249 Acked Write 8 Sent Done Done false
18736 6250 Acked Read 8 Sent Done Done false
18737 6251 Acked Read 5 Sent Done Done false
18738 6252 Acked Flush 0 Sent Done Done false
18739 6253 Acked Write 9 Sent Done Done false
18740 6254 Acked Read 8 Sent Done Done false
18741 6255 Acked Write 8 Sent Done Done false
18742 6256 Acked Read 8 Sent Done Done false
18743 6257 Acked Write 3 Sent Done Done false
18744 6258 Acked Write 4 Sent Done Done false
18745 6259 Acked Write 8 Sent Done Done false
18746 6260 Acked Read 7 Sent Done Done false
18747 6261 Acked Write 7 Sent Done Done false
18748 6262 Acked Write 1 Sent Done Done false
18749 6263 Acked Read 3 Sent Done Done false
18750 6264 Acked Write 1 Sent Done Done false
18751 6265 Acked Read 7 Sent Done Done false
18752 6266 Acked Read 2 Sent Done Done false
18753 6267 Acked Write 1 Sent Done Done false
18754 6268 Acked Read 5 Sent Done Done false
18755 6269 Acked Write 8 Sent Done Done false
18756 6270 Acked Read 1 Sent Done Done false
18757 6271 Acked Read 9 Sent Done Done false
18758 6272 Acked Write 10 Sent Done Done false
18759 6273 Acked Flush 0 Sent Done Done false
18760 6274 Acked Write 6 Sent Done Done false
18761 6275 Acked Read 2 Sent Done Done false
18762 6276 Acked Write 1 Sent Done Done false
18763 6277 Acked Flush 0 Sent Done Done false
18764 6278 Acked Read 1 Sent Done Done false
18765 6279 Acked Read 8 Sent Done Done false
18766 6280 Acked Read 7 Sent Done Done false
18767 6281 Acked Read 8 Sent Done Done false
18768 6282 Acked Write 5 Sent Done Done false
18769 6283 Acked Read 4 Sent Done Done false
18770 6284 Acked Write 2 Sent Done Done false
18771 6285 Acked Flush 0 Sent Done Done false
18772 6286 Acked Write 1 Sent Done Done false
18773 6287 Acked Read 7 Sent Done Done false
18774 6288 Acked Write 9 Sent Done Done false
18775 6289 Acked Read 10 Sent Done Done false
18776 6290 Acked Read 7 Sent Done Done false
18777 6291 Acked Read 7 Sent Done Done false
18778 6292 Acked Write 3 Sent Done Done false
18779 6293 Acked Write 7 Sent Done Done false
18780 6294 Acked Flush 0 Sent Done Done false
18781 6295 Acked Write 7 Sent Done Done false
18782 6296 Acked Write 4 Sent Done Done false
18783 6297 Acked Read 9 Sent Done Done false
18784 6298 Acked Write 9 Sent Done Done false
18785 6299 Acked Write 6 Sent Done Done false
18786 6300 Acked Read 8 Sent Done Done false
18787 6301 Acked Write 2 Sent Done Done false
18788 6302 Acked Read 1 Sent Done Done false
18789 6303 Acked Write 4 Sent Done Done false
18790 6304 Acked Read 3 Sent Done Done false
18791 6305 Acked Read 4 Sent Done Done false
18792 6306 Acked Write 6 Sent Done Done false
18793 6307 Acked Write 6 Sent Done Done false
18794 6308 Acked Write 4 Sent Done Done false
18795 6309 Acked Flush 0 Sent Done Done false
18796 6310 Acked Read 10 Sent Done Done false
18797 6311 Acked Flush 0 Sent Done Done false
18798 6312 Acked Write 4 Sent Done Done false
18799 6313 Acked Write 9 Sent Done Done false
18800 6314 Acked Write 8 Sent Done Done false
18801 6315 Acked Flush 0 Sent Done Done false
18802 6316 Acked Read 6 Sent Done Done false
18803 6317 Acked Write 9 Sent Done Done false
18804 6318 Acked Read 8 Sent Done Done false
18805 6319 Acked Write 2 Sent Done Done false
18806 6320 Acked Write 5 Sent Done Done false
18807 6321 Acked Flush 0 Sent Done Done false
18808 6322 Acked Write 6 Sent Done Done false
18809 6323 Acked Read 3 Sent Done Done false
18810 6324 Acked Write 5 Sent Done Done false
18811 6325 Acked Flush 0 Sent Done Done false
18812 6326 Acked Write 6 Sent Done Done false
18813 6327 Acked Read 5 Sent Done Done false
18814 6328 Acked Write 10 Sent Done Done false
18815 6329 Acked Read 4 Sent Done Done false
18816 6330 Acked Read 2 Sent Done Done false
18817 6331 Acked Read 4 Sent Done Done false
18818 6332 Acked Write 1 Sent Done Done false
18819 6333 Acked Read 5 Sent Done Done false
18820 6334 Acked Flush 0 Sent Done Done false
18821 6335 Acked Write 8 Sent Done Done false
18822 6336 Acked Read 7 Sent Done Done false
18823 6337 Acked Write 9 Sent Done Done false
18824 6338 Acked Read 9 Sent Done Done false
18825 6339 Acked Read 6 Sent Done Done false
18826 6340 Acked Read 4 Sent Done Done false
18827 6341 Acked Write 2 Sent Done Done false
18828 6342 Acked Read 1 Sent Done Done false
18829 6343 Acked Write 1 Sent Done Done false
18830 6344 Acked Read 8 Sent Done Done false
18831 6345 Acked Read 2 Sent Done Done false
18832 6346 Acked Read 5 Sent Done Done false
18833 6347 Acked Read 10 Sent Done Done false
18834 6348 Acked Read 10 Sent Done Done false
18835 6349 Acked Write 1 Sent Done Done false
18836 6350 Acked Read 2 Sent Done Done false
18837 6351 Acked Read 10 Sent Done Done false
18838 6352 Acked Read 7 Sent Done Done false
18839 6353 Acked Write 6 Sent Done Done false
18840 6354 Acked Read 8 Sent Done Done false
18841 6355 Acked Read 7 Sent Done Done false
18842 6356 Acked Write 3 Sent Done Done false
18843 6357 Acked Flush 0 Sent Done Done false
18844 6358 Acked Flush 0 Sent Done Done false
18845 6359 Acked Write 4 Sent Done Done false
18846 6360 Acked Flush 0 Sent Done Done false
18847 6361 Acked Write 6 Sent Done Done false
18848 6362 Acked Read 2 Sent Done Done false
18849 6363 Acked Read 10 Sent Done Done false
18850 6364 Acked Write 2 Sent Done Done false
18851 6365 Acked Read 5 Sent Done Done false
18852 6366 Acked Read 5 Sent Done Done false
18853 6367 Acked Write 9 Sent Done Done false
18854 6368 Acked Read 1 Sent Done Done false
18855 6369 Acked Read 9 Sent Done Done false
18856 6370 Acked Write 5 Sent Done Done false
18857 6371 Acked Read 2 Sent Done Done false
18858 6372 Acked Read 2 Sent Done Done false
18859 6373 Acked Write 10 Sent Done Done false
18860 6374 Acked Write 7 Sent Done Done false
18861 6375 Acked Flush 0 Sent Done Done false
18862 6376 Acked Write 3 Sent Done Done false
18863 6377 Acked Flush 0 Sent Done Done false
18864 6378 Acked Write 4 Sent Done Done false
18865 6379 Acked Read 5 Sent Done Done false
18866 6380 Acked Write 6 Sent Done Done false
18867 6381 Acked Write 4 Sent Done Done false
18868 6382 Acked Read 1 Sent Done Done false
18869 6383 Acked Read 8 Sent Done Done false
18870 6384 Acked Write 7 Sent Done Done false
18871 6385 Acked Read 10 Sent Done Done false
18872 6386 Acked Read 6 Sent Done Done false
18873 6387 Acked Read 10 Sent Done Done false
18874 6388 Acked Write 8 Sent Done Done false
18875 6389 Acked Write 6 Sent Done Done false
18876 6390 Acked Flush 0 Sent Done Done false
18877 6391 Acked Read 9 Sent Done Done false
18878 6392 Acked Read 10 Sent Done Done false
18879 6393 Acked Write 6 Sent Done Done false
18880 6394 Acked Read 6 Sent Done Done false
18881 6395 Acked Read 9 Sent Done Done false
18882 6396 Acked Flush 0 Sent Done Done false
18883 6397 Acked Write 3 Sent Done Done false
18884 6398 Acked Read 3 Sent Done Done false
18885 6399 Acked Read 2 Sent Done Done false
18886 6400 Acked Write 8 Sent Done Done false
18887 6401 Acked Write 5 Sent Done Done false
18888 6402 Acked Read 8 Sent Done Done false
18889 6403 Acked Write 3 Sent Done Done false
18890 6404 Acked Read 6 Sent Done Done false
18891 6405 Acked Flush 0 Sent Done Done false
18892 6406 Acked Read 9 Sent Done Done false
18893 6407 Acked Write 1 Sent Done Done false
18894 6408 Acked Write 10 Sent Done Done false
18895 6409 Acked Read 8 Sent Done Done false
18896 6410 Acked Read 3 Sent Done Done false
18897 6411 Acked Read 5 Sent Done Done false
18898 6412 Acked Write 7 Sent Done Done false
18899 6413 Acked Read 1 Sent Done Done false
18900 6414 Acked Write 10 Sent Done Done false
18901 6415 Acked Write 10 Sent Done Done false
18902 6416 Acked Read 8 Sent Done Done false
18903 6417 Acked Write 4 Sent Done Done false
18904 6418 Acked Read 7 Sent Done Done false
18905 6419 Acked Read 2 Sent Done Done false
18906 6420 Acked Write 7 Sent Done Done false
18907 6421 Acked Write 9 Sent Done Done false
18908 6422 Acked Read 8 Sent Done Done false
18909 6423 Acked Write 10 Sent Done Done false
18910 6424 Acked Read 6 Sent Done Done false
18911 6425 Acked Read 9 Sent Done Done false
18912 6426 Acked Read 8 Sent Done Done false
18913 6427 Acked Write 3 Sent Done Done false
18914 6428 Acked Read 9 Sent Done Done false
18915 6429 Acked Read 9 Sent Done Done false
18916 6430 Acked Read 1 Sent Done Done false
18917 6431 Acked Write 7 Sent Done Done false
18918 6432 Acked Flush 0 Sent Done Done false
18919 STATES DS:0 DS:1 DS:2 TOTAL
18920 Sent 302 0 0 302
18921 Done 0 302 302 604
18922 Skipped 0 0 0 0
18923 Error 0 0 0 0
18924 Last Flush: 6130 6432 6432
18925 Downstairs last five completed: 6130 6129 6128 6127 6126
18926 Upstairs last five completed: 6432 6431 6430 6429 6428
18927 CLIENT: Up:0 ds:302 act:2
189282025-01-14T16:14:34.067ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
189292025-01-14T16:14:34.067ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64604) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
189302025-01-14T16:14:34.068ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
189312025-01-14T16:14:34.069ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
189322025-01-14T16:14:34.069ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
189332025-01-14T16:14:34.069ZINFOcrucible: send last flush ID to this DS: 6130 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
189342025-01-14T16:14:34.069ZINFOcrucible: Replied this last flush ID: 6130 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
189352025-01-14T16:14:34.069ZINFOcrucible: [0] client re-new 302 jobs since flush 6130 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
189362025-01-14T16:14:34.069ZINFOcrucible: [0] Replayed 302 jobs since flush: 6130 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
18937 ----------------------------------------------------------------
18938 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
18939 Downstairs last five completed: 6432 6431 6430 6429 6428
18940 Upstairs last five completed: 6432 6431 6430 6429 6428
18941 CLIENT: Up:0 ds:0 act:3
18942 Replay: All jobs finished
18943 Main task has work to do, go find it
18944 got dsc Stop(1)
18945 stop 1
18946 Jan 14 16:14:36.313 INFO request completed, latency_us: 305, response_code: 204, uri:[8820] Got stop action so:false kr:false
18947 /stop/cid/1, method: POST, req_id: e1a995a3-eea8-490c-9b88-f21d4e5389ed, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
18948 Jan 14 16:14:36.314 INFO request completed, latency_us: 173, response_code: 200, uri: /state/cid/1, method: GET, req_id: 4e4c9543-f880-43bc-a672-3a96ddee3a8d, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
189492025-01-14T16:14:36.314ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
189502025-01-14T16:14:36.314ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
189512025-01-14T16:14:36.314ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
189522025-01-14T16:14:36.314ZINFOcrucible: client task is exiting = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
18953 [8820] Exited with: ExitStatus(unix_wait_status(9))
18954 [8820][1] reports Exit
18955 Jan 14 16:14:40.316 INFO request completed, latency_us: 212, response_code: 200, uri: /state/cid/1, method: GET, req_id: 3a00d46c-6b44-4846-a481-11f4c0655f2a, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
18956 001/300 Read block 1783 len 40960
18957 002/300 Read block 4178 len 24576
18958 003/300 Flush
18959 004/300 Read block 2518 len 24576
18960 005/300 Read block 2910 len 36864
18961 006/300 Flush
18962 007/300 Read block 509 len 40960
18963 008/300 Read block 1194 len 36864
18964 009/300 Write block 2780 len 40960 data: 5 4 4 3 3 3 3 3 3 3
18965 010/300 Read block 2180 len 12288
18966 011/300 Write block 2188 len 12288 data: 5 4 5
18967 012/300 Read block 741 len 16384
18968 013/300 Read block 1427 len 28672
18969 014/300 Write block 60 len 4096 data: 5
18970 015/300 Flush
18971 016/300 Write block 3607 len 28672 data: 5 6 6 6 6 5 6
18972 017/300 Write block 2376 len 24576 data: 5 5 5 4 4 4
18973 018/300 Read block 2053 len 36864
18974 019/300 Write block 3256 len 20480 data: 6 5 4 4 3
18975 020/300 Read block 434 len 32768
18976 021/300 Write block 3617 len 4096 data: 5
18977 022/300 Read block 3538 len 28672
18978 023/300 Write block 3389 len 8192 data: 6 6
18979 024/300 Flush
18980 025/300 Write block 1070 len 28672 data: 4 5 4 4 7 7 6
18981 026/300 Write block 4236 len 12288 data: 3 4 3
18982 027/300 Write block 4081 len 36864 data: 4 4 4 5 6 7 6 7 8
18983 028/300 Write block 4061 len 16384 data: 3 3 4 4
18984 029/300 Write block 3910 len 16384 data: 5 5 5 6
18985 030/300 Write block 3610 len 36864 data: 7 7 6 7 6 5 5 6 5
18986 031/300 Read block 4356 len 8192
18987 032/300 Read block 4749 len 16384
18988 033/300 Write block 2723 len 12288 data: 4 4 5
18989 034/300 Flush
18990 035/300 Read block 1726 len 40960
18991 036/300 Read block 4775 len 4096
18992 037/300 Flush
18993 038/300 Write block 1348 len 32768 data: 2 2 2 3 4 5 4 5
18994 039/300 Read block 2758 len 20480
18995 040/300 Read block 329 len 28672
18996 041/300 Read block 459 len 4096
18997 042/300 Read block 3285 len 24576
18998 043/300 Flush
18999 044/300 Flush
19000 045/300 Read block 2307 len 36864
19001 046/300 Read block 2084 len 12288
19002 047/300 Read block 4202 len 20480
19003 048/300 Read block 339 len 40960
19004 049/300 Read block 333 len 40960
19005 050/300 Read block 2764 len 28672
19006 051/300 Read block 4168 len 32768
19007 052/300 Write block 4057 len 12288 data: 5 5 6
19008 053/300 Write block 1911 len 8192 data: 3 3
19009 054/300 Read block 1448 len 8192
19010 055/300 Write block 2336 len 40960 data: 5 5 4 4 5 5 4 5 5 3
19011 056/300 Read block 4556 len 16384
19012 057/300 Write block 4619 len 32768 data: 2 2 3 3 5 5 5 4
19013 058/300 Read block 1367 len 32768
19014 059/300 Read block 4122 len 16384
19015 060/300 Write block 3621 len 40960 data: 6 5 6 5 5 4 4 4 4 3
19016 061/300 Read block 3129 len 36864
19017 062/300 Read block 4123 len 4096
19018 063/300 Read block 3328 len 24576
19019 064/300 Flush
19020 065/300 Write block 574 len 8192 data: 4 4
19021 066/300 Write block 3220 len 32768 data: 4 4 3 3 3 3 3 6
19022 067/300 Flush
19023 068/300 Read block 2039 len 32768
19024 069/300 Write block 3328 len 32768 data: 3 3 3 3 3 4 4 4
19025 070/300 Write block 819 len 40960 data: 5 5 4 5 3 4 2 4 4 3
19026 071/300 Write block 2015 len 28672 data: 3 3 3 3 4 4 5
19027 072/300 Write block 2434 len 20480 data: 4 2 3 3 3
19028 073/300 Flush
19029 074/300 Write block 242 len 16384 data: 5 5 5 4
19030 075/300 Read block 2258 len 20480
19031 076/300 Read block 1496 len 24576
19032 077/300 Read block 1751 len 36864
19033 078/300 Read block 1659 len 24576
19034 079/300 Read block 4198 len 12288
19035 080/300 Write block 22 len 4096 data: 6
19036 081/300 Flush
19037 082/300 Write block 3844 len 32768 data: 5 5 6 6 7 6 5 4
19038 083/300 Read block 1350 len 20480
19039 084/300 Write block 1415 len 12288 data: 3 3 4
19040 085/300 Write block 920 len 36864 data: 4 4 4 4 5 6 7 8 8
19041 086/300 Write block 1558 len 40960 data: 7 4 5 4 3 4 4 5 6 7
19042 087/300 Flush
19043 088/300 Write block 3671 len 12288 data: 3 3 3
19044 089/300 Write block 3815 len 16384 data: 4 4 5 5
19045 090/300 Read block 4806 len 12288
19046 091/300 Read block 2135 len 40960
19047 092/300 Write block 3969 len 28672 data: 4 3 5 5 7 7 7
19048 093/300 Read block 411 len 16384
19049 094/300 Write block 589 len 20480 data: 6 7 6 6 5
19050 095/300 Write block 948 len 20480 data: 4 4 4 4 3
19051 096/300 Read block 2790 len 40960
19052 097/300 Write block 4959 len 24576 data: 6 5 3 3 5 3
19053 098/300 Read block 4738 len 16384
19054 099/300 Write block 4028 len 12288 data: 5 4 4
19055 100/300 Flush
19056 101/300 Write block 2470 len 4096 data: 4
19057 102/300 Write block 4346 len 36864 data: 3 3 4 5 6 6 5 6 5
19058 103/300 Read block 2102 len 24576
19059 104/300 Write block 4740 len 12288 data: 4 5 5
19060 105/300 Write block 3366 len 36864 data: 5 5 6 6 5 4 4 4 4
19061 106/300 Write block 3310 len 16384 data: 4 4 5 5
19062 107/300 Flush
19063 108/300 Write block 4557 len 20480 data: 4 4 4 5 5
19064 109/300 Flush
19065 110/300 Write block 2598 len 28672 data: 3 3 4 4 4 4 4
19066 111/300 Read block 4060 len 40960
19067 112/300 Write block 1162 len 8192 data: 3 3
19068 113/300 Read block 1317 len 28672
19069 114/300 Flush
19070 115/300 Read block 3247 len 36864
19071 116/300 Read block 4720 len 32768
19072 117/300 Write block 3593 len 40960 data: 2 2 2 2 2 2 2 2 2 4
19073 118/300 Read block 1386 len 16384
19074 119/300 Write block 3622 len 4096 data: 6
19075 120/300 Read block 2788 len 28672
19076 121/300 Read block 2587 len 4096
19077 122/300 Write block 18 len 24576 data: 4 4 4 5 7 6
19078 123/300 Read block 1441 len 12288
19079 124/300 Write block 1952 len 4096 data: 5
19080 125/300 Write block 3514 len 20480 data: 4 5 6 7 6
19081 126/300 Read block 4481 len 36864
19082 127/300 Write block 4529 len 24576 data: 2 2 2 2 3 3
19083 128/300 Read block 2534 len 12288
19084 129/300 Write block 3033 len 32768 data: 3 4 4 3 3 3 3 2
19085 130/300 Read block 2163 len 8192
19086 131/300 Read block 3752 len 4096
19087 132/300 Write block 4237 len 32768 data: 5 4 4 4 4 5 4 4
19088 133/300 Read block 2903 len 4096
19089 134/300 Read block 2034 len 16384
19090 135/300 Read block 4637 len 8192
19091 136/300 Read block 1850 len 20480
19092 137/300 Read block 3408 len 24576
19093 138/300 Flush
19094 139/300 Flush
19095 140/300 Write block 1739 len 24576 data: 3 3 6 5 5 4
19096 141/300 Read block 1643 len 12288
19097 142/300 Write block 1038 len 16384 data: 5 5 4 3
19098 143/300 Write block 731 len 12288 data: 5 4 3
19099 144/300 Write block 3722 len 36864 data: 4 3 3 3 4 5 4 4 3
19100 145/300 Read block 2641 len 12288
19101 146/300 Read block 1455 len 4096
19102 147/300 Write block 2277 len 24576 data: 4 3 5 6 6 6
19103 148/300 Flush
19104 149/300 Read block 267 len 40960
19105 150/300 Write block 2726 len 32768 data: 6 6 6 5 5 5 4 4
19106 151/300 Write block 1093 len 36864 data: 3 3 3 4 4 4 5 3 3
19107 152/300 Read block 3595 len 36864
19108 153/300 Read block 4682 len 28672
19109 154/300 Flush
19110 155/300 Read block 3643 len 20480
19111 156/300 Read block 4082 len 40960
19112 157/300 Write block 3848 len 36864 data: 8 7 6 5 4 3 2 2 2
19113 158/300 Flush
19114 159/300 Flush
19115 160/300 Read block 236 len 12288
19116 161/300 Read block 366 len 28672
19117 162/300 Write block 2175 len 36864 data: 5 4 3 2 2 2 2 2 2
19118 163/300 Read block 2300 len 28672
19119 164/300 Write block 217 len 28672 data: 3 4 5 5 5 5 5
19120 165/300 Read block 1636 len 16384
19121 166/300 Write block 3017 len 36864 data: 5 4 4 5 5 4 4 8 8
19122 167/300 Read block 1523 len 28672
19123 168/300 Write block 899 len 24576 data: 6 6 5 5 5 6
19124 169/300 Write block 1261 len 8192 data: 5 3
19125 170/300 Write block 565 len 8192 data: 5 3
19126 171/300 Write block 256 len 32768 data: 6 5 4 4 5 4 4 5
19127 172/300 Read block 1891 len 24576
19128 173/300 Write block 3483 len 8192 data: 3 3
19129 174/300 Write block 4172 len 32768 data: 5 7 6 6 7 7 7 6
19130 175/300 Write block 3629 len 12288 data: 5 4 5
19131 176/300 Flush
19132 177/300 Read block 4031 len 24576
19133 178/300 Read block 4452 len 28672
19134 179/300 Write block 2308 len 28672 data: 4 4 5 5 8 9 9
19135 180/300 Write block 90 len 4096 data: 2
19136 181/300 Write block 3502 len 28672 data: 3 2 4 4 3 2 2
19137 182/300 Write block 390 len 12288 data: 3 3 2
19138 183/300 Read block 1696 len 28672
19139 184/300 Read block 1785 len 20480
19140 185/300 Write block 4520 len 28672 data: 7 6 5 5 5 4 3
19141 186/300 Flush
19142 187/300 Read block 2222 len 28672
19143 188/300 Read block 795 len 4096
19144 189/300 Flush
19145 190/300 Read block 333 len 16384
19146 191/300 Write block 2609 len 24576 data: 4 3 5 6 8 7
19147 192/300 Write block 2849 len 8192 data: 3 4
19148 193/300 Read block 2121 len 8192
19149 194/300 Read block 3241 len 28672
19150 195/300 Flush
19151 196/300 Read block 434 len 28672
19152 197/300 Write block 3078 len 20480 data: 4 4 4 4 3
19153 198/300 Flush
19154 199/300 Write block 2748 len 24576 data: 4 4 3 3 3 4
19155 200/300 Read block 1070 len 24576
19156 201/300 Write block 314 len 20480 data: 6 6 6 7 6
19157 202/300 Write block 3197 len 12288 data: 7 7 6
19158 203/300 Write block 1675 len 8192 data: 6 6
19159 204/300 Read block 1104 len 40960
19160 205/300 Write block 2377 len 28672 data: 6 6 5 5 5 3 3
19161 206/300 Write block 1706 len 20480 data: 4 5 3 3 3
19162 207/300 Flush
19163 208/300 Read block 1700 len 4096
19164 209/300 Write block 269 len 40960 data: 3 3 4 4 4 4 5 5 6 5
19165 210/300 Write block 4746 len 32768 data: 4 3 3 4 5 5 5 5
19166 211/300 Read block 1878 len 16384
19167 212/300 Read block 1379 len 12288
19168 213/300 Write block 3915 len 12288 data: 6 6 4
19169 214/300 Write block 2735 len 20480 data: 3 3 3 3 3
19170 215/300 Read block 1327 len 16384
19171 216/300 Read block 3371 len 40960
19172 217/300 Write block 3399 len 28672 data: 5 5 6 4 4 3 3
19173 218/300 Write block 1031 len 16384 data: 4 5 6 5
19174 219/300 Read block 2073 len 40960
19175 220/300 Write block 2876 len 40960 data: 7 7 8 7 4 4 3 3 4 4
19176 221/300 Write block 430 len 4096 data: 3
19177 222/300 Read block 1574 len 28672
19178 223/300 Write block 582 len 16384 data: 4 4 5 6
19179 224/300 Write block 1770 len 36864 data: 4 4 5 4 4 3 3 3 3
19180 225/300 Write block 3804 len 40960 data: 4 3 6 6 6 6 6 6 6 3
19181 226/300 Write block 1939 len 28672 data: 4 5 5 5 5 4 4
19182 227/300 Read block 351 len 20480
19183 228/300 Flush
19184 229/300 Write block 2212 len 20480 data: 6 5 5 4 5
19185 230/300 Write block 1155 len 12288 data: 6 6 5
19186 231/300 Read block 1267 len 24576
19187 232/300 Read block 2298 len 4096
19188 233/300 Read block 466 len 8192
19189 234/300 Write block 4738 len 28672 data: 4 3 5 6 6 5 5
19190 235/300 Read block 4715 len 36864
19191 236/300 Read block 3801 len 24576
19192 237/300 Write block 2708 len 16384 data: 3 3 3 4
19193 238/300 Read block 1692 len 16384
19194 239/300 Write block 1398 len 28672 data: 5 5 4 5 4 5 5
19195 240/300 Read block 1615 len 16384
19196 241/300 Read block 681 len 32768
19197 242/300 Read block 4231 len 40960
19198 243/300 Write block 1233 len 28672 data: 4 4 4 5 5 5 5
19199 244/300 Read block 3768 len 4096
19200 245/300 Write block 1238 len 16384 data: 6 6 5 3
19201 246/300 Write block 804 len 24576 data: 4 6 6 7 7 7
19202 247/300 Write block 2887 len 12288 data: 5 4 4
19203 248/300 Read block 4582 len 8192
19204 249/300 Read block 3822 len 16384
19205 250/300 Write block 4049 len 4096 data: 6
19206 251/300 Read block 3383 len 16384
19207 252/300 Read block 2694 len 32768
19208 253/300 Write block 4450 len 16384 data: 3 3 3 2
19209 254/300 Read block 2416 len 12288
19210 255/300 Write block 1762 len 4096 data: 3
19211 256/300 Read block 2423 len 20480
19212 257/300 Read block 3398 len 16384
19213 258/300 Read block 3302 len 12288
19214 259/300 Read block 3650 len 40960
19215 260/300 Flush
19216 261/300 Read block 3506 len 32768
19217 262/300 Write block 2728 len 4096 data: 7
19218 263/300 Read block 1258 len 16384
19219 264/300 Read block 1811 len 16384
19220 265/300 Flush
19221 266/300 Read block 1395 len 32768
19222 267/300 Write block 1012 len 20480 data: 4 5 5 5 5
19223 268/300 Write block 4338 len 40960 data: 2 2 2 2 3 3 3 3 4 4
19224 269/300 Read block 3752 len 8192
19225 270/300 Write block 4943 len 40960 data: 5 6 5 4 4 4 5 5 3 4
19226 271/300 Flush
19227 272/300 Flush
19228 273/300 Read block 1826 len 4096
19229 274/300 Read block 2627 len 4096
19230 275/300 Read block 3464 len 32768
19231 276/300 Read block 3707 len 32768
19232 277/300 Read block 4771 len 16384
19233 278/300 Read block 109 len 12288
19234 279/300 Read block 1496 len 4096
19235 280/300 Read block 2693 len 16384
19236 281/300 Read block 381 len 24576
19237 282/300 Read block 771 len 12288
19238 283/300 Write block 559 len 24576 data: 7 9 7 6 6 7
19239 284/300 Read block 1358 len 36864
19240 285/300 Read block 3199 len 36864
19241 286/300 Flush
19242 287/300 Read block 1890 len 36864
19243 288/300 Write block 3546 len 12288 data: 3 3 3
19244 289/300 Write block 1109 len 28672 data: 2 2 2 2 2 2 2
19245 290/300 Write block 823 len 8192 data: 4 5
19246 291/300 Flush
19247 292/300 Write block 2057 len 32768 data: 5 4 4 4 4 2 2 3
19248 293/300 Read block 748 len 28672
19249 294/300 Write block 2272 len 32768 data: 4 4 4 5 5 5 4 6
19250 295/300 Read block 4176 len 36864
19251 296/300 Read block 1312 len 32768
19252 297/300 Read block 4675 len 36864
19253 298/300 Read block 2555 len 8192
19254 299/300 Read block 3993 len 20480
19255 300/300 Read block 2537 len 28672
19256 301/300 Write block 2535 len 28672 data: 3 4 5 4 4 4 4
19257 Main task has work to do, go find it
19258 got dsc Start(1)
19259 start 1
19260 Jan 14 16:14:40.552 INFO request completed, latency_us: 262, response_code: 204, uri: /start/cid/1, method: POST, req_id: ee222741-ce82-4176-9c9c-d1aede4f377c, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
19261 Replay: started 1, returned:Ok(())
19262 ----------------------------------------------------------------
19263 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
19264 States: Active Offline Active
19265 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19266 6433 Acked Read 10 Done Sent Done false
19267 6434 Acked Read 6 Done Sent Done false
19268 6435 Acked Flush 0 Done Sent Done false
19269 6436 Acked Read 6 Done Sent Done false
19270 6437 Acked Read 9 Done Sent Done false
19271 6438 Acked Flush 0 Done Sent Done false
19272 6439 Acked Read 10 Done Sent Done false
19273 6440 Acked Read 9 Done Sent Done false
19274 6441 Acked Write 10 Done Sent Done false
19275 6442 Acked Read 3 Done Sent Done false
19276 6443 Acked Write 3 Done Sent Done false
19277 6444 Acked Read 4 Done Sent Done false
19278 6445 Acked Read 7 Done Sent Done false
19279 6446 Acked Write 1 Done Sent Done false
19280 6447 Acked Flush 0 Done Sent Done false
19281 6448 Acked Write 7 Done Sent Done false
19282 6449 Acked Write 6 Done Sent Done false
19283 6450 Acked Read 9 Done Sent Done false
19284 6451 Acked Write 5 Done Sent Done false
19285 6452 Acked Read 8 Done Sent Done false
19286 6453 Acked Write 1 Done Sent Done false
19287 6454 Acked Read 7 Done Sent Done false
19288 6455 Acked Write 2 Done Sent Done false
19289 6456 Acked Flush 0 Done Sent Done false
19290 6457 Acked Write 7 Done Sent Done false
19291 6458 Acked Write 3 Done Sent Done false
19292 6459 Acked Write 9 Done Sent Done false
19293 6460 Acked Write 4 Done Sent Done false
19294 6461 Acked Write 4 Done Sent Done false
19295 6462 Acked Write 9 Done Sent Done false
19296 6463 Acked Read 2 Done Sent Done false
19297 6464 Acked Read 4 Done Sent Done false
19298 6465 Acked Write 3 Done Sent Done false
19299 6466 Acked Flush 0 Done Sent Done false
19300 6467 Acked Read 10 Done Sent Done false
19301 6468 Acked Read 1 Done Sent Done false
19302 6469 Acked Flush 0 Done Sent Done false
19303 6470 Acked Write 8 Done Sent Done false
19304 6471 Acked Read 5 Done Sent Done false
19305 6472 Acked Read 7 Done Sent Done false
19306 6473 Acked Read 1 Done Sent Done false
19307 6474 Acked Read 6 Done Sent Done false
19308 6475 Acked Flush 0 Done Sent Done false
19309 6476 Acked Flush 0 Done Sent Done false
19310 6477 Acked Read 9 Done Sent Done false
19311 6478 Acked Read 3 Done Sent Done false
19312 6479 Acked Read 5 Done Sent Done false
19313 6480 Acked Read 10 Done Sent Done false
19314 6481 Acked Read 10 Done Sent Done false
19315 6482 Acked Read 7 Done Sent Done false
19316 6483 Acked Read 8 Done Sent Done false
19317 6484 Acked Write 3 Done Sent Done false
19318 6485 Acked Write 2 Done Sent Done false
19319 6486 Acked Read 2 Done Sent Done false
19320 6487 Acked Write 10 Done Sent Done false
19321 6488 Acked Read 4 Done Sent Done false
19322 6489 Acked Write 8 Done Sent Done false
19323 6490 Acked Read 8 Done Sent Done false
19324 6491 Acked Read 4 Done Sent Done false
19325 6492 Acked Write 10 Done Sent Done false
19326 6493 Acked Read 9 Done Sent Done false
19327 6494 Acked Read 1 Done Sent Done false
19328 6495 Acked Read 6 Done Sent Done false
19329 6496 Acked Flush 0 Done Sent Done false
19330 6497 Acked Write 2 Done Sent Done false
19331 6498 Acked Write 8 Done Sent Done false
19332 6499 Acked Flush 0 Done Sent Done false
19333 6500 Acked Read 8 Done Sent Done false
19334 6501 Acked Write 8 Done Sent Done false
19335 6502 Acked Write 10 Done Sent Done false
19336 6503 Acked Write 7 Done Sent Done false
19337 6504 Acked Write 5 Done Sent Done false
19338 6505 Acked Flush 0 Done Sent Done false
19339 6506 Acked Write 4 Done Sent Done false
19340 6507 Acked Read 5 Done Sent Done false
19341 6508 Acked Read 6 Done Sent Done false
19342 6509 Acked Read 9 Done Sent Done false
19343 6510 Acked Read 6 Done Sent Done false
19344 6511 Acked Read 3 Done Sent Done false
19345 6512 Acked Write 1 Done Sent Done false
19346 6513 Acked Flush 0 Done Sent Done false
19347 6514 Acked Write 8 Done Sent Done false
19348 6515 Acked Read 5 Done Sent Done false
19349 6516 Acked Write 3 Done Sent Done false
19350 6517 Acked Write 9 Done Sent Done false
19351 6518 Acked Write 10 Done Sent Done false
19352 6519 Acked Flush 0 Done Sent Done false
19353 6520 Acked Write 3 Done Sent Done false
19354 6521 Acked Write 4 Done Sent Done false
19355 6522 Acked Read 3 Done Sent Done false
19356 6523 Acked Read 10 Done Sent Done false
19357 6524 Acked Write 7 Done Sent Done false
19358 6525 Acked Read 4 Done Sent Done false
19359 6526 Acked Write 5 Done Sent Done false
19360 6527 Acked Write 5 Done Sent Done false
19361 6528 Acked Read 10 Done Sent Done false
19362 6529 Acked Write 6 Done Sent Done false
19363 6530 Acked Read 4 Done Sent Done false
19364 6531 Acked Write 3 Done Sent Done false
19365 6532 Acked Flush 0 Done Sent Done false
19366 6533 Acked Write 1 Done Sent Done false
19367 6534 Acked Write 9 Done Sent Done false
19368 6535 Acked Read 6 Done Sent Done false
19369 6536 Acked Write 3 Done Sent Done false
19370 6537 Acked Write 9 Done Sent Done false
19371 6538 Acked Write 4 Done Sent Done false
19372 6539 Acked Flush 0 Done Sent Done false
19373 6540 Acked Write 5 Done Sent Done false
19374 6541 Acked Flush 0 Done Sent Done false
19375 6542 Acked Write 7 Done Sent Done false
19376 6543 Acked Read 10 Done Sent Done false
19377 6544 Acked Write 2 Done Sent Done false
19378 6545 Acked Read 7 Done Sent Done false
19379 6546 Acked Flush 0 Done Sent Done false
19380 6547 Acked Read 9 Done Sent Done false
19381 6548 Acked Read 8 Done Sent Done false
19382 6549 Acked Write 10 Done Sent Done false
19383 6550 Acked Read 4 Done Sent Done false
19384 6551 Acked Write 1 Done Sent Done false
19385 6552 Acked Read 7 Done Sent Done false
19386 6553 Acked Read 1 Done Sent Done false
19387 6554 Acked Write 6 Done Sent Done false
19388 6555 Acked Read 3 Done Sent Done false
19389 6556 Acked Write 1 Done Sent Done false
19390 6557 Acked Write 5 Done Sent Done false
19391 6558 Acked Read 9 Done Sent Done false
19392 6559 Acked Write 6 Done Sent Done false
19393 6560 Acked Read 3 Done Sent Done false
19394 6561 Acked Write 8 Done Sent Done false
19395 6562 Acked Read 2 Done Sent Done false
19396 6563 Acked Read 1 Done Sent Done false
19397 6564 Acked Write 8 Done Sent Done false
19398 6565 Acked Read 1 Done Sent Done false
19399 6566 Acked Read 4 Done Sent Done false
19400 6567 Acked Read 2 Done Sent Done false
19401 6568 Acked Read 5 Done Sent Done false
19402 6569 Acked Read 6 Done Sent Done false
19403 6570 Acked Flush 0 Done Sent Done false
19404 6571 Acked Flush 0 Done Sent Done false
19405 6572 Acked Write 6 Done Sent Done false
19406 6573 Acked Read 3 Done Sent Done false
19407 6574 Acked Write 4 Done Sent Done false
19408 6575 Acked Write 3 Done Sent Done false
19409 6576 Acked Write 9 Done Sent Done false
19410 6577 Acked Read 3 Done Sent Done false
19411 6578 Acked Read 1 Done Sent Done false
19412 6579 Acked Write 6 Done Sent Done false
19413 6580 Acked Flush 0 Done Sent Done false
19414 6581 Acked Read 10 Done Sent Done false
19415 6582 Acked Write 8 Done Sent Done false
19416 6583 Acked Write 9 Done Sent Done false
19417 6584 Acked Read 9 Done Sent Done false
19418 6585 Acked Read 7 Done Sent Done false
19419 6586 Acked Flush 0 Done Sent Done false
19420 6587 Acked Read 5 Done Sent Done false
19421 6588 Acked Read 10 Done Sent Done false
19422 6589 Acked Write 9 Done Sent Done false
19423 6590 Acked Flush 0 Done Sent Done false
19424 6591 Acked Flush 0 Done Sent Done false
19425 6592 Acked Read 3 Done Sent Done false
19426 6593 Acked Read 7 Done Sent Done false
19427 6594 Acked Write 9 Done Sent Done false
19428 6595 Acked Read 7 Done Sent Done false
19429 6596 Acked Write 7 Done Sent Done false
19430 6597 Acked Read 4 Done Sent Done false
19431 6598 Acked Write 9 Done Sent Done false
19432 6599 Acked Read 7 Done Sent Done false
19433 6600 Acked Write 6 Done Sent Done false
19434 6601 Acked Write 2 Done Sent Done false
19435 6602 Acked Write 2 Done Sent Done false
19436 6603 Acked Write 8 Done Sent Done false
19437 6604 Acked Read 6 Done Sent Done false
19438 6605 Acked Write 2 Done Sent Done false
19439 6606 Acked Write 8 Done Sent Done false
19440 6607 Acked Write 3 Done Sent Done false
19441 6608 Acked Flush 0 Done Sent Done false
19442 6609 Acked Read 6 Done Sent Done false
19443 6610 Acked Read 7 Done Sent Done false
19444 6611 Acked Write 7 Done Sent Done false
19445 6612 Acked Write 1 Done Sent Done false
19446 6613 Acked Write 7 Done Sent Done false
19447 6614 Acked Write 3 Done Sent Done false
19448 6615 Acked Read 7 Done Sent Done false
19449 6616 Acked Read 5 Done Sent Done false
19450 6617 Acked Write 7 Done Sent Done false
19451 6618 Acked Flush 0 Done Sent Done false
19452 6619 Acked Read 7 Done Sent Done false
19453 6620 Acked Read 1 Done Sent Done false
19454 6621 Acked Flush 0 Done Sent Done false
19455 6622 Acked Read 4 Done Sent Done false
19456 6623 Acked Write 6 Done Sent Done false
19457 6624 Acked Write 2 Done Sent Done false
19458 6625 Acked Read 2 Done Sent Done false
19459 6626 Acked Read 7 Done Sent Done false
19460 6627 Acked Flush 0 Done Sent Done false
19461 6628 Acked Read 7 Done Sent Done false
19462 6629 Acked Write 5 Done Sent Done false
19463 6630 Acked Flush 0 Done Sent Done false
19464 6631 Acked Write 6 Done Sent Done false
19465 6632 Acked Read 6 Done Sent Done false
19466 6633 Acked Write 5 Done Sent Done false
19467 6634 Acked Write 3 Done Sent Done false
19468 6635 Acked Write 2 Done Sent Done false
19469 6636 Acked Read 10 Done Sent Done false
19470 6637 Acked Write 7 Done Sent Done false
19471 6638 Acked Write 5 Done Sent Done false
19472 6639 Acked Flush 0 Done Sent Done false
19473 6640 Acked Read 1 Done Sent Done false
19474 6641 Acked Write 10 Done Sent Done false
19475 6642 Acked Write 8 Done Sent Done false
19476 6643 Acked Read 4 Done Sent Done false
19477 6644 Acked Read 3 Done Sent Done false
19478 6645 Acked Write 3 Done Sent Done false
19479 6646 Acked Write 5 Done Sent Done false
19480 6647 Acked Read 4 Done Sent Done false
19481 6648 Acked Read 10 Done Sent Done false
19482 6649 Acked Write 7 Done Sent Done false
19483 6650 Acked Write 4 Done Sent Done false
19484 6651 Acked Read 10 Done Sent Done false
19485 6652 Acked Write 10 Done Sent Done false
19486 6653 Acked Write 1 Done Sent Done false
19487 6654 Acked Read 7 Done Sent Done false
19488 6655 Acked Write 4 Done Sent Done false
19489 6656 Acked Write 9 Done Sent Done false
19490 6657 Acked Write 10 Done Sent Done false
19491 6658 Acked Write 7 Done Sent Done false
19492 6659 Acked Read 5 Done Sent Done false
19493 6660 Acked Flush 0 Done Sent Done false
19494 6661 Acked Write 5 Done Sent Done false
19495 6662 Acked Write 3 Done Sent Done false
19496 6663 Acked Read 6 Done Sent Done false
19497 6664 Acked Read 1 Done Sent Done false
19498 6665 Acked Read 2 Done Sent Done false
19499 6666 Acked Write 7 Done Sent Done false
19500 6667 Acked Read 9 Done Sent Done false
19501 6668 Acked Read 6 Done Sent Done false
19502 6669 Acked Write 4 Done Sent Done false
19503 6670 Acked Read 4 Done Sent Done false
19504 6671 Acked Write 7 Done Sent Done false
19505 6672 Acked Read 4 Done Sent Done false
19506 6673 Acked Read 8 Done Sent Done false
19507 6674 Acked Read 10 Done Sent Done false
19508 6675 Acked Write 7 Done Sent Done false
19509 6676 Acked Read 1 Done Sent Done false
19510 6677 Acked Write 4 Done Sent Done false
19511 6678 Acked Write 6 Done Sent Done false
19512 6679 Acked Write 3 Done Sent Done false
19513 6680 Acked Read 2 Done Sent Done false
19514 6681 Acked Read 4 Done Sent Done false
19515 6682 Acked Write 1 Done Sent Done false
19516 6683 Acked Read 4 Done Sent Done false
19517 6684 Acked Read 8 Done Sent Done false
19518 6685 Acked Write 4 Done Sent Done false
19519 6686 Acked Read 3 Done Sent Done false
19520 6687 Acked Write 1 Done Sent Done false
19521 6688 Acked Read 5 Done Sent Done false
19522 6689 Acked Read 4 Done Sent Done false
19523 6690 Acked Read 3 Done Sent Done false
19524 6691 Acked Read 10 Done Sent Done false
19525 6692 Acked Flush 0 Done Sent Done false
19526 6693 Acked Read 8 Done Sent Done false
19527 6694 Acked Write 1 Done Sent Done false
19528 6695 Acked Read 4 Done Sent Done false
19529 6696 Acked Read 4 Done Sent Done false
19530 6697 Acked Flush 0 Done Sent Done false
19531 6698 Acked Read 8 Done Sent Done false
19532 6699 Acked Write 5 Done Sent Done false
19533 6700 Acked Write 10 Done Sent Done false
19534 6701 Acked Read 2 Done Sent Done false
19535 6702 Acked Write 10 Done Sent Done false
19536 6703 Acked Flush 0 Done Sent Done false
19537 6704 Acked Flush 0 Done Sent Done false
19538 6705 Acked Read 1 Done Sent Done false
19539 6706 Acked Read 1 Done Sent Done false
19540 6707 Acked Read 8 Done Sent Done false
19541 6708 Acked Read 8 Done Sent Done false
19542 6709 Acked Read 4 Done Sent Done false
19543 6710 Acked Read 3 Done Sent Done false
19544 6711 Acked Read 1 Done Sent Done false
19545 6712 Acked Read 4 Done Sent Done false
19546 6713 Acked Read 6 Done Sent Done false
19547 6714 Acked Read 3 Done Sent Done false
19548 6715 Acked Write 6 Done Sent Done false
19549 6716 Acked Read 9 Done Sent Done false
19550 6717 Acked Read 9 Done Sent Done false
19551 6718 Acked Flush 0 Done Sent Done false
19552 6719 Acked Read 9 Done Sent Done false
19553 6720 Acked Write 3 Done Sent Done false
19554 6721 Acked Write 7 Done Sent Done false
19555 6722 Acked Write 2 Done Sent Done false
19556 6723 Acked Flush 0 Done Sent Done false
19557 6724 Acked Write 8 Done Sent Done false
19558 6725 Acked Read 7 Done Sent Done false
19559 6726 Acked Write 8 Done Sent Done false
19560 6727 Acked Read 9 Done Sent Done false
19561 6728 Acked Read 8 Done Sent Done false
19562 6729 Acked Read 9 Done Sent Done false
19563 6730 Acked Read 2 Done Sent Done false
19564 6731 Acked Read 5 Done Sent Done false
19565 6732 Acked Read 7 Done Sent Done false
19566 6733 Acked Write 7 Done Sent Done false
19567 STATES DS:0 DS:1 DS:2 TOTAL
19568 Sent 0 301 0 301
19569 Done 301 0 301 602
19570 Skipped 0 0 0 0
19571 Error 0 0 0 0
19572 Last Flush: 6723 6432 6723
19573 Downstairs last five completed: 6432 6431 6430 6429 6428
19574 Upstairs last five completed: 6733 6732 6731 6730 6729
19575 CLIENT: Up:0 ds:301 act:2
19576 [8820] got start action
19577 [8820] I am going to restart
19578 Starting downstairs at port 8820
19579 Make output file at "/tmp/dsc/downstairs-8820.txt"
19580 [8820][1] reports Starting
19581 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1221)
19582 [8820][1] reports Running
19583 ----------------------------------------------------------------
19584 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19585 States: Active Offline Active
19586 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19587 6433 Acked Read 10 Done Sent Done false
19588 6434 Acked Read 6 Done Sent Done false
19589 6435 Acked Flush 0 Done Sent Done false
19590 6436 Acked Read 6 Done Sent Done false
19591 6437 Acked Read 9 Done Sent Done false
19592 6438 Acked Flush 0 Done Sent Done false
19593 6439 Acked Read 10 Done Sent Done false
19594 6440 Acked Read 9 Done Sent Done false
19595 6441 Acked Write 10 Done Sent Done false
19596 6442 Acked Read 3 Done Sent Done false
19597 6443 Acked Write 3 Done Sent Done false
19598 6444 Acked Read 4 Done Sent Done false
19599 6445 Acked Read 7 Done Sent Done false
19600 6446 Acked Write 1 Done Sent Done false
19601 6447 Acked Flush 0 Done Sent Done false
19602 6448 Acked Write 7 Done Sent Done false
19603 6449 Acked Write 6 Done Sent Done false
19604 6450 Acked Read 9 Done Sent Done false
19605 6451 Acked Write 5 Done Sent Done false
19606 6452 Acked Read 8 Done Sent Done false
19607 6453 Acked Write 1 Done Sent Done false
19608 6454 Acked Read 7 Done Sent Done false
19609 6455 Acked Write 2 Done Sent Done false
19610 6456 Acked Flush 0 Done Sent Done false
19611 6457 Acked Write 7 Done Sent Done false
19612 6458 Acked Write 3 Done Sent Done false
19613 6459 Acked Write 9 Done Sent Done false
19614 6460 Acked Write 4 Done Sent Done false
19615 6461 Acked Write 4 Done Sent Done false
19616 6462 Acked Write 9 Done Sent Done false
19617 6463 Acked Read 2 Done Sent Done false
19618 6464 Acked Read 4 Done Sent Done false
19619 6465 Acked Write 3 Done Sent Done false
19620 6466 Acked Flush 0 Done Sent Done false
19621 6467 Acked Read 10 Done Sent Done false
19622 6468 Acked Read 1 Done Sent Done false
19623 6469 Acked Flush 0 Done Sent Done false
19624 6470 Acked Write 8 Done Sent Done false
19625 6471 Acked Read 5 Done Sent Done false
19626 6472 Acked Read 7 Done Sent Done false
19627 6473 Acked Read 1 Done Sent Done false
19628 6474 Acked Read 6 Done Sent Done false
19629 6475 Acked Flush 0 Done Sent Done false
19630 6476 Acked Flush 0 Done Sent Done false
19631 6477 Acked Read 9 Done Sent Done false
19632 6478 Acked Read 3 Done Sent Done false
19633 6479 Acked Read 5 Done Sent Done false
19634 6480 Acked Read 10 Done Sent Done false
19635 6481 Acked Read 10 Done Sent Done false
19636 6482 Acked Read 7 Done Sent Done false
19637 6483 Acked Read 8 Done Sent Done false
19638 6484 Acked Write 3 Done Sent Done false
19639 6485 Acked Write 2 Done Sent Done false
19640 6486 Acked Read 2 Done Sent Done false
19641 6487 Acked Write 10 Done Sent Done false
19642 6488 Acked Read 4 Done Sent Done false
19643 6489 Acked Write 8 Done Sent Done false
19644 6490 Acked Read 8 Done Sent Done false
19645 6491 Acked Read 4 Done Sent Done false
19646 6492 Acked Write 10 Done Sent Done false
19647 6493 Acked Read 9 Done Sent Done false
19648 6494 Acked Read 1 Done Sent Done false
19649 6495 Acked Read 6 Done Sent Done false
19650 6496 Acked Flush 0 Done Sent Done false
19651 6497 Acked Write 2 Done Sent Done false
19652 6498 Acked Write 8 Done Sent Done false
19653 6499 Acked Flush 0 Done Sent Done false
19654 6500 Acked Read 8 Done Sent Done false
19655 6501 Acked Write 8 Done Sent Done false
19656 6502 Acked Write 10 Done Sent Done false
19657 6503 Acked Write 7 Done Sent Done false
19658 6504 Acked Write 5 Done Sent Done false
19659 6505 Acked Flush 0 Done Sent Done false
19660 6506 Acked Write 4 Done Sent Done false
19661 6507 Acked Read 5 Done Sent Done false
19662 6508 Acked Read 6 Done Sent Done false
19663 6509 Acked Read 9 Done Sent Done false
19664 6510 Acked Read 6 Done Sent Done false
19665 6511 Acked Read 3 Done Sent Done false
19666 6512 Acked Write 1 Done Sent Done false
19667 6513 Acked Flush 0 Done Sent Done false
19668 6514 Acked Write 8 Done Sent Done false
19669 6515 Acked Read 5 Done Sent Done false
19670 6516 Acked Write 3 Done Sent Done false
19671 6517 Acked Write 9 Done Sent Done false
19672 6518 Acked Write 10 Done Sent Done false
19673 6519 Acked Flush 0 Done Sent Done false
19674 6520 Acked Write 3 Done Sent Done false
19675 6521 Acked Write 4 Done Sent Done false
19676 6522 Acked Read 3 Done Sent Done false
19677 6523 Acked Read 10 Done Sent Done false
19678 6524 Acked Write 7 Done Sent Done false
19679 6525 Acked Read 4 Done Sent Done false
19680 6526 Acked Write 5 Done Sent Done false
19681 6527 Acked Write 5 Done Sent Done false
19682 6528 Acked Read 10 Done Sent Done false
19683 6529 Acked Write 6 Done Sent Done false
19684 6530 Acked Read 4 Done Sent Done false
19685 6531 Acked Write 3 Done Sent Done false
19686 6532 Acked Flush 0 Done Sent Done false
19687 6533 Acked Write 1 Done Sent Done false
19688 6534 Acked Write 9 Done Sent Done false
19689 6535 Acked Read 6 Done Sent Done false
19690 6536 Acked Write 3 Done Sent Done false
19691 6537 Acked Write 9 Done Sent Done false
19692 6538 Acked Write 4 Done Sent Done false
19693 6539 Acked Flush 0 Done Sent Done false
19694 6540 Acked Write 5 Done Sent Done false
19695 6541 Acked Flush 0 Done Sent Done false
19696 6542 Acked Write 7 Done Sent Done false
19697 6543 Acked Read 10 Done Sent Done false
19698 6544 Acked Write 2 Done Sent Done false
19699 6545 Acked Read 7 Done Sent Done false
19700 6546 Acked Flush 0 Done Sent Done false
19701 6547 Acked Read 9 Done Sent Done false
19702 6548 Acked Read 8 Done Sent Done false
19703 6549 Acked Write 10 Done Sent Done false
19704 6550 Acked Read 4 Done Sent Done false
19705 6551 Acked Write 1 Done Sent Done false
19706 6552 Acked Read 7 Done Sent Done false
19707 6553 Acked Read 1 Done Sent Done false
19708 6554 Acked Write 6 Done Sent Done false
19709 6555 Acked Read 3 Done Sent Done false
19710 6556 Acked Write 1 Done Sent Done false
19711 6557 Acked Write 5 Done Sent Done false
19712 6558 Acked Read 9 Done Sent Done false
19713 6559 Acked Write 6 Done Sent Done false
19714 6560 Acked Read 3 Done Sent Done false
19715 6561 Acked Write 8 Done Sent Done false
19716 6562 Acked Read 2 Done Sent Done false
19717 6563 Acked Read 1 Done Sent Done false
19718 6564 Acked Write 8 Done Sent Done false
19719 6565 Acked Read 1 Done Sent Done false
19720 6566 Acked Read 4 Done Sent Done false
19721 6567 Acked Read 2 Done Sent Done false
19722 6568 Acked Read 5 Done Sent Done false
19723 6569 Acked Read 6 Done Sent Done false
19724 6570 Acked Flush 0 Done Sent Done false
19725 6571 Acked Flush 0 Done Sent Done false
19726 6572 Acked Write 6 Done Sent Done false
19727 6573 Acked Read 3 Done Sent Done false
19728 6574 Acked Write 4 Done Sent Done false
19729 6575 Acked Write 3 Done Sent Done false
19730 6576 Acked Write 9 Done Sent Done false
19731 6577 Acked Read 3 Done Sent Done false
19732 6578 Acked Read 1 Done Sent Done false
19733 6579 Acked Write 6 Done Sent Done false
19734 6580 Acked Flush 0 Done Sent Done false
19735 6581 Acked Read 10 Done Sent Done false
19736 6582 Acked Write 8 Done Sent Done false
19737 6583 Acked Write 9 Done Sent Done false
19738 6584 Acked Read 9 Done Sent Done false
19739 6585 Acked Read 7 Done Sent Done false
19740 6586 Acked Flush 0 Done Sent Done false
19741 6587 Acked Read 5 Done Sent Done false
19742 6588 Acked Read 10 Done Sent Done false
19743 6589 Acked Write 9 Done Sent Done false
19744 6590 Acked Flush 0 Done Sent Done false
19745 6591 Acked Flush 0 Done Sent Done false
19746 6592 Acked Read 3 Done Sent Done false
19747 6593 Acked Read 7 Done Sent Done false
19748 6594 Acked Write 9 Done Sent Done false
19749 6595 Acked Read 7 Done Sent Done false
19750 6596 Acked Write 7 Done Sent Done false
19751 6597 Acked Read 4 Done Sent Done false
19752 6598 Acked Write 9 Done Sent Done false
19753 6599 Acked Read 7 Done Sent Done false
19754 6600 Acked Write 6 Done Sent Done false
19755 6601 Acked Write 2 Done Sent Done false
19756 6602 Acked Write 2 Done Sent Done false
19757 6603 Acked Write 8 Done Sent Done false
19758 6604 Acked Read 6 Done Sent Done false
19759 6605 Acked Write 2 Done Sent Done false
19760 6606 Acked Write 8 Done Sent Done false
19761 6607 Acked Write 3 Done Sent Done false
19762 6608 Acked Flush 0 Done Sent Done false
19763 6609 Acked Read 6 Done Sent Done false
19764 6610 Acked Read 7 Done Sent Done false
19765 6611 Acked Write 7 Done Sent Done false
19766 6612 Acked Write 1 Done Sent Done false
19767 6613 Acked Write 7 Done Sent Done false
19768 6614 Acked Write 3 Done Sent Done false
19769 6615 Acked Read 7 Done Sent Done false
19770 6616 Acked Read 5 Done Sent Done false
19771 6617 Acked Write 7 Done Sent Done false
19772 6618 Acked Flush 0 Done Sent Done false
19773 6619 Acked Read 7 Done Sent Done false
19774 6620 Acked Read 1 Done Sent Done false
19775 6621 Acked Flush 0 Done Sent Done false
19776 6622 Acked Read 4 Done Sent Done false
19777 6623 Acked Write 6 Done Sent Done false
19778 6624 Acked Write 2 Done Sent Done false
19779 6625 Acked Read 2 Done Sent Done false
19780 6626 Acked Read 7 Done Sent Done false
19781 6627 Acked Flush 0 Done Sent Done false
19782 6628 Acked Read 7 Done Sent Done false
19783 6629 Acked Write 5 Done Sent Done false
19784 6630 Acked Flush 0 Done Sent Done false
19785 6631 Acked Write 6 Done Sent Done false
19786 6632 Acked Read 6 Done Sent Done false
19787 6633 Acked Write 5 Done Sent Done false
19788 6634 Acked Write 3 Done Sent Done false
19789 6635 Acked Write 2 Done Sent Done false
19790 6636 Acked Read 10 Done Sent Done false
19791 6637 Acked Write 7 Done Sent Done false
19792 6638 Acked Write 5 Done Sent Done false
19793 6639 Acked Flush 0 Done Sent Done false
19794 6640 Acked Read 1 Done Sent Done false
19795 6641 Acked Write 10 Done Sent Done false
19796 6642 Acked Write 8 Done Sent Done false
19797 6643 Acked Read 4 Done Sent Done false
19798 6644 Acked Read 3 Done Sent Done false
19799 6645 Acked Write 3 Done Sent Done false
19800 6646 Acked Write 5 Done Sent Done false
19801 6647 Acked Read 4 Done Sent Done false
19802 6648 Acked Read 10 Done Sent Done false
19803 6649 Acked Write 7 Done Sent Done false
19804 6650 Acked Write 4 Done Sent Done false
19805 6651 Acked Read 10 Done Sent Done false
19806 6652 Acked Write 10 Done Sent Done false
19807 6653 Acked Write 1 Done Sent Done false
19808 6654 Acked Read 7 Done Sent Done false
19809 6655 Acked Write 4 Done Sent Done false
19810 6656 Acked Write 9 Done Sent Done false
19811 6657 Acked Write 10 Done Sent Done false
19812 6658 Acked Write 7 Done Sent Done false
19813 6659 Acked Read 5 Done Sent Done false
19814 6660 Acked Flush 0 Done Sent Done false
19815 6661 Acked Write 5 Done Sent Done false
19816 6662 Acked Write 3 Done Sent Done false
19817 6663 Acked Read 6 Done Sent Done false
19818 6664 Acked Read 1 Done Sent Done false
19819 6665 Acked Read 2 Done Sent Done false
19820 6666 Acked Write 7 Done Sent Done false
19821 6667 Acked Read 9 Done Sent Done false
19822 6668 Acked Read 6 Done Sent Done false
19823 6669 Acked Write 4 Done Sent Done false
19824 6670 Acked Read 4 Done Sent Done false
19825 6671 Acked Write 7 Done Sent Done false
19826 6672 Acked Read 4 Done Sent Done false
19827 6673 Acked Read 8 Done Sent Done false
19828 6674 Acked Read 10 Done Sent Done false
19829 6675 Acked Write 7 Done Sent Done false
19830 6676 Acked Read 1 Done Sent Done false
19831 6677 Acked Write 4 Done Sent Done false
19832 6678 Acked Write 6 Done Sent Done false
19833 6679 Acked Write 3 Done Sent Done false
19834 6680 Acked Read 2 Done Sent Done false
19835 6681 Acked Read 4 Done Sent Done false
19836 6682 Acked Write 1 Done Sent Done false
19837 6683 Acked Read 4 Done Sent Done false
19838 6684 Acked Read 8 Done Sent Done false
19839 6685 Acked Write 4 Done Sent Done false
19840 6686 Acked Read 3 Done Sent Done false
19841 6687 Acked Write 1 Done Sent Done false
19842 6688 Acked Read 5 Done Sent Done false
19843 6689 Acked Read 4 Done Sent Done false
19844 6690 Acked Read 3 Done Sent Done false
19845 6691 Acked Read 10 Done Sent Done false
19846 6692 Acked Flush 0 Done Sent Done false
19847 6693 Acked Read 8 Done Sent Done false
19848 6694 Acked Write 1 Done Sent Done false
19849 6695 Acked Read 4 Done Sent Done false
19850 6696 Acked Read 4 Done Sent Done false
19851 6697 Acked Flush 0 Done Sent Done false
19852 6698 Acked Read 8 Done Sent Done false
19853 6699 Acked Write 5 Done Sent Done false
19854 6700 Acked Write 10 Done Sent Done false
19855 6701 Acked Read 2 Done Sent Done false
19856 6702 Acked Write 10 Done Sent Done false
19857 6703 Acked Flush 0 Done Sent Done false
19858 6704 Acked Flush 0 Done Sent Done false
19859 6705 Acked Read 1 Done Sent Done false
19860 6706 Acked Read 1 Done Sent Done false
19861 6707 Acked Read 8 Done Sent Done false
19862 6708 Acked Read 8 Done Sent Done false
19863 6709 Acked Read 4 Done Sent Done false
19864 6710 Acked Read 3 Done Sent Done false
19865 6711 Acked Read 1 Done Sent Done false
19866 6712 Acked Read 4 Done Sent Done false
19867 6713 Acked Read 6 Done Sent Done false
19868 6714 Acked Read 3 Done Sent Done false
19869 6715 Acked Write 6 Done Sent Done false
19870 6716 Acked Read 9 Done Sent Done false
19871 6717 Acked Read 9 Done Sent Done false
19872 6718 Acked Flush 0 Done Sent Done false
19873 6719 Acked Read 9 Done Sent Done false
19874 6720 Acked Write 3 Done Sent Done false
19875 6721 Acked Write 7 Done Sent Done false
19876 6722 Acked Write 2 Done Sent Done false
19877 6723 Acked Flush 0 Done Sent Done false
19878 6724 Acked Write 8 Done Sent Done false
19879 6725 Acked Read 7 Done Sent Done false
19880 6726 Acked Write 8 Done Sent Done false
19881 6727 Acked Read 9 Done Sent Done false
19882 6728 Acked Read 8 Done Sent Done false
19883 6729 Acked Read 9 Done Sent Done false
19884 6730 Acked Read 2 Done Sent Done false
19885 6731 Acked Read 5 Done Sent Done false
19886 6732 Acked Read 7 Done Sent Done false
19887 6733 Acked Write 7 Done Sent Done false
19888 6734 Acked Flush 0 Done Sent Done false
19889 STATES DS:0 DS:1 DS:2 TOTAL
19890 Sent 0 302 0 302
19891 Done 302 0 302 604
19892 Skipped 0 0 0 0
19893 Error 0 0 0 0
19894 Last Flush: 6734 6432 6734
19895 Downstairs last five completed: 6432 6431 6430 6429 6428
19896 Upstairs last five completed: 6734 6733 6732 6731 6730
19897 CLIENT: Up:0 ds:302 act:2
198982025-01-14T16:14:46.315ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
198992025-01-14T16:14:46.315ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:43188) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
199002025-01-14T16:14:46.316ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
199012025-01-14T16:14:46.317ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
199022025-01-14T16:14:46.317ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
199032025-01-14T16:14:46.317ZINFOcrucible: send last flush ID to this DS: 6432 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
199042025-01-14T16:14:46.317ZINFOcrucible: Replied this last flush ID: 6432 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
199052025-01-14T16:14:46.317ZINFOcrucible: [1] client re-new 302 jobs since flush 6432 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
199062025-01-14T16:14:46.317ZINFOcrucible: [1] Replayed 302 jobs since flush: 6432 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
19907 ----------------------------------------------------------------
19908 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
19909 Downstairs last five completed: 6734 6733 6732 6731 6730
19910 Upstairs last five completed: 6734 6733 6732 6731 6730
19911 CLIENT: Up:0 ds:0 act:3
19912 Replay: All jobs finished
19913 Main task has work to do, go find it
19914 got dsc Stop(0)
19915 stop 0
19916 Jan 14 16:14:48.557 INFO request completed, latency_us: 341, response_code:[8810] Got stop action so:false kr:false
19917 204, uri: /stop/cid/0, method: POST, req_id: f7ac7095-248a-4192-a53b-9457126b0816, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
199182025-01-14T16:14:48.557ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
199192025-01-14T16:14:48.558ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
199202025-01-14T16:14:48.558ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
19921 Jan 14 16:14:48.558 INFO request completed, latency_us: 122, response_code: 200, uri: /state/cid/0, method: GET, req_id: c71602d4-aa67-4d8b-9809-4a7d1db76743, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
199222025-01-14T16:14:48.558ZINFOcrucible: client task is exiting = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
19923 [8810] Exited with: ExitStatus(unix_wait_status(9))
19924 [8810][0] reports Exit
19925 Jan 14 16:14:52.559 INFO request completed, latency_us: 246, response_code: 200, uri: /state/cid/0, method: GET, req_id: a61f0c09-4131-44e8-938e-88a6b8d78a71, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
19926 001/300 Write block 4205 len 32768 data: 4 5 6 4 4 6 7 6
19927 002/300 Read block 3751 len 36864
19928 003/300 Read block 4116 len 12288
19929 004/300 Read block 1812 len 28672
19930 005/300 Write block 1661 len 12288 data: 4 4 4
19931 006/300 Read block 604 len 36864
19932 007/300 Write block 884 len 40960 data: 7 8 7 7 5 5 5 5 6 6
19933 008/300 Write block 1486 len 20480 data: 4 4 4 5 6
19934 009/300 Write block 1179 len 28672 data: 5 5 4 5 6 5 6
19935 010/300 Flush
19936 011/300 Read block 4041 len 8192
19937 012/300 Flush
19938 013/300 Read block 2070 len 28672
19939 014/300 Read block 1281 len 20480
19940 015/300 Write block 2600 len 36864 data: 5 5 5 5 5 6 6 4 4
19941 016/300 Write block 2307 len 20480 data: 5 5 5 6 6
19942 017/300 Read block 4504 len 24576
19943 018/300 Write block 3101 len 4096 data: 4
19944 019/300 Write block 4711 len 32768 data: 2 3 4 3 3 4 5 3
19945 020/300 Write block 3582 len 32768 data: 5 4 5 4 3 3 3 2
19946 021/300 Write block 2708 len 4096 data: 4
19947 022/300 Read block 4049 len 40960
19948 023/300 Read block 850 len 16384
19949 024/300 Write block 4906 len 20480 data: 4 4 2 2 2
19950 025/300 Read block 3525 len 20480
19951 026/300 Read block 4408 len 4096
19952 027/300 Read block 391 len 28672
19953 028/300 Read block 2371 len 32768
19954 029/300 Read block 2578 len 4096
19955 030/300 Read block 3615 len 20480
19956 031/300 Read block 3706 len 8192
19957 032/300 Read block 2968 len 36864
19958 033/300 Write block 1368 len 36864 data: 4 2 2 2 3 3 3 6 6
19959 034/300 Read block 3786 len 16384
19960 035/300 Read block 2063 len 32768
19961 036/300 Read block 2298 len 12288
19962 037/300 Write block 4280 len 8192 data: 6 6
19963 038/300 Read block 932 len 32768
19964 039/300 Read block 1622 len 16384
19965 040/300 Write block 3579 len 28672 data: 5 6 5 6 5 6 5
19966 041/300 Read block 80 len 40960
19967 042/300 Read block 2852 len 20480
19968 043/300 Read block 643 len 24576
19969 044/300 Write block 3186 len 12288 data: 4 5 4
19970 045/300 Write block 1133 len 40960 data: 3 3 2 3 3 3 2 2 3 3
19971 046/300 Read block 979 len 24576
19972 047/300 Read block 3669 len 40960
19973 048/300 Read block 221 len 20480
19974 049/300 Read block 315 len 24576
19975 050/300 Read block 2904 len 12288
19976 051/300 Write block 137 len 36864 data: 5 5 4 6 4 6 8 8 8
19977 052/300 Write block 3326 len 40960 data: 3 3 4 4 4 4 4 5 5 5
19978 053/300 Write block 476 len 12288 data: 4 5 5
19979 054/300 Write block 755 len 32768 data: 2 3 4 4 4 4 4 4
19980 055/300 Read block 3465 len 28672
19981 056/300 Read block 4266 len 4096
19982 057/300 Write block 3846 len 32768 data: 7 7 9 8 7 6 5 4
19983 058/300 Read block 2448 len 32768
19984 059/300 Read block 4720 len 36864
19985 060/300 Flush
19986 061/300 Write block 4920 len 16384 data: 5 5 4 3
19987 062/300 Read block 1469 len 20480
19988 063/300 Write block 1956 len 40960 data: 6 7 4 4 4 5 5 3 3 3
19989 064/300 Read block 1902 len 4096
19990 065/300 Read block 3233 len 4096
19991 066/300 Read block 352 len 32768
19992 067/300 Write block 3188 len 40960 data: 5 4 5 5 4 6 5 5 6 8
19993 068/300 Read block 3377 len 36864
19994 069/300 Read block 2030 len 20480
19995 070/300 Read block 1436 len 8192
19996 071/300 Read block 355 len 24576
19997 072/300 Read block 3244 len 20480
19998 073/300 Write block 4361 len 12288 data: 2 2 2
19999 074/300 Read block 3115 len 40960
20000 075/300 Read block 3764 len 28672
20001 076/300 Flush
20002 077/300 Write block 134 len 12288 data: 5 5 5
20003 078/300 Read block 994 len 40960
20004 079/300 Read block 1854 len 8192
20005 080/300 Write block 2837 len 24576 data: 5 4 4 3 3 4
20006 081/300 Write block 2510 len 36864 data: 4 3 4 3 3 2 2 3 3
20007 082/300 Read block 2752 len 4096
20008 083/300 Read block 390 len 32768
20009 084/300 Write block 56 len 4096 data: 5
20010 085/300 Write block 3720 len 4096 data: 4
20011 086/300 Write block 1519 len 24576 data: 2 2 4 4 5 6
20012 087/300 Write block 3735 len 8192 data: 3 4
20013 088/300 Read block 4609 len 24576
20014 089/300 Write block 438 len 8192 data: 5 4
20015 090/300 Read block 3320 len 12288
20016 091/300 Write block 2922 len 20480 data: 7 6 6 6 6
20017 092/300 Read block 2096 len 4096
20018 093/300 Flush
20019 094/300 Write block 3424 len 12288 data: 10 9 9
20020 095/300 Read block 3306 len 20480
20021 096/300 Write block 1422 len 28672 data: 4 4 4 5 4 4 2
20022 097/300 Read block 815 len 32768
20023 098/300 Flush
20024 099/300 Write block 90 len 24576 data: 3 2 2 2 3 5
20025 100/300 Write block 3595 len 24576 data: 3 3 3 3 3 3
20026 101/300 Write block 1084 len 28672 data: 4 4 4 3 2 2 2
20027 102/300 Read block 2862 len 36864
20028 103/300 Read block 1692 len 28672
20029 104/300 Write block 1671 len 28672 data: 6 6 7 7 7 7 6
20030 105/300 Write block 1940 len 8192 data: 6 6
20031 106/300 Read block 4506 len 24576
20032 107/300 Read block 1296 len 36864
20033 108/300 Flush
20034 109/300 Read block 2978 len 32768
20035 110/300 Write block 1278 len 28672 data: 7 7 6 5 5 3 3
20036 111/300 Read block 2261 len 32768
20037 112/300 Write block 3908 len 20480 data: 4 5 6 6 6
20038 113/300 Write block 1478 len 20480 data: 4 4 4 4 4
20039 114/300 Read block 500 len 24576
20040 115/300 Write block 4010 len 16384 data: 7 6 5 5
20041 116/300 Write block 2505 len 16384 data: 3 4 3 4
20042 117/300 Read block 4467 len 32768
20043 118/300 Write block 635 len 20480 data: 3 3 3 3 2
20044 119/300 Write block 4636 len 12288 data: 4 4 4
20045 120/300 Flush
20046 121/300 Write block 3842 len 24576 data: 4 4 6 6 8 8
20047 122/300 Read block 1717 len 12288
20048 123/300 Read block 464 len 24576
20049 124/300 Flush
20050 125/300 Read block 1244 len 40960
20051 126/300 Write block 2217 len 20480 data: 6 5 5 3 4
20052 127/300 Read block 4864 len 24576
20053 128/300 Write block 4138 len 32768 data: 7 7 6 6 6 6 5 4
20054 129/300 Write block 4951 len 12288 data: 4 5 6
20055 130/300 Write block 2852 len 12288 data: 4 4 4
20056 131/300 Write block 3173 len 8192 data: 3 4
20057 132/300 Write block 513 len 12288 data: 8 8 8
20058 133/300 Flush
20059 134/300 Flush
20060 135/300 Read block 3856 len 20480
20061 136/300 Write block 2835 len 16384 data: 5 6 6 5
20062 137/300 Read block 1690 len 32768
20063 138/300 Write block 244 len 28672 data: 6 5 4 3 3 2 3
20064 139/300 Write block 607 len 20480 data: 3 3 4 5 5
20065 140/300 Write block 1442 len 36864 data: 7 6 6 6 7 7 5 5 6
20066 141/300 Write block 199 len 16384 data: 4 4 4 4
20067 142/300 Read block 1206 len 16384
20068 143/300 Write block 3753 len 20480 data: 3 3 4 4 5
20069 144/300 Write block 1754 len 20480 data: 9 9 8 7 5
20070 145/300 Read block 833 len 28672
20071 146/300 Write block 1524 len 24576 data: 7 6 5 5 6 6
20072 147/300 Write block 3420 len 16384 data: 10 9 8 9
20073 148/300 Flush
20074 149/300 Flush
20075 150/300 Read block 1268 len 36864
20076 151/300 Write block 2939 len 4096 data: 4
20077 152/300 Write block 3591 len 16384 data: 2 2 3 3
20078 153/300 Read block 3990 len 36864
20079 154/300 Write block 815 len 4096 data: 7
20080 155/300 Read block 4644 len 20480
20081 156/300 Write block 3895 len 4096 data: 5
20082 157/300 Read block 1652 len 16384
20083 158/300 Write block 2277 len 28672 data: 6 5 7 7 7 7 7
20084 159/300 Read block 2765 len 8192
20085 160/300 Write block 1685 len 20480 data: 4 3 4 4 3
20086 161/300 Read block 2526 len 12288
20087 162/300 Read block 2382 len 8192
20088 163/300 Read block 4129 len 36864
20089 164/300 Read block 1929 len 40960
20090 165/300 Read block 2833 len 24576
20091 166/300 Read block 1220 len 28672
20092 167/300 Write block 3943 len 4096 data: 5
20093 168/300 Read block 793 len 36864
20094 169/300 Read block 1987 len 36864
20095 170/300 Read block 4312 len 40960
20096 171/300 Read block 4500 len 32768
20097 172/300 Write block 290 len 16384 data: 5 6 6 6
20098 173/300 Write block 2410 len 16384 data: 4 4 4 4
20099 174/300 Read block 4847 len 4096
20100 175/300 Write block 1220 len 16384 data: 6 6 6 5
20101 176/300 Read block 4521 len 28672
20102 177/300 Write block 287 len 24576 data: 4 6 5 6 7 7
20103 178/300 Read block 840 len 36864
20104 179/300 Read block 3069 len 24576
20105 180/300 Write block 1355 len 28672 data: 6 6 6 5 5 3 3
20106 181/300 Write block 2582 len 28672 data: 7 4 4 4 3 3 2
20107 182/300 Write block 1069 len 16384 data: 4 5 6 5
20108 183/300 Write block 2614 len 20480 data: 8 7 7 8 10
20109 184/300 Flush
20110 185/300 Read block 3953 len 4096
20111 186/300 Write block 1181 len 40960 data: 5 6 7 6 7 5 4 5 5 7
20112 187/300 Flush
20113 188/300 Read block 3996 len 36864
20114 189/300 Write block 3154 len 24576 data: 5 6 6 7 5 4
20115 190/300 Read block 1789 len 16384
20116 191/300 Read block 3286 len 16384
20117 192/300 Read block 99 len 36864
20118 193/300 Read block 4053 len 16384
20119 194/300 Read block 2620 len 4096
20120 195/300 Write block 3911 len 24576 data: 7 7 7 5 7 7
20121 196/300 Write block 368 len 32768 data: 3 4 4 4 5 5 7 6
20122 197/300 Write block 1598 len 4096 data: 6
20123 198/300 Write block 2438 len 4096 data: 4
20124 199/300 Write block 2628 len 40960 data: 2 2 4 3 3 3 4 4 4 4
20125 200/300 Write block 2692 len 32768 data: 3 6 8 8 8 9 7 6
20126 201/300 Write block 1657 len 20480 data: 3 5 6 3 5
20127 202/300 Write block 1970 len 16384 data: 4 4 3 3
20128 203/300 Write block 1859 len 28672 data: 6 5 4 4 3 2 2
20129 204/300 Write block 2316 len 40960 data: 8 8 7 7 5 4 3 4 5 5
20130 205/300 Write block 595 len 40960 data: 4 5 4 3 3 3 2 2 2 2
20131 206/300 Write block 2984 len 28672 data: 5 4 4 3 2 4 4
20132 207/300 Read block 807 len 20480
20133 208/300 Read block 3448 len 36864
20134 209/300 Read block 4053 len 24576
20135 210/300 Write block 2097 len 40960 data: 2 3 4 5 6 5 5 4 3 5
20136 211/300 Write block 2674 len 16384 data: 5 5 3 3
20137 212/300 Write block 1841 len 20480 data: 5 4 4 5 4
20138 213/300 Read block 2967 len 4096
20139 214/300 Write block 2977 len 28672 data: 4 6 6 7 6 6 6
20140 215/300 Read block 3718 len 28672
20141 216/300 Read block 3593 len 32768
20142 217/300 Write block 4482 len 20480 data: 3 3 4 5 5
20143 218/300 Read block 2447 len 36864
20144 219/300 Write block 38 len 8192 data: 4 5
20145 220/300 Read block 556 len 4096
20146 221/300 Read block 4793 len 12288
20147 222/300 Write block 968 len 8192 data: 4 4
20148 223/300 Read block 579 len 24576
20149 224/300 Read block 3376 len 32768
20150 225/300 Read block 4639 len 8192
20151 226/300 Read block 3156 len 24576
20152 227/300 Write block 1965 len 28672 data: 4 3 3 3 3 5 5
20153 228/300 Flush
20154 229/300 Read block 355 len 40960
20155 230/300 Write block 4217 len 40960 data: 4 6 6 6 6 7 4 3 5 4
20156 231/300 Read block 693 len 28672
20157 232/300 Read block 4351 len 24576
20158 233/300 Read block 4056 len 32768
20159 234/300 Write block 563 len 8192 data: 7 8
20160 235/300 Write block 2798 len 20480 data: 5 5 5 6 4
20161 236/300 Read block 3167 len 4096
20162 237/300 Read block 3776 len 36864
20163 238/300 Read block 966 len 4096
20164 239/300 Flush
20165 240/300 Read block 3253 len 24576
20166 241/300 Write block 1649 len 40960 data: 4 4 4 4 3 3 4 4 4 6
20167 242/300 Write block 886 len 32768 data: 8 8 6 6 6 6 7 7
20168 243/300 Read block 861 len 28672
20169 244/300 Flush
20170 245/300 Read block 4316 len 12288
20171 246/300 Write block 2819 len 24576 data: 3 3 3 3 4 3
20172 247/300 Flush
20173 248/300 Read block 513 len 8192
20174 249/300 Read block 3672 len 28672
20175 250/300 Read block 841 len 36864
20176 251/300 Flush
20177 252/300 Read block 2170 len 24576
20178 253/300 Read block 2678 len 16384
20179 254/300 Read block 260 len 32768
20180 255/300 Read block 2047 len 4096
20181 256/300 Write block 2870 len 40960 data: 8 8 8 8 7 6 8 8 9 8
20182 257/300 Write block 1023 len 36864 data: 4 5 4 4 3 3 4 4 5
20183 258/300 Write block 2371 len 36864 data: 6 4 4 4 4 6 7 7 6
20184 259/300 Write block 1993 len 28672 data: 10 7 8 9 8 7 6
20185 260/300 Write block 792 len 28672 data: 6 6 6 6 6 7 7
20186 261/300 Read block 2959 len 36864
20187 262/300 Write block 4982 len 32768 data: 5 5 5 4 3 3 4 4
20188 263/300 Write block 4259 len 32768 data: 3 2 3 3 4 4 4 4
20189 264/300 Write block 1613 len 28672 data: 4 3 3 3 5 5 5
20190 265/300 Flush
20191 266/300 Read block 700 len 28672
20192 267/300 Read block 4598 len 40960
20193 268/300 Write block 669 len 20480 data: 4 4 5 5 6
20194 269/300 Read block 3803 len 8192
20195 270/300 Flush
20196 271/300 Write block 637 len 20480 data: 4 4 3 3 2
20197 272/300 Write block 1018 len 36864 data: 5 4 6 6 5 5 6 5 5
20198 273/300 Read block 386 len 16384
20199 274/300 Write block 1686 len 32768 data: 4 5 5 4 3 3 3 4
20200 275/300 Read block 3125 len 12288
20201 276/300 Write block 1273 len 20480 data: 4 4 6 7 8
20202 277/300 Read block 3671 len 20480
20203 278/300 Read block 4162 len 24576
20204 279/300 Read block 4009 len 36864
20205 280/300 Read block 1334 len 20480
20206 281/300 Read block 4331 len 28672
20207 282/300 Write block 320 len 8192 data: 7 5
20208 283/300 Read block 3806 len 4096
20209 284/300 Flush
20210 285/300 Read block 2337 len 40960
20211 286/300 Write block 2956 len 4096 data: 3
20212 287/300 Flush
20213 288/300 Read block 1968 len 8192
20214 289/300 Read block 4392 len 4096
20215 290/300 Read block 213 len 8192
20216 291/300 Write block 4782 len 28672 data: 3 3 3 3 3 3 4
20217 292/300 Read block 2272 len 12288
20218 293/300 Read block 1598 len 36864
20219 294/300 Read block 2632 len 32768
20220 295/300 Write block 2160 len 12288 data: 5 5 4
20221 296/300 Write block 4966 len 40960 data: 3 4 3 3 3 4 4 4 5 6
20222 297/300 Write block 2485 len 40960 data: 4 4 3 2 4 4 4 5 5 4
20223 298/300 Read block 88 len 28672
20224 299/300 Write block 1939 len 20480 data: 5 7 7 6 6
20225 300/300 Read block 4656 len 20480
20226 301/300 Write block 1510 len 16384 data: 5 4 4 4
20227 Main task has work to do, go find it
20228 got dsc Start(0)
20229 start 0
20230 Jan 14 16:14:52.798 INFO request completed, latency_us: 237, response_code: 204, uri: /start/cid/0, method: POST, req_id: ccb1a743-4c19-4094-8dff-cc3692b5b9c4, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
20231 Replay: started 0, returned:Ok(())
20232 ----------------------------------------------------------------
20233 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
20234 States: Offline Active Active
20235 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20236 6735 Acked Write 8 Sent Done Done false
20237 6736 Acked Read 9 Sent Done Done false
20238 6737 Acked Read 3 Sent Done Done false
20239 6738 Acked Read 7 Sent Done Done false
20240 6739 Acked Write 3 Sent Done Done false
20241 6740 Acked Read 9 Sent Done Done false
20242 6741 Acked Write 10 Sent Done Done false
20243 6742 Acked Write 5 Sent Done Done false
20244 6743 Acked Write 7 Sent Done Done false
20245 6744 Acked Flush 0 Sent Done Done false
20246 6745 Acked Read 2 Sent Done Done false
20247 6746 Acked Flush 0 Sent Done Done false
20248 6747 Acked Read 7 Sent Done Done false
20249 6748 Acked Read 5 Sent Done Done false
20250 6749 Acked Write 9 Sent Done Done false
20251 6750 Acked Write 5 Sent Done Done false
20252 6751 Acked Read 6 Sent Done Done false
20253 6752 Acked Write 1 Sent Done Done false
20254 6753 Acked Write 8 Sent Done Done false
20255 6754 Acked Write 8 Sent Done Done false
20256 6755 Acked Write 1 Sent Done Done false
20257 6756 Acked Read 10 Sent Done Done false
20258 6757 Acked Read 4 Sent Done Done false
20259 6758 Acked Write 5 Sent Done Done false
20260 6759 Acked Read 5 Sent Done Done false
20261 6760 Acked Read 1 Sent Done Done false
20262 6761 Acked Read 7 Sent Done Done false
20263 6762 Acked Read 8 Sent Done Done false
20264 6763 Acked Read 1 Sent Done Done false
20265 6764 Acked Read 5 Sent Done Done false
20266 6765 Acked Read 2 Sent Done Done false
20267 6766 Acked Read 9 Sent Done Done false
20268 6767 Acked Write 9 Sent Done Done false
20269 6768 Acked Read 4 Sent Done Done false
20270 6769 Acked Read 8 Sent Done Done false
20271 6770 Acked Read 3 Sent Done Done false
20272 6771 Acked Write 2 Sent Done Done false
20273 6772 Acked Read 8 Sent Done Done false
20274 6773 Acked Read 4 Sent Done Done false
20275 6774 Acked Write 7 Sent Done Done false
20276 6775 Acked Read 10 Sent Done Done false
20277 6776 Acked Read 5 Sent Done Done false
20278 6777 Acked Read 6 Sent Done Done false
20279 6778 Acked Write 3 Sent Done Done false
20280 6779 Acked Write 10 Sent Done Done false
20281 6780 Acked Read 6 Sent Done Done false
20282 6781 Acked Read 10 Sent Done Done false
20283 6782 Acked Read 5 Sent Done Done false
20284 6783 Acked Read 6 Sent Done Done false
20285 6784 Acked Read 3 Sent Done Done false
20286 6785 Acked Write 9 Sent Done Done false
20287 6786 Acked Write 10 Sent Done Done false
20288 6787 Acked Write 3 Sent Done Done false
20289 6788 Acked Write 8 Sent Done Done false
20290 6789 Acked Read 7 Sent Done Done false
20291 6790 Acked Read 1 Sent Done Done false
20292 6791 Acked Write 8 Sent Done Done false
20293 6792 Acked Read 8 Sent Done Done false
20294 6793 Acked Read 9 Sent Done Done false
20295 6794 Acked Flush 0 Sent Done Done false
20296 6795 Acked Write 4 Sent Done Done false
20297 6796 Acked Read 5 Sent Done Done false
20298 6797 Acked Write 10 Sent Done Done false
20299 6798 Acked Read 1 Sent Done Done false
20300 6799 Acked Read 1 Sent Done Done false
20301 6800 Acked Read 8 Sent Done Done false
20302 6801 Acked Write 10 Sent Done Done false
20303 6802 Acked Read 9 Sent Done Done false
20304 6803 Acked Read 5 Sent Done Done false
20305 6804 Acked Read 2 Sent Done Done false
20306 6805 Acked Read 6 Sent Done Done false
20307 6806 Acked Read 5 Sent Done Done false
20308 6807 Acked Write 3 Sent Done Done false
20309 6808 Acked Read 10 Sent Done Done false
20310 6809 Acked Read 7 Sent Done Done false
20311 6810 Acked Flush 0 Sent Done Done false
20312 6811 Acked Write 3 Sent Done Done false
20313 6812 Acked Read 10 Sent Done Done false
20314 6813 Acked Read 2 Sent Done Done false
20315 6814 Acked Write 6 Sent Done Done false
20316 6815 Acked Write 9 Sent Done Done false
20317 6816 Acked Read 1 Sent Done Done false
20318 6817 Acked Read 8 Sent Done Done false
20319 6818 Acked Write 1 Sent Done Done false
20320 6819 Acked Write 1 Sent Done Done false
20321 6820 Acked Write 6 Sent Done Done false
20322 6821 Acked Write 2 Sent Done Done false
20323 6822 Acked Read 6 Sent Done Done false
20324 6823 Acked Write 2 Sent Done Done false
20325 6824 Acked Read 3 Sent Done Done false
20326 6825 Acked Write 5 Sent Done Done false
20327 6826 Acked Read 1 Sent Done Done false
20328 6827 Acked Flush 0 Sent Done Done false
20329 6828 Acked Write 3 Sent Done Done false
20330 6829 Acked Read 5 Sent Done Done false
20331 6830 Acked Write 7 Sent Done Done false
20332 6831 Acked Read 8 Sent Done Done false
20333 6832 Acked Flush 0 Sent Done Done false
20334 6833 Acked Write 6 Sent Done Done false
20335 6834 Acked Write 6 Sent Done Done false
20336 6835 Acked Write 7 Sent Done Done false
20337 6836 Acked Read 9 Sent Done Done false
20338 6837 Acked Read 7 Sent Done Done false
20339 6838 Acked Write 7 Sent Done Done false
20340 6839 Acked Write 2 Sent Done Done false
20341 6840 Acked Read 6 Sent Done Done false
20342 6841 Acked Read 9 Sent Done Done false
20343 6842 Acked Flush 0 Sent Done Done false
20344 6843 Acked Read 8 Sent Done Done false
20345 6844 Acked Write 7 Sent Done Done false
20346 6845 Acked Read 8 Sent Done Done false
20347 6846 Acked Write 5 Sent Done Done false
20348 6847 Acked Write 5 Sent Done Done false
20349 6848 Acked Read 6 Sent Done Done false
20350 6849 Acked Write 4 Sent Done Done false
20351 6850 Acked Write 4 Sent Done Done false
20352 6851 Acked Read 8 Sent Done Done false
20353 6852 Acked Write 5 Sent Done Done false
20354 6853 Acked Write 3 Sent Done Done false
20355 6854 Acked Flush 0 Sent Done Done false
20356 6855 Acked Write 6 Sent Done Done false
20357 6856 Acked Read 3 Sent Done Done false
20358 6857 Acked Read 6 Sent Done Done false
20359 6858 Acked Flush 0 Sent Done Done false
20360 6859 Acked Read 10 Sent Done Done false
20361 6860 Acked Write 5 Sent Done Done false
20362 6861 Acked Read 6 Sent Done Done false
20363 6862 Acked Write 8 Sent Done Done false
20364 6863 Acked Write 3 Sent Done Done false
20365 6864 Acked Write 3 Sent Done Done false
20366 6865 Acked Write 2 Sent Done Done false
20367 6866 Acked Write 3 Sent Done Done false
20368 6867 Acked Flush 0 Sent Done Done false
20369 6868 Acked Flush 0 Sent Done Done false
20370 6869 Acked Read 5 Sent Done Done false
20371 6870 Acked Write 4 Sent Done Done false
20372 6871 Acked Read 8 Sent Done Done false
20373 6872 Acked Write 7 Sent Done Done false
20374 6873 Acked Write 5 Sent Done Done false
20375 6874 Acked Write 9 Sent Done Done false
20376 6875 Acked Write 4 Sent Done Done false
20377 6876 Acked Read 4 Sent Done Done false
20378 6877 Acked Write 5 Sent Done Done false
20379 6878 Acked Write 5 Sent Done Done false
20380 6879 Acked Read 7 Sent Done Done false
20381 6880 Acked Write 6 Sent Done Done false
20382 6881 Acked Write 4 Sent Done Done false
20383 6882 Acked Flush 0 Sent Done Done false
20384 6883 Acked Flush 0 Sent Done Done false
20385 6884 Acked Read 9 Sent Done Done false
20386 6885 Acked Write 1 Sent Done Done false
20387 6886 Acked Write 4 Sent Done Done false
20388 6887 Acked Read 9 Sent Done Done false
20389 6888 Acked Write 1 Sent Done Done false
20390 6889 Acked Read 5 Sent Done Done false
20391 6890 Acked Write 1 Sent Done Done false
20392 6891 Acked Read 4 Sent Done Done false
20393 6892 Acked Write 7 Sent Done Done false
20394 6893 Acked Read 2 Sent Done Done false
20395 6894 Acked Write 5 Sent Done Done false
20396 6895 Acked Read 3 Sent Done Done false
20397 6896 Acked Read 2 Sent Done Done false
20398 6897 Acked Read 9 Sent Done Done false
20399 6898 Acked Read 10 Sent Done Done false
20400 6899 Acked Read 6 Sent Done Done false
20401 6900 Acked Read 7 Sent Done Done false
20402 6901 Acked Write 1 Sent Done Done false
20403 6902 Acked Read 9 Sent Done Done false
20404 6903 Acked Read 9 Sent Done Done false
20405 6904 Acked Read 10 Sent Done Done false
20406 6905 Acked Read 8 Sent Done Done false
20407 6906 Acked Write 4 Sent Done Done false
20408 6907 Acked Write 4 Sent Done Done false
20409 6908 Acked Read 1 Sent Done Done false
20410 6909 Acked Write 4 Sent Done Done false
20411 6910 Acked Read 7 Sent Done Done false
20412 6911 Acked Write 6 Sent Done Done false
20413 6912 Acked Read 9 Sent Done Done false
20414 6913 Acked Read 6 Sent Done Done false
20415 6914 Acked Write 7 Sent Done Done false
20416 6915 Acked Write 7 Sent Done Done false
20417 6916 Acked Write 4 Sent Done Done false
20418 6917 Acked Write 5 Sent Done Done false
20419 6918 Acked Flush 0 Sent Done Done false
20420 6919 Acked Read 1 Sent Done Done false
20421 6920 Acked Write 10 Sent Done Done false
20422 6921 Acked Flush 0 Sent Done Done false
20423 6922 Acked Read 9 Sent Done Done false
20424 6923 Acked Write 6 Sent Done Done false
20425 6924 Acked Read 4 Sent Done Done false
20426 6925 Acked Read 4 Sent Done Done false
20427 6926 Acked Read 9 Sent Done Done false
20428 6927 Acked Read 4 Sent Done Done false
20429 6928 Acked Read 1 Sent Done Done false
20430 6929 Acked Write 6 Sent Done Done false
20431 6930 Acked Write 8 Sent Done Done false
20432 6931 Acked Write 1 Sent Done Done false
20433 6932 Acked Write 1 Sent Done Done false
20434 6933 Acked Write 10 Sent Done Done false
20435 6934 Acked Write 8 Sent Done Done false
20436 6935 Acked Write 5 Sent Done Done false
20437 6936 Acked Write 4 Sent Done Done false
20438 6937 Acked Write 7 Sent Done Done false
20439 6938 Acked Write 10 Sent Done Done false
20440 6939 Acked Write 10 Sent Done Done false
20441 6940 Acked Write 7 Sent Done Done false
20442 6941 Acked Read 5 Sent Done Done false
20443 6942 Acked Read 9 Sent Done Done false
20444 6943 Acked Read 6 Sent Done Done false
20445 6944 Acked Write 10 Sent Done Done false
20446 6945 Acked Write 4 Sent Done Done false
20447 6946 Acked Write 5 Sent Done Done false
20448 6947 Acked Read 1 Sent Done Done false
20449 6948 Acked Write 7 Sent Done Done false
20450 6949 Acked Read 7 Sent Done Done false
20451 6950 Acked Read 8 Sent Done Done false
20452 6951 Acked Write 5 Sent Done Done false
20453 6952 Acked Read 9 Sent Done Done false
20454 6953 Acked Write 2 Sent Done Done false
20455 6954 Acked Read 1 Sent Done Done false
20456 6955 Acked Read 3 Sent Done Done false
20457 6956 Acked Write 2 Sent Done Done false
20458 6957 Acked Read 6 Sent Done Done false
20459 6958 Acked Read 8 Sent Done Done false
20460 6959 Acked Read 2 Sent Done Done false
20461 6960 Acked Read 6 Sent Done Done false
20462 6961 Acked Write 7 Sent Done Done false
20463 6962 Acked Flush 0 Sent Done Done false
20464 6963 Acked Read 10 Sent Done Done false
20465 6964 Acked Write 10 Sent Done Done false
20466 6965 Acked Read 7 Sent Done Done false
20467 6966 Acked Read 6 Sent Done Done false
20468 6967 Acked Read 8 Sent Done Done false
20469 6968 Acked Write 2 Sent Done Done false
20470 6969 Acked Write 5 Sent Done Done false
20471 6970 Acked Read 1 Sent Done Done false
20472 6971 Acked Read 9 Sent Done Done false
20473 6972 Acked Read 1 Sent Done Done false
20474 6973 Acked Flush 0 Sent Done Done false
20475 6974 Acked Read 6 Sent Done Done false
20476 6975 Acked Write 10 Sent Done Done false
20477 6976 Acked Write 8 Sent Done Done false
20478 6977 Acked Read 7 Sent Done Done false
20479 6978 Acked Flush 0 Sent Done Done false
20480 6979 Acked Read 3 Sent Done Done false
20481 6980 Acked Write 6 Sent Done Done false
20482 6981 Acked Flush 0 Sent Done Done false
20483 6982 Acked Read 2 Sent Done Done false
20484 6983 Acked Read 7 Sent Done Done false
20485 6984 Acked Read 9 Sent Done Done false
20486 6985 Acked Flush 0 Sent Done Done false
20487 6986 Acked Read 6 Sent Done Done false
20488 6987 Acked Read 4 Sent Done Done false
20489 6988 Acked Read 8 Sent Done Done false
20490 6989 Acked Read 1 Sent Done Done false
20491 6990 Acked Write 10 Sent Done Done false
20492 6991 Acked Write 9 Sent Done Done false
20493 6992 Acked Write 9 Sent Done Done false
20494 6993 Acked Write 7 Sent Done Done false
20495 6994 Acked Write 7 Sent Done Done false
20496 6995 Acked Read 9 Sent Done Done false
20497 6996 Acked Write 8 Sent Done Done false
20498 6997 Acked Write 8 Sent Done Done false
20499 6998 Acked Write 7 Sent Done Done false
20500 6999 Acked Flush 0 Sent Done Done false
20501 7000 Acked Read 7 Sent Done Done false
20502 7001 Acked Read 10 Sent Done Done false
20503 7002 Acked Write 5 Sent Done Done false
20504 7003 Acked Read 2 Sent Done Done false
20505 7004 Acked Flush 0 Sent Done Done false
20506 7005 Acked Write 5 Sent Done Done false
20507 7006 Acked Write 9 Sent Done Done false
20508 7007 Acked Read 4 Sent Done Done false
20509 7008 Acked Write 8 Sent Done Done false
20510 7009 Acked Read 3 Sent Done Done false
20511 7010 Acked Write 5 Sent Done Done false
20512 7011 Acked Read 5 Sent Done Done false
20513 7012 Acked Read 6 Sent Done Done false
20514 7013 Acked Read 9 Sent Done Done false
20515 7014 Acked Read 5 Sent Done Done false
20516 7015 Acked Read 7 Sent Done Done false
20517 7016 Acked Write 2 Sent Done Done false
20518 7017 Acked Read 1 Sent Done Done false
20519 7018 Acked Flush 0 Sent Done Done false
20520 7019 Acked Read 10 Sent Done Done false
20521 7020 Acked Write 1 Sent Done Done false
20522 7021 Acked Flush 0 Sent Done Done false
20523 7022 Acked Read 2 Sent Done Done false
20524 7023 Acked Read 1 Sent Done Done false
20525 7024 Acked Read 2 Sent Done Done false
20526 7025 Acked Write 7 Sent Done Done false
20527 7026 Acked Read 3 Sent Done Done false
20528 7027 Acked Read 9 Sent Done Done false
20529 7028 Acked Read 8 Sent Done Done false
20530 7029 Acked Write 3 Sent Done Done false
20531 7030 Acked Write 10 Sent Done Done false
20532 7031 Acked Write 10 Sent Done Done false
20533 7032 Acked Read 7 Sent Done Done false
20534 7033 Acked Write 5 Sent Done Done false
20535 7034 Acked Read 5 Sent Done Done false
20536 7035 Acked Write 4 Sent Done Done false
20537 STATES DS:0 DS:1 DS:2 TOTAL
20538 Sent 301 0 0 301
20539 Done 0 301 301 602
20540 Skipped 0 0 0 0
20541 Error 0 0 0 0
20542 Last Flush: 6734 7021 7021
20543 Downstairs last five completed: 6734 6733 6732 6731 6730
20544 Upstairs last five completed: 7035 7034 7033 7032 7031
20545 CLIENT: Up:0 ds:301 act:2
20546 [8810] got start action
20547 [8810] I am going to restart
20548 Starting downstairs at port 8810
20549 Make output file at "/tmp/dsc/downstairs-8810.txt"
20550 [8810][0] reports Starting
20551 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1222)
20552 [8810][0] reports Running
20553 ----------------------------------------------------------------
20554 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20555 States: Offline Active Active
20556 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20557 6735 Acked Write 8 Sent Done Done false
20558 6736 Acked Read 9 Sent Done Done false
20559 6737 Acked Read 3 Sent Done Done false
20560 6738 Acked Read 7 Sent Done Done false
20561 6739 Acked Write 3 Sent Done Done false
20562 6740 Acked Read 9 Sent Done Done false
20563 6741 Acked Write 10 Sent Done Done false
20564 6742 Acked Write 5 Sent Done Done false
20565 6743 Acked Write 7 Sent Done Done false
20566 6744 Acked Flush 0 Sent Done Done false
20567 6745 Acked Read 2 Sent Done Done false
20568 6746 Acked Flush 0 Sent Done Done false
20569 6747 Acked Read 7 Sent Done Done false
20570 6748 Acked Read 5 Sent Done Done false
20571 6749 Acked Write 9 Sent Done Done false
20572 6750 Acked Write 5 Sent Done Done false
20573 6751 Acked Read 6 Sent Done Done false
20574 6752 Acked Write 1 Sent Done Done false
20575 6753 Acked Write 8 Sent Done Done false
20576 6754 Acked Write 8 Sent Done Done false
20577 6755 Acked Write 1 Sent Done Done false
20578 6756 Acked Read 10 Sent Done Done false
20579 6757 Acked Read 4 Sent Done Done false
20580 6758 Acked Write 5 Sent Done Done false
20581 6759 Acked Read 5 Sent Done Done false
20582 6760 Acked Read 1 Sent Done Done false
20583 6761 Acked Read 7 Sent Done Done false
20584 6762 Acked Read 8 Sent Done Done false
20585 6763 Acked Read 1 Sent Done Done false
20586 6764 Acked Read 5 Sent Done Done false
20587 6765 Acked Read 2 Sent Done Done false
20588 6766 Acked Read 9 Sent Done Done false
20589 6767 Acked Write 9 Sent Done Done false
20590 6768 Acked Read 4 Sent Done Done false
20591 6769 Acked Read 8 Sent Done Done false
20592 6770 Acked Read 3 Sent Done Done false
20593 6771 Acked Write 2 Sent Done Done false
20594 6772 Acked Read 8 Sent Done Done false
20595 6773 Acked Read 4 Sent Done Done false
20596 6774 Acked Write 7 Sent Done Done false
20597 6775 Acked Read 10 Sent Done Done false
20598 6776 Acked Read 5 Sent Done Done false
20599 6777 Acked Read 6 Sent Done Done false
20600 6778 Acked Write 3 Sent Done Done false
20601 6779 Acked Write 10 Sent Done Done false
20602 6780 Acked Read 6 Sent Done Done false
20603 6781 Acked Read 10 Sent Done Done false
20604 6782 Acked Read 5 Sent Done Done false
20605 6783 Acked Read 6 Sent Done Done false
20606 6784 Acked Read 3 Sent Done Done false
20607 6785 Acked Write 9 Sent Done Done false
20608 6786 Acked Write 10 Sent Done Done false
20609 6787 Acked Write 3 Sent Done Done false
20610 6788 Acked Write 8 Sent Done Done false
20611 6789 Acked Read 7 Sent Done Done false
20612 6790 Acked Read 1 Sent Done Done false
20613 6791 Acked Write 8 Sent Done Done false
20614 6792 Acked Read 8 Sent Done Done false
20615 6793 Acked Read 9 Sent Done Done false
20616 6794 Acked Flush 0 Sent Done Done false
20617 6795 Acked Write 4 Sent Done Done false
20618 6796 Acked Read 5 Sent Done Done false
20619 6797 Acked Write 10 Sent Done Done false
20620 6798 Acked Read 1 Sent Done Done false
20621 6799 Acked Read 1 Sent Done Done false
20622 6800 Acked Read 8 Sent Done Done false
20623 6801 Acked Write 10 Sent Done Done false
20624 6802 Acked Read 9 Sent Done Done false
20625 6803 Acked Read 5 Sent Done Done false
20626 6804 Acked Read 2 Sent Done Done false
20627 6805 Acked Read 6 Sent Done Done false
20628 6806 Acked Read 5 Sent Done Done false
20629 6807 Acked Write 3 Sent Done Done false
20630 6808 Acked Read 10 Sent Done Done false
20631 6809 Acked Read 7 Sent Done Done false
20632 6810 Acked Flush 0 Sent Done Done false
20633 6811 Acked Write 3 Sent Done Done false
20634 6812 Acked Read 10 Sent Done Done false
20635 6813 Acked Read 2 Sent Done Done false
20636 6814 Acked Write 6 Sent Done Done false
20637 6815 Acked Write 9 Sent Done Done false
20638 6816 Acked Read 1 Sent Done Done false
20639 6817 Acked Read 8 Sent Done Done false
20640 6818 Acked Write 1 Sent Done Done false
20641 6819 Acked Write 1 Sent Done Done false
20642 6820 Acked Write 6 Sent Done Done false
20643 6821 Acked Write 2 Sent Done Done false
20644 6822 Acked Read 6 Sent Done Done false
20645 6823 Acked Write 2 Sent Done Done false
20646 6824 Acked Read 3 Sent Done Done false
20647 6825 Acked Write 5 Sent Done Done false
20648 6826 Acked Read 1 Sent Done Done false
20649 6827 Acked Flush 0 Sent Done Done false
20650 6828 Acked Write 3 Sent Done Done false
20651 6829 Acked Read 5 Sent Done Done false
20652 6830 Acked Write 7 Sent Done Done false
20653 6831 Acked Read 8 Sent Done Done false
20654 6832 Acked Flush 0 Sent Done Done false
20655 6833 Acked Write 6 Sent Done Done false
20656 6834 Acked Write 6 Sent Done Done false
20657 6835 Acked Write 7 Sent Done Done false
20658 6836 Acked Read 9 Sent Done Done false
20659 6837 Acked Read 7 Sent Done Done false
20660 6838 Acked Write 7 Sent Done Done false
20661 6839 Acked Write 2 Sent Done Done false
20662 6840 Acked Read 6 Sent Done Done false
20663 6841 Acked Read 9 Sent Done Done false
20664 6842 Acked Flush 0 Sent Done Done false
20665 6843 Acked Read 8 Sent Done Done false
20666 6844 Acked Write 7 Sent Done Done false
20667 6845 Acked Read 8 Sent Done Done false
20668 6846 Acked Write 5 Sent Done Done false
20669 6847 Acked Write 5 Sent Done Done false
20670 6848 Acked Read 6 Sent Done Done false
20671 6849 Acked Write 4 Sent Done Done false
20672 6850 Acked Write 4 Sent Done Done false
20673 6851 Acked Read 8 Sent Done Done false
20674 6852 Acked Write 5 Sent Done Done false
20675 6853 Acked Write 3 Sent Done Done false
20676 6854 Acked Flush 0 Sent Done Done false
20677 6855 Acked Write 6 Sent Done Done false
20678 6856 Acked Read 3 Sent Done Done false
20679 6857 Acked Read 6 Sent Done Done false
20680 6858 Acked Flush 0 Sent Done Done false
20681 6859 Acked Read 10 Sent Done Done false
20682 6860 Acked Write 5 Sent Done Done false
20683 6861 Acked Read 6 Sent Done Done false
20684 6862 Acked Write 8 Sent Done Done false
20685 6863 Acked Write 3 Sent Done Done false
20686 6864 Acked Write 3 Sent Done Done false
20687 6865 Acked Write 2 Sent Done Done false
20688 6866 Acked Write 3 Sent Done Done false
20689 6867 Acked Flush 0 Sent Done Done false
20690 6868 Acked Flush 0 Sent Done Done false
20691 6869 Acked Read 5 Sent Done Done false
20692 6870 Acked Write 4 Sent Done Done false
20693 6871 Acked Read 8 Sent Done Done false
20694 6872 Acked Write 7 Sent Done Done false
20695 6873 Acked Write 5 Sent Done Done false
20696 6874 Acked Write 9 Sent Done Done false
20697 6875 Acked Write 4 Sent Done Done false
20698 6876 Acked Read 4 Sent Done Done false
20699 6877 Acked Write 5 Sent Done Done false
20700 6878 Acked Write 5 Sent Done Done false
20701 6879 Acked Read 7 Sent Done Done false
20702 6880 Acked Write 6 Sent Done Done false
20703 6881 Acked Write 4 Sent Done Done false
20704 6882 Acked Flush 0 Sent Done Done false
20705 6883 Acked Flush 0 Sent Done Done false
20706 6884 Acked Read 9 Sent Done Done false
20707 6885 Acked Write 1 Sent Done Done false
20708 6886 Acked Write 4 Sent Done Done false
20709 6887 Acked Read 9 Sent Done Done false
20710 6888 Acked Write 1 Sent Done Done false
20711 6889 Acked Read 5 Sent Done Done false
20712 6890 Acked Write 1 Sent Done Done false
20713 6891 Acked Read 4 Sent Done Done false
20714 6892 Acked Write 7 Sent Done Done false
20715 6893 Acked Read 2 Sent Done Done false
20716 6894 Acked Write 5 Sent Done Done false
20717 6895 Acked Read 3 Sent Done Done false
20718 6896 Acked Read 2 Sent Done Done false
20719 6897 Acked Read 9 Sent Done Done false
20720 6898 Acked Read 10 Sent Done Done false
20721 6899 Acked Read 6 Sent Done Done false
20722 6900 Acked Read 7 Sent Done Done false
20723 6901 Acked Write 1 Sent Done Done false
20724 6902 Acked Read 9 Sent Done Done false
20725 6903 Acked Read 9 Sent Done Done false
20726 6904 Acked Read 10 Sent Done Done false
20727 6905 Acked Read 8 Sent Done Done false
20728 6906 Acked Write 4 Sent Done Done false
20729 6907 Acked Write 4 Sent Done Done false
20730 6908 Acked Read 1 Sent Done Done false
20731 6909 Acked Write 4 Sent Done Done false
20732 6910 Acked Read 7 Sent Done Done false
20733 6911 Acked Write 6 Sent Done Done false
20734 6912 Acked Read 9 Sent Done Done false
20735 6913 Acked Read 6 Sent Done Done false
20736 6914 Acked Write 7 Sent Done Done false
20737 6915 Acked Write 7 Sent Done Done false
20738 6916 Acked Write 4 Sent Done Done false
20739 6917 Acked Write 5 Sent Done Done false
20740 6918 Acked Flush 0 Sent Done Done false
20741 6919 Acked Read 1 Sent Done Done false
20742 6920 Acked Write 10 Sent Done Done false
20743 6921 Acked Flush 0 Sent Done Done false
20744 6922 Acked Read 9 Sent Done Done false
20745 6923 Acked Write 6 Sent Done Done false
20746 6924 Acked Read 4 Sent Done Done false
20747 6925 Acked Read 4 Sent Done Done false
20748 6926 Acked Read 9 Sent Done Done false
20749 6927 Acked Read 4 Sent Done Done false
20750 6928 Acked Read 1 Sent Done Done false
20751 6929 Acked Write 6 Sent Done Done false
20752 6930 Acked Write 8 Sent Done Done false
20753 6931 Acked Write 1 Sent Done Done false
20754 6932 Acked Write 1 Sent Done Done false
20755 6933 Acked Write 10 Sent Done Done false
20756 6934 Acked Write 8 Sent Done Done false
20757 6935 Acked Write 5 Sent Done Done false
20758 6936 Acked Write 4 Sent Done Done false
20759 6937 Acked Write 7 Sent Done Done false
20760 6938 Acked Write 10 Sent Done Done false
20761 6939 Acked Write 10 Sent Done Done false
20762 6940 Acked Write 7 Sent Done Done false
20763 6941 Acked Read 5 Sent Done Done false
20764 6942 Acked Read 9 Sent Done Done false
20765 6943 Acked Read 6 Sent Done Done false
20766 6944 Acked Write 10 Sent Done Done false
20767 6945 Acked Write 4 Sent Done Done false
20768 6946 Acked Write 5 Sent Done Done false
20769 6947 Acked Read 1 Sent Done Done false
20770 6948 Acked Write 7 Sent Done Done false
20771 6949 Acked Read 7 Sent Done Done false
20772 6950 Acked Read 8 Sent Done Done false
20773 6951 Acked Write 5 Sent Done Done false
20774 6952 Acked Read 9 Sent Done Done false
20775 6953 Acked Write 2 Sent Done Done false
20776 6954 Acked Read 1 Sent Done Done false
20777 6955 Acked Read 3 Sent Done Done false
20778 6956 Acked Write 2 Sent Done Done false
20779 6957 Acked Read 6 Sent Done Done false
20780 6958 Acked Read 8 Sent Done Done false
20781 6959 Acked Read 2 Sent Done Done false
20782 6960 Acked Read 6 Sent Done Done false
20783 6961 Acked Write 7 Sent Done Done false
20784 6962 Acked Flush 0 Sent Done Done false
20785 6963 Acked Read 10 Sent Done Done false
20786 6964 Acked Write 10 Sent Done Done false
20787 6965 Acked Read 7 Sent Done Done false
20788 6966 Acked Read 6 Sent Done Done false
20789 6967 Acked Read 8 Sent Done Done false
20790 6968 Acked Write 2 Sent Done Done false
20791 6969 Acked Write 5 Sent Done Done false
20792 6970 Acked Read 1 Sent Done Done false
20793 6971 Acked Read 9 Sent Done Done false
20794 6972 Acked Read 1 Sent Done Done false
20795 6973 Acked Flush 0 Sent Done Done false
20796 6974 Acked Read 6 Sent Done Done false
20797 6975 Acked Write 10 Sent Done Done false
20798 6976 Acked Write 8 Sent Done Done false
20799 6977 Acked Read 7 Sent Done Done false
20800 6978 Acked Flush 0 Sent Done Done false
20801 6979 Acked Read 3 Sent Done Done false
20802 6980 Acked Write 6 Sent Done Done false
20803 6981 Acked Flush 0 Sent Done Done false
20804 6982 Acked Read 2 Sent Done Done false
20805 6983 Acked Read 7 Sent Done Done false
20806 6984 Acked Read 9 Sent Done Done false
20807 6985 Acked Flush 0 Sent Done Done false
20808 6986 Acked Read 6 Sent Done Done false
20809 6987 Acked Read 4 Sent Done Done false
20810 6988 Acked Read 8 Sent Done Done false
20811 6989 Acked Read 1 Sent Done Done false
20812 6990 Acked Write 10 Sent Done Done false
20813 6991 Acked Write 9 Sent Done Done false
20814 6992 Acked Write 9 Sent Done Done false
20815 6993 Acked Write 7 Sent Done Done false
20816 6994 Acked Write 7 Sent Done Done false
20817 6995 Acked Read 9 Sent Done Done false
20818 6996 Acked Write 8 Sent Done Done false
20819 6997 Acked Write 8 Sent Done Done false
20820 6998 Acked Write 7 Sent Done Done false
20821 6999 Acked Flush 0 Sent Done Done false
20822 7000 Acked Read 7 Sent Done Done false
20823 7001 Acked Read 10 Sent Done Done false
20824 7002 Acked Write 5 Sent Done Done false
20825 7003 Acked Read 2 Sent Done Done false
20826 7004 Acked Flush 0 Sent Done Done false
20827 7005 Acked Write 5 Sent Done Done false
20828 7006 Acked Write 9 Sent Done Done false
20829 7007 Acked Read 4 Sent Done Done false
20830 7008 Acked Write 8 Sent Done Done false
20831 7009 Acked Read 3 Sent Done Done false
20832 7010 Acked Write 5 Sent Done Done false
20833 7011 Acked Read 5 Sent Done Done false
20834 7012 Acked Read 6 Sent Done Done false
20835 7013 Acked Read 9 Sent Done Done false
20836 7014 Acked Read 5 Sent Done Done false
20837 7015 Acked Read 7 Sent Done Done false
20838 7016 Acked Write 2 Sent Done Done false
20839 7017 Acked Read 1 Sent Done Done false
20840 7018 Acked Flush 0 Sent Done Done false
20841 7019 Acked Read 10 Sent Done Done false
20842 7020 Acked Write 1 Sent Done Done false
20843 7021 Acked Flush 0 Sent Done Done false
20844 7022 Acked Read 2 Sent Done Done false
20845 7023 Acked Read 1 Sent Done Done false
20846 7024 Acked Read 2 Sent Done Done false
20847 7025 Acked Write 7 Sent Done Done false
20848 7026 Acked Read 3 Sent Done Done false
20849 7027 Acked Read 9 Sent Done Done false
20850 7028 Acked Read 8 Sent Done Done false
20851 7029 Acked Write 3 Sent Done Done false
20852 7030 Acked Write 10 Sent Done Done false
20853 7031 Acked Write 10 Sent Done Done false
20854 7032 Acked Read 7 Sent Done Done false
20855 7033 Acked Write 5 Sent Done Done false
20856 7034 Acked Read 5 Sent Done Done false
20857 7035 Acked Write 4 Sent Done Done false
20858 7036 Acked Flush 0 Sent Done Done false
20859 STATES DS:0 DS:1 DS:2 TOTAL
20860 Sent 302 0 0 302
20861 Done 0 302 302 604
20862 Skipped 0 0 0 0
20863 Error 0 0 0 0
20864 Last Flush: 6734 7036 7036
20865 Downstairs last five completed: 6734 6733 6732 6731 6730
20866 Upstairs last five completed: 7036 7035 7034 7033 7032
20867 CLIENT: Up:0 ds:302 act:2
208682025-01-14T16:14:58.559ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
208692025-01-14T16:14:58.559ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54636) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
208702025-01-14T16:14:58.560ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
208712025-01-14T16:14:58.560ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
208722025-01-14T16:14:58.560ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
208732025-01-14T16:14:58.560ZINFOcrucible: send last flush ID to this DS: 6734 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
208742025-01-14T16:14:58.561ZINFOcrucible: Replied this last flush ID: 6734 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
208752025-01-14T16:14:58.561ZINFOcrucible: [0] client re-new 302 jobs since flush 6734 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
208762025-01-14T16:14:58.561ZINFOcrucible: [0] Replayed 302 jobs since flush: 6734 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
20877 ----------------------------------------------------------------
20878 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
20879 Downstairs last five completed: 7036 7035 7034 7033 7032
20880 Upstairs last five completed: 7036 7035 7034 7033 7032
20881 CLIENT: Up:0 ds:0 act:3
20882 Replay: All jobs finished
20883 Main task has work to do, go find it
20884 got dsc Stop(2)
20885 stop 2
20886 [8830] Got stop action so:false kr:false
20887 Jan 14 16:15:00.803 INFO request completed, latency_us: 322, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 524e68e5-4e86-41d0-ab8f-688c03685821, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
208882025-01-14T16:15:00.804ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
20889 Jan 14 16:15:00.804 INFO request completed, latency_us: 139, response_code:{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-14T16:15:00.804634749Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
20890 200, uri: /state/cid/2, method: GET, req_id: 62a7e9c6-8289-484c-a02e-ec6ba9590c63, remote_addr:{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-14T16:15:00.804661925Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
20891 127.0.0.1:61942, local_addr: 127.0.0.1:9998
208922025-01-14T16:15:00.804ZINFOcrucible: client task is exiting = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
20893 [8830] Exited with: ExitStatus(unix_wait_status(9))
20894 [8830][2] reports Exit
20895 Jan 14 16:15:04.806 INFO request completed, latency_us: 184, response_code: 200, uri: /state/cid/2, method: GET, req_id: f4d7dc68-b4ea-4f2d-823a-d464676c808b, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
20896 001/300 Write block 4410 len 36864 data: 4 4 5 5 4 5 6 7 6
20897 002/300 Flush
20898 003/300 Read block 3068 len 12288
20899 004/300 Write block 3208 len 36864 data: 5 5 4 4 3 3 3 4 3
20900 005/300 Read block 671 len 16384
20901 006/300 Read block 2851 len 36864
20902 007/300 Write block 2472 len 24576 data: 4 4 5 6 6 6
20903 008/300 Flush
20904 009/300 Write block 4125 len 16384 data: 8 7 7 8
20905 010/300 Write block 2547 len 32768 data: 5 7 6 6 5 5 5 5
20906 011/300 Write block 1494 len 16384 data: 5 6 6 5
20907 012/300 Read block 4146 len 40960
20908 013/300 Read block 3953 len 20480
20909 014/300 Write block 1057 len 24576 data: 3 4 4 5 5 4
20910 015/300 Write block 4941 len 16384 data: 3 3 6 7
20911 016/300 Read block 3514 len 36864
20912 017/300 Read block 1864 len 12288
20913 018/300 Read block 1691 len 4096
20914 019/300 Read block 4798 len 4096
20915 020/300 Flush
20916 021/300 Read block 1517 len 32768
20917 022/300 Read block 1216 len 40960
20918 023/300 Write block 3842 len 4096 data: 5
20919 024/300 Read block 3059 len 8192
20920 025/300 Flush
20921 026/300 Read block 485 len 16384
20922 027/300 Read block 4655 len 12288
20923 028/300 Write block 267 len 4096 data: 3
20924 029/300 Read block 87 len 40960
20925 030/300 Write block 4024 len 32768 data: 3 3 5 4 6 5 5 4
20926 031/300 Write block 1458 len 4096 data: 4
20927 032/300 Read block 2033 len 24576
20928 033/300 Write block 4004 len 20480 data: 6 6 6 8 7
20929 034/300 Write block 4292 len 36864 data: 4 5 5 7 6 5 5 6 5
20930 035/300 Read block 4257 len 28672
20931 036/300 Read block 4605 len 40960
20932 037/300 Write block 2451 len 28672 data: 4 4 5 6 3 3 3
20933 038/300 Read block 1998 len 32768
20934 039/300 Read block 2544 len 36864
20935 040/300 Write block 4048 len 16384 data: 6 7 6 4
20936 041/300 Read block 1367 len 32768
20937 042/300 Write block 2813 len 28672 data: 6 7 6 6 5 5 4
20938 043/300 Write block 1226 len 28672 data: 2 2 2 3 3 4 4
20939 044/300 Read block 1203 len 36864
20940 045/300 Write block 846 len 8192 data: 2 2
20941 046/300 Write block 4659 len 20480 data: 4 4 7 8 8
20942 047/300 Read block 3835 len 20480
20943 048/300 Read block 4632 len 28672
20944 049/300 Read block 2483 len 36864
20945 050/300 Read block 23 len 24576
20946 051/300 Read block 4337 len 12288
20947 052/300 Read block 2661 len 32768
20948 053/300 Flush
20949 054/300 Read block 3660 len 4096
20950 055/300 Write block 1254 len 12288 data: 4 4 4
20951 056/300 Read block 4239 len 20480
20952 057/300 Read block 4093 len 12288
20953 058/300 Read block 4273 len 36864
20954 059/300 Write block 2521 len 4096 data: 4
20955 060/300 Read block 760 len 4096
20956 061/300 Read block 4176 len 28672
20957 062/300 Write block 1769 len 8192 data: 5 5
20958 063/300 Write block 4551 len 8192 data: 2 2
20959 064/300 Write block 3562 len 40960 data: 3 3 3 3 3 4 4 5 5 4
20960 065/300 Read block 1 len 36864
20961 066/300 Write block 836 len 36864 data: 4 6 6 6 6 6 6 6 6
20962 067/300 Write block 2439 len 20480 data: 3 2 2 2 2
20963 068/300 Flush
20964 069/300 Read block 2902 len 40960
20965 070/300 Write block 1801 len 4096 data: 4
20966 071/300 Read block 4804 len 36864
20967 072/300 Read block 1518 len 4096
20968 073/300 Read block 1010 len 28672
20969 074/300 Write block 4418 len 32768 data: 7 6 7 6 6 7 7 7
20970 075/300 Flush
20971 076/300 Write block 4835 len 24576 data: 3 3 4 5 5 6
20972 077/300 Write block 1655 len 32768 data: 5 5 5 7 7 4 6 5
20973 078/300 Write block 2927 len 12288 data: 7 6 6
20974 079/300 Flush
20975 080/300 Flush
20976 081/300 Write block 3911 len 28672 data: 8 8 8 6 8 8 5
20977 082/300 Write block 4286 len 40960 data: 3 3 3 3 3 4 5 6 6 8
20978 083/300 Read block 2337 len 32768
20979 084/300 Read block 1813 len 12288
20980 085/300 Read block 4844 len 8192
20981 086/300 Write block 3533 len 20480 data: 5 5 6 6 7
20982 087/300 Flush
20983 088/300 Read block 4806 len 28672
20984 089/300 Write block 3567 len 20480 data: 5 5 6 6 5
20985 090/300 Read block 775 len 40960
20986 091/300 Read block 2405 len 8192
20987 092/300 Read block 790 len 16384
20988 093/300 Read block 399 len 20480
20989 094/300 Read block 2214 len 16384
20990 095/300 Write block 372 len 20480 data: 6 6 8 7 7
20991 096/300 Read block 1723 len 20480
20992 097/300 Flush
20993 098/300 Write block 3734 len 36864 data: 3 4 5 4 4 5 6 6 6
20994 099/300 Write block 28 len 36864 data: 8 7 6 5 5 5 3 4 4
20995 100/300 Write block 1351 len 8192 data: 4 5
20996 101/300 Flush
20997 102/300 Write block 4672 len 36864 data: 3 3 2 4 4 4 4 4 6
20998 103/300 Write block 4519 len 28672 data: 5 8 7 6 6 6 5
20999 104/300 Read block 2036 len 8192
21000 105/300 Read block 3042 len 40960
21001 106/300 Read block 4979 len 36864
21002 107/300 Write block 1769 len 28672 data: 6 6 5 6 5 5 4
21003 108/300 Read block 4239 len 16384
21004 109/300 Read block 4431 len 4096
21005 110/300 Write block 2341 len 24576 data: 6 5 6 6 4 2
21006 111/300 Write block 4820 len 12288 data: 3 3 4
21007 112/300 Write block 2189 len 32768 data: 5 6 6 7 7 8 7 6
21008 113/300 Read block 2088 len 32768
21009 114/300 Read block 3046 len 24576
21010 115/300 Flush
21011 116/300 Read block 3521 len 8192
21012 117/300 Read block 3225 len 36864
21013 118/300 Write block 2695 len 36864 data: 9 9 10 8 7 5 6 6 8
21014 119/300 Read block 4275 len 36864
21015 120/300 Read block 1400 len 24576
21016 121/300 Read block 2684 len 32768
21017 122/300 Write block 3217 len 36864 data: 3 3 3 5 5 4 4 4 4
21018 123/300 Read block 3732 len 40960
21019 124/300 Write block 358 len 36864 data: 2 2 2 2 2 3 2 2 2
21020 125/300 Read block 3041 len 24576
21021 126/300 Read block 118 len 40960
21022 127/300 Read block 2836 len 12288
21023 128/300 Flush
21024 129/300 Read block 2777 len 4096
21025 130/300 Flush
21026 131/300 Read block 4682 len 28672
21027 132/300 Write block 2013 len 32768 data: 2 2 4 4 4 4 5 5
21028 133/300 Write block 938 len 16384 data: 6 6 5 4
21029 134/300 Write block 2254 len 28672 data: 3 3 2 3 3 4 4
21030 135/300 Read block 4218 len 4096
21031 136/300 Read block 2008 len 8192
21032 137/300 Read block 240 len 28672
21033 138/300 Write block 1907 len 12288 data: 4 4 4
21034 139/300 Read block 668 len 28672
21035 140/300 Write block 2897 len 32768 data: 4 4 5 7 6 5 6 7
21036 141/300 Read block 2843 len 16384
21037 142/300 Read block 3465 len 40960
21038 143/300 Flush
21039 144/300 Read block 4585 len 8192
21040 145/300 Read block 2737 len 36864
21041 146/300 Read block 1605 len 20480
21042 147/300 Write block 4176 len 12288 data: 8 8 8
21043 148/300 Flush
21044 149/300 Read block 402 len 4096
21045 150/300 Read block 4707 len 4096
21046 151/300 Read block 3842 len 16384
21047 152/300 Read block 2890 len 32768
21048 153/300 Read block 278 len 32768
21049 154/300 Write block 4286 len 40960 data: 4 4 4 4 4 5 6 7 7 9
21050 155/300 Read block 2185 len 12288
21051 156/300 Write block 1916 len 28672 data: 2 2 3 3 3 3 4
21052 157/300 Write block 3718 len 40960 data: 5 3 5 4 5 4 4 4 5 6
21053 158/300 Write block 4917 len 36864 data: 6 5 5 6 6 5 4 3 4
21054 159/300 Write block 4665 len 4096 data: 11
21055 160/300 Read block 4380 len 28672
21056 161/300 Read block 1078 len 16384
21057 162/300 Write block 903 len 40960 data: 6 7 6 6 6 7 7 7 7 6
21058 163/300 Write block 4146 len 8192 data: 4 4
21059 164/300 Write block 4670 len 8192 data: 6 5
21060 165/300 Read block 3710 len 12288
21061 166/300 Write block 177 len 24576 data: 2 3 4 4 4 4
21062 167/300 Write block 4766 len 28672 data: 5 5 7 7 9 8 8
21063 168/300 Read block 3415 len 4096
21064 169/300 Read block 1909 len 20480
21065 170/300 Write block 1596 len 28672 data: 8 7 7 3 5 4 5
21066 171/300 Read block 2433 len 8192
21067 172/300 Flush
21068 173/300 Write block 784 len 8192 data: 3 4
21069 174/300 Write block 2735 len 24576 data: 4 4 4 4 4 2
21070 175/300 Read block 1776 len 12288
21071 176/300 Write block 4654 len 40960 data: 2 2 2 3 3 5 5 8 9 9
21072 177/300 Read block 1239 len 20480
21073 178/300 Read block 2480 len 36864
21074 179/300 Read block 2816 len 16384
21075 180/300 Write block 1996 len 12288 data: 10 9 8
21076 181/300 Read block 368 len 16384
21077 182/300 Read block 3363 len 32768
21078 183/300 Read block 1400 len 12288
21079 184/300 Read block 1484 len 8192
21080 185/300 Read block 2732 len 36864
21081 186/300 Read block 2033 len 24576
21082 187/300 Read block 2694 len 28672
21083 188/300 Read block 2958 len 24576
21084 189/300 Write block 4298 len 4096 data: 6
21085 190/300 Write block 2654 len 20480 data: 3 3 2 3 3
21086 191/300 Write block 3401 len 36864 data: 7 5 5 4 4 3 3 3 3
21087 192/300 Read block 2533 len 24576
21088 193/300 Read block 3149 len 8192
21089 194/300 Write block 3449 len 24576 data: 4 5 6 5 5 6
21090 195/300 Flush
21091 196/300 Write block 4259 len 40960 data: 4 3 4 4 5 5 5 5 4 4
21092 197/300 Write block 1518 len 28672 data: 3 3 3 5 5 6 8
21093 198/300 Read block 3812 len 4096
21094 199/300 Write block 4270 len 8192 data: 3 3
21095 200/300 Read block 2561 len 16384
21096 201/300 Read block 1400 len 4096
21097 202/300 Write block 3531 len 36864 data: 3 4 6 6 7 7 8 6 6
21098 203/300 Read block 1889 len 8192
21099 204/300 Flush
21100 205/300 Read block 1052 len 16384
21101 206/300 Write block 2506 len 28672 data: 5 4 5 4 5 4 5
21102 207/300 Write block 2429 len 20480 data: 4 3 4 5 5
21103 208/300 Write block 527 len 32768 data: 5 4 6 5 5 4 3 5
21104 209/300 Write block 4578 len 20480 data: 5 4 4 4 4
21105 210/300 Write block 4509 len 28672 data: 4 5 5 4 5 5 4
21106 211/300 Read block 4653 len 20480
21107 212/300 Read block 4794 len 12288
21108 213/300 Read block 223 len 4096
21109 214/300 Write block 751 len 28672 data: 4 4 4 2 3 4 5
21110 215/300 Read block 3969 len 36864
21111 216/300 Read block 260 len 16384
21112 217/300 Read block 3200 len 16384
21113 218/300 Read block 679 len 28672
21114 219/300 Read block 3517 len 24576
21115 220/300 Read block 3043 len 32768
21116 221/300 Write block 4182 len 36864 data: 3 3 3 2 2 2 4 4 4
21117 222/300 Read block 853 len 24576
21118 223/300 Flush
21119 224/300 Read block 329 len 32768
21120 225/300 Read block 1987 len 20480
21121 226/300 Write block 2497 len 24576 data: 3 2 2 3 3 4
21122 227/300 Write block 2643 len 32768 data: 6 6 6 5 5 5 5 6
21123 228/300 Flush
21124 229/300 Read block 878 len 24576
21125 230/300 Flush
21126 231/300 Write block 3946 len 4096 data: 4
21127 232/300 Flush
21128 233/300 Read block 4952 len 8192
21129 234/300 Flush
21130 235/300 Read block 3097 len 20480
21131 236/300 Flush
21132 237/300 Read block 3882 len 32768
21133 238/300 Write block 3843 len 12288 data: 5 7 7
21134 239/300 Read block 3831 len 40960
21135 240/300 Read block 974 len 16384
21136 241/300 Write block 2155 len 20480 data: 3 4 4 4 5
21137 242/300 Read block 106 len 24576
21138 243/300 Read block 857 len 8192
21139 244/300 Write block 1947 len 40960 data: 2 4 4 4 6 6 5 6 7 7
21140 245/300 Read block 4535 len 20480
21141 246/300 Read block 567 len 40960
21142 247/300 Flush
21143 248/300 Write block 2882 len 32768 data: 4 4 5 5 4 6 5 5
21144 249/300 Write block 1101 len 8192 data: 4 2
21145 250/300 Read block 3453 len 28672
21146 251/300 Flush
21147 252/300 Read block 3038 len 36864
21148 253/300 Write block 2089 len 28672 data: 5 5 5 5 4 5 5
21149 254/300 Read block 1759 len 36864
21150 255/300 Read block 1637 len 16384
21151 256/300 Read block 1351 len 36864
21152 257/300 Read block 1 len 36864
21153 258/300 Read block 1127 len 8192
21154 259/300 Read block 3178 len 24576
21155 260/300 Flush
21156 261/300 Write block 3010 len 40960 data: 3 2 2 2 2 4 5 6 5 5
21157 262/300 Read block 4795 len 8192
21158 263/300 Read block 611 len 32768
21159 264/300 Read block 2229 len 4096
21160 265/300 Write block 4798 len 24576 data: 4 3 3 3 3 3
21161 266/300 Read block 2233 len 4096
21162 267/300 Read block 2179 len 8192
21163 268/300 Write block 2932 len 4096 data: 4
21164 269/300 Read block 4751 len 16384
21165 270/300 Read block 1528 len 16384
21166 271/300 Flush
21167 272/300 Read block 2718 len 32768
21168 273/300 Write block 3525 len 20480 data: 4 3 3 3 3
21169 274/300 Write block 1878 len 28672 data: 4 4 4 4 4 3 3
21170 275/300 Read block 3161 len 12288
21171 276/300 Read block 4212 len 36864
21172 277/300 Read block 3954 len 32768
21173 278/300 Read block 1805 len 20480
21174 279/300 Write block 875 len 24576 data: 3 3 3 5 5 5
21175 280/300 Read block 979 len 40960
21176 281/300 Read block 3376 len 8192
21177 282/300 Read block 3937 len 32768
21178 283/300 Flush
21179 284/300 Read block 2143 len 24576
21180 285/300 Write block 3047 len 16384 data: 5 5 6 8
21181 286/300 Write block 2028 len 12288 data: 4 4 5
21182 287/300 Read block 1700 len 16384
21183 288/300 Read block 2502 len 12288
21184 289/300 Write block 4638 len 12288 data: 5 4 4
21185 290/300 Read block 4214 len 4096
21186 291/300 Write block 3149 len 28672 data: 7 7 6 6 5 6 7
21187 292/300 Write block 4568 len 8192 data: 5 4
21188 293/300 Flush
21189 294/300 Read block 198 len 4096
21190 295/300 Write block 399 len 40960 data: 4 4 5 4 5 7 5 4 4 5
21191 296/300 Read block 3286 len 32768
21192 297/300 Flush
21193 298/300 Write block 4223 len 32768 data: 5 4 6 5 3 4 4 3
21194 299/300 Write block 160 len 36864 data: 5 5 5 6 6 7 7 7 6
21195 300/300 Read block 4260 len 28672
21196 301/300 Read block 141 len 20480
21197 Main task has work to do, go find it
21198 got dsc Start(2)
21199 start 2
21200 Jan 14 16:15:05.047 INFO request completed, latency_us: 199, response_code: 204, uri: /start/cid/2, method: POST, req_id: 79ee86c0-6b15-438f-ac72-19912605c02a, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
21201 Replay: started 2, returned:Ok(())
21202 ----------------------------------------------------------------
21203 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
21204 States: Active Active Offline
21205 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21206 7037 Acked Write 9 Done Done Sent false
21207 7038 Acked Flush 0 Done Done Sent false
21208 7039 Acked Read 3 Done Done Sent false
21209 7040 Acked Write 9 Done Done Sent false
21210 7041 Acked Read 4 Done Done Sent false
21211 7042 Acked Read 9 Done Done Sent false
21212 7043 Acked Write 6 Done Done Sent false
21213 7044 Acked Flush 0 Done Done Sent false
21214 7045 Acked Write 4 Done Done Sent false
21215 7046 Acked Write 8 Done Done Sent false
21216 7047 Acked Write 4 Done Done Sent false
21217 7048 Acked Read 10 Done Done Sent false
21218 7049 Acked Read 5 Done Done Sent false
21219 7050 Acked Write 6 Done Done Sent false
21220 7051 Acked Write 4 Done Done Sent false
21221 7052 Acked Read 9 Done Done Sent false
21222 7053 Acked Read 3 Done Done Sent false
21223 7054 Acked Read 1 Done Done Sent false
21224 7055 Acked Read 1 Done Done Sent false
21225 7056 Acked Flush 0 Done Done Sent false
21226 7057 Acked Read 8 Done Done Sent false
21227 7058 Acked Read 10 Done Done Sent false
21228 7059 Acked Write 1 Done Done Sent false
21229 7060 Acked Read 2 Done Done Sent false
21230 7061 Acked Flush 0 Done Done Sent false
21231 7062 Acked Read 4 Done Done Sent false
21232 7063 Acked Read 3 Done Done Sent false
21233 7064 Acked Write 1 Done Done Sent false
21234 7065 Acked Read 10 Done Done Sent false
21235 7066 Acked Write 8 Done Done Sent false
21236 7067 Acked Write 1 Done Done Sent false
21237 7068 Acked Read 6 Done Done Sent false
21238 7069 Acked Write 5 Done Done Sent false
21239 7070 Acked Write 9 Done Done Sent false
21240 7071 Acked Read 7 Done Done Sent false
21241 7072 Acked Read 10 Done Done Sent false
21242 7073 Acked Write 7 Done Done Sent false
21243 7074 Acked Read 8 Done Done Sent false
21244 7075 Acked Read 9 Done Done Sent false
21245 7076 Acked Write 4 Done Done Sent false
21246 7077 Acked Read 8 Done Done Sent false
21247 7078 Acked Write 7 Done Done Sent false
21248 7079 Acked Write 7 Done Done Sent false
21249 7080 Acked Read 9 Done Done Sent false
21250 7081 Acked Write 2 Done Done Sent false
21251 7082 Acked Write 5 Done Done Sent false
21252 7083 Acked Read 5 Done Done Sent false
21253 7084 Acked Read 7 Done Done Sent false
21254 7085 Acked Read 9 Done Done Sent false
21255 7086 Acked Read 6 Done Done Sent false
21256 7087 Acked Read 3 Done Done Sent false
21257 7088 Acked Read 8 Done Done Sent false
21258 7089 Acked Flush 0 Done Done Sent false
21259 7090 Acked Read 1 Done Done Sent false
21260 7091 Acked Write 3 Done Done Sent false
21261 7092 Acked Read 5 Done Done Sent false
21262 7093 Acked Read 3 Done Done Sent false
21263 7094 Acked Read 9 Done Done Sent false
21264 7095 Acked Write 1 Done Done Sent false
21265 7096 Acked Read 1 Done Done Sent false
21266 7097 Acked Read 7 Done Done Sent false
21267 7098 Acked Write 2 Done Done Sent false
21268 7099 Acked Write 2 Done Done Sent false
21269 7100 Acked Write 10 Done Done Sent false
21270 7101 Acked Read 9 Done Done Sent false
21271 7102 Acked Write 9 Done Done Sent false
21272 7103 Acked Write 5 Done Done Sent false
21273 7104 Acked Flush 0 Done Done Sent false
21274 7105 Acked Read 10 Done Done Sent false
21275 7106 Acked Write 1 Done Done Sent false
21276 7107 Acked Read 9 Done Done Sent false
21277 7108 Acked Read 1 Done Done Sent false
21278 7109 Acked Read 7 Done Done Sent false
21279 7110 Acked Write 8 Done Done Sent false
21280 7111 Acked Flush 0 Done Done Sent false
21281 7112 Acked Write 6 Done Done Sent false
21282 7113 Acked Write 8 Done Done Sent false
21283 7114 Acked Write 3 Done Done Sent false
21284 7115 Acked Flush 0 Done Done Sent false
21285 7116 Acked Flush 0 Done Done Sent false
21286 7117 Acked Write 7 Done Done Sent false
21287 7118 Acked Write 10 Done Done Sent false
21288 7119 Acked Read 8 Done Done Sent false
21289 7120 Acked Read 3 Done Done Sent false
21290 7121 Acked Read 2 Done Done Sent false
21291 7122 Acked Write 5 Done Done Sent false
21292 7123 Acked Flush 0 Done Done Sent false
21293 7124 Acked Read 7 Done Done Sent false
21294 7125 Acked Write 5 Done Done Sent false
21295 7126 Acked Read 10 Done Done Sent false
21296 7127 Acked Read 2 Done Done Sent false
21297 7128 Acked Read 4 Done Done Sent false
21298 7129 Acked Read 5 Done Done Sent false
21299 7130 Acked Read 4 Done Done Sent false
21300 7131 Acked Write 5 Done Done Sent false
21301 7132 Acked Read 5 Done Done Sent false
21302 7133 Acked Flush 0 Done Done Sent false
21303 7134 Acked Write 9 Done Done Sent false
21304 7135 Acked Write 9 Done Done Sent false
21305 7136 Acked Write 2 Done Done Sent false
21306 7137 Acked Flush 0 Done Done Sent false
21307 7138 Acked Write 9 Done Done Sent false
21308 7139 Acked Write 7 Done Done Sent false
21309 7140 Acked Read 2 Done Done Sent false
21310 7141 Acked Read 10 Done Done Sent false
21311 7142 Acked Read 9 Done Done Sent false
21312 7143 Acked Write 7 Done Done Sent false
21313 7144 Acked Read 4 Done Done Sent false
21314 7145 Acked Read 1 Done Done Sent false
21315 7146 Acked Write 6 Done Done Sent false
21316 7147 Acked Write 3 Done Done Sent false
21317 7148 Acked Write 8 Done Done Sent false
21318 7149 Acked Read 8 Done Done Sent false
21319 7150 Acked Read 6 Done Done Sent false
21320 7151 Acked Flush 0 Done Done Sent false
21321 7152 Acked Read 2 Done Done Sent false
21322 7153 Acked Read 9 Done Done Sent false
21323 7154 Acked Write 9 Done Done Sent false
21324 7155 Acked Read 9 Done Done Sent false
21325 7156 Acked Read 6 Done Done Sent false
21326 7157 Acked Read 8 Done Done Sent false
21327 7158 Acked Write 9 Done Done Sent false
21328 7159 Acked Read 10 Done Done Sent false
21329 7160 Acked Write 9 Done Done Sent false
21330 7161 Acked Read 6 Done Done Sent false
21331 7162 Acked Read 10 Done Done Sent false
21332 7163 Acked Read 3 Done Done Sent false
21333 7164 Acked Flush 0 Done Done Sent false
21334 7165 Acked Read 1 Done Done Sent false
21335 7166 Acked Flush 0 Done Done Sent false
21336 7167 Acked Read 7 Done Done Sent false
21337 7168 Acked Write 8 Done Done Sent false
21338 7169 Acked Write 4 Done Done Sent false
21339 7170 Acked Write 7 Done Done Sent false
21340 7171 Acked Read 1 Done Done Sent false
21341 7172 Acked Read 2 Done Done Sent false
21342 7173 Acked Read 7 Done Done Sent false
21343 7174 Acked Write 3 Done Done Sent false
21344 7175 Acked Read 7 Done Done Sent false
21345 7176 Acked Write 8 Done Done Sent false
21346 7177 Acked Read 4 Done Done Sent false
21347 7178 Acked Read 10 Done Done Sent false
21348 7179 Acked Flush 0 Done Done Sent false
21349 7180 Acked Read 2 Done Done Sent false
21350 7181 Acked Read 9 Done Done Sent false
21351 7182 Acked Read 5 Done Done Sent false
21352 7183 Acked Write 3 Done Done Sent false
21353 7184 Acked Flush 0 Done Done Sent false
21354 7185 Acked Read 1 Done Done Sent false
21355 7186 Acked Read 1 Done Done Sent false
21356 7187 Acked Read 4 Done Done Sent false
21357 7188 Acked Read 8 Done Done Sent false
21358 7189 Acked Read 8 Done Done Sent false
21359 7190 Acked Write 10 Done Done Sent false
21360 7191 Acked Read 3 Done Done Sent false
21361 7192 Acked Write 7 Done Done Sent false
21362 7193 Acked Write 10 Done Done Sent false
21363 7194 Acked Write 9 Done Done Sent false
21364 7195 Acked Write 1 Done Done Sent false
21365 7196 Acked Read 7 Done Done Sent false
21366 7197 Acked Read 4 Done Done Sent false
21367 7198 Acked Write 10 Done Done Sent false
21368 7199 Acked Write 2 Done Done Sent false
21369 7200 Acked Write 2 Done Done Sent false
21370 7201 Acked Read 3 Done Done Sent false
21371 7202 Acked Write 6 Done Done Sent false
21372 7203 Acked Write 7 Done Done Sent false
21373 7204 Acked Read 1 Done Done Sent false
21374 7205 Acked Read 5 Done Done Sent false
21375 7206 Acked Write 7 Done Done Sent false
21376 7207 Acked Read 2 Done Done Sent false
21377 7208 Acked Flush 0 Done Done Sent false
21378 7209 Acked Write 2 Done Done Sent false
21379 7210 Acked Write 6 Done Done Sent false
21380 7211 Acked Read 3 Done Done Sent false
21381 7212 Acked Write 10 Done Done Sent false
21382 7213 Acked Read 5 Done Done Sent false
21383 7214 Acked Read 9 Done Done Sent false
21384 7215 Acked Read 4 Done Done Sent false
21385 7216 Acked Write 3 Done Done Sent false
21386 7217 Acked Read 4 Done Done Sent false
21387 7218 Acked Read 8 Done Done Sent false
21388 7219 Acked Read 3 Done Done Sent false
21389 7220 Acked Read 2 Done Done Sent false
21390 7221 Acked Read 9 Done Done Sent false
21391 7222 Acked Read 6 Done Done Sent false
21392 7223 Acked Read 7 Done Done Sent false
21393 7224 Acked Read 6 Done Done Sent false
21394 7225 Acked Write 1 Done Done Sent false
21395 7226 Acked Write 5 Done Done Sent false
21396 7227 Acked Write 9 Done Done Sent false
21397 7228 Acked Read 6 Done Done Sent false
21398 7229 Acked Read 2 Done Done Sent false
21399 7230 Acked Write 6 Done Done Sent false
21400 7231 Acked Flush 0 Done Done Sent false
21401 7232 Acked Write 10 Done Done Sent false
21402 7233 Acked Write 7 Done Done Sent false
21403 7234 Acked Read 1 Done Done Sent false
21404 7235 Acked Write 2 Done Done Sent false
21405 7236 Acked Read 4 Done Done Sent false
21406 7237 Acked Read 1 Done Done Sent false
21407 7238 Acked Write 9 Done Done Sent false
21408 7239 Acked Read 2 Done Done Sent false
21409 7240 Acked Flush 0 Done Done Sent false
21410 7241 Acked Read 4 Done Done Sent false
21411 7242 Acked Write 7 Done Done Sent false
21412 7243 Acked Write 5 Done Done Sent false
21413 7244 Acked Write 8 Done Done Sent false
21414 7245 Acked Write 5 Done Done Sent false
21415 7246 Acked Write 7 Done Done Sent false
21416 7247 Acked Read 5 Done Done Sent false
21417 7248 Acked Read 3 Done Done Sent false
21418 7249 Acked Read 1 Done Done Sent false
21419 7250 Acked Write 7 Done Done Sent false
21420 7251 Acked Read 9 Done Done Sent false
21421 7252 Acked Read 4 Done Done Sent false
21422 7253 Acked Read 4 Done Done Sent false
21423 7254 Acked Read 7 Done Done Sent false
21424 7255 Acked Read 6 Done Done Sent false
21425 7256 Acked Read 8 Done Done Sent false
21426 7257 Acked Write 9 Done Done Sent false
21427 7258 Acked Read 6 Done Done Sent false
21428 7259 Acked Flush 0 Done Done Sent false
21429 7260 Acked Read 8 Done Done Sent false
21430 7261 Acked Read 5 Done Done Sent false
21431 7262 Acked Write 6 Done Done Sent false
21432 7263 Acked Write 8 Done Done Sent false
21433 7264 Acked Flush 0 Done Done Sent false
21434 7265 Acked Read 6 Done Done Sent false
21435 7266 Acked Flush 0 Done Done Sent false
21436 7267 Acked Write 1 Done Done Sent false
21437 7268 Acked Flush 0 Done Done Sent false
21438 7269 Acked Read 2 Done Done Sent false
21439 7270 Acked Flush 0 Done Done Sent false
21440 7271 Acked Read 5 Done Done Sent false
21441 7272 Acked Flush 0 Done Done Sent false
21442 7273 Acked Read 8 Done Done Sent false
21443 7274 Acked Write 3 Done Done Sent false
21444 7275 Acked Read 10 Done Done Sent false
21445 7276 Acked Read 4 Done Done Sent false
21446 7277 Acked Write 5 Done Done Sent false
21447 7278 Acked Read 6 Done Done Sent false
21448 7279 Acked Read 2 Done Done Sent false
21449 7280 Acked Write 10 Done Done Sent false
21450 7281 Acked Read 5 Done Done Sent false
21451 7282 Acked Read 10 Done Done Sent false
21452 7283 Acked Flush 0 Done Done Sent false
21453 7284 Acked Write 8 Done Done Sent false
21454 7285 Acked Write 2 Done Done Sent false
21455 7286 Acked Read 7 Done Done Sent false
21456 7287 Acked Flush 0 Done Done Sent false
21457 7288 Acked Read 9 Done Done Sent false
21458 7289 Acked Write 7 Done Done Sent false
21459 7290 Acked Read 9 Done Done Sent false
21460 7291 Acked Read 4 Done Done Sent false
21461 7292 Acked Read 9 Done Done Sent false
21462 7293 Acked Read 9 Done Done Sent false
21463 7294 Acked Read 2 Done Done Sent false
21464 7295 Acked Read 6 Done Done Sent false
21465 7296 Acked Flush 0 Done Done Sent false
21466 7297 Acked Write 10 Done Done Sent false
21467 7298 Acked Read 2 Done Done Sent false
21468 7299 Acked Read 8 Done Done Sent false
21469 7300 Acked Read 1 Done Done Sent false
21470 7301 Acked Write 6 Done Done Sent false
21471 7302 Acked Read 1 Done Done Sent false
21472 7303 Acked Read 2 Done Done Sent false
21473 7304 Acked Write 1 Done Done Sent false
21474 7305 Acked Read 4 Done Done Sent false
21475 7306 Acked Read 4 Done Done Sent false
21476 7307 Acked Flush 0 Done Done Sent false
21477 7308 Acked Read 8 Done Done Sent false
21478 7309 Acked Write 5 Done Done Sent false
21479 7310 Acked Write 7 Done Done Sent false
21480 7311 Acked Read 3 Done Done Sent false
21481 7312 Acked Read 9 Done Done Sent false
21482 7313 Acked Read 8 Done Done Sent false
21483 7314 Acked Read 5 Done Done Sent false
21484 7315 Acked Write 6 Done Done Sent false
21485 7316 Acked Read 10 Done Done Sent false
21486 7317 Acked Read 2 Done Done Sent false
21487 7318 Acked Read 8 Done Done Sent false
21488 7319 Acked Flush 0 Done Done Sent false
21489 7320 Acked Read 6 Done Done Sent false
21490 7321 Acked Write 4 Done Done Sent false
21491 7322 Acked Write 3 Done Done Sent false
21492 7323 Acked Read 4 Done Done Sent false
21493 7324 Acked Read 3 Done Done Sent false
21494 7325 Acked Write 3 Done Done Sent false
21495 7326 Acked Read 1 Done Done Sent false
21496 7327 Acked Write 7 Done Done Sent false
21497 7328 Acked Write 2 Done Done Sent false
21498 7329 Acked Flush 0 Done Done Sent false
21499 7330 Acked Read 1 Done Done Sent false
21500 7331 Acked Write 10 Done Done Sent false
21501 7332 Acked Read 8 Done Done Sent false
21502 7333 Acked Flush 0 Done Done Sent false
21503 7334 Acked Write 8 Done Done Sent false
21504 7335 Acked Write 9 Done Done Sent false
21505 7336 Acked Read 7 Done Done Sent false
21506 7337 Acked Read 5 Done Done Sent false
21507 STATES DS:0 DS:1 DS:2 TOTAL
21508 Sent 0 0 301 301
21509 Done 301 301 0 602
21510 Skipped 0 0 0 0
21511 Error 0 0 0 0
21512 Last Flush: 7333 7333 7036
21513 Downstairs last five completed: 7036 7035 7034 7033 7032
21514 Upstairs last five completed: 7337 7336 7335 7334 7333
21515 CLIENT: Up:0 ds:301 act:2
21516 [8830] got start action
21517 [8830] I am going to restart
21518 Starting downstairs at port 8830
21519 Make output file at "/tmp/dsc/downstairs-8830.txt"
21520 [8830][2] reports Starting
21521 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1223)
21522 [8830][2] reports Running
21523 ----------------------------------------------------------------
21524 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21525 States: Active Active Offline
21526 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21527 7037 Acked Write 9 Done Done Sent false
21528 7038 Acked Flush 0 Done Done Sent false
21529 7039 Acked Read 3 Done Done Sent false
21530 7040 Acked Write 9 Done Done Sent false
21531 7041 Acked Read 4 Done Done Sent false
21532 7042 Acked Read 9 Done Done Sent false
21533 7043 Acked Write 6 Done Done Sent false
21534 7044 Acked Flush 0 Done Done Sent false
21535 7045 Acked Write 4 Done Done Sent false
21536 7046 Acked Write 8 Done Done Sent false
21537 7047 Acked Write 4 Done Done Sent false
21538 7048 Acked Read 10 Done Done Sent false
21539 7049 Acked Read 5 Done Done Sent false
21540 7050 Acked Write 6 Done Done Sent false
21541 7051 Acked Write 4 Done Done Sent false
21542 7052 Acked Read 9 Done Done Sent false
21543 7053 Acked Read 3 Done Done Sent false
21544 7054 Acked Read 1 Done Done Sent false
21545 7055 Acked Read 1 Done Done Sent false
21546 7056 Acked Flush 0 Done Done Sent false
21547 7057 Acked Read 8 Done Done Sent false
21548 7058 Acked Read 10 Done Done Sent false
21549 7059 Acked Write 1 Done Done Sent false
21550 7060 Acked Read 2 Done Done Sent false
21551 7061 Acked Flush 0 Done Done Sent false
21552 7062 Acked Read 4 Done Done Sent false
21553 7063 Acked Read 3 Done Done Sent false
21554 7064 Acked Write 1 Done Done Sent false
21555 7065 Acked Read 10 Done Done Sent false
21556 7066 Acked Write 8 Done Done Sent false
21557 7067 Acked Write 1 Done Done Sent false
21558 7068 Acked Read 6 Done Done Sent false
21559 7069 Acked Write 5 Done Done Sent false
21560 7070 Acked Write 9 Done Done Sent false
21561 7071 Acked Read 7 Done Done Sent false
21562 7072 Acked Read 10 Done Done Sent false
21563 7073 Acked Write 7 Done Done Sent false
21564 7074 Acked Read 8 Done Done Sent false
21565 7075 Acked Read 9 Done Done Sent false
21566 7076 Acked Write 4 Done Done Sent false
21567 7077 Acked Read 8 Done Done Sent false
21568 7078 Acked Write 7 Done Done Sent false
21569 7079 Acked Write 7 Done Done Sent false
21570 7080 Acked Read 9 Done Done Sent false
21571 7081 Acked Write 2 Done Done Sent false
21572 7082 Acked Write 5 Done Done Sent false
21573 7083 Acked Read 5 Done Done Sent false
21574 7084 Acked Read 7 Done Done Sent false
21575 7085 Acked Read 9 Done Done Sent false
21576 7086 Acked Read 6 Done Done Sent false
21577 7087 Acked Read 3 Done Done Sent false
21578 7088 Acked Read 8 Done Done Sent false
21579 7089 Acked Flush 0 Done Done Sent false
21580 7090 Acked Read 1 Done Done Sent false
21581 7091 Acked Write 3 Done Done Sent false
21582 7092 Acked Read 5 Done Done Sent false
21583 7093 Acked Read 3 Done Done Sent false
21584 7094 Acked Read 9 Done Done Sent false
21585 7095 Acked Write 1 Done Done Sent false
21586 7096 Acked Read 1 Done Done Sent false
21587 7097 Acked Read 7 Done Done Sent false
21588 7098 Acked Write 2 Done Done Sent false
21589 7099 Acked Write 2 Done Done Sent false
21590 7100 Acked Write 10 Done Done Sent false
21591 7101 Acked Read 9 Done Done Sent false
21592 7102 Acked Write 9 Done Done Sent false
21593 7103 Acked Write 5 Done Done Sent false
21594 7104 Acked Flush 0 Done Done Sent false
21595 7105 Acked Read 10 Done Done Sent false
21596 7106 Acked Write 1 Done Done Sent false
21597 7107 Acked Read 9 Done Done Sent false
21598 7108 Acked Read 1 Done Done Sent false
21599 7109 Acked Read 7 Done Done Sent false
21600 7110 Acked Write 8 Done Done Sent false
21601 7111 Acked Flush 0 Done Done Sent false
21602 7112 Acked Write 6 Done Done Sent false
21603 7113 Acked Write 8 Done Done Sent false
21604 7114 Acked Write 3 Done Done Sent false
21605 7115 Acked Flush 0 Done Done Sent false
21606 7116 Acked Flush 0 Done Done Sent false
21607 7117 Acked Write 7 Done Done Sent false
21608 7118 Acked Write 10 Done Done Sent false
21609 7119 Acked Read 8 Done Done Sent false
21610 7120 Acked Read 3 Done Done Sent false
21611 7121 Acked Read 2 Done Done Sent false
21612 7122 Acked Write 5 Done Done Sent false
21613 7123 Acked Flush 0 Done Done Sent false
21614 7124 Acked Read 7 Done Done Sent false
21615 7125 Acked Write 5 Done Done Sent false
21616 7126 Acked Read 10 Done Done Sent false
21617 7127 Acked Read 2 Done Done Sent false
21618 7128 Acked Read 4 Done Done Sent false
21619 7129 Acked Read 5 Done Done Sent false
21620 7130 Acked Read 4 Done Done Sent false
21621 7131 Acked Write 5 Done Done Sent false
21622 7132 Acked Read 5 Done Done Sent false
21623 7133 Acked Flush 0 Done Done Sent false
21624 7134 Acked Write 9 Done Done Sent false
21625 7135 Acked Write 9 Done Done Sent false
21626 7136 Acked Write 2 Done Done Sent false
21627 7137 Acked Flush 0 Done Done Sent false
21628 7138 Acked Write 9 Done Done Sent false
21629 7139 Acked Write 7 Done Done Sent false
21630 7140 Acked Read 2 Done Done Sent false
21631 7141 Acked Read 10 Done Done Sent false
21632 7142 Acked Read 9 Done Done Sent false
21633 7143 Acked Write 7 Done Done Sent false
21634 7144 Acked Read 4 Done Done Sent false
21635 7145 Acked Read 1 Done Done Sent false
21636 7146 Acked Write 6 Done Done Sent false
21637 7147 Acked Write 3 Done Done Sent false
21638 7148 Acked Write 8 Done Done Sent false
21639 7149 Acked Read 8 Done Done Sent false
21640 7150 Acked Read 6 Done Done Sent false
21641 7151 Acked Flush 0 Done Done Sent false
21642 7152 Acked Read 2 Done Done Sent false
21643 7153 Acked Read 9 Done Done Sent false
21644 7154 Acked Write 9 Done Done Sent false
21645 7155 Acked Read 9 Done Done Sent false
21646 7156 Acked Read 6 Done Done Sent false
21647 7157 Acked Read 8 Done Done Sent false
21648 7158 Acked Write 9 Done Done Sent false
21649 7159 Acked Read 10 Done Done Sent false
21650 7160 Acked Write 9 Done Done Sent false
21651 7161 Acked Read 6 Done Done Sent false
21652 7162 Acked Read 10 Done Done Sent false
21653 7163 Acked Read 3 Done Done Sent false
21654 7164 Acked Flush 0 Done Done Sent false
21655 7165 Acked Read 1 Done Done Sent false
21656 7166 Acked Flush 0 Done Done Sent false
21657 7167 Acked Read 7 Done Done Sent false
21658 7168 Acked Write 8 Done Done Sent false
21659 7169 Acked Write 4 Done Done Sent false
21660 7170 Acked Write 7 Done Done Sent false
21661 7171 Acked Read 1 Done Done Sent false
21662 7172 Acked Read 2 Done Done Sent false
21663 7173 Acked Read 7 Done Done Sent false
21664 7174 Acked Write 3 Done Done Sent false
21665 7175 Acked Read 7 Done Done Sent false
21666 7176 Acked Write 8 Done Done Sent false
21667 7177 Acked Read 4 Done Done Sent false
21668 7178 Acked Read 10 Done Done Sent false
21669 7179 Acked Flush 0 Done Done Sent false
21670 7180 Acked Read 2 Done Done Sent false
21671 7181 Acked Read 9 Done Done Sent false
21672 7182 Acked Read 5 Done Done Sent false
21673 7183 Acked Write 3 Done Done Sent false
21674 7184 Acked Flush 0 Done Done Sent false
21675 7185 Acked Read 1 Done Done Sent false
21676 7186 Acked Read 1 Done Done Sent false
21677 7187 Acked Read 4 Done Done Sent false
21678 7188 Acked Read 8 Done Done Sent false
21679 7189 Acked Read 8 Done Done Sent false
21680 7190 Acked Write 10 Done Done Sent false
21681 7191 Acked Read 3 Done Done Sent false
21682 7192 Acked Write 7 Done Done Sent false
21683 7193 Acked Write 10 Done Done Sent false
21684 7194 Acked Write 9 Done Done Sent false
21685 7195 Acked Write 1 Done Done Sent false
21686 7196 Acked Read 7 Done Done Sent false
21687 7197 Acked Read 4 Done Done Sent false
21688 7198 Acked Write 10 Done Done Sent false
21689 7199 Acked Write 2 Done Done Sent false
21690 7200 Acked Write 2 Done Done Sent false
21691 7201 Acked Read 3 Done Done Sent false
21692 7202 Acked Write 6 Done Done Sent false
21693 7203 Acked Write 7 Done Done Sent false
21694 7204 Acked Read 1 Done Done Sent false
21695 7205 Acked Read 5 Done Done Sent false
21696 7206 Acked Write 7 Done Done Sent false
21697 7207 Acked Read 2 Done Done Sent false
21698 7208 Acked Flush 0 Done Done Sent false
21699 7209 Acked Write 2 Done Done Sent false
21700 7210 Acked Write 6 Done Done Sent false
21701 7211 Acked Read 3 Done Done Sent false
21702 7212 Acked Write 10 Done Done Sent false
21703 7213 Acked Read 5 Done Done Sent false
21704 7214 Acked Read 9 Done Done Sent false
21705 7215 Acked Read 4 Done Done Sent false
21706 7216 Acked Write 3 Done Done Sent false
21707 7217 Acked Read 4 Done Done Sent false
21708 7218 Acked Read 8 Done Done Sent false
21709 7219 Acked Read 3 Done Done Sent false
21710 7220 Acked Read 2 Done Done Sent false
21711 7221 Acked Read 9 Done Done Sent false
21712 7222 Acked Read 6 Done Done Sent false
21713 7223 Acked Read 7 Done Done Sent false
21714 7224 Acked Read 6 Done Done Sent false
21715 7225 Acked Write 1 Done Done Sent false
21716 7226 Acked Write 5 Done Done Sent false
21717 7227 Acked Write 9 Done Done Sent false
21718 7228 Acked Read 6 Done Done Sent false
21719 7229 Acked Read 2 Done Done Sent false
21720 7230 Acked Write 6 Done Done Sent false
21721 7231 Acked Flush 0 Done Done Sent false
21722 7232 Acked Write 10 Done Done Sent false
21723 7233 Acked Write 7 Done Done Sent false
21724 7234 Acked Read 1 Done Done Sent false
21725 7235 Acked Write 2 Done Done Sent false
21726 7236 Acked Read 4 Done Done Sent false
21727 7237 Acked Read 1 Done Done Sent false
21728 7238 Acked Write 9 Done Done Sent false
21729 7239 Acked Read 2 Done Done Sent false
21730 7240 Acked Flush 0 Done Done Sent false
21731 7241 Acked Read 4 Done Done Sent false
21732 7242 Acked Write 7 Done Done Sent false
21733 7243 Acked Write 5 Done Done Sent false
21734 7244 Acked Write 8 Done Done Sent false
21735 7245 Acked Write 5 Done Done Sent false
21736 7246 Acked Write 7 Done Done Sent false
21737 7247 Acked Read 5 Done Done Sent false
21738 7248 Acked Read 3 Done Done Sent false
21739 7249 Acked Read 1 Done Done Sent false
21740 7250 Acked Write 7 Done Done Sent false
21741 7251 Acked Read 9 Done Done Sent false
21742 7252 Acked Read 4 Done Done Sent false
21743 7253 Acked Read 4 Done Done Sent false
21744 7254 Acked Read 7 Done Done Sent false
21745 7255 Acked Read 6 Done Done Sent false
21746 7256 Acked Read 8 Done Done Sent false
21747 7257 Acked Write 9 Done Done Sent false
21748 7258 Acked Read 6 Done Done Sent false
21749 7259 Acked Flush 0 Done Done Sent false
21750 7260 Acked Read 8 Done Done Sent false
21751 7261 Acked Read 5 Done Done Sent false
21752 7262 Acked Write 6 Done Done Sent false
21753 7263 Acked Write 8 Done Done Sent false
21754 7264 Acked Flush 0 Done Done Sent false
21755 7265 Acked Read 6 Done Done Sent false
21756 7266 Acked Flush 0 Done Done Sent false
21757 7267 Acked Write 1 Done Done Sent false
21758 7268 Acked Flush 0 Done Done Sent false
21759 7269 Acked Read 2 Done Done Sent false
21760 7270 Acked Flush 0 Done Done Sent false
21761 7271 Acked Read 5 Done Done Sent false
21762 7272 Acked Flush 0 Done Done Sent false
21763 7273 Acked Read 8 Done Done Sent false
21764 7274 Acked Write 3 Done Done Sent false
21765 7275 Acked Read 10 Done Done Sent false
21766 7276 Acked Read 4 Done Done Sent false
21767 7277 Acked Write 5 Done Done Sent false
21768 7278 Acked Read 6 Done Done Sent false
21769 7279 Acked Read 2 Done Done Sent false
21770 7280 Acked Write 10 Done Done Sent false
21771 7281 Acked Read 5 Done Done Sent false
21772 7282 Acked Read 10 Done Done Sent false
21773 7283 Acked Flush 0 Done Done Sent false
21774 7284 Acked Write 8 Done Done Sent false
21775 7285 Acked Write 2 Done Done Sent false
21776 7286 Acked Read 7 Done Done Sent false
21777 7287 Acked Flush 0 Done Done Sent false
21778 7288 Acked Read 9 Done Done Sent false
21779 7289 Acked Write 7 Done Done Sent false
21780 7290 Acked Read 9 Done Done Sent false
21781 7291 Acked Read 4 Done Done Sent false
21782 7292 Acked Read 9 Done Done Sent false
21783 7293 Acked Read 9 Done Done Sent false
21784 7294 Acked Read 2 Done Done Sent false
21785 7295 Acked Read 6 Done Done Sent false
21786 7296 Acked Flush 0 Done Done Sent false
21787 7297 Acked Write 10 Done Done Sent false
21788 7298 Acked Read 2 Done Done Sent false
21789 7299 Acked Read 8 Done Done Sent false
21790 7300 Acked Read 1 Done Done Sent false
21791 7301 Acked Write 6 Done Done Sent false
21792 7302 Acked Read 1 Done Done Sent false
21793 7303 Acked Read 2 Done Done Sent false
21794 7304 Acked Write 1 Done Done Sent false
21795 7305 Acked Read 4 Done Done Sent false
21796 7306 Acked Read 4 Done Done Sent false
21797 7307 Acked Flush 0 Done Done Sent false
21798 7308 Acked Read 8 Done Done Sent false
21799 7309 Acked Write 5 Done Done Sent false
21800 7310 Acked Write 7 Done Done Sent false
21801 7311 Acked Read 3 Done Done Sent false
21802 7312 Acked Read 9 Done Done Sent false
21803 7313 Acked Read 8 Done Done Sent false
21804 7314 Acked Read 5 Done Done Sent false
21805 7315 Acked Write 6 Done Done Sent false
21806 7316 Acked Read 10 Done Done Sent false
21807 7317 Acked Read 2 Done Done Sent false
21808 7318 Acked Read 8 Done Done Sent false
21809 7319 Acked Flush 0 Done Done Sent false
21810 7320 Acked Read 6 Done Done Sent false
21811 7321 Acked Write 4 Done Done Sent false
21812 7322 Acked Write 3 Done Done Sent false
21813 7323 Acked Read 4 Done Done Sent false
21814 7324 Acked Read 3 Done Done Sent false
21815 7325 Acked Write 3 Done Done Sent false
21816 7326 Acked Read 1 Done Done Sent false
21817 7327 Acked Write 7 Done Done Sent false
21818 7328 Acked Write 2 Done Done Sent false
21819 7329 Acked Flush 0 Done Done Sent false
21820 7330 Acked Read 1 Done Done Sent false
21821 7331 Acked Write 10 Done Done Sent false
21822 7332 Acked Read 8 Done Done Sent false
21823 7333 Acked Flush 0 Done Done Sent false
21824 7334 Acked Write 8 Done Done Sent false
21825 7335 Acked Write 9 Done Done Sent false
21826 7336 Acked Read 7 Done Done Sent false
21827 7337 Acked Read 5 Done Done Sent false
21828 7338 Acked Flush 0 Done Done Sent false
21829 STATES DS:0 DS:1 DS:2 TOTAL
21830 Sent 0 0 302 302
21831 Done 302 302 0 604
21832 Skipped 0 0 0 0
21833 Error 0 0 0 0
21834 Last Flush: 7338 7338 7036
21835 Downstairs last five completed: 7036 7035 7034 7033 7032
21836 Upstairs last five completed: 7338 7337 7336 7335 7334
21837 CLIENT: Up:0 ds:302 act:2
218382025-01-14T16:15:10.806ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
218392025-01-14T16:15:10.806ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49903) = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
218402025-01-14T16:15:10.807ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
218412025-01-14T16:15:10.808ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
218422025-01-14T16:15:10.808ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
218432025-01-14T16:15:10.808ZINFOcrucible: send last flush ID to this DS: 7036 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
218442025-01-14T16:15:10.808ZINFOcrucible: Replied this last flush ID: 7036 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
218452025-01-14T16:15:10.808ZINFOcrucible: [2] client re-new 302 jobs since flush 7036 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
218462025-01-14T16:15:10.808ZINFOcrucible: [2] Replayed 302 jobs since flush: 7036 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
21847 ----------------------------------------------------------------
21848 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
21849 Downstairs last five completed: 7338 7337 7336 7335 7334
21850 Upstairs last five completed: 7338 7337 7336 7335 7334
21851 CLIENT: Up:0 ds:0 act:3
21852 Replay: All jobs finished
21853 Main task has work to do, go find it
21854 got dsc Stop(0)
21855 stop 0
21856 [8810] Got stop action so:false kr:false
21857 Jan 14 16:15:13.053 INFO request completed, latency_us: 330, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 25eafba7-ad94-49e9-9490-5b6883e93827, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
218582025-01-14T16:15:13.054ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
218592025-01-14T16:15:13.054ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
218602025-01-14T16:15:13.054ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
21861 Jan 14 16:15:13.054 INFO request completed, latency_us: 173, response_code: 200, uri: /state/cid/0, method: GET, req_id: 5c6bf62d-cffd-4c02-b398-4de359d3d3c3, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
218622025-01-14T16:15:13.054ZINFOcrucible: client task is exiting = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
21863 [8810] Exited with: ExitStatus(unix_wait_status(9))
21864 [8810][0] reports Exit
21865 Jan 14 16:15:17.056 INFO request completed, latency_us: 197, response_code: 200, uri: /state/cid/0, method: GET, req_id: 3b83fe0b-919d-4600-8119-60be816bd9bd, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
21866 001/300 Read block 4205 len 20480
21867 002/300 Read block 3353 len 28672
21868 003/300 Write block 4294 len 12288 data: 8 10 7
21869 004/300 Write block 4344 len 40960 data: 4 4 5 5 5 6 7 7 6 7
21870 005/300 Write block 1806 len 16384 data: 4 4 5 5
21871 006/300 Read block 1990 len 20480
21872 007/300 Write block 2770 len 4096 data: 3
21873 008/300 Write block 1659 len 24576 data: 8 5 7 6 5 4
21874 009/300 Read block 4401 len 12288
21875 010/300 Flush
21876 011/300 Read block 711 len 40960
21877 012/300 Read block 2034 len 28672
21878 013/300 Write block 39 len 12288 data: 6 5 7
21879 014/300 Write block 1686 len 32768 data: 5 6 6 5 4 4 4 5
21880 015/300 Read block 4855 len 16384
21881 016/300 Read block 1437 len 36864
21882 017/300 Read block 1876 len 8192
21883 018/300 Write block 3930 len 16384 data: 5 5 6 6
21884 019/300 Read block 215 len 4096
21885 020/300 Write block 2595 len 32768 data: 4 3 3 4 4 6 6 6
21886 021/300 Flush
21887 022/300 Write block 3657 len 8192 data: 3 4
21888 023/300 Read block 1334 len 36864
21889 024/300 Read block 3963 len 32768
21890 025/300 Write block 3101 len 28672 data: 5 4 4 4 3 4 4
21891 026/300 Read block 4654 len 16384
21892 027/300 Write block 3499 len 4096 data: 3
21893 028/300 Read block 2864 len 24576
21894 029/300 Flush
21895 030/300 Read block 4501 len 24576
21896 031/300 Write block 385 len 36864 data: 4 3 4 3 3 4 4 3 3
21897 032/300 Write block 4109 len 16384 data: 3 3 2 3
21898 033/300 Read block 1383 len 40960
21899 034/300 Flush
21900 035/300 Read block 4933 len 28672
21901 036/300 Read block 2135 len 4096
21902 037/300 Read block 3230 len 28672
21903 038/300 Read block 3937 len 28672
21904 039/300 Write block 3942 len 4096 data: 4
21905 040/300 Read block 2900 len 4096
21906 041/300 Write block 337 len 24576 data: 5 5 3 4 3 4
21907 042/300 Write block 1883 len 36864 data: 4 4 2 2 3 4 6 7 8
21908 043/300 Flush
21909 044/300 Read block 310 len 32768
21910 045/300 Flush
21911 046/300 Write block 442 len 24576 data: 5 5 5 5 4 5
21912 047/300 Read block 2416 len 40960
21913 048/300 Read block 4014 len 4096
21914 049/300 Read block 4905 len 4096
21915 050/300 Write block 1122 len 28672 data: 3 3 4 4 4 4 4
21916 051/300 Read block 2004 len 20480
21917 052/300 Write block 1541 len 20480 data: 4 5 6 4 5
21918 053/300 Write block 56 len 28672 data: 6 4 6 5 6 6 6
21919 054/300 Write block 490 len 24576 data: 3 4 2 2 3 4
21920 055/300 Write block 4425 len 20480 data: 8 7 7 5 5
21921 056/300 Write block 1008 len 12288 data: 5 4 3
21922 057/300 Read block 3964 len 32768
21923 058/300 Read block 4913 len 28672
21924 059/300 Read block 68 len 8192
21925 060/300 Read block 1845 len 20480
21926 061/300 Flush
21927 062/300 Write block 1599 len 4096 data: 4
21928 063/300 Write block 1091 len 4096 data: 2
21929 064/300 Read block 4261 len 28672
21930 065/300 Read block 2902 len 20480
21931 066/300 Write block 1279 len 24576 data: 8 7 6 6 4 4
21932 067/300 Read block 3786 len 40960
21933 068/300 Write block 4798 len 28672 data: 5 4 4 4 4 4 5
21934 069/300 Write block 4756 len 32768 data: 4 4 5 4 3 4 4 4
21935 070/300 Write block 2238 len 4096 data: 6
21936 071/300 Read block 434 len 28672
21937 072/300 Read block 3872 len 40960
21938 073/300 Read block 2106 len 36864
21939 074/300 Read block 2106 len 8192
21940 075/300 Read block 2248 len 24576
21941 076/300 Write block 1347 len 40960 data: 2 3 3 3 5 6 6 5 7 7
21942 077/300 Write block 2986 len 20480 data: 5 4 3 5 5
21943 078/300 Read block 2852 len 24576
21944 079/300 Write block 3004 len 36864 data: 6 5 5 5 3 3 4 3 3
21945 080/300 Read block 467 len 28672
21946 081/300 Write block 23 len 28672 data: 7 8 8 8 7 9 8
21947 082/300 Read block 2752 len 40960
21948 083/300 Write block 1848 len 28672 data: 8 6 5 6 7 6 6
21949 084/300 Write block 1441 len 36864 data: 8 8 7 7 7 8 8 6 6
21950 085/300 Read block 4731 len 24576
21951 086/300 Write block 3181 len 20480 data: 6 5 4 3 3
21952 087/300 Read block 2331 len 40960
21953 088/300 Write block 3672 len 20480 data: 4 4 3 3 3
21954 089/300 Read block 2469 len 24576
21955 090/300 Read block 2231 len 32768
21956 091/300 Write block 70 len 40960 data: 5 5 6 4 4 5 4 4 6 8
21957 092/300 Read block 1300 len 24576
21958 093/300 Read block 1667 len 12288
21959 094/300 Read block 4787 len 24576
21960 095/300 Write block 3940 len 4096 data: 5
21961 096/300 Read block 3550 len 8192
21962 097/300 Flush
21963 098/300 Write block 1747 len 36864 data: 6 8 6 6 6 7 7 10 10
21964 099/300 Write block 2739 len 36864 data: 5 3 2 2 3 3 3 3 3
21965 100/300 Read block 2529 len 32768
21966 101/300 Write block 1671 len 40960 data: 7 7 8 8 8 8 7 6 5 4
21967 102/300 Write block 4872 len 32768 data: 3 3 3 5 5 5 3 2
21968 103/300 Read block 3537 len 20480
21969 104/300 Read block 3365 len 8192
21970 105/300 Read block 426 len 16384
21971 106/300 Read block 3126 len 28672
21972 107/300 Read block 2398 len 40960
21973 108/300 Read block 3289 len 28672
21974 109/300 Write block 3441 len 16384 data: 4 4 4 2
21975 110/300 Write block 2306 len 12288 data: 5 6 6
21976 111/300 Write block 4466 len 28672 data: 3 4 3 3 3 3 4
21977 112/300 Write block 535 len 28672 data: 5 6 5 3 3 4 4
21978 113/300 Read block 1586 len 24576
21979 114/300 Read block 2113 len 4096
21980 115/300 Read block 1136 len 20480
21981 116/300 Read block 3080 len 20480
21982 117/300 Read block 4411 len 40960
21983 118/300 Write block 227 len 24576 data: 4 4 3 3 3 3
21984 119/300 Read block 807 len 24576
21985 120/300 Read block 2637 len 36864
21986 121/300 Read block 1268 len 4096
21987 122/300 Write block 4276 len 16384 data: 5 5 5 5
21988 123/300 Read block 613 len 16384
21989 124/300 Write block 807 len 8192 data: 8 8
21990 125/300 Read block 3832 len 32768
21991 126/300 Read block 3571 len 28672
21992 127/300 Read block 4421 len 16384
21993 128/300 Write block 4027 len 12288 data: 5 7 6
21994 129/300 Read block 4217 len 12288
21995 130/300 Read block 1431 len 32768
21996 131/300 Write block 149 len 8192 data: 7 6
21997 132/300 Write block 1402 len 8192 data: 5 6
21998 133/300 Read block 674 len 4096
21999 134/300 Read block 461 len 40960
22000 135/300 Read block 3635 len 24576
22001 136/300 Flush
22002 137/300 Write block 1723 len 36864 data: 5 6 5 5 5 5 4 4 4
22003 138/300 Write block 2307 len 36864 data: 7 7 6 7 7 9 10 10 8
22004 139/300 Write block 1659 len 8192 data: 9 6
22005 140/300 Write block 3912 len 36864 data: 9 9 7 9 9 6 4 2 3
22006 141/300 Write block 592 len 40960 data: 7 6 4 5 6 5 4 4 4 3
22007 142/300 Read block 2555 len 8192
22008 143/300 Flush
22009 144/300 Write block 3662 len 28672 data: 4 4 3 4 4 3 3
22010 145/300 Write block 183 len 24576 data: 4 4 5 7 8 6
22011 146/300 Read block 355 len 24576
22012 147/300 Write block 3262 len 4096 data: 5
22013 148/300 Write block 2630 len 28672 data: 5 4 4 4 5 5 5
22014 149/300 Write block 2733 len 32768 data: 5 3 5 5 5 5 6 4
22015 150/300 Flush
22016 151/300 Read block 86 len 32768
22017 152/300 Read block 2216 len 4096
22018 153/300 Write block 815 len 28672 data: 8 7 6 5 6 6 5
22019 154/300 Write block 3660 len 36864 data: 3 3 5 5 4 5 5 4 4
22020 155/300 Read block 2623 len 8192
22021 156/300 Write block 669 len 36864 data: 5 5 6 6 7 6 7 6 5
22022 157/300 Read block 2868 len 16384
22023 158/300 Read block 1169 len 36864
22024 159/300 Flush
22025 160/300 Read block 3553 len 28672
22026 161/300 Flush
22027 162/300 Read block 1349 len 36864
22028 163/300 Write block 217 len 12288 data: 4 5 6
22029 164/300 Flush
22030 165/300 Write block 4787 len 16384 data: 4 5 4 6
22031 166/300 Read block 1138 len 20480
22032 167/300 Flush
22033 168/300 Write block 4227 len 40960 data: 4 5 5 4 4 3 2 2 2 4
22034 169/300 Write block 3558 len 28672 data: 3 3 2 2 4 4 4
22035 170/300 Read block 370 len 8192
22036 171/300 Write block 3829 len 20480 data: 5 5 6 6 6
22037 172/300 Read block 1781 len 28672
22038 173/300 Write block 2861 len 16384 data: 6 5 5 5
22039 174/300 Write block 448 len 16384 data: 6 8 5 6
22040 175/300 Read block 2285 len 8192
22041 176/300 Read block 2733 len 12288
22042 177/300 Flush
22043 178/300 Write block 2936 len 8192 data: 3 3
22044 179/300 Write block 1530 len 28672 data: 7 7 8 8 7 5 5
22045 180/300 Write block 697 len 40960 data: 6 6 6 5 6 5 5 6 6 6
22046 181/300 Write block 2637 len 40960 data: 5 4 5 5 5 5 7 7 7 6
22047 182/300 Write block 891 len 8192 data: 7 8
22048 183/300 Read block 4814 len 4096
22049 184/300 Read block 1980 len 12288
22050 185/300 Write block 292 len 36864 data: 8 7 6 6 4 4 4 4 4
22051 186/300 Write block 4762 len 40960 data: 5 5 5 4 6 6 8 8 10 9
22052 187/300 Flush
22053 188/300 Read block 3337 len 24576
22054 189/300 Write block 1790 len 12288 data: 4 4 3
22055 190/300 Read block 790 len 12288
22056 191/300 Read block 1596 len 36864
22057 192/300 Read block 4881 len 40960
22058 193/300 Write block 1143 len 20480 data: 3 3 2 2 3
22059 194/300 Read block 2748 len 40960
22060 195/300 Write block 2769 len 12288 data: 4 4 3
22061 196/300 Read block 4026 len 28672
22062 197/300 Read block 1224 len 12288
22063 198/300 Flush
22064 199/300 Write block 3300 len 16384 data: 4 4 3 4
22065 200/300 Read block 206 len 16384
22066 201/300 Write block 233 len 8192 data: 4 4
22067 202/300 Read block 1896 len 28672
22068 203/300 Write block 1691 len 24576 data: 5 5 6 4 5 7
22069 204/300 Write block 160 len 12288 data: 6 6 6
22070 205/300 Read block 2142 len 40960
22071 206/300 Read block 3355 len 28672
22072 207/300 Write block 2702 len 28672 data: 7 9 5 5 3 4 5
22073 208/300 Flush
22074 209/300 Write block 4273 len 28672 data: 4 4 5 6 6 6 6
22075 210/300 Write block 3609 len 12288 data: 7 8 8
22076 211/300 Read block 2732 len 20480
22077 212/300 Read block 4190 len 40960
22078 213/300 Read block 3285 len 32768
22079 214/300 Read block 1405 len 8192
22080 215/300 Write block 4132 len 12288 data: 4 4 4
22081 216/300 Write block 2831 len 16384 data: 7 6 6 6
22082 217/300 Write block 4350 len 36864 data: 8 8 7 8 6 5 5 4 5
22083 218/300 Write block 222 len 32768 data: 6 6 4 4 3 5 5 4
22084 219/300 Read block 587 len 8192
22085 220/300 Read block 99 len 8192
22086 221/300 Read block 281 len 4096
22087 222/300 Read block 4304 len 8192
22088 223/300 Flush
22089 224/300 Read block 2603 len 12288
22090 225/300 Read block 773 len 24576
22091 226/300 Read block 3723 len 12288
22092 227/300 Flush
22093 228/300 Flush
22094 229/300 Write block 4397 len 12288 data: 6 7 6
22095 230/300 Read block 91 len 24576
22096 231/300 Read block 4591 len 8192
22097 232/300 Read block 1935 len 16384
22098 233/300 Read block 2457 len 24576
22099 234/300 Read block 4322 len 40960
22100 235/300 Write block 1588 len 36864 data: 2 3 3 5 6 7 7 8 9
22101 236/300 Write block 3904 len 16384 data: 4 4 5 5
22102 237/300 Read block 848 len 40960
22103 238/300 Read block 76 len 28672
22104 239/300 Flush
22105 240/300 Read block 4549 len 16384
22106 241/300 Read block 2157 len 16384
22107 242/300 Read block 827 len 8192
22108 243/300 Flush
22109 244/300 Read block 2951 len 36864
22110 245/300 Read block 2186 len 4096
22111 246/300 Read block 930 len 8192
22112 247/300 Read block 616 len 40960
22113 248/300 Write block 1877 len 8192 data: 5 5
22114 249/300 Write block 2514 len 20480 data: 4 3 3 4 4
22115 250/300 Read block 2194 len 36864
22116 251/300 Write block 4166 len 20480 data: 6 3 4 3 4
22117 252/300 Read block 3032 len 20480
22118 253/300 Read block 2803 len 36864
22119 254/300 Read block 1820 len 20480
22120 255/300 Read block 491 len 36864
22121 256/300 Flush
22122 257/300 Read block 239 len 40960
22123 258/300 Write block 4671 len 16384 data: 6 4 4 3
22124 259/300 Read block 3375 len 8192
22125 260/300 Read block 982 len 32768
22126 261/300 Read block 3951 len 36864
22127 262/300 Flush
22128 263/300 Read block 661 len 20480
22129 264/300 Read block 2896 len 32768
22130 265/300 Read block 1259 len 4096
22131 266/300 Read block 2725 len 24576
22132 267/300 Write block 1388 len 24576 data: 4 5 5 5 5 5
22133 268/300 Read block 685 len 4096
22134 269/300 Read block 4230 len 32768
22135 270/300 Read block 533 len 8192
22136 271/300 Read block 4715 len 16384
22137 272/300 Read block 3522 len 4096
22138 273/300 Write block 1061 len 8192 data: 6 5
22139 274/300 Read block 4598 len 28672
22140 275/300 Write block 1649 len 8192 data: 5 5
22141 276/300 Read block 518 len 20480
22142 277/300 Read block 3610 len 16384
22143 278/300 Write block 1095 len 28672 data: 4 5 5 5 6 4 5
22144 279/300 Write block 3368 len 24576 data: 7 7 6 5 5 5
22145 280/300 Write block 3392 len 20480 data: 7 5 5 4 5
22146 281/300 Write block 3562 len 4096 data: 5
22147 282/300 Write block 3122 len 16384 data: 4 4 2 2
22148 283/300 Read block 3704 len 16384
22149 284/300 Read block 215 len 20480
22150 285/300 Write block 2759 len 20480 data: 3 3 4 4 5
22151 286/300 Read block 2403 len 20480
22152 287/300 Read block 209 len 12288
22153 288/300 Read block 2469 len 4096
22154 289/300 Write block 4816 len 28672 data: 3 3 3 3 4 4 5
22155 290/300 Write block 3000 len 28672 data: 5 5 6 6 7 6 6
22156 291/300 Read block 4125 len 32768
22157 292/300 Read block 4475 len 4096
22158 293/300 Write block 2691 len 4096 data: 3
22159 294/300 Read block 2798 len 12288
22160 295/300 Write block 4789 len 40960 data: 5 7 5 4 5 5 5 4 4 6
22161 296/300 Read block 684 len 40960
22162 297/300 Read block 916 len 20480
22163 298/300 Write block 2435 len 16384 data: 3 4 4 5
22164 299/300 Read block 1162 len 20480
22165 300/300 Read block 1469 len 8192
22166 301/300 Write block 3952 len 32768 data: 5 4 6 6 6 6 7 8
22167 Main task has work to do, go find it
22168 got dsc Start(0)
22169 start 0
22170 Jan 14 16:15:17.295 INFO request completed, latency_us: 207, response_code: 204, uri: /start/cid/0, method: POST, req_id: 21724d80-17f3-4960-8b28-f9edb2fd29fa, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
22171 Replay: started 0, returned:Ok(())
22172 ----------------------------------------------------------------
22173 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
22174 States: Offline Active Active
22175 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22176 7339 Acked Read 5 Sent Done Done false
22177 7340 Acked Read 7 Sent Done Done false
22178 7341 Acked Write 3 Sent Done Done false
22179 7342 Acked Write 10 Sent Done Done false
22180 7343 Acked Write 4 Sent Done Done false
22181 7344 Acked Read 5 Sent Done Done false
22182 7345 Acked Write 1 Sent Done Done false
22183 7346 Acked Write 6 Sent Done Done false
22184 7347 Acked Read 3 Sent Done Done false
22185 7348 Acked Flush 0 Sent Done Done false
22186 7349 Acked Read 10 Sent Done Done false
22187 7350 Acked Read 7 Sent Done Done false
22188 7351 Acked Write 3 Sent Done Done false
22189 7352 Acked Write 8 Sent Done Done false
22190 7353 Acked Read 4 Sent Done Done false
22191 7354 Acked Read 9 Sent Done Done false
22192 7355 Acked Read 2 Sent Done Done false
22193 7356 Acked Write 4 Sent Done Done false
22194 7357 Acked Read 1 Sent Done Done false
22195 7358 Acked Write 8 Sent Done Done false
22196 7359 Acked Flush 0 Sent Done Done false
22197 7360 Acked Write 2 Sent Done Done false
22198 7361 Acked Read 9 Sent Done Done false
22199 7362 Acked Read 8 Sent Done Done false
22200 7363 Acked Write 7 Sent Done Done false
22201 7364 Acked Read 4 Sent Done Done false
22202 7365 Acked Write 1 Sent Done Done false
22203 7366 Acked Read 6 Sent Done Done false
22204 7367 Acked Flush 0 Sent Done Done false
22205 7368 Acked Read 6 Sent Done Done false
22206 7369 Acked Write 9 Sent Done Done false
22207 7370 Acked Write 4 Sent Done Done false
22208 7371 Acked Read 10 Sent Done Done false
22209 7372 Acked Flush 0 Sent Done Done false
22210 7373 Acked Read 7 Sent Done Done false
22211 7374 Acked Read 1 Sent Done Done false
22212 7375 Acked Read 7 Sent Done Done false
22213 7376 Acked Read 7 Sent Done Done false
22214 7377 Acked Write 1 Sent Done Done false
22215 7378 Acked Read 1 Sent Done Done false
22216 7379 Acked Write 6 Sent Done Done false
22217 7380 Acked Write 9 Sent Done Done false
22218 7381 Acked Flush 0 Sent Done Done false
22219 7382 Acked Read 8 Sent Done Done false
22220 7383 Acked Flush 0 Sent Done Done false
22221 7384 Acked Write 6 Sent Done Done false
22222 7385 Acked Read 10 Sent Done Done false
22223 7386 Acked Read 1 Sent Done Done false
22224 7387 Acked Read 1 Sent Done Done false
22225 7388 Acked Write 7 Sent Done Done false
22226 7389 Acked Read 5 Sent Done Done false
22227 7390 Acked Write 5 Sent Done Done false
22228 7391 Acked Write 7 Sent Done Done false
22229 7392 Acked Write 6 Sent Done Done false
22230 7393 Acked Write 5 Sent Done Done false
22231 7394 Acked Write 3 Sent Done Done false
22232 7395 Acked Read 8 Sent Done Done false
22233 7396 Acked Read 7 Sent Done Done false
22234 7397 Acked Read 2 Sent Done Done false
22235 7398 Acked Read 5 Sent Done Done false
22236 7399 Acked Flush 0 Sent Done Done false
22237 7400 Acked Write 1 Sent Done Done false
22238 7401 Acked Write 1 Sent Done Done false
22239 7402 Acked Read 7 Sent Done Done false
22240 7403 Acked Read 5 Sent Done Done false
22241 7404 Acked Write 6 Sent Done Done false
22242 7405 Acked Read 10 Sent Done Done false
22243 7406 Acked Write 7 Sent Done Done false
22244 7407 Acked Write 8 Sent Done Done false
22245 7408 Acked Write 1 Sent Done Done false
22246 7409 Acked Read 7 Sent Done Done false
22247 7410 Acked Read 10 Sent Done Done false
22248 7411 Acked Read 9 Sent Done Done false
22249 7412 Acked Read 2 Sent Done Done false
22250 7413 Acked Read 6 Sent Done Done false
22251 7414 Acked Write 10 Sent Done Done false
22252 7415 Acked Write 5 Sent Done Done false
22253 7416 Acked Read 6 Sent Done Done false
22254 7417 Acked Write 9 Sent Done Done false
22255 7418 Acked Read 7 Sent Done Done false
22256 7419 Acked Write 7 Sent Done Done false
22257 7420 Acked Read 10 Sent Done Done false
22258 7421 Acked Write 7 Sent Done Done false
22259 7422 Acked Write 9 Sent Done Done false
22260 7423 Acked Read 6 Sent Done Done false
22261 7424 Acked Write 5 Sent Done Done false
22262 7425 Acked Read 10 Sent Done Done false
22263 7426 Acked Write 5 Sent Done Done false
22264 7427 Acked Read 6 Sent Done Done false
22265 7428 Acked Read 8 Sent Done Done false
22266 7429 Acked Write 10 Sent Done Done false
22267 7430 Acked Read 6 Sent Done Done false
22268 7431 Acked Read 3 Sent Done Done false
22269 7432 Acked Read 6 Sent Done Done false
22270 7433 Acked Write 1 Sent Done Done false
22271 7434 Acked Read 2 Sent Done Done false
22272 7435 Acked Flush 0 Sent Done Done false
22273 7436 Acked Write 9 Sent Done Done false
22274 7437 Acked Write 9 Sent Done Done false
22275 7438 Acked Read 8 Sent Done Done false
22276 7439 Acked Write 10 Sent Done Done false
22277 7440 Acked Write 8 Sent Done Done false
22278 7441 Acked Read 5 Sent Done Done false
22279 7442 Acked Read 2 Sent Done Done false
22280 7443 Acked Read 4 Sent Done Done false
22281 7444 Acked Read 7 Sent Done Done false
22282 7445 Acked Read 10 Sent Done Done false
22283 7446 Acked Read 7 Sent Done Done false
22284 7447 Acked Write 4 Sent Done Done false
22285 7448 Acked Write 3 Sent Done Done false
22286 7449 Acked Write 7 Sent Done Done false
22287 7450 Acked Write 7 Sent Done Done false
22288 7451 Acked Read 6 Sent Done Done false
22289 7452 Acked Read 1 Sent Done Done false
22290 7453 Acked Read 5 Sent Done Done false
22291 7454 Acked Read 5 Sent Done Done false
22292 7455 Acked Read 10 Sent Done Done false
22293 7456 Acked Write 6 Sent Done Done false
22294 7457 Acked Read 6 Sent Done Done false
22295 7458 Acked Read 9 Sent Done Done false
22296 7459 Acked Read 1 Sent Done Done false
22297 7460 Acked Write 4 Sent Done Done false
22298 7461 Acked Read 4 Sent Done Done false
22299 7462 Acked Write 2 Sent Done Done false
22300 7463 Acked Read 8 Sent Done Done false
22301 7464 Acked Read 7 Sent Done Done false
22302 7465 Acked Read 4 Sent Done Done false
22303 7466 Acked Write 3 Sent Done Done false
22304 7467 Acked Read 3 Sent Done Done false
22305 7468 Acked Read 8 Sent Done Done false
22306 7469 Acked Write 2 Sent Done Done false
22307 7470 Acked Write 2 Sent Done Done false
22308 7471 Acked Read 1 Sent Done Done false
22309 7472 Acked Read 10 Sent Done Done false
22310 7473 Acked Read 6 Sent Done Done false
22311 7474 Acked Flush 0 Sent Done Done false
22312 7475 Acked Write 9 Sent Done Done false
22313 7476 Acked Write 9 Sent Done Done false
22314 7477 Acked Write 2 Sent Done Done false
22315 7478 Acked Write 9 Sent Done Done false
22316 7479 Acked Write 10 Sent Done Done false
22317 7480 Acked Read 2 Sent Done Done false
22318 7481 Acked Flush 0 Sent Done Done false
22319 7482 Acked Write 7 Sent Done Done false
22320 7483 Acked Write 6 Sent Done Done false
22321 7484 Acked Read 6 Sent Done Done false
22322 7485 Acked Write 1 Sent Done Done false
22323 7486 Acked Write 7 Sent Done Done false
22324 7487 Acked Write 8 Sent Done Done false
22325 7488 Acked Flush 0 Sent Done Done false
22326 7489 Acked Read 8 Sent Done Done false
22327 7490 Acked Read 1 Sent Done Done false
22328 7491 Acked Write 7 Sent Done Done false
22329 7492 Acked Write 9 Sent Done Done false
22330 7493 Acked Read 2 Sent Done Done false
22331 7494 Acked Write 9 Sent Done Done false
22332 7495 Acked Read 4 Sent Done Done false
22333 7496 Acked Read 9 Sent Done Done false
22334 7497 Acked Flush 0 Sent Done Done false
22335 7498 Acked Read 7 Sent Done Done false
22336 7499 Acked Flush 0 Sent Done Done false
22337 7500 Acked Read 9 Sent Done Done false
22338 7501 Acked Write 3 Sent Done Done false
22339 7502 Acked Flush 0 Sent Done Done false
22340 7503 Acked Write 4 Sent Done Done false
22341 7504 Acked Read 5 Sent Done Done false
22342 7505 Acked Flush 0 Sent Done Done false
22343 7506 Acked Write 10 Sent Done Done false
22344 7507 Acked Write 7 Sent Done Done false
22345 7508 Acked Read 2 Sent Done Done false
22346 7509 Acked Write 5 Sent Done Done false
22347 7510 Acked Read 7 Sent Done Done false
22348 7511 Acked Write 4 Sent Done Done false
22349 7512 Acked Write 4 Sent Done Done false
22350 7513 Acked Read 2 Sent Done Done false
22351 7514 Acked Read 3 Sent Done Done false
22352 7515 Acked Flush 0 Sent Done Done false
22353 7516 Acked Write 2 Sent Done Done false
22354 7517 Acked Write 7 Sent Done Done false
22355 7518 Acked Write 10 Sent Done Done false
22356 7519 Acked Write 10 Sent Done Done false
22357 7520 Acked Write 2 Sent Done Done false
22358 7521 Acked Read 1 Sent Done Done false
22359 7522 Acked Read 3 Sent Done Done false
22360 7523 Acked Write 9 Sent Done Done false
22361 7524 Acked Write 10 Sent Done Done false
22362 7525 Acked Flush 0 Sent Done Done false
22363 7526 Acked Read 6 Sent Done Done false
22364 7527 Acked Write 3 Sent Done Done false
22365 7528 Acked Read 3 Sent Done Done false
22366 7529 Acked Read 9 Sent Done Done false
22367 7530 Acked Read 10 Sent Done Done false
22368 7531 Acked Write 5 Sent Done Done false
22369 7532 Acked Read 10 Sent Done Done false
22370 7533 Acked Write 3 Sent Done Done false
22371 7534 Acked Read 7 Sent Done Done false
22372 7535 Acked Read 3 Sent Done Done false
22373 7536 Acked Flush 0 Sent Done Done false
22374 7537 Acked Write 4 Sent Done Done false
22375 7538 Acked Read 4 Sent Done Done false
22376 7539 Acked Write 2 Sent Done Done false
22377 7540 Acked Read 7 Sent Done Done false
22378 7541 Acked Write 6 Sent Done Done false
22379 7542 Acked Write 3 Sent Done Done false
22380 7543 Acked Read 10 Sent Done Done false
22381 7544 Acked Read 7 Sent Done Done false
22382 7545 Acked Write 7 Sent Done Done false
22383 7546 Acked Flush 0 Sent Done Done false
22384 7547 Acked Write 7 Sent Done Done false
22385 7548 Acked Write 3 Sent Done Done false
22386 7549 Acked Read 5 Sent Done Done false
22387 7550 Acked Read 10 Sent Done Done false
22388 7551 Acked Read 8 Sent Done Done false
22389 7552 Acked Read 2 Sent Done Done false
22390 7553 Acked Write 3 Sent Done Done false
22391 7554 Acked Write 4 Sent Done Done false
22392 7555 Acked Write 9 Sent Done Done false
22393 7556 Acked Write 8 Sent Done Done false
22394 7557 Acked Read 2 Sent Done Done false
22395 7558 Acked Read 2 Sent Done Done false
22396 7559 Acked Read 1 Sent Done Done false
22397 7560 Acked Read 2 Sent Done Done false
22398 7561 Acked Flush 0 Sent Done Done false
22399 7562 Acked Read 3 Sent Done Done false
22400 7563 Acked Read 6 Sent Done Done false
22401 7564 Acked Read 3 Sent Done Done false
22402 7565 Acked Flush 0 Sent Done Done false
22403 7566 Acked Flush 0 Sent Done Done false
22404 7567 Acked Write 3 Sent Done Done false
22405 7568 Acked Read 6 Sent Done Done false
22406 7569 Acked Read 2 Sent Done Done false
22407 7570 Acked Read 4 Sent Done Done false
22408 7571 Acked Read 6 Sent Done Done false
22409 7572 Acked Read 10 Sent Done Done false
22410 7573 Acked Write 9 Sent Done Done false
22411 7574 Acked Write 4 Sent Done Done false
22412 7575 Acked Read 10 Sent Done Done false
22413 7576 Acked Read 7 Sent Done Done false
22414 7577 Acked Flush 0 Sent Done Done false
22415 7578 Acked Read 4 Sent Done Done false
22416 7579 Acked Read 4 Sent Done Done false
22417 7580 Acked Read 2 Sent Done Done false
22418 7581 Acked Flush 0 Sent Done Done false
22419 7582 Acked Read 9 Sent Done Done false
22420 7583 Acked Read 1 Sent Done Done false
22421 7584 Acked Read 2 Sent Done Done false
22422 7585 Acked Read 10 Sent Done Done false
22423 7586 Acked Write 2 Sent Done Done false
22424 7587 Acked Write 5 Sent Done Done false
22425 7588 Acked Read 9 Sent Done Done false
22426 7589 Acked Write 5 Sent Done Done false
22427 7590 Acked Read 5 Sent Done Done false
22428 7591 Acked Read 9 Sent Done Done false
22429 7592 Acked Read 5 Sent Done Done false
22430 7593 Acked Read 9 Sent Done Done false
22431 7594 Acked Flush 0 Sent Done Done false
22432 7595 Acked Read 10 Sent Done Done false
22433 7596 Acked Write 4 Sent Done Done false
22434 7597 Acked Read 2 Sent Done Done false
22435 7598 Acked Read 8 Sent Done Done false
22436 7599 Acked Read 9 Sent Done Done false
22437 7600 Acked Flush 0 Sent Done Done false
22438 7601 Acked Read 5 Sent Done Done false
22439 7602 Acked Read 8 Sent Done Done false
22440 7603 Acked Read 1 Sent Done Done false
22441 7604 Acked Read 6 Sent Done Done false
22442 7605 Acked Write 6 Sent Done Done false
22443 7606 Acked Read 1 Sent Done Done false
22444 7607 Acked Read 8 Sent Done Done false
22445 7608 Acked Read 2 Sent Done Done false
22446 7609 Acked Read 4 Sent Done Done false
22447 7610 Acked Read 1 Sent Done Done false
22448 7611 Acked Write 2 Sent Done Done false
22449 7612 Acked Read 7 Sent Done Done false
22450 7613 Acked Write 2 Sent Done Done false
22451 7614 Acked Read 5 Sent Done Done false
22452 7615 Acked Read 4 Sent Done Done false
22453 7616 Acked Write 7 Sent Done Done false
22454 7617 Acked Write 6 Sent Done Done false
22455 7618 Acked Write 5 Sent Done Done false
22456 7619 Acked Write 1 Sent Done Done false
22457 7620 Acked Write 4 Sent Done Done false
22458 7621 Acked Read 4 Sent Done Done false
22459 7622 Acked Read 5 Sent Done Done false
22460 7623 Acked Write 5 Sent Done Done false
22461 7624 Acked Read 5 Sent Done Done false
22462 7625 Acked Read 3 Sent Done Done false
22463 7626 Acked Read 1 Sent Done Done false
22464 7627 Acked Write 7 Sent Done Done false
22465 7628 Acked Write 7 Sent Done Done false
22466 7629 Acked Read 8 Sent Done Done false
22467 7630 Acked Read 1 Sent Done Done false
22468 7631 Acked Write 1 Sent Done Done false
22469 7632 Acked Read 3 Sent Done Done false
22470 7633 Acked Write 10 Sent Done Done false
22471 7634 Acked Read 10 Sent Done Done false
22472 7635 Acked Read 5 Sent Done Done false
22473 7636 Acked Write 4 Sent Done Done false
22474 7637 Acked Read 5 Sent Done Done false
22475 7638 Acked Read 2 Sent Done Done false
22476 7639 Acked Write 8 Sent Done Done false
22477 STATES DS:0 DS:1 DS:2 TOTAL
22478 Sent 301 0 0 301
22479 Done 0 301 301 602
22480 Skipped 0 0 0 0
22481 Error 0 0 0 0
22482 Last Flush: 7338 7600 7600
22483 Downstairs last five completed: 7338 7337 7336 7335 7334
22484 Upstairs last five completed: 7639 7638 7637 7636 7635
22485 CLIENT: Up:0 ds:301 act:2
22486 [8810] got start action
22487 [8810] I am going to restart
22488 Starting downstairs at port 8810
22489 Make output file at "/tmp/dsc/downstairs-8810.txt"
22490 [8810][0] reports Starting
22491 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1224)
22492 [8810][0] reports Running
22493 ----------------------------------------------------------------
22494 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22495 States: Offline Active Active
22496 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22497 7339 Acked Read 5 Sent Done Done false
22498 7340 Acked Read 7 Sent Done Done false
22499 7341 Acked Write 3 Sent Done Done false
22500 7342 Acked Write 10 Sent Done Done false
22501 7343 Acked Write 4 Sent Done Done false
22502 7344 Acked Read 5 Sent Done Done false
22503 7345 Acked Write 1 Sent Done Done false
22504 7346 Acked Write 6 Sent Done Done false
22505 7347 Acked Read 3 Sent Done Done false
22506 7348 Acked Flush 0 Sent Done Done false
22507 7349 Acked Read 10 Sent Done Done false
22508 7350 Acked Read 7 Sent Done Done false
22509 7351 Acked Write 3 Sent Done Done false
22510 7352 Acked Write 8 Sent Done Done false
22511 7353 Acked Read 4 Sent Done Done false
22512 7354 Acked Read 9 Sent Done Done false
22513 7355 Acked Read 2 Sent Done Done false
22514 7356 Acked Write 4 Sent Done Done false
22515 7357 Acked Read 1 Sent Done Done false
22516 7358 Acked Write 8 Sent Done Done false
22517 7359 Acked Flush 0 Sent Done Done false
22518 7360 Acked Write 2 Sent Done Done false
22519 7361 Acked Read 9 Sent Done Done false
22520 7362 Acked Read 8 Sent Done Done false
22521 7363 Acked Write 7 Sent Done Done false
22522 7364 Acked Read 4 Sent Done Done false
22523 7365 Acked Write 1 Sent Done Done false
22524 7366 Acked Read 6 Sent Done Done false
22525 7367 Acked Flush 0 Sent Done Done false
22526 7368 Acked Read 6 Sent Done Done false
22527 7369 Acked Write 9 Sent Done Done false
22528 7370 Acked Write 4 Sent Done Done false
22529 7371 Acked Read 10 Sent Done Done false
22530 7372 Acked Flush 0 Sent Done Done false
22531 7373 Acked Read 7 Sent Done Done false
22532 7374 Acked Read 1 Sent Done Done false
22533 7375 Acked Read 7 Sent Done Done false
22534 7376 Acked Read 7 Sent Done Done false
22535 7377 Acked Write 1 Sent Done Done false
22536 7378 Acked Read 1 Sent Done Done false
22537 7379 Acked Write 6 Sent Done Done false
22538 7380 Acked Write 9 Sent Done Done false
22539 7381 Acked Flush 0 Sent Done Done false
22540 7382 Acked Read 8 Sent Done Done false
22541 7383 Acked Flush 0 Sent Done Done false
22542 7384 Acked Write 6 Sent Done Done false
22543 7385 Acked Read 10 Sent Done Done false
22544 7386 Acked Read 1 Sent Done Done false
22545 7387 Acked Read 1 Sent Done Done false
22546 7388 Acked Write 7 Sent Done Done false
22547 7389 Acked Read 5 Sent Done Done false
22548 7390 Acked Write 5 Sent Done Done false
22549 7391 Acked Write 7 Sent Done Done false
22550 7392 Acked Write 6 Sent Done Done false
22551 7393 Acked Write 5 Sent Done Done false
22552 7394 Acked Write 3 Sent Done Done false
22553 7395 Acked Read 8 Sent Done Done false
22554 7396 Acked Read 7 Sent Done Done false
22555 7397 Acked Read 2 Sent Done Done false
22556 7398 Acked Read 5 Sent Done Done false
22557 7399 Acked Flush 0 Sent Done Done false
22558 7400 Acked Write 1 Sent Done Done false
22559 7401 Acked Write 1 Sent Done Done false
22560 7402 Acked Read 7 Sent Done Done false
22561 7403 Acked Read 5 Sent Done Done false
22562 7404 Acked Write 6 Sent Done Done false
22563 7405 Acked Read 10 Sent Done Done false
22564 7406 Acked Write 7 Sent Done Done false
22565 7407 Acked Write 8 Sent Done Done false
22566 7408 Acked Write 1 Sent Done Done false
22567 7409 Acked Read 7 Sent Done Done false
22568 7410 Acked Read 10 Sent Done Done false
22569 7411 Acked Read 9 Sent Done Done false
22570 7412 Acked Read 2 Sent Done Done false
22571 7413 Acked Read 6 Sent Done Done false
22572 7414 Acked Write 10 Sent Done Done false
22573 7415 Acked Write 5 Sent Done Done false
22574 7416 Acked Read 6 Sent Done Done false
22575 7417 Acked Write 9 Sent Done Done false
22576 7418 Acked Read 7 Sent Done Done false
22577 7419 Acked Write 7 Sent Done Done false
22578 7420 Acked Read 10 Sent Done Done false
22579 7421 Acked Write 7 Sent Done Done false
22580 7422 Acked Write 9 Sent Done Done false
22581 7423 Acked Read 6 Sent Done Done false
22582 7424 Acked Write 5 Sent Done Done false
22583 7425 Acked Read 10 Sent Done Done false
22584 7426 Acked Write 5 Sent Done Done false
22585 7427 Acked Read 6 Sent Done Done false
22586 7428 Acked Read 8 Sent Done Done false
22587 7429 Acked Write 10 Sent Done Done false
22588 7430 Acked Read 6 Sent Done Done false
22589 7431 Acked Read 3 Sent Done Done false
22590 7432 Acked Read 6 Sent Done Done false
22591 7433 Acked Write 1 Sent Done Done false
22592 7434 Acked Read 2 Sent Done Done false
22593 7435 Acked Flush 0 Sent Done Done false
22594 7436 Acked Write 9 Sent Done Done false
22595 7437 Acked Write 9 Sent Done Done false
22596 7438 Acked Read 8 Sent Done Done false
22597 7439 Acked Write 10 Sent Done Done false
22598 7440 Acked Write 8 Sent Done Done false
22599 7441 Acked Read 5 Sent Done Done false
22600 7442 Acked Read 2 Sent Done Done false
22601 7443 Acked Read 4 Sent Done Done false
22602 7444 Acked Read 7 Sent Done Done false
22603 7445 Acked Read 10 Sent Done Done false
22604 7446 Acked Read 7 Sent Done Done false
22605 7447 Acked Write 4 Sent Done Done false
22606 7448 Acked Write 3 Sent Done Done false
22607 7449 Acked Write 7 Sent Done Done false
22608 7450 Acked Write 7 Sent Done Done false
22609 7451 Acked Read 6 Sent Done Done false
22610 7452 Acked Read 1 Sent Done Done false
22611 7453 Acked Read 5 Sent Done Done false
22612 7454 Acked Read 5 Sent Done Done false
22613 7455 Acked Read 10 Sent Done Done false
22614 7456 Acked Write 6 Sent Done Done false
22615 7457 Acked Read 6 Sent Done Done false
22616 7458 Acked Read 9 Sent Done Done false
22617 7459 Acked Read 1 Sent Done Done false
22618 7460 Acked Write 4 Sent Done Done false
22619 7461 Acked Read 4 Sent Done Done false
22620 7462 Acked Write 2 Sent Done Done false
22621 7463 Acked Read 8 Sent Done Done false
22622 7464 Acked Read 7 Sent Done Done false
22623 7465 Acked Read 4 Sent Done Done false
22624 7466 Acked Write 3 Sent Done Done false
22625 7467 Acked Read 3 Sent Done Done false
22626 7468 Acked Read 8 Sent Done Done false
22627 7469 Acked Write 2 Sent Done Done false
22628 7470 Acked Write 2 Sent Done Done false
22629 7471 Acked Read 1 Sent Done Done false
22630 7472 Acked Read 10 Sent Done Done false
22631 7473 Acked Read 6 Sent Done Done false
22632 7474 Acked Flush 0 Sent Done Done false
22633 7475 Acked Write 9 Sent Done Done false
22634 7476 Acked Write 9 Sent Done Done false
22635 7477 Acked Write 2 Sent Done Done false
22636 7478 Acked Write 9 Sent Done Done false
22637 7479 Acked Write 10 Sent Done Done false
22638 7480 Acked Read 2 Sent Done Done false
22639 7481 Acked Flush 0 Sent Done Done false
22640 7482 Acked Write 7 Sent Done Done false
22641 7483 Acked Write 6 Sent Done Done false
22642 7484 Acked Read 6 Sent Done Done false
22643 7485 Acked Write 1 Sent Done Done false
22644 7486 Acked Write 7 Sent Done Done false
22645 7487 Acked Write 8 Sent Done Done false
22646 7488 Acked Flush 0 Sent Done Done false
22647 7489 Acked Read 8 Sent Done Done false
22648 7490 Acked Read 1 Sent Done Done false
22649 7491 Acked Write 7 Sent Done Done false
22650 7492 Acked Write 9 Sent Done Done false
22651 7493 Acked Read 2 Sent Done Done false
22652 7494 Acked Write 9 Sent Done Done false
22653 7495 Acked Read 4 Sent Done Done false
22654 7496 Acked Read 9 Sent Done Done false
22655 7497 Acked Flush 0 Sent Done Done false
22656 7498 Acked Read 7 Sent Done Done false
22657 7499 Acked Flush 0 Sent Done Done false
22658 7500 Acked Read 9 Sent Done Done false
22659 7501 Acked Write 3 Sent Done Done false
22660 7502 Acked Flush 0 Sent Done Done false
22661 7503 Acked Write 4 Sent Done Done false
22662 7504 Acked Read 5 Sent Done Done false
22663 7505 Acked Flush 0 Sent Done Done false
22664 7506 Acked Write 10 Sent Done Done false
22665 7507 Acked Write 7 Sent Done Done false
22666 7508 Acked Read 2 Sent Done Done false
22667 7509 Acked Write 5 Sent Done Done false
22668 7510 Acked Read 7 Sent Done Done false
22669 7511 Acked Write 4 Sent Done Done false
22670 7512 Acked Write 4 Sent Done Done false
22671 7513 Acked Read 2 Sent Done Done false
22672 7514 Acked Read 3 Sent Done Done false
22673 7515 Acked Flush 0 Sent Done Done false
22674 7516 Acked Write 2 Sent Done Done false
22675 7517 Acked Write 7 Sent Done Done false
22676 7518 Acked Write 10 Sent Done Done false
22677 7519 Acked Write 10 Sent Done Done false
22678 7520 Acked Write 2 Sent Done Done false
22679 7521 Acked Read 1 Sent Done Done false
22680 7522 Acked Read 3 Sent Done Done false
22681 7523 Acked Write 9 Sent Done Done false
22682 7524 Acked Write 10 Sent Done Done false
22683 7525 Acked Flush 0 Sent Done Done false
22684 7526 Acked Read 6 Sent Done Done false
22685 7527 Acked Write 3 Sent Done Done false
22686 7528 Acked Read 3 Sent Done Done false
22687 7529 Acked Read 9 Sent Done Done false
22688 7530 Acked Read 10 Sent Done Done false
22689 7531 Acked Write 5 Sent Done Done false
22690 7532 Acked Read 10 Sent Done Done false
22691 7533 Acked Write 3 Sent Done Done false
22692 7534 Acked Read 7 Sent Done Done false
22693 7535 Acked Read 3 Sent Done Done false
22694 7536 Acked Flush 0 Sent Done Done false
22695 7537 Acked Write 4 Sent Done Done false
22696 7538 Acked Read 4 Sent Done Done false
22697 7539 Acked Write 2 Sent Done Done false
22698 7540 Acked Read 7 Sent Done Done false
22699 7541 Acked Write 6 Sent Done Done false
22700 7542 Acked Write 3 Sent Done Done false
22701 7543 Acked Read 10 Sent Done Done false
22702 7544 Acked Read 7 Sent Done Done false
22703 7545 Acked Write 7 Sent Done Done false
22704 7546 Acked Flush 0 Sent Done Done false
22705 7547 Acked Write 7 Sent Done Done false
22706 7548 Acked Write 3 Sent Done Done false
22707 7549 Acked Read 5 Sent Done Done false
22708 7550 Acked Read 10 Sent Done Done false
22709 7551 Acked Read 8 Sent Done Done false
22710 7552 Acked Read 2 Sent Done Done false
22711 7553 Acked Write 3 Sent Done Done false
22712 7554 Acked Write 4 Sent Done Done false
22713 7555 Acked Write 9 Sent Done Done false
22714 7556 Acked Write 8 Sent Done Done false
22715 7557 Acked Read 2 Sent Done Done false
22716 7558 Acked Read 2 Sent Done Done false
22717 7559 Acked Read 1 Sent Done Done false
22718 7560 Acked Read 2 Sent Done Done false
22719 7561 Acked Flush 0 Sent Done Done false
22720 7562 Acked Read 3 Sent Done Done false
22721 7563 Acked Read 6 Sent Done Done false
22722 7564 Acked Read 3 Sent Done Done false
22723 7565 Acked Flush 0 Sent Done Done false
22724 7566 Acked Flush 0 Sent Done Done false
22725 7567 Acked Write 3 Sent Done Done false
22726 7568 Acked Read 6 Sent Done Done false
22727 7569 Acked Read 2 Sent Done Done false
22728 7570 Acked Read 4 Sent Done Done false
22729 7571 Acked Read 6 Sent Done Done false
22730 7572 Acked Read 10 Sent Done Done false
22731 7573 Acked Write 9 Sent Done Done false
22732 7574 Acked Write 4 Sent Done Done false
22733 7575 Acked Read 10 Sent Done Done false
22734 7576 Acked Read 7 Sent Done Done false
22735 7577 Acked Flush 0 Sent Done Done false
22736 7578 Acked Read 4 Sent Done Done false
22737 7579 Acked Read 4 Sent Done Done false
22738 7580 Acked Read 2 Sent Done Done false
22739 7581 Acked Flush 0 Sent Done Done false
22740 7582 Acked Read 9 Sent Done Done false
22741 7583 Acked Read 1 Sent Done Done false
22742 7584 Acked Read 2 Sent Done Done false
22743 7585 Acked Read 10 Sent Done Done false
22744 7586 Acked Write 2 Sent Done Done false
22745 7587 Acked Write 5 Sent Done Done false
22746 7588 Acked Read 9 Sent Done Done false
22747 7589 Acked Write 5 Sent Done Done false
22748 7590 Acked Read 5 Sent Done Done false
22749 7591 Acked Read 9 Sent Done Done false
22750 7592 Acked Read 5 Sent Done Done false
22751 7593 Acked Read 9 Sent Done Done false
22752 7594 Acked Flush 0 Sent Done Done false
22753 7595 Acked Read 10 Sent Done Done false
22754 7596 Acked Write 4 Sent Done Done false
22755 7597 Acked Read 2 Sent Done Done false
22756 7598 Acked Read 8 Sent Done Done false
22757 7599 Acked Read 9 Sent Done Done false
22758 7600 Acked Flush 0 Sent Done Done false
22759 7601 Acked Read 5 Sent Done Done false
22760 7602 Acked Read 8 Sent Done Done false
22761 7603 Acked Read 1 Sent Done Done false
22762 7604 Acked Read 6 Sent Done Done false
22763 7605 Acked Write 6 Sent Done Done false
22764 7606 Acked Read 1 Sent Done Done false
22765 7607 Acked Read 8 Sent Done Done false
22766 7608 Acked Read 2 Sent Done Done false
22767 7609 Acked Read 4 Sent Done Done false
22768 7610 Acked Read 1 Sent Done Done false
22769 7611 Acked Write 2 Sent Done Done false
22770 7612 Acked Read 7 Sent Done Done false
22771 7613 Acked Write 2 Sent Done Done false
22772 7614 Acked Read 5 Sent Done Done false
22773 7615 Acked Read 4 Sent Done Done false
22774 7616 Acked Write 7 Sent Done Done false
22775 7617 Acked Write 6 Sent Done Done false
22776 7618 Acked Write 5 Sent Done Done false
22777 7619 Acked Write 1 Sent Done Done false
22778 7620 Acked Write 4 Sent Done Done false
22779 7621 Acked Read 4 Sent Done Done false
22780 7622 Acked Read 5 Sent Done Done false
22781 7623 Acked Write 5 Sent Done Done false
22782 7624 Acked Read 5 Sent Done Done false
22783 7625 Acked Read 3 Sent Done Done false
22784 7626 Acked Read 1 Sent Done Done false
22785 7627 Acked Write 7 Sent Done Done false
22786 7628 Acked Write 7 Sent Done Done false
22787 7629 Acked Read 8 Sent Done Done false
22788 7630 Acked Read 1 Sent Done Done false
22789 7631 Acked Write 1 Sent Done Done false
22790 7632 Acked Read 3 Sent Done Done false
22791 7633 Acked Write 10 Sent Done Done false
22792 7634 Acked Read 10 Sent Done Done false
22793 7635 Acked Read 5 Sent Done Done false
22794 7636 Acked Write 4 Sent Done Done false
22795 7637 Acked Read 5 Sent Done Done false
22796 7638 Acked Read 2 Sent Done Done false
22797 7639 Acked Write 8 Sent Done Done false
22798 7640 Acked Flush 0 Sent Done Done false
22799 STATES DS:0 DS:1 DS:2 TOTAL
22800 Sent 302 0 0 302
22801 Done 0 302 302 604
22802 Skipped 0 0 0 0
22803 Error 0 0 0 0
22804 Last Flush: 7338 7640 7640
22805 Downstairs last five completed: 7338 7337 7336 7335 7334
22806 Upstairs last five completed: 7640 7639 7638 7637 7636
22807 CLIENT: Up:0 ds:302 act:2
228082025-01-14T16:15:23.055ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
228092025-01-14T16:15:23.055ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57954) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
228102025-01-14T16:15:23.056ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
228112025-01-14T16:15:23.057ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
228122025-01-14T16:15:23.057ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
228132025-01-14T16:15:23.057ZINFOcrucible: send last flush ID to this DS: 7338 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
228142025-01-14T16:15:23.057ZINFOcrucible: Replied this last flush ID: 7338 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
228152025-01-14T16:15:23.057ZINFOcrucible: [0] client re-new 302 jobs since flush 7338 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
228162025-01-14T16:15:23.057ZINFOcrucible: [0] Replayed 302 jobs since flush: 7338 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
22817 ----------------------------------------------------------------
22818 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
22819 Downstairs last five completed: 7640 7639 7638 7637 7636
22820 Upstairs last five completed: 7640 7639 7638 7637 7636
22821 CLIENT: Up:0 ds:0 act:3
22822 Replay: All jobs finished
22823 Main task has work to do, go find it
22824 got dsc Stop(2)
22825 stop 2
22826 Jan 14 16:15:25.301 INFO request completed, latency_us: 315, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 4bb33627-4b5f-4d19-8b24-3aa63958cf5d, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
22827 [8830] Got stop action so:false kr:false
22828 Jan 14 16:15:25.302 INFO request completed, latency_us: 147, response_code: 200, uri: /state/cid/2, method: GET, req_id: 0f370314-83ef-488e-b6f2-2c112af70eda, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
228292025-01-14T16:15:25.302ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
228302025-01-14T16:15:25.302ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
228312025-01-14T16:15:25.302ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
228322025-01-14T16:15:25.302ZINFOcrucible: client task is exiting = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
22833 [8830] Exited with: ExitStatus(unix_wait_status(9))
22834 [8830][2] reports Exit
22835 Jan 14 16:15:29.303 INFO request completed, latency_us: 197, response_code: 200, uri: /state/cid/2, method: GET, req_id: 295d48b4-046d-4b54-ae59-68e8234b0ac1, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
22836 001/300 Write block 1793 len 4096 data: 4
22837 002/300 Read block 2747 len 8192
22838 003/300 Read block 601 len 28672
22839 004/300 Read block 3922 len 12288
22840 005/300 Write block 572 len 16384 data: 3 4 5 5
22841 006/300 Write block 2482 len 24576 data: 7 5 5 5 5 4
22842 007/300 Write block 3276 len 12288 data: 4 4 4
22843 008/300 Read block 2512 len 28672
22844 009/300 Write block 707 len 20480 data: 6 5 5 5 5
22845 010/300 Read block 2306 len 8192
22846 011/300 Write block 1728 len 32768 data: 6 5 5 5 4 4 4 4
22847 012/300 Write block 3834 len 16384 data: 5 5 6 5
22848 013/300 Write block 2583 len 16384 data: 5 5 5 4
22849 014/300 Read block 4197 len 24576
22850 015/300 Flush
22851 016/300 Read block 2175 len 24576
22852 017/300 Write block 3663 len 8192 data: 6 5
22853 018/300 Write block 181 len 4096 data: 5
22854 019/300 Write block 903 len 16384 data: 7 8 7 7
22855 020/300 Write block 3445 len 4096 data: 2
22856 021/300 Write block 189 len 32768 data: 3 3 2 2 3 3 4 4
22857 022/300 Read block 3799 len 12288
22858 023/300 Write block 681 len 20480 data: 5 6 6 6 6
22859 024/300 Flush
22860 025/300 Write block 1661 len 32768 data: 8 7 6 5 4 4 5 4
22861 026/300 Read block 445 len 12288
22862 027/300 Read block 1702 len 24576
22863 028/300 Write block 2556 len 4096 data: 6
22864 029/300 Read block 3828 len 8192
22865 030/300 Read block 2312 len 8192
22866 031/300 Write block 1488 len 40960 data: 5 6 7 5 5 5 6 7 7 6
22867 032/300 Write block 1839 len 24576 data: 5 5 6 5 5 6
22868 033/300 Write block 864 len 32768 data: 3 3 3 3 3 3 4 4
22869 034/300 Flush
22870 035/300 Write block 2679 len 20480 data: 5 4 5 5 5
22871 036/300 Write block 3371 len 20480 data: 6 6 6 5 3
22872 037/300 Read block 916 len 24576
22873 038/300 Write block 2555 len 36864 data: 5 7 3 4 5 5 6 7 7
22874 039/300 Flush
22875 040/300 Read block 2458 len 16384
22876 041/300 Read block 3884 len 24576
22877 042/300 Read block 2341 len 28672
22878 043/300 Read block 2931 len 32768
22879 044/300 Write block 445 len 32768 data: 6 5 6 7 9 6 7 6
22880 045/300 Read block 2712 len 28672
22881 046/300 Read block 1670 len 16384
22882 047/300 Write block 4045 len 12288 data: 6 7 6
22883 048/300 Read block 4572 len 12288
22884 049/300 Read block 631 len 20480
22885 050/300 Read block 3320 len 16384
22886 051/300 Read block 1188 len 32768
22887 052/300 Read block 2298 len 12288
22888 053/300 Write block 4250 len 12288 data: 5 5 3
22889 054/300 Write block 744 len 24576 data: 4 4 4 3 3 4
22890 055/300 Read block 2397 len 20480
22891 056/300 Read block 4649 len 20480
22892 057/300 Read block 1907 len 24576
22893 058/300 Read block 3246 len 36864
22894 059/300 Write block 3625 len 8192 data: 6 5
22895 060/300 Read block 1985 len 20480
22896 061/300 Write block 3830 len 20480 data: 6 7 7 7 6
22897 062/300 Read block 1032 len 40960
22898 063/300 Read block 838 len 12288
22899 064/300 Read block 3034 len 12288
22900 065/300 Read block 4313 len 40960
22901 066/300 Write block 4971 len 32768 data: 5 5 5 6 7 7 7 6
22902 067/300 Read block 994 len 8192
22903 068/300 Read block 4295 len 4096
22904 069/300 Write block 4663 len 20480 data: 10 10 12 12 11
22905 070/300 Write block 3157 len 32768 data: 8 6 5 5 5 3 3 3
22906 071/300 Read block 4435 len 12288
22907 072/300 Write block 4677 len 32768 data: 5 5 5 7 5 5 5 4
22908 073/300 Write block 3401 len 40960 data: 8 6 6 5 5 4 4 4 4 3
22909 074/300 Write block 4004 len 40960 data: 7 7 7 9 8 6 8 7 6 6
22910 075/300 Read block 1537 len 8192
22911 076/300 Read block 962 len 36864
22912 077/300 Read block 2803 len 4096
22913 078/300 Read block 664 len 20480
22914 079/300 Read block 2412 len 24576
22915 080/300 Read block 2984 len 4096
22916 081/300 Read block 4134 len 32768
22917 082/300 Flush
22918 083/300 Read block 1407 len 16384
22919 084/300 Read block 3376 len 32768
22920 085/300 Write block 666 len 28672 data: 4 4 4 6 6 7 7
22921 086/300 Read block 4352 len 20480
22922 087/300 Read block 3355 len 36864
22923 088/300 Write block 4910 len 40960 data: 3 2 3 4 5 5 6 7 6 6
22924 089/300 Write block 4972 len 36864 data: 6 6 7 8 8 8 7 5 6
22925 090/300 Write block 1733 len 32768 data: 5 5 5 3 4 4 4 4
22926 091/300 Write block 1886 len 28672 data: 3 4 5 7 8 9 8
22927 092/300 Flush
22928 093/300 Write block 4825 len 8192 data: 5 5
22929 094/300 Read block 101 len 4096
22930 095/300 Flush
22931 096/300 Write block 1902 len 28672 data: 5 5 4 5 5 5 5
22932 097/300 Write block 3611 len 4096 data: 9
22933 098/300 Write block 2696 len 20480 data: 10 11 9 8 6
22934 099/300 Read block 848 len 4096
22935 100/300 Write block 4022 len 8192 data: 2 2
22936 101/300 Write block 209 len 32768 data: 6 5 5 6 6 7 6 4
22937 102/300 Write block 354 len 40960 data: 4 5 3 3 3 3 3 3 3 4
22938 103/300 Flush
22939 104/300 Read block 2348 len 40960
22940 105/300 Flush
22941 106/300 Write block 1615 len 28672 data: 4 4 6 6 6 5 5
22942 107/300 Read block 1629 len 8192
22943 108/300 Write block 908 len 12288 data: 8 8 8
22944 109/300 Write block 3588 len 16384 data: 4 3 2 3
22945 110/300 Write block 1445 len 4096 data: 8
22946 111/300 Write block 421 len 12288 data: 7 8 7
22947 112/300 Read block 4440 len 40960
22948 113/300 Read block 3139 len 8192
22949 114/300 Write block 2942 len 16384 data: 3 3 3 4
22950 115/300 Read block 2124 len 8192
22951 116/300 Write block 712 len 24576 data: 4 3 2 2 3 3
22952 117/300 Write block 3046 len 24576 data: 5 6 6 7 9 7
22953 118/300 Read block 522 len 36864
22954 119/300 Read block 3147 len 28672
22955 120/300 Read block 4096 len 20480
22956 121/300 Read block 3021 len 12288
22957 122/300 Flush
22958 123/300 Read block 4947 len 12288
22959 124/300 Read block 773 len 20480
22960 125/300 Read block 4370 len 24576
22961 126/300 Read block 661 len 8192
22962 127/300 Flush
22963 128/300 Read block 3607 len 32768
22964 129/300 Read block 4095 len 16384
22965 130/300 Read block 2553 len 16384
22966 131/300 Read block 1000 len 20480
22967 132/300 Read block 730 len 32768
22968 133/300 Write block 2657 len 24576 data: 4 4 3 4 5 5
22969 134/300 Read block 3993 len 24576
22970 135/300 Write block 1903 len 12288 data: 6 5 6
22971 136/300 Write block 461 len 32768 data: 6 7 7 6 5 4 5 3
22972 137/300 Read block 2086 len 8192
22973 138/300 Read block 1864 len 12288
22974 139/300 Write block 30 len 40960 data: 7 6 6 6 4 5 5 4 5 7
22975 140/300 Flush
22976 141/300 Write block 2866 len 32768 data: 3 3 5 5 9 9 9 9
22977 142/300 Read block 1506 len 40960
22978 143/300 Read block 2572 len 24576
22979 144/300 Read block 1092 len 16384
22980 145/300 Write block 2902 len 24576 data: 6 7 8 8 7 8
22981 146/300 Flush
22982 147/300 Read block 3058 len 28672
22983 148/300 Write block 3799 len 36864 data: 6 6 4 4 4 5 4 7 7
22984 149/300 Read block 1680 len 8192
22985 150/300 Write block 444 len 4096 data: 6
22986 151/300 Read block 501 len 24576
22987 152/300 Read block 1416 len 16384
22988 153/300 Read block 4683 len 36864
22989 154/300 Read block 1499 len 28672
22990 155/300 Read block 1845 len 40960
22991 156/300 Read block 3407 len 8192
22992 157/300 Read block 4349 len 16384
22993 158/300 Read block 4171 len 20480
22994 159/300 Read block 2003 len 40960
22995 160/300 Flush
22996 161/300 Read block 2500 len 36864
22997 162/300 Flush
22998 163/300 Write block 2384 len 40960 data: 3 3 3 3 3 5 6 6 6 6
22999 164/300 Read block 4396 len 4096
23000 165/300 Write block 134 len 20480 data: 6 6 6 6 6
23001 166/300 Read block 912 len 4096
23002 167/300 Flush
23003 168/300 Write block 15 len 24576 data: 5 5 4 5 5 5
23004 169/300 Write block 283 len 12288 data: 3 3 3
23005 170/300 Write block 2845 len 16384 data: 3 3 3 3
23006 171/300 Read block 3693 len 24576
23007 172/300 Write block 1427 len 8192 data: 5 3
23008 173/300 Read block 209 len 8192
23009 174/300 Read block 4223 len 20480
23010 175/300 Write block 2567 len 12288 data: 7 6 5
23011 176/300 Write block 1071 len 24576 data: 7 6 5 8 8 7
23012 177/300 Write block 918 len 12288 data: 3 4 5
23013 178/300 Read block 2106 len 24576
23014 179/300 Flush
23015 180/300 Write block 1068 len 8192 data: 5 5
23016 181/300 Read block 4129 len 36864
23017 182/300 Flush
23018 183/300 Write block 80 len 12288 data: 8 6 7
23019 184/300 Write block 57 len 8192 data: 5 7
23020 185/300 Flush
23021 186/300 Flush
23022 187/300 Read block 1407 len 12288
23023 188/300 Read block 2097 len 36864
23024 189/300 Write block 2239 len 4096 data: 6
23025 190/300 Write block 4646 len 4096 data: 6
23026 191/300 Read block 3375 len 4096
23027 192/300 Write block 3431 len 32768 data: 6 5 3 3 3 3 3 3
23028 193/300 Read block 366 len 28672
23029 194/300 Read block 3613 len 32768
23030 195/300 Write block 1101 len 40960 data: 6 3 2 2 2 2 2 2 3 3
23031 196/300 Write block 2262 len 32768 data: 5 5 4 4 3 3 5 4
23032 197/300 Write block 1412 len 24576 data: 3 4 4 4 4 5
23033 198/300 Flush
23034 199/300 Read block 2479 len 16384
23035 200/300 Read block 3318 len 32768
23036 201/300 Flush
23037 202/300 Read block 384 len 8192
23038 203/300 Write block 1577 len 40960 data: 4 4 3 3 3 3 3 3 3 3
23039 204/300 Flush
23040 205/300 Read block 2672 len 36864
23041 206/300 Read block 2012 len 16384
23042 207/300 Write block 1815 len 28672 data: 5 6 7 7 7 7 5
23043 208/300 Read block 541 len 16384
23044 209/300 Read block 3748 len 4096
23045 210/300 Read block 4787 len 36864
23046 211/300 Write block 3585 len 4096 data: 6
23047 212/300 Read block 1410 len 36864
23048 213/300 Read block 2529 len 40960
23049 214/300 Read block 16 len 24576
23050 215/300 Read block 4688 len 8192
23051 216/300 Read block 3537 len 20480
23052 217/300 Flush
23053 218/300 Read block 2562 len 12288
23054 219/300 Write block 3573 len 28672 data: 5 6 6 5 5 5 6
23055 220/300 Write block 2053 len 36864 data: 6 6 6 6 6 5 5 5 5
23056 221/300 Read block 4068 len 32768
23057 222/300 Read block 1881 len 8192
23058 223/300 Read block 2493 len 8192
23059 224/300 Flush
23060 225/300 Write block 1000 len 28672 data: 6 7 6 7 6 6 6
23061 226/300 Read block 4675 len 36864
23062 227/300 Write block 4846 len 32768 data: 10 8 7 7 7 4 3 4
23063 228/300 Read block 2802 len 4096
23064 229/300 Write block 1195 len 8192 data: 4 3
23065 230/300 Flush
23066 231/300 Write block 579 len 40960 data: 4 2 4 5 5 6 7 6 6 5
23067 232/300 Read block 1627 len 4096
23068 233/300 Read block 4229 len 40960
23069 234/300 Write block 2649 len 40960 data: 6 7 4 4 4 4 4 3 5 5
23070 235/300 Write block 6 len 40960 data: 3 3 4 4 4 2 3 4 6 6
23071 236/300 Read block 28 len 16384
23072 237/300 Write block 3565 len 12288 data: 4 4 6
23073 238/300 Read block 4376 len 16384
23074 239/300 Read block 3232 len 32768
23075 240/300 Write block 627 len 40960 data: 3 3 3 3 3 3 4 3 4 4
23076 241/300 Read block 132 len 24576
23077 242/300 Read block 2488 len 36864
23078 243/300 Write block 2866 len 20480 data: 4 4 6 6 10
23079 244/300 Write block 1435 len 24576 data: 6 4 4 6 7 7
23080 245/300 Read block 3283 len 20480
23081 246/300 Write block 644 len 12288 data: 5 6 5
23082 247/300 Read block 61 len 24576
23083 248/300 Read block 858 len 16384
23084 249/300 Read block 957 len 40960
23085 250/300 Write block 574 len 28672 data: 6 6 5 5 5 5 3
23086 251/300 Write block 3059 len 36864 data: 4 4 5 3 4 5 5 4 4
23087 252/300 Flush
23088 253/300 Read block 785 len 36864
23089 254/300 Read block 4168 len 24576
23090 255/300 Flush
23091 256/300 Write block 2346 len 20480 data: 3 2 3 2 2
23092 257/300 Read block 4239 len 28672
23093 258/300 Write block 1552 len 28672 data: 6 6 5 5 6 5 8
23094 259/300 Read block 1253 len 32768
23095 260/300 Write block 738 len 24576 data: 4 4 4 5 5 4
23096 261/300 Write block 3983 len 4096 data: 2
23097 262/300 Read block 3373 len 12288
23098 263/300 Flush
23099 264/300 Flush
23100 265/300 Write block 4702 len 12288 data: 3 4 2
23101 266/300 Write block 2566 len 40960 data: 6 8 7 6 5 4 4 5 4 4
23102 267/300 Write block 3063 len 12288 data: 5 6 6
23103 268/300 Read block 3990 len 28672
23104 269/300 Read block 1739 len 16384
23105 270/300 Read block 55 len 32768
23106 271/300 Read block 4459 len 40960
23107 272/300 Write block 119 len 36864 data: 5 4 5 6 7 8 8 7 6
23108 273/300 Flush
23109 274/300 Read block 3750 len 40960
23110 275/300 Write block 3442 len 20480 data: 5 5 3 3 4
23111 276/300 Flush
23112 277/300 Read block 820 len 40960
23113 278/300 Read block 4202 len 32768
23114 279/300 Write block 1241 len 32768 data: 4 2 2 3 4 5 4 4
23115 280/300 Write block 205 len 40960 data: 5 5 7 6 7 6 6 7 7 8
23116 281/300 Write block 4968 len 28672 data: 4 4 4 6 7 7 8
23117 282/300 Read block 1185 len 4096
23118 283/300 Read block 144 len 28672
23119 284/300 Write block 1638 len 8192 data: 3 4
23120 285/300 Write block 1394 len 12288 data: 4 3 4
23121 286/300 Write block 2723 len 8192 data: 5 5
23122 287/300 Write block 1450 len 40960 data: 7 7 7 5 5 6 5 4 5 4
23123 288/300 Read block 866 len 28672
23124 289/300 Read block 2774 len 4096
23125 290/300 Read block 2790 len 12288
23126 291/300 Read block 1107 len 20480
23127 292/300 Read block 2257 len 24576
23128 293/300 Read block 4278 len 20480
23129 294/300 Read block 1407 len 24576
23130 295/300 Flush
23131 296/300 Write block 4727 len 20480 data: 3 3 3 2 2
23132 297/300 Flush
23133 298/300 Read block 1077 len 12288
23134 299/300 Read block 1252 len 28672
23135 300/300 Write block 3694 len 40960 data: 3 3 3 3 3 4 4 4 4 5
23136 301/300 Write block 957 len 16384 data: 4 4 4 4
23137 Main task has work to do, go find it
23138 got dsc Start(2)
23139 start 2
23140 Jan 14 16:15:29.538 INFO request completed, latency_us: 327, response_code: 204, uri: /start/cid/2, method: POST, req_id: 2449bbdd-7788-45c9-98b3-b96ebb52e4ee, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
23141 Replay: started 2, returned:Ok(())
23142 ----------------------------------------------------------------
23143 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
23144 States: Active Active Offline
23145 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23146 7641 Acked Write 1 Done Done Sent false
23147 7642 Acked Read 2 Done Done Sent false
23148 7643 Acked Read 7 Done Done Sent false
23149 7644 Acked Read 3 Done Done Sent false
23150 7645 Acked Write 4 Done Done Sent false
23151 7646 Acked Write 6 Done Done Sent false
23152 7647 Acked Write 3 Done Done Sent false
23153 7648 Acked Read 7 Done Done Sent false
23154 7649 Acked Write 5 Done Done Sent false
23155 7650 Acked Read 2 Done Done Sent false
23156 7651 Acked Write 8 Done Done Sent false
23157 7652 Acked Write 4 Done Done Sent false
23158 7653 Acked Write 4 Done Done Sent false
23159 7654 Acked Read 6 Done Done Sent false
23160 7655 Acked Flush 0 Done Done Sent false
23161 7656 Acked Read 6 Done Done Sent false
23162 7657 Acked Write 2 Done Done Sent false
23163 7658 Acked Write 1 Done Done Sent false
23164 7659 Acked Write 4 Done Done Sent false
23165 7660 Acked Write 1 Done Done Sent false
23166 7661 Acked Write 8 Done Done Sent false
23167 7662 Acked Read 3 Done Done Sent false
23168 7663 Acked Write 5 Done Done Sent false
23169 7664 Acked Flush 0 Done Done Sent false
23170 7665 Acked Write 8 Done Done Sent false
23171 7666 Acked Read 3 Done Done Sent false
23172 7667 Acked Read 6 Done Done Sent false
23173 7668 Acked Write 1 Done Done Sent false
23174 7669 Acked Read 2 Done Done Sent false
23175 7670 Acked Read 2 Done Done Sent false
23176 7671 Acked Write 10 Done Done Sent false
23177 7672 Acked Write 6 Done Done Sent false
23178 7673 Acked Write 8 Done Done Sent false
23179 7674 Acked Flush 0 Done Done Sent false
23180 7675 Acked Write 5 Done Done Sent false
23181 7676 Acked Write 5 Done Done Sent false
23182 7677 Acked Read 6 Done Done Sent false
23183 7678 Acked Write 9 Done Done Sent false
23184 7679 Acked Flush 0 Done Done Sent false
23185 7680 Acked Read 4 Done Done Sent false
23186 7681 Acked Read 6 Done Done Sent false
23187 7682 Acked Read 7 Done Done Sent false
23188 7683 Acked Read 8 Done Done Sent false
23189 7684 Acked Write 8 Done Done Sent false
23190 7685 Acked Read 7 Done Done Sent false
23191 7686 Acked Read 4 Done Done Sent false
23192 7687 Acked Write 3 Done Done Sent false
23193 7688 Acked Read 3 Done Done Sent false
23194 7689 Acked Read 5 Done Done Sent false
23195 7690 Acked Read 4 Done Done Sent false
23196 7691 Acked Read 8 Done Done Sent false
23197 7692 Acked Read 3 Done Done Sent false
23198 7693 Acked Write 3 Done Done Sent false
23199 7694 Acked Write 6 Done Done Sent false
23200 7695 Acked Read 5 Done Done Sent false
23201 7696 Acked Read 5 Done Done Sent false
23202 7697 Acked Read 6 Done Done Sent false
23203 7698 Acked Read 9 Done Done Sent false
23204 7699 Acked Write 2 Done Done Sent false
23205 7700 Acked Read 5 Done Done Sent false
23206 7701 Acked Write 5 Done Done Sent false
23207 7702 Acked Read 10 Done Done Sent false
23208 7703 Acked Read 3 Done Done Sent false
23209 7704 Acked Read 3 Done Done Sent false
23210 7705 Acked Read 10 Done Done Sent false
23211 7706 Acked Write 8 Done Done Sent false
23212 7707 Acked Read 2 Done Done Sent false
23213 7708 Acked Read 1 Done Done Sent false
23214 7709 Acked Write 5 Done Done Sent false
23215 7710 Acked Write 8 Done Done Sent false
23216 7711 Acked Read 3 Done Done Sent false
23217 7712 Acked Write 8 Done Done Sent false
23218 7713 Acked Write 10 Done Done Sent false
23219 7714 Acked Write 10 Done Done Sent false
23220 7715 Acked Read 2 Done Done Sent false
23221 7716 Acked Read 9 Done Done Sent false
23222 7717 Acked Read 1 Done Done Sent false
23223 7718 Acked Read 5 Done Done Sent false
23224 7719 Acked Read 6 Done Done Sent false
23225 7720 Acked Read 1 Done Done Sent false
23226 7721 Acked Read 8 Done Done Sent false
23227 7722 Acked Flush 0 Done Done Sent false
23228 7723 Acked Read 4 Done Done Sent false
23229 7724 Acked Read 8 Done Done Sent false
23230 7725 Acked Write 7 Done Done Sent false
23231 7726 Acked Read 5 Done Done Sent false
23232 7727 Acked Read 9 Done Done Sent false
23233 7728 Acked Write 10 Done Done Sent false
23234 7729 Acked Write 9 Done Done Sent false
23235 7730 Acked Write 8 Done Done Sent false
23236 7731 Acked Write 7 Done Done Sent false
23237 7732 Acked Flush 0 Done Done Sent false
23238 7733 Acked Write 2 Done Done Sent false
23239 7734 Acked Read 1 Done Done Sent false
23240 7735 Acked Flush 0 Done Done Sent false
23241 7736 Acked Write 7 Done Done Sent false
23242 7737 Acked Write 1 Done Done Sent false
23243 7738 Acked Write 5 Done Done Sent false
23244 7739 Acked Read 1 Done Done Sent false
23245 7740 Acked Write 2 Done Done Sent false
23246 7741 Acked Write 8 Done Done Sent false
23247 7742 Acked Write 10 Done Done Sent false
23248 7743 Acked Flush 0 Done Done Sent false
23249 7744 Acked Read 10 Done Done Sent false
23250 7745 Acked Flush 0 Done Done Sent false
23251 7746 Acked Write 7 Done Done Sent false
23252 7747 Acked Read 2 Done Done Sent false
23253 7748 Acked Write 3 Done Done Sent false
23254 7749 Acked Write 4 Done Done Sent false
23255 7750 Acked Write 1 Done Done Sent false
23256 7751 Acked Write 3 Done Done Sent false
23257 7752 Acked Read 10 Done Done Sent false
23258 7753 Acked Read 2 Done Done Sent false
23259 7754 Acked Write 4 Done Done Sent false
23260 7755 Acked Read 2 Done Done Sent false
23261 7756 Acked Write 6 Done Done Sent false
23262 7757 Acked Write 6 Done Done Sent false
23263 7758 Acked Read 9 Done Done Sent false
23264 7759 Acked Read 7 Done Done Sent false
23265 7760 Acked Read 5 Done Done Sent false
23266 7761 Acked Read 3 Done Done Sent false
23267 7762 Acked Flush 0 Done Done Sent false
23268 7763 Acked Read 3 Done Done Sent false
23269 7764 Acked Read 5 Done Done Sent false
23270 7765 Acked Read 6 Done Done Sent false
23271 7766 Acked Read 2 Done Done Sent false
23272 7767 Acked Flush 0 Done Done Sent false
23273 7768 Acked Read 8 Done Done Sent false
23274 7769 Acked Read 4 Done Done Sent false
23275 7770 Acked Read 4 Done Done Sent false
23276 7771 Acked Read 5 Done Done Sent false
23277 7772 Acked Read 8 Done Done Sent false
23278 7773 Acked Write 6 Done Done Sent false
23279 7774 Acked Read 6 Done Done Sent false
23280 7775 Acked Write 3 Done Done Sent false
23281 7776 Acked Write 8 Done Done Sent false
23282 7777 Acked Read 2 Done Done Sent false
23283 7778 Acked Read 3 Done Done Sent false
23284 7779 Acked Write 10 Done Done Sent false
23285 7780 Acked Flush 0 Done Done Sent false
23286 7781 Acked Write 8 Done Done Sent false
23287 7782 Acked Read 10 Done Done Sent false
23288 7783 Acked Read 6 Done Done Sent false
23289 7784 Acked Read 4 Done Done Sent false
23290 7785 Acked Write 6 Done Done Sent false
23291 7786 Acked Flush 0 Done Done Sent false
23292 7787 Acked Read 7 Done Done Sent false
23293 7788 Acked Write 9 Done Done Sent false
23294 7789 Acked Read 2 Done Done Sent false
23295 7790 Acked Write 1 Done Done Sent false
23296 7791 Acked Read 6 Done Done Sent false
23297 7792 Acked Read 4 Done Done Sent false
23298 7793 Acked Read 9 Done Done Sent false
23299 7794 Acked Read 7 Done Done Sent false
23300 7795 Acked Read 10 Done Done Sent false
23301 7796 Acked Read 2 Done Done Sent false
23302 7797 Acked Read 4 Done Done Sent false
23303 7798 Acked Read 5 Done Done Sent false
23304 7799 Acked Read 10 Done Done Sent false
23305 7800 Acked Flush 0 Done Done Sent false
23306 7801 Acked Read 9 Done Done Sent false
23307 7802 Acked Flush 0 Done Done Sent false
23308 7803 Acked Write 10 Done Done Sent false
23309 7804 Acked Read 1 Done Done Sent false
23310 7805 Acked Write 5 Done Done Sent false
23311 7806 Acked Read 1 Done Done Sent false
23312 7807 Acked Flush 0 Done Done Sent false
23313 7808 Acked Write 6 Done Done Sent false
23314 7809 Acked Write 3 Done Done Sent false
23315 7810 Acked Write 4 Done Done Sent false
23316 7811 Acked Read 6 Done Done Sent false
23317 7812 Acked Write 2 Done Done Sent false
23318 7813 Acked Read 2 Done Done Sent false
23319 7814 Acked Read 5 Done Done Sent false
23320 7815 Acked Write 3 Done Done Sent false
23321 7816 Acked Write 6 Done Done Sent false
23322 7817 Acked Write 3 Done Done Sent false
23323 7818 Acked Read 6 Done Done Sent false
23324 7819 Acked Flush 0 Done Done Sent false
23325 7820 Acked Write 2 Done Done Sent false
23326 7821 Acked Read 9 Done Done Sent false
23327 7822 Acked Flush 0 Done Done Sent false
23328 7823 Acked Write 3 Done Done Sent false
23329 7824 Acked Write 2 Done Done Sent false
23330 7825 Acked Flush 0 Done Done Sent false
23331 7826 Acked Flush 0 Done Done Sent false
23332 7827 Acked Read 3 Done Done Sent false
23333 7828 Acked Read 9 Done Done Sent false
23334 7829 Acked Write 1 Done Done Sent false
23335 7830 Acked Write 1 Done Done Sent false
23336 7831 Acked Read 1 Done Done Sent false
23337 7832 Acked Write 8 Done Done Sent false
23338 7833 Acked Read 7 Done Done Sent false
23339 7834 Acked Read 8 Done Done Sent false
23340 7835 Acked Write 10 Done Done Sent false
23341 7836 Acked Write 8 Done Done Sent false
23342 7837 Acked Write 6 Done Done Sent false
23343 7838 Acked Flush 0 Done Done Sent false
23344 7839 Acked Read 4 Done Done Sent false
23345 7840 Acked Read 8 Done Done Sent false
23346 7841 Acked Flush 0 Done Done Sent false
23347 7842 Acked Read 2 Done Done Sent false
23348 7843 Acked Write 10 Done Done Sent false
23349 7844 Acked Flush 0 Done Done Sent false
23350 7845 Acked Read 9 Done Done Sent false
23351 7846 Acked Read 4 Done Done Sent false
23352 7847 Acked Write 7 Done Done Sent false
23353 7848 Acked Read 4 Done Done Sent false
23354 7849 Acked Read 1 Done Done Sent false
23355 7850 Acked Read 9 Done Done Sent false
23356 7851 Acked Write 1 Done Done Sent false
23357 7852 Acked Read 9 Done Done Sent false
23358 7853 Acked Read 10 Done Done Sent false
23359 7854 Acked Read 6 Done Done Sent false
23360 7855 Acked Read 2 Done Done Sent false
23361 7856 Acked Read 5 Done Done Sent false
23362 7857 Acked Flush 0 Done Done Sent false
23363 7858 Acked Read 3 Done Done Sent false
23364 7859 Acked Write 7 Done Done Sent false
23365 7860 Acked Write 9 Done Done Sent false
23366 7861 Acked Read 8 Done Done Sent false
23367 7862 Acked Read 2 Done Done Sent false
23368 7863 Acked Read 2 Done Done Sent false
23369 7864 Acked Flush 0 Done Done Sent false
23370 7865 Acked Write 7 Done Done Sent false
23371 7866 Acked Read 9 Done Done Sent false
23372 7867 Acked Write 8 Done Done Sent false
23373 7868 Acked Read 1 Done Done Sent false
23374 7869 Acked Write 2 Done Done Sent false
23375 7870 Acked Flush 0 Done Done Sent false
23376 7871 Acked Write 10 Done Done Sent false
23377 7872 Acked Read 1 Done Done Sent false
23378 7873 Acked Read 10 Done Done Sent false
23379 7874 Acked Write 10 Done Done Sent false
23380 7875 Acked Write 10 Done Done Sent false
23381 7876 Acked Read 4 Done Done Sent false
23382 7877 Acked Write 3 Done Done Sent false
23383 7878 Acked Read 4 Done Done Sent false
23384 7879 Acked Read 8 Done Done Sent false
23385 7880 Acked Write 10 Done Done Sent false
23386 7881 Acked Read 6 Done Done Sent false
23387 7882 Acked Read 9 Done Done Sent false
23388 7883 Acked Write 5 Done Done Sent false
23389 7884 Acked Write 6 Done Done Sent false
23390 7885 Acked Read 5 Done Done Sent false
23391 7886 Acked Write 3 Done Done Sent false
23392 7887 Acked Read 6 Done Done Sent false
23393 7888 Acked Read 4 Done Done Sent false
23394 7889 Acked Read 10 Done Done Sent false
23395 7890 Acked Write 7 Done Done Sent false
23396 7891 Acked Write 9 Done Done Sent false
23397 7892 Acked Flush 0 Done Done Sent false
23398 7893 Acked Read 9 Done Done Sent false
23399 7894 Acked Read 6 Done Done Sent false
23400 7895 Acked Flush 0 Done Done Sent false
23401 7896 Acked Write 5 Done Done Sent false
23402 7897 Acked Read 7 Done Done Sent false
23403 7898 Acked Write 7 Done Done Sent false
23404 7899 Acked Read 8 Done Done Sent false
23405 7900 Acked Write 6 Done Done Sent false
23406 7901 Acked Write 1 Done Done Sent false
23407 7902 Acked Read 3 Done Done Sent false
23408 7903 Acked Flush 0 Done Done Sent false
23409 7904 Acked Flush 0 Done Done Sent false
23410 7905 Acked Write 3 Done Done Sent false
23411 7906 Acked Write 10 Done Done Sent false
23412 7907 Acked Write 3 Done Done Sent false
23413 7908 Acked Read 7 Done Done Sent false
23414 7909 Acked Read 4 Done Done Sent false
23415 7910 Acked Read 8 Done Done Sent false
23416 7911 Acked Read 10 Done Done Sent false
23417 7912 Acked Write 9 Done Done Sent false
23418 7913 Acked Flush 0 Done Done Sent false
23419 7914 Acked Read 10 Done Done Sent false
23420 7915 Acked Write 5 Done Done Sent false
23421 7916 Acked Flush 0 Done Done Sent false
23422 7917 Acked Read 10 Done Done Sent false
23423 7918 Acked Read 8 Done Done Sent false
23424 7919 Acked Write 8 Done Done Sent false
23425 7920 Acked Write 10 Done Done Sent false
23426 7921 Acked Write 7 Done Done Sent false
23427 7922 Acked Read 1 Done Done Sent false
23428 7923 Acked Read 7 Done Done Sent false
23429 7924 Acked Write 2 Done Done Sent false
23430 7925 Acked Write 3 Done Done Sent false
23431 7926 Acked Write 2 Done Done Sent false
23432 7927 Acked Write 10 Done Done Sent false
23433 7928 Acked Read 7 Done Done Sent false
23434 7929 Acked Read 1 Done Done Sent false
23435 7930 Acked Read 3 Done Done Sent false
23436 7931 Acked Read 5 Done Done Sent false
23437 7932 Acked Read 6 Done Done Sent false
23438 7933 Acked Read 5 Done Done Sent false
23439 7934 Acked Read 6 Done Done Sent false
23440 7935 Acked Flush 0 Done Done Sent false
23441 7936 Acked Write 5 Done Done Sent false
23442 7937 Acked Flush 0 Done Done Sent false
23443 7938 Acked Read 3 Done Done Sent false
23444 7939 Acked Read 7 Done Done Sent false
23445 7940 Acked Write 10 Done Done Sent false
23446 7941 Acked Write 4 Done Done Sent false
23447 STATES DS:0 DS:1 DS:2 TOTAL
23448 Sent 0 0 301 301
23449 Done 301 301 0 602
23450 Skipped 0 0 0 0
23451 Error 0 0 0 0
23452 Last Flush: 7937 7937 7640
23453 Downstairs last five completed: 7640 7639 7638 7637 7636
23454 Upstairs last five completed: 7941 7940 7939 7938 7937
23455 CLIENT: Up:0 ds:301 act:2
23456 [8830] got start action
23457 [8830] I am going to restart
23458 Starting downstairs at port 8830
23459 Make output file at "/tmp/dsc/downstairs-8830.txt"
23460 [8830][2] reports Starting
23461 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1225)
23462 [8830][2] reports Running
23463 ----------------------------------------------------------------
23464 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23465 States: Active Active Offline
23466 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23467 7641 Acked Write 1 Done Done Sent false
23468 7642 Acked Read 2 Done Done Sent false
23469 7643 Acked Read 7 Done Done Sent false
23470 7644 Acked Read 3 Done Done Sent false
23471 7645 Acked Write 4 Done Done Sent false
23472 7646 Acked Write 6 Done Done Sent false
23473 7647 Acked Write 3 Done Done Sent false
23474 7648 Acked Read 7 Done Done Sent false
23475 7649 Acked Write 5 Done Done Sent false
23476 7650 Acked Read 2 Done Done Sent false
23477 7651 Acked Write 8 Done Done Sent false
23478 7652 Acked Write 4 Done Done Sent false
23479 7653 Acked Write 4 Done Done Sent false
23480 7654 Acked Read 6 Done Done Sent false
23481 7655 Acked Flush 0 Done Done Sent false
23482 7656 Acked Read 6 Done Done Sent false
23483 7657 Acked Write 2 Done Done Sent false
23484 7658 Acked Write 1 Done Done Sent false
23485 7659 Acked Write 4 Done Done Sent false
23486 7660 Acked Write 1 Done Done Sent false
23487 7661 Acked Write 8 Done Done Sent false
23488 7662 Acked Read 3 Done Done Sent false
23489 7663 Acked Write 5 Done Done Sent false
23490 7664 Acked Flush 0 Done Done Sent false
23491 7665 Acked Write 8 Done Done Sent false
23492 7666 Acked Read 3 Done Done Sent false
23493 7667 Acked Read 6 Done Done Sent false
23494 7668 Acked Write 1 Done Done Sent false
23495 7669 Acked Read 2 Done Done Sent false
23496 7670 Acked Read 2 Done Done Sent false
23497 7671 Acked Write 10 Done Done Sent false
23498 7672 Acked Write 6 Done Done Sent false
23499 7673 Acked Write 8 Done Done Sent false
23500 7674 Acked Flush 0 Done Done Sent false
23501 7675 Acked Write 5 Done Done Sent false
23502 7676 Acked Write 5 Done Done Sent false
23503 7677 Acked Read 6 Done Done Sent false
23504 7678 Acked Write 9 Done Done Sent false
23505 7679 Acked Flush 0 Done Done Sent false
23506 7680 Acked Read 4 Done Done Sent false
23507 7681 Acked Read 6 Done Done Sent false
23508 7682 Acked Read 7 Done Done Sent false
23509 7683 Acked Read 8 Done Done Sent false
23510 7684 Acked Write 8 Done Done Sent false
23511 7685 Acked Read 7 Done Done Sent false
23512 7686 Acked Read 4 Done Done Sent false
23513 7687 Acked Write 3 Done Done Sent false
23514 7688 Acked Read 3 Done Done Sent false
23515 7689 Acked Read 5 Done Done Sent false
23516 7690 Acked Read 4 Done Done Sent false
23517 7691 Acked Read 8 Done Done Sent false
23518 7692 Acked Read 3 Done Done Sent false
23519 7693 Acked Write 3 Done Done Sent false
23520 7694 Acked Write 6 Done Done Sent false
23521 7695 Acked Read 5 Done Done Sent false
23522 7696 Acked Read 5 Done Done Sent false
23523 7697 Acked Read 6 Done Done Sent false
23524 7698 Acked Read 9 Done Done Sent false
23525 7699 Acked Write 2 Done Done Sent false
23526 7700 Acked Read 5 Done Done Sent false
23527 7701 Acked Write 5 Done Done Sent false
23528 7702 Acked Read 10 Done Done Sent false
23529 7703 Acked Read 3 Done Done Sent false
23530 7704 Acked Read 3 Done Done Sent false
23531 7705 Acked Read 10 Done Done Sent false
23532 7706 Acked Write 8 Done Done Sent false
23533 7707 Acked Read 2 Done Done Sent false
23534 7708 Acked Read 1 Done Done Sent false
23535 7709 Acked Write 5 Done Done Sent false
23536 7710 Acked Write 8 Done Done Sent false
23537 7711 Acked Read 3 Done Done Sent false
23538 7712 Acked Write 8 Done Done Sent false
23539 7713 Acked Write 10 Done Done Sent false
23540 7714 Acked Write 10 Done Done Sent false
23541 7715 Acked Read 2 Done Done Sent false
23542 7716 Acked Read 9 Done Done Sent false
23543 7717 Acked Read 1 Done Done Sent false
23544 7718 Acked Read 5 Done Done Sent false
23545 7719 Acked Read 6 Done Done Sent false
23546 7720 Acked Read 1 Done Done Sent false
23547 7721 Acked Read 8 Done Done Sent false
23548 7722 Acked Flush 0 Done Done Sent false
23549 7723 Acked Read 4 Done Done Sent false
23550 7724 Acked Read 8 Done Done Sent false
23551 7725 Acked Write 7 Done Done Sent false
23552 7726 Acked Read 5 Done Done Sent false
23553 7727 Acked Read 9 Done Done Sent false
23554 7728 Acked Write 10 Done Done Sent false
23555 7729 Acked Write 9 Done Done Sent false
23556 7730 Acked Write 8 Done Done Sent false
23557 7731 Acked Write 7 Done Done Sent false
23558 7732 Acked Flush 0 Done Done Sent false
23559 7733 Acked Write 2 Done Done Sent false
23560 7734 Acked Read 1 Done Done Sent false
23561 7735 Acked Flush 0 Done Done Sent false
23562 7736 Acked Write 7 Done Done Sent false
23563 7737 Acked Write 1 Done Done Sent false
23564 7738 Acked Write 5 Done Done Sent false
23565 7739 Acked Read 1 Done Done Sent false
23566 7740 Acked Write 2 Done Done Sent false
23567 7741 Acked Write 8 Done Done Sent false
23568 7742 Acked Write 10 Done Done Sent false
23569 7743 Acked Flush 0 Done Done Sent false
23570 7744 Acked Read 10 Done Done Sent false
23571 7745 Acked Flush 0 Done Done Sent false
23572 7746 Acked Write 7 Done Done Sent false
23573 7747 Acked Read 2 Done Done Sent false
23574 7748 Acked Write 3 Done Done Sent false
23575 7749 Acked Write 4 Done Done Sent false
23576 7750 Acked Write 1 Done Done Sent false
23577 7751 Acked Write 3 Done Done Sent false
23578 7752 Acked Read 10 Done Done Sent false
23579 7753 Acked Read 2 Done Done Sent false
23580 7754 Acked Write 4 Done Done Sent false
23581 7755 Acked Read 2 Done Done Sent false
23582 7756 Acked Write 6 Done Done Sent false
23583 7757 Acked Write 6 Done Done Sent false
23584 7758 Acked Read 9 Done Done Sent false
23585 7759 Acked Read 7 Done Done Sent false
23586 7760 Acked Read 5 Done Done Sent false
23587 7761 Acked Read 3 Done Done Sent false
23588 7762 Acked Flush 0 Done Done Sent false
23589 7763 Acked Read 3 Done Done Sent false
23590 7764 Acked Read 5 Done Done Sent false
23591 7765 Acked Read 6 Done Done Sent false
23592 7766 Acked Read 2 Done Done Sent false
23593 7767 Acked Flush 0 Done Done Sent false
23594 7768 Acked Read 8 Done Done Sent false
23595 7769 Acked Read 4 Done Done Sent false
23596 7770 Acked Read 4 Done Done Sent false
23597 7771 Acked Read 5 Done Done Sent false
23598 7772 Acked Read 8 Done Done Sent false
23599 7773 Acked Write 6 Done Done Sent false
23600 7774 Acked Read 6 Done Done Sent false
23601 7775 Acked Write 3 Done Done Sent false
23602 7776 Acked Write 8 Done Done Sent false
23603 7777 Acked Read 2 Done Done Sent false
23604 7778 Acked Read 3 Done Done Sent false
23605 7779 Acked Write 10 Done Done Sent false
23606 7780 Acked Flush 0 Done Done Sent false
23607 7781 Acked Write 8 Done Done Sent false
23608 7782 Acked Read 10 Done Done Sent false
23609 7783 Acked Read 6 Done Done Sent false
23610 7784 Acked Read 4 Done Done Sent false
23611 7785 Acked Write 6 Done Done Sent false
23612 7786 Acked Flush 0 Done Done Sent false
23613 7787 Acked Read 7 Done Done Sent false
23614 7788 Acked Write 9 Done Done Sent false
23615 7789 Acked Read 2 Done Done Sent false
23616 7790 Acked Write 1 Done Done Sent false
23617 7791 Acked Read 6 Done Done Sent false
23618 7792 Acked Read 4 Done Done Sent false
23619 7793 Acked Read 9 Done Done Sent false
23620 7794 Acked Read 7 Done Done Sent false
23621 7795 Acked Read 10 Done Done Sent false
23622 7796 Acked Read 2 Done Done Sent false
23623 7797 Acked Read 4 Done Done Sent false
23624 7798 Acked Read 5 Done Done Sent false
23625 7799 Acked Read 10 Done Done Sent false
23626 7800 Acked Flush 0 Done Done Sent false
23627 7801 Acked Read 9 Done Done Sent false
23628 7802 Acked Flush 0 Done Done Sent false
23629 7803 Acked Write 10 Done Done Sent false
23630 7804 Acked Read 1 Done Done Sent false
23631 7805 Acked Write 5 Done Done Sent false
23632 7806 Acked Read 1 Done Done Sent false
23633 7807 Acked Flush 0 Done Done Sent false
23634 7808 Acked Write 6 Done Done Sent false
23635 7809 Acked Write 3 Done Done Sent false
23636 7810 Acked Write 4 Done Done Sent false
23637 7811 Acked Read 6 Done Done Sent false
23638 7812 Acked Write 2 Done Done Sent false
23639 7813 Acked Read 2 Done Done Sent false
23640 7814 Acked Read 5 Done Done Sent false
23641 7815 Acked Write 3 Done Done Sent false
23642 7816 Acked Write 6 Done Done Sent false
23643 7817 Acked Write 3 Done Done Sent false
23644 7818 Acked Read 6 Done Done Sent false
23645 7819 Acked Flush 0 Done Done Sent false
23646 7820 Acked Write 2 Done Done Sent false
23647 7821 Acked Read 9 Done Done Sent false
23648 7822 Acked Flush 0 Done Done Sent false
23649 7823 Acked Write 3 Done Done Sent false
23650 7824 Acked Write 2 Done Done Sent false
23651 7825 Acked Flush 0 Done Done Sent false
23652 7826 Acked Flush 0 Done Done Sent false
23653 7827 Acked Read 3 Done Done Sent false
23654 7828 Acked Read 9 Done Done Sent false
23655 7829 Acked Write 1 Done Done Sent false
23656 7830 Acked Write 1 Done Done Sent false
23657 7831 Acked Read 1 Done Done Sent false
23658 7832 Acked Write 8 Done Done Sent false
23659 7833 Acked Read 7 Done Done Sent false
23660 7834 Acked Read 8 Done Done Sent false
23661 7835 Acked Write 10 Done Done Sent false
23662 7836 Acked Write 8 Done Done Sent false
23663 7837 Acked Write 6 Done Done Sent false
23664 7838 Acked Flush 0 Done Done Sent false
23665 7839 Acked Read 4 Done Done Sent false
23666 7840 Acked Read 8 Done Done Sent false
23667 7841 Acked Flush 0 Done Done Sent false
23668 7842 Acked Read 2 Done Done Sent false
23669 7843 Acked Write 10 Done Done Sent false
23670 7844 Acked Flush 0 Done Done Sent false
23671 7845 Acked Read 9 Done Done Sent false
23672 7846 Acked Read 4 Done Done Sent false
23673 7847 Acked Write 7 Done Done Sent false
23674 7848 Acked Read 4 Done Done Sent false
23675 7849 Acked Read 1 Done Done Sent false
23676 7850 Acked Read 9 Done Done Sent false
23677 7851 Acked Write 1 Done Done Sent false
23678 7852 Acked Read 9 Done Done Sent false
23679 7853 Acked Read 10 Done Done Sent false
23680 7854 Acked Read 6 Done Done Sent false
23681 7855 Acked Read 2 Done Done Sent false
23682 7856 Acked Read 5 Done Done Sent false
23683 7857 Acked Flush 0 Done Done Sent false
23684 7858 Acked Read 3 Done Done Sent false
23685 7859 Acked Write 7 Done Done Sent false
23686 7860 Acked Write 9 Done Done Sent false
23687 7861 Acked Read 8 Done Done Sent false
23688 7862 Acked Read 2 Done Done Sent false
23689 7863 Acked Read 2 Done Done Sent false
23690 7864 Acked Flush 0 Done Done Sent false
23691 7865 Acked Write 7 Done Done Sent false
23692 7866 Acked Read 9 Done Done Sent false
23693 7867 Acked Write 8 Done Done Sent false
23694 7868 Acked Read 1 Done Done Sent false
23695 7869 Acked Write 2 Done Done Sent false
23696 7870 Acked Flush 0 Done Done Sent false
23697 7871 Acked Write 10 Done Done Sent false
23698 7872 Acked Read 1 Done Done Sent false
23699 7873 Acked Read 10 Done Done Sent false
23700 7874 Acked Write 10 Done Done Sent false
23701 7875 Acked Write 10 Done Done Sent false
23702 7876 Acked Read 4 Done Done Sent false
23703 7877 Acked Write 3 Done Done Sent false
23704 7878 Acked Read 4 Done Done Sent false
23705 7879 Acked Read 8 Done Done Sent false
23706 7880 Acked Write 10 Done Done Sent false
23707 7881 Acked Read 6 Done Done Sent false
23708 7882 Acked Read 9 Done Done Sent false
23709 7883 Acked Write 5 Done Done Sent false
23710 7884 Acked Write 6 Done Done Sent false
23711 7885 Acked Read 5 Done Done Sent false
23712 7886 Acked Write 3 Done Done Sent false
23713 7887 Acked Read 6 Done Done Sent false
23714 7888 Acked Read 4 Done Done Sent false
23715 7889 Acked Read 10 Done Done Sent false
23716 7890 Acked Write 7 Done Done Sent false
23717 7891 Acked Write 9 Done Done Sent false
23718 7892 Acked Flush 0 Done Done Sent false
23719 7893 Acked Read 9 Done Done Sent false
23720 7894 Acked Read 6 Done Done Sent false
23721 7895 Acked Flush 0 Done Done Sent false
23722 7896 Acked Write 5 Done Done Sent false
23723 7897 Acked Read 7 Done Done Sent false
23724 7898 Acked Write 7 Done Done Sent false
23725 7899 Acked Read 8 Done Done Sent false
23726 7900 Acked Write 6 Done Done Sent false
23727 7901 Acked Write 1 Done Done Sent false
23728 7902 Acked Read 3 Done Done Sent false
23729 7903 Acked Flush 0 Done Done Sent false
23730 7904 Acked Flush 0 Done Done Sent false
23731 7905 Acked Write 3 Done Done Sent false
23732 7906 Acked Write 10 Done Done Sent false
23733 7907 Acked Write 3 Done Done Sent false
23734 7908 Acked Read 7 Done Done Sent false
23735 7909 Acked Read 4 Done Done Sent false
23736 7910 Acked Read 8 Done Done Sent false
23737 7911 Acked Read 10 Done Done Sent false
23738 7912 Acked Write 9 Done Done Sent false
23739 7913 Acked Flush 0 Done Done Sent false
23740 7914 Acked Read 10 Done Done Sent false
23741 7915 Acked Write 5 Done Done Sent false
23742 7916 Acked Flush 0 Done Done Sent false
23743 7917 Acked Read 10 Done Done Sent false
23744 7918 Acked Read 8 Done Done Sent false
23745 7919 Acked Write 8 Done Done Sent false
23746 7920 Acked Write 10 Done Done Sent false
23747 7921 Acked Write 7 Done Done Sent false
23748 7922 Acked Read 1 Done Done Sent false
23749 7923 Acked Read 7 Done Done Sent false
23750 7924 Acked Write 2 Done Done Sent false
23751 7925 Acked Write 3 Done Done Sent false
23752 7926 Acked Write 2 Done Done Sent false
23753 7927 Acked Write 10 Done Done Sent false
23754 7928 Acked Read 7 Done Done Sent false
23755 7929 Acked Read 1 Done Done Sent false
23756 7930 Acked Read 3 Done Done Sent false
23757 7931 Acked Read 5 Done Done Sent false
23758 7932 Acked Read 6 Done Done Sent false
23759 7933 Acked Read 5 Done Done Sent false
23760 7934 Acked Read 6 Done Done Sent false
23761 7935 Acked Flush 0 Done Done Sent false
23762 7936 Acked Write 5 Done Done Sent false
23763 7937 Acked Flush 0 Done Done Sent false
23764 7938 Acked Read 3 Done Done Sent false
23765 7939 Acked Read 7 Done Done Sent false
23766 7940 Acked Write 10 Done Done Sent false
23767 7941 Acked Write 4 Done Done Sent false
23768 7942 Acked Flush 0 Done Done Sent false
23769 STATES DS:0 DS:1 DS:2 TOTAL
23770 Sent 0 0 302 302
23771 Done 302 302 0 604
23772 Skipped 0 0 0 0
23773 Error 0 0 0 0
23774 Last Flush: 7942 7942 7640
23775 Downstairs last five completed: 7640 7639 7638 7637 7636
23776 Upstairs last five completed: 7942 7941 7940 7939 7938
23777 CLIENT: Up:0 ds:302 act:2
237782025-01-14T16:15:35.303ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
237792025-01-14T16:15:35.304ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39973) = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
237802025-01-14T16:15:35.304ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
237812025-01-14T16:15:35.305ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
237822025-01-14T16:15:35.305ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
237832025-01-14T16:15:35.305ZINFOcrucible: send last flush ID to this DS: 7640 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
237842025-01-14T16:15:35.305ZINFOcrucible: Replied this last flush ID: 7640 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
237852025-01-14T16:15:35.305ZINFOcrucible: [2] client re-new 302 jobs since flush 7640 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
237862025-01-14T16:15:35.306ZINFOcrucible: [2] Replayed 302 jobs since flush: 7640 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
23787 ----------------------------------------------------------------
23788 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
23789 Downstairs last five completed: 7942 7941 7940 7939 7938
23790 Upstairs last five completed: 7942 7941 7940 7939 7938
23791 CLIENT: Up:0 ds:0 act:3
23792 Replay: All jobs finished
23793 Main task has work to do, go find it
23794 got dsc Stop(1)
23795 stop 1
23796 Jan 14 16:15:37.543 INFO request completed, latency_us: 342[8820] Got stop action so:false kr:false
23797 , response_code: 204, uri: /stop/cid/1, method: POST, req_id: b8756efa-1222-4031-a0c7-e7aac00d8fe2, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
237982025-01-14T16:15:37.544ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
237992025-01-14T16:15:37.544ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
238002025-01-14T16:15:37.544ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
23801 Jan 14 16:15:37.544 INFO request completed, latency_us: 165, response_code: 200, uri: /state/cid/1, method: GET, req_id: bd4617a5-f6f6-46fc-965f-b0cd2e806847, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
238022025-01-14T16:15:37.544ZINFOcrucible: client task is exiting = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
23803 [8820] Exited with: ExitStatus(unix_wait_status(9))
23804 [8820][1] reports Exit
23805 Jan 14 16:15:41.545 INFO request completed, latency_us: 182, response_code: 200, uri: /state/cid/1, method: GET, req_id: 65e742d8-7cef-4c8a-9b4e-c2efa1092b26, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
23806 001/300 Write block 2410 len 36864 data: 5 5 5 5 4 4 5 5 4
23807 002/300 Read block 3697 len 32768
23808 003/300 Read block 4497 len 32768
23809 004/300 Write block 4833 len 12288 data: 2 2 4
23810 005/300 Read block 4451 len 12288
23811 006/300 Read block 4757 len 28672
23812 007/300 Read block 1888 len 36864
23813 008/300 Write block 4555 len 36864 data: 3 4 5 5 5 6 6 6 4
23814 009/300 Read block 4407 len 20480
23815 010/300 Write block 609 len 4096 data: 5
23816 011/300 Write block 4637 len 28672 data: 5 6 5 5 5 5 6
23817 012/300 Read block 1139 len 12288
23818 013/300 Write block 913 len 28672 data: 4 3 3 4 4 4 5
23819 014/300 Read block 922 len 8192
23820 015/300 Read block 4336 len 24576
23821 016/300 Read block 2015 len 20480
23822 017/300 Read block 2818 len 28672
23823 018/300 Read block 4780 len 28672
23824 019/300 Read block 4975 len 16384
23825 020/300 Write block 728 len 16384 data: 3 3 4 6
23826 021/300 Read block 1463 len 28672
23827 022/300 Write block 4974 len 32768 data: 9 9 9 9 8 6 7 4
23828 023/300 Write block 1990 len 40960 data: 7 9 9 11 8 9 11 10 9 7
23829 024/300 Flush
23830 025/300 Read block 3005 len 24576
23831 026/300 Read block 115 len 28672
23832 027/300 Read block 3710 len 4096
23833 028/300 Read block 3324 len 20480
23834 029/300 Write block 12 len 20480 data: 4 5 7 7 6
23835 030/300 Read block 4116 len 4096
23836 031/300 Write block 2613 len 36864 data: 9 9 8 8 9 11 9 8 6
23837 032/300 Write block 884 len 4096 data: 8
23838 033/300 Read block 2356 len 28672
23839 034/300 Read block 399 len 20480
23840 035/300 Write block 2720 len 36864 data: 7 6 5 6 6 6 7 7 8
23841 036/300 Write block 2579 len 32768 data: 4 4 5 8 6 6 6 5
23842 037/300 Flush
23843 038/300 Read block 3538 len 36864
23844 039/300 Flush
23845 040/300 Read block 2604 len 28672
23846 041/300 Write block 2297 len 20480 data: 2 2 2 2 2
23847 042/300 Read block 251 len 36864
23848 043/300 Read block 5 len 20480
23849 044/300 Write block 4641 len 12288 data: 6 6 7
23850 045/300 Read block 4539 len 12288
23851 046/300 Write block 1037 len 4096 data: 5
23852 047/300 Read block 2858 len 36864
23853 048/300 Write block 857 len 40960 data: 4 4 5 5 5 4 3 4 4 4
23854 049/300 Write block 376 len 12288 data: 8 4 3
23855 050/300 Read block 3270 len 24576
23856 051/300 Write block 563 len 36864 data: 8 9 6 4 3 4 4 2 2
23857 052/300 Write block 109 len 24576 data: 4 4 4 3 5 5
23858 053/300 Write block 3563 len 20480 data: 5 5 5 5 7
23859 054/300 Read block 2019 len 8192
23860 055/300 Read block 2931 len 8192
23861 056/300 Read block 715 len 28672
23862 057/300 Write block 3477 len 8192 data: 3 3
23863 058/300 Read block 4725 len 8192
23864 059/300 Write block 1837 len 12288 data: 6 7 6
23865 060/300 Read block 48 len 12288
23866 061/300 Read block 3847 len 16384
23867 062/300 Read block 2198 len 40960
23868 063/300 Write block 4575 len 32768 data: 4 4 5 6 5 5 5 5
23869 064/300 Write block 4408 len 20480 data: 4 3 5 5 6
23870 065/300 Write block 2980 len 12288 data: 8 7 7
23871 066/300 Write block 4535 len 20480 data: 3 4 4 5 5
23872 067/300 Write block 1326 len 32768 data: 5 5 6 5 5 6 6 6
23873 068/300 Write block 2709 len 28672 data: 4 4 5 4 4 4 6
23874 069/300 Read block 3158 len 16384
23875 070/300 Write block 3532 len 36864 data: 5 7 7 8 8 9 7 7 5
23876 071/300 Read block 1237 len 36864
23877 072/300 Read block 4785 len 24576
23878 073/300 Flush
23879 074/300 Read block 2667 len 24576
23880 075/300 Flush
23881 076/300 Flush
23882 077/300 Write block 129 len 28672 data: 5 5 7 7 7 7 7
23883 078/300 Read block 1654 len 12288
23884 079/300 Read block 706 len 40960
23885 080/300 Write block 3127 len 40960 data: 2 4 5 5 5 5 5 5 4 4
23886 081/300 Read block 3002 len 32768
23887 082/300 Flush
23888 083/300 Write block 1543 len 8192 data: 7 5
23889 084/300 Read block 4181 len 12288
23890 085/300 Read block 325 len 36864
23891 086/300 Write block 2987 len 32768 data: 5 4 6 6 5 5 6 6
23892 087/300 Read block 3582 len 28672
23893 088/300 Read block 483 len 20480
23894 089/300 Write block 1847 len 28672 data: 7 9 7 6 7 8 7
23895 090/300 Write block 4273 len 32768 data: 5 5 6 7 7 7 7 7
23896 091/300 Read block 3104 len 28672
23897 092/300 Read block 2351 len 12288
23898 093/300 Write block 2627 len 4096 data: 4
23899 094/300 Read block 773 len 20480
23900 095/300 Read block 2737 len 8192
23901 096/300 Read block 2763 len 40960
23902 097/300 Read block 2501 len 4096
23903 098/300 Read block 4915 len 40960
23904 099/300 Read block 1843 len 4096
23905 100/300 Read block 3396 len 24576
23906 101/300 Read block 396 len 36864
23907 102/300 Read block 889 len 24576
23908 103/300 Read block 4286 len 20480
23909 104/300 Read block 3189 len 24576
23910 105/300 Read block 3709 len 8192
23911 106/300 Read block 2710 len 16384
23912 107/300 Write block 3627 len 16384 data: 5 5 6 5
23913 108/300 Read block 118 len 8192
23914 109/300 Write block 151 len 16384 data: 5 4 5 4
23915 110/300 Write block 2895 len 28672 data: 6 5 5 5 6 8 7
23916 111/300 Write block 2579 len 24576 data: 5 5 6 9 7 7
23917 112/300 Read block 1019 len 20480
23918 113/300 Write block 1337 len 4096 data: 5
23919 114/300 Read block 1596 len 8192
23920 115/300 Read block 1169 len 36864
23921 116/300 Read block 2129 len 12288
23922 117/300 Read block 2998 len 4096
23923 118/300 Read block 3233 len 16384
23924 119/300 Write block 57 len 36864 data: 6 8 6 7 7 7 6 5 6
23925 120/300 Write block 519 len 16384 data: 5 5 5 3
23926 121/300 Write block 4622 len 28672 data: 4 6 6 6 5 2 4
23927 122/300 Write block 3272 len 32768 data: 4 3 3 4 5 5 5 4
23928 123/300 Write block 1091 len 32768 data: 3 3 4 4 5 6 6 6
23929 124/300 Write block 4723 len 20480 data: 4 4 4 3 4
23930 125/300 Write block 2420 len 16384 data: 4 3 3 3
23931 126/300 Write block 369 len 8192 data: 5 5
23932 127/300 Flush
23933 128/300 Read block 2031 len 12288
23934 129/300 Read block 1144 len 24576
23935 130/300 Read block 4250 len 40960
23936 131/300 Write block 4466 len 20480 data: 4 5 4 4 4
23937 132/300 Read block 2190 len 16384
23938 133/300 Flush
23939 134/300 Write block 2874 len 40960 data: 8 7 9 9 10 9 5 5 5 5
23940 135/300 Read block 2439 len 16384
23941 136/300 Write block 2608 len 4096 data: 5
23942 137/300 Read block 3388 len 8192
23943 138/300 Write block 3480 len 36864 data: 3 3 3 4 4 3 3 3 2
23944 139/300 Flush
23945 140/300 Write block 2965 len 40960 data: 2 4 4 5 6 5 5 4 4 3
23946 141/300 Read block 1377 len 36864
23947 142/300 Write block 2272 len 8192 data: 5 5
23948 143/300 Read block 3584 len 28672
23949 144/300 Read block 1561 len 32768
23950 145/300 Flush
23951 146/300 Flush
23952 147/300 Write block 2013 len 20480 data: 3 3 5 5 5
23953 148/300 Read block 3162 len 32768
23954 149/300 Read block 2015 len 16384
23955 150/300 Read block 1086 len 36864
23956 151/300 Read block 425 len 12288
23957 152/300 Read block 3510 len 32768
23958 153/300 Read block 4321 len 24576
23959 154/300 Read block 3246 len 32768
23960 155/300 Flush
23961 156/300 Write block 818 len 16384 data: 6 7 7 6
23962 157/300 Read block 61 len 12288
23963 158/300 Write block 3030 len 16384 data: 6 7 6 4
23964 159/300 Read block 107 len 40960
23965 160/300 Read block 3018 len 40960
23966 161/300 Read block 1320 len 20480
23967 162/300 Write block 153 len 40960 data: 6 5 4 5 5 4 4 7 7 7
23968 163/300 Read block 2748 len 28672
23969 164/300 Read block 4058 len 32768
23970 165/300 Write block 4120 len 12288 data: 4 4 5
23971 166/300 Write block 4898 len 28672 data: 6 4 4 4 3 3 4
23972 167/300 Write block 634 len 4096 data: 4
23973 168/300 Write block 2877 len 32768 data: 10 11 10 6 6 6 6 6
23974 169/300 Read block 661 len 4096
23975 170/300 Read block 1308 len 28672
23976 171/300 Read block 663 len 8192
23977 172/300 Read block 1858 len 20480
23978 173/300 Read block 4507 len 16384
23979 174/300 Read block 4528 len 36864
23980 175/300 Read block 3484 len 32768
23981 176/300 Read block 2957 len 36864
23982 177/300 Write block 1509 len 4096 data: 4
23983 178/300 Read block 876 len 8192
23984 179/300 Read block 1023 len 32768
23985 180/300 Write block 4635 len 40960 data: 4 5 6 7 6 6 7 7 8 6
23986 181/300 Write block 3119 len 12288 data: 4 4 4
23987 182/300 Flush
23988 183/300 Write block 3908 len 4096 data: 5
23989 184/300 Write block 4988 len 12288 data: 5 5 4
23990 185/300 Read block 1408 len 24576
23991 186/300 Read block 1432 len 40960
23992 187/300 Read block 4943 len 16384
23993 188/300 Write block 2273 len 16384 data: 6 5 6 6
23994 189/300 Read block 1797 len 4096
23995 190/300 Write block 2071 len 4096 data: 2
23996 191/300 Read block 1375 len 12288
23997 192/300 Write block 3023 len 16384 data: 5 9 9 7
23998 193/300 Read block 1838 len 24576
23999 194/300 Read block 3148 len 28672
24000 195/300 Read block 635 len 28672
24001 196/300 Read block 421 len 20480
24002 197/300 Write block 3201 len 32768 data: 3 3 4 3 4 4 5 6
24003 198/300 Read block 2097 len 16384
24004 199/300 Write block 92 len 24576 data: 3 3 4 6 4 3
24005 200/300 Read block 283 len 8192
24006 201/300 Read block 2534 len 32768
24007 202/300 Write block 714 len 8192 data: 3 3
24008 203/300 Write block 4448 len 4096 data: 2
24009 204/300 Read block 4223 len 36864
24010 205/300 Write block 2390 len 32768 data: 7 7 7 7 6 6 6 7
24011 206/300 Read block 154 len 28672
24012 207/300 Read block 698 len 20480
24013 208/300 Read block 3417 len 8192
24014 209/300 Flush
24015 210/300 Read block 933 len 40960
24016 211/300 Flush
24017 212/300 Read block 1645 len 36864
24018 213/300 Flush
24019 214/300 Write block 684 len 36864 data: 7 7 7 8 8 8 7 7 8
24020 215/300 Read block 211 len 8192
24021 216/300 Read block 2658 len 32768
24022 217/300 Write block 2855 len 40960 data: 5 6 6 6 6 5 7 6 6 6
24023 218/300 Read block 1785 len 12288
24024 219/300 Write block 3883 len 24576 data: 5 4 4 4 4 5
24025 220/300 Read block 4401 len 28672
24026 221/300 Read block 3398 len 12288
24027 222/300 Read block 3218 len 36864
24028 223/300 Read block 3134 len 32768
24029 224/300 Write block 1759 len 12288 data: 4 4 3
24030 225/300 Read block 1720 len 4096
24031 226/300 Read block 3885 len 28672
24032 227/300 Read block 299 len 20480
24033 228/300 Write block 141 len 24576 data: 5 7 9 9 9 8
24034 229/300 Read block 2781 len 12288
24035 230/300 Read block 2942 len 32768
24036 231/300 Write block 1226 len 8192 data: 3 3
24037 232/300 Write block 3574 len 24576 data: 7 7 6 6 6 7
24038 233/300 Read block 3158 len 8192
24039 234/300 Write block 4639 len 16384 data: 7 7 8 8
24040 235/300 Read block 563 len 8192
24041 236/300 Read block 2313 len 4096
24042 237/300 Write block 4083 len 20480 data: 5 6 7 8 7
24043 238/300 Read block 1230 len 32768
24044 239/300 Read block 4426 len 32768
24045 240/300 Write block 3877 len 24576 data: 4 4 5 5 5 5
24046 241/300 Read block 2151 len 4096
24047 242/300 Write block 3769 len 12288 data: 5 5 5
24048 243/300 Read block 2999 len 12288
24049 244/300 Flush
24050 245/300 Write block 1948 len 32768 data: 5 5 5 7 7 6 7 8
24051 246/300 Write block 1483 len 24576 data: 4 3 3 5 5 6
24052 247/300 Write block 2967 len 36864 data: 5 6 7 6 6 5 5 4 3
24053 248/300 Read block 993 len 24576
24054 249/300 Read block 15 len 16384
24055 250/300 Read block 633 len 24576
24056 251/300 Write block 1198 len 32768 data: 4 3 2 2 2 2 3 4
24057 252/300 Write block 2707 len 16384 data: 5 6 5 5
24058 253/300 Read block 603 len 40960
24059 254/300 Write block 2048 len 4096 data: 4
24060 255/300 Write block 4342 len 12288 data: 4 4 5
24061 256/300 Read block 2405 len 28672
24062 257/300 Write block 4035 len 28672 data: 3 3 2 2 2 2 3
24063 258/300 Write block 583 len 4096 data: 6
24064 259/300 Flush
24065 260/300 Write block 4327 len 8192 data: 6 5
24066 261/300 Read block 962 len 8192
24067 262/300 Read block 2699 len 16384
24068 263/300 Write block 52 len 40960 data: 4 4 4 4 7 7 9 7 8 8
24069 264/300 Write block 3311 len 36864 data: 5 6 6 4 4 4 3 3 4
24070 265/300 Read block 1308 len 24576
24071 266/300 Read block 39 len 36864
24072 267/300 Read block 3870 len 20480
24073 268/300 Write block 1585 len 16384 data: 4 4 3 3
24074 269/300 Read block 4170 len 4096
24075 270/300 Read block 1019 len 8192
24076 271/300 Write block 156 len 16384 data: 6 6 5 5
24077 272/300 Write block 2059 len 8192 data: 6 6
24078 273/300 Read block 821 len 16384
24079 274/300 Read block 4003 len 36864
24080 275/300 Write block 1758 len 4096 data: 6
24081 276/300 Read block 3143 len 36864
24082 277/300 Read block 909 len 8192
24083 278/300 Flush
24084 279/300 Read block 3972 len 12288
24085 280/300 Write block 2849 len 4096 data: 4
24086 281/300 Write block 3323 len 36864 data: 4 4 3 4 4 5 5 5 5
24087 282/300 Write block 4942 len 4096 data: 4
24088 283/300 Read block 579 len 36864
24089 284/300 Read block 4090 len 8192
24090 285/300 Write block 1736 len 40960 data: 4 5 5 5 5 7 6 6 5 6
24091 286/300 Read block 475 len 32768
24092 287/300 Write block 4199 len 24576 data: 4 4 5 5 4 4
24093 288/300 Write block 3099 len 24576 data: 3 3 6 5 5 5
24094 289/300 Read block 846 len 32768
24095 290/300 Write block 3004 len 36864 data: 8 7 7 6 4 4 5 4 4
24096 291/300 Write block 2836 len 28672 data: 7 7 6 5 4 4 5
24097 292/300 Read block 2998 len 8192
24098 293/300 Write block 576 len 8192 data: 6 6
24099 294/300 Read block 4734 len 8192
24100 295/300 Write block 1682 len 12288 data: 3 4 4
24101 296/300 Read block 2291 len 32768
24102 297/300 Read block 4957 len 20480
24103 298/300 Flush
24104 299/300 Read block 11 len 8192
24105 300/300 Read block 4340 len 36864
24106 301/300 Read block 3235 len 8192
24107 Main task has work to do, go find it
24108 got dsc Start(1)
24109 start 1
24110 Jan 14 16:15:41.787 INFO request completed, latency_us: 242, response_code: 204, uri: /start/cid/1, method: POST, req_id: 77bbfde1-1873-4a89-bea4-3dc881594204, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
24111 Replay: started 1, returned:Ok(())
24112 ----------------------------------------------------------------
24113 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
24114 States: Active Offline Active
24115 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24116 7943 Acked Write 9 Done Sent Done false
24117 7944 Acked Read 8 Done Sent Done false
24118 7945 Acked Read 8 Done Sent Done false
24119 7946 Acked Write 3 Done Sent Done false
24120 7947 Acked Read 3 Done Sent Done false
24121 7948 Acked Read 7 Done Sent Done false
24122 7949 Acked Read 9 Done Sent Done false
24123 7950 Acked Write 9 Done Sent Done false
24124 7951 Acked Read 5 Done Sent Done false
24125 7952 Acked Write 1 Done Sent Done false
24126 7953 Acked Write 7 Done Sent Done false
24127 7954 Acked Read 3 Done Sent Done false
24128 7955 Acked Write 7 Done Sent Done false
24129 7956 Acked Read 2 Done Sent Done false
24130 7957 Acked Read 6 Done Sent Done false
24131 7958 Acked Read 5 Done Sent Done false
24132 7959 Acked Read 7 Done Sent Done false
24133 7960 Acked Read 7 Done Sent Done false
24134 7961 Acked Read 4 Done Sent Done false
24135 7962 Acked Write 4 Done Sent Done false
24136 7963 Acked Read 7 Done Sent Done false
24137 7964 Acked Write 8 Done Sent Done false
24138 7965 Acked Write 10 Done Sent Done false
24139 7966 Acked Flush 0 Done Sent Done false
24140 7967 Acked Read 6 Done Sent Done false
24141 7968 Acked Read 7 Done Sent Done false
24142 7969 Acked Read 1 Done Sent Done false
24143 7970 Acked Read 5 Done Sent Done false
24144 7971 Acked Write 5 Done Sent Done false
24145 7972 Acked Read 1 Done Sent Done false
24146 7973 Acked Write 9 Done Sent Done false
24147 7974 Acked Write 1 Done Sent Done false
24148 7975 Acked Read 7 Done Sent Done false
24149 7976 Acked Read 5 Done Sent Done false
24150 7977 Acked Write 9 Done Sent Done false
24151 7978 Acked Write 8 Done Sent Done false
24152 7979 Acked Flush 0 Done Sent Done false
24153 7980 Acked Read 9 Done Sent Done false
24154 7981 Acked Flush 0 Done Sent Done false
24155 7982 Acked Read 7 Done Sent Done false
24156 7983 Acked Write 5 Done Sent Done false
24157 7984 Acked Read 9 Done Sent Done false
24158 7985 Acked Read 5 Done Sent Done false
24159 7986 Acked Write 3 Done Sent Done false
24160 7987 Acked Read 3 Done Sent Done false
24161 7988 Acked Write 1 Done Sent Done false
24162 7989 Acked Read 9 Done Sent Done false
24163 7990 Acked Write 10 Done Sent Done false
24164 7991 Acked Write 3 Done Sent Done false
24165 7992 Acked Read 6 Done Sent Done false
24166 7993 Acked Write 9 Done Sent Done false
24167 7994 Acked Write 6 Done Sent Done false
24168 7995 Acked Write 5 Done Sent Done false
24169 7996 Acked Read 2 Done Sent Done false
24170 7997 Acked Read 2 Done Sent Done false
24171 7998 Acked Read 7 Done Sent Done false
24172 7999 Acked Write 2 Done Sent Done false
24173 8000 Acked Read 2 Done Sent Done false
24174 8001 Acked Write 3 Done Sent Done false
24175 8002 Acked Read 3 Done Sent Done false
24176 8003 Acked Read 4 Done Sent Done false
24177 8004 Acked Read 10 Done Sent Done false
24178 8005 Acked Write 8 Done Sent Done false
24179 8006 Acked Write 5 Done Sent Done false
24180 8007 Acked Write 3 Done Sent Done false
24181 8008 Acked Write 5 Done Sent Done false
24182 8009 Acked Write 8 Done Sent Done false
24183 8010 Acked Write 7 Done Sent Done false
24184 8011 Acked Read 4 Done Sent Done false
24185 8012 Acked Write 9 Done Sent Done false
24186 8013 Acked Read 9 Done Sent Done false
24187 8014 Acked Read 6 Done Sent Done false
24188 8015 Acked Flush 0 Done Sent Done false
24189 8016 Acked Read 6 Done Sent Done false
24190 8017 Acked Flush 0 Done Sent Done false
24191 8018 Acked Flush 0 Done Sent Done false
24192 8019 Acked Write 7 Done Sent Done false
24193 8020 Acked Read 3 Done Sent Done false
24194 8021 Acked Read 10 Done Sent Done false
24195 8022 Acked Write 10 Done Sent Done false
24196 8023 Acked Read 8 Done Sent Done false
24197 8024 Acked Flush 0 Done Sent Done false
24198 8025 Acked Write 2 Done Sent Done false
24199 8026 Acked Read 3 Done Sent Done false
24200 8027 Acked Read 9 Done Sent Done false
24201 8028 Acked Write 8 Done Sent Done false
24202 8029 Acked Read 7 Done Sent Done false
24203 8030 Acked Read 5 Done Sent Done false
24204 8031 Acked Write 7 Done Sent Done false
24205 8032 Acked Write 8 Done Sent Done false
24206 8033 Acked Read 7 Done Sent Done false
24207 8034 Acked Read 3 Done Sent Done false
24208 8035 Acked Write 1 Done Sent Done false
24209 8036 Acked Read 5 Done Sent Done false
24210 8037 Acked Read 2 Done Sent Done false
24211 8038 Acked Read 10 Done Sent Done false
24212 8039 Acked Read 1 Done Sent Done false
24213 8040 Acked Read 10 Done Sent Done false
24214 8041 Acked Read 1 Done Sent Done false
24215 8042 Acked Read 6 Done Sent Done false
24216 8043 Acked Read 9 Done Sent Done false
24217 8044 Acked Read 6 Done Sent Done false
24218 8045 Acked Read 5 Done Sent Done false
24219 8046 Acked Read 6 Done Sent Done false
24220 8047 Acked Read 2 Done Sent Done false
24221 8048 Acked Read 4 Done Sent Done false
24222 8049 Acked Write 4 Done Sent Done false
24223 8050 Acked Read 2 Done Sent Done false
24224 8051 Acked Write 4 Done Sent Done false
24225 8052 Acked Write 7 Done Sent Done false
24226 8053 Acked Write 6 Done Sent Done false
24227 8054 Acked Read 5 Done Sent Done false
24228 8055 Acked Write 1 Done Sent Done false
24229 8056 Acked Read 2 Done Sent Done false
24230 8057 Acked Read 9 Done Sent Done false
24231 8058 Acked Read 3 Done Sent Done false
24232 8059 Acked Read 1 Done Sent Done false
24233 8060 Acked Read 4 Done Sent Done false
24234 8061 Acked Write 9 Done Sent Done false
24235 8062 Acked Write 4 Done Sent Done false
24236 8063 Acked Write 7 Done Sent Done false
24237 8064 Acked Write 8 Done Sent Done false
24238 8065 Acked Write 8 Done Sent Done false
24239 8066 Acked Write 5 Done Sent Done false
24240 8067 Acked Write 4 Done Sent Done false
24241 8068 Acked Write 2 Done Sent Done false
24242 8069 Acked Flush 0 Done Sent Done false
24243 8070 Acked Read 3 Done Sent Done false
24244 8071 Acked Read 6 Done Sent Done false
24245 8072 Acked Read 10 Done Sent Done false
24246 8073 Acked Write 5 Done Sent Done false
24247 8074 Acked Read 4 Done Sent Done false
24248 8075 Acked Flush 0 Done Sent Done false
24249 8076 Acked Write 10 Done Sent Done false
24250 8077 Acked Read 4 Done Sent Done false
24251 8078 Acked Write 1 Done Sent Done false
24252 8079 Acked Read 2 Done Sent Done false
24253 8080 Acked Write 9 Done Sent Done false
24254 8081 Acked Flush 0 Done Sent Done false
24255 8082 Acked Write 10 Done Sent Done false
24256 8083 Acked Read 9 Done Sent Done false
24257 8084 Acked Write 2 Done Sent Done false
24258 8085 Acked Read 7 Done Sent Done false
24259 8086 Acked Read 8 Done Sent Done false
24260 8087 Acked Flush 0 Done Sent Done false
24261 8088 Acked Flush 0 Done Sent Done false
24262 8089 Acked Write 5 Done Sent Done false
24263 8090 Acked Read 8 Done Sent Done false
24264 8091 Acked Read 4 Done Sent Done false
24265 8092 Acked Read 9 Done Sent Done false
24266 8093 Acked Read 3 Done Sent Done false
24267 8094 Acked Read 8 Done Sent Done false
24268 8095 Acked Read 6 Done Sent Done false
24269 8096 Acked Read 8 Done Sent Done false
24270 8097 Acked Flush 0 Done Sent Done false
24271 8098 Acked Write 4 Done Sent Done false
24272 8099 Acked Read 3 Done Sent Done false
24273 8100 Acked Write 4 Done Sent Done false
24274 8101 Acked Read 10 Done Sent Done false
24275 8102 Acked Read 10 Done Sent Done false
24276 8103 Acked Read 5 Done Sent Done false
24277 8104 Acked Write 10 Done Sent Done false
24278 8105 Acked Read 7 Done Sent Done false
24279 8106 Acked Read 8 Done Sent Done false
24280 8107 Acked Write 3 Done Sent Done false
24281 8108 Acked Write 7 Done Sent Done false
24282 8109 Acked Write 1 Done Sent Done false
24283 8110 Acked Write 8 Done Sent Done false
24284 8111 Acked Read 1 Done Sent Done false
24285 8112 Acked Read 7 Done Sent Done false
24286 8113 Acked Read 2 Done Sent Done false
24287 8114 Acked Read 5 Done Sent Done false
24288 8115 Acked Read 4 Done Sent Done false
24289 8116 Acked Read 9 Done Sent Done false
24290 8117 Acked Read 8 Done Sent Done false
24291 8118 Acked Read 9 Done Sent Done false
24292 8119 Acked Write 1 Done Sent Done false
24293 8120 Acked Read 2 Done Sent Done false
24294 8121 Acked Read 8 Done Sent Done false
24295 8122 Acked Write 10 Done Sent Done false
24296 8123 Acked Write 3 Done Sent Done false
24297 8124 Acked Flush 0 Done Sent Done false
24298 8125 Acked Write 1 Done Sent Done false
24299 8126 Acked Write 3 Done Sent Done false
24300 8127 Acked Read 6 Done Sent Done false
24301 8128 Acked Read 10 Done Sent Done false
24302 8129 Acked Read 4 Done Sent Done false
24303 8130 Acked Write 4 Done Sent Done false
24304 8131 Acked Read 1 Done Sent Done false
24305 8132 Acked Write 1 Done Sent Done false
24306 8133 Acked Read 3 Done Sent Done false
24307 8134 Acked Write 4 Done Sent Done false
24308 8135 Acked Read 6 Done Sent Done false
24309 8136 Acked Read 7 Done Sent Done false
24310 8137 Acked Read 7 Done Sent Done false
24311 8138 Acked Read 5 Done Sent Done false
24312 8139 Acked Write 8 Done Sent Done false
24313 8140 Acked Read 4 Done Sent Done false
24314 8141 Acked Write 6 Done Sent Done false
24315 8142 Acked Read 2 Done Sent Done false
24316 8143 Acked Read 8 Done Sent Done false
24317 8144 Acked Write 2 Done Sent Done false
24318 8145 Acked Write 1 Done Sent Done false
24319 8146 Acked Read 9 Done Sent Done false
24320 8147 Acked Write 8 Done Sent Done false
24321 8148 Acked Read 7 Done Sent Done false
24322 8149 Acked Read 5 Done Sent Done false
24323 8150 Acked Read 2 Done Sent Done false
24324 8151 Acked Flush 0 Done Sent Done false
24325 8152 Acked Read 10 Done Sent Done false
24326 8153 Acked Flush 0 Done Sent Done false
24327 8154 Acked Read 9 Done Sent Done false
24328 8155 Acked Flush 0 Done Sent Done false
24329 8156 Acked Write 9 Done Sent Done false
24330 8157 Acked Read 2 Done Sent Done false
24331 8158 Acked Read 8 Done Sent Done false
24332 8159 Acked Write 10 Done Sent Done false
24333 8160 Acked Read 3 Done Sent Done false
24334 8161 Acked Write 6 Done Sent Done false
24335 8162 Acked Read 7 Done Sent Done false
24336 8163 Acked Read 3 Done Sent Done false
24337 8164 Acked Read 9 Done Sent Done false
24338 8165 Acked Read 8 Done Sent Done false
24339 8166 Acked Write 3 Done Sent Done false
24340 8167 Acked Read 1 Done Sent Done false
24341 8168 Acked Read 7 Done Sent Done false
24342 8169 Acked Read 5 Done Sent Done false
24343 8170 Acked Write 6 Done Sent Done false
24344 8171 Acked Read 3 Done Sent Done false
24345 8172 Acked Read 8 Done Sent Done false
24346 8173 Acked Write 2 Done Sent Done false
24347 8174 Acked Write 6 Done Sent Done false
24348 8175 Acked Read 2 Done Sent Done false
24349 8176 Acked Write 4 Done Sent Done false
24350 8177 Acked Read 2 Done Sent Done false
24351 8178 Acked Read 1 Done Sent Done false
24352 8179 Acked Write 5 Done Sent Done false
24353 8180 Acked Read 8 Done Sent Done false
24354 8181 Acked Read 8 Done Sent Done false
24355 8182 Acked Write 6 Done Sent Done false
24356 8183 Acked Read 1 Done Sent Done false
24357 8184 Acked Write 3 Done Sent Done false
24358 8185 Acked Read 3 Done Sent Done false
24359 8186 Acked Flush 0 Done Sent Done false
24360 8187 Acked Write 8 Done Sent Done false
24361 8188 Acked Write 6 Done Sent Done false
24362 8189 Acked Write 9 Done Sent Done false
24363 8190 Acked Read 6 Done Sent Done false
24364 8191 Acked Read 4 Done Sent Done false
24365 8192 Acked Read 6 Done Sent Done false
24366 8193 Acked Write 8 Done Sent Done false
24367 8194 Acked Write 4 Done Sent Done false
24368 8195 Acked Read 10 Done Sent Done false
24369 8196 Acked Write 1 Done Sent Done false
24370 8197 Acked Write 3 Done Sent Done false
24371 8198 Acked Read 7 Done Sent Done false
24372 8199 Acked Write 7 Done Sent Done false
24373 8200 Acked Write 1 Done Sent Done false
24374 8201 Acked Flush 0 Done Sent Done false
24375 8202 Acked Write 2 Done Sent Done false
24376 8203 Acked Read 2 Done Sent Done false
24377 8204 Acked Read 4 Done Sent Done false
24378 8205 Acked Write 10 Done Sent Done false
24379 8206 Acked Write 9 Done Sent Done false
24380 8207 Acked Read 6 Done Sent Done false
24381 8208 Acked Read 9 Done Sent Done false
24382 8209 Acked Read 5 Done Sent Done false
24383 8210 Acked Write 4 Done Sent Done false
24384 8211 Acked Read 1 Done Sent Done false
24385 8212 Acked Read 2 Done Sent Done false
24386 8213 Acked Write 4 Done Sent Done false
24387 8214 Acked Write 2 Done Sent Done false
24388 8215 Acked Read 4 Done Sent Done false
24389 8216 Acked Read 9 Done Sent Done false
24390 8217 Acked Write 1 Done Sent Done false
24391 8218 Acked Read 9 Done Sent Done false
24392 8219 Acked Read 2 Done Sent Done false
24393 8220 Acked Flush 0 Done Sent Done false
24394 8221 Acked Read 3 Done Sent Done false
24395 8222 Acked Write 1 Done Sent Done false
24396 8223 Acked Write 9 Done Sent Done false
24397 8224 Acked Write 1 Done Sent Done false
24398 8225 Acked Read 9 Done Sent Done false
24399 8226 Acked Read 2 Done Sent Done false
24400 8227 Acked Write 10 Done Sent Done false
24401 8228 Acked Read 8 Done Sent Done false
24402 8229 Acked Write 6 Done Sent Done false
24403 8230 Acked Write 6 Done Sent Done false
24404 8231 Acked Read 8 Done Sent Done false
24405 8232 Acked Write 9 Done Sent Done false
24406 8233 Acked Write 7 Done Sent Done false
24407 8234 Acked Read 2 Done Sent Done false
24408 8235 Acked Write 2 Done Sent Done false
24409 8236 Acked Read 2 Done Sent Done false
24410 8237 Acked Write 3 Done Sent Done false
24411 8238 Acked Read 8 Done Sent Done false
24412 8239 Acked Read 5 Done Sent Done false
24413 8240 Acked Flush 0 Done Sent Done false
24414 8241 Acked Read 2 Done Sent Done false
24415 8242 Acked Read 9 Done Sent Done false
24416 8243 Acked Read 2 Done Sent Done false
24417 STATES DS:0 DS:1 DS:2 TOTAL
24418 Sent 0 301 0 301
24419 Done 301 0 301 602
24420 Skipped 0 0 0 0
24421 Error 0 0 0 0
24422 Last Flush: 8240 7942 8240
24423 Downstairs last five completed: 7942 7941 7940 7939 7938
24424 Upstairs last five completed: 8243 8242 8241 8240 8239
24425 CLIENT: Up:0 ds:301 act:2
24426 ----------------------------------------------------------------
24427 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24428 States: Active Offline Active
24429 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24430 7943 Acked Write 9 Done Sent Done false
24431 7944 Acked Read 8 Done Sent Done false
24432 7945 Acked Read 8 Done Sent Done false
24433 7946 Acked Write 3 Done Sent Done false
24434 7947 Acked Read 3 Done Sent Done false
24435 7948 Acked Read 7 Done Sent Done false
24436 7949 Acked Read 9 Done Sent Done false
24437 7950 Acked Write 9 Done Sent Done false
24438 7951 Acked Read 5 Done Sent Done false
24439 7952 Acked Write 1 Done Sent Done false
24440 7953 Acked Write 7 Done Sent Done false
24441 7954 Acked Read 3 Done Sent Done false
24442 7955 Acked Write 7 Done Sent Done false
24443 7956 Acked Read 2 Done Sent Done false
24444 7957 Acked Read 6 Done Sent Done false
24445 7958 Acked Read 5 Done Sent Done false
24446 7959 Acked Read 7 Done Sent Done false
24447 7960 Acked Read 7 Done Sent Done false
24448 7961 Acked Read 4 Done Sent Done false
24449 7962 Acked Write 4 Done Sent Done false
24450 7963 Acked Read 7 Done Sent Done false
24451 7964 Acked Write 8 Done Sent Done false
24452 7965 Acked Write 10 Done Sent Done false
24453 7966 Acked Flush 0 Done Sent Done false
24454 7967 Acked Read 6 Done Sent Done false
24455 7968 Acked Read 7 Done Sent Done false
24456 7969 Acked Read 1 Done Sent Done false
24457 7970 Acked Read 5 Done Sent Done false
24458 7971 Acked Write 5 Done Sent Done false
24459 7972 Acked Read 1 Done Sent Done false
24460 7973 Acked Write 9 Done Sent Done false
24461 7974 Acked Write 1 Done Sent Done false
24462 7975 Acked Read 7 Done Sent Done false
24463 7976 Acked Read 5 Done Sent Done false
24464 7977 Acked Write 9 Done Sent Done false
24465 7978 Acked Write 8 Done Sent Done false
24466 7979 Acked Flush 0 Done Sent Done false
24467 7980 Acked Read 9 Done Sent Done false
24468 7981 Acked Flush 0 Done Sent Done false
24469 7982 Acked Read 7 Done Sent Done false
24470 7983 Acked Write 5 Done Sent Done false
24471 7984 Acked Read 9 Done Sent Done false
24472 7985 Acked Read 5 Done Sent Done false
24473 7986 Acked Write 3 Done Sent Done false
24474 7987 Acked Read 3 Done Sent Done false
24475 7988 Acked Write 1 Done Sent Done false
24476 7989 Acked Read 9 Done Sent Done false
24477 7990 Acked Write 10 Done Sent Done false
24478 7991 Acked Write 3 Done Sent Done false
24479 7992 Acked Read 6 Done Sent Done false
24480 7993 Acked Write 9 Done Sent Done false
24481 7994 Acked Write 6 Done Sent Done false
24482 7995 Acked Write 5 Done Sent Done false
24483 7996 Acked Read 2 Done Sent Done false
24484 7997 Acked Read 2 Done Sent Done false
24485 7998 Acked Read 7 Done Sent Done false
24486 7999 Acked Write 2 Done Sent Done false
24487 8000 Acked Read 2 Done Sent Done false
24488 8001 Acked Write 3 Done Sent Done false
24489 8002 Acked Read 3 Done Sent Done false
24490 8003 Acked Read 4 Done Sent Done false
24491 8004 Acked Read 10 Done Sent Done false
24492 8005 Acked Write 8 Done Sent Done false
24493 8006 Acked Write 5 Done Sent Done false
24494 8007 Acked Write 3 Done Sent Done false
24495 8008 Acked Write 5 Done Sent Done false
24496 8009 Acked Write 8 Done Sent Done false
24497 8010 Acked Write 7 Done Sent Done false
24498 8011 Acked Read 4 Done Sent Done false
24499 8012 Acked Write 9 Done Sent Done false
24500 8013 Acked Read 9 Done Sent Done false
24501 8014 Acked Read 6 Done Sent Done false
24502 8015 Acked Flush 0 Done Sent Done false
24503 8016 Acked Read 6 Done Sent Done false
24504 8017 Acked Flush 0 Done Sent Done false
24505 8018 Acked Flush 0 Done Sent Done false
24506 8019 Acked Write 7 Done Sent Done false
24507 8020 Acked Read 3 Done Sent Done false
24508 8021 Acked Read 10 Done Sent Done false
24509 8022 Acked Write 10 Done Sent Done false
24510 8023 Acked Read 8 Done Sent Done false
24511 8024 Acked Flush 0 Done Sent Done false
24512 8025 Acked Write 2 Done Sent Done false
24513 8026 Acked Read 3 Done Sent Done false
24514 8027 Acked Read 9 Done Sent Done false
24515 8028 Acked Write 8 Done Sent Done false
24516 8029 Acked Read 7 Done Sent Done false
24517 8030 Acked Read 5 Done Sent Done false
24518 8031 Acked Write 7 Done Sent Done false
24519 8032 Acked Write 8 Done Sent Done false
24520 8033 Acked Read 7 Done Sent Done false
24521 8034 Acked Read 3 Done Sent Done false
24522 8035 Acked Write 1 Done Sent Done false
24523 8036 Acked Read 5 Done Sent Done false
24524 8037 Acked Read 2 Done Sent Done false
24525 8038 Acked Read 10 Done Sent Done false
24526 8039 Acked Read 1 Done Sent Done false
24527 8040 Acked Read 10 Done Sent Done false
24528 8041 Acked Read 1 Done Sent Done false
24529 8042 Acked Read 6 Done Sent Done false
24530 8043 Acked Read 9 Done Sent Done false
24531 8044 Acked Read 6 Done Sent Done false
24532 8045 Acked Read 5 Done Sent Done false
24533 8046 Acked Read 6 Done Sent Done false
24534 8047 Acked Read 2 Done Sent Done false
24535 8048 Acked Read 4 Done Sent Done false
24536 8049 Acked Write 4 Done Sent Done false
24537 8050 Acked Read 2 Done Sent Done false
24538 8051 Acked Write 4 Done Sent Done false
24539 8052 Acked Write 7 Done Sent Done false
24540 8053 Acked Write 6 Done Sent Done false
24541 8054 Acked Read 5 Done Sent Done false
24542 8055 Acked Write 1 Done Sent Done false
24543 8056 Acked Read 2 Done Sent Done false
24544 8057 Acked Read 9 Done Sent Done false
24545 8058 Acked Read 3 Done Sent Done false
24546 8059 Acked Read 1 Done Sent Done false
24547 8060 Acked Read 4 Done Sent Done false
24548 8061 Acked Write 9 Done Sent Done false
24549 8062 Acked Write 4 Done Sent Done false
24550 8063 Acked Write 7 Done Sent Done false
24551 8064 Acked Write 8 Done Sent Done false
24552 8065 Acked Write 8 Done Sent Done false
24553 8066 Acked Write 5 Done Sent Done false
24554 8067 Acked Write 4 Done Sent Done false
24555 8068 Acked Write 2 Done Sent Done false
24556 8069 Acked Flush 0 Done Sent Done false
24557 8070 Acked Read 3 Done Sent Done false
24558 8071 Acked Read 6 Done Sent Done false
24559 8072 Acked Read 10 Done Sent Done false
24560 8073 Acked Write 5 Done Sent Done false
24561 8074 Acked Read 4 Done Sent Done false
24562 8075 Acked Flush 0 Done Sent Done false
24563 8076 Acked Write 10 Done Sent Done false
24564 8077 Acked Read 4 Done Sent Done false
24565 8078 Acked Write 1 Done Sent Done false
24566 8079 Acked Read 2 Done Sent Done false
24567 8080 Acked Write 9 Done Sent Done false
24568 8081 Acked Flush 0 Done Sent Done false
24569 8082 Acked Write 10 Done Sent Done false
24570 8083 Acked Read 9 Done Sent Done false
24571 8084 Acked Write 2 Done Sent Done false
24572 8085 Acked Read 7 Done Sent Done false
24573 8086 Acked Read 8 Done Sent Done false
24574 8087 Acked Flush 0 Done Sent Done false
24575 8088 Acked Flush 0 Done Sent Done false
24576 8089 Acked Write 5 Done Sent Done false
24577 8090 Acked Read 8 Done Sent Done false
24578 8091 Acked Read 4 Done Sent Done false
24579 8092 Acked Read 9 Done Sent Done false
24580 8093 Acked Read 3 Done Sent Done false
24581 8094 Acked Read 8 Done Sent Done false
24582 8095 Acked Read 6 Done Sent Done false
24583 8096 Acked Read 8 Done Sent Done false
24584 8097 Acked Flush 0 Done Sent Done false
24585 8098 Acked Write 4 Done Sent Done false
24586 8099 Acked Read 3 Done Sent Done false
24587 8100 Acked Write 4 Done Sent Done false
24588 8101 Acked Read 10 Done Sent Done false
24589 8102 Acked Read 10 Done Sent Done false
24590 8103 Acked Read 5 Done Sent Done false
24591 8104 Acked Write 10 Done Sent Done false
24592 8105 Acked Read 7 Done Sent Done false
24593 8106 Acked Read 8 Done Sent Done false
24594 8107 Acked Write 3 Done Sent Done false
24595 8108 Acked Write 7 Done Sent Done false
24596 8109 Acked Write 1 Done Sent Done false
24597 8110 Acked Write 8 Done Sent Done false
24598 8111 Acked Read 1 Done Sent Done false
24599 8112 Acked Read 7 Done Sent Done false
24600 8113 Acked Read 2 Done Sent Done false
24601 8114 Acked Read 5 Done Sent Done false
24602 8115 Acked Read 4 Done Sent Done false
24603 8116 Acked Read 9 Done Sent Done false
24604 8117 Acked Read 8 Done Sent Done false
24605 8118 Acked Read 9 Done Sent Done false
24606 8119 Acked Write 1 Done Sent Done false
24607 8120 Acked Read 2 Done Sent Done false
24608 8121 Acked Read 8 Done Sent Done false
24609 8122 Acked Write 10 Done Sent Done false
24610 8123 Acked Write 3 Done Sent Done false
24611 8124 Acked Flush 0 Done Sent Done false
24612 8125 Acked Write 1 Done Sent Done false
24613 8126 Acked Write 3 Done Sent Done false
24614 8127 Acked Read 6 Done Sent Done false
24615 8128 Acked Read 10 Done Sent Done false
24616 8129 Acked Read 4 Done Sent Done false
24617 8130 Acked Write 4 Done Sent Done false
24618 8131 Acked Read 1 Done Sent Done false
24619 8132 Acked Write 1 Done Sent Done false
24620 8133 Acked Read 3 Done Sent Done false
24621 8134 Acked Write 4 Done Sent Done false
24622 8135 Acked Read 6 Done Sent Done false
24623 8136 Acked Read 7 Done Sent Done false
24624 8137 Acked Read 7 Done Sent Done false
24625 8138 Acked Read 5 Done Sent Done false
24626 8139 Acked Write 8 Done Sent Done false
24627 8140 Acked Read 4 Done Sent Done false
24628 8141 Acked Write 6 Done Sent Done false
24629 8142 Acked Read 2 Done Sent Done false
24630 8143 Acked Read 8 Done Sent Done false
24631 8144 Acked Write 2 Done Sent Done false
24632 8145 Acked Write 1 Done Sent Done false
24633 8146 Acked Read 9 Done Sent Done false
24634 8147 Acked Write 8 Done Sent Done false
24635 8148 Acked Read 7 Done Sent Done false
24636 8149 Acked Read 5 Done Sent Done false
24637 8150 Acked Read 2 Done Sent Done false
24638 8151 Acked Flush 0 Done Sent Done false
24639 8152 Acked Read 10 Done Sent Done false
24640 8153 Acked Flush 0 Done Sent Done false
24641 8154 Acked Read 9 Done Sent Done false
24642 8155 Acked Flush 0 Done Sent Done false
24643 8156 Acked Write 9 Done Sent Done false
24644 8157 Acked Read 2 Done Sent Done false
24645 8158 Acked Read 8 Done Sent Done false
24646 8159 Acked Write 10 Done Sent Done false
24647 8160 Acked Read 3 Done Sent Done false
24648 8161 Acked Write 6 Done Sent Done false
24649 8162 Acked Read 7 Done Sent Done false
24650 8163 Acked Read 3 Done Sent Done false
24651 8164 Acked Read 9 Done Sent Done false
24652 8165 Acked Read 8 Done Sent Done false
24653 8166 Acked Write 3 Done Sent Done false
24654 8167 Acked Read 1 Done Sent Done false
24655 8168 Acked Read 7 Done Sent Done false
24656 8169 Acked Read 5 Done Sent Done false
24657 8170 Acked Write 6 Done Sent Done false
24658 8171 Acked Read 3 Done Sent Done false
24659 8172 Acked Read 8 Done Sent Done false
24660 8173 Acked Write 2 Done Sent Done false
24661 8174 Acked Write 6 Done Sent Done false
24662 8175 Acked Read 2 Done Sent Done false
24663 8176 Acked Write 4 Done Sent Done false
24664 8177 Acked Read 2 Done Sent Done false
24665 8178 Acked Read 1 Done Sent Done false
24666 8179 Acked Write 5 Done Sent Done false
24667 8180 Acked Read 8 Done Sent Done false
24668 8181 Acked Read 8 Done Sent Done false
24669 8182 Acked Write 6 Done Sent Done false
24670 8183 Acked Read 1 Done Sent Done false
24671 8184 Acked Write 3 Done Sent Done false
24672 8185 Acked Read 3 Done Sent Done false
24673 8186 Acked Flush 0 Done Sent Done false
24674 8187 Acked Write 8 Done Sent Done false
24675 8188 Acked Write 6 Done Sent Done false
24676 8189 Acked Write 9 Done Sent Done false
24677 8190 Acked Read 6 Done Sent Done false
24678 8191 Acked Read 4 Done Sent Done false
24679 8192 Acked Read 6 Done Sent Done false
24680 8193 Acked Write 8 Done Sent Done false
24681 8194 Acked Write 4 Done Sent Done false
24682 8195 Acked Read 10 Done Sent Done false
24683 8196 Acked Write 1 Done Sent Done false
24684 8197 Acked Write 3 Done Sent Done false
24685 8198 Acked Read 7 Done Sent Done false
24686 8199 Acked Write 7 Done Sent Done false
24687 8200 Acked Write 1 Done Sent Done false
24688 8201 Acked Flush 0 Done Sent Done false
24689 8202 Acked Write 2 Done Sent Done false
24690 8203 Acked Read 2 Done Sent Done false
24691 8204 Acked Read 4 Done Sent Done false
24692 8205 Acked Write 10 Done Sent Done false
24693 8206 Acked Write 9 Done Sent Done false
24694 8207 Acked Read 6 Done Sent Done false
24695 8208 Acked Read 9 Done Sent Done false
24696 8209 Acked Read 5 Done Sent Done false
24697 8210 Acked Write 4 Done Sent Done false
24698 8211 Acked Read 1 Done Sent Done false
24699 8212 Acked Read 2 Done Sent Done false
24700 8213 Acked Write 4 Done Sent Done false
24701 8214 Acked Write 2 Done Sent Done false
24702 8215 Acked Read 4 Done Sent Done false
24703 8216 Acked Read 9 Done Sent Done false
24704 8217 Acked Write 1 Done Sent Done false
24705 8218 Acked Read 9 Done Sent Done false
24706 8219 Acked Read 2 Done Sent Done false
24707 8220 Acked Flush 0 Done Sent Done false
24708 8221 Acked Read 3 Done Sent Done false
24709 8222 Acked Write 1 Done Sent Done false
24710 8223 Acked Write 9 Done Sent Done false
24711 8224 Acked Write 1 Done Sent Done false
24712 8225 Acked Read 9 Done Sent Done false
24713 8226 Acked Read 2 Done Sent Done false
24714 8227 Acked Write 10 Done Sent Done false
24715 8228 Acked Read 8 Done Sent Done false
24716 8229 Acked Write 6 Done Sent Done false
24717 8230 Acked Write 6 Done Sent Done false
24718 8231 Acked Read 8 Done Sent Done false
24719 8232 Acked Write 9 Done Sent Done false
24720 8233 Acked Write 7 Done Sent Done false
24721 8234 Acked Read 2 Done Sent Done false
24722 8235 Acked Write 2 Done Sent Done false
24723 8236 Acked Read 2 Done Sent Done false
24724 8237 Acked Write 3 Done Sent Done false
24725 8238 Acked Read 8 Done Sent Done false
24726 8239 Acked Read 5 Done Sent Done false
24727 8240 Acked Flush 0 Done Sent Done false
24728 8241 Acked Read 2 Done Sent Done false
24729 8242 Acked Read 9 Done Sent Done false
24730 8243 Acked Read 2 Done Sent Done false
24731 8244 Acked Flush 0 Done Sent Done false
24732 STATES DS:0 DS:1 DS:2 TOTAL
24733 Sent 0 302 0 302
24734 Done 302 0 302 604
24735 Skipped 0 0 0 0
24736 Error 0 0 0 0
24737 Last Flush: 8244 7942 8244
24738 Downstairs last five completed: 7942 7941 7940 7939 7938
24739 Upstairs last five completed: 8244 8243 8242 8241 8240
24740 CLIENT: Up:0 ds:302 act:2
247412025-01-14T16:15:47.545ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
247422025-01-14T16:15:47.545ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
247432025-01-14T16:15:47.545ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
247442025-01-14T16:15:47.545ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = c2423ef0-728e-4895-997a-d9044c50948b
247452025-01-14T16:15:47.545ZINFOcrucible: client task is exiting = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
24746 ----------------------------------------------------------------
24747 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24748 States: Active Offline Active
24749 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24750 7943 Acked Write 9 Done Sent Done false
24751 7944 Acked Read 8 Done Sent Done false
24752 7945 Acked Read 8 Done Sent Done false
24753 7946 Acked Write 3 Done Sent Done false
24754 7947 Acked Read 3 Done Sent Done false
24755 7948 Acked Read 7 Done Sent Done false
24756 7949 Acked Read 9 Done Sent Done false
24757 7950 Acked Write 9 Done Sent Done false
24758 7951 Acked Read 5 Done Sent Done false
24759 7952 Acked Write 1 Done Sent Done false
24760 7953 Acked Write 7 Done Sent Done false
24761 7954 Acked Read 3 Done Sent Done false
24762 7955 Acked Write 7 Done Sent Done false
24763 7956 Acked Read 2 Done Sent Done false
24764 7957 Acked Read 6 Done Sent Done false
24765 7958 Acked Read 5 Done Sent Done false
24766 7959 Acked Read 7 Done Sent Done false
24767 7960 Acked Read 7 Done Sent Done false
24768 7961 Acked Read 4 Done Sent Done false
24769 7962 Acked Write 4 Done Sent Done false
24770 7963 Acked Read 7 Done Sent Done false
24771 7964 Acked Write 8 Done Sent Done false
24772 7965 Acked Write 10 Done Sent Done false
24773 7966 Acked Flush 0 Done Sent Done false
24774 7967 Acked Read 6 Done Sent Done false
24775 7968 Acked Read 7 Done Sent Done false
24776 7969 Acked Read 1 Done Sent Done false
24777 7970 Acked Read 5 Done Sent Done false
24778 7971 Acked Write 5 Done Sent Done false
24779 7972 Acked Read 1 Done Sent Done false
24780 7973 Acked Write 9 Done Sent Done false
24781 7974 Acked Write 1 Done Sent Done false
24782 7975 Acked Read 7 Done Sent Done false
24783 7976 Acked Read 5 Done Sent Done false
24784 7977 Acked Write 9 Done Sent Done false
24785 7978 Acked Write 8 Done Sent Done false
24786 7979 Acked Flush 0 Done Sent Done false
24787 7980 Acked Read 9 Done Sent Done false
24788 7981 Acked Flush 0 Done Sent Done false
24789 7982 Acked Read 7 Done Sent Done false
24790 7983 Acked Write 5 Done Sent Done false
24791 7984 Acked Read 9 Done Sent Done false
24792 7985 Acked Read 5 Done Sent Done false
24793 7986 Acked Write 3 Done Sent Done false
24794 7987 Acked Read 3 Done Sent Done false
24795 7988 Acked Write 1 Done Sent Done false
24796 7989 Acked Read 9 Done Sent Done false
24797 7990 Acked Write 10 Done Sent Done false
24798 7991 Acked Write 3 Done Sent Done false
24799 7992 Acked Read 6 Done Sent Done false
24800 7993 Acked Write 9 Done Sent Done false
24801 7994 Acked Write 6 Done Sent Done false
24802 7995 Acked Write 5 Done Sent Done false
24803 7996 Acked Read 2 Done Sent Done false
24804 7997 Acked Read 2 Done Sent Done false
24805 7998 Acked Read 7 Done Sent Done false
24806 7999 Acked Write 2 Done Sent Done false
24807 8000 Acked Read 2 Done Sent Done false
24808 8001 Acked Write 3 Done Sent Done false
24809 8002 Acked Read 3 Done Sent Done false
24810 8003 Acked Read 4 Done Sent Done false
24811 8004 Acked Read 10 Done Sent Done false
24812 8005 Acked Write 8 Done Sent Done false
24813 8006 Acked Write 5 Done Sent Done false
24814 8007 Acked Write 3 Done Sent Done false
24815 8008 Acked Write 5 Done Sent Done false
24816 8009 Acked Write 8 Done Sent Done false
24817 8010 Acked Write 7 Done Sent Done false
24818 8011 Acked Read 4 Done Sent Done false
24819 8012 Acked Write 9 Done Sent Done false
24820 8013 Acked Read 9 Done Sent Done false
24821 8014 Acked Read 6 Done Sent Done false
24822 8015 Acked Flush 0 Done Sent Done false
24823 8016 Acked Read 6 Done Sent Done false
24824 8017 Acked Flush 0 Done Sent Done false
24825 8018 Acked Flush 0 Done Sent Done false
24826 8019 Acked Write 7 Done Sent Done false
24827 8020 Acked Read 3 Done Sent Done false
24828 8021 Acked Read 10 Done Sent Done false
24829 8022 Acked Write 10 Done Sent Done false
24830 8023 Acked Read 8 Done Sent Done false
24831 8024 Acked Flush 0 Done Sent Done false
24832 8025 Acked Write 2 Done Sent Done false
24833 8026 Acked Read 3 Done Sent Done false
24834 8027 Acked Read 9 Done Sent Done false
24835 8028 Acked Write 8 Done Sent Done false
24836 8029 Acked Read 7 Done Sent Done false
24837 8030 Acked Read 5 Done Sent Done false
24838 8031 Acked Write 7 Done Sent Done false
24839 8032 Acked Write 8 Done Sent Done false
24840 8033 Acked Read 7 Done Sent Done false
24841 8034 Acked Read 3 Done Sent Done false
24842 8035 Acked Write 1 Done Sent Done false
24843 8036 Acked Read 5 Done Sent Done false
24844 8037 Acked Read 2 Done Sent Done false
24845 8038 Acked Read 10 Done Sent Done false
24846 8039 Acked Read 1 Done Sent Done false
24847 8040 Acked Read 10 Done Sent Done false
24848 8041 Acked Read 1 Done Sent Done false
24849 8042 Acked Read 6 Done Sent Done false
24850 8043 Acked Read 9 Done Sent Done false
24851 8044 Acked Read 6 Done Sent Done false
24852 8045 Acked Read 5 Done Sent Done false
24853 8046 Acked Read 6 Done Sent Done false
24854 8047 Acked Read 2 Done Sent Done false
24855 8048 Acked Read 4 Done Sent Done false
24856 8049 Acked Write 4 Done Sent Done false
24857 8050 Acked Read 2 Done Sent Done false
24858 8051 Acked Write 4 Done Sent Done false
24859 8052 Acked Write 7 Done Sent Done false
24860 8053 Acked Write 6 Done Sent Done false
24861 8054 Acked Read 5 Done Sent Done false
24862 8055 Acked Write 1 Done Sent Done false
24863 8056 Acked Read 2 Done Sent Done false
24864 8057 Acked Read 9 Done Sent Done false
24865 8058 Acked Read 3 Done Sent Done false
24866 8059 Acked Read 1 Done Sent Done false
24867 8060 Acked Read 4 Done Sent Done false
24868 8061 Acked Write 9 Done Sent Done false
24869 8062 Acked Write 4 Done Sent Done false
24870 8063 Acked Write 7 Done Sent Done false
24871 8064 Acked Write 8 Done Sent Done false
24872 8065 Acked Write 8 Done Sent Done false
24873 8066 Acked Write 5 Done Sent Done false
24874 8067 Acked Write 4 Done Sent Done false
24875 8068 Acked Write 2 Done Sent Done false
24876 8069 Acked Flush 0 Done Sent Done false
24877 8070 Acked Read 3 Done Sent Done false
24878 8071 Acked Read 6 Done Sent Done false
24879 8072 Acked Read 10 Done Sent Done false
24880 8073 Acked Write 5 Done Sent Done false
24881 8074 Acked Read 4 Done Sent Done false
24882 8075 Acked Flush 0 Done Sent Done false
24883 8076 Acked Write 10 Done Sent Done false
24884 8077 Acked Read 4 Done Sent Done false
24885 8078 Acked Write 1 Done Sent Done false
24886 8079 Acked Read 2 Done Sent Done false
24887 8080 Acked Write 9 Done Sent Done false
24888 8081 Acked Flush 0 Done Sent Done false
24889 8082 Acked Write 10 Done Sent Done false
24890 8083 Acked Read 9 Done Sent Done false
24891 8084 Acked Write 2 Done Sent Done false
24892 8085 Acked Read 7 Done Sent Done false
24893 8086 Acked Read 8 Done Sent Done false
24894 8087 Acked Flush 0 Done Sent Done false
24895 8088 Acked Flush 0 Done Sent Done false
24896 8089 Acked Write 5 Done Sent Done false
24897 8090 Acked Read 8 Done Sent Done false
24898 8091 Acked Read 4 Done Sent Done false
24899 8092 Acked Read 9 Done Sent Done false
24900 8093 Acked Read 3 Done Sent Done false
24901 8094 Acked Read 8 Done Sent Done false
24902 8095 Acked Read 6 Done Sent Done false
24903 8096 Acked Read 8 Done Sent Done false
24904 8097 Acked Flush 0 Done Sent Done false
24905 8098 Acked Write 4 Done Sent Done false
24906 8099 Acked Read 3 Done Sent Done false
24907 8100 Acked Write 4 Done Sent Done false
24908 8101 Acked Read 10 Done Sent Done false
24909 8102 Acked Read 10 Done Sent Done false
24910 8103 Acked Read 5 Done Sent Done false
24911 8104 Acked Write 10 Done Sent Done false
24912 8105 Acked Read 7 Done Sent Done false
24913 8106 Acked Read 8 Done Sent Done false
24914 8107 Acked Write 3 Done Sent Done false
24915 8108 Acked Write 7 Done Sent Done false
24916 8109 Acked Write 1 Done Sent Done false
24917 8110 Acked Write 8 Done Sent Done false
24918 8111 Acked Read 1 Done Sent Done false
24919 8112 Acked Read 7 Done Sent Done false
24920 8113 Acked Read 2 Done Sent Done false
24921 8114 Acked Read 5 Done Sent Done false
24922 8115 Acked Read 4 Done Sent Done false
24923 8116 Acked Read 9 Done Sent Done false
24924 8117 Acked Read 8 Done Sent Done false
24925 8118 Acked Read 9 Done Sent Done false
24926 8119 Acked Write 1 Done Sent Done false
24927 8120 Acked Read 2 Done Sent Done false
24928 8121 Acked Read 8 Done Sent Done false
24929 8122 Acked Write 10 Done Sent Done false
24930 8123 Acked Write 3 Done Sent Done false
24931 8124 Acked Flush 0 Done Sent Done false
24932 8125 Acked Write 1 Done Sent Done false
24933 8126 Acked Write 3 Done Sent Done false
24934 8127 Acked Read 6 Done Sent Done false
24935 8128 Acked Read 10 Done Sent Done false
24936 8129 Acked Read 4 Done Sent Done false
24937 8130 Acked Write 4 Done Sent Done false
24938 8131 Acked Read 1 Done Sent Done false
24939 8132 Acked Write 1 Done Sent Done false
24940 8133 Acked Read 3 Done Sent Done false
24941 8134 Acked Write 4 Done Sent Done false
24942 8135 Acked Read 6 Done Sent Done false
24943 8136 Acked Read 7 Done Sent Done false
24944 8137 Acked Read 7 Done Sent Done false
24945 8138 Acked Read 5 Done Sent Done false
24946 8139 Acked Write 8 Done Sent Done false
24947 8140 Acked Read 4 Done Sent Done false
24948 8141 Acked Write 6 Done Sent Done false
24949 8142 Acked Read 2 Done Sent Done false
24950 8143 Acked Read 8 Done Sent Done false
24951 8144 Acked Write 2 Done Sent Done false
24952 8145 Acked Write 1 Done Sent Done false
24953 8146 Acked Read 9 Done Sent Done false
24954 8147 Acked Write 8 Done Sent Done false
24955 8148 Acked Read 7 Done Sent Done false
24956 8149 Acked Read 5 Done Sent Done false
24957 8150 Acked Read 2 Done Sent Done false
24958 8151 Acked Flush 0 Done Sent Done false
24959 8152 Acked Read 10 Done Sent Done false
24960 8153 Acked Flush 0 Done Sent Done false
24961 8154 Acked Read 9 Done Sent Done false
24962 8155 Acked Flush 0 Done Sent Done false
24963 8156 Acked Write 9 Done Sent Done false
24964 8157 Acked Read 2 Done Sent Done false
24965 8158 Acked Read 8 Done Sent Done false
24966 8159 Acked Write 10 Done Sent Done false
24967 8160 Acked Read 3 Done Sent Done false
24968 8161 Acked Write 6 Done Sent Done false
24969 8162 Acked Read 7 Done Sent Done false
24970 8163 Acked Read 3 Done Sent Done false
24971 8164 Acked Read 9 Done Sent Done false
24972 8165 Acked Read 8 Done Sent Done false
24973 8166 Acked Write 3 Done Sent Done false
24974 8167 Acked Read 1 Done Sent Done false
24975 8168 Acked Read 7 Done Sent Done false
24976 8169 Acked Read 5 Done Sent Done false
24977 8170 Acked Write 6 Done Sent Done false
24978 8171 Acked Read 3 Done Sent Done false
24979 8172 Acked Read 8 Done Sent Done false
24980 8173 Acked Write 2 Done Sent Done false
24981 8174 Acked Write 6 Done Sent Done false
24982 8175 Acked Read 2 Done Sent Done false
24983 8176 Acked Write 4 Done Sent Done false
24984 8177 Acked Read 2 Done Sent Done false
24985 8178 Acked Read 1 Done Sent Done false
24986 8179 Acked Write 5 Done Sent Done false
24987 8180 Acked Read 8 Done Sent Done false
24988 8181 Acked Read 8 Done Sent Done false
24989 8182 Acked Write 6 Done Sent Done false
24990 8183 Acked Read 1 Done Sent Done false
24991 8184 Acked Write 3 Done Sent Done false
24992 8185 Acked Read 3 Done Sent Done false
24993 8186 Acked Flush 0 Done Sent Done false
24994 8187 Acked Write 8 Done Sent Done false
24995 8188 Acked Write 6 Done Sent Done false
24996 8189 Acked Write 9 Done Sent Done false
24997 8190 Acked Read 6 Done Sent Done false
24998 8191 Acked Read 4 Done Sent Done false
24999 8192 Acked Read 6 Done Sent Done false
25000 8193 Acked Write 8 Done Sent Done false
25001 8194 Acked Write 4 Done Sent Done false
25002 8195 Acked Read 10 Done Sent Done false
25003 8196 Acked Write 1 Done Sent Done false
25004 8197 Acked Write 3 Done Sent Done false
25005 8198 Acked Read 7 Done Sent Done false
25006 8199 Acked Write 7 Done Sent Done false
25007 8200 Acked Write 1 Done Sent Done false
25008 8201 Acked Flush 0 Done Sent Done false
25009 8202 Acked Write 2 Done Sent Done false
25010 8203 Acked Read 2 Done Sent Done false
25011 8204 Acked Read 4 Done Sent Done false
25012 8205 Acked Write 10 Done Sent Done false
25013 8206 Acked Write 9 Done Sent Done false
25014 8207 Acked Read 6 Done Sent Done false
25015 8208 Acked Read 9 Done Sent Done false
25016 8209 Acked Read 5 Done Sent Done false
25017 8210 Acked Write 4 Done Sent Done false
25018 8211 Acked Read 1 Done Sent Done false
25019 8212 Acked Read 2 Done Sent Done false
25020 8213 Acked Write 4 Done Sent Done false
25021 8214 Acked Write 2 Done Sent Done false
25022 8215 Acked Read 4 Done Sent Done false
25023 8216 Acked Read 9 Done Sent Done false
25024 8217 Acked Write 1 Done Sent Done false
25025 8218 Acked Read 9 Done Sent Done false
25026 8219 Acked Read 2 Done Sent Done false
25027 8220 Acked Flush 0 Done Sent Done false
25028 8221 Acked Read 3 Done Sent Done false
25029 8222 Acked Write 1 Done Sent Done false
25030 8223 Acked Write 9 Done Sent Done false
25031 8224 Acked Write 1 Done Sent Done false
25032 8225 Acked Read 9 Done Sent Done false
25033 8226 Acked Read 2 Done Sent Done false
25034 8227 Acked Write 10 Done Sent Done false
25035 8228 Acked Read 8 Done Sent Done false
25036 8229 Acked Write 6 Done Sent Done false
25037 8230 Acked Write 6 Done Sent Done false
25038 8231 Acked Read 8 Done Sent Done false
25039 8232 Acked Write 9 Done Sent Done false
25040 8233 Acked Write 7 Done Sent Done false
25041 8234 Acked Read 2 Done Sent Done false
25042 8235 Acked Write 2 Done Sent Done false
25043 8236 Acked Read 2 Done Sent Done false
25044 8237 Acked Write 3 Done Sent Done false
25045 8238 Acked Read 8 Done Sent Done false
25046 8239 Acked Read 5 Done Sent Done false
25047 8240 Acked Flush 0 Done Sent Done false
25048 8241 Acked Read 2 Done Sent Done false
25049 8242 Acked Read 9 Done Sent Done false
25050 8243 Acked Read 2 Done Sent Done false
25051 8244 Acked Flush 0 Done Sent Done false
25052 STATES DS:0 DS:1 DS:2 TOTAL
25053 Sent 0 302 0 302
25054 Done 302 0 302 604
25055 Skipped 0 0 0 0
25056 Error 0 0 0 0
25057 Last Flush: 8244 7942 8244
25058 Downstairs last five completed: 7942 7941 7940 7939 7938
25059 Upstairs last five completed: 8244 8243 8242 8241 8240
25060 CLIENT: Up:0 ds:302 act:2
25061 [8820] got start action
25062 [8820] I am going to restart
25063 Starting downstairs at port 8820
25064 Make output file at "/tmp/dsc/downstairs-8820.txt"
25065 [8820][1] reports Starting
25066 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1226)
25067 [8820][1] reports Running
25068 ----------------------------------------------------------------
25069 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
25070 States: Active Offline Active
25071 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25072 7943 Acked Write 9 Done Sent Done false
25073 7944 Acked Read 8 Done Sent Done false
25074 7945 Acked Read 8 Done Sent Done false
25075 7946 Acked Write 3 Done Sent Done false
25076 7947 Acked Read 3 Done Sent Done false
25077 7948 Acked Read 7 Done Sent Done false
25078 7949 Acked Read 9 Done Sent Done false
25079 7950 Acked Write 9 Done Sent Done false
25080 7951 Acked Read 5 Done Sent Done false
25081 7952 Acked Write 1 Done Sent Done false
25082 7953 Acked Write 7 Done Sent Done false
25083 7954 Acked Read 3 Done Sent Done false
25084 7955 Acked Write 7 Done Sent Done false
25085 7956 Acked Read 2 Done Sent Done false
25086 7957 Acked Read 6 Done Sent Done false
25087 7958 Acked Read 5 Done Sent Done false
25088 7959 Acked Read 7 Done Sent Done false
25089 7960 Acked Read 7 Done Sent Done false
25090 7961 Acked Read 4 Done Sent Done false
25091 7962 Acked Write 4 Done Sent Done false
25092 7963 Acked Read 7 Done Sent Done false
25093 7964 Acked Write 8 Done Sent Done false
25094 7965 Acked Write 10 Done Sent Done false
25095 7966 Acked Flush 0 Done Sent Done false
25096 7967 Acked Read 6 Done Sent Done false
25097 7968 Acked Read 7 Done Sent Done false
25098 7969 Acked Read 1 Done Sent Done false
25099 7970 Acked Read 5 Done Sent Done false
25100 7971 Acked Write 5 Done Sent Done false
25101 7972 Acked Read 1 Done Sent Done false
25102 7973 Acked Write 9 Done Sent Done false
25103 7974 Acked Write 1 Done Sent Done false
25104 7975 Acked Read 7 Done Sent Done false
25105 7976 Acked Read 5 Done Sent Done false
25106 7977 Acked Write 9 Done Sent Done false
25107 7978 Acked Write 8 Done Sent Done false
25108 7979 Acked Flush 0 Done Sent Done false
25109 7980 Acked Read 9 Done Sent Done false
25110 7981 Acked Flush 0 Done Sent Done false
25111 7982 Acked Read 7 Done Sent Done false
25112 7983 Acked Write 5 Done Sent Done false
25113 7984 Acked Read 9 Done Sent Done false
25114 7985 Acked Read 5 Done Sent Done false
25115 7986 Acked Write 3 Done Sent Done false
25116 7987 Acked Read 3 Done Sent Done false
25117 7988 Acked Write 1 Done Sent Done false
25118 7989 Acked Read 9 Done Sent Done false
25119 7990 Acked Write 10 Done Sent Done false
25120 7991 Acked Write 3 Done Sent Done false
25121 7992 Acked Read 6 Done Sent Done false
25122 7993 Acked Write 9 Done Sent Done false
25123 7994 Acked Write 6 Done Sent Done false
25124 7995 Acked Write 5 Done Sent Done false
25125 7996 Acked Read 2 Done Sent Done false
25126 7997 Acked Read 2 Done Sent Done false
25127 7998 Acked Read 7 Done Sent Done false
25128 7999 Acked Write 2 Done Sent Done false
25129 8000 Acked Read 2 Done Sent Done false
25130 8001 Acked Write 3 Done Sent Done false
25131 8002 Acked Read 3 Done Sent Done false
25132 8003 Acked Read 4 Done Sent Done false
25133 8004 Acked Read 10 Done Sent Done false
25134 8005 Acked Write 8 Done Sent Done false
25135 8006 Acked Write 5 Done Sent Done false
25136 8007 Acked Write 3 Done Sent Done false
25137 8008 Acked Write 5 Done Sent Done false
25138 8009 Acked Write 8 Done Sent Done false
25139 8010 Acked Write 7 Done Sent Done false
25140 8011 Acked Read 4 Done Sent Done false
25141 8012 Acked Write 9 Done Sent Done false
25142 8013 Acked Read 9 Done Sent Done false
25143 8014 Acked Read 6 Done Sent Done false
25144 8015 Acked Flush 0 Done Sent Done false
25145 8016 Acked Read 6 Done Sent Done false
25146 8017 Acked Flush 0 Done Sent Done false
25147 8018 Acked Flush 0 Done Sent Done false
25148 8019 Acked Write 7 Done Sent Done false
25149 8020 Acked Read 3 Done Sent Done false
25150 8021 Acked Read 10 Done Sent Done false
25151 8022 Acked Write 10 Done Sent Done false
25152 8023 Acked Read 8 Done Sent Done false
25153 8024 Acked Flush 0 Done Sent Done false
25154 8025 Acked Write 2 Done Sent Done false
25155 8026 Acked Read 3 Done Sent Done false
25156 8027 Acked Read 9 Done Sent Done false
25157 8028 Acked Write 8 Done Sent Done false
25158 8029 Acked Read 7 Done Sent Done false
25159 8030 Acked Read 5 Done Sent Done false
25160 8031 Acked Write 7 Done Sent Done false
25161 8032 Acked Write 8 Done Sent Done false
25162 8033 Acked Read 7 Done Sent Done false
25163 8034 Acked Read 3 Done Sent Done false
25164 8035 Acked Write 1 Done Sent Done false
25165 8036 Acked Read 5 Done Sent Done false
25166 8037 Acked Read 2 Done Sent Done false
25167 8038 Acked Read 10 Done Sent Done false
25168 8039 Acked Read 1 Done Sent Done false
25169 8040 Acked Read 10 Done Sent Done false
25170 8041 Acked Read 1 Done Sent Done false
25171 8042 Acked Read 6 Done Sent Done false
25172 8043 Acked Read 9 Done Sent Done false
25173 8044 Acked Read 6 Done Sent Done false
25174 8045 Acked Read 5 Done Sent Done false
25175 8046 Acked Read 6 Done Sent Done false
25176 8047 Acked Read 2 Done Sent Done false
25177 8048 Acked Read 4 Done Sent Done false
25178 8049 Acked Write 4 Done Sent Done false
25179 8050 Acked Read 2 Done Sent Done false
25180 8051 Acked Write 4 Done Sent Done false
25181 8052 Acked Write 7 Done Sent Done false
25182 8053 Acked Write 6 Done Sent Done false
25183 8054 Acked Read 5 Done Sent Done false
25184 8055 Acked Write 1 Done Sent Done false
25185 8056 Acked Read 2 Done Sent Done false
25186 8057 Acked Read 9 Done Sent Done false
25187 8058 Acked Read 3 Done Sent Done false
25188 8059 Acked Read 1 Done Sent Done false
25189 8060 Acked Read 4 Done Sent Done false
25190 8061 Acked Write 9 Done Sent Done false
25191 8062 Acked Write 4 Done Sent Done false
25192 8063 Acked Write 7 Done Sent Done false
25193 8064 Acked Write 8 Done Sent Done false
25194 8065 Acked Write 8 Done Sent Done false
25195 8066 Acked Write 5 Done Sent Done false
25196 8067 Acked Write 4 Done Sent Done false
25197 8068 Acked Write 2 Done Sent Done false
25198 8069 Acked Flush 0 Done Sent Done false
25199 8070 Acked Read 3 Done Sent Done false
25200 8071 Acked Read 6 Done Sent Done false
25201 8072 Acked Read 10 Done Sent Done false
25202 8073 Acked Write 5 Done Sent Done false
25203 8074 Acked Read 4 Done Sent Done false
25204 8075 Acked Flush 0 Done Sent Done false
25205 8076 Acked Write 10 Done Sent Done false
25206 8077 Acked Read 4 Done Sent Done false
25207 8078 Acked Write 1 Done Sent Done false
25208 8079 Acked Read 2 Done Sent Done false
25209 8080 Acked Write 9 Done Sent Done false
25210 8081 Acked Flush 0 Done Sent Done false
25211 8082 Acked Write 10 Done Sent Done false
25212 8083 Acked Read 9 Done Sent Done false
25213 8084 Acked Write 2 Done Sent Done false
25214 8085 Acked Read 7 Done Sent Done false
25215 8086 Acked Read 8 Done Sent Done false
25216 8087 Acked Flush 0 Done Sent Done false
25217 8088 Acked Flush 0 Done Sent Done false
25218 8089 Acked Write 5 Done Sent Done false
25219 8090 Acked Read 8 Done Sent Done false
25220 8091 Acked Read 4 Done Sent Done false
25221 8092 Acked Read 9 Done Sent Done false
25222 8093 Acked Read 3 Done Sent Done false
25223 8094 Acked Read 8 Done Sent Done false
25224 8095 Acked Read 6 Done Sent Done false
25225 8096 Acked Read 8 Done Sent Done false
25226 8097 Acked Flush 0 Done Sent Done false
25227 8098 Acked Write 4 Done Sent Done false
25228 8099 Acked Read 3 Done Sent Done false
25229 8100 Acked Write 4 Done Sent Done false
25230 8101 Acked Read 10 Done Sent Done false
25231 8102 Acked Read 10 Done Sent Done false
25232 8103 Acked Read 5 Done Sent Done false
25233 8104 Acked Write 10 Done Sent Done false
25234 8105 Acked Read 7 Done Sent Done false
25235 8106 Acked Read 8 Done Sent Done false
25236 8107 Acked Write 3 Done Sent Done false
25237 8108 Acked Write 7 Done Sent Done false
25238 8109 Acked Write 1 Done Sent Done false
25239 8110 Acked Write 8 Done Sent Done false
25240 8111 Acked Read 1 Done Sent Done false
25241 8112 Acked Read 7 Done Sent Done false
25242 8113 Acked Read 2 Done Sent Done false
25243 8114 Acked Read 5 Done Sent Done false
25244 8115 Acked Read 4 Done Sent Done false
25245 8116 Acked Read 9 Done Sent Done false
25246 8117 Acked Read 8 Done Sent Done false
25247 8118 Acked Read 9 Done Sent Done false
25248 8119 Acked Write 1 Done Sent Done false
25249 8120 Acked Read 2 Done Sent Done false
25250 8121 Acked Read 8 Done Sent Done false
25251 8122 Acked Write 10 Done Sent Done false
25252 8123 Acked Write 3 Done Sent Done false
25253 8124 Acked Flush 0 Done Sent Done false
25254 8125 Acked Write 1 Done Sent Done false
25255 8126 Acked Write 3 Done Sent Done false
25256 8127 Acked Read 6 Done Sent Done false
25257 8128 Acked Read 10 Done Sent Done false
25258 8129 Acked Read 4 Done Sent Done false
25259 8130 Acked Write 4 Done Sent Done false
25260 8131 Acked Read 1 Done Sent Done false
25261 8132 Acked Write 1 Done Sent Done false
25262 8133 Acked Read 3 Done Sent Done false
25263 8134 Acked Write 4 Done Sent Done false
25264 8135 Acked Read 6 Done Sent Done false
25265 8136 Acked Read 7 Done Sent Done false
25266 8137 Acked Read 7 Done Sent Done false
25267 8138 Acked Read 5 Done Sent Done false
25268 8139 Acked Write 8 Done Sent Done false
25269 8140 Acked Read 4 Done Sent Done false
25270 8141 Acked Write 6 Done Sent Done false
25271 8142 Acked Read 2 Done Sent Done false
25272 8143 Acked Read 8 Done Sent Done false
25273 8144 Acked Write 2 Done Sent Done false
25274 8145 Acked Write 1 Done Sent Done false
25275 8146 Acked Read 9 Done Sent Done false
25276 8147 Acked Write 8 Done Sent Done false
25277 8148 Acked Read 7 Done Sent Done false
25278 8149 Acked Read 5 Done Sent Done false
25279 8150 Acked Read 2 Done Sent Done false
25280 8151 Acked Flush 0 Done Sent Done false
25281 8152 Acked Read 10 Done Sent Done false
25282 8153 Acked Flush 0 Done Sent Done false
25283 8154 Acked Read 9 Done Sent Done false
25284 8155 Acked Flush 0 Done Sent Done false
25285 8156 Acked Write 9 Done Sent Done false
25286 8157 Acked Read 2 Done Sent Done false
25287 8158 Acked Read 8 Done Sent Done false
25288 8159 Acked Write 10 Done Sent Done false
25289 8160 Acked Read 3 Done Sent Done false
25290 8161 Acked Write 6 Done Sent Done false
25291 8162 Acked Read 7 Done Sent Done false
25292 8163 Acked Read 3 Done Sent Done false
25293 8164 Acked Read 9 Done Sent Done false
25294 8165 Acked Read 8 Done Sent Done false
25295 8166 Acked Write 3 Done Sent Done false
25296 8167 Acked Read 1 Done Sent Done false
25297 8168 Acked Read 7 Done Sent Done false
25298 8169 Acked Read 5 Done Sent Done false
25299 8170 Acked Write 6 Done Sent Done false
25300 8171 Acked Read 3 Done Sent Done false
25301 8172 Acked Read 8 Done Sent Done false
25302 8173 Acked Write 2 Done Sent Done false
25303 8174 Acked Write 6 Done Sent Done false
25304 8175 Acked Read 2 Done Sent Done false
25305 8176 Acked Write 4 Done Sent Done false
25306 8177 Acked Read 2 Done Sent Done false
25307 8178 Acked Read 1 Done Sent Done false
25308 8179 Acked Write 5 Done Sent Done false
25309 8180 Acked Read 8 Done Sent Done false
25310 8181 Acked Read 8 Done Sent Done false
25311 8182 Acked Write 6 Done Sent Done false
25312 8183 Acked Read 1 Done Sent Done false
25313 8184 Acked Write 3 Done Sent Done false
25314 8185 Acked Read 3 Done Sent Done false
25315 8186 Acked Flush 0 Done Sent Done false
25316 8187 Acked Write 8 Done Sent Done false
25317 8188 Acked Write 6 Done Sent Done false
25318 8189 Acked Write 9 Done Sent Done false
25319 8190 Acked Read 6 Done Sent Done false
25320 8191 Acked Read 4 Done Sent Done false
25321 8192 Acked Read 6 Done Sent Done false
25322 8193 Acked Write 8 Done Sent Done false
25323 8194 Acked Write 4 Done Sent Done false
25324 8195 Acked Read 10 Done Sent Done false
25325 8196 Acked Write 1 Done Sent Done false
25326 8197 Acked Write 3 Done Sent Done false
25327 8198 Acked Read 7 Done Sent Done false
25328 8199 Acked Write 7 Done Sent Done false
25329 8200 Acked Write 1 Done Sent Done false
25330 8201 Acked Flush 0 Done Sent Done false
25331 8202 Acked Write 2 Done Sent Done false
25332 8203 Acked Read 2 Done Sent Done false
25333 8204 Acked Read 4 Done Sent Done false
25334 8205 Acked Write 10 Done Sent Done false
25335 8206 Acked Write 9 Done Sent Done false
25336 8207 Acked Read 6 Done Sent Done false
25337 8208 Acked Read 9 Done Sent Done false
25338 8209 Acked Read 5 Done Sent Done false
25339 8210 Acked Write 4 Done Sent Done false
25340 8211 Acked Read 1 Done Sent Done false
25341 8212 Acked Read 2 Done Sent Done false
25342 8213 Acked Write 4 Done Sent Done false
25343 8214 Acked Write 2 Done Sent Done false
25344 8215 Acked Read 4 Done Sent Done false
25345 8216 Acked Read 9 Done Sent Done false
25346 8217 Acked Write 1 Done Sent Done false
25347 8218 Acked Read 9 Done Sent Done false
25348 8219 Acked Read 2 Done Sent Done false
25349 8220 Acked Flush 0 Done Sent Done false
25350 8221 Acked Read 3 Done Sent Done false
25351 8222 Acked Write 1 Done Sent Done false
25352 8223 Acked Write 9 Done Sent Done false
25353 8224 Acked Write 1 Done Sent Done false
25354 8225 Acked Read 9 Done Sent Done false
25355 8226 Acked Read 2 Done Sent Done false
25356 8227 Acked Write 10 Done Sent Done false
25357 8228 Acked Read 8 Done Sent Done false
25358 8229 Acked Write 6 Done Sent Done false
25359 8230 Acked Write 6 Done Sent Done false
25360 8231 Acked Read 8 Done Sent Done false
25361 8232 Acked Write 9 Done Sent Done false
25362 8233 Acked Write 7 Done Sent Done false
25363 8234 Acked Read 2 Done Sent Done false
25364 8235 Acked Write 2 Done Sent Done false
25365 8236 Acked Read 2 Done Sent Done false
25366 8237 Acked Write 3 Done Sent Done false
25367 8238 Acked Read 8 Done Sent Done false
25368 8239 Acked Read 5 Done Sent Done false
25369 8240 Acked Flush 0 Done Sent Done false
25370 8241 Acked Read 2 Done Sent Done false
25371 8242 Acked Read 9 Done Sent Done false
25372 8243 Acked Read 2 Done Sent Done false
25373 8244 Acked Flush 0 Done Sent Done false
25374 STATES DS:0 DS:1 DS:2 TOTAL
25375 Sent 0 302 0 302
25376 Done 302 0 302 604
25377 Skipped 0 0 0 0
25378 Error 0 0 0 0
25379 Last Flush: 8244 7942 8244
25380 Downstairs last five completed: 7942 7941 7940 7939 7938
25381 Upstairs last five completed: 8244 8243 8242 8241 8240
25382 CLIENT: Up:0 ds:302 act:2
253832025-01-14T16:15:57.547ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
253842025-01-14T16:15:57.547ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:60635) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
253852025-01-14T16:15:57.548ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
253862025-01-14T16:15:57.549ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
253872025-01-14T16:15:57.549ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
253882025-01-14T16:15:57.549ZINFOcrucible: send last flush ID to this DS: 7942 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
253892025-01-14T16:15:57.549ZINFOcrucible: Replied this last flush ID: 7942 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
253902025-01-14T16:15:57.549ZINFOcrucible: [1] client re-new 302 jobs since flush 7942 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
253912025-01-14T16:15:57.549ZINFOcrucible: [1] Replayed 302 jobs since flush: 7942 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
25392 ----------------------------------------------------------------
25393 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
25394 Downstairs last five completed: 8244 8243 8242 8241 8240
25395 Upstairs last five completed: 8244 8243 8242 8241 8240
25396 CLIENT: Up:0 ds:0 act:3
25397 Replay: All jobs finished
25398 Main task has work to do, go find it
25399 got dsc Stop(2)
25400 stop 2
25401 Jan 14 16:15:57.796 INFO request completed, [8830] Got stop action so:false kr:false
25402 latency_us: 340, response_code: 204, uri: /stop/cid/2, method: POST, req_id: fdf7032d-87ef-4f9b-9666-7fa4fe9d6db6, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
254032025-01-14T16:15:57.797ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
254042025-01-14T16:15:57.797ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
254052025-01-14T16:15:57.797ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
25406 Jan 14 16:15:57.797 INFO request completed, latency_us: 118, response_code: 200, uri: /state/cid/2, method: GET, req_id: 474715dd-9bad-4e99-826c-a4c6a9a9048b, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
254072025-01-14T16:15:57.797ZINFOcrucible: client task is exiting = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
25408 [8830] Exited with: ExitStatus(unix_wait_status(9))
25409 [8830][2] reports Exit
25410 Jan 14 16:16:01.800 INFO request completed, latency_us: 237, response_code: 200, uri: /state/cid/2, method: GET, req_id: 05255ad7-64bd-41d7-be61-1b758f4b19a1, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
25411 001/300 Read block 60 len 36864
25412 002/300 Write block 3888 len 40960 data: 6 5 4 4 3 2 4 6 7 6
25413 003/300 Write block 1545 len 32768 data: 6 5 5 5 5 5 5 7
25414 004/300 Write block 420 len 16384 data: 7 8 9 8
25415 005/300 Read block 1906 len 40960
25416 006/300 Read block 1652 len 40960
25417 007/300 Read block 1443 len 12288
25418 008/300 Write block 401 len 32768 data: 6 5 6 8 6 5 5 6
25419 009/300 Read block 3977 len 32768
25420 010/300 Read block 2999 len 32768
25421 011/300 Flush
25422 012/300 Write block 1530 len 8192 data: 8 8
25423 013/300 Write block 1924 len 40960 data: 4 5 5 5 4 3 4 3 3 3
25424 014/300 Flush
25425 015/300 Read block 881 len 20480
25426 016/300 Read block 3425 len 12288
25427 017/300 Read block 3832 len 12288
25428 018/300 Write block 3279 len 36864 data: 5 5 6 8 7 7 6 6 5
25429 019/300 Read block 4862 len 16384
25430 020/300 Read block 1890 len 16384
25431 021/300 Write block 3717 len 16384 data: 5 6 4 6
25432 022/300 Read block 3658 len 40960
25433 023/300 Write block 2905 len 28672 data: 9 8 9 9 7 7 5
25434 024/300 Write block 3890 len 24576 data: 5 5 4 3 5 7
25435 025/300 Read block 4346 len 28672
25436 026/300 Read block 4600 len 4096
25437 027/300 Write block 527 len 24576 data: 6 5 7 6 6 5
25438 028/300 Write block 4466 len 32768 data: 5 6 5 5 5 4 5 5
25439 029/300 Read block 14 len 36864
25440 030/300 Write block 904 len 28672 data: 9 8 8 7 9 9 9
25441 031/300 Read block 3835 len 40960
25442 032/300 Read block 2424 len 12288
25443 033/300 Write block 3629 len 12288 data: 7 6 6
25444 034/300 Write block 3990 len 20480 data: 4 5 6 6 6
25445 035/300 Read block 2970 len 36864
25446 036/300 Write block 2402 len 36864 data: 8 7 8 8 6 4 5 4 6
25447 037/300 Write block 3433 len 24576 data: 4 4 4 4 4 4
25448 038/300 Read block 3650 len 24576
25449 039/300 Read block 2402 len 36864
25450 040/300 Write block 2796 len 20480 data: 7 6 6 6 6
25451 041/300 Write block 4801 len 24576 data: 5 5 5 6 5 5
25452 042/300 Write block 122 len 28672 data: 7 8 9 9 8 7 6
25453 043/300 Write block 1470 len 40960 data: 6 6 6 6 6 5 4 4 5 5
25454 044/300 Write block 4261 len 28672 data: 5 5 6 6 6 6 5
25455 045/300 Read block 586 len 36864
25456 046/300 Read block 2374 len 40960
25457 047/300 Read block 4483 len 20480
25458 048/300 Write block 4452 len 32768 data: 4 3 2 2 2 2 2 3
25459 049/300 Write block 2259 len 8192 data: 5 5
25460 050/300 Write block 2787 len 16384 data: 4 4 4 3
25461 051/300 Write block 1781 len 28672 data: 4 5 4 3 3 3 3
25462 052/300 Write block 1614 len 32768 data: 4 5 5 7 7 7 6 6
25463 053/300 Read block 2927 len 28672
25464 054/300 Read block 2541 len 4096
25465 055/300 Read block 4754 len 28672
25466 056/300 Flush
25467 057/300 Read block 1562 len 28672
25468 058/300 Read block 1864 len 16384
25469 059/300 Read block 2047 len 28672
25470 060/300 Read block 1572 len 20480
25471 061/300 Write block 2743 len 20480 data: 4 4 4 4 4
25472 062/300 Read block 4574 len 40960
25473 063/300 Write block 1571 len 36864 data: 5 5 3 3 3 3 5 5 4
25474 064/300 Read block 313 len 20480
25475 065/300 Flush
25476 066/300 Read block 4715 len 16384
25477 067/300 Read block 4805 len 40960
25478 068/300 Read block 3479 len 4096
25479 069/300 Read block 4482 len 20480
25480 070/300 Write block 1081 len 12288 data: 6 4 4
25481 071/300 Flush
25482 072/300 Read block 3784 len 24576
25483 073/300 Write block 2383 len 12288 data: 4 4 4
25484 074/300 Flush
25485 075/300 Write block 4476 len 28672 data: 4 4 3 3 3 3 4
25486 076/300 Read block 4932 len 4096
25487 077/300 Write block 2763 len 20480 data: 6 3 4 3 4
25488 078/300 Write block 4434 len 4096 data: 6
25489 079/300 Write block 4019 len 40960 data: 3 3 2 3 3 4 4 6 6 8
25490 080/300 Read block 84 len 4096
25491 081/300 Read block 4372 len 24576
25492 082/300 Read block 3632 len 32768
25493 083/300 Read block 1158 len 8192
25494 084/300 Read block 2507 len 8192
25495 085/300 Write block 3188 len 28672 data: 6 5 6 6 5 7 6
25496 086/300 Write block 4182 len 8192 data: 4 4
25497 087/300 Write block 1199 len 40960 data: 4 3 3 3 3 4 5 5 5 3
25498 088/300 Read block 4533 len 28672
25499 089/300 Write block 2889 len 28672 data: 6 4 4 4 4 6 7
25500 090/300 Write block 2321 len 32768 data: 5 4 5 6 6 3 3 3
25501 091/300 Read block 2168 len 36864
25502 092/300 Read block 81 len 24576
25503 093/300 Read block 2656 len 36864
25504 094/300 Write block 352 len 40960 data: 5 4 5 6 4 4 4 4 4 4
25505 095/300 Read block 4436 len 16384
25506 096/300 Write block 1980 len 32768 data: 4 4 4 3 3 2 4 4
25507 097/300 Read block 1360 len 20480
25508 098/300 Write block 1773 len 12288 data: 6 6 5
25509 099/300 Write block 1503 len 24576 data: 4 4 5 5 4 4
25510 100/300 Read block 1552 len 36864
25511 101/300 Flush
25512 102/300 Write block 3391 len 12288 data: 7 8 6
25513 103/300 Flush
25514 104/300 Read block 3979 len 12288
25515 105/300 Write block 3351 len 20480 data: 5 7 7 6 5
25516 106/300 Write block 642 len 4096 data: 3
25517 107/300 Read block 1434 len 24576
25518 108/300 Write block 1708 len 28672 data: 4 4 4 3 4 5 6
25519 109/300 Write block 115 len 28672 data: 4 4 6 5 6 5 6
25520 110/300 Write block 1190 len 16384 data: 8 7 6 5
25521 111/300 Write block 3554 len 16384 data: 4 4 3 3
25522 112/300 Flush
25523 113/300 Flush
25524 114/300 Write block 2667 len 8192 data: 6 6
25525 115/300 Flush
25526 116/300 Read block 1019 len 32768
25527 117/300 Read block 560 len 24576
25528 118/300 Read block 3036 len 12288
25529 119/300 Read block 2815 len 16384
25530 120/300 Write block 2425 len 12288 data: 4 4 4
25531 121/300 Read block 2847 len 40960
25532 122/300 Read block 2638 len 28672
25533 123/300 Read block 9 len 32768
25534 124/300 Read block 2306 len 24576
25535 125/300 Flush
25536 126/300 Read block 4209 len 24576
25537 127/300 Flush
25538 128/300 Write block 2321 len 32768 data: 6 5 6 7 7 4 4 4
25539 129/300 Write block 1804 len 32768 data: 4 5 5 5 6 6 5 5
25540 130/300 Read block 790 len 16384
25541 131/300 Read block 2739 len 12288
25542 132/300 Write block 2959 len 12288 data: 5 5 3
25543 133/300 Write block 1982 len 24576 data: 5 4 4 3 5 5
25544 134/300 Write block 675 len 36864 data: 8 7 6 4 6 5 6 7 7
25545 135/300 Flush
25546 136/300 Flush
25547 137/300 Read block 1900 len 32768
25548 138/300 Write block 4847 len 20480 data: 9 8 8 8 5
25549 139/300 Write block 1205 len 24576 data: 6 6 6 4 3 3
25550 140/300 Read block 2059 len 32768
25551 141/300 Write block 3158 len 20480 data: 7 6 6 6 4
25552 142/300 Read block 2239 len 20480
25553 143/300 Write block 126 len 28672 data: 9 8 7 6 6 8 8
25554 144/300 Read block 4243 len 40960
25555 145/300 Write block 586 len 12288 data: 7 7 6
25556 146/300 Write block 3302 len 28672 data: 4 5 4 5 5 4 4
25557 147/300 Write block 1151 len 36864 data: 5 8 6 6 7 7 6 6 5
25558 148/300 Read block 1436 len 32768
25559 149/300 Write block 3400 len 16384 data: 6 9 7 7
25560 150/300 Flush
25561 151/300 Write block 757 len 8192 data: 6 5
25562 152/300 Read block 376 len 12288
25563 153/300 Flush
25564 154/300 Flush
25565 155/300 Flush
25566 156/300 Write block 79 len 24576 data: 9 9 7 8 6 5
25567 157/300 Write block 569 len 8192 data: 5 3
25568 158/300 Write block 3318 len 28672 data: 4 5 3 3 4 5 5
25569 159/300 Write block 3984 len 12288 data: 2 2 3
25570 160/300 Write block 2392 len 32768 data: 8 8 7 7 7 8 7 8
25571 161/300 Read block 1658 len 8192
25572 162/300 Write block 3716 len 24576 data: 5 6 7 5 7 5
25573 163/300 Read block 2911 len 20480
25574 164/300 Flush
25575 165/300 Write block 1833 len 28672 data: 6 5 6 6 7 8 7
25576 166/300 Flush
25577 167/300 Read block 1227 len 8192
25578 168/300 Read block 218 len 32768
25579 169/300 Write block 630 len 24576 data: 4 4 4 5 5 5
25580 170/300 Flush
25581 171/300 Read block 4590 len 28672
25582 172/300 Write block 1867 len 4096 data: 2
25583 173/300 Flush
25584 174/300 Flush
25585 175/300 Read block 81 len 4096
25586 176/300 Write block 1929 len 20480 data: 4 5 4 4 4
25587 177/300 Write block 3268 len 36864 data: 5 4 4 3 5 4 4 5 6
25588 178/300 Read block 1945 len 28672
25589 179/300 Read block 3020 len 36864
25590 180/300 Read block 4331 len 20480
25591 181/300 Write block 1583 len 4096 data: 4
25592 182/300 Write block 4719 len 24576 data: 3 2 3 4 5 5
25593 183/300 Read block 4809 len 28672
25594 184/300 Write block 3978 len 36864 data: 3 2 2 2 2 3 3 3 4
25595 185/300 Write block 1096 len 4096 data: 7
25596 186/300 Write block 2876 len 4096 data: 10
25597 187/300 Read block 3488 len 24576
25598 188/300 Write block 2163 len 40960 data: 4 4 3 3 3 4 4 5 5 5
25599 189/300 Write block 4040 len 32768 data: 3 4 3 5 6 7 8 7
25600 190/300 Write block 618 len 20480 data: 7 6 5 5 6
25601 191/300 Flush
25602 192/300 Write block 584 len 28672 data: 7 8 8 8 7 7 8
25603 193/300 Write block 3626 len 4096 data: 6
25604 194/300 Flush
25605 195/300 Read block 4014 len 28672
25606 196/300 Flush
25607 197/300 Write block 3224 len 12288 data: 5 5 4
25608 198/300 Read block 3018 len 24576
25609 199/300 Flush
25610 200/300 Read block 4978 len 12288
25611 201/300 Flush
25612 202/300 Write block 1542 len 28672 data: 6 8 6 7 6 6 6
25613 203/300 Write block 905 len 32768 data: 9 9 8 10 10 10 8 7
25614 204/300 Read block 2757 len 28672
25615 205/300 Read block 458 len 4096
25616 206/300 Read block 1118 len 36864
25617 207/300 Read block 66 len 28672
25618 208/300 Read block 4433 len 36864
25619 209/300 Read block 3029 len 40960
25620 210/300 Write block 2645 len 40960 data: 8 7 6 6 7 8 5 5 5 5
25621 211/300 Read block 2439 len 12288
25622 212/300 Flush
25623 213/300 Flush
25624 214/300 Write block 1741 len 4096 data: 8
25625 215/300 Write block 4129 len 12288 data: 8 6 6
25626 216/300 Read block 2203 len 16384
25627 217/300 Read block 963 len 36864
25628 218/300 Write block 3505 len 24576 data: 5 4 3 3 3 4
25629 219/300 Write block 201 len 32768 data: 5 5 4 4 6 6 8 7
25630 220/300 Read block 3158 len 8192
25631 221/300 Flush
25632 222/300 Write block 850 len 16384 data: 2 5 6 6
25633 223/300 Flush
25634 224/300 Read block 2139 len 12288
25635 225/300 Read block 1912 len 8192
25636 226/300 Write block 393 len 36864 data: 4 3 3 3 3 3 5 5 7
25637 227/300 Write block 2751 len 40960 data: 4 4 5 4 4 3 2 2 4 4
25638 228/300 Write block 2460 len 12288 data: 6 4 5
25639 229/300 Read block 3668 len 8192
25640 230/300 Write block 1349 len 16384 data: 4 4 6 7
25641 231/300 Write block 3087 len 40960 data: 3 3 4 5 5 4 3 3 3 3
25642 232/300 Write block 4939 len 16384 data: 2 2 4 5
25643 233/300 Flush
25644 234/300 Read block 4831 len 20480
25645 235/300 Flush
25646 236/300 Write block 3769 len 24576 data: 6 6 6 5 7 8
25647 237/300 Write block 1966 len 4096 data: 4
25648 238/300 Write block 1346 len 8192 data: 3 3
25649 239/300 Write block 2188 len 24576 data: 6 6 7 7 8 8
25650 240/300 Write block 4614 len 32768 data: 3 3 2 2 2 3 3 4
25651 241/300 Read block 1288 len 20480
25652 242/300 Read block 1345 len 4096
25653 243/300 Read block 1409 len 20480
25654 244/300 Read block 4457 len 32768
25655 245/300 Read block 1412 len 32768
25656 246/300 Read block 894 len 12288
25657 247/300 Read block 1235 len 36864
25658 248/300 Read block 4365 len 12288
25659 249/300 Flush
25660 250/300 Read block 1087 len 24576
25661 251/300 Read block 4805 len 12288
25662 252/300 Flush
25663 253/300 Flush
25664 254/300 Read block 4488 len 36864
25665 255/300 Write block 2740 len 4096 data: 5
25666 256/300 Read block 157 len 4096
25667 257/300 Read block 2045 len 40960
25668 258/300 Write block 931 len 12288 data: 5 5 4
25669 259/300 Read block 8 len 20480
25670 260/300 Flush
25671 261/300 Flush
25672 262/300 Read block 2177 len 36864
25673 263/300 Write block 84 len 24576 data: 6 4 4 4 3 2
25674 264/300 Flush
25675 265/300 Read block 3972 len 36864
25676 266/300 Read block 1193 len 16384
25677 267/300 Read block 3695 len 8192
25678 268/300 Write block 271 len 8192 data: 5 5
25679 269/300 Read block 96 len 8192
25680 270/300 Write block 4566 len 12288 data: 5 5 6
25681 271/300 Write block 4195 len 40960 data: 2 3 4 3 5 5 6 6 5 5
25682 272/300 Read block 219 len 32768
25683 273/300 Read block 2334 len 20480
25684 274/300 Write block 4734 len 36864 data: 4 5 5 4 5 4 6 7 7
25685 275/300 Write block 2250 len 36864 data: 3 3 3 3 4 4 3 4 4
25686 276/300 Write block 36 len 8192 data: 6 5
25687 277/300 Write block 191 len 8192 data: 3 3
25688 278/300 Read block 4050 len 28672
25689 279/300 Flush
25690 280/300 Write block 1435 len 16384 data: 7 5 5 7
25691 281/300 Read block 3536 len 24576
25692 282/300 Read block 2712 len 24576
25693 283/300 Read block 2446 len 28672
25694 284/300 Write block 1935 len 16384 data: 3 3 4 4
25695 285/300 Read block 3602 len 16384
25696 286/300 Write block 3224 len 20480 data: 6 6 5 7 6
25697 287/300 Read block 4911 len 8192
25698 288/300 Read block 2689 len 28672
25699 289/300 Read block 4031 len 12288
25700 290/300 Write block 4029 len 24576 data: 7 6 5 4 4 3
25701 291/300 Flush
25702 292/300 Flush
25703 293/300 Read block 3290 len 20480
25704 294/300 Flush
25705 295/300 Write block 4169 len 32768 data: 4 5 5 6 8 7 7 9
25706 296/300 Write block 1313 len 16384 data: 4 4 3 3
25707 297/300 Write block 47 len 20480 data: 3 3 3 4 4
25708 298/300 Write block 2642 len 12288 data: 6 8 8
25709 299/300 Flush
25710 300/300 Read block 3166 len 24576
25711 301/300 Write block 2244 len 32768 data: 4 4 5 5 5 4 4 4
25712 Main task has work to do, go find it
25713 got dsc Start(2)
25714 start 2
25715 Jan 14 16:16:02.028 INFO request completed, latency_us: 204, response_code: 204, uri: /start/cid/2, method: POST, req_id: 902e0ffc-216a-41d1-9bff-dab68c4a05c7, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
25716 Replay: started 2, returned:Ok(())
25717 ----------------------------------------------------------------
25718 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
25719 States: Active Active Offline
25720 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25721 8245 Acked Read 9 Done Done Sent false
25722 8246 Acked Write 10 Done Done Sent false
25723 8247 Acked Write 8 Done Done Sent false
25724 8248 Acked Write 4 Done Done Sent false
25725 8249 Acked Read 10 Done Done Sent false
25726 8250 Acked Read 10 Done Done Sent false
25727 8251 Acked Read 3 Done Done Sent false
25728 8252 Acked Write 8 Done Done Sent false
25729 8253 Acked Read 8 Done Done Sent false
25730 8254 Acked Read 8 Done Done Sent false
25731 8255 Acked Flush 0 Done Done Sent false
25732 8256 Acked Write 2 Done Done Sent false
25733 8257 Acked Write 10 Done Done Sent false
25734 8258 Acked Flush 0 Done Done Sent false
25735 8259 Acked Read 5 Done Done Sent false
25736 8260 Acked Read 3 Done Done Sent false
25737 8261 Acked Read 3 Done Done Sent false
25738 8262 Acked Write 9 Done Done Sent false
25739 8263 Acked Read 4 Done Done Sent false
25740 8264 Acked Read 4 Done Done Sent false
25741 8265 Acked Write 4 Done Done Sent false
25742 8266 Acked Read 10 Done Done Sent false
25743 8267 Acked Write 7 Done Done Sent false
25744 8268 Acked Write 6 Done Done Sent false
25745 8269 Acked Read 7 Done Done Sent false
25746 8270 Acked Read 1 Done Done Sent false
25747 8271 Acked Write 6 Done Done Sent false
25748 8272 Acked Write 8 Done Done Sent false
25749 8273 Acked Read 9 Done Done Sent false
25750 8274 Acked Write 7 Done Done Sent false
25751 8275 Acked Read 10 Done Done Sent false
25752 8276 Acked Read 3 Done Done Sent false
25753 8277 Acked Write 3 Done Done Sent false
25754 8278 Acked Write 5 Done Done Sent false
25755 8279 Acked Read 9 Done Done Sent false
25756 8280 Acked Write 9 Done Done Sent false
25757 8281 Acked Write 6 Done Done Sent false
25758 8282 Acked Read 6 Done Done Sent false
25759 8283 Acked Read 9 Done Done Sent false
25760 8284 Acked Write 5 Done Done Sent false
25761 8285 Acked Write 6 Done Done Sent false
25762 8286 Acked Write 7 Done Done Sent false
25763 8287 Acked Write 10 Done Done Sent false
25764 8288 Acked Write 7 Done Done Sent false
25765 8289 Acked Read 9 Done Done Sent false
25766 8290 Acked Read 10 Done Done Sent false
25767 8291 Acked Read 5 Done Done Sent false
25768 8292 Acked Write 8 Done Done Sent false
25769 8293 Acked Write 2 Done Done Sent false
25770 8294 Acked Write 4 Done Done Sent false
25771 8295 Acked Write 7 Done Done Sent false
25772 8296 Acked Write 8 Done Done Sent false
25773 8297 Acked Read 7 Done Done Sent false
25774 8298 Acked Read 1 Done Done Sent false
25775 8299 Acked Read 7 Done Done Sent false
25776 8300 Acked Flush 0 Done Done Sent false
25777 8301 Acked Read 7 Done Done Sent false
25778 8302 Acked Read 4 Done Done Sent false
25779 8303 Acked Read 7 Done Done Sent false
25780 8304 Acked Read 5 Done Done Sent false
25781 8305 Acked Write 5 Done Done Sent false
25782 8306 Acked Read 10 Done Done Sent false
25783 8307 Acked Write 9 Done Done Sent false
25784 8308 Acked Read 5 Done Done Sent false
25785 8309 Acked Flush 0 Done Done Sent false
25786 8310 Acked Read 4 Done Done Sent false
25787 8311 Acked Read 10 Done Done Sent false
25788 8312 Acked Read 1 Done Done Sent false
25789 8313 Acked Read 5 Done Done Sent false
25790 8314 Acked Write 3 Done Done Sent false
25791 8315 Acked Flush 0 Done Done Sent false
25792 8316 Acked Read 6 Done Done Sent false
25793 8317 Acked Write 3 Done Done Sent false
25794 8318 Acked Flush 0 Done Done Sent false
25795 8319 Acked Write 7 Done Done Sent false
25796 8320 Acked Read 1 Done Done Sent false
25797 8321 Acked Write 5 Done Done Sent false
25798 8322 Acked Write 1 Done Done Sent false
25799 8323 Acked Write 10 Done Done Sent false
25800 8324 Acked Read 1 Done Done Sent false
25801 8325 Acked Read 6 Done Done Sent false
25802 8326 Acked Read 8 Done Done Sent false
25803 8327 Acked Read 2 Done Done Sent false
25804 8328 Acked Read 2 Done Done Sent false
25805 8329 Acked Write 7 Done Done Sent false
25806 8330 Acked Write 2 Done Done Sent false
25807 8331 Acked Write 10 Done Done Sent false
25808 8332 Acked Read 7 Done Done Sent false
25809 8333 Acked Write 7 Done Done Sent false
25810 8334 Acked Write 8 Done Done Sent false
25811 8335 Acked Read 9 Done Done Sent false
25812 8336 Acked Read 6 Done Done Sent false
25813 8337 Acked Read 9 Done Done Sent false
25814 8338 Acked Write 10 Done Done Sent false
25815 8339 Acked Read 4 Done Done Sent false
25816 8340 Acked Write 8 Done Done Sent false
25817 8341 Acked Read 5 Done Done Sent false
25818 8342 Acked Write 3 Done Done Sent false
25819 8343 Acked Write 6 Done Done Sent false
25820 8344 Acked Read 9 Done Done Sent false
25821 8345 Acked Flush 0 Done Done Sent false
25822 8346 Acked Write 3 Done Done Sent false
25823 8347 Acked Flush 0 Done Done Sent false
25824 8348 Acked Read 3 Done Done Sent false
25825 8349 Acked Write 5 Done Done Sent false
25826 8350 Acked Write 1 Done Done Sent false
25827 8351 Acked Read 6 Done Done Sent false
25828 8352 Acked Write 7 Done Done Sent false
25829 8353 Acked Write 7 Done Done Sent false
25830 8354 Acked Write 4 Done Done Sent false
25831 8355 Acked Write 4 Done Done Sent false
25832 8356 Acked Flush 0 Done Done Sent false
25833 8357 Acked Flush 0 Done Done Sent false
25834 8358 Acked Write 2 Done Done Sent false
25835 8359 Acked Flush 0 Done Done Sent false
25836 8360 Acked Read 8 Done Done Sent false
25837 8361 Acked Read 6 Done Done Sent false
25838 8362 Acked Read 3 Done Done Sent false
25839 8363 Acked Read 4 Done Done Sent false
25840 8364 Acked Write 3 Done Done Sent false
25841 8365 Acked Read 10 Done Done Sent false
25842 8366 Acked Read 7 Done Done Sent false
25843 8367 Acked Read 8 Done Done Sent false
25844 8368 Acked Read 6 Done Done Sent false
25845 8369 Acked Flush 0 Done Done Sent false
25846 8370 Acked Read 6 Done Done Sent false
25847 8371 Acked Flush 0 Done Done Sent false
25848 8372 Acked Write 8 Done Done Sent false
25849 8373 Acked Write 8 Done Done Sent false
25850 8374 Acked Read 4 Done Done Sent false
25851 8375 Acked Read 3 Done Done Sent false
25852 8376 Acked Write 3 Done Done Sent false
25853 8377 Acked Write 6 Done Done Sent false
25854 8378 Acked Write 9 Done Done Sent false
25855 8379 Acked Flush 0 Done Done Sent false
25856 8380 Acked Flush 0 Done Done Sent false
25857 8381 Acked Read 8 Done Done Sent false
25858 8382 Acked Write 5 Done Done Sent false
25859 8383 Acked Write 6 Done Done Sent false
25860 8384 Acked Read 8 Done Done Sent false
25861 8385 Acked Write 5 Done Done Sent false
25862 8386 Acked Read 5 Done Done Sent false
25863 8387 Acked Write 7 Done Done Sent false
25864 8388 Acked Read 10 Done Done Sent false
25865 8389 Acked Write 3 Done Done Sent false
25866 8390 Acked Write 7 Done Done Sent false
25867 8391 Acked Write 9 Done Done Sent false
25868 8392 Acked Read 8 Done Done Sent false
25869 8393 Acked Write 4 Done Done Sent false
25870 8394 Acked Flush 0 Done Done Sent false
25871 8395 Acked Write 2 Done Done Sent false
25872 8396 Acked Read 3 Done Done Sent false
25873 8397 Acked Flush 0 Done Done Sent false
25874 8398 Acked Flush 0 Done Done Sent false
25875 8399 Acked Flush 0 Done Done Sent false
25876 8400 Acked Write 6 Done Done Sent false
25877 8401 Acked Write 2 Done Done Sent false
25878 8402 Acked Write 7 Done Done Sent false
25879 8403 Acked Write 3 Done Done Sent false
25880 8404 Acked Write 8 Done Done Sent false
25881 8405 Acked Read 2 Done Done Sent false
25882 8406 Acked Write 6 Done Done Sent false
25883 8407 Acked Read 5 Done Done Sent false
25884 8408 Acked Flush 0 Done Done Sent false
25885 8409 Acked Write 7 Done Done Sent false
25886 8410 Acked Flush 0 Done Done Sent false
25887 8411 Acked Read 2 Done Done Sent false
25888 8412 Acked Read 8 Done Done Sent false
25889 8413 Acked Write 6 Done Done Sent false
25890 8414 Acked Flush 0 Done Done Sent false
25891 8415 Acked Read 7 Done Done Sent false
25892 8416 Acked Write 1 Done Done Sent false
25893 8417 Acked Flush 0 Done Done Sent false
25894 8418 Acked Flush 0 Done Done Sent false
25895 8419 Acked Read 1 Done Done Sent false
25896 8420 Acked Write 5 Done Done Sent false
25897 8421 Acked Write 9 Done Done Sent false
25898 8422 Acked Read 7 Done Done Sent false
25899 8423 Acked Read 9 Done Done Sent false
25900 8424 Acked Read 5 Done Done Sent false
25901 8425 Acked Write 1 Done Done Sent false
25902 8426 Acked Write 6 Done Done Sent false
25903 8427 Acked Read 7 Done Done Sent false
25904 8428 Acked Write 9 Done Done Sent false
25905 8429 Acked Write 1 Done Done Sent false
25906 8430 Acked Write 1 Done Done Sent false
25907 8431 Acked Read 6 Done Done Sent false
25908 8432 Acked Write 10 Done Done Sent false
25909 8433 Acked Write 8 Done Done Sent false
25910 8434 Acked Write 5 Done Done Sent false
25911 8435 Acked Flush 0 Done Done Sent false
25912 8436 Acked Write 7 Done Done Sent false
25913 8437 Acked Write 1 Done Done Sent false
25914 8438 Acked Flush 0 Done Done Sent false
25915 8439 Acked Read 7 Done Done Sent false
25916 8440 Acked Flush 0 Done Done Sent false
25917 8441 Acked Write 3 Done Done Sent false
25918 8442 Acked Read 6 Done Done Sent false
25919 8443 Acked Flush 0 Done Done Sent false
25920 8444 Acked Read 3 Done Done Sent false
25921 8445 Acked Flush 0 Done Done Sent false
25922 8446 Acked Write 7 Done Done Sent false
25923 8447 Acked Write 8 Done Done Sent false
25924 8448 Acked Read 7 Done Done Sent false
25925 8449 Acked Read 1 Done Done Sent false
25926 8450 Acked Read 9 Done Done Sent false
25927 8451 Acked Read 7 Done Done Sent false
25928 8452 Acked Read 9 Done Done Sent false
25929 8453 Acked Read 10 Done Done Sent false
25930 8454 Acked Write 10 Done Done Sent false
25931 8455 Acked Read 3 Done Done Sent false
25932 8456 Acked Flush 0 Done Done Sent false
25933 8457 Acked Flush 0 Done Done Sent false
25934 8458 Acked Write 1 Done Done Sent false
25935 8459 Acked Write 3 Done Done Sent false
25936 8460 Acked Read 4 Done Done Sent false
25937 8461 Acked Read 9 Done Done Sent false
25938 8462 Acked Write 6 Done Done Sent false
25939 8463 Acked Write 8 Done Done Sent false
25940 8464 Acked Read 2 Done Done Sent false
25941 8465 Acked Flush 0 Done Done Sent false
25942 8466 Acked Write 4 Done Done Sent false
25943 8467 Acked Flush 0 Done Done Sent false
25944 8468 Acked Read 3 Done Done Sent false
25945 8469 Acked Read 2 Done Done Sent false
25946 8470 Acked Write 9 Done Done Sent false
25947 8471 Acked Write 10 Done Done Sent false
25948 8472 Acked Write 3 Done Done Sent false
25949 8473 Acked Read 2 Done Done Sent false
25950 8474 Acked Write 4 Done Done Sent false
25951 8475 Acked Write 10 Done Done Sent false
25952 8476 Acked Write 4 Done Done Sent false
25953 8477 Acked Flush 0 Done Done Sent false
25954 8478 Acked Read 5 Done Done Sent false
25955 8479 Acked Flush 0 Done Done Sent false
25956 8480 Acked Write 6 Done Done Sent false
25957 8481 Acked Write 1 Done Done Sent false
25958 8482 Acked Write 2 Done Done Sent false
25959 8483 Acked Write 6 Done Done Sent false
25960 8484 Acked Write 8 Done Done Sent false
25961 8485 Acked Read 5 Done Done Sent false
25962 8486 Acked Read 1 Done Done Sent false
25963 8487 Acked Read 5 Done Done Sent false
25964 8488 Acked Read 8 Done Done Sent false
25965 8489 Acked Read 8 Done Done Sent false
25966 8490 Acked Read 3 Done Done Sent false
25967 8491 Acked Read 9 Done Done Sent false
25968 8492 Acked Read 3 Done Done Sent false
25969 8493 Acked Flush 0 Done Done Sent false
25970 8494 Acked Read 6 Done Done Sent false
25971 8495 Acked Read 3 Done Done Sent false
25972 8496 Acked Flush 0 Done Done Sent false
25973 8497 Acked Flush 0 Done Done Sent false
25974 8498 Acked Read 9 Done Done Sent false
25975 8499 Acked Write 1 Done Done Sent false
25976 8500 Acked Read 1 Done Done Sent false
25977 8501 Acked Read 10 Done Done Sent false
25978 8502 Acked Write 3 Done Done Sent false
25979 8503 Acked Read 5 Done Done Sent false
25980 8504 Acked Flush 0 Done Done Sent false
25981 8505 Acked Flush 0 Done Done Sent false
25982 8506 Acked Read 9 Done Done Sent false
25983 8507 Acked Write 6 Done Done Sent false
25984 8508 Acked Flush 0 Done Done Sent false
25985 8509 Acked Read 9 Done Done Sent false
25986 8510 Acked Read 4 Done Done Sent false
25987 8511 Acked Read 2 Done Done Sent false
25988 8512 Acked Write 2 Done Done Sent false
25989 8513 Acked Read 2 Done Done Sent false
25990 8514 Acked Write 3 Done Done Sent false
25991 8515 Acked Write 10 Done Done Sent false
25992 8516 Acked Read 8 Done Done Sent false
25993 8517 Acked Read 5 Done Done Sent false
25994 8518 Acked Write 9 Done Done Sent false
25995 8519 Acked Write 9 Done Done Sent false
25996 8520 Acked Write 2 Done Done Sent false
25997 8521 Acked Write 2 Done Done Sent false
25998 8522 Acked Read 7 Done Done Sent false
25999 8523 Acked Flush 0 Done Done Sent false
26000 8524 Acked Write 4 Done Done Sent false
26001 8525 Acked Read 6 Done Done Sent false
26002 8526 Acked Read 6 Done Done Sent false
26003 8527 Acked Read 7 Done Done Sent false
26004 8528 Acked Write 4 Done Done Sent false
26005 8529 Acked Read 4 Done Done Sent false
26006 8530 Acked Write 5 Done Done Sent false
26007 8531 Acked Read 2 Done Done Sent false
26008 8532 Acked Read 7 Done Done Sent false
26009 8533 Acked Read 3 Done Done Sent false
26010 8534 Acked Write 6 Done Done Sent false
26011 8535 Acked Flush 0 Done Done Sent false
26012 8536 Acked Flush 0 Done Done Sent false
26013 8537 Acked Read 5 Done Done Sent false
26014 8538 Acked Flush 0 Done Done Sent false
26015 8539 Acked Write 8 Done Done Sent false
26016 8540 Acked Write 4 Done Done Sent false
26017 8541 Acked Write 5 Done Done Sent false
26018 8542 Acked Write 3 Done Done Sent false
26019 8543 Acked Flush 0 Done Done Sent false
26020 8544 Acked Read 6 Done Done Sent false
26021 8545 Acked Write 8 Done Done Sent false
26022 STATES DS:0 DS:1 DS:2 TOTAL
26023 Sent 0 0 301 301
26024 Done 301 301 0 602
26025 Skipped 0 0 0 0
26026 Error 0 0 0 0
26027 Last Flush: 8543 8543 8244
26028 Downstairs last five completed: 8244 8243 8242 8241 8240
26029 Upstairs last five completed: 8545 8544 8543 8542 8541
26030 CLIENT: Up:0 ds:301 act:2
26031 [8830] got start action
26032 [8830] I am going to restart
26033 Starting downstairs at port 8830
26034 Make output file at "/tmp/dsc/downstairs-8830.txt"
26035 [8830][2] reports Starting
26036 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1227)
26037 [8830][2] reports Running
26038 ----------------------------------------------------------------
26039 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26040 States: Active Active Offline
26041 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26042 8245 Acked Read 9 Done Done Sent false
26043 8246 Acked Write 10 Done Done Sent false
26044 8247 Acked Write 8 Done Done Sent false
26045 8248 Acked Write 4 Done Done Sent false
26046 8249 Acked Read 10 Done Done Sent false
26047 8250 Acked Read 10 Done Done Sent false
26048 8251 Acked Read 3 Done Done Sent false
26049 8252 Acked Write 8 Done Done Sent false
26050 8253 Acked Read 8 Done Done Sent false
26051 8254 Acked Read 8 Done Done Sent false
26052 8255 Acked Flush 0 Done Done Sent false
26053 8256 Acked Write 2 Done Done Sent false
26054 8257 Acked Write 10 Done Done Sent false
26055 8258 Acked Flush 0 Done Done Sent false
26056 8259 Acked Read 5 Done Done Sent false
26057 8260 Acked Read 3 Done Done Sent false
26058 8261 Acked Read 3 Done Done Sent false
26059 8262 Acked Write 9 Done Done Sent false
26060 8263 Acked Read 4 Done Done Sent false
26061 8264 Acked Read 4 Done Done Sent false
26062 8265 Acked Write 4 Done Done Sent false
26063 8266 Acked Read 10 Done Done Sent false
26064 8267 Acked Write 7 Done Done Sent false
26065 8268 Acked Write 6 Done Done Sent false
26066 8269 Acked Read 7 Done Done Sent false
26067 8270 Acked Read 1 Done Done Sent false
26068 8271 Acked Write 6 Done Done Sent false
26069 8272 Acked Write 8 Done Done Sent false
26070 8273 Acked Read 9 Done Done Sent false
26071 8274 Acked Write 7 Done Done Sent false
26072 8275 Acked Read 10 Done Done Sent false
26073 8276 Acked Read 3 Done Done Sent false
26074 8277 Acked Write 3 Done Done Sent false
26075 8278 Acked Write 5 Done Done Sent false
26076 8279 Acked Read 9 Done Done Sent false
26077 8280 Acked Write 9 Done Done Sent false
26078 8281 Acked Write 6 Done Done Sent false
26079 8282 Acked Read 6 Done Done Sent false
26080 8283 Acked Read 9 Done Done Sent false
26081 8284 Acked Write 5 Done Done Sent false
26082 8285 Acked Write 6 Done Done Sent false
26083 8286 Acked Write 7 Done Done Sent false
26084 8287 Acked Write 10 Done Done Sent false
26085 8288 Acked Write 7 Done Done Sent false
26086 8289 Acked Read 9 Done Done Sent false
26087 8290 Acked Read 10 Done Done Sent false
26088 8291 Acked Read 5 Done Done Sent false
26089 8292 Acked Write 8 Done Done Sent false
26090 8293 Acked Write 2 Done Done Sent false
26091 8294 Acked Write 4 Done Done Sent false
26092 8295 Acked Write 7 Done Done Sent false
26093 8296 Acked Write 8 Done Done Sent false
26094 8297 Acked Read 7 Done Done Sent false
26095 8298 Acked Read 1 Done Done Sent false
26096 8299 Acked Read 7 Done Done Sent false
26097 8300 Acked Flush 0 Done Done Sent false
26098 8301 Acked Read 7 Done Done Sent false
26099 8302 Acked Read 4 Done Done Sent false
26100 8303 Acked Read 7 Done Done Sent false
26101 8304 Acked Read 5 Done Done Sent false
26102 8305 Acked Write 5 Done Done Sent false
26103 8306 Acked Read 10 Done Done Sent false
26104 8307 Acked Write 9 Done Done Sent false
26105 8308 Acked Read 5 Done Done Sent false
26106 8309 Acked Flush 0 Done Done Sent false
26107 8310 Acked Read 4 Done Done Sent false
26108 8311 Acked Read 10 Done Done Sent false
26109 8312 Acked Read 1 Done Done Sent false
26110 8313 Acked Read 5 Done Done Sent false
26111 8314 Acked Write 3 Done Done Sent false
26112 8315 Acked Flush 0 Done Done Sent false
26113 8316 Acked Read 6 Done Done Sent false
26114 8317 Acked Write 3 Done Done Sent false
26115 8318 Acked Flush 0 Done Done Sent false
26116 8319 Acked Write 7 Done Done Sent false
26117 8320 Acked Read 1 Done Done Sent false
26118 8321 Acked Write 5 Done Done Sent false
26119 8322 Acked Write 1 Done Done Sent false
26120 8323 Acked Write 10 Done Done Sent false
26121 8324 Acked Read 1 Done Done Sent false
26122 8325 Acked Read 6 Done Done Sent false
26123 8326 Acked Read 8 Done Done Sent false
26124 8327 Acked Read 2 Done Done Sent false
26125 8328 Acked Read 2 Done Done Sent false
26126 8329 Acked Write 7 Done Done Sent false
26127 8330 Acked Write 2 Done Done Sent false
26128 8331 Acked Write 10 Done Done Sent false
26129 8332 Acked Read 7 Done Done Sent false
26130 8333 Acked Write 7 Done Done Sent false
26131 8334 Acked Write 8 Done Done Sent false
26132 8335 Acked Read 9 Done Done Sent false
26133 8336 Acked Read 6 Done Done Sent false
26134 8337 Acked Read 9 Done Done Sent false
26135 8338 Acked Write 10 Done Done Sent false
26136 8339 Acked Read 4 Done Done Sent false
26137 8340 Acked Write 8 Done Done Sent false
26138 8341 Acked Read 5 Done Done Sent false
26139 8342 Acked Write 3 Done Done Sent false
26140 8343 Acked Write 6 Done Done Sent false
26141 8344 Acked Read 9 Done Done Sent false
26142 8345 Acked Flush 0 Done Done Sent false
26143 8346 Acked Write 3 Done Done Sent false
26144 8347 Acked Flush 0 Done Done Sent false
26145 8348 Acked Read 3 Done Done Sent false
26146 8349 Acked Write 5 Done Done Sent false
26147 8350 Acked Write 1 Done Done Sent false
26148 8351 Acked Read 6 Done Done Sent false
26149 8352 Acked Write 7 Done Done Sent false
26150 8353 Acked Write 7 Done Done Sent false
26151 8354 Acked Write 4 Done Done Sent false
26152 8355 Acked Write 4 Done Done Sent false
26153 8356 Acked Flush 0 Done Done Sent false
26154 8357 Acked Flush 0 Done Done Sent false
26155 8358 Acked Write 2 Done Done Sent false
26156 8359 Acked Flush 0 Done Done Sent false
26157 8360 Acked Read 8 Done Done Sent false
26158 8361 Acked Read 6 Done Done Sent false
26159 8362 Acked Read 3 Done Done Sent false
26160 8363 Acked Read 4 Done Done Sent false
26161 8364 Acked Write 3 Done Done Sent false
26162 8365 Acked Read 10 Done Done Sent false
26163 8366 Acked Read 7 Done Done Sent false
26164 8367 Acked Read 8 Done Done Sent false
26165 8368 Acked Read 6 Done Done Sent false
26166 8369 Acked Flush 0 Done Done Sent false
26167 8370 Acked Read 6 Done Done Sent false
26168 8371 Acked Flush 0 Done Done Sent false
26169 8372 Acked Write 8 Done Done Sent false
26170 8373 Acked Write 8 Done Done Sent false
26171 8374 Acked Read 4 Done Done Sent false
26172 8375 Acked Read 3 Done Done Sent false
26173 8376 Acked Write 3 Done Done Sent false
26174 8377 Acked Write 6 Done Done Sent false
26175 8378 Acked Write 9 Done Done Sent false
26176 8379 Acked Flush 0 Done Done Sent false
26177 8380 Acked Flush 0 Done Done Sent false
26178 8381 Acked Read 8 Done Done Sent false
26179 8382 Acked Write 5 Done Done Sent false
26180 8383 Acked Write 6 Done Done Sent false
26181 8384 Acked Read 8 Done Done Sent false
26182 8385 Acked Write 5 Done Done Sent false
26183 8386 Acked Read 5 Done Done Sent false
26184 8387 Acked Write 7 Done Done Sent false
26185 8388 Acked Read 10 Done Done Sent false
26186 8389 Acked Write 3 Done Done Sent false
26187 8390 Acked Write 7 Done Done Sent false
26188 8391 Acked Write 9 Done Done Sent false
26189 8392 Acked Read 8 Done Done Sent false
26190 8393 Acked Write 4 Done Done Sent false
26191 8394 Acked Flush 0 Done Done Sent false
26192 8395 Acked Write 2 Done Done Sent false
26193 8396 Acked Read 3 Done Done Sent false
26194 8397 Acked Flush 0 Done Done Sent false
26195 8398 Acked Flush 0 Done Done Sent false
26196 8399 Acked Flush 0 Done Done Sent false
26197 8400 Acked Write 6 Done Done Sent false
26198 8401 Acked Write 2 Done Done Sent false
26199 8402 Acked Write 7 Done Done Sent false
26200 8403 Acked Write 3 Done Done Sent false
26201 8404 Acked Write 8 Done Done Sent false
26202 8405 Acked Read 2 Done Done Sent false
26203 8406 Acked Write 6 Done Done Sent false
26204 8407 Acked Read 5 Done Done Sent false
26205 8408 Acked Flush 0 Done Done Sent false
26206 8409 Acked Write 7 Done Done Sent false
26207 8410 Acked Flush 0 Done Done Sent false
26208 8411 Acked Read 2 Done Done Sent false
26209 8412 Acked Read 8 Done Done Sent false
26210 8413 Acked Write 6 Done Done Sent false
26211 8414 Acked Flush 0 Done Done Sent false
26212 8415 Acked Read 7 Done Done Sent false
26213 8416 Acked Write 1 Done Done Sent false
26214 8417 Acked Flush 0 Done Done Sent false
26215 8418 Acked Flush 0 Done Done Sent false
26216 8419 Acked Read 1 Done Done Sent false
26217 8420 Acked Write 5 Done Done Sent false
26218 8421 Acked Write 9 Done Done Sent false
26219 8422 Acked Read 7 Done Done Sent false
26220 8423 Acked Read 9 Done Done Sent false
26221 8424 Acked Read 5 Done Done Sent false
26222 8425 Acked Write 1 Done Done Sent false
26223 8426 Acked Write 6 Done Done Sent false
26224 8427 Acked Read 7 Done Done Sent false
26225 8428 Acked Write 9 Done Done Sent false
26226 8429 Acked Write 1 Done Done Sent false
26227 8430 Acked Write 1 Done Done Sent false
26228 8431 Acked Read 6 Done Done Sent false
26229 8432 Acked Write 10 Done Done Sent false
26230 8433 Acked Write 8 Done Done Sent false
26231 8434 Acked Write 5 Done Done Sent false
26232 8435 Acked Flush 0 Done Done Sent false
26233 8436 Acked Write 7 Done Done Sent false
26234 8437 Acked Write 1 Done Done Sent false
26235 8438 Acked Flush 0 Done Done Sent false
26236 8439 Acked Read 7 Done Done Sent false
26237 8440 Acked Flush 0 Done Done Sent false
26238 8441 Acked Write 3 Done Done Sent false
26239 8442 Acked Read 6 Done Done Sent false
26240 8443 Acked Flush 0 Done Done Sent false
26241 8444 Acked Read 3 Done Done Sent false
26242 8445 Acked Flush 0 Done Done Sent false
26243 8446 Acked Write 7 Done Done Sent false
26244 8447 Acked Write 8 Done Done Sent false
26245 8448 Acked Read 7 Done Done Sent false
26246 8449 Acked Read 1 Done Done Sent false
26247 8450 Acked Read 9 Done Done Sent false
26248 8451 Acked Read 7 Done Done Sent false
26249 8452 Acked Read 9 Done Done Sent false
26250 8453 Acked Read 10 Done Done Sent false
26251 8454 Acked Write 10 Done Done Sent false
26252 8455 Acked Read 3 Done Done Sent false
26253 8456 Acked Flush 0 Done Done Sent false
26254 8457 Acked Flush 0 Done Done Sent false
26255 8458 Acked Write 1 Done Done Sent false
26256 8459 Acked Write 3 Done Done Sent false
26257 8460 Acked Read 4 Done Done Sent false
26258 8461 Acked Read 9 Done Done Sent false
26259 8462 Acked Write 6 Done Done Sent false
26260 8463 Acked Write 8 Done Done Sent false
26261 8464 Acked Read 2 Done Done Sent false
26262 8465 Acked Flush 0 Done Done Sent false
26263 8466 Acked Write 4 Done Done Sent false
26264 8467 Acked Flush 0 Done Done Sent false
26265 8468 Acked Read 3 Done Done Sent false
26266 8469 Acked Read 2 Done Done Sent false
26267 8470 Acked Write 9 Done Done Sent false
26268 8471 Acked Write 10 Done Done Sent false
26269 8472 Acked Write 3 Done Done Sent false
26270 8473 Acked Read 2 Done Done Sent false
26271 8474 Acked Write 4 Done Done Sent false
26272 8475 Acked Write 10 Done Done Sent false
26273 8476 Acked Write 4 Done Done Sent false
26274 8477 Acked Flush 0 Done Done Sent false
26275 8478 Acked Read 5 Done Done Sent false
26276 8479 Acked Flush 0 Done Done Sent false
26277 8480 Acked Write 6 Done Done Sent false
26278 8481 Acked Write 1 Done Done Sent false
26279 8482 Acked Write 2 Done Done Sent false
26280 8483 Acked Write 6 Done Done Sent false
26281 8484 Acked Write 8 Done Done Sent false
26282 8485 Acked Read 5 Done Done Sent false
26283 8486 Acked Read 1 Done Done Sent false
26284 8487 Acked Read 5 Done Done Sent false
26285 8488 Acked Read 8 Done Done Sent false
26286 8489 Acked Read 8 Done Done Sent false
26287 8490 Acked Read 3 Done Done Sent false
26288 8491 Acked Read 9 Done Done Sent false
26289 8492 Acked Read 3 Done Done Sent false
26290 8493 Acked Flush 0 Done Done Sent false
26291 8494 Acked Read 6 Done Done Sent false
26292 8495 Acked Read 3 Done Done Sent false
26293 8496 Acked Flush 0 Done Done Sent false
26294 8497 Acked Flush 0 Done Done Sent false
26295 8498 Acked Read 9 Done Done Sent false
26296 8499 Acked Write 1 Done Done Sent false
26297 8500 Acked Read 1 Done Done Sent false
26298 8501 Acked Read 10 Done Done Sent false
26299 8502 Acked Write 3 Done Done Sent false
26300 8503 Acked Read 5 Done Done Sent false
26301 8504 Acked Flush 0 Done Done Sent false
26302 8505 Acked Flush 0 Done Done Sent false
26303 8506 Acked Read 9 Done Done Sent false
26304 8507 Acked Write 6 Done Done Sent false
26305 8508 Acked Flush 0 Done Done Sent false
26306 8509 Acked Read 9 Done Done Sent false
26307 8510 Acked Read 4 Done Done Sent false
26308 8511 Acked Read 2 Done Done Sent false
26309 8512 Acked Write 2 Done Done Sent false
26310 8513 Acked Read 2 Done Done Sent false
26311 8514 Acked Write 3 Done Done Sent false
26312 8515 Acked Write 10 Done Done Sent false
26313 8516 Acked Read 8 Done Done Sent false
26314 8517 Acked Read 5 Done Done Sent false
26315 8518 Acked Write 9 Done Done Sent false
26316 8519 Acked Write 9 Done Done Sent false
26317 8520 Acked Write 2 Done Done Sent false
26318 8521 Acked Write 2 Done Done Sent false
26319 8522 Acked Read 7 Done Done Sent false
26320 8523 Acked Flush 0 Done Done Sent false
26321 8524 Acked Write 4 Done Done Sent false
26322 8525 Acked Read 6 Done Done Sent false
26323 8526 Acked Read 6 Done Done Sent false
26324 8527 Acked Read 7 Done Done Sent false
26325 8528 Acked Write 4 Done Done Sent false
26326 8529 Acked Read 4 Done Done Sent false
26327 8530 Acked Write 5 Done Done Sent false
26328 8531 Acked Read 2 Done Done Sent false
26329 8532 Acked Read 7 Done Done Sent false
26330 8533 Acked Read 3 Done Done Sent false
26331 8534 Acked Write 6 Done Done Sent false
26332 8535 Acked Flush 0 Done Done Sent false
26333 8536 Acked Flush 0 Done Done Sent false
26334 8537 Acked Read 5 Done Done Sent false
26335 8538 Acked Flush 0 Done Done Sent false
26336 8539 Acked Write 8 Done Done Sent false
26337 8540 Acked Write 4 Done Done Sent false
26338 8541 Acked Write 5 Done Done Sent false
26339 8542 Acked Write 3 Done Done Sent false
26340 8543 Acked Flush 0 Done Done Sent false
26341 8544 Acked Read 6 Done Done Sent false
26342 8545 Acked Write 8 Done Done Sent false
26343 8546 Acked Flush 0 Done Done Sent false
26344 STATES DS:0 DS:1 DS:2 TOTAL
26345 Sent 0 0 302 302
26346 Done 302 302 0 604
26347 Skipped 0 0 0 0
26348 Error 0 0 0 0
26349 Last Flush: 8546 8546 8244
26350 Downstairs last five completed: 8244 8243 8242 8241 8240
26351 Upstairs last five completed: 8546 8545 8544 8543 8542
26352 CLIENT: Up:0 ds:302 act:2
263532025-01-14T16:16:07.799ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
263542025-01-14T16:16:07.799ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36486) = io task client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
263552025-01-14T16:16:07.800ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
263562025-01-14T16:16:07.800ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
263572025-01-14T16:16:07.800ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
263582025-01-14T16:16:07.800ZINFOcrucible: send last flush ID to this DS: 8244 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
263592025-01-14T16:16:07.801ZINFOcrucible: Replied this last flush ID: 8244 = downstairs client = 2 session_id = c2423ef0-728e-4895-997a-d9044c50948b
263602025-01-14T16:16:07.801ZINFOcrucible: [2] client re-new 302 jobs since flush 8244 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
263612025-01-14T16:16:07.801ZINFOcrucible: [2] Replayed 302 jobs since flush: 8244 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
26362 ----------------------------------------------------------------
26363 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
26364 Downstairs last five completed: 8546 8545 8544 8543 8542
26365 Upstairs last five completed: 8546 8545 8544 8543 8542
26366 CLIENT: Up:0 ds:0 act:3
26367 Replay: All jobs finished
26368 Main task has work to do, go find it
26369 got dsc Stop(0)
26370 Jan 14 16:16:10.033stop 0
26371 INFO request completed, latency_us: 320, response_code: 204, uri: /stop/cid/0, method: POST, req_id: b3c7ecc3-ecc7-4d0e-b634-52a5b4887d47, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
26372 [8810] Got stop action so:false kr:false
26373 Jan 14 16:16:10.034 INFO request completed, latency_us: 127, response_code: 200, uri: /state/cid/0, method: GET, req_id: b81cb1eb-1098-4475-aac9-4eb50ac47499, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
263742025-01-14T16:16:10.034ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
263752025-01-14T16:16:10.034ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
263762025-01-14T16:16:10.034ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
263772025-01-14T16:16:10.034ZINFOcrucible: client task is exiting = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
26378 [8810] Exited with: ExitStatus(unix_wait_status(9))
26379 [8810][0] reports Exit
26380 Jan 14 16:16:14.037 INFO request completed, latency_us: 196, response_code: 200, uri: /state/cid/0, method: GET, req_id: ace340fa-dd4d-4209-bd3a-363986d39233, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
26381 001/300 Read block 2127 len 40960
26382 002/300 Read block 4959 len 24576
26383 003/300 Write block 2593 len 4096 data: 5
26384 004/300 Write block 1568 len 8192 data: 8 7
26385 005/300 Read block 1837 len 8192
26386 006/300 Write block 1497 len 36864 data: 7 5 4 5 4 4 5 5 6
26387 007/300 Read block 3557 len 12288
26388 008/300 Read block 1175 len 12288
26389 009/300 Read block 3832 len 20480
26390 010/300 Read block 829 len 28672
26391 011/300 Read block 2927 len 32768
26392 012/300 Write block 4651 len 8192 data: 2 2
26393 013/300 Write block 3787 len 36864 data: 5 5 5 5 4 4 4 6 6
26394 014/300 Read block 4767 len 36864
26395 015/300 Write block 4763 len 4096 data: 6
26396 016/300 Write block 1253 len 4096 data: 3
26397 017/300 Write block 2746 len 4096 data: 5
26398 018/300 Write block 357 len 40960 data: 5 5 5 5 5 4 5 3 3 3
26399 019/300 Read block 4173 len 28672
26400 020/300 Write block 2839 len 20480 data: 6 5 5 6 3
26401 021/300 Write block 2212 len 4096 data: 7
26402 022/300 Write block 594 len 12288 data: 5 6 7
26403 023/300 Read block 710 len 28672
26404 024/300 Read block 1106 len 20480
26405 025/300 Read block 1087 len 12288
26406 026/300 Read block 2159 len 4096
26407 027/300 Write block 776 len 32768 data: 7 5 5 4 4 4 4 4
26408 028/300 Read block 3226 len 40960
26409 029/300 Write block 4653 len 32768 data: 3 3 3 3 4 4 6 6
26410 030/300 Read block 751 len 4096
26411 031/300 Read block 2967 len 36864
26412 032/300 Write block 2109 len 4096 data: 5
26413 033/300 Write block 743 len 16384 data: 5 5 5 5
26414 034/300 Flush
26415 035/300 Read block 2605 len 36864
26416 036/300 Write block 1068 len 12288 data: 6 6 6
26417 037/300 Read block 1687 len 4096
26418 038/300 Write block 4650 len 28672 data: 2 3 3 4 4 4 4
26419 039/300 Read block 1681 len 36864
26420 040/300 Read block 4487 len 28672
26421 041/300 Read block 2256 len 40960
26422 042/300 Read block 4201 len 32768
26423 043/300 Read block 3729 len 24576
26424 044/300 Write block 622 len 8192 data: 7 5
26425 045/300 Write block 1490 len 16384 data: 8 6 6 6
26426 046/300 Write block 613 len 8192 data: 5 5
26427 047/300 Write block 2833 len 28672 data: 7 7 6 8 8 7 7
26428 048/300 Flush
26429 049/300 Write block 2926 len 40960 data: 7 8 7 7 4 4 5 4 4 3
26430 050/300 Write block 4477 len 24576 data: 5 4 4 4 4 5
26431 051/300 Read block 776 len 28672
26432 052/300 Read block 2673 len 20480
26433 053/300 Read block 544 len 40960
26434 054/300 Flush
26435 055/300 Read block 979 len 32768
26436 056/300 Read block 1248 len 36864
26437 057/300 Write block 2346 len 16384 data: 4 3 4 3
26438 058/300 Read block 2546 len 36864
26439 059/300 Write block 575 len 8192 data: 7 7
26440 060/300 Read block 1426 len 24576
26441 061/300 Write block 2469 len 8192 data: 4 5
26442 062/300 Write block 4980 len 16384 data: 8 5 6 6
26443 063/300 Read block 1512 len 24576
26444 064/300 Read block 1792 len 36864
26445 065/300 Flush
26446 066/300 Write block 2793 len 4096 data: 4
26447 067/300 Read block 3731 len 28672
26448 068/300 Flush
26449 069/300 Write block 4847 len 36864 data: 10 9 9 9 6 4 5 4 4
26450 070/300 Write block 4557 len 28672 data: 6 6 6 7 7 7 5
26451 071/300 Read block 1382 len 32768
26452 072/300 Read block 3386 len 16384
26453 073/300 Write block 1628 len 32768 data: 4 3 3 3 5 4 4 3
26454 074/300 Write block 2945 len 8192 data: 5 5
26455 075/300 Write block 4604 len 32768 data: 3 3 3 2 2 3 3 4
26456 076/300 Read block 2547 len 32768
26457 077/300 Read block 3211 len 28672
26458 078/300 Write block 1622 len 28672 data: 4 3 4 6 6 4 5
26459 079/300 Write block 3871 len 16384 data: 5 4 3 4
26460 080/300 Write block 4611 len 12288 data: 5 4 3
26461 081/300 Write block 4090 len 16384 data: 8 10 9 8
26462 082/300 Write block 253 len 20480 data: 5 5 5 7 6
26463 083/300 Flush
26464 084/300 Write block 1097 len 8192 data: 7 7
26465 085/300 Flush
26466 086/300 Read block 2884 len 12288
26467 087/300 Read block 1668 len 8192
26468 088/300 Flush
26469 089/300 Write block 1747 len 12288 data: 7 9 7
26470 090/300 Write block 492 len 24576 data: 3 3 4 5 4 5
26471 091/300 Read block 2529 len 20480
26472 092/300 Read block 4907 len 16384
26473 093/300 Read block 3523 len 12288
26474 094/300 Write block 4089 len 36864 data: 9 9 11 10 9 6 5 3 5
26475 095/300 Read block 3468 len 32768
26476 096/300 Write block 371 len 8192 data: 5 7
26477 097/300 Read block 2896 len 28672
26478 098/300 Read block 2098 len 32768
26479 099/300 Flush
26480 100/300 Read block 4936 len 24576
26481 101/300 Read block 139 len 32768
26482 102/300 Read block 3128 len 36864
26483 103/300 Write block 116 len 20480 data: 5 7 6 7 6
26484 104/300 Read block 1551 len 4096
26485 105/300 Write block 2408 len 24576 data: 6 5 7 6 6 6
26486 106/300 Write block 4991 len 12288 data: 4 3 5
26487 107/300 Read block 919 len 16384
26488 108/300 Read block 671 len 24576
26489 109/300 Write block 1119 len 12288 data: 3 3 3
26490 110/300 Write block 16 len 4096 data: 7
26491 111/300 Read block 839 len 8192
26492 112/300 Read block 1234 len 12288
26493 113/300 Write block 2815 len 16384 data: 7 7 6 6
26494 114/300 Read block 2901 len 32768
26495 115/300 Write block 417 len 4096 data: 5
26496 116/300 Write block 1270 len 36864 data: 3 5 4 5 5 7 8 9 8
26497 117/300 Write block 1230 len 36864 data: 4 5 5 5 5 5 6 6 7
26498 118/300 Write block 2145 len 40960 data: 2 2 2 2 2 2 2 3 3 3
26499 119/300 Flush
26500 120/300 Read block 4125 len 24576
26501 121/300 Read block 1313 len 40960
26502 122/300 Write block 3980 len 8192 data: 3 3
26503 123/300 Read block 1491 len 28672
26504 124/300 Write block 2620 len 12288 data: 9 7 7
26505 125/300 Read block 2489 len 4096
26506 126/300 Write block 262 len 20480 data: 5 6 4 4 4
26507 127/300 Write block 2244 len 4096 data: 5
26508 128/300 Write block 3395 len 40960 data: 5 6 5 5 6 7 10 8 8 6
26509 129/300 Flush
26510 130/300 Read block 4661 len 8192
26511 131/300 Flush
26512 132/300 Write block 4700 len 12288 data: 3 3 4
26513 133/300 Read block 3836 len 36864
26514 134/300 Read block 1783 len 12288
26515 135/300 Read block 2022 len 40960
26516 136/300 Read block 1349 len 12288
26517 137/300 Read block 3365 len 20480
26518 138/300 Write block 252 len 28672 data: 4 6 6 6 8 7 5
26519 139/300 Read block 1366 len 12288
26520 140/300 Read block 4530 len 28672
26521 141/300 Read block 14 len 16384
26522 142/300 Read block 4491 len 24576
26523 143/300 Write block 978 len 32768 data: 3 3 2 2 3 4 4 4
26524 144/300 Read block 3054 len 36864
26525 145/300 Read block 4651 len 20480
26526 146/300 Read block 2560 len 40960
26527 147/300 Write block 1923 len 28672 data: 4 5 6 6 6 5 5
26528 148/300 Read block 3448 len 12288
26529 149/300 Read block 3296 len 16384
26530 150/300 Write block 2315 len 32768 data: 9 9 9 8 8 6 7 6
26531 151/300 Write block 1950 len 8192 data: 6 8
26532 152/300 Read block 1901 len 8192
26533 153/300 Write block 110 len 4096 data: 5
26534 154/300 Flush
26535 155/300 Read block 2583 len 20480
26536 156/300 Read block 3649 len 8192
26537 157/300 Read block 1204 len 20480
26538 158/300 Write block 1728 len 16384 data: 7 6 6 6
26539 159/300 Write block 2203 len 12288 data: 6 6 6
26540 160/300 Read block 2395 len 40960
26541 161/300 Flush
26542 162/300 Write block 3693 len 20480 data: 5 4 4 4 4
26543 163/300 Read block 2633 len 28672
26544 164/300 Write block 1060 len 12288 data: 6 7 6
26545 165/300 Read block 468 len 40960
26546 166/300 Write block 3672 len 40960 data: 5 5 4 4 4 2 2 3 3 3
26547 167/300 Read block 385 len 4096
26548 168/300 Read block 1800 len 40960
26549 169/300 Read block 1165 len 40960
26550 170/300 Write block 3136 len 32768 data: 5 4 4 4 4 5 6 6
26551 171/300 Read block 3105 len 40960
26552 172/300 Read block 511 len 40960
26553 173/300 Read block 2497 len 24576
26554 174/300 Read block 1984 len 28672
26555 175/300 Read block 1071 len 32768
26556 176/300 Read block 3285 len 8192
26557 177/300 Read block 3135 len 32768
26558 178/300 Read block 450 len 12288
26559 179/300 Read block 73 len 32768
26560 180/300 Flush
26561 181/300 Read block 516 len 40960
26562 182/300 Read block 1355 len 20480
26563 183/300 Read block 4607 len 36864
26564 184/300 Read block 132 len 20480
26565 185/300 Read block 934 len 40960
26566 186/300 Read block 1276 len 28672
26567 187/300 Write block 2457 len 36864 data: 4 5 5 7 5 6 5 4 3
26568 188/300 Read block 2140 len 12288
26569 189/300 Write block 4108 len 24576 data: 3 4 4 3 4 2
26570 190/300 Read block 2067 len 8192
26571 191/300 Read block 3610 len 8192
26572 192/300 Write block 387 len 4096 data: 5
26573 193/300 Read block 330 len 4096
26574 194/300 Write block 3559 len 36864 data: 4 3 3 6 6 6 6 6 8
26575 195/300 Write block 700 len 4096 data: 6
26576 196/300 Flush
26577 197/300 Read block 1384 len 40960
26578 198/300 Read block 4214 len 24576
26579 199/300 Read block 258 len 40960
26580 200/300 Write block 4348 len 28672 data: 6 7 9 9 8 9 7
26581 201/300 Read block 2850 len 16384
26582 202/300 Write block 4838 len 40960 data: 6 6 7 6 8 7 10 10 11 11
26583 203/300 Read block 2809 len 32768
26584 204/300 Read block 4702 len 16384
26585 205/300 Read block 975 len 40960
26586 206/300 Write block 2961 len 20480 data: 4 2 2 2 3
26587 207/300 Read block 4008 len 4096
26588 208/300 Write block 4089 len 16384 data: 10 10 12 11
26589 209/300 Read block 1437 len 20480
26590 210/300 Flush
26591 211/300 Read block 3987 len 40960
26592 212/300 Read block 2616 len 28672
26593 213/300 Read block 77 len 28672
26594 214/300 Read block 4307 len 4096
26595 215/300 Write block 2719 len 20480 data: 6 8 7 6 7
26596 216/300 Read block 3513 len 32768
26597 217/300 Read block 1007 len 4096
26598 218/300 Flush
26599 219/300 Read block 1998 len 28672
26600 220/300 Read block 1552 len 4096
26601 221/300 Read block 703 len 32768
26602 222/300 Read block 391 len 32768
26603 223/300 Read block 4093 len 8192
26604 224/300 Write block 599 len 32768 data: 5 5 4 3 3 3 3 3
26605 225/300 Read block 2538 len 20480
26606 226/300 Read block 1475 len 4096
26607 227/300 Read block 3037 len 36864
26608 228/300 Write block 3976 len 24576 data: 5 4 4 3 4 4
26609 229/300 Write block 483 len 20480 data: 6 5 3 3 3
26610 230/300 Write block 1526 len 20480 data: 6 6 7 7 9
26611 231/300 Read block 3536 len 4096
26612 232/300 Write block 3815 len 4096 data: 5
26613 233/300 Write block 1514 len 40960 data: 4 4 3 3 4 4 4 6 6 7
26614 234/300 Read block 3195 len 32768
26615 235/300 Write block 2568 len 32768 data: 8 7 6 5 5 6 5 5
26616 236/300 Write block 4243 len 36864 data: 5 5 4 5 6 6 5 6 6
26617 237/300 Write block 135 len 4096 data: 8
26618 238/300 Write block 2936 len 20480 data: 4 4 4 5 4
26619 239/300 Write block 1452 len 36864 data: 8 6 6 7 6 5 6 5 4
26620 240/300 Write block 95 len 36864 data: 7 5 4 3 3 3 3 3 2
26621 241/300 Read block 3879 len 8192
26622 242/300 Write block 4328 len 24576 data: 6 5 5 5 4 3
26623 243/300 Read block 4588 len 28672
26624 244/300 Read block 3411 len 40960
26625 245/300 Read block 4603 len 32768
26626 246/300 Write block 3518 len 4096 data: 7
26627 247/300 Read block 2626 len 32768
26628 248/300 Write block 3280 len 28672 data: 6 7 9 8 8 7 7
26629 249/300 Read block 3471 len 4096
26630 250/300 Flush
26631 251/300 Write block 1396 len 40960 data: 5 4 6 6 5 6 6 7 6 5
26632 252/300 Write block 2675 len 16384 data: 6 4 4 4
26633 253/300 Write block 187 len 16384 data: 9 7 4 4
26634 254/300 Write block 2712 len 16384 data: 5 5 5 7
26635 255/300 Write block 4859 len 28672 data: 4 5 5 7 6 5 4
26636 256/300 Read block 1741 len 12288
26637 257/300 Read block 1630 len 20480
26638 258/300 Write block 3572 len 28672 data: 4 6 8 8 7 7 7
26639 259/300 Read block 2288 len 4096
26640 260/300 Flush
26641 261/300 Read block 772 len 16384
26642 262/300 Read block 1573 len 24576
26643 263/300 Write block 3585 len 12288 data: 7 4 4
26644 264/300 Write block 3994 len 28672 data: 7 6 7 6 8 10 8
26645 265/300 Read block 689 len 16384
26646 266/300 Write block 1819 len 4096 data: 8
26647 267/300 Write block 1063 len 4096 data: 5
26648 268/300 Read block 745 len 24576
26649 269/300 Read block 3626 len 36864
26650 270/300 Read block 4992 len 28672
26651 271/300 Read block 4687 len 20480
26652 272/300 Read block 2257 len 28672
26653 273/300 Write block 2848 len 40960 data: 4 5 5 5 5 5 5 6 7 7
26654 274/300 Write block 1684 len 8192 data: 5 5
26655 275/300 Read block 4287 len 36864
26656 276/300 Read block 1826 len 8192
26657 277/300 Read block 4281 len 32768
26658 278/300 Read block 3853 len 8192
26659 279/300 Write block 312 len 32768 data: 7 7 7 7 7 8 7 7
26660 280/300 Read block 3361 len 28672
26661 281/300 Read block 3165 len 16384
26662 282/300 Write block 171 len 36864 data: 5 4 5 4 3 3 3 4 5
26663 283/300 Write block 147 len 8192 data: 8 9
26664 284/300 Flush
26665 285/300 Write block 2556 len 4096 data: 8
26666 286/300 Read block 4976 len 16384
26667 287/300 Write block 3971 len 36864 data: 6 6 8 8 8 6 5 5 4
26668 288/300 Read block 838 len 12288
26669 289/300 Read block 4647 len 4096
26670 290/300 Write block 3511 len 28672 data: 4 4 4 5 6 7 8
26671 291/300 Write block 3503 len 32768 data: 3 5 6 5 4 4 4 5
26672 292/300 Write block 2967 len 28672 data: 6 7 8 7 7 6 6
26673 293/300 Read block 4331 len 32768
26674 294/300 Read block 4107 len 28672
26675 295/300 Write block 1421 len 40960 data: 3 5 5 5 6 5 6 4 3 4
26676 296/300 Read block 3717 len 24576
26677 297/300 Read block 3313 len 32768
26678 298/300 Read block 1603 len 36864
26679 299/300 Write block 2270 len 4096 data: 4
26680 300/300 Read block 2548 len 24576
26681 301/300 Write block 88 len 32768 data: 4 3 4 3 4 4 5 8
26682 Main task has work to do, go find it
26683 got dsc Start(0)
26684 start 0
26685 Jan 14 16:16:14.281 INFO request completed, latency_us: 194, response_code: 204, uri: /start/cid/0, method: POST, req_id: 226d5b91-e3e2-4abb-94cc-38d9f2979bb7, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
26686 Replay: started 0, returned:Ok(())
26687 ----------------------------------------------------------------
26688 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
26689 States: Offline Active Active
26690 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26691 8547 Acked Read 10 Sent Done Done false
26692 8548 Acked Read 6 Sent Done Done false
26693 8549 Acked Write 1 Sent Done Done false
26694 8550 Acked Write 2 Sent Done Done false
26695 8551 Acked Read 2 Sent Done Done false
26696 8552 Acked Write 9 Sent Done Done false
26697 8553 Acked Read 3 Sent Done Done false
26698 8554 Acked Read 3 Sent Done Done false
26699 8555 Acked Read 5 Sent Done Done false
26700 8556 Acked Read 7 Sent Done Done false
26701 8557 Acked Read 8 Sent Done Done false
26702 8558 Acked Write 2 Sent Done Done false
26703 8559 Acked Write 9 Sent Done Done false
26704 8560 Acked Read 9 Sent Done Done false
26705 8561 Acked Write 1 Sent Done Done false
26706 8562 Acked Write 1 Sent Done Done false
26707 8563 Acked Write 1 Sent Done Done false
26708 8564 Acked Write 10 Sent Done Done false
26709 8565 Acked Read 7 Sent Done Done false
26710 8566 Acked Write 5 Sent Done Done false
26711 8567 Acked Write 1 Sent Done Done false
26712 8568 Acked Write 3 Sent Done Done false
26713 8569 Acked Read 7 Sent Done Done false
26714 8570 Acked Read 5 Sent Done Done false
26715 8571 Acked Read 3 Sent Done Done false
26716 8572 Acked Read 1 Sent Done Done false
26717 8573 Acked Write 8 Sent Done Done false
26718 8574 Acked Read 10 Sent Done Done false
26719 8575 Acked Write 8 Sent Done Done false
26720 8576 Acked Read 1 Sent Done Done false
26721 8577 Acked Read 9 Sent Done Done false
26722 8578 Acked Write 1 Sent Done Done false
26723 8579 Acked Write 4 Sent Done Done false
26724 8580 Acked Flush 0 Sent Done Done false
26725 8581 Acked Read 9 Sent Done Done false
26726 8582 Acked Write 3 Sent Done Done false
26727 8583 Acked Read 1 Sent Done Done false
26728 8584 Acked Write 7 Sent Done Done false
26729 8585 Acked Read 9 Sent Done Done false
26730 8586 Acked Read 7 Sent Done Done false
26731 8587 Acked Read 10 Sent Done Done false
26732 8588 Acked Read 8 Sent Done Done false
26733 8589 Acked Read 6 Sent Done Done false
26734 8590 Acked Write 2 Sent Done Done false
26735 8591 Acked Write 4 Sent Done Done false
26736 8592 Acked Write 2 Sent Done Done false
26737 8593 Acked Write 7 Sent Done Done false
26738 8594 Acked Flush 0 Sent Done Done false
26739 8595 Acked Write 10 Sent Done Done false
26740 8596 Acked Write 6 Sent Done Done false
26741 8597 Acked Read 7 Sent Done Done false
26742 8598 Acked Read 5 Sent Done Done false
26743 8599 Acked Read 10 Sent Done Done false
26744 8600 Acked Flush 0 Sent Done Done false
26745 8601 Acked Read 8 Sent Done Done false
26746 8602 Acked Read 9 Sent Done Done false
26747 8603 Acked Write 4 Sent Done Done false
26748 8604 Acked Read 9 Sent Done Done false
26749 8605 Acked Write 2 Sent Done Done false
26750 8606 Acked Read 6 Sent Done Done false
26751 8607 Acked Write 2 Sent Done Done false
26752 8608 Acked Write 4 Sent Done Done false
26753 8609 Acked Read 6 Sent Done Done false
26754 8610 Acked Read 9 Sent Done Done false
26755 8611 Acked Flush 0 Sent Done Done false
26756 8612 Acked Write 1 Sent Done Done false
26757 8613 Acked Read 7 Sent Done Done false
26758 8614 Acked Flush 0 Sent Done Done false
26759 8615 Acked Write 9 Sent Done Done false
26760 8616 Acked Write 7 Sent Done Done false
26761 8617 Acked Read 8 Sent Done Done false
26762 8618 Acked Read 4 Sent Done Done false
26763 8619 Acked Write 8 Sent Done Done false
26764 8620 Acked Write 2 Sent Done Done false
26765 8621 Acked Write 8 Sent Done Done false
26766 8622 Acked Read 8 Sent Done Done false
26767 8623 Acked Read 7 Sent Done Done false
26768 8624 Acked Write 7 Sent Done Done false
26769 8625 Acked Write 4 Sent Done Done false
26770 8626 Acked Write 3 Sent Done Done false
26771 8627 Acked Write 4 Sent Done Done false
26772 8628 Acked Write 5 Sent Done Done false
26773 8629 Acked Flush 0 Sent Done Done false
26774 8630 Acked Write 2 Sent Done Done false
26775 8631 Acked Flush 0 Sent Done Done false
26776 8632 Acked Read 3 Sent Done Done false
26777 8633 Acked Read 2 Sent Done Done false
26778 8634 Acked Flush 0 Sent Done Done false
26779 8635 Acked Write 3 Sent Done Done false
26780 8636 Acked Write 6 Sent Done Done false
26781 8637 Acked Read 5 Sent Done Done false
26782 8638 Acked Read 4 Sent Done Done false
26783 8639 Acked Read 3 Sent Done Done false
26784 8640 Acked Write 9 Sent Done Done false
26785 8641 Acked Read 8 Sent Done Done false
26786 8642 Acked Write 2 Sent Done Done false
26787 8643 Acked Read 7 Sent Done Done false
26788 8644 Acked Read 8 Sent Done Done false
26789 8645 Acked Flush 0 Sent Done Done false
26790 8646 Acked Read 6 Sent Done Done false
26791 8647 Acked Read 8 Sent Done Done false
26792 8648 Acked Read 9 Sent Done Done false
26793 8649 Acked Write 5 Sent Done Done false
26794 8650 Acked Read 1 Sent Done Done false
26795 8651 Acked Write 6 Sent Done Done false
26796 8652 Acked Write 3 Sent Done Done false
26797 8653 Acked Read 4 Sent Done Done false
26798 8654 Acked Read 6 Sent Done Done false
26799 8655 Acked Write 3 Sent Done Done false
26800 8656 Acked Write 1 Sent Done Done false
26801 8657 Acked Read 2 Sent Done Done false
26802 8658 Acked Read 3 Sent Done Done false
26803 8659 Acked Write 4 Sent Done Done false
26804 8660 Acked Read 8 Sent Done Done false
26805 8661 Acked Write 1 Sent Done Done false
26806 8662 Acked Write 9 Sent Done Done false
26807 8663 Acked Write 9 Sent Done Done false
26808 8664 Acked Write 10 Sent Done Done false
26809 8665 Acked Flush 0 Sent Done Done false
26810 8666 Acked Read 6 Sent Done Done false
26811 8667 Acked Read 10 Sent Done Done false
26812 8668 Acked Write 2 Sent Done Done false
26813 8669 Acked Read 7 Sent Done Done false
26814 8670 Acked Write 3 Sent Done Done false
26815 8671 Acked Read 1 Sent Done Done false
26816 8672 Acked Write 5 Sent Done Done false
26817 8673 Acked Write 1 Sent Done Done false
26818 8674 Acked Write 10 Sent Done Done false
26819 8675 Acked Flush 0 Sent Done Done false
26820 8676 Acked Read 2 Sent Done Done false
26821 8677 Acked Flush 0 Sent Done Done false
26822 8678 Acked Write 3 Sent Done Done false
26823 8679 Acked Read 9 Sent Done Done false
26824 8680 Acked Read 3 Sent Done Done false
26825 8681 Acked Read 10 Sent Done Done false
26826 8682 Acked Read 3 Sent Done Done false
26827 8683 Acked Read 5 Sent Done Done false
26828 8684 Acked Write 7 Sent Done Done false
26829 8685 Acked Read 3 Sent Done Done false
26830 8686 Acked Read 7 Sent Done Done false
26831 8687 Acked Read 4 Sent Done Done false
26832 8688 Acked Read 6 Sent Done Done false
26833 8689 Acked Write 8 Sent Done Done false
26834 8690 Acked Read 9 Sent Done Done false
26835 8691 Acked Read 5 Sent Done Done false
26836 8692 Acked Read 10 Sent Done Done false
26837 8693 Acked Write 7 Sent Done Done false
26838 8694 Acked Read 3 Sent Done Done false
26839 8695 Acked Read 4 Sent Done Done false
26840 8696 Acked Write 8 Sent Done Done false
26841 8697 Acked Write 2 Sent Done Done false
26842 8698 Acked Read 2 Sent Done Done false
26843 8699 Acked Write 1 Sent Done Done false
26844 8700 Acked Flush 0 Sent Done Done false
26845 8701 Acked Read 5 Sent Done Done false
26846 8702 Acked Read 2 Sent Done Done false
26847 8703 Acked Read 5 Sent Done Done false
26848 8704 Acked Write 4 Sent Done Done false
26849 8705 Acked Write 3 Sent Done Done false
26850 8706 Acked Read 10 Sent Done Done false
26851 8707 Acked Flush 0 Sent Done Done false
26852 8708 Acked Write 5 Sent Done Done false
26853 8709 Acked Read 7 Sent Done Done false
26854 8710 Acked Write 3 Sent Done Done false
26855 8711 Acked Read 10 Sent Done Done false
26856 8712 Acked Write 10 Sent Done Done false
26857 8713 Acked Read 1 Sent Done Done false
26858 8714 Acked Read 10 Sent Done Done false
26859 8715 Acked Read 10 Sent Done Done false
26860 8716 Acked Write 8 Sent Done Done false
26861 8717 Acked Read 10 Sent Done Done false
26862 8718 Acked Read 10 Sent Done Done false
26863 8719 Acked Read 6 Sent Done Done false
26864 8720 Acked Read 7 Sent Done Done false
26865 8721 Acked Read 8 Sent Done Done false
26866 8722 Acked Read 2 Sent Done Done false
26867 8723 Acked Read 8 Sent Done Done false
26868 8724 Acked Read 3 Sent Done Done false
26869 8725 Acked Read 8 Sent Done Done false
26870 8726 Acked Flush 0 Sent Done Done false
26871 8727 Acked Read 10 Sent Done Done false
26872 8728 Acked Read 5 Sent Done Done false
26873 8729 Acked Read 9 Sent Done Done false
26874 8730 Acked Read 5 Sent Done Done false
26875 8731 Acked Read 10 Sent Done Done false
26876 8732 Acked Read 7 Sent Done Done false
26877 8733 Acked Write 9 Sent Done Done false
26878 8734 Acked Read 3 Sent Done Done false
26879 8735 Acked Write 6 Sent Done Done false
26880 8736 Acked Read 2 Sent Done Done false
26881 8737 Acked Read 2 Sent Done Done false
26882 8738 Acked Write 1 Sent Done Done false
26883 8739 Acked Read 1 Sent Done Done false
26884 8740 Acked Write 9 Sent Done Done false
26885 8741 Acked Write 1 Sent Done Done false
26886 8742 Acked Flush 0 Sent Done Done false
26887 8743 Acked Read 10 Sent Done Done false
26888 8744 Acked Read 6 Sent Done Done false
26889 8745 Acked Read 10 Sent Done Done false
26890 8746 Acked Write 7 Sent Done Done false
26891 8747 Acked Read 4 Sent Done Done false
26892 8748 Acked Write 10 Sent Done Done false
26893 8749 Acked Read 8 Sent Done Done false
26894 8750 Acked Read 4 Sent Done Done false
26895 8751 Acked Read 10 Sent Done Done false
26896 8752 Acked Write 5 Sent Done Done false
26897 8753 Acked Read 1 Sent Done Done false
26898 8754 Acked Write 4 Sent Done Done false
26899 8755 Acked Read 5 Sent Done Done false
26900 8756 Acked Flush 0 Sent Done Done false
26901 8757 Acked Read 10 Sent Done Done false
26902 8758 Acked Read 7 Sent Done Done false
26903 8759 Acked Read 7 Sent Done Done false
26904 8760 Acked Read 1 Sent Done Done false
26905 8761 Acked Write 5 Sent Done Done false
26906 8762 Acked Read 8 Sent Done Done false
26907 8763 Acked Read 1 Sent Done Done false
26908 8764 Acked Flush 0 Sent Done Done false
26909 8765 Acked Read 7 Sent Done Done false
26910 8766 Acked Read 1 Sent Done Done false
26911 8767 Acked Read 8 Sent Done Done false
26912 8768 Acked Read 8 Sent Done Done false
26913 8769 Acked Read 2 Sent Done Done false
26914 8770 Acked Write 8 Sent Done Done false
26915 8771 Acked Read 5 Sent Done Done false
26916 8772 Acked Read 1 Sent Done Done false
26917 8773 Acked Read 9 Sent Done Done false
26918 8774 Acked Write 6 Sent Done Done false
26919 8775 Acked Write 5 Sent Done Done false
26920 8776 Acked Write 5 Sent Done Done false
26921 8777 Acked Read 1 Sent Done Done false
26922 8778 Acked Write 1 Sent Done Done false
26923 8779 Acked Write 10 Sent Done Done false
26924 8780 Acked Read 8 Sent Done Done false
26925 8781 Acked Write 8 Sent Done Done false
26926 8782 Acked Write 9 Sent Done Done false
26927 8783 Acked Write 1 Sent Done Done false
26928 8784 Acked Write 5 Sent Done Done false
26929 8785 Acked Write 9 Sent Done Done false
26930 8786 Acked Write 9 Sent Done Done false
26931 8787 Acked Read 2 Sent Done Done false
26932 8788 Acked Write 6 Sent Done Done false
26933 8789 Acked Read 7 Sent Done Done false
26934 8790 Acked Read 10 Sent Done Done false
26935 8791 Acked Read 8 Sent Done Done false
26936 8792 Acked Write 1 Sent Done Done false
26937 8793 Acked Read 8 Sent Done Done false
26938 8794 Acked Write 7 Sent Done Done false
26939 8795 Acked Read 1 Sent Done Done false
26940 8796 Acked Flush 0 Sent Done Done false
26941 8797 Acked Write 10 Sent Done Done false
26942 8798 Acked Write 4 Sent Done Done false
26943 8799 Acked Write 4 Sent Done Done false
26944 8800 Acked Write 4 Sent Done Done false
26945 8801 Acked Write 7 Sent Done Done false
26946 8802 Acked Read 3 Sent Done Done false
26947 8803 Acked Read 5 Sent Done Done false
26948 8804 Acked Write 7 Sent Done Done false
26949 8805 Acked Read 1 Sent Done Done false
26950 8806 Acked Flush 0 Sent Done Done false
26951 8807 Acked Read 4 Sent Done Done false
26952 8808 Acked Read 6 Sent Done Done false
26953 8809 Acked Write 3 Sent Done Done false
26954 8810 Acked Write 7 Sent Done Done false
26955 8811 Acked Read 4 Sent Done Done false
26956 8812 Acked Write 1 Sent Done Done false
26957 8813 Acked Write 1 Sent Done Done false
26958 8814 Acked Read 6 Sent Done Done false
26959 8815 Acked Read 9 Sent Done Done false
26960 8816 Acked Read 7 Sent Done Done false
26961 8817 Acked Read 5 Sent Done Done false
26962 8818 Acked Read 7 Sent Done Done false
26963 8819 Acked Write 10 Sent Done Done false
26964 8820 Acked Write 2 Sent Done Done false
26965 8821 Acked Read 9 Sent Done Done false
26966 8822 Acked Read 2 Sent Done Done false
26967 8823 Acked Read 8 Sent Done Done false
26968 8824 Acked Read 2 Sent Done Done false
26969 8825 Acked Write 8 Sent Done Done false
26970 8826 Acked Read 7 Sent Done Done false
26971 8827 Acked Read 4 Sent Done Done false
26972 8828 Acked Write 9 Sent Done Done false
26973 8829 Acked Write 2 Sent Done Done false
26974 8830 Acked Flush 0 Sent Done Done false
26975 8831 Acked Write 1 Sent Done Done false
26976 8832 Acked Read 4 Sent Done Done false
26977 8833 Acked Write 9 Sent Done Done false
26978 8834 Acked Read 3 Sent Done Done false
26979 8835 Acked Read 1 Sent Done Done false
26980 8836 Acked Write 7 Sent Done Done false
26981 8837 Acked Write 8 Sent Done Done false
26982 8838 Acked Write 7 Sent Done Done false
26983 8839 Acked Read 8 Sent Done Done false
26984 8840 Acked Read 7 Sent Done Done false
26985 8841 Acked Write 10 Sent Done Done false
26986 8842 Acked Read 6 Sent Done Done false
26987 8843 Acked Read 8 Sent Done Done false
26988 8844 Acked Read 9 Sent Done Done false
26989 8845 Acked Write 1 Sent Done Done false
26990 8846 Acked Read 6 Sent Done Done false
26991 8847 Acked Write 8 Sent Done Done false
26992 STATES DS:0 DS:1 DS:2 TOTAL
26993 Sent 301 0 0 301
26994 Done 0 301 301 602
26995 Skipped 0 0 0 0
26996 Error 0 0 0 0
26997 Last Flush: 8546 8830 8830
26998 Downstairs last five completed: 8546 8545 8544 8543 8542
26999 Upstairs last five completed: 8847 8846 8845 8844 8843
27000 CLIENT: Up:0 ds:301 act:2
27001 ----------------------------------------------------------------
27002 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27003 States: Offline Active Active
27004 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27005 8547 Acked Read 10 Sent Done Done false
27006 8548 Acked Read 6 Sent Done Done false
27007 8549 Acked Write 1 Sent Done Done false
27008 8550 Acked Write 2 Sent Done Done false
27009 8551 Acked Read 2 Sent Done Done false
27010 8552 Acked Write 9 Sent Done Done false
27011 8553 Acked Read 3 Sent Done Done false
27012 8554 Acked Read 3 Sent Done Done false
27013 8555 Acked Read 5 Sent Done Done false
27014 8556 Acked Read 7 Sent Done Done false
27015 8557 Acked Read 8 Sent Done Done false
27016 8558 Acked Write 2 Sent Done Done false
27017 8559 Acked Write 9 Sent Done Done false
27018 8560 Acked Read 9 Sent Done Done false
27019 8561 Acked Write 1 Sent Done Done false
27020 8562 Acked Write 1 Sent Done Done false
27021 8563 Acked Write 1 Sent Done Done false
27022 8564 Acked Write 10 Sent Done Done false
27023 8565 Acked Read 7 Sent Done Done false
27024 8566 Acked Write 5 Sent Done Done false
27025 8567 Acked Write 1 Sent Done Done false
27026 8568 Acked Write 3 Sent Done Done false
27027 8569 Acked Read 7 Sent Done Done false
27028 8570 Acked Read 5 Sent Done Done false
27029 8571 Acked Read 3 Sent Done Done false
27030 8572 Acked Read 1 Sent Done Done false
27031 8573 Acked Write 8 Sent Done Done false
27032 8574 Acked Read 10 Sent Done Done false
27033 8575 Acked Write 8 Sent Done Done false
27034 8576 Acked Read 1 Sent Done Done false
27035 8577 Acked Read 9 Sent Done Done false
27036 8578 Acked Write 1 Sent Done Done false
27037 8579 Acked Write 4 Sent Done Done false
27038 8580 Acked Flush 0 Sent Done Done false
27039 8581 Acked Read 9 Sent Done Done false
27040 8582 Acked Write 3 Sent Done Done false
27041 8583 Acked Read 1 Sent Done Done false
27042 8584 Acked Write 7 Sent Done Done false
27043 8585 Acked Read 9 Sent Done Done false
27044 8586 Acked Read 7 Sent Done Done false
27045 8587 Acked Read 10 Sent Done Done false
27046 8588 Acked Read 8 Sent Done Done false
27047 8589 Acked Read 6 Sent Done Done false
27048 8590 Acked Write 2 Sent Done Done false
27049 8591 Acked Write 4 Sent Done Done false
27050 8592 Acked Write 2 Sent Done Done false
27051 8593 Acked Write 7 Sent Done Done false
27052 8594 Acked Flush 0 Sent Done Done false
27053 8595 Acked Write 10 Sent Done Done false
27054 8596 Acked Write 6 Sent Done Done false
27055 8597 Acked Read 7 Sent Done Done false
27056 8598 Acked Read 5 Sent Done Done false
27057 8599 Acked Read 10 Sent Done Done false
27058 8600 Acked Flush 0 Sent Done Done false
27059 8601 Acked Read 8 Sent Done Done false
27060 8602 Acked Read 9 Sent Done Done false
27061 8603 Acked Write 4 Sent Done Done false
27062 8604 Acked Read 9 Sent Done Done false
27063 8605 Acked Write 2 Sent Done Done false
27064 8606 Acked Read 6 Sent Done Done false
27065 8607 Acked Write 2 Sent Done Done false
27066 8608 Acked Write 4 Sent Done Done false
27067 8609 Acked Read 6 Sent Done Done false
27068 8610 Acked Read 9 Sent Done Done false
27069 8611 Acked Flush 0 Sent Done Done false
27070 8612 Acked Write 1 Sent Done Done false
27071 8613 Acked Read 7 Sent Done Done false
27072 8614 Acked Flush 0 Sent Done Done false
27073 8615 Acked Write 9 Sent Done Done false
27074 8616 Acked Write 7 Sent Done Done false
27075 8617 Acked Read 8 Sent Done Done false
27076 8618 Acked Read 4 Sent Done Done false
27077 8619 Acked Write 8 Sent Done Done false
27078 8620 Acked Write 2 Sent Done Done false
27079 8621 Acked Write 8 Sent Done Done false
27080 8622 Acked Read 8 Sent Done Done false
27081 8623 Acked Read 7 Sent Done Done false
27082 8624 Acked Write 7 Sent Done Done false
27083 8625 Acked Write 4 Sent Done Done false
27084 8626 Acked Write 3 Sent Done Done false
27085 8627 Acked Write 4 Sent Done Done false
27086 8628 Acked Write 5 Sent Done Done false
27087 8629 Acked Flush 0 Sent Done Done false
27088 8630 Acked Write 2 Sent Done Done false
27089 8631 Acked Flush 0 Sent Done Done false
27090 8632 Acked Read 3 Sent Done Done false
27091 8633 Acked Read 2 Sent Done Done false
27092 8634 Acked Flush 0 Sent Done Done false
27093 8635 Acked Write 3 Sent Done Done false
27094 8636 Acked Write 6 Sent Done Done false
27095 8637 Acked Read 5 Sent Done Done false
27096 8638 Acked Read 4 Sent Done Done false
27097 8639 Acked Read 3 Sent Done Done false
27098 8640 Acked Write 9 Sent Done Done false
27099 8641 Acked Read 8 Sent Done Done false
27100 8642 Acked Write 2 Sent Done Done false
27101 8643 Acked Read 7 Sent Done Done false
27102 8644 Acked Read 8 Sent Done Done false
27103 8645 Acked Flush 0 Sent Done Done false
27104 8646 Acked Read 6 Sent Done Done false
27105 8647 Acked Read 8 Sent Done Done false
27106 8648 Acked Read 9 Sent Done Done false
27107 8649 Acked Write 5 Sent Done Done false
27108 8650 Acked Read 1 Sent Done Done false
27109 8651 Acked Write 6 Sent Done Done false
27110 8652 Acked Write 3 Sent Done Done false
27111 8653 Acked Read 4 Sent Done Done false
27112 8654 Acked Read 6 Sent Done Done false
27113 8655 Acked Write 3 Sent Done Done false
27114 8656 Acked Write 1 Sent Done Done false
27115 8657 Acked Read 2 Sent Done Done false
27116 8658 Acked Read 3 Sent Done Done false
27117 8659 Acked Write 4 Sent Done Done false
27118 8660 Acked Read 8 Sent Done Done false
27119 8661 Acked Write 1 Sent Done Done false
27120 8662 Acked Write 9 Sent Done Done false
27121 8663 Acked Write 9 Sent Done Done false
27122 8664 Acked Write 10 Sent Done Done false
27123 8665 Acked Flush 0 Sent Done Done false
27124 8666 Acked Read 6 Sent Done Done false
27125 8667 Acked Read 10 Sent Done Done false
27126 8668 Acked Write 2 Sent Done Done false
27127 8669 Acked Read 7 Sent Done Done false
27128 8670 Acked Write 3 Sent Done Done false
27129 8671 Acked Read 1 Sent Done Done false
27130 8672 Acked Write 5 Sent Done Done false
27131 8673 Acked Write 1 Sent Done Done false
27132 8674 Acked Write 10 Sent Done Done false
27133 8675 Acked Flush 0 Sent Done Done false
27134 8676 Acked Read 2 Sent Done Done false
27135 8677 Acked Flush 0 Sent Done Done false
27136 8678 Acked Write 3 Sent Done Done false
27137 8679 Acked Read 9 Sent Done Done false
27138 8680 Acked Read 3 Sent Done Done false
27139 8681 Acked Read 10 Sent Done Done false
27140 8682 Acked Read 3 Sent Done Done false
27141 8683 Acked Read 5 Sent Done Done false
27142 8684 Acked Write 7 Sent Done Done false
27143 8685 Acked Read 3 Sent Done Done false
27144 8686 Acked Read 7 Sent Done Done false
27145 8687 Acked Read 4 Sent Done Done false
27146 8688 Acked Read 6 Sent Done Done false
27147 8689 Acked Write 8 Sent Done Done false
27148 8690 Acked Read 9 Sent Done Done false
27149 8691 Acked Read 5 Sent Done Done false
27150 8692 Acked Read 10 Sent Done Done false
27151 8693 Acked Write 7 Sent Done Done false
27152 8694 Acked Read 3 Sent Done Done false
27153 8695 Acked Read 4 Sent Done Done false
27154 8696 Acked Write 8 Sent Done Done false
27155 8697 Acked Write 2 Sent Done Done false
27156 8698 Acked Read 2 Sent Done Done false
27157 8699 Acked Write 1 Sent Done Done false
27158 8700 Acked Flush 0 Sent Done Done false
27159 8701 Acked Read 5 Sent Done Done false
27160 8702 Acked Read 2 Sent Done Done false
27161 8703 Acked Read 5 Sent Done Done false
27162 8704 Acked Write 4 Sent Done Done false
27163 8705 Acked Write 3 Sent Done Done false
27164 8706 Acked Read 10 Sent Done Done false
27165 8707 Acked Flush 0 Sent Done Done false
27166 8708 Acked Write 5 Sent Done Done false
27167 8709 Acked Read 7 Sent Done Done false
27168 8710 Acked Write 3 Sent Done Done false
27169 8711 Acked Read 10 Sent Done Done false
27170 8712 Acked Write 10 Sent Done Done false
27171 8713 Acked Read 1 Sent Done Done false
27172 8714 Acked Read 10 Sent Done Done false
27173 8715 Acked Read 10 Sent Done Done false
27174 8716 Acked Write 8 Sent Done Done false
27175 8717 Acked Read 10 Sent Done Done false
27176 8718 Acked Read 10 Sent Done Done false
27177 8719 Acked Read 6 Sent Done Done false
27178 8720 Acked Read 7 Sent Done Done false
27179 8721 Acked Read 8 Sent Done Done false
27180 8722 Acked Read 2 Sent Done Done false
27181 8723 Acked Read 8 Sent Done Done false
27182 8724 Acked Read 3 Sent Done Done false
27183 8725 Acked Read 8 Sent Done Done false
27184 8726 Acked Flush 0 Sent Done Done false
27185 8727 Acked Read 10 Sent Done Done false
27186 8728 Acked Read 5 Sent Done Done false
27187 8729 Acked Read 9 Sent Done Done false
27188 8730 Acked Read 5 Sent Done Done false
27189 8731 Acked Read 10 Sent Done Done false
27190 8732 Acked Read 7 Sent Done Done false
27191 8733 Acked Write 9 Sent Done Done false
27192 8734 Acked Read 3 Sent Done Done false
27193 8735 Acked Write 6 Sent Done Done false
27194 8736 Acked Read 2 Sent Done Done false
27195 8737 Acked Read 2 Sent Done Done false
27196 8738 Acked Write 1 Sent Done Done false
27197 8739 Acked Read 1 Sent Done Done false
27198 8740 Acked Write 9 Sent Done Done false
27199 8741 Acked Write 1 Sent Done Done false
27200 8742 Acked Flush 0 Sent Done Done false
27201 8743 Acked Read 10 Sent Done Done false
27202 8744 Acked Read 6 Sent Done Done false
27203 8745 Acked Read 10 Sent Done Done false
27204 8746 Acked Write 7 Sent Done Done false
27205 8747 Acked Read 4 Sent Done Done false
27206 8748 Acked Write 10 Sent Done Done false
27207 8749 Acked Read 8 Sent Done Done false
27208 8750 Acked Read 4 Sent Done Done false
27209 8751 Acked Read 10 Sent Done Done false
27210 8752 Acked Write 5 Sent Done Done false
27211 8753 Acked Read 1 Sent Done Done false
27212 8754 Acked Write 4 Sent Done Done false
27213 8755 Acked Read 5 Sent Done Done false
27214 8756 Acked Flush 0 Sent Done Done false
27215 8757 Acked Read 10 Sent Done Done false
27216 8758 Acked Read 7 Sent Done Done false
27217 8759 Acked Read 7 Sent Done Done false
27218 8760 Acked Read 1 Sent Done Done false
27219 8761 Acked Write 5 Sent Done Done false
27220 8762 Acked Read 8 Sent Done Done false
27221 8763 Acked Read 1 Sent Done Done false
27222 8764 Acked Flush 0 Sent Done Done false
27223 8765 Acked Read 7 Sent Done Done false
27224 8766 Acked Read 1 Sent Done Done false
27225 8767 Acked Read 8 Sent Done Done false
27226 8768 Acked Read 8 Sent Done Done false
27227 8769 Acked Read 2 Sent Done Done false
27228 8770 Acked Write 8 Sent Done Done false
27229 8771 Acked Read 5 Sent Done Done false
27230 8772 Acked Read 1 Sent Done Done false
27231 8773 Acked Read 9 Sent Done Done false
27232 8774 Acked Write 6 Sent Done Done false
27233 8775 Acked Write 5 Sent Done Done false
27234 8776 Acked Write 5 Sent Done Done false
27235 8777 Acked Read 1 Sent Done Done false
27236 8778 Acked Write 1 Sent Done Done false
27237 8779 Acked Write 10 Sent Done Done false
27238 8780 Acked Read 8 Sent Done Done false
27239 8781 Acked Write 8 Sent Done Done false
27240 8782 Acked Write 9 Sent Done Done false
27241 8783 Acked Write 1 Sent Done Done false
27242 8784 Acked Write 5 Sent Done Done false
27243 8785 Acked Write 9 Sent Done Done false
27244 8786 Acked Write 9 Sent Done Done false
27245 8787 Acked Read 2 Sent Done Done false
27246 8788 Acked Write 6 Sent Done Done false
27247 8789 Acked Read 7 Sent Done Done false
27248 8790 Acked Read 10 Sent Done Done false
27249 8791 Acked Read 8 Sent Done Done false
27250 8792 Acked Write 1 Sent Done Done false
27251 8793 Acked Read 8 Sent Done Done false
27252 8794 Acked Write 7 Sent Done Done false
27253 8795 Acked Read 1 Sent Done Done false
27254 8796 Acked Flush 0 Sent Done Done false
27255 8797 Acked Write 10 Sent Done Done false
27256 8798 Acked Write 4 Sent Done Done false
27257 8799 Acked Write 4 Sent Done Done false
27258 8800 Acked Write 4 Sent Done Done false
27259 8801 Acked Write 7 Sent Done Done false
27260 8802 Acked Read 3 Sent Done Done false
27261 8803 Acked Read 5 Sent Done Done false
27262 8804 Acked Write 7 Sent Done Done false
27263 8805 Acked Read 1 Sent Done Done false
27264 8806 Acked Flush 0 Sent Done Done false
27265 8807 Acked Read 4 Sent Done Done false
27266 8808 Acked Read 6 Sent Done Done false
27267 8809 Acked Write 3 Sent Done Done false
27268 8810 Acked Write 7 Sent Done Done false
27269 8811 Acked Read 4 Sent Done Done false
27270 8812 Acked Write 1 Sent Done Done false
27271 8813 Acked Write 1 Sent Done Done false
27272 8814 Acked Read 6 Sent Done Done false
27273 8815 Acked Read 9 Sent Done Done false
27274 8816 Acked Read 7 Sent Done Done false
27275 8817 Acked Read 5 Sent Done Done false
27276 8818 Acked Read 7 Sent Done Done false
27277 8819 Acked Write 10 Sent Done Done false
27278 8820 Acked Write 2 Sent Done Done false
27279 8821 Acked Read 9 Sent Done Done false
27280 8822 Acked Read 2 Sent Done Done false
27281 8823 Acked Read 8 Sent Done Done false
27282 8824 Acked Read 2 Sent Done Done false
27283 8825 Acked Write 8 Sent Done Done false
27284 8826 Acked Read 7 Sent Done Done false
27285 8827 Acked Read 4 Sent Done Done false
27286 8828 Acked Write 9 Sent Done Done false
27287 8829 Acked Write 2 Sent Done Done false
27288 8830 Acked Flush 0 Sent Done Done false
27289 8831 Acked Write 1 Sent Done Done false
27290 8832 Acked Read 4 Sent Done Done false
27291 8833 Acked Write 9 Sent Done Done false
27292 8834 Acked Read 3 Sent Done Done false
27293 8835 Acked Read 1 Sent Done Done false
27294 8836 Acked Write 7 Sent Done Done false
27295 8837 Acked Write 8 Sent Done Done false
27296 8838 Acked Write 7 Sent Done Done false
27297 8839 Acked Read 8 Sent Done Done false
27298 8840 Acked Read 7 Sent Done Done false
27299 8841 Acked Write 10 Sent Done Done false
27300 8842 Acked Read 6 Sent Done Done false
27301 8843 Acked Read 8 Sent Done Done false
27302 8844 Acked Read 9 Sent Done Done false
27303 8845 Acked Write 1 Sent Done Done false
27304 8846 Acked Read 6 Sent Done Done false
27305 8847 Acked Write 8 Sent Done Done false
27306 8848 Acked Flush 0 Sent Done Done false
27307 STATES DS:0 DS:1 DS:2 TOTAL
27308 Sent 302 0 0 302
27309 Done 0 302 302 604
27310 Skipped 0 0 0 0
27311 Error 0 0 0 0
27312 Last Flush: 8546 8848 8848
27313 Downstairs last five completed: 8546 8545 8544 8543 8542
27314 Upstairs last five completed: 8848 8847 8846 8845 8844
27315 CLIENT: Up:0 ds:302 act:2
273162025-01-14T16:16:20.036ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
273172025-01-14T16:16:20.036ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
273182025-01-14T16:16:20.036ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
273192025-01-14T16:16:20.036ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = c2423ef0-728e-4895-997a-d9044c50948b
273202025-01-14T16:16:20.036ZINFOcrucible: client task is exiting = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
27321 [8810] got start action
27322 [8810] I am going to restart
27323 Starting downstairs at port 8810
27324 Make output file at "/tmp/dsc/downstairs-8810.txt"
27325 [8810][0] reports Starting
27326 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1228)
27327 [8810][0] reports Running
27328 ----------------------------------------------------------------
27329 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27330 States: Offline Active Active
27331 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27332 8547 Acked Read 10 Sent Done Done false
27333 8548 Acked Read 6 Sent Done Done false
27334 8549 Acked Write 1 Sent Done Done false
27335 8550 Acked Write 2 Sent Done Done false
27336 8551 Acked Read 2 Sent Done Done false
27337 8552 Acked Write 9 Sent Done Done false
27338 8553 Acked Read 3 Sent Done Done false
27339 8554 Acked Read 3 Sent Done Done false
27340 8555 Acked Read 5 Sent Done Done false
27341 8556 Acked Read 7 Sent Done Done false
27342 8557 Acked Read 8 Sent Done Done false
27343 8558 Acked Write 2 Sent Done Done false
27344 8559 Acked Write 9 Sent Done Done false
27345 8560 Acked Read 9 Sent Done Done false
27346 8561 Acked Write 1 Sent Done Done false
27347 8562 Acked Write 1 Sent Done Done false
27348 8563 Acked Write 1 Sent Done Done false
27349 8564 Acked Write 10 Sent Done Done false
27350 8565 Acked Read 7 Sent Done Done false
27351 8566 Acked Write 5 Sent Done Done false
27352 8567 Acked Write 1 Sent Done Done false
27353 8568 Acked Write 3 Sent Done Done false
27354 8569 Acked Read 7 Sent Done Done false
27355 8570 Acked Read 5 Sent Done Done false
27356 8571 Acked Read 3 Sent Done Done false
27357 8572 Acked Read 1 Sent Done Done false
27358 8573 Acked Write 8 Sent Done Done false
27359 8574 Acked Read 10 Sent Done Done false
27360 8575 Acked Write 8 Sent Done Done false
27361 8576 Acked Read 1 Sent Done Done false
27362 8577 Acked Read 9 Sent Done Done false
27363 8578 Acked Write 1 Sent Done Done false
27364 8579 Acked Write 4 Sent Done Done false
27365 8580 Acked Flush 0 Sent Done Done false
27366 8581 Acked Read 9 Sent Done Done false
27367 8582 Acked Write 3 Sent Done Done false
27368 8583 Acked Read 1 Sent Done Done false
27369 8584 Acked Write 7 Sent Done Done false
27370 8585 Acked Read 9 Sent Done Done false
27371 8586 Acked Read 7 Sent Done Done false
27372 8587 Acked Read 10 Sent Done Done false
27373 8588 Acked Read 8 Sent Done Done false
27374 8589 Acked Read 6 Sent Done Done false
27375 8590 Acked Write 2 Sent Done Done false
27376 8591 Acked Write 4 Sent Done Done false
27377 8592 Acked Write 2 Sent Done Done false
27378 8593 Acked Write 7 Sent Done Done false
27379 8594 Acked Flush 0 Sent Done Done false
27380 8595 Acked Write 10 Sent Done Done false
27381 8596 Acked Write 6 Sent Done Done false
27382 8597 Acked Read 7 Sent Done Done false
27383 8598 Acked Read 5 Sent Done Done false
27384 8599 Acked Read 10 Sent Done Done false
27385 8600 Acked Flush 0 Sent Done Done false
27386 8601 Acked Read 8 Sent Done Done false
27387 8602 Acked Read 9 Sent Done Done false
27388 8603 Acked Write 4 Sent Done Done false
27389 8604 Acked Read 9 Sent Done Done false
27390 8605 Acked Write 2 Sent Done Done false
27391 8606 Acked Read 6 Sent Done Done false
27392 8607 Acked Write 2 Sent Done Done false
27393 8608 Acked Write 4 Sent Done Done false
27394 8609 Acked Read 6 Sent Done Done false
27395 8610 Acked Read 9 Sent Done Done false
27396 8611 Acked Flush 0 Sent Done Done false
27397 8612 Acked Write 1 Sent Done Done false
27398 8613 Acked Read 7 Sent Done Done false
27399 8614 Acked Flush 0 Sent Done Done false
27400 8615 Acked Write 9 Sent Done Done false
27401 8616 Acked Write 7 Sent Done Done false
27402 8617 Acked Read 8 Sent Done Done false
27403 8618 Acked Read 4 Sent Done Done false
27404 8619 Acked Write 8 Sent Done Done false
27405 8620 Acked Write 2 Sent Done Done false
27406 8621 Acked Write 8 Sent Done Done false
27407 8622 Acked Read 8 Sent Done Done false
27408 8623 Acked Read 7 Sent Done Done false
27409 8624 Acked Write 7 Sent Done Done false
27410 8625 Acked Write 4 Sent Done Done false
27411 8626 Acked Write 3 Sent Done Done false
27412 8627 Acked Write 4 Sent Done Done false
27413 8628 Acked Write 5 Sent Done Done false
27414 8629 Acked Flush 0 Sent Done Done false
27415 8630 Acked Write 2 Sent Done Done false
27416 8631 Acked Flush 0 Sent Done Done false
27417 8632 Acked Read 3 Sent Done Done false
27418 8633 Acked Read 2 Sent Done Done false
27419 8634 Acked Flush 0 Sent Done Done false
27420 8635 Acked Write 3 Sent Done Done false
27421 8636 Acked Write 6 Sent Done Done false
27422 8637 Acked Read 5 Sent Done Done false
27423 8638 Acked Read 4 Sent Done Done false
27424 8639 Acked Read 3 Sent Done Done false
27425 8640 Acked Write 9 Sent Done Done false
27426 8641 Acked Read 8 Sent Done Done false
27427 8642 Acked Write 2 Sent Done Done false
27428 8643 Acked Read 7 Sent Done Done false
27429 8644 Acked Read 8 Sent Done Done false
27430 8645 Acked Flush 0 Sent Done Done false
27431 8646 Acked Read 6 Sent Done Done false
27432 8647 Acked Read 8 Sent Done Done false
27433 8648 Acked Read 9 Sent Done Done false
27434 8649 Acked Write 5 Sent Done Done false
27435 8650 Acked Read 1 Sent Done Done false
27436 8651 Acked Write 6 Sent Done Done false
27437 8652 Acked Write 3 Sent Done Done false
27438 8653 Acked Read 4 Sent Done Done false
27439 8654 Acked Read 6 Sent Done Done false
27440 8655 Acked Write 3 Sent Done Done false
27441 8656 Acked Write 1 Sent Done Done false
27442 8657 Acked Read 2 Sent Done Done false
27443 8658 Acked Read 3 Sent Done Done false
27444 8659 Acked Write 4 Sent Done Done false
27445 8660 Acked Read 8 Sent Done Done false
27446 8661 Acked Write 1 Sent Done Done false
27447 8662 Acked Write 9 Sent Done Done false
27448 8663 Acked Write 9 Sent Done Done false
27449 8664 Acked Write 10 Sent Done Done false
27450 8665 Acked Flush 0 Sent Done Done false
27451 8666 Acked Read 6 Sent Done Done false
27452 8667 Acked Read 10 Sent Done Done false
27453 8668 Acked Write 2 Sent Done Done false
27454 8669 Acked Read 7 Sent Done Done false
27455 8670 Acked Write 3 Sent Done Done false
27456 8671 Acked Read 1 Sent Done Done false
27457 8672 Acked Write 5 Sent Done Done false
27458 8673 Acked Write 1 Sent Done Done false
27459 8674 Acked Write 10 Sent Done Done false
27460 8675 Acked Flush 0 Sent Done Done false
27461 8676 Acked Read 2 Sent Done Done false
27462 8677 Acked Flush 0 Sent Done Done false
27463 8678 Acked Write 3 Sent Done Done false
27464 8679 Acked Read 9 Sent Done Done false
27465 8680 Acked Read 3 Sent Done Done false
27466 8681 Acked Read 10 Sent Done Done false
27467 8682 Acked Read 3 Sent Done Done false
27468 8683 Acked Read 5 Sent Done Done false
27469 8684 Acked Write 7 Sent Done Done false
27470 8685 Acked Read 3 Sent Done Done false
27471 8686 Acked Read 7 Sent Done Done false
27472 8687 Acked Read 4 Sent Done Done false
27473 8688 Acked Read 6 Sent Done Done false
27474 8689 Acked Write 8 Sent Done Done false
27475 8690 Acked Read 9 Sent Done Done false
27476 8691 Acked Read 5 Sent Done Done false
27477 8692 Acked Read 10 Sent Done Done false
27478 8693 Acked Write 7 Sent Done Done false
27479 8694 Acked Read 3 Sent Done Done false
27480 8695 Acked Read 4 Sent Done Done false
27481 8696 Acked Write 8 Sent Done Done false
27482 8697 Acked Write 2 Sent Done Done false
27483 8698 Acked Read 2 Sent Done Done false
27484 8699 Acked Write 1 Sent Done Done false
27485 8700 Acked Flush 0 Sent Done Done false
27486 8701 Acked Read 5 Sent Done Done false
27487 8702 Acked Read 2 Sent Done Done false
27488 8703 Acked Read 5 Sent Done Done false
27489 8704 Acked Write 4 Sent Done Done false
27490 8705 Acked Write 3 Sent Done Done false
27491 8706 Acked Read 10 Sent Done Done false
27492 8707 Acked Flush 0 Sent Done Done false
27493 8708 Acked Write 5 Sent Done Done false
27494 8709 Acked Read 7 Sent Done Done false
27495 8710 Acked Write 3 Sent Done Done false
27496 8711 Acked Read 10 Sent Done Done false
27497 8712 Acked Write 10 Sent Done Done false
27498 8713 Acked Read 1 Sent Done Done false
27499 8714 Acked Read 10 Sent Done Done false
27500 8715 Acked Read 10 Sent Done Done false
27501 8716 Acked Write 8 Sent Done Done false
27502 8717 Acked Read 10 Sent Done Done false
27503 8718 Acked Read 10 Sent Done Done false
27504 8719 Acked Read 6 Sent Done Done false
27505 8720 Acked Read 7 Sent Done Done false
27506 8721 Acked Read 8 Sent Done Done false
27507 8722 Acked Read 2 Sent Done Done false
27508 8723 Acked Read 8 Sent Done Done false
27509 8724 Acked Read 3 Sent Done Done false
27510 8725 Acked Read 8 Sent Done Done false
27511 8726 Acked Flush 0 Sent Done Done false
27512 8727 Acked Read 10 Sent Done Done false
27513 8728 Acked Read 5 Sent Done Done false
27514 8729 Acked Read 9 Sent Done Done false
27515 8730 Acked Read 5 Sent Done Done false
27516 8731 Acked Read 10 Sent Done Done false
27517 8732 Acked Read 7 Sent Done Done false
27518 8733 Acked Write 9 Sent Done Done false
27519 8734 Acked Read 3 Sent Done Done false
27520 8735 Acked Write 6 Sent Done Done false
27521 8736 Acked Read 2 Sent Done Done false
27522 8737 Acked Read 2 Sent Done Done false
27523 8738 Acked Write 1 Sent Done Done false
27524 8739 Acked Read 1 Sent Done Done false
27525 8740 Acked Write 9 Sent Done Done false
27526 8741 Acked Write 1 Sent Done Done false
27527 8742 Acked Flush 0 Sent Done Done false
27528 8743 Acked Read 10 Sent Done Done false
27529 8744 Acked Read 6 Sent Done Done false
27530 8745 Acked Read 10 Sent Done Done false
27531 8746 Acked Write 7 Sent Done Done false
27532 8747 Acked Read 4 Sent Done Done false
27533 8748 Acked Write 10 Sent Done Done false
27534 8749 Acked Read 8 Sent Done Done false
27535 8750 Acked Read 4 Sent Done Done false
27536 8751 Acked Read 10 Sent Done Done false
27537 8752 Acked Write 5 Sent Done Done false
27538 8753 Acked Read 1 Sent Done Done false
27539 8754 Acked Write 4 Sent Done Done false
27540 8755 Acked Read 5 Sent Done Done false
27541 8756 Acked Flush 0 Sent Done Done false
27542 8757 Acked Read 10 Sent Done Done false
27543 8758 Acked Read 7 Sent Done Done false
27544 8759 Acked Read 7 Sent Done Done false
27545 8760 Acked Read 1 Sent Done Done false
27546 8761 Acked Write 5 Sent Done Done false
27547 8762 Acked Read 8 Sent Done Done false
27548 8763 Acked Read 1 Sent Done Done false
27549 8764 Acked Flush 0 Sent Done Done false
27550 8765 Acked Read 7 Sent Done Done false
27551 8766 Acked Read 1 Sent Done Done false
27552 8767 Acked Read 8 Sent Done Done false
27553 8768 Acked Read 8 Sent Done Done false
27554 8769 Acked Read 2 Sent Done Done false
27555 8770 Acked Write 8 Sent Done Done false
27556 8771 Acked Read 5 Sent Done Done false
27557 8772 Acked Read 1 Sent Done Done false
27558 8773 Acked Read 9 Sent Done Done false
27559 8774 Acked Write 6 Sent Done Done false
27560 8775 Acked Write 5 Sent Done Done false
27561 8776 Acked Write 5 Sent Done Done false
27562 8777 Acked Read 1 Sent Done Done false
27563 8778 Acked Write 1 Sent Done Done false
27564 8779 Acked Write 10 Sent Done Done false
27565 8780 Acked Read 8 Sent Done Done false
27566 8781 Acked Write 8 Sent Done Done false
27567 8782 Acked Write 9 Sent Done Done false
27568 8783 Acked Write 1 Sent Done Done false
27569 8784 Acked Write 5 Sent Done Done false
27570 8785 Acked Write 9 Sent Done Done false
27571 8786 Acked Write 9 Sent Done Done false
27572 8787 Acked Read 2 Sent Done Done false
27573 8788 Acked Write 6 Sent Done Done false
27574 8789 Acked Read 7 Sent Done Done false
27575 8790 Acked Read 10 Sent Done Done false
27576 8791 Acked Read 8 Sent Done Done false
27577 8792 Acked Write 1 Sent Done Done false
27578 8793 Acked Read 8 Sent Done Done false
27579 8794 Acked Write 7 Sent Done Done false
27580 8795 Acked Read 1 Sent Done Done false
27581 8796 Acked Flush 0 Sent Done Done false
27582 8797 Acked Write 10 Sent Done Done false
27583 8798 Acked Write 4 Sent Done Done false
27584 8799 Acked Write 4 Sent Done Done false
27585 8800 Acked Write 4 Sent Done Done false
27586 8801 Acked Write 7 Sent Done Done false
27587 8802 Acked Read 3 Sent Done Done false
27588 8803 Acked Read 5 Sent Done Done false
27589 8804 Acked Write 7 Sent Done Done false
27590 8805 Acked Read 1 Sent Done Done false
27591 8806 Acked Flush 0 Sent Done Done false
27592 8807 Acked Read 4 Sent Done Done false
27593 8808 Acked Read 6 Sent Done Done false
27594 8809 Acked Write 3 Sent Done Done false
27595 8810 Acked Write 7 Sent Done Done false
27596 8811 Acked Read 4 Sent Done Done false
27597 8812 Acked Write 1 Sent Done Done false
27598 8813 Acked Write 1 Sent Done Done false
27599 8814 Acked Read 6 Sent Done Done false
27600 8815 Acked Read 9 Sent Done Done false
27601 8816 Acked Read 7 Sent Done Done false
27602 8817 Acked Read 5 Sent Done Done false
27603 8818 Acked Read 7 Sent Done Done false
27604 8819 Acked Write 10 Sent Done Done false
27605 8820 Acked Write 2 Sent Done Done false
27606 8821 Acked Read 9 Sent Done Done false
27607 8822 Acked Read 2 Sent Done Done false
27608 8823 Acked Read 8 Sent Done Done false
27609 8824 Acked Read 2 Sent Done Done false
27610 8825 Acked Write 8 Sent Done Done false
27611 8826 Acked Read 7 Sent Done Done false
27612 8827 Acked Read 4 Sent Done Done false
27613 8828 Acked Write 9 Sent Done Done false
27614 8829 Acked Write 2 Sent Done Done false
27615 8830 Acked Flush 0 Sent Done Done false
27616 8831 Acked Write 1 Sent Done Done false
27617 8832 Acked Read 4 Sent Done Done false
27618 8833 Acked Write 9 Sent Done Done false
27619 8834 Acked Read 3 Sent Done Done false
27620 8835 Acked Read 1 Sent Done Done false
27621 8836 Acked Write 7 Sent Done Done false
27622 8837 Acked Write 8 Sent Done Done false
27623 8838 Acked Write 7 Sent Done Done false
27624 8839 Acked Read 8 Sent Done Done false
27625 8840 Acked Read 7 Sent Done Done false
27626 8841 Acked Write 10 Sent Done Done false
27627 8842 Acked Read 6 Sent Done Done false
27628 8843 Acked Read 8 Sent Done Done false
27629 8844 Acked Read 9 Sent Done Done false
27630 8845 Acked Write 1 Sent Done Done false
27631 8846 Acked Read 6 Sent Done Done false
27632 8847 Acked Write 8 Sent Done Done false
27633 8848 Acked Flush 0 Sent Done Done false
27634 STATES DS:0 DS:1 DS:2 TOTAL
27635 Sent 302 0 0 302
27636 Done 0 302 302 604
27637 Skipped 0 0 0 0
27638 Error 0 0 0 0
27639 Last Flush: 8546 8848 8848
27640 Downstairs last five completed: 8546 8545 8544 8543 8542
27641 Upstairs last five completed: 8848 8847 8846 8845 8844
27642 CLIENT: Up:0 ds:302 act:2
27643 ----------------------------------------------------------------
27644 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27645 States: Offline Active Active
27646 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27647 8547 Acked Read 10 Sent Done Done false
27648 8548 Acked Read 6 Sent Done Done false
27649 8549 Acked Write 1 Sent Done Done false
27650 8550 Acked Write 2 Sent Done Done false
27651 8551 Acked Read 2 Sent Done Done false
27652 8552 Acked Write 9 Sent Done Done false
27653 8553 Acked Read 3 Sent Done Done false
27654 8554 Acked Read 3 Sent Done Done false
27655 8555 Acked Read 5 Sent Done Done false
27656 8556 Acked Read 7 Sent Done Done false
27657 8557 Acked Read 8 Sent Done Done false
27658 8558 Acked Write 2 Sent Done Done false
27659 8559 Acked Write 9 Sent Done Done false
27660 8560 Acked Read 9 Sent Done Done false
27661 8561 Acked Write 1 Sent Done Done false
27662 8562 Acked Write 1 Sent Done Done false
27663 8563 Acked Write 1 Sent Done Done false
27664 8564 Acked Write 10 Sent Done Done false
27665 8565 Acked Read 7 Sent Done Done false
27666 8566 Acked Write 5 Sent Done Done false
27667 8567 Acked Write 1 Sent Done Done false
27668 8568 Acked Write 3 Sent Done Done false
27669 8569 Acked Read 7 Sent Done Done false
27670 8570 Acked Read 5 Sent Done Done false
27671 8571 Acked Read 3 Sent Done Done false
27672 8572 Acked Read 1 Sent Done Done false
27673 8573 Acked Write 8 Sent Done Done false
27674 8574 Acked Read 10 Sent Done Done false
27675 8575 Acked Write 8 Sent Done Done false
27676 8576 Acked Read 1 Sent Done Done false
27677 8577 Acked Read 9 Sent Done Done false
27678 8578 Acked Write 1 Sent Done Done false
27679 8579 Acked Write 4 Sent Done Done false
27680 8580 Acked Flush 0 Sent Done Done false
27681 8581 Acked Read 9 Sent Done Done false
27682 8582 Acked Write 3 Sent Done Done false
27683 8583 Acked Read 1 Sent Done Done false
27684 8584 Acked Write 7 Sent Done Done false
27685 8585 Acked Read 9 Sent Done Done false
27686 8586 Acked Read 7 Sent Done Done false
27687 8587 Acked Read 10 Sent Done Done false
27688 8588 Acked Read 8 Sent Done Done false
27689 8589 Acked Read 6 Sent Done Done false
27690 8590 Acked Write 2 Sent Done Done false
27691 8591 Acked Write 4 Sent Done Done false
27692 8592 Acked Write 2 Sent Done Done false
27693 8593 Acked Write 7 Sent Done Done false
27694 8594 Acked Flush 0 Sent Done Done false
27695 8595 Acked Write 10 Sent Done Done false
27696 8596 Acked Write 6 Sent Done Done false
27697 8597 Acked Read 7 Sent Done Done false
27698 8598 Acked Read 5 Sent Done Done false
27699 8599 Acked Read 10 Sent Done Done false
27700 8600 Acked Flush 0 Sent Done Done false
27701 8601 Acked Read 8 Sent Done Done false
27702 8602 Acked Read 9 Sent Done Done false
27703 8603 Acked Write 4 Sent Done Done false
27704 8604 Acked Read 9 Sent Done Done false
27705 8605 Acked Write 2 Sent Done Done false
27706 8606 Acked Read 6 Sent Done Done false
27707 8607 Acked Write 2 Sent Done Done false
27708 8608 Acked Write 4 Sent Done Done false
27709 8609 Acked Read 6 Sent Done Done false
27710 8610 Acked Read 9 Sent Done Done false
27711 8611 Acked Flush 0 Sent Done Done false
27712 8612 Acked Write 1 Sent Done Done false
27713 8613 Acked Read 7 Sent Done Done false
27714 8614 Acked Flush 0 Sent Done Done false
27715 8615 Acked Write 9 Sent Done Done false
27716 8616 Acked Write 7 Sent Done Done false
27717 8617 Acked Read 8 Sent Done Done false
27718 8618 Acked Read 4 Sent Done Done false
27719 8619 Acked Write 8 Sent Done Done false
27720 8620 Acked Write 2 Sent Done Done false
27721 8621 Acked Write 8 Sent Done Done false
27722 8622 Acked Read 8 Sent Done Done false
27723 8623 Acked Read 7 Sent Done Done false
27724 8624 Acked Write 7 Sent Done Done false
27725 8625 Acked Write 4 Sent Done Done false
27726 8626 Acked Write 3 Sent Done Done false
27727 8627 Acked Write 4 Sent Done Done false
27728 8628 Acked Write 5 Sent Done Done false
27729 8629 Acked Flush 0 Sent Done Done false
27730 8630 Acked Write 2 Sent Done Done false
27731 8631 Acked Flush 0 Sent Done Done false
27732 8632 Acked Read 3 Sent Done Done false
27733 8633 Acked Read 2 Sent Done Done false
27734 8634 Acked Flush 0 Sent Done Done false
27735 8635 Acked Write 3 Sent Done Done false
27736 8636 Acked Write 6 Sent Done Done false
27737 8637 Acked Read 5 Sent Done Done false
27738 8638 Acked Read 4 Sent Done Done false
27739 8639 Acked Read 3 Sent Done Done false
27740 8640 Acked Write 9 Sent Done Done false
27741 8641 Acked Read 8 Sent Done Done false
27742 8642 Acked Write 2 Sent Done Done false
27743 8643 Acked Read 7 Sent Done Done false
27744 8644 Acked Read 8 Sent Done Done false
27745 8645 Acked Flush 0 Sent Done Done false
27746 8646 Acked Read 6 Sent Done Done false
27747 8647 Acked Read 8 Sent Done Done false
27748 8648 Acked Read 9 Sent Done Done false
27749 8649 Acked Write 5 Sent Done Done false
27750 8650 Acked Read 1 Sent Done Done false
27751 8651 Acked Write 6 Sent Done Done false
27752 8652 Acked Write 3 Sent Done Done false
27753 8653 Acked Read 4 Sent Done Done false
27754 8654 Acked Read 6 Sent Done Done false
27755 8655 Acked Write 3 Sent Done Done false
27756 8656 Acked Write 1 Sent Done Done false
27757 8657 Acked Read 2 Sent Done Done false
27758 8658 Acked Read 3 Sent Done Done false
27759 8659 Acked Write 4 Sent Done Done false
27760 8660 Acked Read 8 Sent Done Done false
27761 8661 Acked Write 1 Sent Done Done false
27762 8662 Acked Write 9 Sent Done Done false
27763 8663 Acked Write 9 Sent Done Done false
27764 8664 Acked Write 10 Sent Done Done false
27765 8665 Acked Flush 0 Sent Done Done false
27766 8666 Acked Read 6 Sent Done Done false
27767 8667 Acked Read 10 Sent Done Done false
27768 8668 Acked Write 2 Sent Done Done false
27769 8669 Acked Read 7 Sent Done Done false
27770 8670 Acked Write 3 Sent Done Done false
27771 8671 Acked Read 1 Sent Done Done false
27772 8672 Acked Write 5 Sent Done Done false
27773 8673 Acked Write 1 Sent Done Done false
27774 8674 Acked Write 10 Sent Done Done false
27775 8675 Acked Flush 0 Sent Done Done false
27776 8676 Acked Read 2 Sent Done Done false
27777 8677 Acked Flush 0 Sent Done Done false
27778 8678 Acked Write 3 Sent Done Done false
27779 8679 Acked Read 9 Sent Done Done false
27780 8680 Acked Read 3 Sent Done Done false
27781 8681 Acked Read 10 Sent Done Done false
27782 8682 Acked Read 3 Sent Done Done false
27783 8683 Acked Read 5 Sent Done Done false
27784 8684 Acked Write 7 Sent Done Done false
27785 8685 Acked Read 3 Sent Done Done false
27786 8686 Acked Read 7 Sent Done Done false
27787 8687 Acked Read 4 Sent Done Done false
27788 8688 Acked Read 6 Sent Done Done false
27789 8689 Acked Write 8 Sent Done Done false
27790 8690 Acked Read 9 Sent Done Done false
27791 8691 Acked Read 5 Sent Done Done false
27792 8692 Acked Read 10 Sent Done Done false
27793 8693 Acked Write 7 Sent Done Done false
27794 8694 Acked Read 3 Sent Done Done false
27795 8695 Acked Read 4 Sent Done Done false
27796 8696 Acked Write 8 Sent Done Done false
27797 8697 Acked Write 2 Sent Done Done false
27798 8698 Acked Read 2 Sent Done Done false
27799 8699 Acked Write 1 Sent Done Done false
27800 8700 Acked Flush 0 Sent Done Done false
27801 8701 Acked Read 5 Sent Done Done false
27802 8702 Acked Read 2 Sent Done Done false
27803 8703 Acked Read 5 Sent Done Done false
27804 8704 Acked Write 4 Sent Done Done false
27805 8705 Acked Write 3 Sent Done Done false
27806 8706 Acked Read 10 Sent Done Done false
27807 8707 Acked Flush 0 Sent Done Done false
27808 8708 Acked Write 5 Sent Done Done false
27809 8709 Acked Read 7 Sent Done Done false
27810 8710 Acked Write 3 Sent Done Done false
27811 8711 Acked Read 10 Sent Done Done false
27812 8712 Acked Write 10 Sent Done Done false
27813 8713 Acked Read 1 Sent Done Done false
27814 8714 Acked Read 10 Sent Done Done false
27815 8715 Acked Read 10 Sent Done Done false
27816 8716 Acked Write 8 Sent Done Done false
27817 8717 Acked Read 10 Sent Done Done false
27818 8718 Acked Read 10 Sent Done Done false
27819 8719 Acked Read 6 Sent Done Done false
27820 8720 Acked Read 7 Sent Done Done false
27821 8721 Acked Read 8 Sent Done Done false
27822 8722 Acked Read 2 Sent Done Done false
27823 8723 Acked Read 8 Sent Done Done false
27824 8724 Acked Read 3 Sent Done Done false
27825 8725 Acked Read 8 Sent Done Done false
27826 8726 Acked Flush 0 Sent Done Done false
27827 8727 Acked Read 10 Sent Done Done false
27828 8728 Acked Read 5 Sent Done Done false
27829 8729 Acked Read 9 Sent Done Done false
27830 8730 Acked Read 5 Sent Done Done false
27831 8731 Acked Read 10 Sent Done Done false
27832 8732 Acked Read 7 Sent Done Done false
27833 8733 Acked Write 9 Sent Done Done false
27834 8734 Acked Read 3 Sent Done Done false
27835 8735 Acked Write 6 Sent Done Done false
27836 8736 Acked Read 2 Sent Done Done false
27837 8737 Acked Read 2 Sent Done Done false
27838 8738 Acked Write 1 Sent Done Done false
27839 8739 Acked Read 1 Sent Done Done false
27840 8740 Acked Write 9 Sent Done Done false
27841 8741 Acked Write 1 Sent Done Done false
27842 8742 Acked Flush 0 Sent Done Done false
27843 8743 Acked Read 10 Sent Done Done false
27844 8744 Acked Read 6 Sent Done Done false
27845 8745 Acked Read 10 Sent Done Done false
27846 8746 Acked Write 7 Sent Done Done false
27847 8747 Acked Read 4 Sent Done Done false
27848 8748 Acked Write 10 Sent Done Done false
27849 8749 Acked Read 8 Sent Done Done false
27850 8750 Acked Read 4 Sent Done Done false
27851 8751 Acked Read 10 Sent Done Done false
27852 8752 Acked Write 5 Sent Done Done false
27853 8753 Acked Read 1 Sent Done Done false
27854 8754 Acked Write 4 Sent Done Done false
27855 8755 Acked Read 5 Sent Done Done false
27856 8756 Acked Flush 0 Sent Done Done false
27857 8757 Acked Read 10 Sent Done Done false
27858 8758 Acked Read 7 Sent Done Done false
27859 8759 Acked Read 7 Sent Done Done false
27860 8760 Acked Read 1 Sent Done Done false
27861 8761 Acked Write 5 Sent Done Done false
27862 8762 Acked Read 8 Sent Done Done false
27863 8763 Acked Read 1 Sent Done Done false
27864 8764 Acked Flush 0 Sent Done Done false
27865 8765 Acked Read 7 Sent Done Done false
27866 8766 Acked Read 1 Sent Done Done false
27867 8767 Acked Read 8 Sent Done Done false
27868 8768 Acked Read 8 Sent Done Done false
27869 8769 Acked Read 2 Sent Done Done false
27870 8770 Acked Write 8 Sent Done Done false
27871 8771 Acked Read 5 Sent Done Done false
27872 8772 Acked Read 1 Sent Done Done false
27873 8773 Acked Read 9 Sent Done Done false
27874 8774 Acked Write 6 Sent Done Done false
27875 8775 Acked Write 5 Sent Done Done false
27876 8776 Acked Write 5 Sent Done Done false
27877 8777 Acked Read 1 Sent Done Done false
27878 8778 Acked Write 1 Sent Done Done false
27879 8779 Acked Write 10 Sent Done Done false
27880 8780 Acked Read 8 Sent Done Done false
27881 8781 Acked Write 8 Sent Done Done false
27882 8782 Acked Write 9 Sent Done Done false
27883 8783 Acked Write 1 Sent Done Done false
27884 8784 Acked Write 5 Sent Done Done false
27885 8785 Acked Write 9 Sent Done Done false
27886 8786 Acked Write 9 Sent Done Done false
27887 8787 Acked Read 2 Sent Done Done false
27888 8788 Acked Write 6 Sent Done Done false
27889 8789 Acked Read 7 Sent Done Done false
27890 8790 Acked Read 10 Sent Done Done false
27891 8791 Acked Read 8 Sent Done Done false
27892 8792 Acked Write 1 Sent Done Done false
27893 8793 Acked Read 8 Sent Done Done false
27894 8794 Acked Write 7 Sent Done Done false
27895 8795 Acked Read 1 Sent Done Done false
27896 8796 Acked Flush 0 Sent Done Done false
27897 8797 Acked Write 10 Sent Done Done false
27898 8798 Acked Write 4 Sent Done Done false
27899 8799 Acked Write 4 Sent Done Done false
27900 8800 Acked Write 4 Sent Done Done false
27901 8801 Acked Write 7 Sent Done Done false
27902 8802 Acked Read 3 Sent Done Done false
27903 8803 Acked Read 5 Sent Done Done false
27904 8804 Acked Write 7 Sent Done Done false
27905 8805 Acked Read 1 Sent Done Done false
27906 8806 Acked Flush 0 Sent Done Done false
27907 8807 Acked Read 4 Sent Done Done false
27908 8808 Acked Read 6 Sent Done Done false
27909 8809 Acked Write 3 Sent Done Done false
27910 8810 Acked Write 7 Sent Done Done false
27911 8811 Acked Read 4 Sent Done Done false
27912 8812 Acked Write 1 Sent Done Done false
27913 8813 Acked Write 1 Sent Done Done false
27914 8814 Acked Read 6 Sent Done Done false
27915 8815 Acked Read 9 Sent Done Done false
27916 8816 Acked Read 7 Sent Done Done false
27917 8817 Acked Read 5 Sent Done Done false
27918 8818 Acked Read 7 Sent Done Done false
27919 8819 Acked Write 10 Sent Done Done false
27920 8820 Acked Write 2 Sent Done Done false
27921 8821 Acked Read 9 Sent Done Done false
27922 8822 Acked Read 2 Sent Done Done false
27923 8823 Acked Read 8 Sent Done Done false
27924 8824 Acked Read 2 Sent Done Done false
27925 8825 Acked Write 8 Sent Done Done false
27926 8826 Acked Read 7 Sent Done Done false
27927 8827 Acked Read 4 Sent Done Done false
27928 8828 Acked Write 9 Sent Done Done false
27929 8829 Acked Write 2 Sent Done Done false
27930 8830 Acked Flush 0 Sent Done Done false
27931 8831 Acked Write 1 Sent Done Done false
27932 8832 Acked Read 4 Sent Done Done false
27933 8833 Acked Write 9 Sent Done Done false
27934 8834 Acked Read 3 Sent Done Done false
27935 8835 Acked Read 1 Sent Done Done false
27936 8836 Acked Write 7 Sent Done Done false
27937 8837 Acked Write 8 Sent Done Done false
27938 8838 Acked Write 7 Sent Done Done false
27939 8839 Acked Read 8 Sent Done Done false
27940 8840 Acked Read 7 Sent Done Done false
27941 8841 Acked Write 10 Sent Done Done false
27942 8842 Acked Read 6 Sent Done Done false
27943 8843 Acked Read 8 Sent Done Done false
27944 8844 Acked Read 9 Sent Done Done false
27945 8845 Acked Write 1 Sent Done Done false
27946 8846 Acked Read 6 Sent Done Done false
27947 8847 Acked Write 8 Sent Done Done false
27948 8848 Acked Flush 0 Sent Done Done false
27949 STATES DS:0 DS:1 DS:2 TOTAL
27950 Sent 302 0 0 302
27951 Done 0 302 302 604
27952 Skipped 0 0 0 0
27953 Error 0 0 0 0
27954 Last Flush: 8546 8848 8848
27955 Downstairs last five completed: 8546 8545 8544 8543 8542
27956 Upstairs last five completed: 8848 8847 8846 8845 8844
27957 CLIENT: Up:0 ds:302 act:2
279582025-01-14T16:16:30.037ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
279592025-01-14T16:16:30.037ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:37038) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
279602025-01-14T16:16:30.038ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
279612025-01-14T16:16:30.039ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
279622025-01-14T16:16:30.039ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
279632025-01-14T16:16:30.039ZINFOcrucible: send last flush ID to this DS: 8546 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
279642025-01-14T16:16:30.039ZINFOcrucible: Replied this last flush ID: 8546 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
279652025-01-14T16:16:30.039ZINFOcrucible: [0] client re-new 302 jobs since flush 8546 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
279662025-01-14T16:16:30.039ZINFOcrucible: [0] Replayed 302 jobs since flush: 8546 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
27967 ----------------------------------------------------------------
27968 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
27969 Downstairs last five completed: 8848 8847 8846 8845 8844
27970 Upstairs last five completed: 8848 8847 8846 8845 8844
27971 CLIENT: Up:0 ds:0 act:3
27972 Replay: All jobs finished
27973 Main task has work to do, go find it
27974 got dsc Stop(1)
27975 stop 1
27976 [8820] Got stop action so:false kr:false
27977 Jan 14 16:16:30.290 INFO request completed, latency_us: 313, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 1e8ac66b-0e49-4562-ad54-c49cd9ce2956, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
279782025-01-14T16:16:30.290ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
279792025-01-14T16:16:30.290ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
279802025-01-14T16:16:30.290ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
279812025-01-14T16:16:30.290ZINFOcrucible: client task is exiting = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
27982 Jan 14 16:16:30.291 INFO request completed, latency_us: 181, response_code: 200, uri: /state/cid/1, method: GET, req_id: 315cf8e4-2682-4749-8dfa-2b13fa3afa6d, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
27983 [8820] Exited with: ExitStatus(unix_wait_status(9))
27984 [8820][1] reports Exit
27985 Jan 14 16:16:34.293 INFO request completed, latency_us: 195, response_code: 200, uri: /state/cid/1, method: GET, req_id: 025bdf42-9d5a-48de-9ab1-5095a8d2f94c, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
27986 001/300 Read block 2282 len 20480
27987 002/300 Read block 4228 len 24576
27988 003/300 Read block 3498 len 16384
27989 004/300 Flush
27990 005/300 Write block 4006 len 4096 data: 8
27991 006/300 Write block 1125 len 12288 data: 5 5 5
27992 007/300 Write block 1730 len 40960 data: 7 7 5 6 6 6 5 6 6 6
27993 008/300 Read block 4140 len 20480
27994 009/300 Read block 465 len 20480
27995 010/300 Write block 3876 len 24576 data: 4 5 5 6 6 6
27996 011/300 Read block 2753 len 16384
27997 012/300 Write block 3767 len 16384 data: 6 5 7 7
27998 013/300 Write block 2053 len 16384 data: 7 7 7 7
27999 014/300 Read block 394 len 36864
28000 015/300 Read block 2930 len 32768
28001 016/300 Read block 883 len 40960
28002 017/300 Read block 4795 len 12288
28003 018/300 Read block 1889 len 40960
28004 019/300 Read block 762 len 32768
28005 020/300 Read block 4079 len 12288
28006 021/300 Read block 2903 len 40960
28007 022/300 Write block 507 len 8192 data: 5 5
28008 023/300 Read block 256 len 24576
28009 024/300 Read block 2304 len 12288
28010 025/300 Read block 1229 len 8192
28011 026/300 Read block 3136 len 40960
28012 027/300 Write block 2205 len 20480 data: 7 6 6 6 6
28013 028/300 Write block 3564 len 16384 data: 7 7 7 9
28014 029/300 Write block 4079 len 8192 data: 4 4
28015 030/300 Write block 4002 len 40960 data: 11 8 8 8 9 10 9 7 9 8
28016 031/300 Read block 3908 len 28672
28017 032/300 Read block 1533 len 20480
28018 033/300 Read block 784 len 8192
28019 034/300 Write block 922 len 20480 data: 5 5 6 7 8
28020 035/300 Read block 3850 len 40960
28021 036/300 Read block 129 len 28672
28022 037/300 Flush
28023 038/300 Read block 744 len 20480
28024 039/300 Write block 2992 len 4096 data: 6
28025 040/300 Write block 687 len 4096 data: 9
28026 041/300 Read block 3357 len 16384
28027 042/300 Write block 1460 len 16384 data: 5 3 2 4
28028 043/300 Flush
28029 044/300 Read block 3065 len 40960
28030 045/300 Write block 477 len 28672 data: 6 6 6 7 7 6 7
28031 046/300 Read block 3252 len 4096
28032 047/300 Write block 1308 len 16384 data: 4 3 3 3
28033 048/300 Write block 833 len 32768 data: 2 3 3 5 7 7 7 7
28034 049/300 Write block 714 len 32768 data: 4 4 4 4 3 3 2 3
28035 050/300 Write block 489 len 12288 data: 3 4 5
28036 051/300 Read block 2022 len 4096
28037 052/300 Read block 506 len 24576
28038 053/300 Write block 168 len 20480 data: 7 6 6 6 5
28039 054/300 Write block 3852 len 24576 data: 6 5 3 3 3 2
28040 055/300 Write block 1041 len 16384 data: 4 2 3 3
28041 056/300 Flush
28042 057/300 Read block 2904 len 8192
28043 058/300 Read block 1797 len 20480
28044 059/300 Flush
28045 060/300 Read block 1108 len 40960
28046 061/300 Write block 1226 len 24576 data: 4 4 3 4 5 6
28047 062/300 Read block 332 len 12288
28048 063/300 Write block 1120 len 24576 data: 4 4 4 4 5 6
28049 064/300 Write block 3496 len 32768 data: 4 3 3 4 5 3 4 4
28050 065/300 Read block 4242 len 20480
28051 066/300 Read block 2052 len 20480
28052 067/300 Write block 2928 len 36864 data: 8 8 5 5 6 5 5 4 5
28053 068/300 Write block 2285 len 16384 data: 7 6 5 6
28054 069/300 Read block 2305 len 20480
28055 070/300 Read block 4052 len 24576
28056 071/300 Write block 1544 len 32768 data: 7 8 7 7 7 6 6 6
28057 072/300 Flush
28058 073/300 Write block 1784 len 24576 data: 4 4 4 4 3 3
28059 074/300 Flush
28060 075/300 Read block 1354 len 16384
28061 076/300 Write block 3730 len 40960 data: 4 3 2 3 4 5 6 5 5 6
28062 077/300 Read block 3615 len 8192
28063 078/300 Write block 1911 len 40960 data: 4 4 3 2 2 3 3 4 4 4
28064 079/300 Read block 4584 len 20480
28065 080/300 Write block 3689 len 24576 data: 7 6 6 6 6 5
28066 081/300 Read block 1608 len 16384
28067 082/300 Flush
28068 083/300 Read block 208 len 20480
28069 084/300 Write block 3253 len 4096 data: 5
28070 085/300 Read block 2099 len 16384
28071 086/300 Write block 3894 len 12288 data: 6 8 8
28072 087/300 Write block 4810 len 20480 data: 4 3 3 4 3
28073 088/300 Write block 2579 len 36864 data: 6 6 7 10 8 8 7 6 4
28074 089/300 Write block 4198 len 4096 data: 4
28075 090/300 Flush
28076 091/300 Write block 2345 len 40960 data: 5 5 4 5 4 3 2 3 3 4
28077 092/300 Write block 1945 len 32768 data: 5 3 3 6 6 7 9 8
28078 093/300 Write block 166 len 40960 data: 8 8 8 7 7 7 6 6 5 4
28079 094/300 Read block 2983 len 32768
28080 095/300 Read block 2016 len 4096
28081 096/300 Read block 2190 len 32768
28082 097/300 Read block 1729 len 16384
28083 098/300 Read block 1594 len 32768
28084 099/300 Write block 4189 len 4096 data: 5
28085 100/300 Write block 4159 len 40960 data: 9 8 8 8 8 9 5 7 4 5
28086 101/300 Write block 1343 len 20480 data: 3 3 4 4 4
28087 102/300 Read block 72 len 28672
28088 103/300 Read block 1682 len 12288
28089 104/300 Read block 3077 len 24576
28090 105/300 Read block 3211 len 36864
28091 106/300 Read block 1749 len 28672
28092 107/300 Write block 44 len 16384 data: 4 4 3 4
28093 108/300 Write block 2562 len 40960 data: 8 8 7 6 7 9 9 8 7 6
28094 109/300 Write block 3058 len 24576 data: 4 5 5 6 4 6
28095 110/300 Read block 2801 len 24576
28096 111/300 Write block 916 len 32768 data: 5 5 5 6 6 5 6 6
28097 112/300 Read block 97 len 40960
28098 113/300 Read block 4262 len 24576
28099 114/300 Read block 3658 len 28672
28100 115/300 Write block 258 len 4096 data: 6
28101 116/300 Write block 3542 len 16384 data: 3 2 2 3
28102 117/300 Flush
28103 118/300 Write block 2863 len 32768 data: 7 7 4 5 5 7 7 11
28104 119/300 Read block 2179 len 4096
28105 120/300 Write block 794 len 4096 data: 7
28106 121/300 Read block 3166 len 20480
28107 122/300 Write block 741 len 8192 data: 6 6
28108 123/300 Write block 4140 len 40960 data: 7 7 7 7 6 5 5 5 2 3
28109 124/300 Flush
28110 125/300 Read block 3321 len 4096
28111 126/300 Read block 3779 len 8192
28112 127/300 Flush
28113 128/300 Write block 4035 len 12288 data: 4 4 3
28114 129/300 Write block 3250 len 12288 data: 2 2 3
28115 130/300 Write block 2167 len 40960 data: 4 5 5 6 6 6 5 6 6 5
28116 131/300 Write block 2783 len 4096 data: 4
28117 132/300 Read block 707 len 8192
28118 133/300 Read block 4470 len 24576
28119 134/300 Write block 1197 len 12288 data: 3 5 5
28120 135/300 Read block 4980 len 12288
28121 136/300 Read block 4435 len 32768
28122 137/300 Write block 3917 len 12288 data: 7 5 3
28123 138/300 Read block 3161 len 16384
28124 139/300 Read block 4825 len 8192
28125 140/300 Flush
28126 141/300 Read block 4381 len 16384
28127 142/300 Read block 32 len 8192
28128 143/300 Read block 2443 len 12288
28129 144/300 Read block 3203 len 4096
28130 145/300 Write block 4933 len 24576 data: 6 4 4 4 4 4
28131 146/300 Write block 4673 len 28672 data: 5 4 5 5 6 6 6
28132 147/300 Write block 779 len 12288 data: 5 5 5
28133 148/300 Read block 1608 len 20480
28134 149/300 Write block 3388 len 20480 data: 5 7 7 8 9
28135 150/300 Read block 4645 len 12288
28136 151/300 Read block 1926 len 32768
28137 152/300 Read block 4228 len 40960
28138 153/300 Write block 2950 len 40960 data: 5 6 3 3 3 3 4 4 4 6
28139 154/300 Read block 263 len 32768
28140 155/300 Read block 650 len 40960
28141 156/300 Flush
28142 157/300 Write block 3022 len 40960 data: 5 6 10 10 8 6 7 8 7 8
28143 158/300 Write block 2141 len 28672 data: 3 2 2 2 3 3 3
28144 159/300 Read block 4855 len 32768
28145 160/300 Read block 2036 len 16384
28146 161/300 Flush
28147 162/300 Write block 1638 len 16384 data: 4 5 4 4
28148 163/300 Write block 834 len 16384 data: 4 4 6 8
28149 164/300 Write block 4373 len 32768 data: 7 7 6 5 5 3 4 3
28150 165/300 Write block 3377 len 40960 data: 3 3 3 3 3 5 4 5 5 5
28151 166/300 Write block 536 len 40960 data: 7 6 4 4 5 5 5 5 4 4
28152 167/300 Write block 696 len 32768 data: 4 7 7 7 7 7 6 6
28153 168/300 Read block 3053 len 40960
28154 169/300 Read block 4487 len 16384
28155 170/300 Write block 3624 len 16384 data: 6 7 7 6
28156 171/300 Read block 445 len 36864
28157 172/300 Read block 3294 len 12288
28158 173/300 Read block 3943 len 40960
28159 174/300 Read block 4352 len 24576
28160 175/300 Flush
28161 176/300 Read block 315 len 24576
28162 177/300 Read block 81 len 12288
28163 178/300 Read block 4429 len 28672
28164 179/300 Write block 359 len 24576 data: 6 6 6 5 6 4
28165 180/300 Read block 3974 len 12288
28166 181/300 Write block 2026 len 20480 data: 5 4 5 5 6
28167 182/300 Read block 1804 len 4096
28168 183/300 Read block 4785 len 16384
28169 184/300 Read block 2772 len 16384
28170 185/300 Read block 2391 len 24576
28171 186/300 Flush
28172 187/300 Write block 1993 len 16384 data: 12 9 10 12
28173 188/300 Flush
28174 189/300 Write block 3423 len 40960 data: 10 11 10 10 9 7 7 6 7 6
28175 190/300 Read block 4812 len 24576
28176 191/300 Flush
28177 192/300 Read block 4159 len 8192
28178 193/300 Read block 3879 len 12288
28179 194/300 Read block 58 len 32768
28180 195/300 Read block 4171 len 4096
28181 196/300 Write block 2942 len 4096 data: 4
28182 197/300 Write block 3892 len 40960 data: 5 4 7 9 9 7 7 7 7 5
28183 198/300 Write block 3832 len 20480 data: 8 8 7 6 7
28184 199/300 Read block 240 len 12288
28185 200/300 Read block 2976 len 36864
28186 201/300 Read block 3156 len 32768
28187 202/300 Write block 3267 len 12288 data: 5 6 5
28188 203/300 Write block 2375 len 28672 data: 5 7 8 8 7 6 6
28189 204/300 Write block 3825 len 28672 data: 6 5 4 5 6 7 8
28190 205/300 Read block 4601 len 40960
28191 206/300 Write block 2964 len 4096 data: 3
28192 207/300 Write block 553 len 24576 data: 6 5 6 6 6 6
28193 208/300 Write block 2586 len 12288 data: 7 5 3
28194 209/300 Write block 2469 len 12288 data: 5 6 5
28195 210/300 Write block 2680 len 32768 data: 5 6 6 6 5 4 3 4
28196 211/300 Read block 83 len 40960
28197 212/300 Read block 4822 len 12288
28198 213/300 Flush
28199 214/300 Read block 225 len 20480
28200 215/300 Write block 1636 len 4096 data: 3
28201 216/300 Read block 2615 len 20480
28202 217/300 Flush
28203 218/300 Read block 1895 len 8192
28204 219/300 Read block 808 len 28672
28205 220/300 Read block 4981 len 8192
28206 221/300 Read block 3268 len 36864
28207 222/300 Write block 743 len 4096 data: 6
28208 223/300 Read block 4071 len 32768
28209 224/300 Read block 4783 len 28672
28210 225/300 Read block 2120 len 16384
28211 226/300 Read block 436 len 12288
28212 227/300 Write block 2241 len 8192 data: 6 5
28213 228/300 Read block 3676 len 32768
28214 229/300 Read block 3619 len 32768
28215 230/300 Read block 1618 len 4096
28216 231/300 Flush
28217 232/300 Write block 4754 len 12288 data: 4 4 5
28218 233/300 Flush
28219 234/300 Read block 3027 len 24576
28220 235/300 Write block 4144 len 4096 data: 7
28221 236/300 Read block 1315 len 40960
28222 237/300 Read block 1771 len 20480
28223 238/300 Write block 2932 len 40960 data: 7 6 6 5 6 5 5 6 5 4
28224 239/300 Read block 4711 len 32768
28225 240/300 Read block 4402 len 12288
28226 241/300 Read block 274 len 4096
28227 242/300 Read block 4 len 32768
28228 243/300 Read block 3423 len 24576
28229 244/300 Read block 1816 len 24576
28230 245/300 Read block 1476 len 4096
28231 246/300 Read block 3795 len 32768
28232 247/300 Write block 2838 len 12288 data: 8 8 6
28233 248/300 Read block 1386 len 8192
28234 249/300 Write block 3565 len 40960 data: 8 8 10 6 7 7 6 5 7 9
28235 250/300 Read block 2172 len 24576
28236 251/300 Read block 3779 len 24576
28237 252/300 Write block 1462 len 12288 data: 3 5 5
28238 253/300 Read block 479 len 32768
28239 254/300 Read block 4008 len 24576
28240 255/300 Read block 2565 len 32768
28241 256/300 Flush
28242 257/300 Write block 604 len 28672 data: 4 4 4 4 4 6 6
28243 258/300 Read block 3437 len 12288
28244 259/300 Write block 1917 len 20480 data: 4 5 5 5 4
28245 260/300 Write block 4690 len 28672 data: 5 5 6 5 4 3 2
28246 261/300 Write block 2255 len 12288 data: 5 4 5
28247 262/300 Write block 1217 len 24576 data: 5 6 7 7 7 7
28248 263/300 Read block 42 len 32768
28249 264/300 Read block 1400 len 24576
28250 265/300 Write block 848 len 4096 data: 2
28251 266/300 Write block 430 len 32768 data: 4 3 3 3 3 4 4 5
28252 267/300 Read block 2921 len 36864
28253 268/300 Read block 2305 len 32768
28254 269/300 Read block 4753 len 36864
28255 270/300 Write block 2589 len 36864 data: 2 2 3 4 6 4 5 4 4
28256 271/300 Read block 3595 len 32768
28257 272/300 Write block 4713 len 20480 data: 5 4 4 5 6
28258 273/300 Read block 3715 len 28672
28259 274/300 Write block 2123 len 20480 data: 3 3 2 3 3
28260 275/300 Write block 18 len 36864 data: 6 6 6 6 8 8 9 9 9
28261 276/300 Write block 3269 len 24576 data: 6 5 4 6 5 5
28262 277/300 Flush
28263 278/300 Read block 2498 len 32768
28264 279/300 Write block 2324 len 16384 data: 8 8 5 5
28265 280/300 Read block 2129 len 12288
28266 281/300 Write block 2860 len 20480 data: 6 8 7 8 8
28267 282/300 Write block 2132 len 12288 data: 5 5 5
28268 283/300 Read block 2346 len 40960
28269 284/300 Write block 3741 len 28672 data: 7 7 5 5 4 4 3
28270 285/300 Flush
28271 286/300 Read block 2688 len 36864
28272 287/300 Read block 3143 len 32768
28273 288/300 Write block 3812 len 16384 data: 7 4 3 6
28274 289/300 Read block 3056 len 36864
28275 290/300 Write block 1356 len 24576 data: 8 7 6 6 4 4
28276 291/300 Read block 2543 len 12288
28277 292/300 Read block 4695 len 4096
28278 293/300 Write block 2215 len 24576 data: 5 6 7 6 6 4
28279 294/300 Read block 2875 len 16384
28280 295/300 Read block 4857 len 16384
28281 296/300 Read block 1785 len 36864
28282 297/300 Read block 2446 len 4096
28283 298/300 Read block 3131 len 12288
28284 299/300 Write block 1766 len 28672 data: 4 4 5 7 7 6 7
28285 300/300 Read block 3023 len 4096
28286 301/300 Write block 1946 len 24576 data: 4 4 7 7 8 10
28287 Main task has work to do, go find it
28288 got dsc Start(1)
28289 start 1
28290 Jan 14 16:16:34.533 INFO request completed, latency_us: 218, response_code: 204, uri: /start/cid/1, method: POST, req_id: 6e8427ea-1d4f-4a75-af5c-cb7e56660e07, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
28291 Replay: started 1, returned:Ok(())
28292 ----------------------------------------------------------------
28293 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
28294 States: Active Offline Active
28295 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28296 8849 Acked Read 5 Done Sent Done false
28297 8850 Acked Read 6 Done Sent Done false
28298 8851 Acked Read 4 Done Sent Done false
28299 8852 Acked Flush 0 Done Sent Done false
28300 8853 Acked Write 1 Done Sent Done false
28301 8854 Acked Write 3 Done Sent Done false
28302 8855 Acked Write 10 Done Sent Done false
28303 8856 Acked Read 5 Done Sent Done false
28304 8857 Acked Read 5 Done Sent Done false
28305 8858 Acked Write 6 Done Sent Done false
28306 8859 Acked Read 4 Done Sent Done false
28307 8860 Acked Write 4 Done Sent Done false
28308 8861 Acked Write 4 Done Sent Done false
28309 8862 Acked Read 9 Done Sent Done false
28310 8863 Acked Read 8 Done Sent Done false
28311 8864 Acked Read 10 Done Sent Done false
28312 8865 Acked Read 3 Done Sent Done false
28313 8866 Acked Read 10 Done Sent Done false
28314 8867 Acked Read 8 Done Sent Done false
28315 8868 Acked Read 3 Done Sent Done false
28316 8869 Acked Read 10 Done Sent Done false
28317 8870 Acked Write 2 Done Sent Done false
28318 8871 Acked Read 6 Done Sent Done false
28319 8872 Acked Read 3 Done Sent Done false
28320 8873 Acked Read 2 Done Sent Done false
28321 8874 Acked Read 10 Done Sent Done false
28322 8875 Acked Write 5 Done Sent Done false
28323 8876 Acked Write 4 Done Sent Done false
28324 8877 Acked Write 2 Done Sent Done false
28325 8878 Acked Write 10 Done Sent Done false
28326 8879 Acked Read 7 Done Sent Done false
28327 8880 Acked Read 5 Done Sent Done false
28328 8881 Acked Read 2 Done Sent Done false
28329 8882 Acked Write 5 Done Sent Done false
28330 8883 Acked Read 10 Done Sent Done false
28331 8884 Acked Read 7 Done Sent Done false
28332 8885 Acked Flush 0 Done Sent Done false
28333 8886 Acked Read 5 Done Sent Done false
28334 8887 Acked Write 1 Done Sent Done false
28335 8888 Acked Write 1 Done Sent Done false
28336 8889 Acked Read 4 Done Sent Done false
28337 8890 Acked Write 4 Done Sent Done false
28338 8891 Acked Flush 0 Done Sent Done false
28339 8892 Acked Read 10 Done Sent Done false
28340 8893 Acked Write 7 Done Sent Done false
28341 8894 Acked Read 1 Done Sent Done false
28342 8895 Acked Write 4 Done Sent Done false
28343 8896 Acked Write 8 Done Sent Done false
28344 8897 Acked Write 8 Done Sent Done false
28345 8898 Acked Write 3 Done Sent Done false
28346 8899 Acked Read 1 Done Sent Done false
28347 8900 Acked Read 6 Done Sent Done false
28348 8901 Acked Write 5 Done Sent Done false
28349 8902 Acked Write 6 Done Sent Done false
28350 8903 Acked Write 4 Done Sent Done false
28351 8904 Acked Flush 0 Done Sent Done false
28352 8905 Acked Read 2 Done Sent Done false
28353 8906 Acked Read 5 Done Sent Done false
28354 8907 Acked Flush 0 Done Sent Done false
28355 8908 Acked Read 10 Done Sent Done false
28356 8909 Acked Write 6 Done Sent Done false
28357 8910 Acked Read 3 Done Sent Done false
28358 8911 Acked Write 6 Done Sent Done false
28359 8912 Acked Write 8 Done Sent Done false
28360 8913 Acked Read 5 Done Sent Done false
28361 8914 Acked Read 5 Done Sent Done false
28362 8915 Acked Write 9 Done Sent Done false
28363 8916 Acked Write 4 Done Sent Done false
28364 8917 Acked Read 5 Done Sent Done false
28365 8918 Acked Read 6 Done Sent Done false
28366 8919 Acked Write 8 Done Sent Done false
28367 8920 Acked Flush 0 Done Sent Done false
28368 8921 Acked Write 6 Done Sent Done false
28369 8922 Acked Flush 0 Done Sent Done false
28370 8923 Acked Read 4 Done Sent Done false
28371 8924 Acked Write 10 Done Sent Done false
28372 8925 Acked Read 2 Done Sent Done false
28373 8926 Acked Write 10 Done Sent Done false
28374 8927 Acked Read 5 Done Sent Done false
28375 8928 Acked Write 6 Done Sent Done false
28376 8929 Acked Read 4 Done Sent Done false
28377 8930 Acked Flush 0 Done Sent Done false
28378 8931 Acked Read 5 Done Sent Done false
28379 8932 Acked Write 1 Done Sent Done false
28380 8933 Acked Read 4 Done Sent Done false
28381 8934 Acked Write 3 Done Sent Done false
28382 8935 Acked Write 5 Done Sent Done false
28383 8936 Acked Write 9 Done Sent Done false
28384 8937 Acked Write 1 Done Sent Done false
28385 8938 Acked Flush 0 Done Sent Done false
28386 8939 Acked Write 10 Done Sent Done false
28387 8940 Acked Write 8 Done Sent Done false
28388 8941 Acked Write 10 Done Sent Done false
28389 8942 Acked Read 8 Done Sent Done false
28390 8943 Acked Read 1 Done Sent Done false
28391 8944 Acked Read 8 Done Sent Done false
28392 8945 Acked Read 4 Done Sent Done false
28393 8946 Acked Read 8 Done Sent Done false
28394 8947 Acked Write 1 Done Sent Done false
28395 8948 Acked Write 10 Done Sent Done false
28396 8949 Acked Write 5 Done Sent Done false
28397 8950 Acked Read 7 Done Sent Done false
28398 8951 Acked Read 3 Done Sent Done false
28399 8952 Acked Read 6 Done Sent Done false
28400 8953 Acked Read 9 Done Sent Done false
28401 8954 Acked Read 7 Done Sent Done false
28402 8955 Acked Write 4 Done Sent Done false
28403 8956 Acked Write 10 Done Sent Done false
28404 8957 Acked Write 6 Done Sent Done false
28405 8958 Acked Read 6 Done Sent Done false
28406 8959 Acked Write 8 Done Sent Done false
28407 8960 Acked Read 10 Done Sent Done false
28408 8961 Acked Read 6 Done Sent Done false
28409 8962 Acked Read 7 Done Sent Done false
28410 8963 Acked Write 1 Done Sent Done false
28411 8964 Acked Write 4 Done Sent Done false
28412 8965 Acked Flush 0 Done Sent Done false
28413 8966 Acked Write 8 Done Sent Done false
28414 8967 Acked Read 1 Done Sent Done false
28415 8968 Acked Write 1 Done Sent Done false
28416 8969 Acked Read 5 Done Sent Done false
28417 8970 Acked Write 2 Done Sent Done false
28418 8971 Acked Write 10 Done Sent Done false
28419 8972 Acked Flush 0 Done Sent Done false
28420 8973 Acked Read 1 Done Sent Done false
28421 8974 Acked Read 2 Done Sent Done false
28422 8975 Acked Flush 0 Done Sent Done false
28423 8976 Acked Write 3 Done Sent Done false
28424 8977 Acked Write 3 Done Sent Done false
28425 8978 Acked Write 10 Done Sent Done false
28426 8979 Acked Write 1 Done Sent Done false
28427 8980 Acked Read 2 Done Sent Done false
28428 8981 Acked Read 6 Done Sent Done false
28429 8982 Acked Write 3 Done Sent Done false
28430 8983 Acked Read 3 Done Sent Done false
28431 8984 Acked Read 8 Done Sent Done false
28432 8985 Acked Write 3 Done Sent Done false
28433 8986 Acked Read 4 Done Sent Done false
28434 8987 Acked Read 2 Done Sent Done false
28435 8988 Acked Flush 0 Done Sent Done false
28436 8989 Acked Read 4 Done Sent Done false
28437 8990 Acked Read 2 Done Sent Done false
28438 8991 Acked Read 3 Done Sent Done false
28439 8992 Acked Read 1 Done Sent Done false
28440 8993 Acked Write 6 Done Sent Done false
28441 8994 Acked Write 7 Done Sent Done false
28442 8995 Acked Write 3 Done Sent Done false
28443 8996 Acked Read 5 Done Sent Done false
28444 8997 Acked Write 5 Done Sent Done false
28445 8998 Acked Read 3 Done Sent Done false
28446 8999 Acked Read 8 Done Sent Done false
28447 9000 Acked Read 10 Done Sent Done false
28448 9001 Acked Write 10 Done Sent Done false
28449 9002 Acked Read 8 Done Sent Done false
28450 9003 Acked Read 10 Done Sent Done false
28451 9004 Acked Flush 0 Done Sent Done false
28452 9005 Acked Write 10 Done Sent Done false
28453 9006 Acked Write 7 Done Sent Done false
28454 9007 Acked Read 8 Done Sent Done false
28455 9008 Acked Read 4 Done Sent Done false
28456 9009 Acked Flush 0 Done Sent Done false
28457 9010 Acked Write 4 Done Sent Done false
28458 9011 Acked Write 4 Done Sent Done false
28459 9012 Acked Write 8 Done Sent Done false
28460 9013 Acked Write 10 Done Sent Done false
28461 9014 Acked Write 10 Done Sent Done false
28462 9015 Acked Write 8 Done Sent Done false
28463 9016 Acked Read 10 Done Sent Done false
28464 9017 Acked Read 4 Done Sent Done false
28465 9018 Acked Write 4 Done Sent Done false
28466 9019 Acked Read 9 Done Sent Done false
28467 9020 Acked Read 3 Done Sent Done false
28468 9021 Acked Read 10 Done Sent Done false
28469 9022 Acked Read 6 Done Sent Done false
28470 9023 Acked Flush 0 Done Sent Done false
28471 9024 Acked Read 6 Done Sent Done false
28472 9025 Acked Read 3 Done Sent Done false
28473 9026 Acked Read 7 Done Sent Done false
28474 9027 Acked Write 6 Done Sent Done false
28475 9028 Acked Read 3 Done Sent Done false
28476 9029 Acked Write 5 Done Sent Done false
28477 9030 Acked Read 1 Done Sent Done false
28478 9031 Acked Read 4 Done Sent Done false
28479 9032 Acked Read 4 Done Sent Done false
28480 9033 Acked Read 6 Done Sent Done false
28481 9034 Acked Flush 0 Done Sent Done false
28482 9035 Acked Write 4 Done Sent Done false
28483 9036 Acked Flush 0 Done Sent Done false
28484 9037 Acked Write 10 Done Sent Done false
28485 9038 Acked Read 6 Done Sent Done false
28486 9039 Acked Flush 0 Done Sent Done false
28487 9040 Acked Read 2 Done Sent Done false
28488 9041 Acked Read 3 Done Sent Done false
28489 9042 Acked Read 8 Done Sent Done false
28490 9043 Acked Read 1 Done Sent Done false
28491 9044 Acked Write 1 Done Sent Done false
28492 9045 Acked Write 10 Done Sent Done false
28493 9046 Acked Write 5 Done Sent Done false
28494 9047 Acked Read 3 Done Sent Done false
28495 9048 Acked Read 9 Done Sent Done false
28496 9049 Acked Read 8 Done Sent Done false
28497 9050 Acked Write 3 Done Sent Done false
28498 9051 Acked Write 7 Done Sent Done false
28499 9052 Acked Write 7 Done Sent Done false
28500 9053 Acked Read 10 Done Sent Done false
28501 9054 Acked Write 1 Done Sent Done false
28502 9055 Acked Write 6 Done Sent Done false
28503 9056 Acked Write 3 Done Sent Done false
28504 9057 Acked Write 3 Done Sent Done false
28505 9058 Acked Write 8 Done Sent Done false
28506 9059 Acked Read 10 Done Sent Done false
28507 9060 Acked Read 3 Done Sent Done false
28508 9061 Acked Flush 0 Done Sent Done false
28509 9062 Acked Read 5 Done Sent Done false
28510 9063 Acked Write 1 Done Sent Done false
28511 9064 Acked Read 5 Done Sent Done false
28512 9065 Acked Flush 0 Done Sent Done false
28513 9066 Acked Read 2 Done Sent Done false
28514 9067 Acked Read 7 Done Sent Done false
28515 9068 Acked Read 2 Done Sent Done false
28516 9069 Acked Read 9 Done Sent Done false
28517 9070 Acked Write 1 Done Sent Done false
28518 9071 Acked Read 8 Done Sent Done false
28519 9072 Acked Read 7 Done Sent Done false
28520 9073 Acked Read 4 Done Sent Done false
28521 9074 Acked Read 3 Done Sent Done false
28522 9075 Acked Write 2 Done Sent Done false
28523 9076 Acked Read 8 Done Sent Done false
28524 9077 Acked Read 8 Done Sent Done false
28525 9078 Acked Read 1 Done Sent Done false
28526 9079 Acked Flush 0 Done Sent Done false
28527 9080 Acked Write 3 Done Sent Done false
28528 9081 Acked Flush 0 Done Sent Done false
28529 9082 Acked Read 6 Done Sent Done false
28530 9083 Acked Write 1 Done Sent Done false
28531 9084 Acked Read 10 Done Sent Done false
28532 9085 Acked Read 5 Done Sent Done false
28533 9086 Acked Write 10 Done Sent Done false
28534 9087 Acked Read 8 Done Sent Done false
28535 9088 Acked Read 3 Done Sent Done false
28536 9089 Acked Read 1 Done Sent Done false
28537 9090 Acked Read 8 Done Sent Done false
28538 9091 Acked Read 6 Done Sent Done false
28539 9092 Acked Read 6 Done Sent Done false
28540 9093 Acked Read 1 Done Sent Done false
28541 9094 Acked Read 8 Done Sent Done false
28542 9095 Acked Write 3 Done Sent Done false
28543 9096 Acked Read 2 Done Sent Done false
28544 9097 Acked Write 10 Done Sent Done false
28545 9098 Acked Read 6 Done Sent Done false
28546 9099 Acked Read 6 Done Sent Done false
28547 9100 Acked Write 3 Done Sent Done false
28548 9101 Acked Read 8 Done Sent Done false
28549 9102 Acked Read 6 Done Sent Done false
28550 9103 Acked Read 8 Done Sent Done false
28551 9104 Acked Flush 0 Done Sent Done false
28552 9105 Acked Write 7 Done Sent Done false
28553 9106 Acked Read 3 Done Sent Done false
28554 9107 Acked Write 5 Done Sent Done false
28555 9108 Acked Write 7 Done Sent Done false
28556 9109 Acked Write 3 Done Sent Done false
28557 9110 Acked Write 6 Done Sent Done false
28558 9111 Acked Read 8 Done Sent Done false
28559 9112 Acked Read 6 Done Sent Done false
28560 9113 Acked Write 1 Done Sent Done false
28561 9114 Acked Write 8 Done Sent Done false
28562 9115 Acked Read 9 Done Sent Done false
28563 9116 Acked Read 8 Done Sent Done false
28564 9117 Acked Read 9 Done Sent Done false
28565 9118 Acked Write 9 Done Sent Done false
28566 9119 Acked Read 8 Done Sent Done false
28567 9120 Acked Write 5 Done Sent Done false
28568 9121 Acked Read 7 Done Sent Done false
28569 9122 Acked Write 5 Done Sent Done false
28570 9123 Acked Write 9 Done Sent Done false
28571 9124 Acked Write 6 Done Sent Done false
28572 9125 Acked Flush 0 Done Sent Done false
28573 9126 Acked Read 8 Done Sent Done false
28574 9127 Acked Write 4 Done Sent Done false
28575 9128 Acked Read 3 Done Sent Done false
28576 9129 Acked Write 5 Done Sent Done false
28577 9130 Acked Write 3 Done Sent Done false
28578 9131 Acked Read 10 Done Sent Done false
28579 9132 Acked Write 7 Done Sent Done false
28580 9133 Acked Flush 0 Done Sent Done false
28581 9134 Acked Read 9 Done Sent Done false
28582 9135 Acked Read 8 Done Sent Done false
28583 9136 Acked Write 4 Done Sent Done false
28584 9137 Acked Read 9 Done Sent Done false
28585 9138 Acked Write 6 Done Sent Done false
28586 9139 Acked Read 3 Done Sent Done false
28587 9140 Acked Read 1 Done Sent Done false
28588 9141 Acked Write 6 Done Sent Done false
28589 9142 Acked Read 4 Done Sent Done false
28590 9143 Acked Read 4 Done Sent Done false
28591 9144 Acked Read 9 Done Sent Done false
28592 9145 Acked Read 1 Done Sent Done false
28593 9146 Acked Read 3 Done Sent Done false
28594 9147 Acked Write 7 Done Sent Done false
28595 9148 Acked Read 1 Done Sent Done false
28596 9149 Acked Write 6 Done Sent Done false
28597 STATES DS:0 DS:1 DS:2 TOTAL
28598 Sent 0 301 0 301
28599 Done 301 0 301 602
28600 Skipped 0 0 0 0
28601 Error 0 0 0 0
28602 Last Flush: 9133 8848 9133
28603 Downstairs last five completed: 8848 8847 8846 8845 8844
28604 Upstairs last five completed: 9149 9148 9147 9146 9145
28605 CLIENT: Up:0 ds:301 act:2
28606 ----------------------------------------------------------------
28607 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28608 States: Active Offline Active
28609 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28610 8849 Acked Read 5 Done Sent Done false
28611 8850 Acked Read 6 Done Sent Done false
28612 8851 Acked Read 4 Done Sent Done false
28613 8852 Acked Flush 0 Done Sent Done false
28614 8853 Acked Write 1 Done Sent Done false
28615 8854 Acked Write 3 Done Sent Done false
28616 8855 Acked Write 10 Done Sent Done false
28617 8856 Acked Read 5 Done Sent Done false
28618 8857 Acked Read 5 Done Sent Done false
28619 8858 Acked Write 6 Done Sent Done false
28620 8859 Acked Read 4 Done Sent Done false
28621 8860 Acked Write 4 Done Sent Done false
28622 8861 Acked Write 4 Done Sent Done false
28623 8862 Acked Read 9 Done Sent Done false
28624 8863 Acked Read 8 Done Sent Done false
28625 8864 Acked Read 10 Done Sent Done false
28626 8865 Acked Read 3 Done Sent Done false
28627 8866 Acked Read 10 Done Sent Done false
28628 8867 Acked Read 8 Done Sent Done false
28629 8868 Acked Read 3 Done Sent Done false
28630 8869 Acked Read 10 Done Sent Done false
28631 8870 Acked Write 2 Done Sent Done false
28632 8871 Acked Read 6 Done Sent Done false
28633 8872 Acked Read 3 Done Sent Done false
28634 8873 Acked Read 2 Done Sent Done false
28635 8874 Acked Read 10 Done Sent Done false
28636 8875 Acked Write 5 Done Sent Done false
28637 8876 Acked Write 4 Done Sent Done false
28638 8877 Acked Write 2 Done Sent Done false
28639 8878 Acked Write 10 Done Sent Done false
28640 8879 Acked Read 7 Done Sent Done false
28641 8880 Acked Read 5 Done Sent Done false
28642 8881 Acked Read 2 Done Sent Done false
28643 8882 Acked Write 5 Done Sent Done false
28644 8883 Acked Read 10 Done Sent Done false
28645 8884 Acked Read 7 Done Sent Done false
28646 8885 Acked Flush 0 Done Sent Done false
28647 8886 Acked Read 5 Done Sent Done false
28648 8887 Acked Write 1 Done Sent Done false
28649 8888 Acked Write 1 Done Sent Done false
28650 8889 Acked Read 4 Done Sent Done false
28651 8890 Acked Write 4 Done Sent Done false
28652 8891 Acked Flush 0 Done Sent Done false
28653 8892 Acked Read 10 Done Sent Done false
28654 8893 Acked Write 7 Done Sent Done false
28655 8894 Acked Read 1 Done Sent Done false
28656 8895 Acked Write 4 Done Sent Done false
28657 8896 Acked Write 8 Done Sent Done false
28658 8897 Acked Write 8 Done Sent Done false
28659 8898 Acked Write 3 Done Sent Done false
28660 8899 Acked Read 1 Done Sent Done false
28661 8900 Acked Read 6 Done Sent Done false
28662 8901 Acked Write 5 Done Sent Done false
28663 8902 Acked Write 6 Done Sent Done false
28664 8903 Acked Write 4 Done Sent Done false
28665 8904 Acked Flush 0 Done Sent Done false
28666 8905 Acked Read 2 Done Sent Done false
28667 8906 Acked Read 5 Done Sent Done false
28668 8907 Acked Flush 0 Done Sent Done false
28669 8908 Acked Read 10 Done Sent Done false
28670 8909 Acked Write 6 Done Sent Done false
28671 8910 Acked Read 3 Done Sent Done false
28672 8911 Acked Write 6 Done Sent Done false
28673 8912 Acked Write 8 Done Sent Done false
28674 8913 Acked Read 5 Done Sent Done false
28675 8914 Acked Read 5 Done Sent Done false
28676 8915 Acked Write 9 Done Sent Done false
28677 8916 Acked Write 4 Done Sent Done false
28678 8917 Acked Read 5 Done Sent Done false
28679 8918 Acked Read 6 Done Sent Done false
28680 8919 Acked Write 8 Done Sent Done false
28681 8920 Acked Flush 0 Done Sent Done false
28682 8921 Acked Write 6 Done Sent Done false
28683 8922 Acked Flush 0 Done Sent Done false
28684 8923 Acked Read 4 Done Sent Done false
28685 8924 Acked Write 10 Done Sent Done false
28686 8925 Acked Read 2 Done Sent Done false
28687 8926 Acked Write 10 Done Sent Done false
28688 8927 Acked Read 5 Done Sent Done false
28689 8928 Acked Write 6 Done Sent Done false
28690 8929 Acked Read 4 Done Sent Done false
28691 8930 Acked Flush 0 Done Sent Done false
28692 8931 Acked Read 5 Done Sent Done false
28693 8932 Acked Write 1 Done Sent Done false
28694 8933 Acked Read 4 Done Sent Done false
28695 8934 Acked Write 3 Done Sent Done false
28696 8935 Acked Write 5 Done Sent Done false
28697 8936 Acked Write 9 Done Sent Done false
28698 8937 Acked Write 1 Done Sent Done false
28699 8938 Acked Flush 0 Done Sent Done false
28700 8939 Acked Write 10 Done Sent Done false
28701 8940 Acked Write 8 Done Sent Done false
28702 8941 Acked Write 10 Done Sent Done false
28703 8942 Acked Read 8 Done Sent Done false
28704 8943 Acked Read 1 Done Sent Done false
28705 8944 Acked Read 8 Done Sent Done false
28706 8945 Acked Read 4 Done Sent Done false
28707 8946 Acked Read 8 Done Sent Done false
28708 8947 Acked Write 1 Done Sent Done false
28709 8948 Acked Write 10 Done Sent Done false
28710 8949 Acked Write 5 Done Sent Done false
28711 8950 Acked Read 7 Done Sent Done false
28712 8951 Acked Read 3 Done Sent Done false
28713 8952 Acked Read 6 Done Sent Done false
28714 8953 Acked Read 9 Done Sent Done false
28715 8954 Acked Read 7 Done Sent Done false
28716 8955 Acked Write 4 Done Sent Done false
28717 8956 Acked Write 10 Done Sent Done false
28718 8957 Acked Write 6 Done Sent Done false
28719 8958 Acked Read 6 Done Sent Done false
28720 8959 Acked Write 8 Done Sent Done false
28721 8960 Acked Read 10 Done Sent Done false
28722 8961 Acked Read 6 Done Sent Done false
28723 8962 Acked Read 7 Done Sent Done false
28724 8963 Acked Write 1 Done Sent Done false
28725 8964 Acked Write 4 Done Sent Done false
28726 8965 Acked Flush 0 Done Sent Done false
28727 8966 Acked Write 8 Done Sent Done false
28728 8967 Acked Read 1 Done Sent Done false
28729 8968 Acked Write 1 Done Sent Done false
28730 8969 Acked Read 5 Done Sent Done false
28731 8970 Acked Write 2 Done Sent Done false
28732 8971 Acked Write 10 Done Sent Done false
28733 8972 Acked Flush 0 Done Sent Done false
28734 8973 Acked Read 1 Done Sent Done false
28735 8974 Acked Read 2 Done Sent Done false
28736 8975 Acked Flush 0 Done Sent Done false
28737 8976 Acked Write 3 Done Sent Done false
28738 8977 Acked Write 3 Done Sent Done false
28739 8978 Acked Write 10 Done Sent Done false
28740 8979 Acked Write 1 Done Sent Done false
28741 8980 Acked Read 2 Done Sent Done false
28742 8981 Acked Read 6 Done Sent Done false
28743 8982 Acked Write 3 Done Sent Done false
28744 8983 Acked Read 3 Done Sent Done false
28745 8984 Acked Read 8 Done Sent Done false
28746 8985 Acked Write 3 Done Sent Done false
28747 8986 Acked Read 4 Done Sent Done false
28748 8987 Acked Read 2 Done Sent Done false
28749 8988 Acked Flush 0 Done Sent Done false
28750 8989 Acked Read 4 Done Sent Done false
28751 8990 Acked Read 2 Done Sent Done false
28752 8991 Acked Read 3 Done Sent Done false
28753 8992 Acked Read 1 Done Sent Done false
28754 8993 Acked Write 6 Done Sent Done false
28755 8994 Acked Write 7 Done Sent Done false
28756 8995 Acked Write 3 Done Sent Done false
28757 8996 Acked Read 5 Done Sent Done false
28758 8997 Acked Write 5 Done Sent Done false
28759 8998 Acked Read 3 Done Sent Done false
28760 8999 Acked Read 8 Done Sent Done false
28761 9000 Acked Read 10 Done Sent Done false
28762 9001 Acked Write 10 Done Sent Done false
28763 9002 Acked Read 8 Done Sent Done false
28764 9003 Acked Read 10 Done Sent Done false
28765 9004 Acked Flush 0 Done Sent Done false
28766 9005 Acked Write 10 Done Sent Done false
28767 9006 Acked Write 7 Done Sent Done false
28768 9007 Acked Read 8 Done Sent Done false
28769 9008 Acked Read 4 Done Sent Done false
28770 9009 Acked Flush 0 Done Sent Done false
28771 9010 Acked Write 4 Done Sent Done false
28772 9011 Acked Write 4 Done Sent Done false
28773 9012 Acked Write 8 Done Sent Done false
28774 9013 Acked Write 10 Done Sent Done false
28775 9014 Acked Write 10 Done Sent Done false
28776 9015 Acked Write 8 Done Sent Done false
28777 9016 Acked Read 10 Done Sent Done false
28778 9017 Acked Read 4 Done Sent Done false
28779 9018 Acked Write 4 Done Sent Done false
28780 9019 Acked Read 9 Done Sent Done false
28781 9020 Acked Read 3 Done Sent Done false
28782 9021 Acked Read 10 Done Sent Done false
28783 9022 Acked Read 6 Done Sent Done false
28784 9023 Acked Flush 0 Done Sent Done false
28785 9024 Acked Read 6 Done Sent Done false
28786 9025 Acked Read 3 Done Sent Done false
28787 9026 Acked Read 7 Done Sent Done false
28788 9027 Acked Write 6 Done Sent Done false
28789 9028 Acked Read 3 Done Sent Done false
28790 9029 Acked Write 5 Done Sent Done false
28791 9030 Acked Read 1 Done Sent Done false
28792 9031 Acked Read 4 Done Sent Done false
28793 9032 Acked Read 4 Done Sent Done false
28794 9033 Acked Read 6 Done Sent Done false
28795 9034 Acked Flush 0 Done Sent Done false
28796 9035 Acked Write 4 Done Sent Done false
28797 9036 Acked Flush 0 Done Sent Done false
28798 9037 Acked Write 10 Done Sent Done false
28799 9038 Acked Read 6 Done Sent Done false
28800 9039 Acked Flush 0 Done Sent Done false
28801 9040 Acked Read 2 Done Sent Done false
28802 9041 Acked Read 3 Done Sent Done false
28803 9042 Acked Read 8 Done Sent Done false
28804 9043 Acked Read 1 Done Sent Done false
28805 9044 Acked Write 1 Done Sent Done false
28806 9045 Acked Write 10 Done Sent Done false
28807 9046 Acked Write 5 Done Sent Done false
28808 9047 Acked Read 3 Done Sent Done false
28809 9048 Acked Read 9 Done Sent Done false
28810 9049 Acked Read 8 Done Sent Done false
28811 9050 Acked Write 3 Done Sent Done false
28812 9051 Acked Write 7 Done Sent Done false
28813 9052 Acked Write 7 Done Sent Done false
28814 9053 Acked Read 10 Done Sent Done false
28815 9054 Acked Write 1 Done Sent Done false
28816 9055 Acked Write 6 Done Sent Done false
28817 9056 Acked Write 3 Done Sent Done false
28818 9057 Acked Write 3 Done Sent Done false
28819 9058 Acked Write 8 Done Sent Done false
28820 9059 Acked Read 10 Done Sent Done false
28821 9060 Acked Read 3 Done Sent Done false
28822 9061 Acked Flush 0 Done Sent Done false
28823 9062 Acked Read 5 Done Sent Done false
28824 9063 Acked Write 1 Done Sent Done false
28825 9064 Acked Read 5 Done Sent Done false
28826 9065 Acked Flush 0 Done Sent Done false
28827 9066 Acked Read 2 Done Sent Done false
28828 9067 Acked Read 7 Done Sent Done false
28829 9068 Acked Read 2 Done Sent Done false
28830 9069 Acked Read 9 Done Sent Done false
28831 9070 Acked Write 1 Done Sent Done false
28832 9071 Acked Read 8 Done Sent Done false
28833 9072 Acked Read 7 Done Sent Done false
28834 9073 Acked Read 4 Done Sent Done false
28835 9074 Acked Read 3 Done Sent Done false
28836 9075 Acked Write 2 Done Sent Done false
28837 9076 Acked Read 8 Done Sent Done false
28838 9077 Acked Read 8 Done Sent Done false
28839 9078 Acked Read 1 Done Sent Done false
28840 9079 Acked Flush 0 Done Sent Done false
28841 9080 Acked Write 3 Done Sent Done false
28842 9081 Acked Flush 0 Done Sent Done false
28843 9082 Acked Read 6 Done Sent Done false
28844 9083 Acked Write 1 Done Sent Done false
28845 9084 Acked Read 10 Done Sent Done false
28846 9085 Acked Read 5 Done Sent Done false
28847 9086 Acked Write 10 Done Sent Done false
28848 9087 Acked Read 8 Done Sent Done false
28849 9088 Acked Read 3 Done Sent Done false
28850 9089 Acked Read 1 Done Sent Done false
28851 9090 Acked Read 8 Done Sent Done false
28852 9091 Acked Read 6 Done Sent Done false
28853 9092 Acked Read 6 Done Sent Done false
28854 9093 Acked Read 1 Done Sent Done false
28855 9094 Acked Read 8 Done Sent Done false
28856 9095 Acked Write 3 Done Sent Done false
28857 9096 Acked Read 2 Done Sent Done false
28858 9097 Acked Write 10 Done Sent Done false
28859 9098 Acked Read 6 Done Sent Done false
28860 9099 Acked Read 6 Done Sent Done false
28861 9100 Acked Write 3 Done Sent Done false
28862 9101 Acked Read 8 Done Sent Done false
28863 9102 Acked Read 6 Done Sent Done false
28864 9103 Acked Read 8 Done Sent Done false
28865 9104 Acked Flush 0 Done Sent Done false
28866 9105 Acked Write 7 Done Sent Done false
28867 9106 Acked Read 3 Done Sent Done false
28868 9107 Acked Write 5 Done Sent Done false
28869 9108 Acked Write 7 Done Sent Done false
28870 9109 Acked Write 3 Done Sent Done false
28871 9110 Acked Write 6 Done Sent Done false
28872 9111 Acked Read 8 Done Sent Done false
28873 9112 Acked Read 6 Done Sent Done false
28874 9113 Acked Write 1 Done Sent Done false
28875 9114 Acked Write 8 Done Sent Done false
28876 9115 Acked Read 9 Done Sent Done false
28877 9116 Acked Read 8 Done Sent Done false
28878 9117 Acked Read 9 Done Sent Done false
28879 9118 Acked Write 9 Done Sent Done false
28880 9119 Acked Read 8 Done Sent Done false
28881 9120 Acked Write 5 Done Sent Done false
28882 9121 Acked Read 7 Done Sent Done false
28883 9122 Acked Write 5 Done Sent Done false
28884 9123 Acked Write 9 Done Sent Done false
28885 9124 Acked Write 6 Done Sent Done false
28886 9125 Acked Flush 0 Done Sent Done false
28887 9126 Acked Read 8 Done Sent Done false
28888 9127 Acked Write 4 Done Sent Done false
28889 9128 Acked Read 3 Done Sent Done false
28890 9129 Acked Write 5 Done Sent Done false
28891 9130 Acked Write 3 Done Sent Done false
28892 9131 Acked Read 10 Done Sent Done false
28893 9132 Acked Write 7 Done Sent Done false
28894 9133 Acked Flush 0 Done Sent Done false
28895 9134 Acked Read 9 Done Sent Done false
28896 9135 Acked Read 8 Done Sent Done false
28897 9136 Acked Write 4 Done Sent Done false
28898 9137 Acked Read 9 Done Sent Done false
28899 9138 Acked Write 6 Done Sent Done false
28900 9139 Acked Read 3 Done Sent Done false
28901 9140 Acked Read 1 Done Sent Done false
28902 9141 Acked Write 6 Done Sent Done false
28903 9142 Acked Read 4 Done Sent Done false
28904 9143 Acked Read 4 Done Sent Done false
28905 9144 Acked Read 9 Done Sent Done false
28906 9145 Acked Read 1 Done Sent Done false
28907 9146 Acked Read 3 Done Sent Done false
28908 9147 Acked Write 7 Done Sent Done false
28909 9148 Acked Read 1 Done Sent Done false
28910 9149 Acked Write 6 Done Sent Done false
28911 9150 Acked Flush 0 Done Sent Done false
28912 STATES DS:0 DS:1 DS:2 TOTAL
28913 Sent 0 302 0 302
28914 Done 302 0 302 604
28915 Skipped 0 0 0 0
28916 Error 0 0 0 0
28917 Last Flush: 9150 8848 9150
28918 Downstairs last five completed: 8848 8847 8846 8845 8844
28919 Upstairs last five completed: 9150 9149 9148 9147 9146
28920 CLIENT: Up:0 ds:302 act:2
289212025-01-14T16:16:40.292ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
289222025-01-14T16:16:40.292ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
289232025-01-14T16:16:40.292ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
289242025-01-14T16:16:40.292ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = c2423ef0-728e-4895-997a-d9044c50948b
289252025-01-14T16:16:40.292ZINFOcrucible: client task is exiting = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
28926 ----------------------------------------------------------------
28927 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28928 States: Active Offline Active
28929 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28930 8849 Acked Read 5 Done Sent Done false
28931 8850 Acked Read 6 Done Sent Done false
28932 8851 Acked Read 4 Done Sent Done false
28933 8852 Acked Flush 0 Done Sent Done false
28934 8853 Acked Write 1 Done Sent Done false
28935 8854 Acked Write 3 Done Sent Done false
28936 8855 Acked Write 10 Done Sent Done false
28937 8856 Acked Read 5 Done Sent Done false
28938 8857 Acked Read 5 Done Sent Done false
28939 8858 Acked Write 6 Done Sent Done false
28940 8859 Acked Read 4 Done Sent Done false
28941 8860 Acked Write 4 Done Sent Done false
28942 8861 Acked Write 4 Done Sent Done false
28943 8862 Acked Read 9 Done Sent Done false
28944 8863 Acked Read 8 Done Sent Done false
28945 8864 Acked Read 10 Done Sent Done false
28946 8865 Acked Read 3 Done Sent Done false
28947 8866 Acked Read 10 Done Sent Done false
28948 8867 Acked Read 8 Done Sent Done false
28949 8868 Acked Read 3 Done Sent Done false
28950 8869 Acked Read 10 Done Sent Done false
28951 8870 Acked Write 2 Done Sent Done false
28952 8871 Acked Read 6 Done Sent Done false
28953 8872 Acked Read 3 Done Sent Done false
28954 8873 Acked Read 2 Done Sent Done false
28955 8874 Acked Read 10 Done Sent Done false
28956 8875 Acked Write 5 Done Sent Done false
28957 8876 Acked Write 4 Done Sent Done false
28958 8877 Acked Write 2 Done Sent Done false
28959 8878 Acked Write 10 Done Sent Done false
28960 8879 Acked Read 7 Done Sent Done false
28961 8880 Acked Read 5 Done Sent Done false
28962 8881 Acked Read 2 Done Sent Done false
28963 8882 Acked Write 5 Done Sent Done false
28964 8883 Acked Read 10 Done Sent Done false
28965 8884 Acked Read 7 Done Sent Done false
28966 8885 Acked Flush 0 Done Sent Done false
28967 8886 Acked Read 5 Done Sent Done false
28968 8887 Acked Write 1 Done Sent Done false
28969 8888 Acked Write 1 Done Sent Done false
28970 8889 Acked Read 4 Done Sent Done false
28971 8890 Acked Write 4 Done Sent Done false
28972 8891 Acked Flush 0 Done Sent Done false
28973 8892 Acked Read 10 Done Sent Done false
28974 8893 Acked Write 7 Done Sent Done false
28975 8894 Acked Read 1 Done Sent Done false
28976 8895 Acked Write 4 Done Sent Done false
28977 8896 Acked Write 8 Done Sent Done false
28978 8897 Acked Write 8 Done Sent Done false
28979 8898 Acked Write 3 Done Sent Done false
28980 8899 Acked Read 1 Done Sent Done false
28981 8900 Acked Read 6 Done Sent Done false
28982 8901 Acked Write 5 Done Sent Done false
28983 8902 Acked Write 6 Done Sent Done false
28984 8903 Acked Write 4 Done Sent Done false
28985 8904 Acked Flush 0 Done Sent Done false
28986 8905 Acked Read 2 Done Sent Done false
28987 8906 Acked Read 5 Done Sent Done false
28988 8907 Acked Flush 0 Done Sent Done false
28989 8908 Acked Read 10 Done Sent Done false
28990 8909 Acked Write 6 Done Sent Done false
28991 8910 Acked Read 3 Done Sent Done false
28992 8911 Acked Write 6 Done Sent Done false
28993 8912 Acked Write 8 Done Sent Done false
28994 8913 Acked Read 5 Done Sent Done false
28995 8914 Acked Read 5 Done Sent Done false
28996 8915 Acked Write 9 Done Sent Done false
28997 8916 Acked Write 4 Done Sent Done false
28998 8917 Acked Read 5 Done Sent Done false
28999 8918 Acked Read 6 Done Sent Done false
29000 8919 Acked Write 8 Done Sent Done false
29001 8920 Acked Flush 0 Done Sent Done false
29002 8921 Acked Write 6 Done Sent Done false
29003 8922 Acked Flush 0 Done Sent Done false
29004 8923 Acked Read 4 Done Sent Done false
29005 8924 Acked Write 10 Done Sent Done false
29006 8925 Acked Read 2 Done Sent Done false
29007 8926 Acked Write 10 Done Sent Done false
29008 8927 Acked Read 5 Done Sent Done false
29009 8928 Acked Write 6 Done Sent Done false
29010 8929 Acked Read 4 Done Sent Done false
29011 8930 Acked Flush 0 Done Sent Done false
29012 8931 Acked Read 5 Done Sent Done false
29013 8932 Acked Write 1 Done Sent Done false
29014 8933 Acked Read 4 Done Sent Done false
29015 8934 Acked Write 3 Done Sent Done false
29016 8935 Acked Write 5 Done Sent Done false
29017 8936 Acked Write 9 Done Sent Done false
29018 8937 Acked Write 1 Done Sent Done false
29019 8938 Acked Flush 0 Done Sent Done false
29020 8939 Acked Write 10 Done Sent Done false
29021 8940 Acked Write 8 Done Sent Done false
29022 8941 Acked Write 10 Done Sent Done false
29023 8942 Acked Read 8 Done Sent Done false
29024 8943 Acked Read 1 Done Sent Done false
29025 8944 Acked Read 8 Done Sent Done false
29026 8945 Acked Read 4 Done Sent Done false
29027 8946 Acked Read 8 Done Sent Done false
29028 8947 Acked Write 1 Done Sent Done false
29029 8948 Acked Write 10 Done Sent Done false
29030 8949 Acked Write 5 Done Sent Done false
29031 8950 Acked Read 7 Done Sent Done false
29032 8951 Acked Read 3 Done Sent Done false
29033 8952 Acked Read 6 Done Sent Done false
29034 8953 Acked Read 9 Done Sent Done false
29035 8954 Acked Read 7 Done Sent Done false
29036 8955 Acked Write 4 Done Sent Done false
29037 8956 Acked Write 10 Done Sent Done false
29038 8957 Acked Write 6 Done Sent Done false
29039 8958 Acked Read 6 Done Sent Done false
29040 8959 Acked Write 8 Done Sent Done false
29041 8960 Acked Read 10 Done Sent Done false
29042 8961 Acked Read 6 Done Sent Done false
29043 8962 Acked Read 7 Done Sent Done false
29044 8963 Acked Write 1 Done Sent Done false
29045 8964 Acked Write 4 Done Sent Done false
29046 8965 Acked Flush 0 Done Sent Done false
29047 8966 Acked Write 8 Done Sent Done false
29048 8967 Acked Read 1 Done Sent Done false
29049 8968 Acked Write 1 Done Sent Done false
29050 8969 Acked Read 5 Done Sent Done false
29051 8970 Acked Write 2 Done Sent Done false
29052 8971 Acked Write 10 Done Sent Done false
29053 8972 Acked Flush 0 Done Sent Done false
29054 8973 Acked Read 1 Done Sent Done false
29055 8974 Acked Read 2 Done Sent Done false
29056 8975 Acked Flush 0 Done Sent Done false
29057 8976 Acked Write 3 Done Sent Done false
29058 8977 Acked Write 3 Done Sent Done false
29059 8978 Acked Write 10 Done Sent Done false
29060 8979 Acked Write 1 Done Sent Done false
29061 8980 Acked Read 2 Done Sent Done false
29062 8981 Acked Read 6 Done Sent Done false
29063 8982 Acked Write 3 Done Sent Done false
29064 8983 Acked Read 3 Done Sent Done false
29065 8984 Acked Read 8 Done Sent Done false
29066 8985 Acked Write 3 Done Sent Done false
29067 8986 Acked Read 4 Done Sent Done false
29068 8987 Acked Read 2 Done Sent Done false
29069 8988 Acked Flush 0 Done Sent Done false
29070 8989 Acked Read 4 Done Sent Done false
29071 8990 Acked Read 2 Done Sent Done false
29072 8991 Acked Read 3 Done Sent Done false
29073 8992 Acked Read 1 Done Sent Done false
29074 8993 Acked Write 6 Done Sent Done false
29075 8994 Acked Write 7 Done Sent Done false
29076 8995 Acked Write 3 Done Sent Done false
29077 8996 Acked Read 5 Done Sent Done false
29078 8997 Acked Write 5 Done Sent Done false
29079 8998 Acked Read 3 Done Sent Done false
29080 8999 Acked Read 8 Done Sent Done false
29081 9000 Acked Read 10 Done Sent Done false
29082 9001 Acked Write 10 Done Sent Done false
29083 9002 Acked Read 8 Done Sent Done false
29084 9003 Acked Read 10 Done Sent Done false
29085 9004 Acked Flush 0 Done Sent Done false
29086 9005 Acked Write 10 Done Sent Done false
29087 9006 Acked Write 7 Done Sent Done false
29088 9007 Acked Read 8 Done Sent Done false
29089 9008 Acked Read 4 Done Sent Done false
29090 9009 Acked Flush 0 Done Sent Done false
29091 9010 Acked Write 4 Done Sent Done false
29092 9011 Acked Write 4 Done Sent Done false
29093 9012 Acked Write 8 Done Sent Done false
29094 9013 Acked Write 10 Done Sent Done false
29095 9014 Acked Write 10 Done Sent Done false
29096 9015 Acked Write 8 Done Sent Done false
29097 9016 Acked Read 10 Done Sent Done false
29098 9017 Acked Read 4 Done Sent Done false
29099 9018 Acked Write 4 Done Sent Done false
29100 9019 Acked Read 9 Done Sent Done false
29101 9020 Acked Read 3 Done Sent Done false
29102 9021 Acked Read 10 Done Sent Done false
29103 9022 Acked Read 6 Done Sent Done false
29104 9023 Acked Flush 0 Done Sent Done false
29105 9024 Acked Read 6 Done Sent Done false
29106 9025 Acked Read 3 Done Sent Done false
29107 9026 Acked Read 7 Done Sent Done false
29108 9027 Acked Write 6 Done Sent Done false
29109 9028 Acked Read 3 Done Sent Done false
29110 9029 Acked Write 5 Done Sent Done false
29111 9030 Acked Read 1 Done Sent Done false
29112 9031 Acked Read 4 Done Sent Done false
29113 9032 Acked Read 4 Done Sent Done false
29114 9033 Acked Read 6 Done Sent Done false
29115 9034 Acked Flush 0 Done Sent Done false
29116 9035 Acked Write 4 Done Sent Done false
29117 9036 Acked Flush 0 Done Sent Done false
29118 9037 Acked Write 10 Done Sent Done false
29119 9038 Acked Read 6 Done Sent Done false
29120 9039 Acked Flush 0 Done Sent Done false
29121 9040 Acked Read 2 Done Sent Done false
29122 9041 Acked Read 3 Done Sent Done false
29123 9042 Acked Read 8 Done Sent Done false
29124 9043 Acked Read 1 Done Sent Done false
29125 9044 Acked Write 1 Done Sent Done false
29126 9045 Acked Write 10 Done Sent Done false
29127 9046 Acked Write 5 Done Sent Done false
29128 9047 Acked Read 3 Done Sent Done false
29129 9048 Acked Read 9 Done Sent Done false
29130 9049 Acked Read 8 Done Sent Done false
29131 9050 Acked Write 3 Done Sent Done false
29132 9051 Acked Write 7 Done Sent Done false
29133 9052 Acked Write 7 Done Sent Done false
29134 9053 Acked Read 10 Done Sent Done false
29135 9054 Acked Write 1 Done Sent Done false
29136 9055 Acked Write 6 Done Sent Done false
29137 9056 Acked Write 3 Done Sent Done false
29138 9057 Acked Write 3 Done Sent Done false
29139 9058 Acked Write 8 Done Sent Done false
29140 9059 Acked Read 10 Done Sent Done false
29141 9060 Acked Read 3 Done Sent Done false
29142 9061 Acked Flush 0 Done Sent Done false
29143 9062 Acked Read 5 Done Sent Done false
29144 9063 Acked Write 1 Done Sent Done false
29145 9064 Acked Read 5 Done Sent Done false
29146 9065 Acked Flush 0 Done Sent Done false
29147 9066 Acked Read 2 Done Sent Done false
29148 9067 Acked Read 7 Done Sent Done false
29149 9068 Acked Read 2 Done Sent Done false
29150 9069 Acked Read 9 Done Sent Done false
29151 9070 Acked Write 1 Done Sent Done false
29152 9071 Acked Read 8 Done Sent Done false
29153 9072 Acked Read 7 Done Sent Done false
29154 9073 Acked Read 4 Done Sent Done false
29155 9074 Acked Read 3 Done Sent Done false
29156 9075 Acked Write 2 Done Sent Done false
29157 9076 Acked Read 8 Done Sent Done false
29158 9077 Acked Read 8 Done Sent Done false
29159 9078 Acked Read 1 Done Sent Done false
29160 9079 Acked Flush 0 Done Sent Done false
29161 9080 Acked Write 3 Done Sent Done false
29162 9081 Acked Flush 0 Done Sent Done false
29163 9082 Acked Read 6 Done Sent Done false
29164 9083 Acked Write 1 Done Sent Done false
29165 9084 Acked Read 10 Done Sent Done false
29166 9085 Acked Read 5 Done Sent Done false
29167 9086 Acked Write 10 Done Sent Done false
29168 9087 Acked Read 8 Done Sent Done false
29169 9088 Acked Read 3 Done Sent Done false
29170 9089 Acked Read 1 Done Sent Done false
29171 9090 Acked Read 8 Done Sent Done false
29172 9091 Acked Read 6 Done Sent Done false
29173 9092 Acked Read 6 Done Sent Done false
29174 9093 Acked Read 1 Done Sent Done false
29175 9094 Acked Read 8 Done Sent Done false
29176 9095 Acked Write 3 Done Sent Done false
29177 9096 Acked Read 2 Done Sent Done false
29178 9097 Acked Write 10 Done Sent Done false
29179 9098 Acked Read 6 Done Sent Done false
29180 9099 Acked Read 6 Done Sent Done false
29181 9100 Acked Write 3 Done Sent Done false
29182 9101 Acked Read 8 Done Sent Done false
29183 9102 Acked Read 6 Done Sent Done false
29184 9103 Acked Read 8 Done Sent Done false
29185 9104 Acked Flush 0 Done Sent Done false
29186 9105 Acked Write 7 Done Sent Done false
29187 9106 Acked Read 3 Done Sent Done false
29188 9107 Acked Write 5 Done Sent Done false
29189 9108 Acked Write 7 Done Sent Done false
29190 9109 Acked Write 3 Done Sent Done false
29191 9110 Acked Write 6 Done Sent Done false
29192 9111 Acked Read 8 Done Sent Done false
29193 9112 Acked Read 6 Done Sent Done false
29194 9113 Acked Write 1 Done Sent Done false
29195 9114 Acked Write 8 Done Sent Done false
29196 9115 Acked Read 9 Done Sent Done false
29197 9116 Acked Read 8 Done Sent Done false
29198 9117 Acked Read 9 Done Sent Done false
29199 9118 Acked Write 9 Done Sent Done false
29200 9119 Acked Read 8 Done Sent Done false
29201 9120 Acked Write 5 Done Sent Done false
29202 9121 Acked Read 7 Done Sent Done false
29203 9122 Acked Write 5 Done Sent Done false
29204 9123 Acked Write 9 Done Sent Done false
29205 9124 Acked Write 6 Done Sent Done false
29206 9125 Acked Flush 0 Done Sent Done false
29207 9126 Acked Read 8 Done Sent Done false
29208 9127 Acked Write 4 Done Sent Done false
29209 9128 Acked Read 3 Done Sent Done false
29210 9129 Acked Write 5 Done Sent Done false
29211 9130 Acked Write 3 Done Sent Done false
29212 9131 Acked Read 10 Done Sent Done false
29213 9132 Acked Write 7 Done Sent Done false
29214 9133 Acked Flush 0 Done Sent Done false
29215 9134 Acked Read 9 Done Sent Done false
29216 9135 Acked Read 8 Done Sent Done false
29217 9136 Acked Write 4 Done Sent Done false
29218 9137 Acked Read 9 Done Sent Done false
29219 9138 Acked Write 6 Done Sent Done false
29220 9139 Acked Read 3 Done Sent Done false
29221 9140 Acked Read 1 Done Sent Done false
29222 9141 Acked Write 6 Done Sent Done false
29223 9142 Acked Read 4 Done Sent Done false
29224 9143 Acked Read 4 Done Sent Done false
29225 9144 Acked Read 9 Done Sent Done false
29226 9145 Acked Read 1 Done Sent Done false
29227 9146 Acked Read 3 Done Sent Done false
29228 9147 Acked Write 7 Done Sent Done false
29229 9148 Acked Read 1 Done Sent Done false
29230 9149 Acked Write 6 Done Sent Done false
29231 9150 Acked Flush 0 Done Sent Done false
29232 STATES DS:0 DS:1 DS:2 TOTAL
29233 Sent 0 302 0 302
29234 Done 302 0 302 604
29235 Skipped 0 0 0 0
29236 Error 0 0 0 0
29237 Last Flush: 9150 8848 9150
29238 Downstairs last five completed: 8848 8847 8846 8845 8844
29239 Upstairs last five completed: 9150 9149 9148 9147 9146
29240 CLIENT: Up:0 ds:302 act:2
29241 ----------------------------------------------------------------
29242 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29243 States: Active Offline Active
29244 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29245 8849 Acked Read 5 Done Sent Done false
29246 8850 Acked Read 6 Done Sent Done false
29247 8851 Acked Read 4 Done Sent Done false
29248 8852 Acked Flush 0 Done Sent Done false
29249 8853 Acked Write 1 Done Sent Done false
29250 8854 Acked Write 3 Done Sent Done false
29251 8855 Acked Write 10 Done Sent Done false
29252 8856 Acked Read 5 Done Sent Done false
29253 8857 Acked Read 5 Done Sent Done false
29254 8858 Acked Write 6 Done Sent Done false
29255 8859 Acked Read 4 Done Sent Done false
29256 8860 Acked Write 4 Done Sent Done false
29257 8861 Acked Write 4 Done Sent Done false
29258 8862 Acked Read 9 Done Sent Done false
29259 8863 Acked Read 8 Done Sent Done false
29260 8864 Acked Read 10 Done Sent Done false
29261 8865 Acked Read 3 Done Sent Done false
29262 8866 Acked Read 10 Done Sent Done false
29263 8867 Acked Read 8 Done Sent Done false
29264 8868 Acked Read 3 Done Sent Done false
29265 8869 Acked Read 10 Done Sent Done false
29266 8870 Acked Write 2 Done Sent Done false
29267 8871 Acked Read 6 Done Sent Done false
29268 8872 Acked Read 3 Done Sent Done false
29269 8873 Acked Read 2 Done Sent Done false
29270 8874 Acked Read 10 Done Sent Done false
29271 8875 Acked Write 5 Done Sent Done false
29272 8876 Acked Write 4 Done Sent Done false
29273 8877 Acked Write 2 Done Sent Done false
29274 8878 Acked Write 10 Done Sent Done false
29275 8879 Acked Read 7 Done Sent Done false
29276 8880 Acked Read 5 Done Sent Done false
29277 8881 Acked Read 2 Done Sent Done false
29278 8882 Acked Write 5 Done Sent Done false
29279 8883 Acked Read 10 Done Sent Done false
29280 8884 Acked Read 7 Done Sent Done false
29281 8885 Acked Flush 0 Done Sent Done false
29282 8886 Acked Read 5 Done Sent Done false
29283 8887 Acked Write 1 Done Sent Done false
29284 8888 Acked Write 1 Done Sent Done false
29285 8889 Acked Read 4 Done Sent Done false
29286 8890 Acked Write 4 Done Sent Done false
29287 8891 Acked Flush 0 Done Sent Done false
29288 8892 Acked Read 10 Done Sent Done false
29289 8893 Acked Write 7 Done Sent Done false
29290 8894 Acked Read 1 Done Sent Done false
29291 8895 Acked Write 4 Done Sent Done false
29292 8896 Acked Write 8 Done Sent Done false
29293 8897 Acked Write 8 Done Sent Done false
29294 8898 Acked Write 3 Done Sent Done false
29295 8899 Acked Read 1 Done Sent Done false
29296 8900 Acked Read 6 Done Sent Done false
29297 8901 Acked Write 5 Done Sent Done false
29298 8902 Acked Write 6 Done Sent Done false
29299 8903 Acked Write 4 Done Sent Done false
29300 8904 Acked Flush 0 Done Sent Done false
29301 8905 Acked Read 2 Done Sent Done false
29302 8906 Acked Read 5 Done Sent Done false
29303 8907 Acked Flush 0 Done Sent Done false
29304 8908 Acked Read 10 Done Sent Done false
29305 8909 Acked Write 6 Done Sent Done false
29306 8910 Acked Read 3 Done Sent Done false
29307 8911 Acked Write 6 Done Sent Done false
29308 8912 Acked Write 8 Done Sent Done false
29309 8913 Acked Read 5 Done Sent Done false
29310 8914 Acked Read 5 Done Sent Done false
29311 8915 Acked Write 9 Done Sent Done false
29312 8916 Acked Write 4 Done Sent Done false
29313 8917 Acked Read 5 Done Sent Done false
29314 8918 Acked Read 6 Done Sent Done false
29315 8919 Acked Write 8 Done Sent Done false
29316 8920 Acked Flush 0 Done Sent Done false
29317 8921 Acked Write 6 Done Sent Done false
29318 8922 Acked Flush 0 Done Sent Done false
29319 8923 Acked Read 4 Done Sent Done false
29320 8924 Acked Write 10 Done Sent Done false
29321 8925 Acked Read 2 Done Sent Done false
29322 8926 Acked Write 10 Done Sent Done false
29323 8927 Acked Read 5 Done Sent Done false
29324 8928 Acked Write 6 Done Sent Done false
29325 8929 Acked Read 4 Done Sent Done false
29326 8930 Acked Flush 0 Done Sent Done false
29327 8931 Acked Read 5 Done Sent Done false
29328 8932 Acked Write 1 Done Sent Done false
29329 8933 Acked Read 4 Done Sent Done false
29330 8934 Acked Write 3 Done Sent Done false
29331 8935 Acked Write 5 Done Sent Done false
29332 8936 Acked Write 9 Done Sent Done false
29333 8937 Acked Write 1 Done Sent Done false
29334 8938 Acked Flush 0 Done Sent Done false
29335 8939 Acked Write 10 Done Sent Done false
29336 8940 Acked Write 8 Done Sent Done false
29337 8941 Acked Write 10 Done Sent Done false
29338 8942 Acked Read 8 Done Sent Done false
29339 8943 Acked Read 1 Done Sent Done false
29340 8944 Acked Read 8 Done Sent Done false
29341 8945 Acked Read 4 Done Sent Done false
29342 8946 Acked Read 8 Done Sent Done false
29343 8947 Acked Write 1 Done Sent Done false
29344 8948 Acked Write 10 Done Sent Done false
29345 8949 Acked Write 5 Done Sent Done false
29346 8950 Acked Read 7 Done Sent Done false
29347 8951 Acked Read 3 Done Sent Done false
29348 8952 Acked Read 6 Done Sent Done false
29349 8953 Acked Read 9 Done Sent Done false
29350 8954 Acked Read 7 Done Sent Done false
29351 8955 Acked Write 4 Done Sent Done false
29352 8956 Acked Write 10 Done Sent Done false
29353 8957 Acked Write 6 Done Sent Done false
29354 8958 Acked Read 6 Done Sent Done false
29355 8959 Acked Write 8 Done Sent Done false
29356 8960 Acked Read 10 Done Sent Done false
29357 8961 Acked Read 6 Done Sent Done false
29358 8962 Acked Read 7 Done Sent Done false
29359 8963 Acked Write 1 Done Sent Done false
29360 8964 Acked Write 4 Done Sent Done false
29361 8965 Acked Flush 0 Done Sent Done false
29362 8966 Acked Write 8 Done Sent Done false
29363 8967 Acked Read 1 Done Sent Done false
29364 8968 Acked Write 1 Done Sent Done false
29365 8969 Acked Read 5 Done Sent Done false
29366 8970 Acked Write 2 Done Sent Done false
29367 8971 Acked Write 10 Done Sent Done false
29368 8972 Acked Flush 0 Done Sent Done false
29369 8973 Acked Read 1 Done Sent Done false
29370 8974 Acked Read 2 Done Sent Done false
29371 8975 Acked Flush 0 Done Sent Done false
29372 8976 Acked Write 3 Done Sent Done false
29373 8977 Acked Write 3 Done Sent Done false
29374 8978 Acked Write 10 Done Sent Done false
29375 8979 Acked Write 1 Done Sent Done false
29376 8980 Acked Read 2 Done Sent Done false
29377 8981 Acked Read 6 Done Sent Done false
29378 8982 Acked Write 3 Done Sent Done false
29379 8983 Acked Read 3 Done Sent Done false
29380 8984 Acked Read 8 Done Sent Done false
29381 8985 Acked Write 3 Done Sent Done false
29382 8986 Acked Read 4 Done Sent Done false
29383 8987 Acked Read 2 Done Sent Done false
29384 8988 Acked Flush 0 Done Sent Done false
29385 8989 Acked Read 4 Done Sent Done false
29386 8990 Acked Read 2 Done Sent Done false
29387 8991 Acked Read 3 Done Sent Done false
29388 8992 Acked Read 1 Done Sent Done false
29389 8993 Acked Write 6 Done Sent Done false
29390 8994 Acked Write 7 Done Sent Done false
29391 8995 Acked Write 3 Done Sent Done false
29392 8996 Acked Read 5 Done Sent Done false
29393 8997 Acked Write 5 Done Sent Done false
29394 8998 Acked Read 3 Done Sent Done false
29395 8999 Acked Read 8 Done Sent Done false
29396 9000 Acked Read 10 Done Sent Done false
29397 9001 Acked Write 10 Done Sent Done false
29398 9002 Acked Read 8 Done Sent Done false
29399 9003 Acked Read 10 Done Sent Done false
29400 9004 Acked Flush 0 Done Sent Done false
29401 9005 Acked Write 10 Done Sent Done false
29402 9006 Acked Write 7 Done Sent Done false
29403 9007 Acked Read 8 Done Sent Done false
29404 9008 Acked Read 4 Done Sent Done false
29405 9009 Acked Flush 0 Done Sent Done false
29406 9010 Acked Write 4 Done Sent Done false
29407 9011 Acked Write 4 Done Sent Done false
29408 9012 Acked Write 8 Done Sent Done false
29409 9013 Acked Write 10 Done Sent Done false
29410 9014 Acked Write 10 Done Sent Done false
29411 9015 Acked Write 8 Done Sent Done false
29412 9016 Acked Read 10 Done Sent Done false
29413 9017 Acked Read 4 Done Sent Done false
29414 9018 Acked Write 4 Done Sent Done false
29415 9019 Acked Read 9 Done Sent Done false
29416 9020 Acked Read 3 Done Sent Done false
29417 9021 Acked Read 10 Done Sent Done false
29418 9022 Acked Read 6 Done Sent Done false
29419 9023 Acked Flush 0 Done Sent Done false
29420 9024 Acked Read 6 Done Sent Done false
29421 9025 Acked Read 3 Done Sent Done false
29422 9026 Acked Read 7 Done Sent Done false
29423 9027 Acked Write 6 Done Sent Done false
29424 9028 Acked Read 3 Done Sent Done false
29425 9029 Acked Write 5 Done Sent Done false
29426 9030 Acked Read 1 Done Sent Done false
29427 9031 Acked Read 4 Done Sent Done false
29428 9032 Acked Read 4 Done Sent Done false
29429 9033 Acked Read 6 Done Sent Done false
29430 9034 Acked Flush 0 Done Sent Done false
29431 9035 Acked Write 4 Done Sent Done false
29432 9036 Acked Flush 0 Done Sent Done false
29433 9037 Acked Write 10 Done Sent Done false
29434 9038 Acked Read 6 Done Sent Done false
29435 9039 Acked Flush 0 Done Sent Done false
29436 9040 Acked Read 2 Done Sent Done false
29437 9041 Acked Read 3 Done Sent Done false
29438 9042 Acked Read 8 Done Sent Done false
29439 9043 Acked Read 1 Done Sent Done false
29440 9044 Acked Write 1 Done Sent Done false
29441 9045 Acked Write 10 Done Sent Done false
29442 9046 Acked Write 5 Done Sent Done false
29443 9047 Acked Read 3 Done Sent Done false
29444 9048 Acked Read 9 Done Sent Done false
29445 9049 Acked Read 8 Done Sent Done false
29446 9050 Acked Write 3 Done Sent Done false
29447 9051 Acked Write 7 Done Sent Done false
29448 9052 Acked Write 7 Done Sent Done false
29449 9053 Acked Read 10 Done Sent Done false
29450 9054 Acked Write 1 Done Sent Done false
29451 9055 Acked Write 6 Done Sent Done false
29452 9056 Acked Write 3 Done Sent Done false
29453 9057 Acked Write 3 Done Sent Done false
29454 9058 Acked Write 8 Done Sent Done false
29455 9059 Acked Read 10 Done Sent Done false
29456 9060 Acked Read 3 Done Sent Done false
29457 9061 Acked Flush 0 Done Sent Done false
29458 9062 Acked Read 5 Done Sent Done false
29459 9063 Acked Write 1 Done Sent Done false
29460 9064 Acked Read 5 Done Sent Done false
29461 9065 Acked Flush 0 Done Sent Done false
29462 9066 Acked Read 2 Done Sent Done false
29463 9067 Acked Read 7 Done Sent Done false
29464 9068 Acked Read 2 Done Sent Done false
29465 9069 Acked Read 9 Done Sent Done false
29466 9070 Acked Write 1 Done Sent Done false
29467 9071 Acked Read 8 Done Sent Done false
29468 9072 Acked Read 7 Done Sent Done false
29469 9073 Acked Read 4 Done Sent Done false
29470 9074 Acked Read 3 Done Sent Done false
29471 9075 Acked Write 2 Done Sent Done false
29472 9076 Acked Read 8 Done Sent Done false
29473 9077 Acked Read 8 Done Sent Done false
29474 9078 Acked Read 1 Done Sent Done false
29475 9079 Acked Flush 0 Done Sent Done false
29476 9080 Acked Write 3 Done Sent Done false
29477 9081 Acked Flush 0 Done Sent Done false
29478 9082 Acked Read 6 Done Sent Done false
29479 9083 Acked Write 1 Done Sent Done false
29480 9084 Acked Read 10 Done Sent Done false
29481 9085 Acked Read 5 Done Sent Done false
29482 9086 Acked Write 10 Done Sent Done false
29483 9087 Acked Read 8 Done Sent Done false
29484 9088 Acked Read 3 Done Sent Done false
29485 9089 Acked Read 1 Done Sent Done false
29486 9090 Acked Read 8 Done Sent Done false
29487 9091 Acked Read 6 Done Sent Done false
29488 9092 Acked Read 6 Done Sent Done false
29489 9093 Acked Read 1 Done Sent Done false
29490 9094 Acked Read 8 Done Sent Done false
29491 9095 Acked Write 3 Done Sent Done false
29492 9096 Acked Read 2 Done Sent Done false
29493 9097 Acked Write 10 Done Sent Done false
29494 9098 Acked Read 6 Done Sent Done false
29495 9099 Acked Read 6 Done Sent Done false
29496 9100 Acked Write 3 Done Sent Done false
29497 9101 Acked Read 8 Done Sent Done false
29498 9102 Acked Read 6 Done Sent Done false
29499 9103 Acked Read 8 Done Sent Done false
29500 9104 Acked Flush 0 Done Sent Done false
29501 9105 Acked Write 7 Done Sent Done false
29502 9106 Acked Read 3 Done Sent Done false
29503 9107 Acked Write 5 Done Sent Done false
29504 9108 Acked Write 7 Done Sent Done false
29505 9109 Acked Write 3 Done Sent Done false
29506 9110 Acked Write 6 Done Sent Done false
29507 9111 Acked Read 8 Done Sent Done false
29508 9112 Acked Read 6 Done Sent Done false
29509 9113 Acked Write 1 Done Sent Done false
29510 9114 Acked Write 8 Done Sent Done false
29511 9115 Acked Read 9 Done Sent Done false
29512 9116 Acked Read 8 Done Sent Done false
29513 9117 Acked Read 9 Done Sent Done false
29514 9118 Acked Write 9 Done Sent Done false
29515 9119 Acked Read 8 Done Sent Done false
29516 9120 Acked Write 5 Done Sent Done false
29517 9121 Acked Read 7 Done Sent Done false
29518 9122 Acked Write 5 Done Sent Done false
29519 9123 Acked Write 9 Done Sent Done false
29520 9124 Acked Write 6 Done Sent Done false
29521 9125 Acked Flush 0 Done Sent Done false
29522 9126 Acked Read 8 Done Sent Done false
29523 9127 Acked Write 4 Done Sent Done false
29524 9128 Acked Read 3 Done Sent Done false
29525 9129 Acked Write 5 Done Sent Done false
29526 9130 Acked Write 3 Done Sent Done false
29527 9131 Acked Read 10 Done Sent Done false
29528 9132 Acked Write 7 Done Sent Done false
29529 9133 Acked Flush 0 Done Sent Done false
29530 9134 Acked Read 9 Done Sent Done false
29531 9135 Acked Read 8 Done Sent Done false
29532 9136 Acked Write 4 Done Sent Done false
29533 9137 Acked Read 9 Done Sent Done false
29534 9138 Acked Write 6 Done Sent Done false
29535 9139 Acked Read 3 Done Sent Done false
29536 9140 Acked Read 1 Done Sent Done false
29537 9141 Acked Write 6 Done Sent Done false
29538 9142 Acked Read 4 Done Sent Done false
29539 9143 Acked Read 4 Done Sent Done false
29540 9144 Acked Read 9 Done Sent Done false
29541 9145 Acked Read 1 Done Sent Done false
29542 9146 Acked Read 3 Done Sent Done false
29543 9147 Acked Write 7 Done Sent Done false
29544 9148 Acked Read 1 Done Sent Done false
29545 9149 Acked Write 6 Done Sent Done false
29546 9150 Acked Flush 0 Done Sent Done false
29547 STATES DS:0 DS:1 DS:2 TOTAL
29548 Sent 0 302 0 302
29549 Done 302 0 302 604
29550 Skipped 0 0 0 0
29551 Error 0 0 0 0
29552 Last Flush: 9150 8848 9150
29553 Downstairs last five completed: 8848 8847 8846 8845 8844
29554 Upstairs last five completed: 9150 9149 9148 9147 9146
29555 CLIENT: Up:0 ds:302 act:2
29556 [8820] got start action
29557 [8820] I am going to restart
29558 Starting downstairs at port 8820
29559 Make output file at "/tmp/dsc/downstairs-8820.txt"
29560 [8820][1] reports Starting
29561 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1229)
29562 [8820][1] reports Running
295632025-01-14T16:16:50.294ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
295642025-01-14T16:16:50.294ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40732) = io task client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
295652025-01-14T16:16:50.295ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
295662025-01-14T16:16:50.296ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
295672025-01-14T16:16:50.296ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
295682025-01-14T16:16:50.296ZINFOcrucible: send last flush ID to this DS: 8848 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
295692025-01-14T16:16:50.296ZINFOcrucible: Replied this last flush ID: 8848 = downstairs client = 1 session_id = c2423ef0-728e-4895-997a-d9044c50948b
295702025-01-14T16:16:50.296ZINFOcrucible: [1] client re-new 302 jobs since flush 8848 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
295712025-01-14T16:16:50.297ZINFOcrucible: [1] Replayed 302 jobs since flush: 8848 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
29572 ----------------------------------------------------------------
29573 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
29574 Downstairs last five completed: 9150 9149 9148 9147 9146
29575 Upstairs last five completed: 9150 9149 9148 9147 9146
29576 CLIENT: Up:0 ds:0 act:3
29577 Replay: All jobs finished
29578 Main task has work to do, go find it
29579 got dsc Stop(0)
29580 stop 0
29581 Jan 14 16:16:50.542 [8810] Got stop action so:false kr:false
29582 INFO request completed, latency_us: 328, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 710810cd-a742-41b2-9561-258cb69c03c0, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
29583 Jan 14 16:16:50.543 INFO request completed, latency_us: 109, response_code: 200, {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-14T16:16:50.543373588Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
29584 uri: /state/cid/0, method: GET, req_id: d5b3c22a-10d8-48fa-a9fb-94ac58ea71d2, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-14T16:16:50.543458238Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"c2423ef0-728e-4895-997a-d9044c50948b"}
29585 
295862025-01-14T16:16:50.543ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
295872025-01-14T16:16:50.543ZINFOcrucible: client task is exiting = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
29588 [8810] Exited with: ExitStatus(unix_wait_status(9))
29589 [8810][0] reports Exit
29590 Jan 14 16:16:54.546 INFO request completed, latency_us: 236, response_code: 200, uri: /state/cid/0, method: GET, req_id: 4930834f-4ad0-45d7-a9ee-831da81a7722, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
29591 001/300 Write block 4328 len 16384 data: 7 6 6 6
29592 002/300 Write block 4624 len 16384 data: 7 7 6 3
29593 003/300 Read block 2151 len 40960
29594 004/300 Read block 684 len 20480
29595 005/300 Read block 2391 len 36864
29596 006/300 Flush
29597 007/300 Write block 1424 len 28672 data: 6 7 6 7 5 4 5
29598 008/300 Read block 3726 len 12288
29599 009/300 Read block 940 len 32768
29600 010/300 Write block 4641 len 8192 data: 9 9
29601 011/300 Read block 49 len 4096
29602 012/300 Read block 2006 len 36864
29603 013/300 Flush
29604 014/300 Read block 1803 len 28672
29605 015/300 Read block 3453 len 20480
29606 016/300 Flush
29607 017/300 Read block 3479 len 16384
29608 018/300 Read block 3036 len 28672
29609 019/300 Write block 282 len 32768 data: 2 4 4 4 4 5 7 6
29610 020/300 Read block 2297 len 36864
29611 021/300 Flush
29612 022/300 Write block 3088 len 16384 data: 4 5 6 6
29613 023/300 Write block 4374 len 24576 data: 8 7 6 6 4 5
29614 024/300 Read block 301 len 40960
29615 025/300 Write block 2678 len 16384 data: 5 6 6 7
29616 026/300 Write block 3913 len 24576 data: 10 8 10 10 8 6
29617 027/300 Read block 53 len 36864
29618 028/300 Write block 3795 len 12288 data: 7 6 6
29619 029/300 Write block 1852 len 8192 data: 9 8
29620 030/300 Write block 1127 len 36864 data: 6 5 5 6 5 4 4 4 3
29621 031/300 Read block 3009 len 20480
29622 032/300 Read block 4155 len 8192
29623 033/300 Read block 2830 len 16384
29624 034/300 Read block 4907 len 12288
29625 035/300 Write block 4666 len 28672 data: 13 12 10 9 7 7 5
29626 036/300 Read block 2779 len 24576
29627 037/300 Write block 3914 len 28672 data: 9 11 11 9 7 4 4
29628 038/300 Write block 2857 len 8192 data: 8 7
29629 039/300 Write block 3409 len 8192 data: 5 4
29630 040/300 Write block 2830 len 12288 data: 6 8 7
29631 041/300 Write block 3061 len 8192 data: 7 5
29632 042/300 Read block 1195 len 24576
29633 043/300 Read block 634 len 32768
29634 044/300 Flush
29635 045/300 Write block 3493 len 16384 data: 3 3 4 5
29636 046/300 Write block 2612 len 4096 data: 7
29637 047/300 Write block 1352 len 20480 data: 8 7 6 8 9
29638 048/300 Write block 3469 len 12288 data: 4 3 4
29639 049/300 Write block 1969 len 4096 data: 4
29640 050/300 Write block 3624 len 28672 data: 7 8 8 7 6 8 7
29641 051/300 Write block 563 len 24576 data: 9 10 7 5 4 5
29642 052/300 Read block 1717 len 16384
29643 053/300 Read block 1985 len 4096
29644 054/300 Read block 660 len 40960
29645 055/300 Read block 2427 len 36864
29646 056/300 Read block 4205 len 28672
29647 057/300 Write block 4206 len 8192 data: 6 7
29648 058/300 Flush
29649 059/300 Write block 96 len 24576 data: 6 5 4 4 4 4
29650 060/300 Write block 1766 len 40960 data: 5 5 6 8 8 7 8 7 7 6
29651 061/300 Write block 1302 len 16384 data: 5 4 5 6
29652 062/300 Flush
29653 063/300 Read block 924 len 40960
29654 064/300 Read block 2948 len 8192
29655 065/300 Write block 4938 len 32768 data: 5 3 3 5 6 7 8 6
29656 066/300 Write block 3685 len 16384 data: 6 5 5 7
29657 067/300 Read block 4184 len 12288
29658 068/300 Write block 4861 len 4096 data: 6
29659 069/300 Read block 3814 len 40960
29660 070/300 Write block 2808 len 36864 data: 4 3 5 3 4 7 8 8 8
29661 071/300 Read block 2090 len 8192
29662 072/300 Read block 2184 len 8192
29663 073/300 Read block 1904 len 36864
29664 074/300 Write block 2510 len 4096 data: 6
29665 075/300 Write block 18 len 24576 data: 7 7 7 7 9 9
29666 076/300 Flush
29667 077/300 Read block 2814 len 4096
29668 078/300 Write block 2339 len 4096 data: 5
29669 079/300 Read block 2680 len 24576
29670 080/300 Read block 2692 len 36864
29671 081/300 Write block 4619 len 28672 data: 4 4 5 5 7 8 8
29672 082/300 Read block 2835 len 36864
29673 083/300 Write block 998 len 36864 data: 5 5 7 8 7 8 7 7 7
29674 084/300 Read block 4015 len 16384
29675 085/300 Write block 275 len 28672 data: 6 6 7 6 5 3 2
29676 086/300 Read block 2837 len 4096
29677 087/300 Read block 4194 len 36864
29678 088/300 Read block 2016 len 20480
29679 089/300 Read block 1888 len 40960
29680 090/300 Read block 1747 len 36864
29681 091/300 Read block 1208 len 12288
29682 092/300 Read block 1745 len 4096
29683 093/300 Read block 3267 len 36864
29684 094/300 Read block 335 len 20480
29685 095/300 Write block 2823 len 28672 data: 5 4 3 3 4 5 4
29686 096/300 Write block 1803 len 20480 data: 5 5 6 6 6
29687 097/300 Read block 3706 len 16384
29688 098/300 Write block 4044 len 12288 data: 7 8 9
29689 099/300 Read block 3506 len 40960
29690 100/300 Read block 2264 len 32768
29691 101/300 Write block 666 len 20480 data: 5 5 5 7 7
29692 102/300 Write block 1797 len 12288 data: 3 3 4
29693 103/300 Read block 4830 len 20480
29694 104/300 Read block 4149 len 8192
29695 105/300 Write block 1265 len 8192 data: 3 3
29696 106/300 Write block 3442 len 16384 data: 6 6 4 4
29697 107/300 Read block 2457 len 4096
29698 108/300 Write block 2748 len 28672 data: 5 5 4 5 5 6 5
29699 109/300 Read block 4233 len 24576
29700 110/300 Flush
29701 111/300 Read block 2799 len 36864
29702 112/300 Read block 1391 len 12288
29703 113/300 Read block 2137 len 16384
29704 114/300 Write block 1454 len 28672 data: 7 8 7 6 7 6 6
29705 115/300 Read block 3440 len 36864
29706 116/300 Read block 3458 len 28672
29707 117/300 Write block 1007 len 40960 data: 6 6 5 4 3 5 6 6 6 6
29708 118/300 Read block 3850 len 28672
29709 119/300 Flush
29710 120/300 Write block 1265 len 20480 data: 4 4 3 4 5
29711 121/300 Flush
29712 122/300 Write block 4981 len 28672 data: 6 7 7 6 5 4 4
29713 123/300 Read block 3829 len 24576
29714 124/300 Write block 3981 len 20480 data: 5 3 4 4 4
29715 125/300 Read block 4393 len 40960
29716 126/300 Read block 1873 len 28672
29717 127/300 Read block 4426 len 4096
29718 128/300 Read block 169 len 36864
29719 129/300 Read block 4422 len 16384
29720 130/300 Read block 18 len 36864
29721 131/300 Read block 3541 len 24576
29722 132/300 Read block 2828 len 8192
29723 133/300 Read block 3693 len 20480
29724 134/300 Flush
29725 135/300 Write block 3555 len 32768 data: 5 4 4 4 5 4 4 7
29726 136/300 Write block 4643 len 12288 data: 9 7 6
29727 137/300 Write block 1746 len 40960 data: 6 8 10 8 7 7 8 8 11 11
29728 138/300 Flush
29729 139/300 Flush
29730 140/300 Write block 1239 len 28672 data: 7 6 5 3 3 4 5
29731 141/300 Write block 1888 len 36864 data: 6 8 9 10 9 8 8 8 6
29732 142/300 Write block 1580 len 28672 data: 4 4 4 5 4 5 5
29733 143/300 Write block 1903 len 36864 data: 7 6 7 6 6 6 5 3 5
29734 144/300 Read block 4969 len 12288
29735 145/300 Read block 3975 len 8192
29736 146/300 Flush
29737 147/300 Read block 2297 len 8192
29738 148/300 Write block 1810 len 36864 data: 6 6 3 5 5 6 7 8 8
29739 149/300 Read block 2495 len 16384
29740 150/300 Flush
29741 151/300 Write block 2769 len 8192 data: 5 5
29742 152/300 Flush
29743 153/300 Write block 4295 len 32768 data: 11 8 6 7 7 6 5 5
29744 154/300 Write block 502 len 36864 data: 3 3 4 4 5 6 6 6 6
29745 155/300 Read block 2575 len 20480
29746 156/300 Read block 242 len 40960
29747 157/300 Read block 1890 len 24576
29748 158/300 Write block 4951 len 24576 data: 5 6 7 6 5 6
29749 159/300 Read block 1402 len 16384
29750 160/300 Read block 566 len 8192
29751 161/300 Flush
29752 162/300 Read block 4942 len 12288
29753 163/300 Read block 4297 len 16384
29754 164/300 Read block 731 len 8192
29755 165/300 Read block 3380 len 36864
29756 166/300 Write block 638 len 24576 data: 5 4 4 3 4 4
29757 167/300 Read block 494 len 8192
29758 168/300 Write block 4881 len 24576 data: 5 5 6 6 6 5
29759 169/300 Write block 3823 len 24576 data: 6 6 7 6 5 6
29760 170/300 Read block 4885 len 40960
29761 171/300 Read block 1667 len 4096
29762 172/300 Write block 3360 len 16384 data: 4 4 3 2
29763 173/300 Read block 2769 len 4096
29764 174/300 Read block 2133 len 32768
29765 175/300 Write block 4005 len 24576 data: 9 10 11 10 8 10
29766 176/300 Read block 508 len 16384
29767 177/300 Write block 2875 len 40960 data: 8 11 11 12 11 7 7 7 7 7
29768 178/300 Read block 1600 len 28672
29769 179/300 Write block 3765 len 16384 data: 6 6 7 6
29770 180/300 Read block 3280 len 4096
29771 181/300 Read block 1816 len 36864
29772 182/300 Read block 4182 len 20480
29773 183/300 Read block 4398 len 36864
29774 184/300 Write block 2614 len 24576 data: 10 9 9 10 12 10
29775 185/300 Read block 2692 len 28672
29776 186/300 Read block 4731 len 12288
29777 187/300 Read block 265 len 20480
29778 188/300 Read block 127 len 20480
29779 189/300 Read block 2600 len 8192
29780 190/300 Read block 2706 len 20480
29781 191/300 Write block 1322 len 32768 data: 4 5 4 5 6 6 7 6
29782 192/300 Write block 800 len 40960 data: 5 3 3 3 5 7 7 9 9 8
29783 193/300 Flush
29784 194/300 Write block 1119 len 20480 data: 4 5 5 5 5
29785 195/300 Write block 1007 len 24576 data: 7 7 6 5 4 6
29786 196/300 Write block 4468 len 32768 data: 6 6 6 5 6 6 5 5
29787 197/300 Write block 4686 len 16384 data: 4 4 4 4
29788 198/300 Write block 4917 len 40960 data: 8 7 7 7 7 6 5 4 5 5
29789 199/300 Write block 428 len 28672 data: 5 5 5 4 4 4 4
29790 200/300 Read block 1063 len 28672
29791 201/300 Write block 3986 len 16384 data: 5 4 4 3
29792 202/300 Read block 2915 len 24576
29793 203/300 Read block 3792 len 4096
29794 204/300 Read block 2340 len 32768
29795 205/300 Read block 1356 len 24576
29796 206/300 Write block 1098 len 24576 data: 8 7 5 7 4 3
29797 207/300 Write block 4040 len 32768 data: 4 5 4 6 8 9 10 8
29798 208/300 Write block 3790 len 36864 data: 6 5 5 5 7 8 7 7 7
29799 209/300 Flush
29800 210/300 Read block 2146 len 20480
29801 211/300 Write block 4807 len 32768 data: 6 5 4 5 4 4 5 4
29802 212/300 Read block 4796 len 20480
29803 213/300 Write block 3988 len 36864 data: 5 4 5 6 7 7 8 7 8
29804 214/300 Write block 250 len 24576 data: 4 4 5 7 7 7
29805 215/300 Read block 600 len 40960
29806 216/300 Read block 2377 len 4096
29807 217/300 Read block 3229 len 4096
29808 218/300 Read block 3796 len 8192
29809 219/300 Write block 4945 len 8192 data: 7 5
29810 220/300 Read block 26 len 12288
29811 221/300 Write block 2027 len 32768 data: 5 6 6 7 5 3 3 4
29812 222/300 Write block 2061 len 4096 data: 6
29813 223/300 Read block 3311 len 28672
29814 224/300 Write block 1056 len 32768 data: 3 4 5 5 7 8 7 6
29815 225/300 Write block 2201 len 32768 data: 4 4 7 7 8 7 7 7
29816 226/300 Write block 2165 len 12288 data: 4 4 5
29817 227/300 Write block 4797 len 16384 data: 5 7 5 5
29818 228/300 Read block 2839 len 4096
29819 229/300 Write block 1063 len 28672 data: 7 6 7 7 6 7 7
29820 230/300 Read block 4703 len 20480
29821 231/300 Read block 2041 len 16384
29822 232/300 Write block 145 len 32768 data: 10 9 9 10 8 7 6 5
29823 233/300 Write block 2407 len 16384 data: 5 7 6 8
29824 234/300 Read block 2673 len 16384
29825 235/300 Write block 2507 len 20480 data: 5 6 5 7 5
29826 236/300 Write block 1838 len 16384 data: 9 8 6 7
29827 237/300 Read block 1697 len 4096
29828 238/300 Read block 1395 len 12288
29829 239/300 Read block 1947 len 4096
29830 240/300 Write block 2013 len 32768 data: 4 4 6 6 6 5 6 6
29831 241/300 Write block 911 len 24576 data: 9 8 5 4 4 6
29832 242/300 Write block 794 len 20480 data: 8 7 7 8 8
29833 243/300 Write block 1292 len 28672 data: 4 3 3 3 5 6 5
29834 244/300 Read block 30 len 4096
29835 245/300 Write block 2317 len 12288 data: 10 9 9
29836 246/300 Write block 4211 len 12288 data: 8 7 6
29837 247/300 Flush
29838 248/300 Read block 1808 len 40960
29839 249/300 Write block 3724 len 20480 data: 5 5 6 7 5
29840 250/300 Read block 62 len 24576
29841 251/300 Write block 1292 len 20480 data: 5 4 4 4 6
29842 252/300 Write block 2662 len 40960 data: 6 6 5 4 5 7 7 4 4 5
29843 253/300 Read block 2005 len 24576
29844 254/300 Write block 4867 len 32768 data: 5 5 4 4 3 4 4 4
29845 255/300 Read block 105 len 40960
29846 256/300 Read block 4541 len 20480
29847 257/300 Read block 1332 len 4096
29848 258/300 Read block 1382 len 8192
29849 259/300 Flush
29850 260/300 Read block 602 len 24576
29851 261/300 Read block 3499 len 24576
29852 262/300 Read block 100 len 32768
29853 263/300 Read block 3260 len 32768
29854 264/300 Read block 798 len 8192
29855 265/300 Read block 4644 len 20480
29856 266/300 Write block 3588 len 16384 data: 5 4 3 4
29857 267/300 Write block 485 len 24576 data: 4 4 4 3 4 5
29858 268/300 Write block 2833 len 8192 data: 8 8
29859 269/300 Read block 644 len 32768
29860 270/300 Flush
29861 271/300 Read block 4529 len 4096
29862 272/300 Read block 263 len 4096
29863 273/300 Write block 4056 len 32768 data: 5 6 6 7 5 4 4 5
29864 274/300 Flush
29865 275/300 Read block 127 len 20480
29866 276/300 Write block 863 len 8192 data: 4 5
29867 277/300 Read block 739 len 36864
29868 278/300 Read block 755 len 20480
29869 279/300 Write block 3964 len 4096 data: 4
29870 280/300 Write block 1382 len 16384 data: 9 7 5 5
29871 281/300 Read block 54 len 16384
29872 282/300 Read block 3870 len 12288
29873 283/300 Write block 917 len 4096 data: 6
29874 284/300 Read block 2679 len 8192
29875 285/300 Read block 3895 len 24576
29876 286/300 Write block 1089 len 12288 data: 3 3 4
29877 287/300 Flush
29878 288/300 Write block 1 len 24576 data: 2 4 5 5 5 4
29879 289/300 Read block 1237 len 16384
29880 290/300 Read block 3249 len 32768
29881 291/300 Flush
29882 292/300 Write block 2689 len 28672 data: 3 3 4 4 7 9 10
29883 293/300 Write block 2403 len 12288 data: 8 9 9
29884 294/300 Write block 423 len 12288 data: 9 7 7
29885 295/300 Read block 686 len 4096
29886 296/300 Read block 2584 len 36864
29887 297/300 Write block 3623 len 24576 data: 7 8 9 9 8 7
29888 298/300 Write block 2906 len 28672 data: 9 10 10 8 8 6 5
29889 299/300 Write block 1694 len 4096 data: 5
29890 300/300 Write block 3563 len 24576 data: 7 8 9 9 11 7
29891 301/300 Flush
29892 Main task has work to do, go find it
29893 got dsc Start(0)
29894 start 0
29895 Jan 14 16:16:54.776 INFO request completed, latency_us: 200, response_code: 204, uri: /start/cid/0, method: POST, req_id: 9568204f-4176-4012-bd59-10aeab491207, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
29896 Replay: started 0, returned:Ok(())
29897 ----------------------------------------------------------------
29898 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
29899 States: Offline Active Active
29900 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29901 9151 Acked Write 4 Sent Done Done false
29902 9152 Acked Write 4 Sent Done Done false
29903 9153 Acked Read 10 Sent Done Done false
29904 9154 Acked Read 5 Sent Done Done false
29905 9155 Acked Read 9 Sent Done Done false
29906 9156 Acked Flush 0 Sent Done Done false
29907 9157 Acked Write 7 Sent Done Done false
29908 9158 Acked Read 3 Sent Done Done false
29909 9159 Acked Read 8 Sent Done Done false
29910 9160 Acked Write 2 Sent Done Done false
29911 9161 Acked Read 1 Sent Done Done false
29912 9162 Acked Read 9 Sent Done Done false
29913 9163 Acked Flush 0 Sent Done Done false
29914 9164 Acked Read 7 Sent Done Done false
29915 9165 Acked Read 5 Sent Done Done false
29916 9166 Acked Flush 0 Sent Done Done false
29917 9167 Acked Read 4 Sent Done Done false
29918 9168 Acked Read 7 Sent Done Done false
29919 9169 Acked Write 8 Sent Done Done false
29920 9170 Acked Read 9 Sent Done Done false
29921 9171 Acked Flush 0 Sent Done Done false
29922 9172 Acked Write 4 Sent Done Done false
29923 9173 Acked Write 6 Sent Done Done false
29924 9174 Acked Read 10 Sent Done Done false
29925 9175 Acked Write 4 Sent Done Done false
29926 9176 Acked Write 6 Sent Done Done false
29927 9177 Acked Read 9 Sent Done Done false
29928 9178 Acked Write 3 Sent Done Done false
29929 9179 Acked Write 2 Sent Done Done false
29930 9180 Acked Write 9 Sent Done Done false
29931 9181 Acked Read 5 Sent Done Done false
29932 9182 Acked Read 2 Sent Done Done false
29933 9183 Acked Read 4 Sent Done Done false
29934 9184 Acked Read 3 Sent Done Done false
29935 9185 Acked Write 7 Sent Done Done false
29936 9186 Acked Read 6 Sent Done Done false
29937 9187 Acked Write 7 Sent Done Done false
29938 9188 Acked Write 2 Sent Done Done false
29939 9189 Acked Write 2 Sent Done Done false
29940 9190 Acked Write 3 Sent Done Done false
29941 9191 Acked Write 2 Sent Done Done false
29942 9192 Acked Read 6 Sent Done Done false
29943 9193 Acked Read 8 Sent Done Done false
29944 9194 Acked Flush 0 Sent Done Done false
29945 9195 Acked Write 4 Sent Done Done false
29946 9196 Acked Write 1 Sent Done Done false
29947 9197 Acked Write 5 Sent Done Done false
29948 9198 Acked Write 3 Sent Done Done false
29949 9199 Acked Write 1 Sent Done Done false
29950 9200 Acked Write 7 Sent Done Done false
29951 9201 Acked Write 6 Sent Done Done false
29952 9202 Acked Read 4 Sent Done Done false
29953 9203 Acked Read 1 Sent Done Done false
29954 9204 Acked Read 10 Sent Done Done false
29955 9205 Acked Read 9 Sent Done Done false
29956 9206 Acked Read 7 Sent Done Done false
29957 9207 Acked Write 2 Sent Done Done false
29958 9208 Acked Flush 0 Sent Done Done false
29959 9209 Acked Write 6 Sent Done Done false
29960 9210 Acked Write 10 Sent Done Done false
29961 9211 Acked Write 4 Sent Done Done false
29962 9212 Acked Flush 0 Sent Done Done false
29963 9213 Acked Read 10 Sent Done Done false
29964 9214 Acked Read 2 Sent Done Done false
29965 9215 Acked Write 8 Sent Done Done false
29966 9216 Acked Write 4 Sent Done Done false
29967 9217 Acked Read 3 Sent Done Done false
29968 9218 Acked Write 1 Sent Done Done false
29969 9219 Acked Read 10 Sent Done Done false
29970 9220 Acked Write 9 Sent Done Done false
29971 9221 Acked Read 2 Sent Done Done false
29972 9222 Acked Read 2 Sent Done Done false
29973 9223 Acked Read 9 Sent Done Done false
29974 9224 Acked Write 1 Sent Done Done false
29975 9225 Acked Write 6 Sent Done Done false
29976 9226 Acked Flush 0 Sent Done Done false
29977 9227 Acked Read 1 Sent Done Done false
29978 9228 Acked Write 1 Sent Done Done false
29979 9229 Acked Read 6 Sent Done Done false
29980 9230 Acked Read 9 Sent Done Done false
29981 9231 Acked Write 7 Sent Done Done false
29982 9232 Acked Read 9 Sent Done Done false
29983 9233 Acked Write 9 Sent Done Done false
29984 9234 Acked Read 4 Sent Done Done false
29985 9235 Acked Write 7 Sent Done Done false
29986 9236 Acked Read 1 Sent Done Done false
29987 9237 Acked Read 9 Sent Done Done false
29988 9238 Acked Read 5 Sent Done Done false
29989 9239 Acked Read 10 Sent Done Done false
29990 9240 Acked Read 9 Sent Done Done false
29991 9241 Acked Read 3 Sent Done Done false
29992 9242 Acked Read 1 Sent Done Done false
29993 9243 Acked Read 9 Sent Done Done false
29994 9244 Acked Read 5 Sent Done Done false
29995 9245 Acked Write 7 Sent Done Done false
29996 9246 Acked Write 5 Sent Done Done false
29997 9247 Acked Read 4 Sent Done Done false
29998 9248 Acked Write 3 Sent Done Done false
29999 9249 Acked Read 10 Sent Done Done false
30000 9250 Acked Read 8 Sent Done Done false
30001 9251 Acked Write 5 Sent Done Done false
30002 9252 Acked Write 3 Sent Done Done false
30003 9253 Acked Read 5 Sent Done Done false
30004 9254 Acked Read 2 Sent Done Done false
30005 9255 Acked Write 2 Sent Done Done false
30006 9256 Acked Write 4 Sent Done Done false
30007 9257 Acked Read 1 Sent Done Done false
30008 9258 Acked Write 7 Sent Done Done false
30009 9259 Acked Read 6 Sent Done Done false
30010 9260 Acked Flush 0 Sent Done Done false
30011 9261 Acked Read 9 Sent Done Done false
30012 9262 Acked Read 3 Sent Done Done false
30013 9263 Acked Read 4 Sent Done Done false
30014 9264 Acked Write 7 Sent Done Done false
30015 9265 Acked Read 9 Sent Done Done false
30016 9266 Acked Read 7 Sent Done Done false
30017 9267 Acked Write 10 Sent Done Done false
30018 9268 Acked Read 7 Sent Done Done false
30019 9269 Acked Flush 0 Sent Done Done false
30020 9270 Acked Write 5 Sent Done Done false
30021 9271 Acked Flush 0 Sent Done Done false
30022 9272 Acked Write 7 Sent Done Done false
30023 9273 Acked Read 6 Sent Done Done false
30024 9274 Acked Write 5 Sent Done Done false
30025 9275 Acked Read 10 Sent Done Done false
30026 9276 Acked Read 7 Sent Done Done false
30027 9277 Acked Read 1 Sent Done Done false
30028 9278 Acked Read 9 Sent Done Done false
30029 9279 Acked Read 4 Sent Done Done false
30030 9280 Acked Read 9 Sent Done Done false
30031 9281 Acked Read 6 Sent Done Done false
30032 9282 Acked Read 2 Sent Done Done false
30033 9283 Acked Read 5 Sent Done Done false
30034 9284 Acked Flush 0 Sent Done Done false
30035 9285 Acked Write 8 Sent Done Done false
30036 9286 Acked Write 3 Sent Done Done false
30037 9287 Acked Write 10 Sent Done Done false
30038 9288 Acked Flush 0 Sent Done Done false
30039 9289 Acked Flush 0 Sent Done Done false
30040 9290 Acked Write 7 Sent Done Done false
30041 9291 Acked Write 9 Sent Done Done false
30042 9292 Acked Write 7 Sent Done Done false
30043 9293 Acked Write 9 Sent Done Done false
30044 9294 Acked Read 3 Sent Done Done false
30045 9295 Acked Read 2 Sent Done Done false
30046 9296 Acked Flush 0 Sent Done Done false
30047 9297 Acked Read 2 Sent Done Done false
30048 9298 Acked Write 9 Sent Done Done false
30049 9299 Acked Read 4 Sent Done Done false
30050 9300 Acked Flush 0 Sent Done Done false
30051 9301 Acked Write 2 Sent Done Done false
30052 9302 Acked Flush 0 Sent Done Done false
30053 9303 Acked Write 8 Sent Done Done false
30054 9304 Acked Write 9 Sent Done Done false
30055 9305 Acked Read 5 Sent Done Done false
30056 9306 Acked Read 10 Sent Done Done false
30057 9307 Acked Read 6 Sent Done Done false
30058 9308 Acked Write 6 Sent Done Done false
30059 9309 Acked Read 4 Sent Done Done false
30060 9310 Acked Read 2 Sent Done Done false
30061 9311 Acked Flush 0 Sent Done Done false
30062 9312 Acked Read 3 Sent Done Done false
30063 9313 Acked Read 4 Sent Done Done false
30064 9314 Acked Read 2 Sent Done Done false
30065 9315 Acked Read 9 Sent Done Done false
30066 9316 Acked Write 6 Sent Done Done false
30067 9317 Acked Read 2 Sent Done Done false
30068 9318 Acked Write 6 Sent Done Done false
30069 9319 Acked Write 6 Sent Done Done false
30070 9320 Acked Read 10 Sent Done Done false
30071 9321 Acked Read 1 Sent Done Done false
30072 9322 Acked Write 4 Sent Done Done false
30073 9323 Acked Read 1 Sent Done Done false
30074 9324 Acked Read 8 Sent Done Done false
30075 9325 Acked Write 6 Sent Done Done false
30076 9326 Acked Read 4 Sent Done Done false
30077 9327 Acked Write 10 Sent Done Done false
30078 9328 Acked Read 7 Sent Done Done false
30079 9329 Acked Write 4 Sent Done Done false
30080 9330 Acked Read 1 Sent Done Done false
30081 9331 Acked Read 9 Sent Done Done false
30082 9332 Acked Read 5 Sent Done Done false
30083 9333 Acked Read 9 Sent Done Done false
30084 9334 Acked Write 6 Sent Done Done false
30085 9335 Acked Read 7 Sent Done Done false
30086 9336 Acked Read 3 Sent Done Done false
30087 9337 Acked Read 5 Sent Done Done false
30088 9338 Acked Read 5 Sent Done Done false
30089 9339 Acked Read 2 Sent Done Done false
30090 9340 Acked Read 5 Sent Done Done false
30091 9341 Acked Write 8 Sent Done Done false
30092 9342 Acked Write 10 Sent Done Done false
30093 9343 Acked Flush 0 Sent Done Done false
30094 9344 Acked Write 5 Sent Done Done false
30095 9345 Acked Write 6 Sent Done Done false
30096 9346 Acked Write 8 Sent Done Done false
30097 9347 Acked Write 4 Sent Done Done false
30098 9348 Acked Write 10 Sent Done Done false
30099 9349 Acked Write 7 Sent Done Done false
30100 9350 Acked Read 7 Sent Done Done false
30101 9351 Acked Write 4 Sent Done Done false
30102 9352 Acked Read 6 Sent Done Done false
30103 9353 Acked Read 1 Sent Done Done false
30104 9354 Acked Read 8 Sent Done Done false
30105 9355 Acked Read 6 Sent Done Done false
30106 9356 Acked Write 6 Sent Done Done false
30107 9357 Acked Write 8 Sent Done Done false
30108 9358 Acked Write 9 Sent Done Done false
30109 9359 Acked Flush 0 Sent Done Done false
30110 9360 Acked Read 5 Sent Done Done false
30111 9361 Acked Write 8 Sent Done Done false
30112 9362 Acked Read 5 Sent Done Done false
30113 9363 Acked Write 9 Sent Done Done false
30114 9364 Acked Write 6 Sent Done Done false
30115 9365 Acked Read 10 Sent Done Done false
30116 9366 Acked Read 1 Sent Done Done false
30117 9367 Acked Read 1 Sent Done Done false
30118 9368 Acked Read 2 Sent Done Done false
30119 9369 Acked Write 2 Sent Done Done false
30120 9370 Acked Read 3 Sent Done Done false
30121 9371 Acked Write 8 Sent Done Done false
30122 9372 Acked Write 1 Sent Done Done false
30123 9373 Acked Read 7 Sent Done Done false
30124 9374 Acked Write 8 Sent Done Done false
30125 9375 Acked Write 8 Sent Done Done false
30126 9376 Acked Write 3 Sent Done Done false
30127 9377 Acked Write 4 Sent Done Done false
30128 9378 Acked Read 1 Sent Done Done false
30129 9379 Acked Write 7 Sent Done Done false
30130 9380 Acked Read 5 Sent Done Done false
30131 9381 Acked Read 4 Sent Done Done false
30132 9382 Acked Write 8 Sent Done Done false
30133 9383 Acked Write 4 Sent Done Done false
30134 9384 Acked Read 4 Sent Done Done false
30135 9385 Acked Write 5 Sent Done Done false
30136 9386 Acked Write 4 Sent Done Done false
30137 9387 Acked Read 1 Sent Done Done false
30138 9388 Acked Read 3 Sent Done Done false
30139 9389 Acked Read 1 Sent Done Done false
30140 9390 Acked Write 8 Sent Done Done false
30141 9391 Acked Write 6 Sent Done Done false
30142 9392 Acked Write 5 Sent Done Done false
30143 9393 Acked Write 7 Sent Done Done false
30144 9394 Acked Read 1 Sent Done Done false
30145 9395 Acked Write 3 Sent Done Done false
30146 9396 Acked Write 3 Sent Done Done false
30147 9397 Acked Flush 0 Sent Done Done false
30148 9398 Acked Read 10 Sent Done Done false
30149 9399 Acked Write 5 Sent Done Done false
30150 9400 Acked Read 6 Sent Done Done false
30151 9401 Acked Write 5 Sent Done Done false
30152 9402 Acked Write 10 Sent Done Done false
30153 9403 Acked Read 6 Sent Done Done false
30154 9404 Acked Write 8 Sent Done Done false
30155 9405 Acked Read 10 Sent Done Done false
30156 9406 Acked Read 5 Sent Done Done false
30157 9407 Acked Read 1 Sent Done Done false
30158 9408 Acked Read 2 Sent Done Done false
30159 9409 Acked Flush 0 Sent Done Done false
30160 9410 Acked Read 6 Sent Done Done false
30161 9411 Acked Read 6 Sent Done Done false
30162 9412 Acked Read 8 Sent Done Done false
30163 9413 Acked Read 8 Sent Done Done false
30164 9414 Acked Read 2 Sent Done Done false
30165 9415 Acked Read 5 Sent Done Done false
30166 9416 Acked Write 4 Sent Done Done false
30167 9417 Acked Write 6 Sent Done Done false
30168 9418 Acked Write 2 Sent Done Done false
30169 9419 Acked Read 8 Sent Done Done false
30170 9420 Acked Flush 0 Sent Done Done false
30171 9421 Acked Read 1 Sent Done Done false
30172 9422 Acked Read 1 Sent Done Done false
30173 9423 Acked Write 8 Sent Done Done false
30174 9424 Acked Flush 0 Sent Done Done false
30175 9425 Acked Read 5 Sent Done Done false
30176 9426 Acked Write 2 Sent Done Done false
30177 9427 Acked Read 9 Sent Done Done false
30178 9428 Acked Read 5 Sent Done Done false
30179 9429 Acked Write 1 Sent Done Done false
30180 9430 Acked Write 4 Sent Done Done false
30181 9431 Acked Read 4 Sent Done Done false
30182 9432 Acked Read 3 Sent Done Done false
30183 9433 Acked Write 1 Sent Done Done false
30184 9434 Acked Read 2 Sent Done Done false
30185 9435 Acked Read 6 Sent Done Done false
30186 9436 Acked Write 3 Sent Done Done false
30187 9437 Acked Flush 0 Sent Done Done false
30188 9438 Acked Write 6 Sent Done Done false
30189 9439 Acked Read 4 Sent Done Done false
30190 9440 Acked Read 8 Sent Done Done false
30191 9441 Acked Flush 0 Sent Done Done false
30192 9442 Acked Write 7 Sent Done Done false
30193 9443 Acked Write 3 Sent Done Done false
30194 9444 Acked Write 3 Sent Done Done false
30195 9445 Acked Read 1 Sent Done Done false
30196 9446 Acked Read 9 Sent Done Done false
30197 9447 Acked Write 6 Sent Done Done false
30198 9448 Acked Write 7 Sent Done Done false
30199 9449 Acked Write 1 Sent Done Done false
30200 9450 Acked Write 6 Sent Done Done false
30201 9451 Acked Flush 0 Sent Done Done false
30202 STATES DS:0 DS:1 DS:2 TOTAL
30203 Sent 301 0 0 301
30204 Done 0 301 301 602
30205 Skipped 0 0 0 0
30206 Error 0 0 0 0
30207 Last Flush: 9150 9451 9451
30208 Downstairs last five completed: 9150 9149 9148 9147 9146
30209 Upstairs last five completed: 9451 9450 9449 9448 9447
30210 CLIENT: Up:0 ds:301 act:2
30211 ----------------------------------------------------------------
30212 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
30213 States: Offline Active Active
30214 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30215 9151 Acked Write 4 Sent Done Done false
30216 9152 Acked Write 4 Sent Done Done false
30217 9153 Acked Read 10 Sent Done Done false
30218 9154 Acked Read 5 Sent Done Done false
30219 9155 Acked Read 9 Sent Done Done false
30220 9156 Acked Flush 0 Sent Done Done false
30221 9157 Acked Write 7 Sent Done Done false
30222 9158 Acked Read 3 Sent Done Done false
30223 9159 Acked Read 8 Sent Done Done false
30224 9160 Acked Write 2 Sent Done Done false
30225 9161 Acked Read 1 Sent Done Done false
30226 9162 Acked Read 9 Sent Done Done false
30227 9163 Acked Flush 0 Sent Done Done false
30228 9164 Acked Read 7 Sent Done Done false
30229 9165 Acked Read 5 Sent Done Done false
30230 9166 Acked Flush 0 Sent Done Done false
30231 9167 Acked Read 4 Sent Done Done false
30232 9168 Acked Read 7 Sent Done Done false
30233 9169 Acked Write 8 Sent Done Done false
30234 9170 Acked Read 9 Sent Done Done false
30235 9171 Acked Flush 0 Sent Done Done false
30236 9172 Acked Write 4 Sent Done Done false
30237 9173 Acked Write 6 Sent Done Done false
30238 9174 Acked Read 10 Sent Done Done false
30239 9175 Acked Write 4 Sent Done Done false
30240 9176 Acked Write 6 Sent Done Done false
30241 9177 Acked Read 9 Sent Done Done false
30242 9178 Acked Write 3 Sent Done Done false
30243 9179 Acked Write 2 Sent Done Done false
30244 9180 Acked Write 9 Sent Done Done false
30245 9181 Acked Read 5 Sent Done Done false
30246 9182 Acked Read 2 Sent Done Done false
30247 9183 Acked Read 4 Sent Done Done false
30248 9184 Acked Read 3 Sent Done Done false
30249 9185 Acked Write 7 Sent Done Done false
30250 9186 Acked Read 6 Sent Done Done false
30251 9187 Acked Write 7 Sent Done Done false
30252 9188 Acked Write 2 Sent Done Done false
30253 9189 Acked Write 2 Sent Done Done false
30254 9190 Acked Write 3 Sent Done Done false
30255 9191 Acked Write 2 Sent Done Done false
30256 9192 Acked Read 6 Sent Done Done false
30257 9193 Acked Read 8 Sent Done Done false
30258 9194 Acked Flush 0 Sent Done Done false
30259 9195 Acked Write 4 Sent Done Done false
30260 9196 Acked Write 1 Sent Done Done false
30261 9197 Acked Write 5 Sent Done Done false
30262 9198 Acked Write 3 Sent Done Done false
30263 9199 Acked Write 1 Sent Done Done false
30264 9200 Acked Write 7 Sent Done Done false
30265 9201 Acked Write 6 Sent Done Done false
30266 9202 Acked Read 4 Sent Done Done false
30267 9203 Acked Read 1 Sent Done Done false
30268 9204 Acked Read 10 Sent Done Done false
30269 9205 Acked Read 9 Sent Done Done false
30270 9206 Acked Read 7 Sent Done Done false
30271 9207 Acked Write 2 Sent Done Done false
30272 9208 Acked Flush 0 Sent Done Done false
30273 9209 Acked Write 6 Sent Done Done false
30274 9210 Acked Write 10 Sent Done Done false
30275 9211 Acked Write 4 Sent Done Done false
30276 9212 Acked Flush 0 Sent Done Done false
30277 9213 Acked Read 10 Sent Done Done false
30278 9214 Acked Read 2 Sent Done Done false
30279 9215 Acked Write 8 Sent Done Done false
30280 9216 Acked Write 4 Sent Done Done false
30281 9217 Acked Read 3 Sent Done Done false
30282 9218 Acked Write 1 Sent Done Done false
30283 9219 Acked Read 10 Sent Done Done false
30284 9220 Acked Write 9 Sent Done Done false
30285 9221 Acked Read 2 Sent Done Done false
30286 9222 Acked Read 2 Sent Done Done false
30287 9223 Acked Read 9 Sent Done Done false
30288 9224 Acked Write 1 Sent Done Done false
30289 9225 Acked Write 6 Sent Done Done false
30290 9226 Acked Flush 0 Sent Done Done false
30291 9227 Acked Read 1 Sent Done Done false
30292 9228 Acked Write 1 Sent Done Done false
30293 9229 Acked Read 6 Sent Done Done false
30294 9230 Acked Read 9 Sent Done Done false
30295 9231 Acked Write 7 Sent Done Done false
30296 9232 Acked Read 9 Sent Done Done false
30297 9233 Acked Write 9 Sent Done Done false
30298 9234 Acked Read 4 Sent Done Done false
30299 9235 Acked Write 7 Sent Done Done false
30300 9236 Acked Read 1 Sent Done Done false
30301 9237 Acked Read 9 Sent Done Done false
30302 9238 Acked Read 5 Sent Done Done false
30303 9239 Acked Read 10 Sent Done Done false
30304 9240 Acked Read 9 Sent Done Done false
30305 9241 Acked Read 3 Sent Done Done false
30306 9242 Acked Read 1 Sent Done Done false
30307 9243 Acked Read 9 Sent Done Done false
30308 9244 Acked Read 5 Sent Done Done false
30309 9245 Acked Write 7 Sent Done Done false
30310 9246 Acked Write 5 Sent Done Done false
30311 9247 Acked Read 4 Sent Done Done false
30312 9248 Acked Write 3 Sent Done Done false
30313 9249 Acked Read 10 Sent Done Done false
30314 9250 Acked Read 8 Sent Done Done false
30315 9251 Acked Write 5 Sent Done Done false
30316 9252 Acked Write 3 Sent Done Done false
30317 9253 Acked Read 5 Sent Done Done false
30318 9254 Acked Read 2 Sent Done Done false
30319 9255 Acked Write 2 Sent Done Done false
30320 9256 Acked Write 4 Sent Done Done false
30321 9257 Acked Read 1 Sent Done Done false
30322 9258 Acked Write 7 Sent Done Done false
30323 9259 Acked Read 6 Sent Done Done false
30324 9260 Acked Flush 0 Sent Done Done false
30325 9261 Acked Read 9 Sent Done Done false
30326 9262 Acked Read 3 Sent Done Done false
30327 9263 Acked Read 4 Sent Done Done false
30328 9264 Acked Write 7 Sent Done Done false
30329 9265 Acked Read 9 Sent Done Done false
30330 9266 Acked Read 7 Sent Done Done false
30331 9267 Acked Write 10 Sent Done Done false
30332 9268 Acked Read 7 Sent Done Done false
30333 9269 Acked Flush 0 Sent Done Done false
30334 9270 Acked Write 5 Sent Done Done false
30335 9271 Acked Flush 0 Sent Done Done false
30336 9272 Acked Write 7 Sent Done Done false
30337 9273 Acked Read 6 Sent Done Done false
30338 9274 Acked Write 5 Sent Done Done false
30339 9275 Acked Read 10 Sent Done Done false
30340 9276 Acked Read 7 Sent Done Done false
30341 9277 Acked Read 1 Sent Done Done false
30342 9278 Acked Read 9 Sent Done Done false
30343 9279 Acked Read 4 Sent Done Done false
30344 9280 Acked Read 9 Sent Done Done false
30345 9281 Acked Read 6 Sent Done Done false
30346 9282 Acked Read 2 Sent Done Done false
30347 9283 Acked Read 5 Sent Done Done false
30348 9284 Acked Flush 0 Sent Done Done false
30349 9285 Acked Write 8 Sent Done Done false
30350 9286 Acked Write 3 Sent Done Done false
30351 9287 Acked Write 10 Sent Done Done false
30352 9288 Acked Flush 0 Sent Done Done false
30353 9289 Acked Flush 0 Sent Done Done false
30354 9290 Acked Write 7 Sent Done Done false
30355 9291 Acked Write 9 Sent Done Done false
30356 9292 Acked Write 7 Sent Done Done false
30357 9293 Acked Write 9 Sent Done Done false
30358 9294 Acked Read 3 Sent Done Done false
30359 9295 Acked Read 2 Sent Done Done false
30360 9296 Acked Flush 0 Sent Done Done false
30361 9297 Acked Read 2 Sent Done Done false
30362 9298 Acked Write 9 Sent Done Done false
30363 9299 Acked Read 4 Sent Done Done false
30364 9300 Acked Flush 0 Sent Done Done false
30365 9301 Acked Write 2 Sent Done Done false
30366 9302 Acked Flush 0 Sent Done Done false
30367 9303 Acked Write 8 Sent Done Done false
30368 9304 Acked Write 9 Sent Done Done false
30369 9305 Acked Read 5 Sent Done Done false
30370 9306 Acked Read 10 Sent Done Done false
30371 9307 Acked Read 6 Sent Done Done false
30372 9308 Acked Write 6 Sent Done Done false
30373 9309 Acked Read 4 Sent Done Done false
30374 9310 Acked Read 2 Sent Done Done false
30375 9311 Acked Flush 0 Sent Done Done false
30376 9312 Acked Read 3 Sent Done Done false
30377 9313 Acked Read 4 Sent Done Done false
30378 9314 Acked Read 2 Sent Done Done false
30379 9315 Acked Read 9 Sent Done Done false
30380 9316 Acked Write 6 Sent Done Done false
30381 9317 Acked Read 2 Sent Done Done false
30382 9318 Acked Write 6 Sent Done Done false
30383 9319 Acked Write 6 Sent Done Done false
30384 9320 Acked Read 10 Sent Done Done false
30385 9321 Acked Read 1 Sent Done Done false
30386 9322 Acked Write 4 Sent Done Done false
30387 9323 Acked Read 1 Sent Done Done false
30388 9324 Acked Read 8 Sent Done Done false
30389 9325 Acked Write 6 Sent Done Done false
30390 9326 Acked Read 4 Sent Done Done false
30391 9327 Acked Write 10 Sent Done Done false
30392 9328 Acked Read 7 Sent Done Done false
30393 9329 Acked Write 4 Sent Done Done false
30394 9330 Acked Read 1 Sent Done Done false
30395 9331 Acked Read 9 Sent Done Done false
30396 9332 Acked Read 5 Sent Done Done false
30397 9333 Acked Read 9 Sent Done Done false
30398 9334 Acked Write 6 Sent Done Done false
30399 9335 Acked Read 7 Sent Done Done false
30400 9336 Acked Read 3 Sent Done Done false
30401 9337 Acked Read 5 Sent Done Done false
30402 9338 Acked Read 5 Sent Done Done false
30403 9339 Acked Read 2 Sent Done Done false
30404 9340 Acked Read 5 Sent Done Done false
30405 9341 Acked Write 8 Sent Done Done false
30406 9342 Acked Write 10 Sent Done Done false
30407 9343 Acked Flush 0 Sent Done Done false
30408 9344 Acked Write 5 Sent Done Done false
30409 9345 Acked Write 6 Sent Done Done false
30410 9346 Acked Write 8 Sent Done Done false
30411 9347 Acked Write 4 Sent Done Done false
30412 9348 Acked Write 10 Sent Done Done false
30413 9349 Acked Write 7 Sent Done Done false
30414 9350 Acked Read 7 Sent Done Done false
30415 9351 Acked Write 4 Sent Done Done false
30416 9352 Acked Read 6 Sent Done Done false
30417 9353 Acked Read 1 Sent Done Done false
30418 9354 Acked Read 8 Sent Done Done false
30419 9355 Acked Read 6 Sent Done Done false
30420 9356 Acked Write 6 Sent Done Done false
30421 9357 Acked Write 8 Sent Done Done false
30422 9358 Acked Write 9 Sent Done Done false
30423 9359 Acked Flush 0 Sent Done Done false
30424 9360 Acked Read 5 Sent Done Done false
30425 9361 Acked Write 8 Sent Done Done false
30426 9362 Acked Read 5 Sent Done Done false
30427 9363 Acked Write 9 Sent Done Done false
30428 9364 Acked Write 6 Sent Done Done false
30429 9365 Acked Read 10 Sent Done Done false
30430 9366 Acked Read 1 Sent Done Done false
30431 9367 Acked Read 1 Sent Done Done false
30432 9368 Acked Read 2 Sent Done Done false
30433 9369 Acked Write 2 Sent Done Done false
30434 9370 Acked Read 3 Sent Done Done false
30435 9371 Acked Write 8 Sent Done Done false
30436 9372 Acked Write 1 Sent Done Done false
30437 9373 Acked Read 7 Sent Done Done false
30438 9374 Acked Write 8 Sent Done Done false
30439 9375 Acked Write 8 Sent Done Done false
30440 9376 Acked Write 3 Sent Done Done false
30441 9377 Acked Write 4 Sent Done Done false
30442 9378 Acked Read 1 Sent Done Done false
30443 9379 Acked Write 7 Sent Done Done false
30444 9380 Acked Read 5 Sent Done Done false
30445 9381 Acked Read 4 Sent Done Done false
30446 9382 Acked Write 8 Sent Done Done false
30447 9383 Acked Write 4 Sent Done Done false
30448 9384 Acked Read 4 Sent Done Done false
30449 9385 Acked Write 5 Sent Done Done false
30450 9386 Acked Write 4 Sent Done Done false
30451 9387 Acked Read 1 Sent Done Done false
30452 9388 Acked Read 3 Sent Done Done false
30453 9389 Acked Read 1 Sent Done Done false
30454 9390 Acked Write 8 Sent Done Done false
30455 9391 Acked Write 6 Sent Done Done false
30456 9392 Acked Write 5 Sent Done Done false
30457 9393 Acked Write 7 Sent Done Done false
30458 9394 Acked Read 1 Sent Done Done false
30459 9395 Acked Write 3 Sent Done Done false
30460 9396 Acked Write 3 Sent Done Done false
30461 9397 Acked Flush 0 Sent Done Done false
30462 9398 Acked Read 10 Sent Done Done false
30463 9399 Acked Write 5 Sent Done Done false
30464 9400 Acked Read 6 Sent Done Done false
30465 9401 Acked Write 5 Sent Done Done false
30466 9402 Acked Write 10 Sent Done Done false
30467 9403 Acked Read 6 Sent Done Done false
30468 9404 Acked Write 8 Sent Done Done false
30469 9405 Acked Read 10 Sent Done Done false
30470 9406 Acked Read 5 Sent Done Done false
30471 9407 Acked Read 1 Sent Done Done false
30472 9408 Acked Read 2 Sent Done Done false
30473 9409 Acked Flush 0 Sent Done Done false
30474 9410 Acked Read 6 Sent Done Done false
30475 9411 Acked Read 6 Sent Done Done false
30476 9412 Acked Read 8 Sent Done Done false
30477 9413 Acked Read 8 Sent Done Done false
30478 9414 Acked Read 2 Sent Done Done false
30479 9415 Acked Read 5 Sent Done Done false
30480 9416 Acked Write 4 Sent Done Done false
30481 9417 Acked Write 6 Sent Done Done false
30482 9418 Acked Write 2 Sent Done Done false
30483 9419 Acked Read 8 Sent Done Done false
30484 9420 Acked Flush 0 Sent Done Done false
30485 9421 Acked Read 1 Sent Done Done false
30486 9422 Acked Read 1 Sent Done Done false
30487 9423 Acked Write 8 Sent Done Done false
30488 9424 Acked Flush 0 Sent Done Done false
30489 9425 Acked Read 5 Sent Done Done false
30490 9426 Acked Write 2 Sent Done Done false
30491 9427 Acked Read 9 Sent Done Done false
30492 9428 Acked Read 5 Sent Done Done false
30493 9429 Acked Write 1 Sent Done Done false
30494 9430 Acked Write 4 Sent Done Done false
30495 9431 Acked Read 4 Sent Done Done false
30496 9432 Acked Read 3 Sent Done Done false
30497 9433 Acked Write 1 Sent Done Done false
30498 9434 Acked Read 2 Sent Done Done false
30499 9435 Acked Read 6 Sent Done Done false
30500 9436 Acked Write 3 Sent Done Done false
30501 9437 Acked Flush 0 Sent Done Done false
30502 9438 Acked Write 6 Sent Done Done false
30503 9439 Acked Read 4 Sent Done Done false
30504 9440 Acked Read 8 Sent Done Done false
30505 9441 Acked Flush 0 Sent Done Done false
30506 9442 Acked Write 7 Sent Done Done false
30507 9443 Acked Write 3 Sent Done Done false
30508 9444 Acked Write 3 Sent Done Done false
30509 9445 Acked Read 1 Sent Done Done false
30510 9446 Acked Read 9 Sent Done Done false
30511 9447 Acked Write 6 Sent Done Done false
30512 9448 Acked Write 7 Sent Done Done false
30513 9449 Acked Write 1 Sent Done Done false
30514 9450 Acked Write 6 Sent Done Done false
30515 9451 Acked Flush 0 Sent Done Done false
30516 STATES DS:0 DS:1 DS:2 TOTAL
30517 Sent 301 0 0 301
30518 Done 0 301 301 602
30519 Skipped 0 0 0 0
30520 Error 0 0 0 0
30521 Last Flush: 9150 9451 9451
30522 Downstairs last five completed: 9150 9149 9148 9147 9146
30523 Upstairs last five completed: 9451 9450 9449 9448 9447
30524 CLIENT: Up:0 ds:301 act:2
305252025-01-14T16:17:00.544ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
305262025-01-14T16:17:00.544ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
305272025-01-14T16:17:00.544ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
305282025-01-14T16:17:00.544ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = c2423ef0-728e-4895-997a-d9044c50948b
305292025-01-14T16:17:00.544ZINFOcrucible: client task is exiting = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
30530 [8810] got start action
30531 [8810] I am going to restart
30532 Starting downstairs at port 8810
30533 Make output file at "/tmp/dsc/downstairs-8810.txt"
30534 [8810][0] reports Starting
30535 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1230)
30536 [8810][0] reports Running
30537 ----------------------------------------------------------------
30538 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
30539 States: Offline Active Active
30540 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30541 9151 Acked Write 4 Sent Done Done false
30542 9152 Acked Write 4 Sent Done Done false
30543 9153 Acked Read 10 Sent Done Done false
30544 9154 Acked Read 5 Sent Done Done false
30545 9155 Acked Read 9 Sent Done Done false
30546 9156 Acked Flush 0 Sent Done Done false
30547 9157 Acked Write 7 Sent Done Done false
30548 9158 Acked Read 3 Sent Done Done false
30549 9159 Acked Read 8 Sent Done Done false
30550 9160 Acked Write 2 Sent Done Done false
30551 9161 Acked Read 1 Sent Done Done false
30552 9162 Acked Read 9 Sent Done Done false
30553 9163 Acked Flush 0 Sent Done Done false
30554 9164 Acked Read 7 Sent Done Done false
30555 9165 Acked Read 5 Sent Done Done false
30556 9166 Acked Flush 0 Sent Done Done false
30557 9167 Acked Read 4 Sent Done Done false
30558 9168 Acked Read 7 Sent Done Done false
30559 9169 Acked Write 8 Sent Done Done false
30560 9170 Acked Read 9 Sent Done Done false
30561 9171 Acked Flush 0 Sent Done Done false
30562 9172 Acked Write 4 Sent Done Done false
30563 9173 Acked Write 6 Sent Done Done false
30564 9174 Acked Read 10 Sent Done Done false
30565 9175 Acked Write 4 Sent Done Done false
30566 9176 Acked Write 6 Sent Done Done false
30567 9177 Acked Read 9 Sent Done Done false
30568 9178 Acked Write 3 Sent Done Done false
30569 9179 Acked Write 2 Sent Done Done false
30570 9180 Acked Write 9 Sent Done Done false
30571 9181 Acked Read 5 Sent Done Done false
30572 9182 Acked Read 2 Sent Done Done false
30573 9183 Acked Read 4 Sent Done Done false
30574 9184 Acked Read 3 Sent Done Done false
30575 9185 Acked Write 7 Sent Done Done false
30576 9186 Acked Read 6 Sent Done Done false
30577 9187 Acked Write 7 Sent Done Done false
30578 9188 Acked Write 2 Sent Done Done false
30579 9189 Acked Write 2 Sent Done Done false
30580 9190 Acked Write 3 Sent Done Done false
30581 9191 Acked Write 2 Sent Done Done false
30582 9192 Acked Read 6 Sent Done Done false
30583 9193 Acked Read 8 Sent Done Done false
30584 9194 Acked Flush 0 Sent Done Done false
30585 9195 Acked Write 4 Sent Done Done false
30586 9196 Acked Write 1 Sent Done Done false
30587 9197 Acked Write 5 Sent Done Done false
30588 9198 Acked Write 3 Sent Done Done false
30589 9199 Acked Write 1 Sent Done Done false
30590 9200 Acked Write 7 Sent Done Done false
30591 9201 Acked Write 6 Sent Done Done false
30592 9202 Acked Read 4 Sent Done Done false
30593 9203 Acked Read 1 Sent Done Done false
30594 9204 Acked Read 10 Sent Done Done false
30595 9205 Acked Read 9 Sent Done Done false
30596 9206 Acked Read 7 Sent Done Done false
30597 9207 Acked Write 2 Sent Done Done false
30598 9208 Acked Flush 0 Sent Done Done false
30599 9209 Acked Write 6 Sent Done Done false
30600 9210 Acked Write 10 Sent Done Done false
30601 9211 Acked Write 4 Sent Done Done false
30602 9212 Acked Flush 0 Sent Done Done false
30603 9213 Acked Read 10 Sent Done Done false
30604 9214 Acked Read 2 Sent Done Done false
30605 9215 Acked Write 8 Sent Done Done false
30606 9216 Acked Write 4 Sent Done Done false
30607 9217 Acked Read 3 Sent Done Done false
30608 9218 Acked Write 1 Sent Done Done false
30609 9219 Acked Read 10 Sent Done Done false
30610 9220 Acked Write 9 Sent Done Done false
30611 9221 Acked Read 2 Sent Done Done false
30612 9222 Acked Read 2 Sent Done Done false
30613 9223 Acked Read 9 Sent Done Done false
30614 9224 Acked Write 1 Sent Done Done false
30615 9225 Acked Write 6 Sent Done Done false
30616 9226 Acked Flush 0 Sent Done Done false
30617 9227 Acked Read 1 Sent Done Done false
30618 9228 Acked Write 1 Sent Done Done false
30619 9229 Acked Read 6 Sent Done Done false
30620 9230 Acked Read 9 Sent Done Done false
30621 9231 Acked Write 7 Sent Done Done false
30622 9232 Acked Read 9 Sent Done Done false
30623 9233 Acked Write 9 Sent Done Done false
30624 9234 Acked Read 4 Sent Done Done false
30625 9235 Acked Write 7 Sent Done Done false
30626 9236 Acked Read 1 Sent Done Done false
30627 9237 Acked Read 9 Sent Done Done false
30628 9238 Acked Read 5 Sent Done Done false
30629 9239 Acked Read 10 Sent Done Done false
30630 9240 Acked Read 9 Sent Done Done false
30631 9241 Acked Read 3 Sent Done Done false
30632 9242 Acked Read 1 Sent Done Done false
30633 9243 Acked Read 9 Sent Done Done false
30634 9244 Acked Read 5 Sent Done Done false
30635 9245 Acked Write 7 Sent Done Done false
30636 9246 Acked Write 5 Sent Done Done false
30637 9247 Acked Read 4 Sent Done Done false
30638 9248 Acked Write 3 Sent Done Done false
30639 9249 Acked Read 10 Sent Done Done false
30640 9250 Acked Read 8 Sent Done Done false
30641 9251 Acked Write 5 Sent Done Done false
30642 9252 Acked Write 3 Sent Done Done false
30643 9253 Acked Read 5 Sent Done Done false
30644 9254 Acked Read 2 Sent Done Done false
30645 9255 Acked Write 2 Sent Done Done false
30646 9256 Acked Write 4 Sent Done Done false
30647 9257 Acked Read 1 Sent Done Done false
30648 9258 Acked Write 7 Sent Done Done false
30649 9259 Acked Read 6 Sent Done Done false
30650 9260 Acked Flush 0 Sent Done Done false
30651 9261 Acked Read 9 Sent Done Done false
30652 9262 Acked Read 3 Sent Done Done false
30653 9263 Acked Read 4 Sent Done Done false
30654 9264 Acked Write 7 Sent Done Done false
30655 9265 Acked Read 9 Sent Done Done false
30656 9266 Acked Read 7 Sent Done Done false
30657 9267 Acked Write 10 Sent Done Done false
30658 9268 Acked Read 7 Sent Done Done false
30659 9269 Acked Flush 0 Sent Done Done false
30660 9270 Acked Write 5 Sent Done Done false
30661 9271 Acked Flush 0 Sent Done Done false
30662 9272 Acked Write 7 Sent Done Done false
30663 9273 Acked Read 6 Sent Done Done false
30664 9274 Acked Write 5 Sent Done Done false
30665 9275 Acked Read 10 Sent Done Done false
30666 9276 Acked Read 7 Sent Done Done false
30667 9277 Acked Read 1 Sent Done Done false
30668 9278 Acked Read 9 Sent Done Done false
30669 9279 Acked Read 4 Sent Done Done false
30670 9280 Acked Read 9 Sent Done Done false
30671 9281 Acked Read 6 Sent Done Done false
30672 9282 Acked Read 2 Sent Done Done false
30673 9283 Acked Read 5 Sent Done Done false
30674 9284 Acked Flush 0 Sent Done Done false
30675 9285 Acked Write 8 Sent Done Done false
30676 9286 Acked Write 3 Sent Done Done false
30677 9287 Acked Write 10 Sent Done Done false
30678 9288 Acked Flush 0 Sent Done Done false
30679 9289 Acked Flush 0 Sent Done Done false
30680 9290 Acked Write 7 Sent Done Done false
30681 9291 Acked Write 9 Sent Done Done false
30682 9292 Acked Write 7 Sent Done Done false
30683 9293 Acked Write 9 Sent Done Done false
30684 9294 Acked Read 3 Sent Done Done false
30685 9295 Acked Read 2 Sent Done Done false
30686 9296 Acked Flush 0 Sent Done Done false
30687 9297 Acked Read 2 Sent Done Done false
30688 9298 Acked Write 9 Sent Done Done false
30689 9299 Acked Read 4 Sent Done Done false
30690 9300 Acked Flush 0 Sent Done Done false
30691 9301 Acked Write 2 Sent Done Done false
30692 9302 Acked Flush 0 Sent Done Done false
30693 9303 Acked Write 8 Sent Done Done false
30694 9304 Acked Write 9 Sent Done Done false
30695 9305 Acked Read 5 Sent Done Done false
30696 9306 Acked Read 10 Sent Done Done false
30697 9307 Acked Read 6 Sent Done Done false
30698 9308 Acked Write 6 Sent Done Done false
30699 9309 Acked Read 4 Sent Done Done false
30700 9310 Acked Read 2 Sent Done Done false
30701 9311 Acked Flush 0 Sent Done Done false
30702 9312 Acked Read 3 Sent Done Done false
30703 9313 Acked Read 4 Sent Done Done false
30704 9314 Acked Read 2 Sent Done Done false
30705 9315 Acked Read 9 Sent Done Done false
30706 9316 Acked Write 6 Sent Done Done false
30707 9317 Acked Read 2 Sent Done Done false
30708 9318 Acked Write 6 Sent Done Done false
30709 9319 Acked Write 6 Sent Done Done false
30710 9320 Acked Read 10 Sent Done Done false
30711 9321 Acked Read 1 Sent Done Done false
30712 9322 Acked Write 4 Sent Done Done false
30713 9323 Acked Read 1 Sent Done Done false
30714 9324 Acked Read 8 Sent Done Done false
30715 9325 Acked Write 6 Sent Done Done false
30716 9326 Acked Read 4 Sent Done Done false
30717 9327 Acked Write 10 Sent Done Done false
30718 9328 Acked Read 7 Sent Done Done false
30719 9329 Acked Write 4 Sent Done Done false
30720 9330 Acked Read 1 Sent Done Done false
30721 9331 Acked Read 9 Sent Done Done false
30722 9332 Acked Read 5 Sent Done Done false
30723 9333 Acked Read 9 Sent Done Done false
30724 9334 Acked Write 6 Sent Done Done false
30725 9335 Acked Read 7 Sent Done Done false
30726 9336 Acked Read 3 Sent Done Done false
30727 9337 Acked Read 5 Sent Done Done false
30728 9338 Acked Read 5 Sent Done Done false
30729 9339 Acked Read 2 Sent Done Done false
30730 9340 Acked Read 5 Sent Done Done false
30731 9341 Acked Write 8 Sent Done Done false
30732 9342 Acked Write 10 Sent Done Done false
30733 9343 Acked Flush 0 Sent Done Done false
30734 9344 Acked Write 5 Sent Done Done false
30735 9345 Acked Write 6 Sent Done Done false
30736 9346 Acked Write 8 Sent Done Done false
30737 9347 Acked Write 4 Sent Done Done false
30738 9348 Acked Write 10 Sent Done Done false
30739 9349 Acked Write 7 Sent Done Done false
30740 9350 Acked Read 7 Sent Done Done false
30741 9351 Acked Write 4 Sent Done Done false
30742 9352 Acked Read 6 Sent Done Done false
30743 9353 Acked Read 1 Sent Done Done false
30744 9354 Acked Read 8 Sent Done Done false
30745 9355 Acked Read 6 Sent Done Done false
30746 9356 Acked Write 6 Sent Done Done false
30747 9357 Acked Write 8 Sent Done Done false
30748 9358 Acked Write 9 Sent Done Done false
30749 9359 Acked Flush 0 Sent Done Done false
30750 9360 Acked Read 5 Sent Done Done false
30751 9361 Acked Write 8 Sent Done Done false
30752 9362 Acked Read 5 Sent Done Done false
30753 9363 Acked Write 9 Sent Done Done false
30754 9364 Acked Write 6 Sent Done Done false
30755 9365 Acked Read 10 Sent Done Done false
30756 9366 Acked Read 1 Sent Done Done false
30757 9367 Acked Read 1 Sent Done Done false
30758 9368 Acked Read 2 Sent Done Done false
30759 9369 Acked Write 2 Sent Done Done false
30760 9370 Acked Read 3 Sent Done Done false
30761 9371 Acked Write 8 Sent Done Done false
30762 9372 Acked Write 1 Sent Done Done false
30763 9373 Acked Read 7 Sent Done Done false
30764 9374 Acked Write 8 Sent Done Done false
30765 9375 Acked Write 8 Sent Done Done false
30766 9376 Acked Write 3 Sent Done Done false
30767 9377 Acked Write 4 Sent Done Done false
30768 9378 Acked Read 1 Sent Done Done false
30769 9379 Acked Write 7 Sent Done Done false
30770 9380 Acked Read 5 Sent Done Done false
30771 9381 Acked Read 4 Sent Done Done false
30772 9382 Acked Write 8 Sent Done Done false
30773 9383 Acked Write 4 Sent Done Done false
30774 9384 Acked Read 4 Sent Done Done false
30775 9385 Acked Write 5 Sent Done Done false
30776 9386 Acked Write 4 Sent Done Done false
30777 9387 Acked Read 1 Sent Done Done false
30778 9388 Acked Read 3 Sent Done Done false
30779 9389 Acked Read 1 Sent Done Done false
30780 9390 Acked Write 8 Sent Done Done false
30781 9391 Acked Write 6 Sent Done Done false
30782 9392 Acked Write 5 Sent Done Done false
30783 9393 Acked Write 7 Sent Done Done false
30784 9394 Acked Read 1 Sent Done Done false
30785 9395 Acked Write 3 Sent Done Done false
30786 9396 Acked Write 3 Sent Done Done false
30787 9397 Acked Flush 0 Sent Done Done false
30788 9398 Acked Read 10 Sent Done Done false
30789 9399 Acked Write 5 Sent Done Done false
30790 9400 Acked Read 6 Sent Done Done false
30791 9401 Acked Write 5 Sent Done Done false
30792 9402 Acked Write 10 Sent Done Done false
30793 9403 Acked Read 6 Sent Done Done false
30794 9404 Acked Write 8 Sent Done Done false
30795 9405 Acked Read 10 Sent Done Done false
30796 9406 Acked Read 5 Sent Done Done false
30797 9407 Acked Read 1 Sent Done Done false
30798 9408 Acked Read 2 Sent Done Done false
30799 9409 Acked Flush 0 Sent Done Done false
30800 9410 Acked Read 6 Sent Done Done false
30801 9411 Acked Read 6 Sent Done Done false
30802 9412 Acked Read 8 Sent Done Done false
30803 9413 Acked Read 8 Sent Done Done false
30804 9414 Acked Read 2 Sent Done Done false
30805 9415 Acked Read 5 Sent Done Done false
30806 9416 Acked Write 4 Sent Done Done false
30807 9417 Acked Write 6 Sent Done Done false
30808 9418 Acked Write 2 Sent Done Done false
30809 9419 Acked Read 8 Sent Done Done false
30810 9420 Acked Flush 0 Sent Done Done false
30811 9421 Acked Read 1 Sent Done Done false
30812 9422 Acked Read 1 Sent Done Done false
30813 9423 Acked Write 8 Sent Done Done false
30814 9424 Acked Flush 0 Sent Done Done false
30815 9425 Acked Read 5 Sent Done Done false
30816 9426 Acked Write 2 Sent Done Done false
30817 9427 Acked Read 9 Sent Done Done false
30818 9428 Acked Read 5 Sent Done Done false
30819 9429 Acked Write 1 Sent Done Done false
30820 9430 Acked Write 4 Sent Done Done false
30821 9431 Acked Read 4 Sent Done Done false
30822 9432 Acked Read 3 Sent Done Done false
30823 9433 Acked Write 1 Sent Done Done false
30824 9434 Acked Read 2 Sent Done Done false
30825 9435 Acked Read 6 Sent Done Done false
30826 9436 Acked Write 3 Sent Done Done false
30827 9437 Acked Flush 0 Sent Done Done false
30828 9438 Acked Write 6 Sent Done Done false
30829 9439 Acked Read 4 Sent Done Done false
30830 9440 Acked Read 8 Sent Done Done false
30831 9441 Acked Flush 0 Sent Done Done false
30832 9442 Acked Write 7 Sent Done Done false
30833 9443 Acked Write 3 Sent Done Done false
30834 9444 Acked Write 3 Sent Done Done false
30835 9445 Acked Read 1 Sent Done Done false
30836 9446 Acked Read 9 Sent Done Done false
30837 9447 Acked Write 6 Sent Done Done false
30838 9448 Acked Write 7 Sent Done Done false
30839 9449 Acked Write 1 Sent Done Done false
30840 9450 Acked Write 6 Sent Done Done false
30841 9451 Acked Flush 0 Sent Done Done false
30842 STATES DS:0 DS:1 DS:2 TOTAL
30843 Sent 301 0 0 301
30844 Done 0 301 301 602
30845 Skipped 0 0 0 0
30846 Error 0 0 0 0
30847 Last Flush: 9150 9451 9451
30848 Downstairs last five completed: 9150 9149 9148 9147 9146
30849 Upstairs last five completed: 9451 9450 9449 9448 9447
30850 CLIENT: Up:0 ds:301 act:2
30851 ----------------------------------------------------------------
30852 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
30853 States: Offline Active Active
30854 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30855 9151 Acked Write 4 Sent Done Done false
30856 9152 Acked Write 4 Sent Done Done false
30857 9153 Acked Read 10 Sent Done Done false
30858 9154 Acked Read 5 Sent Done Done false
30859 9155 Acked Read 9 Sent Done Done false
30860 9156 Acked Flush 0 Sent Done Done false
30861 9157 Acked Write 7 Sent Done Done false
30862 9158 Acked Read 3 Sent Done Done false
30863 9159 Acked Read 8 Sent Done Done false
30864 9160 Acked Write 2 Sent Done Done false
30865 9161 Acked Read 1 Sent Done Done false
30866 9162 Acked Read 9 Sent Done Done false
30867 9163 Acked Flush 0 Sent Done Done false
30868 9164 Acked Read 7 Sent Done Done false
30869 9165 Acked Read 5 Sent Done Done false
30870 9166 Acked Flush 0 Sent Done Done false
30871 9167 Acked Read 4 Sent Done Done false
30872 9168 Acked Read 7 Sent Done Done false
30873 9169 Acked Write 8 Sent Done Done false
30874 9170 Acked Read 9 Sent Done Done false
30875 9171 Acked Flush 0 Sent Done Done false
30876 9172 Acked Write 4 Sent Done Done false
30877 9173 Acked Write 6 Sent Done Done false
30878 9174 Acked Read 10 Sent Done Done false
30879 9175 Acked Write 4 Sent Done Done false
30880 9176 Acked Write 6 Sent Done Done false
30881 9177 Acked Read 9 Sent Done Done false
30882 9178 Acked Write 3 Sent Done Done false
30883 9179 Acked Write 2 Sent Done Done false
30884 9180 Acked Write 9 Sent Done Done false
30885 9181 Acked Read 5 Sent Done Done false
30886 9182 Acked Read 2 Sent Done Done false
30887 9183 Acked Read 4 Sent Done Done false
30888 9184 Acked Read 3 Sent Done Done false
30889 9185 Acked Write 7 Sent Done Done false
30890 9186 Acked Read 6 Sent Done Done false
30891 9187 Acked Write 7 Sent Done Done false
30892 9188 Acked Write 2 Sent Done Done false
30893 9189 Acked Write 2 Sent Done Done false
30894 9190 Acked Write 3 Sent Done Done false
30895 9191 Acked Write 2 Sent Done Done false
30896 9192 Acked Read 6 Sent Done Done false
30897 9193 Acked Read 8 Sent Done Done false
30898 9194 Acked Flush 0 Sent Done Done false
30899 9195 Acked Write 4 Sent Done Done false
30900 9196 Acked Write 1 Sent Done Done false
30901 9197 Acked Write 5 Sent Done Done false
30902 9198 Acked Write 3 Sent Done Done false
30903 9199 Acked Write 1 Sent Done Done false
30904 9200 Acked Write 7 Sent Done Done false
30905 9201 Acked Write 6 Sent Done Done false
30906 9202 Acked Read 4 Sent Done Done false
30907 9203 Acked Read 1 Sent Done Done false
30908 9204 Acked Read 10 Sent Done Done false
30909 9205 Acked Read 9 Sent Done Done false
30910 9206 Acked Read 7 Sent Done Done false
30911 9207 Acked Write 2 Sent Done Done false
30912 9208 Acked Flush 0 Sent Done Done false
30913 9209 Acked Write 6 Sent Done Done false
30914 9210 Acked Write 10 Sent Done Done false
30915 9211 Acked Write 4 Sent Done Done false
30916 9212 Acked Flush 0 Sent Done Done false
30917 9213 Acked Read 10 Sent Done Done false
30918 9214 Acked Read 2 Sent Done Done false
30919 9215 Acked Write 8 Sent Done Done false
30920 9216 Acked Write 4 Sent Done Done false
30921 9217 Acked Read 3 Sent Done Done false
30922 9218 Acked Write 1 Sent Done Done false
30923 9219 Acked Read 10 Sent Done Done false
30924 9220 Acked Write 9 Sent Done Done false
30925 9221 Acked Read 2 Sent Done Done false
30926 9222 Acked Read 2 Sent Done Done false
30927 9223 Acked Read 9 Sent Done Done false
30928 9224 Acked Write 1 Sent Done Done false
30929 9225 Acked Write 6 Sent Done Done false
30930 9226 Acked Flush 0 Sent Done Done false
30931 9227 Acked Read 1 Sent Done Done false
30932 9228 Acked Write 1 Sent Done Done false
30933 9229 Acked Read 6 Sent Done Done false
30934 9230 Acked Read 9 Sent Done Done false
30935 9231 Acked Write 7 Sent Done Done false
30936 9232 Acked Read 9 Sent Done Done false
30937 9233 Acked Write 9 Sent Done Done false
30938 9234 Acked Read 4 Sent Done Done false
30939 9235 Acked Write 7 Sent Done Done false
30940 9236 Acked Read 1 Sent Done Done false
30941 9237 Acked Read 9 Sent Done Done false
30942 9238 Acked Read 5 Sent Done Done false
30943 9239 Acked Read 10 Sent Done Done false
30944 9240 Acked Read 9 Sent Done Done false
30945 9241 Acked Read 3 Sent Done Done false
30946 9242 Acked Read 1 Sent Done Done false
30947 9243 Acked Read 9 Sent Done Done false
30948 9244 Acked Read 5 Sent Done Done false
30949 9245 Acked Write 7 Sent Done Done false
30950 9246 Acked Write 5 Sent Done Done false
30951 9247 Acked Read 4 Sent Done Done false
30952 9248 Acked Write 3 Sent Done Done false
30953 9249 Acked Read 10 Sent Done Done false
30954 9250 Acked Read 8 Sent Done Done false
30955 9251 Acked Write 5 Sent Done Done false
30956 9252 Acked Write 3 Sent Done Done false
30957 9253 Acked Read 5 Sent Done Done false
30958 9254 Acked Read 2 Sent Done Done false
30959 9255 Acked Write 2 Sent Done Done false
30960 9256 Acked Write 4 Sent Done Done false
30961 9257 Acked Read 1 Sent Done Done false
30962 9258 Acked Write 7 Sent Done Done false
30963 9259 Acked Read 6 Sent Done Done false
30964 9260 Acked Flush 0 Sent Done Done false
30965 9261 Acked Read 9 Sent Done Done false
30966 9262 Acked Read 3 Sent Done Done false
30967 9263 Acked Read 4 Sent Done Done false
30968 9264 Acked Write 7 Sent Done Done false
30969 9265 Acked Read 9 Sent Done Done false
30970 9266 Acked Read 7 Sent Done Done false
30971 9267 Acked Write 10 Sent Done Done false
30972 9268 Acked Read 7 Sent Done Done false
30973 9269 Acked Flush 0 Sent Done Done false
30974 9270 Acked Write 5 Sent Done Done false
30975 9271 Acked Flush 0 Sent Done Done false
30976 9272 Acked Write 7 Sent Done Done false
30977 9273 Acked Read 6 Sent Done Done false
30978 9274 Acked Write 5 Sent Done Done false
30979 9275 Acked Read 10 Sent Done Done false
30980 9276 Acked Read 7 Sent Done Done false
30981 9277 Acked Read 1 Sent Done Done false
30982 9278 Acked Read 9 Sent Done Done false
30983 9279 Acked Read 4 Sent Done Done false
30984 9280 Acked Read 9 Sent Done Done false
30985 9281 Acked Read 6 Sent Done Done false
30986 9282 Acked Read 2 Sent Done Done false
30987 9283 Acked Read 5 Sent Done Done false
30988 9284 Acked Flush 0 Sent Done Done false
30989 9285 Acked Write 8 Sent Done Done false
30990 9286 Acked Write 3 Sent Done Done false
30991 9287 Acked Write 10 Sent Done Done false
30992 9288 Acked Flush 0 Sent Done Done false
30993 9289 Acked Flush 0 Sent Done Done false
30994 9290 Acked Write 7 Sent Done Done false
30995 9291 Acked Write 9 Sent Done Done false
30996 9292 Acked Write 7 Sent Done Done false
30997 9293 Acked Write 9 Sent Done Done false
30998 9294 Acked Read 3 Sent Done Done false
30999 9295 Acked Read 2 Sent Done Done false
31000 9296 Acked Flush 0 Sent Done Done false
31001 9297 Acked Read 2 Sent Done Done false
31002 9298 Acked Write 9 Sent Done Done false
31003 9299 Acked Read 4 Sent Done Done false
31004 9300 Acked Flush 0 Sent Done Done false
31005 9301 Acked Write 2 Sent Done Done false
31006 9302 Acked Flush 0 Sent Done Done false
31007 9303 Acked Write 8 Sent Done Done false
31008 9304 Acked Write 9 Sent Done Done false
31009 9305 Acked Read 5 Sent Done Done false
31010 9306 Acked Read 10 Sent Done Done false
31011 9307 Acked Read 6 Sent Done Done false
31012 9308 Acked Write 6 Sent Done Done false
31013 9309 Acked Read 4 Sent Done Done false
31014 9310 Acked Read 2 Sent Done Done false
31015 9311 Acked Flush 0 Sent Done Done false
31016 9312 Acked Read 3 Sent Done Done false
31017 9313 Acked Read 4 Sent Done Done false
31018 9314 Acked Read 2 Sent Done Done false
31019 9315 Acked Read 9 Sent Done Done false
31020 9316 Acked Write 6 Sent Done Done false
31021 9317 Acked Read 2 Sent Done Done false
31022 9318 Acked Write 6 Sent Done Done false
31023 9319 Acked Write 6 Sent Done Done false
31024 9320 Acked Read 10 Sent Done Done false
31025 9321 Acked Read 1 Sent Done Done false
31026 9322 Acked Write 4 Sent Done Done false
31027 9323 Acked Read 1 Sent Done Done false
31028 9324 Acked Read 8 Sent Done Done false
31029 9325 Acked Write 6 Sent Done Done false
31030 9326 Acked Read 4 Sent Done Done false
31031 9327 Acked Write 10 Sent Done Done false
31032 9328 Acked Read 7 Sent Done Done false
31033 9329 Acked Write 4 Sent Done Done false
31034 9330 Acked Read 1 Sent Done Done false
31035 9331 Acked Read 9 Sent Done Done false
31036 9332 Acked Read 5 Sent Done Done false
31037 9333 Acked Read 9 Sent Done Done false
31038 9334 Acked Write 6 Sent Done Done false
31039 9335 Acked Read 7 Sent Done Done false
31040 9336 Acked Read 3 Sent Done Done false
31041 9337 Acked Read 5 Sent Done Done false
31042 9338 Acked Read 5 Sent Done Done false
31043 9339 Acked Read 2 Sent Done Done false
31044 9340 Acked Read 5 Sent Done Done false
31045 9341 Acked Write 8 Sent Done Done false
31046 9342 Acked Write 10 Sent Done Done false
31047 9343 Acked Flush 0 Sent Done Done false
31048 9344 Acked Write 5 Sent Done Done false
31049 9345 Acked Write 6 Sent Done Done false
31050 9346 Acked Write 8 Sent Done Done false
31051 9347 Acked Write 4 Sent Done Done false
31052 9348 Acked Write 10 Sent Done Done false
31053 9349 Acked Write 7 Sent Done Done false
31054 9350 Acked Read 7 Sent Done Done false
31055 9351 Acked Write 4 Sent Done Done false
31056 9352 Acked Read 6 Sent Done Done false
31057 9353 Acked Read 1 Sent Done Done false
31058 9354 Acked Read 8 Sent Done Done false
31059 9355 Acked Read 6 Sent Done Done false
31060 9356 Acked Write 6 Sent Done Done false
31061 9357 Acked Write 8 Sent Done Done false
31062 9358 Acked Write 9 Sent Done Done false
31063 9359 Acked Flush 0 Sent Done Done false
31064 9360 Acked Read 5 Sent Done Done false
31065 9361 Acked Write 8 Sent Done Done false
31066 9362 Acked Read 5 Sent Done Done false
31067 9363 Acked Write 9 Sent Done Done false
31068 9364 Acked Write 6 Sent Done Done false
31069 9365 Acked Read 10 Sent Done Done false
31070 9366 Acked Read 1 Sent Done Done false
31071 9367 Acked Read 1 Sent Done Done false
31072 9368 Acked Read 2 Sent Done Done false
31073 9369 Acked Write 2 Sent Done Done false
31074 9370 Acked Read 3 Sent Done Done false
31075 9371 Acked Write 8 Sent Done Done false
31076 9372 Acked Write 1 Sent Done Done false
31077 9373 Acked Read 7 Sent Done Done false
31078 9374 Acked Write 8 Sent Done Done false
31079 9375 Acked Write 8 Sent Done Done false
31080 9376 Acked Write 3 Sent Done Done false
31081 9377 Acked Write 4 Sent Done Done false
31082 9378 Acked Read 1 Sent Done Done false
31083 9379 Acked Write 7 Sent Done Done false
31084 9380 Acked Read 5 Sent Done Done false
31085 9381 Acked Read 4 Sent Done Done false
31086 9382 Acked Write 8 Sent Done Done false
31087 9383 Acked Write 4 Sent Done Done false
31088 9384 Acked Read 4 Sent Done Done false
31089 9385 Acked Write 5 Sent Done Done false
31090 9386 Acked Write 4 Sent Done Done false
31091 9387 Acked Read 1 Sent Done Done false
31092 9388 Acked Read 3 Sent Done Done false
31093 9389 Acked Read 1 Sent Done Done false
31094 9390 Acked Write 8 Sent Done Done false
31095 9391 Acked Write 6 Sent Done Done false
31096 9392 Acked Write 5 Sent Done Done false
31097 9393 Acked Write 7 Sent Done Done false
31098 9394 Acked Read 1 Sent Done Done false
31099 9395 Acked Write 3 Sent Done Done false
31100 9396 Acked Write 3 Sent Done Done false
31101 9397 Acked Flush 0 Sent Done Done false
31102 9398 Acked Read 10 Sent Done Done false
31103 9399 Acked Write 5 Sent Done Done false
31104 9400 Acked Read 6 Sent Done Done false
31105 9401 Acked Write 5 Sent Done Done false
31106 9402 Acked Write 10 Sent Done Done false
31107 9403 Acked Read 6 Sent Done Done false
31108 9404 Acked Write 8 Sent Done Done false
31109 9405 Acked Read 10 Sent Done Done false
31110 9406 Acked Read 5 Sent Done Done false
31111 9407 Acked Read 1 Sent Done Done false
31112 9408 Acked Read 2 Sent Done Done false
31113 9409 Acked Flush 0 Sent Done Done false
31114 9410 Acked Read 6 Sent Done Done false
31115 9411 Acked Read 6 Sent Done Done false
31116 9412 Acked Read 8 Sent Done Done false
31117 9413 Acked Read 8 Sent Done Done false
31118 9414 Acked Read 2 Sent Done Done false
31119 9415 Acked Read 5 Sent Done Done false
31120 9416 Acked Write 4 Sent Done Done false
31121 9417 Acked Write 6 Sent Done Done false
31122 9418 Acked Write 2 Sent Done Done false
31123 9419 Acked Read 8 Sent Done Done false
31124 9420 Acked Flush 0 Sent Done Done false
31125 9421 Acked Read 1 Sent Done Done false
31126 9422 Acked Read 1 Sent Done Done false
31127 9423 Acked Write 8 Sent Done Done false
31128 9424 Acked Flush 0 Sent Done Done false
31129 9425 Acked Read 5 Sent Done Done false
31130 9426 Acked Write 2 Sent Done Done false
31131 9427 Acked Read 9 Sent Done Done false
31132 9428 Acked Read 5 Sent Done Done false
31133 9429 Acked Write 1 Sent Done Done false
31134 9430 Acked Write 4 Sent Done Done false
31135 9431 Acked Read 4 Sent Done Done false
31136 9432 Acked Read 3 Sent Done Done false
31137 9433 Acked Write 1 Sent Done Done false
31138 9434 Acked Read 2 Sent Done Done false
31139 9435 Acked Read 6 Sent Done Done false
31140 9436 Acked Write 3 Sent Done Done false
31141 9437 Acked Flush 0 Sent Done Done false
31142 9438 Acked Write 6 Sent Done Done false
31143 9439 Acked Read 4 Sent Done Done false
31144 9440 Acked Read 8 Sent Done Done false
31145 9441 Acked Flush 0 Sent Done Done false
31146 9442 Acked Write 7 Sent Done Done false
31147 9443 Acked Write 3 Sent Done Done false
31148 9444 Acked Write 3 Sent Done Done false
31149 9445 Acked Read 1 Sent Done Done false
31150 9446 Acked Read 9 Sent Done Done false
31151 9447 Acked Write 6 Sent Done Done false
31152 9448 Acked Write 7 Sent Done Done false
31153 9449 Acked Write 1 Sent Done Done false
31154 9450 Acked Write 6 Sent Done Done false
31155 9451 Acked Flush 0 Sent Done Done false
31156 STATES DS:0 DS:1 DS:2 TOTAL
31157 Sent 301 0 0 301
31158 Done 0 301 301 602
31159 Skipped 0 0 0 0
31160 Error 0 0 0 0
31161 Last Flush: 9150 9451 9451
31162 Downstairs last five completed: 9150 9149 9148 9147 9146
31163 Upstairs last five completed: 9451 9450 9449 9448 9447
31164 CLIENT: Up:0 ds:301 act:2
311652025-01-14T16:17:10.545ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
311662025-01-14T16:17:10.545ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42173) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
311672025-01-14T16:17:10.546ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
311682025-01-14T16:17:10.547ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
311692025-01-14T16:17:10.547ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
311702025-01-14T16:17:10.547ZINFOcrucible: send last flush ID to this DS: 9150 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
311712025-01-14T16:17:10.547ZINFOcrucible: Replied this last flush ID: 9150 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
311722025-01-14T16:17:10.547ZINFOcrucible: [0] client re-new 301 jobs since flush 9150 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
311732025-01-14T16:17:10.548ZINFOcrucible: [0] Replayed 301 jobs since flush: 9150 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
31174 ----------------------------------------------------------------
31175 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
31176 Downstairs last five completed: 9451 9450 9449 9448 9447
31177 Upstairs last five completed: 9451 9450 9449 9448 9447
31178 CLIENT: Up:0 ds:0 act:3
31179 Replay: All jobs finished
31180 Main task has work to do, go find it
31181 got dsc Stop(0)
31182 stop 0
31183 Jan 14 16:17:10.786 INFO request completed, latency_us: 335, response_code[8810] Got stop action so:false kr:false
31184 : 204, uri: /stop/cid/0, method: POST, req_id: dce6ad40-66ef-42ee-9d0e-ec295d459834, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
311852025-01-14T16:17:10.787ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
311862025-01-14T16:17:10.787ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
311872025-01-14T16:17:10.787ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
31188 Jan 14 16:17:10.787 INFO request completed, latency_us: 138, response_code: 200, uri: /state/cid/0, method: GET, req_id: b56d1f50-e7f8-4403-b637-e73995f0cda2, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
311892025-01-14T16:17:10.787ZINFOcrucible: client task is exiting = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
31190 [8810] Exited with: ExitStatus(unix_wait_status(9))
31191 [8810][0] reports Exit
31192 Jan 14 16:17:14.789 INFO request completed, latency_us: 197, response_code: 200, uri: /state/cid/0, method: GET, req_id: 1ab85a12-e525-40fd-9457-d2c90b5981da, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
31193 001/300 Read block 444 len 40960
31194 002/300 Read block 888 len 20480
31195 003/300 Write block 3150 len 24576 data: 8 7 7 6 7 8
31196 004/300 Write block 809 len 28672 data: 9 8 7 6 5 6 9
31197 005/300 Read block 1714 len 36864
31198 006/300 Write block 3248 len 28672 data: 5 3 3 3 4 6 6
31199 007/300 Write block 2502 len 36864 data: 5 4 4 4 6 6 7 6 8
31200 008/300 Read block 2440 len 24576
31201 009/300 Flush
31202 010/300 Read block 2736 len 28672
31203 011/300 Read block 1183 len 12288
31204 012/300 Read block 2197 len 4096
31205 013/300 Write block 2120 len 4096 data: 4
31206 014/300 Write block 4959 len 4096 data: 7
31207 015/300 Read block 2331 len 4096
31208 016/300 Read block 1293 len 12288
31209 017/300 Read block 1936 len 20480
31210 018/300 Read block 557 len 20480
31211 019/300 Write block 4116 len 32768 data: 2 2 3 3 5 5 6 6
31212 020/300 Flush
31213 021/300 Read block 4087 len 12288
31214 022/300 Read block 4585 len 12288
31215 023/300 Write block 1024 len 28672 data: 7 6 6 4 4 5 5
31216 024/300 Write block 206 len 36864 data: 7 9 8 8 7 7 8 8 9
31217 025/300 Read block 3955 len 32768
31218 026/300 Write block 1877 len 28672 data: 6 6 5 5 5 5 5
31219 027/300 Read block 737 len 28672
31220 028/300 Read block 4547 len 24576
31221 029/300 Read block 3598 len 4096
31222 030/300 Write block 1079 len 32768 data: 5 5 7 5 5 5 5 5
31223 031/300 Read block 3415 len 20480
31224 032/300 Flush
31225 033/300 Write block 871 len 20480 data: 5 5 5 3 4
31226 034/300 Write block 2323 len 12288 data: 7 9 9
31227 035/300 Read block 2228 len 12288
31228 036/300 Read block 69 len 16384
31229 037/300 Read block 1691 len 16384
31230 038/300 Write block 2483 len 28672 data: 6 6 6 6 5 3 5
31231 039/300 Read block 72 len 36864
31232 040/300 Read block 1297 len 16384
31233 041/300 Write block 3369 len 36864 data: 8 7 7 7 7 6 4 2 4
31234 042/300 Read block 4911 len 28672
31235 043/300 Read block 4041 len 8192
31236 044/300 Read block 741 len 4096
31237 045/300 Write block 1831 len 40960 data: 5 5 7 6 7 7 8 10 9 7
31238 046/300 Write block 4792 len 8192 data: 5 6
31239 047/300 Read block 3482 len 8192
31240 048/300 Flush
31241 049/300 Write block 3455 len 24576 data: 6 6 6 5 4 5
31242 050/300 Read block 241 len 16384
31243 051/300 Write block 3147 len 16384 data: 4 6 8 9
31244 052/300 Read block 4775 len 8192
31245 053/300 Write block 1310 len 28672 data: 4 4 4 5 5 4 4
31246 054/300 Read block 3220 len 12288
31247 055/300 Write block 4158 len 12288 data: 7 10 9
31248 056/300 Write block 675 len 12288 data: 9 8 7
31249 057/300 Read block 1054 len 36864
31250 058/300 Write block 3684 len 12288 data: 6 7 6
31251 059/300 Write block 3996 len 20480 data: 9 7 9 11 9
31252 060/300 Read block 4785 len 32768
31253 061/300 Read block 2861 len 16384
31254 062/300 Write block 2170 len 36864 data: 7 7 7 6 7 7 6 4 3
31255 063/300 Write block 2253 len 32768 data: 4 5 6 5 6 5 6 6
31256 064/300 Write block 87 len 16384 data: 5 5 4 5
31257 065/300 Read block 841 len 4096
31258 066/300 Write block 3006 len 32768 data: 8 7 5 5 6 5 5 3
31259 067/300 Read block 1657 len 12288
31260 068/300 Read block 2231 len 24576
31261 069/300 Write block 65 len 32768 data: 7 6 5 6 4 6 6 7
31262 070/300 Read block 2731 len 24576
31263 071/300 Flush
31264 072/300 Read block 1762 len 28672
31265 073/300 Write block 3735 len 20480 data: 6 7 6 6 7
31266 074/300 Read block 3736 len 4096
31267 075/300 Read block 4240 len 24576
31268 076/300 Write block 4407 len 12288 data: 4 5 4
31269 077/300 Read block 3146 len 16384
31270 078/300 Read block 647 len 36864
31271 079/300 Write block 4855 len 4096 data: 5
31272 080/300 Read block 4710 len 32768
31273 081/300 Read block 1566 len 28672
31274 082/300 Write block 2018 len 40960 data: 6 7 7 6 5 5 6 5 6 6
31275 083/300 Write block 4436 len 28672 data: 7 5 4 5 5 4 4
31276 084/300 Read block 2590 len 36864
31277 085/300 Flush
31278 086/300 Flush
31279 087/300 Read block 2038 len 40960
31280 088/300 Read block 3585 len 20480
31281 089/300 Read block 1975 len 4096
31282 090/300 Read block 3439 len 8192
31283 091/300 Write block 2539 len 20480 data: 5 5 5 5 3
31284 092/300 Write block 1237 len 20480 data: 7 8 8 7 6
31285 093/300 Write block 129 len 40960 data: 7 7 9 9 8 8 9 7 7 7
31286 094/300 Write block 1949 len 32768 data: 8 9 11 9 7 8 9 8
31287 095/300 Read block 707 len 12288
31288 096/300 Read block 3071 len 36864
31289 097/300 Read block 1986 len 40960
31290 098/300 Read block 3684 len 16384
31291 099/300 Flush
31292 100/300 Flush
31293 101/300 Read block 2862 len 20480
31294 102/300 Write block 4150 len 12288 data: 2 2 3
31295 103/300 Read block 2603 len 32768
31296 104/300 Write block 2544 len 20480 data: 3 3 3 6 8
31297 105/300 Write block 4447 len 20480 data: 2 3 2 4 4
31298 106/300 Write block 3107 len 4096 data: 5
31299 107/300 Write block 3354 len 32768 data: 7 6 5 5 5 4 5 5
31300 108/300 Write block 1491 len 32768 data: 7 7 7 7 8 8 8 6
31301 109/300 Read block 4477 len 28672
31302 110/300 Read block 4501 len 28672
31303 111/300 Read block 4045 len 20480
31304 112/300 Write block 1749 len 8192 data: 9 8
31305 113/300 Write block 3226 len 24576 data: 6 8 7 6 6 6
31306 114/300 Flush
31307 115/300 Write block 3224 len 12288 data: 7 7 7
31308 116/300 Write block 1303 len 32768 data: 5 6 7 5 4 5 4 5
31309 117/300 Write block 2441 len 24576 data: 3 3 3 2 2 3
31310 118/300 Flush
31311 119/300 Read block 4443 len 20480
31312 120/300 Write block 3756 len 16384 data: 5 6 4 4
31313 121/300 Read block 1495 len 28672
31314 122/300 Read block 2659 len 16384
31315 123/300 Read block 4353 len 24576
31316 124/300 Read block 3689 len 36864
31317 125/300 Write block 718 len 40960 data: 4 4 3 4 3 3 4 6 5 4
31318 126/300 Write block 497 len 24576 data: 6 5 4 3 4 4
31319 127/300 Write block 3585 len 20480 data: 8 5 5 6 5
31320 128/300 Write block 3836 len 40960 data: 8 6 3 3 4 4 6 6 8 8
31321 129/300 Read block 746 len 32768
31322 130/300 Write block 919 len 24576 data: 7 7 6 7 7 7
31323 131/300 Flush
31324 132/300 Write block 3360 len 8192 data: 6 6
31325 133/300 Read block 2063 len 24576
31326 134/300 Write block 4802 len 32768 data: 6 6 7 6 6 7 6 5
31327 135/300 Write block 1567 len 8192 data: 8 9
31328 136/300 Flush
31329 137/300 Write block 4041 len 16384 data: 6 5 7 9
31330 138/300 Write block 2449 len 32768 data: 3 3 5 5 6 7 4 4
31331 139/300 Write block 2753 len 16384 data: 7 6 5 4
31332 140/300 Flush
31333 141/300 Write block 265 len 16384 data: 5 5 4 3
31334 142/300 Flush
31335 143/300 Read block 734 len 16384
31336 144/300 Flush
31337 145/300 Write block 4983 len 4096 data: 8
31338 146/300 Write block 3507 len 4096 data: 5
31339 147/300 Write block 3170 len 40960 data: 3 2 2 4 5 5 5 6 6 5
31340 148/300 Read block 3193 len 8192
31341 149/300 Read block 2235 len 16384
31342 150/300 Read block 3668 len 12288
31343 151/300 Read block 718 len 8192
31344 152/300 Read block 3307 len 36864
31345 153/300 Read block 413 len 4096
31346 154/300 Write block 3387 len 16384 data: 4 6 8 8
31347 155/300 Write block 4616 len 28672 data: 3 3 3 5 5 6 6
31348 156/300 Write block 1087 len 24576 data: 4 3 4 4 5 4
31349 157/300 Read block 1083 len 40960
31350 158/300 Read block 1935 len 28672
31351 159/300 Write block 70 len 8192 data: 7 7
31352 160/300 Write block 4635 len 32768 data: 5 6 7 8 8 8 10 10
31353 161/300 Write block 3300 len 12288 data: 5 5 5
31354 162/300 Write block 4797 len 28672 data: 6 8 6 6 6 7 7
31355 163/300 Write block 4349 len 36864 data: 8 10 10 9 10 8 6 6 5
31356 164/300 Read block 1773 len 16384
31357 165/300 Write block 4392 len 8192 data: 10 10
31358 166/300 Write block 3257 len 20480 data: 6 5 5 4 4
31359 167/300 Read block 2220 len 20480
31360 168/300 Flush
31361 169/300 Read block 4454 len 24576
31362 170/300 Write block 2581 len 36864 data: 8 11 9 9 8 8 6 4 3
31363 171/300 Write block 3767 len 12288 data: 8 7 8
31364 172/300 Write block 1898 len 24576 data: 5 4 6 6 6 8
31365 173/300 Read block 2158 len 36864
31366 174/300 Write block 1397 len 28672 data: 5 7 7 6 7 7 8
31367 175/300 Read block 649 len 8192
31368 176/300 Read block 18 len 28672
31369 177/300 Flush
31370 178/300 Read block 2988 len 28672
31371 179/300 Read block 3185 len 12288
31372 180/300 Flush
31373 181/300 Read block 2444 len 8192
31374 182/300 Write block 745 len 32768 data: 6 6 4 4 5 4 5 5
31375 183/300 Read block 1318 len 40960
31376 184/300 Read block 4785 len 4096
31377 185/300 Write block 2210 len 16384 data: 5 5 8 6
31378 186/300 Read block 1394 len 32768
31379 187/300 Write block 4057 len 4096 data: 7
31380 188/300 Read block 416 len 4096
31381 189/300 Write block 51 len 40960 data: 5 5 5 5 5 8 8 10 8 9
31382 190/300 Flush
31383 191/300 Read block 802 len 20480
31384 192/300 Write block 3985 len 24576 data: 5 6 5 6 5 6
31385 193/300 Write block 2695 len 40960 data: 11 11 12 10 9 7 7 8 10 6
31386 194/300 Write block 177 len 8192 data: 4 5
31387 195/300 Read block 773 len 24576
31388 196/300 Read block 1549 len 28672
31389 197/300 Write block 3672 len 16384 data: 6 6 5 5
31390 198/300 Read block 1893 len 40960
31391 199/300 Write block 3303 len 24576 data: 6 5 6 6 5 5
31392 200/300 Write block 3688 len 12288 data: 8 8 7
31393 201/300 Read block 1069 len 36864
31394 202/300 Read block 1310 len 32768
31395 203/300 Read block 1911 len 24576
31396 204/300 Read block 1931 len 32768
31397 205/300 Read block 807 len 24576
31398 206/300 Read block 2833 len 16384
31399 207/300 Read block 3621 len 12288
31400 208/300 Write block 1338 len 28672 data: 5 5 4 4 3 4 4
31401 209/300 Read block 3920 len 28672
31402 210/300 Read block 1704 len 4096
31403 211/300 Read block 2468 len 20480
31404 212/300 Read block 1547 len 40960
31405 213/300 Read block 3695 len 8192
31406 214/300 Read block 396 len 24576
31407 215/300 Read block 2071 len 40960
31408 216/300 Read block 3252 len 8192
31409 217/300 Flush
31410 218/300 Read block 4933 len 28672
31411 219/300 Read block 4070 len 40960
31412 220/300 Write block 149 len 28672 data: 9 8 7 6 7 6 5
31413 221/300 Flush
31414 222/300 Read block 1399 len 24576
31415 223/300 Read block 3456 len 40960
31416 224/300 Write block 4164 len 12288 data: 10 6 8
31417 225/300 Write block 1138 len 8192 data: 4 3
31418 226/300 Write block 1123 len 28672 data: 6 6 7 6 7 6 6
31419 227/300 Flush
31420 228/300 Read block 3229 len 40960
31421 229/300 Write block 4945 len 28672 data: 8 6 5 5 6 6 6
31422 230/300 Read block 3532 len 8192
31423 231/300 Write block 368 len 12288 data: 4 6 6
31424 232/300 Write block 3099 len 36864 data: 4 4 7 6 6 6 4 5 6
31425 233/300 Write block 4658 len 12288 data: 5 7 7
31426 234/300 Flush
31427 235/300 Read block 469 len 32768
31428 236/300 Write block 4654 len 8192 data: 5 5
31429 237/300 Write block 2522 len 28672 data: 3 3 3 4 4 4 4
31430 238/300 Write block 2635 len 8192 data: 6 6
31431 239/300 Write block 3711 len 32768 data: 5 4 5 5 5 6 7 8
31432 240/300 Read block 2867 len 40960
31433 241/300 Write block 2300 len 36864 data: 3 3 2 4 4 4 6 8 8
31434 242/300 Write block 4272 len 8192 data: 4 6
31435 243/300 Write block 4612 len 24576 data: 5 4 4 4 4 4
31436 244/300 Write block 1622 len 8192 data: 5 4
31437 245/300 Flush
31438 246/300 Read block 4819 len 16384
31439 247/300 Read block 3230 len 12288
31440 248/300 Read block 1201 len 20480
31441 249/300 Read block 974 len 20480
31442 250/300 Read block 2879 len 12288
31443 251/300 Read block 1426 len 8192
31444 252/300 Write block 4953 len 8192 data: 8 7
31445 253/300 Read block 2055 len 4096
31446 254/300 Read block 1682 len 40960
31447 255/300 Read block 711 len 20480
31448 256/300 Read block 2033 len 32768
31449 257/300 Read block 3160 len 12288
31450 258/300 Write block 1822 len 12288 data: 5 4 4
31451 259/300 Read block 4095 len 16384
31452 260/300 Write block 2964 len 40960 data: 4 4 5 7 8 9 8 8 7 7
31453 261/300 Read block 180 len 28672
31454 262/300 Write block 3786 len 20480 data: 4 6 6 6 7
31455 263/300 Write block 567 len 20480 data: 5 6 6 4 3
31456 264/300 Flush
31457 265/300 Write block 393 len 36864 data: 5 4 4 4 4 4 6 6 8
31458 266/300 Read block 2190 len 24576
31459 267/300 Flush
31460 268/300 Read block 1259 len 40960
31461 269/300 Read block 1363 len 8192
31462 270/300 Flush
31463 271/300 Write block 1031 len 16384 data: 6 6 7 6
31464 272/300 Read block 4363 len 16384
31465 273/300 Read block 1109 len 40960
31466 274/300 Read block 4594 len 12288
31467 275/300 Write block 266 len 20480 data: 6 5 4 4 4
31468 276/300 Read block 3181 len 16384
31469 277/300 Write block 949 len 40960 data: 5 5 5 4 2 4 3 4 5 5
31470 278/300 Read block 3337 len 8192
31471 279/300 Read block 1356 len 36864
31472 280/300 Flush
31473 281/300 Write block 3263 len 36864 data: 7 6 6 6 6 7 7 6 5
31474 282/300 Read block 1622 len 36864
31475 283/300 Read block 4974 len 4096
31476 284/300 Write block 2565 len 8192 data: 7 8
31477 285/300 Write block 143 len 16384 data: 10 10 11 10
31478 286/300 Write block 2751 len 20480 data: 6 6 8 7 6
31479 287/300 Write block 4709 len 4096 data: 2
31480 288/300 Read block 570 len 28672
31481 289/300 Read block 3185 len 8192
31482 290/300 Write block 503 len 12288 data: 4 5 5
31483 291/300 Read block 994 len 16384
31484 292/300 Write block 2203 len 36864 data: 8 8 9 8 8 8 7 6 6
31485 293/300 Read block 104 len 36864
31486 294/300 Read block 1833 len 28672
31487 295/300 Write block 1309 len 32768 data: 5 6 5 5 6 6 5 5
31488 296/300 Read block 4738 len 12288
31489 297/300 Read block 2316 len 4096
31490 298/300 Write block 4935 len 16384 data: 5 5 5 6
31491 299/300 Read block 4381 len 16384
31492 300/300 Read block 4692 len 20480
31493 301/300 Write block 4250 len 32768 data: 7 7 4 3 3 4 4 3
31494 Main task has work to do, go find it
31495 got dsc Start(0)
31496 start 0
31497 Jan 14 16:17:15.019 INFO request completed, latency_us: 217, response_code: 204, uri: /start/cid/0, method: POST, req_id: e58d047e-fde7-45a2-8ea3-3955ac1e5af7, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
31498 Replay: started 0, returned:Ok(())
31499 ----------------------------------------------------------------
31500 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
31501 States: Offline Active Active
31502 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31503 9452 Acked Read 10 Sent Done Done false
31504 9453 Acked Read 5 Sent Done Done false
31505 9454 Acked Write 6 Sent Done Done false
31506 9455 Acked Write 7 Sent Done Done false
31507 9456 Acked Read 9 Sent Done Done false
31508 9457 Acked Write 7 Sent Done Done false
31509 9458 Acked Write 9 Sent Done Done false
31510 9459 Acked Read 6 Sent Done Done false
31511 9460 Acked Flush 0 Sent Done Done false
31512 9461 Acked Read 7 Sent Done Done false
31513 9462 Acked Read 3 Sent Done Done false
31514 9463 Acked Read 1 Sent Done Done false
31515 9464 Acked Write 1 Sent Done Done false
31516 9465 Acked Write 1 Sent Done Done false
31517 9466 Acked Read 1 Sent Done Done false
31518 9467 Acked Read 3 Sent Done Done false
31519 9468 Acked Read 5 Sent Done Done false
31520 9469 Acked Read 5 Sent Done Done false
31521 9470 Acked Write 8 Sent Done Done false
31522 9471 Acked Flush 0 Sent Done Done false
31523 9472 Acked Read 3 Sent Done Done false
31524 9473 Acked Read 3 Sent Done Done false
31525 9474 Acked Write 7 Sent Done Done false
31526 9475 Acked Write 9 Sent Done Done false
31527 9476 Acked Read 8 Sent Done Done false
31528 9477 Acked Write 7 Sent Done Done false
31529 9478 Acked Read 7 Sent Done Done false
31530 9479 Acked Read 6 Sent Done Done false
31531 9480 Acked Read 1 Sent Done Done false
31532 9481 Acked Write 8 Sent Done Done false
31533 9482 Acked Read 5 Sent Done Done false
31534 9483 Acked Flush 0 Sent Done Done false
31535 9484 Acked Write 5 Sent Done Done false
31536 9485 Acked Write 3 Sent Done Done false
31537 9486 Acked Read 3 Sent Done Done false
31538 9487 Acked Read 4 Sent Done Done false
31539 9488 Acked Read 4 Sent Done Done false
31540 9489 Acked Write 7 Sent Done Done false
31541 9490 Acked Read 9 Sent Done Done false
31542 9491 Acked Read 4 Sent Done Done false
31543 9492 Acked Write 9 Sent Done Done false
31544 9493 Acked Read 7 Sent Done Done false
31545 9494 Acked Read 2 Sent Done Done false
31546 9495 Acked Read 1 Sent Done Done false
31547 9496 Acked Write 10 Sent Done Done false
31548 9497 Acked Write 2 Sent Done Done false
31549 9498 Acked Read 2 Sent Done Done false
31550 9499 Acked Flush 0 Sent Done Done false
31551 9500 Acked Write 6 Sent Done Done false
31552 9501 Acked Read 4 Sent Done Done false
31553 9502 Acked Write 4 Sent Done Done false
31554 9503 Acked Read 2 Sent Done Done false
31555 9504 Acked Write 7 Sent Done Done false
31556 9505 Acked Read 3 Sent Done Done false
31557 9506 Acked Write 3 Sent Done Done false
31558 9507 Acked Write 3 Sent Done Done false
31559 9508 Acked Read 9 Sent Done Done false
31560 9509 Acked Write 3 Sent Done Done false
31561 9510 Acked Write 5 Sent Done Done false
31562 9511 Acked Read 8 Sent Done Done false
31563 9512 Acked Read 4 Sent Done Done false
31564 9513 Acked Write 9 Sent Done Done false
31565 9514 Acked Write 8 Sent Done Done false
31566 9515 Acked Write 4 Sent Done Done false
31567 9516 Acked Read 1 Sent Done Done false
31568 9517 Acked Write 8 Sent Done Done false
31569 9518 Acked Read 3 Sent Done Done false
31570 9519 Acked Read 6 Sent Done Done false
31571 9520 Acked Write 8 Sent Done Done false
31572 9521 Acked Read 6 Sent Done Done false
31573 9522 Acked Flush 0 Sent Done Done false
31574 9523 Acked Read 7 Sent Done Done false
31575 9524 Acked Write 5 Sent Done Done false
31576 9525 Acked Read 1 Sent Done Done false
31577 9526 Acked Read 6 Sent Done Done false
31578 9527 Acked Write 3 Sent Done Done false
31579 9528 Acked Read 4 Sent Done Done false
31580 9529 Acked Read 9 Sent Done Done false
31581 9530 Acked Write 1 Sent Done Done false
31582 9531 Acked Read 8 Sent Done Done false
31583 9532 Acked Read 7 Sent Done Done false
31584 9533 Acked Write 10 Sent Done Done false
31585 9534 Acked Write 7 Sent Done Done false
31586 9535 Acked Read 9 Sent Done Done false
31587 9536 Acked Flush 0 Sent Done Done false
31588 9537 Acked Flush 0 Sent Done Done false
31589 9538 Acked Read 10 Sent Done Done false
31590 9539 Acked Read 5 Sent Done Done false
31591 9540 Acked Read 1 Sent Done Done false
31592 9541 Acked Read 2 Sent Done Done false
31593 9542 Acked Write 5 Sent Done Done false
31594 9543 Acked Write 5 Sent Done Done false
31595 9544 Acked Write 10 Sent Done Done false
31596 9545 Acked Write 8 Sent Done Done false
31597 9546 Acked Read 3 Sent Done Done false
31598 9547 Acked Read 9 Sent Done Done false
31599 9548 Acked Read 10 Sent Done Done false
31600 9549 Acked Read 4 Sent Done Done false
31601 9550 Acked Flush 0 Sent Done Done false
31602 9551 Acked Flush 0 Sent Done Done false
31603 9552 Acked Read 5 Sent Done Done false
31604 9553 Acked Write 3 Sent Done Done false
31605 9554 Acked Read 8 Sent Done Done false
31606 9555 Acked Write 5 Sent Done Done false
31607 9556 Acked Write 5 Sent Done Done false
31608 9557 Acked Write 1 Sent Done Done false
31609 9558 Acked Write 8 Sent Done Done false
31610 9559 Acked Write 8 Sent Done Done false
31611 9560 Acked Read 7 Sent Done Done false
31612 9561 Acked Read 7 Sent Done Done false
31613 9562 Acked Read 5 Sent Done Done false
31614 9563 Acked Write 2 Sent Done Done false
31615 9564 Acked Write 6 Sent Done Done false
31616 9565 Acked Flush 0 Sent Done Done false
31617 9566 Acked Write 3 Sent Done Done false
31618 9567 Acked Write 8 Sent Done Done false
31619 9568 Acked Write 6 Sent Done Done false
31620 9569 Acked Flush 0 Sent Done Done false
31621 9570 Acked Read 5 Sent Done Done false
31622 9571 Acked Write 4 Sent Done Done false
31623 9572 Acked Read 7 Sent Done Done false
31624 9573 Acked Read 4 Sent Done Done false
31625 9574 Acked Read 6 Sent Done Done false
31626 9575 Acked Read 9 Sent Done Done false
31627 9576 Acked Write 10 Sent Done Done false
31628 9577 Acked Write 6 Sent Done Done false
31629 9578 Acked Write 5 Sent Done Done false
31630 9579 Acked Write 10 Sent Done Done false
31631 9580 Acked Read 8 Sent Done Done false
31632 9581 Acked Write 6 Sent Done Done false
31633 9582 Acked Flush 0 Sent Done Done false
31634 9583 Acked Write 2 Sent Done Done false
31635 9584 Acked Read 6 Sent Done Done false
31636 9585 Acked Write 8 Sent Done Done false
31637 9586 Acked Write 2 Sent Done Done false
31638 9587 Acked Flush 0 Sent Done Done false
31639 9588 Acked Write 4 Sent Done Done false
31640 9589 Acked Write 8 Sent Done Done false
31641 9590 Acked Write 4 Sent Done Done false
31642 9591 Acked Flush 0 Sent Done Done false
31643 9592 Acked Write 4 Sent Done Done false
31644 9593 Acked Flush 0 Sent Done Done false
31645 9594 Acked Read 4 Sent Done Done false
31646 9595 Acked Flush 0 Sent Done Done false
31647 9596 Acked Write 1 Sent Done Done false
31648 9597 Acked Write 1 Sent Done Done false
31649 9598 Acked Write 10 Sent Done Done false
31650 9599 Acked Read 2 Sent Done Done false
31651 9600 Acked Read 4 Sent Done Done false
31652 9601 Acked Read 3 Sent Done Done false
31653 9602 Acked Read 2 Sent Done Done false
31654 9603 Acked Read 9 Sent Done Done false
31655 9604 Acked Read 1 Sent Done Done false
31656 9605 Acked Write 4 Sent Done Done false
31657 9606 Acked Write 7 Sent Done Done false
31658 9607 Acked Write 6 Sent Done Done false
31659 9608 Acked Read 10 Sent Done Done false
31660 9609 Acked Read 7 Sent Done Done false
31661 9610 Acked Write 2 Sent Done Done false
31662 9611 Acked Write 8 Sent Done Done false
31663 9612 Acked Write 3 Sent Done Done false
31664 9613 Acked Write 7 Sent Done Done false
31665 9614 Acked Write 9 Sent Done Done false
31666 9615 Acked Read 4 Sent Done Done false
31667 9616 Acked Write 2 Sent Done Done false
31668 9617 Acked Write 5 Sent Done Done false
31669 9618 Acked Read 5 Sent Done Done false
31670 9619 Acked Flush 0 Sent Done Done false
31671 9620 Acked Read 6 Sent Done Done false
31672 9621 Acked Write 9 Sent Done Done false
31673 9622 Acked Write 3 Sent Done Done false
31674 9623 Acked Write 6 Sent Done Done false
31675 9624 Acked Read 9 Sent Done Done false
31676 9625 Acked Write 7 Sent Done Done false
31677 9626 Acked Read 2 Sent Done Done false
31678 9627 Acked Read 7 Sent Done Done false
31679 9628 Acked Flush 0 Sent Done Done false
31680 9629 Acked Read 7 Sent Done Done false
31681 9630 Acked Read 3 Sent Done Done false
31682 9631 Acked Flush 0 Sent Done Done false
31683 9632 Acked Read 2 Sent Done Done false
31684 9633 Acked Write 8 Sent Done Done false
31685 9634 Acked Read 10 Sent Done Done false
31686 9635 Acked Read 1 Sent Done Done false
31687 9636 Acked Write 4 Sent Done Done false
31688 9637 Acked Read 8 Sent Done Done false
31689 9638 Acked Write 1 Sent Done Done false
31690 9639 Acked Read 1 Sent Done Done false
31691 9640 Acked Write 10 Sent Done Done false
31692 9641 Acked Flush 0 Sent Done Done false
31693 9642 Acked Read 5 Sent Done Done false
31694 9643 Acked Write 6 Sent Done Done false
31695 9644 Acked Write 10 Sent Done Done false
31696 9645 Acked Write 2 Sent Done Done false
31697 9646 Acked Read 6 Sent Done Done false
31698 9647 Acked Read 7 Sent Done Done false
31699 9648 Acked Write 4 Sent Done Done false
31700 9649 Acked Read 10 Sent Done Done false
31701 9650 Acked Write 6 Sent Done Done false
31702 9651 Acked Write 3 Sent Done Done false
31703 9652 Acked Read 9 Sent Done Done false
31704 9653 Acked Read 8 Sent Done Done false
31705 9654 Acked Read 6 Sent Done Done false
31706 9655 Acked Read 8 Sent Done Done false
31707 9656 Acked Read 6 Sent Done Done false
31708 9657 Acked Read 4 Sent Done Done false
31709 9658 Acked Read 3 Sent Done Done false
31710 9659 Acked Write 7 Sent Done Done false
31711 9660 Acked Read 7 Sent Done Done false
31712 9661 Acked Read 1 Sent Done Done false
31713 9662 Acked Read 5 Sent Done Done false
31714 9663 Acked Read 10 Sent Done Done false
31715 9664 Acked Read 2 Sent Done Done false
31716 9665 Acked Read 6 Sent Done Done false
31717 9666 Acked Read 10 Sent Done Done false
31718 9667 Acked Read 2 Sent Done Done false
31719 9668 Acked Flush 0 Sent Done Done false
31720 9669 Acked Read 7 Sent Done Done false
31721 9670 Acked Read 10 Sent Done Done false
31722 9671 Acked Write 7 Sent Done Done false
31723 9672 Acked Flush 0 Sent Done Done false
31724 9673 Acked Read 6 Sent Done Done false
31725 9674 Acked Read 10 Sent Done Done false
31726 9675 Acked Write 3 Sent Done Done false
31727 9676 Acked Write 2 Sent Done Done false
31728 9677 Acked Write 7 Sent Done Done false
31729 9678 Acked Flush 0 Sent Done Done false
31730 9679 Acked Read 10 Sent Done Done false
31731 9680 Acked Write 7 Sent Done Done false
31732 9681 Acked Read 2 Sent Done Done false
31733 9682 Acked Write 3 Sent Done Done false
31734 9683 Acked Write 9 Sent Done Done false
31735 9684 Acked Write 3 Sent Done Done false
31736 9685 Acked Flush 0 Sent Done Done false
31737 9686 Acked Read 8 Sent Done Done false
31738 9687 Acked Write 2 Sent Done Done false
31739 9688 Acked Write 7 Sent Done Done false
31740 9689 Acked Write 2 Sent Done Done false
31741 9690 Acked Write 8 Sent Done Done false
31742 9691 Acked Read 10 Sent Done Done false
31743 9692 Acked Write 9 Sent Done Done false
31744 9693 Acked Write 2 Sent Done Done false
31745 9694 Acked Write 6 Sent Done Done false
31746 9695 Acked Write 2 Sent Done Done false
31747 9696 Acked Flush 0 Sent Done Done false
31748 9697 Acked Read 4 Sent Done Done false
31749 9698 Acked Read 3 Sent Done Done false
31750 9699 Acked Read 5 Sent Done Done false
31751 9700 Acked Read 5 Sent Done Done false
31752 9701 Acked Read 3 Sent Done Done false
31753 9702 Acked Read 2 Sent Done Done false
31754 9703 Acked Write 2 Sent Done Done false
31755 9704 Acked Read 1 Sent Done Done false
31756 9705 Acked Read 10 Sent Done Done false
31757 9706 Acked Read 5 Sent Done Done false
31758 9707 Acked Read 8 Sent Done Done false
31759 9708 Acked Read 3 Sent Done Done false
31760 9709 Acked Write 3 Sent Done Done false
31761 9710 Acked Read 4 Sent Done Done false
31762 9711 Acked Write 10 Sent Done Done false
31763 9712 Acked Read 7 Sent Done Done false
31764 9713 Acked Write 5 Sent Done Done false
31765 9714 Acked Write 5 Sent Done Done false
31766 9715 Acked Flush 0 Sent Done Done false
31767 9716 Acked Write 9 Sent Done Done false
31768 9717 Acked Read 6 Sent Done Done false
31769 9718 Acked Flush 0 Sent Done Done false
31770 9719 Acked Read 10 Sent Done Done false
31771 9720 Acked Read 2 Sent Done Done false
31772 9721 Acked Flush 0 Sent Done Done false
31773 9722 Acked Write 4 Sent Done Done false
31774 9723 Acked Read 4 Sent Done Done false
31775 9724 Acked Read 10 Sent Done Done false
31776 9725 Acked Read 3 Sent Done Done false
31777 9726 Acked Write 5 Sent Done Done false
31778 9727 Acked Read 4 Sent Done Done false
31779 9728 Acked Write 10 Sent Done Done false
31780 9729 Acked Read 2 Sent Done Done false
31781 9730 Acked Read 9 Sent Done Done false
31782 9731 Acked Flush 0 Sent Done Done false
31783 9732 Acked Write 9 Sent Done Done false
31784 9733 Acked Read 9 Sent Done Done false
31785 9734 Acked Read 1 Sent Done Done false
31786 9735 Acked Write 2 Sent Done Done false
31787 9736 Acked Write 4 Sent Done Done false
31788 9737 Acked Write 5 Sent Done Done false
31789 9738 Acked Write 1 Sent Done Done false
31790 9739 Acked Read 7 Sent Done Done false
31791 9740 Acked Read 2 Sent Done Done false
31792 9741 Acked Write 3 Sent Done Done false
31793 9742 Acked Read 4 Sent Done Done false
31794 9743 Acked Write 9 Sent Done Done false
31795 9744 Acked Read 9 Sent Done Done false
31796 9745 Acked Read 7 Sent Done Done false
31797 9746 Acked Write 8 Sent Done Done false
31798 9747 Acked Read 3 Sent Done Done false
31799 9748 Acked Read 1 Sent Done Done false
31800 9749 Acked Write 4 Sent Done Done false
31801 9750 Acked Read 4 Sent Done Done false
31802 9751 Acked Read 5 Sent Done Done false
31803 9752 Acked Write 8 Sent Done Done false
31804 STATES DS:0 DS:1 DS:2 TOTAL
31805 Sent 301 0 0 301
31806 Done 0 301 301 602
31807 Skipped 0 0 0 0
31808 Error 0 0 0 0
31809 Last Flush: 9451 9731 9731
31810 Downstairs last five completed: 9451 9450 9449 9448 9447
31811 Upstairs last five completed: 9752 9751 9750 9749 9748
31812 CLIENT: Up:0 ds:301 act:2
31813 [8810] got start action
31814 [8810] I am going to restart
31815 Starting downstairs at port 8810
31816 Make output file at "/tmp/dsc/downstairs-8810.txt"
31817 [8810][0] reports Starting
31818 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1231)
31819 [8810][0] reports Running
31820 ----------------------------------------------------------------
31821 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31822 States: Offline Active Active
31823 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31824 9452 Acked Read 10 Sent Done Done false
31825 9453 Acked Read 5 Sent Done Done false
31826 9454 Acked Write 6 Sent Done Done false
31827 9455 Acked Write 7 Sent Done Done false
31828 9456 Acked Read 9 Sent Done Done false
31829 9457 Acked Write 7 Sent Done Done false
31830 9458 Acked Write 9 Sent Done Done false
31831 9459 Acked Read 6 Sent Done Done false
31832 9460 Acked Flush 0 Sent Done Done false
31833 9461 Acked Read 7 Sent Done Done false
31834 9462 Acked Read 3 Sent Done Done false
31835 9463 Acked Read 1 Sent Done Done false
31836 9464 Acked Write 1 Sent Done Done false
31837 9465 Acked Write 1 Sent Done Done false
31838 9466 Acked Read 1 Sent Done Done false
31839 9467 Acked Read 3 Sent Done Done false
31840 9468 Acked Read 5 Sent Done Done false
31841 9469 Acked Read 5 Sent Done Done false
31842 9470 Acked Write 8 Sent Done Done false
31843 9471 Acked Flush 0 Sent Done Done false
31844 9472 Acked Read 3 Sent Done Done false
31845 9473 Acked Read 3 Sent Done Done false
31846 9474 Acked Write 7 Sent Done Done false
31847 9475 Acked Write 9 Sent Done Done false
31848 9476 Acked Read 8 Sent Done Done false
31849 9477 Acked Write 7 Sent Done Done false
31850 9478 Acked Read 7 Sent Done Done false
31851 9479 Acked Read 6 Sent Done Done false
31852 9480 Acked Read 1 Sent Done Done false
31853 9481 Acked Write 8 Sent Done Done false
31854 9482 Acked Read 5 Sent Done Done false
31855 9483 Acked Flush 0 Sent Done Done false
31856 9484 Acked Write 5 Sent Done Done false
31857 9485 Acked Write 3 Sent Done Done false
31858 9486 Acked Read 3 Sent Done Done false
31859 9487 Acked Read 4 Sent Done Done false
31860 9488 Acked Read 4 Sent Done Done false
31861 9489 Acked Write 7 Sent Done Done false
31862 9490 Acked Read 9 Sent Done Done false
31863 9491 Acked Read 4 Sent Done Done false
31864 9492 Acked Write 9 Sent Done Done false
31865 9493 Acked Read 7 Sent Done Done false
31866 9494 Acked Read 2 Sent Done Done false
31867 9495 Acked Read 1 Sent Done Done false
31868 9496 Acked Write 10 Sent Done Done false
31869 9497 Acked Write 2 Sent Done Done false
31870 9498 Acked Read 2 Sent Done Done false
31871 9499 Acked Flush 0 Sent Done Done false
31872 9500 Acked Write 6 Sent Done Done false
31873 9501 Acked Read 4 Sent Done Done false
31874 9502 Acked Write 4 Sent Done Done false
31875 9503 Acked Read 2 Sent Done Done false
31876 9504 Acked Write 7 Sent Done Done false
31877 9505 Acked Read 3 Sent Done Done false
31878 9506 Acked Write 3 Sent Done Done false
31879 9507 Acked Write 3 Sent Done Done false
31880 9508 Acked Read 9 Sent Done Done false
31881 9509 Acked Write 3 Sent Done Done false
31882 9510 Acked Write 5 Sent Done Done false
31883 9511 Acked Read 8 Sent Done Done false
31884 9512 Acked Read 4 Sent Done Done false
31885 9513 Acked Write 9 Sent Done Done false
31886 9514 Acked Write 8 Sent Done Done false
31887 9515 Acked Write 4 Sent Done Done false
31888 9516 Acked Read 1 Sent Done Done false
31889 9517 Acked Write 8 Sent Done Done false
31890 9518 Acked Read 3 Sent Done Done false
31891 9519 Acked Read 6 Sent Done Done false
31892 9520 Acked Write 8 Sent Done Done false
31893 9521 Acked Read 6 Sent Done Done false
31894 9522 Acked Flush 0 Sent Done Done false
31895 9523 Acked Read 7 Sent Done Done false
31896 9524 Acked Write 5 Sent Done Done false
31897 9525 Acked Read 1 Sent Done Done false
31898 9526 Acked Read 6 Sent Done Done false
31899 9527 Acked Write 3 Sent Done Done false
31900 9528 Acked Read 4 Sent Done Done false
31901 9529 Acked Read 9 Sent Done Done false
31902 9530 Acked Write 1 Sent Done Done false
31903 9531 Acked Read 8 Sent Done Done false
31904 9532 Acked Read 7 Sent Done Done false
31905 9533 Acked Write 10 Sent Done Done false
31906 9534 Acked Write 7 Sent Done Done false
31907 9535 Acked Read 9 Sent Done Done false
31908 9536 Acked Flush 0 Sent Done Done false
31909 9537 Acked Flush 0 Sent Done Done false
31910 9538 Acked Read 10 Sent Done Done false
31911 9539 Acked Read 5 Sent Done Done false
31912 9540 Acked Read 1 Sent Done Done false
31913 9541 Acked Read 2 Sent Done Done false
31914 9542 Acked Write 5 Sent Done Done false
31915 9543 Acked Write 5 Sent Done Done false
31916 9544 Acked Write 10 Sent Done Done false
31917 9545 Acked Write 8 Sent Done Done false
31918 9546 Acked Read 3 Sent Done Done false
31919 9547 Acked Read 9 Sent Done Done false
31920 9548 Acked Read 10 Sent Done Done false
31921 9549 Acked Read 4 Sent Done Done false
31922 9550 Acked Flush 0 Sent Done Done false
31923 9551 Acked Flush 0 Sent Done Done false
31924 9552 Acked Read 5 Sent Done Done false
31925 9553 Acked Write 3 Sent Done Done false
31926 9554 Acked Read 8 Sent Done Done false
31927 9555 Acked Write 5 Sent Done Done false
31928 9556 Acked Write 5 Sent Done Done false
31929 9557 Acked Write 1 Sent Done Done false
31930 9558 Acked Write 8 Sent Done Done false
31931 9559 Acked Write 8 Sent Done Done false
31932 9560 Acked Read 7 Sent Done Done false
31933 9561 Acked Read 7 Sent Done Done false
31934 9562 Acked Read 5 Sent Done Done false
31935 9563 Acked Write 2 Sent Done Done false
31936 9564 Acked Write 6 Sent Done Done false
31937 9565 Acked Flush 0 Sent Done Done false
31938 9566 Acked Write 3 Sent Done Done false
31939 9567 Acked Write 8 Sent Done Done false
31940 9568 Acked Write 6 Sent Done Done false
31941 9569 Acked Flush 0 Sent Done Done false
31942 9570 Acked Read 5 Sent Done Done false
31943 9571 Acked Write 4 Sent Done Done false
31944 9572 Acked Read 7 Sent Done Done false
31945 9573 Acked Read 4 Sent Done Done false
31946 9574 Acked Read 6 Sent Done Done false
31947 9575 Acked Read 9 Sent Done Done false
31948 9576 Acked Write 10 Sent Done Done false
31949 9577 Acked Write 6 Sent Done Done false
31950 9578 Acked Write 5 Sent Done Done false
31951 9579 Acked Write 10 Sent Done Done false
31952 9580 Acked Read 8 Sent Done Done false
31953 9581 Acked Write 6 Sent Done Done false
31954 9582 Acked Flush 0 Sent Done Done false
31955 9583 Acked Write 2 Sent Done Done false
31956 9584 Acked Read 6 Sent Done Done false
31957 9585 Acked Write 8 Sent Done Done false
31958 9586 Acked Write 2 Sent Done Done false
31959 9587 Acked Flush 0 Sent Done Done false
31960 9588 Acked Write 4 Sent Done Done false
31961 9589 Acked Write 8 Sent Done Done false
31962 9590 Acked Write 4 Sent Done Done false
31963 9591 Acked Flush 0 Sent Done Done false
31964 9592 Acked Write 4 Sent Done Done false
31965 9593 Acked Flush 0 Sent Done Done false
31966 9594 Acked Read 4 Sent Done Done false
31967 9595 Acked Flush 0 Sent Done Done false
31968 9596 Acked Write 1 Sent Done Done false
31969 9597 Acked Write 1 Sent Done Done false
31970 9598 Acked Write 10 Sent Done Done false
31971 9599 Acked Read 2 Sent Done Done false
31972 9600 Acked Read 4 Sent Done Done false
31973 9601 Acked Read 3 Sent Done Done false
31974 9602 Acked Read 2 Sent Done Done false
31975 9603 Acked Read 9 Sent Done Done false
31976 9604 Acked Read 1 Sent Done Done false
31977 9605 Acked Write 4 Sent Done Done false
31978 9606 Acked Write 7 Sent Done Done false
31979 9607 Acked Write 6 Sent Done Done false
31980 9608 Acked Read 10 Sent Done Done false
31981 9609 Acked Read 7 Sent Done Done false
31982 9610 Acked Write 2 Sent Done Done false
31983 9611 Acked Write 8 Sent Done Done false
31984 9612 Acked Write 3 Sent Done Done false
31985 9613 Acked Write 7 Sent Done Done false
31986 9614 Acked Write 9 Sent Done Done false
31987 9615 Acked Read 4 Sent Done Done false
31988 9616 Acked Write 2 Sent Done Done false
31989 9617 Acked Write 5 Sent Done Done false
31990 9618 Acked Read 5 Sent Done Done false
31991 9619 Acked Flush 0 Sent Done Done false
31992 9620 Acked Read 6 Sent Done Done false
31993 9621 Acked Write 9 Sent Done Done false
31994 9622 Acked Write 3 Sent Done Done false
31995 9623 Acked Write 6 Sent Done Done false
31996 9624 Acked Read 9 Sent Done Done false
31997 9625 Acked Write 7 Sent Done Done false
31998 9626 Acked Read 2 Sent Done Done false
31999 9627 Acked Read 7 Sent Done Done false
32000 9628 Acked Flush 0 Sent Done Done false
32001 9629 Acked Read 7 Sent Done Done false
32002 9630 Acked Read 3 Sent Done Done false
32003 9631 Acked Flush 0 Sent Done Done false
32004 9632 Acked Read 2 Sent Done Done false
32005 9633 Acked Write 8 Sent Done Done false
32006 9634 Acked Read 10 Sent Done Done false
32007 9635 Acked Read 1 Sent Done Done false
32008 9636 Acked Write 4 Sent Done Done false
32009 9637 Acked Read 8 Sent Done Done false
32010 9638 Acked Write 1 Sent Done Done false
32011 9639 Acked Read 1 Sent Done Done false
32012 9640 Acked Write 10 Sent Done Done false
32013 9641 Acked Flush 0 Sent Done Done false
32014 9642 Acked Read 5 Sent Done Done false
32015 9643 Acked Write 6 Sent Done Done false
32016 9644 Acked Write 10 Sent Done Done false
32017 9645 Acked Write 2 Sent Done Done false
32018 9646 Acked Read 6 Sent Done Done false
32019 9647 Acked Read 7 Sent Done Done false
32020 9648 Acked Write 4 Sent Done Done false
32021 9649 Acked Read 10 Sent Done Done false
32022 9650 Acked Write 6 Sent Done Done false
32023 9651 Acked Write 3 Sent Done Done false
32024 9652 Acked Read 9 Sent Done Done false
32025 9653 Acked Read 8 Sent Done Done false
32026 9654 Acked Read 6 Sent Done Done false
32027 9655 Acked Read 8 Sent Done Done false
32028 9656 Acked Read 6 Sent Done Done false
32029 9657 Acked Read 4 Sent Done Done false
32030 9658 Acked Read 3 Sent Done Done false
32031 9659 Acked Write 7 Sent Done Done false
32032 9660 Acked Read 7 Sent Done Done false
32033 9661 Acked Read 1 Sent Done Done false
32034 9662 Acked Read 5 Sent Done Done false
32035 9663 Acked Read 10 Sent Done Done false
32036 9664 Acked Read 2 Sent Done Done false
32037 9665 Acked Read 6 Sent Done Done false
32038 9666 Acked Read 10 Sent Done Done false
32039 9667 Acked Read 2 Sent Done Done false
32040 9668 Acked Flush 0 Sent Done Done false
32041 9669 Acked Read 7 Sent Done Done false
32042 9670 Acked Read 10 Sent Done Done false
32043 9671 Acked Write 7 Sent Done Done false
32044 9672 Acked Flush 0 Sent Done Done false
32045 9673 Acked Read 6 Sent Done Done false
32046 9674 Acked Read 10 Sent Done Done false
32047 9675 Acked Write 3 Sent Done Done false
32048 9676 Acked Write 2 Sent Done Done false
32049 9677 Acked Write 7 Sent Done Done false
32050 9678 Acked Flush 0 Sent Done Done false
32051 9679 Acked Read 10 Sent Done Done false
32052 9680 Acked Write 7 Sent Done Done false
32053 9681 Acked Read 2 Sent Done Done false
32054 9682 Acked Write 3 Sent Done Done false
32055 9683 Acked Write 9 Sent Done Done false
32056 9684 Acked Write 3 Sent Done Done false
32057 9685 Acked Flush 0 Sent Done Done false
32058 9686 Acked Read 8 Sent Done Done false
32059 9687 Acked Write 2 Sent Done Done false
32060 9688 Acked Write 7 Sent Done Done false
32061 9689 Acked Write 2 Sent Done Done false
32062 9690 Acked Write 8 Sent Done Done false
32063 9691 Acked Read 10 Sent Done Done false
32064 9692 Acked Write 9 Sent Done Done false
32065 9693 Acked Write 2 Sent Done Done false
32066 9694 Acked Write 6 Sent Done Done false
32067 9695 Acked Write 2 Sent Done Done false
32068 9696 Acked Flush 0 Sent Done Done false
32069 9697 Acked Read 4 Sent Done Done false
32070 9698 Acked Read 3 Sent Done Done false
32071 9699 Acked Read 5 Sent Done Done false
32072 9700 Acked Read 5 Sent Done Done false
32073 9701 Acked Read 3 Sent Done Done false
32074 9702 Acked Read 2 Sent Done Done false
32075 9703 Acked Write 2 Sent Done Done false
32076 9704 Acked Read 1 Sent Done Done false
32077 9705 Acked Read 10 Sent Done Done false
32078 9706 Acked Read 5 Sent Done Done false
32079 9707 Acked Read 8 Sent Done Done false
32080 9708 Acked Read 3 Sent Done Done false
32081 9709 Acked Write 3 Sent Done Done false
32082 9710 Acked Read 4 Sent Done Done false
32083 9711 Acked Write 10 Sent Done Done false
32084 9712 Acked Read 7 Sent Done Done false
32085 9713 Acked Write 5 Sent Done Done false
32086 9714 Acked Write 5 Sent Done Done false
32087 9715 Acked Flush 0 Sent Done Done false
32088 9716 Acked Write 9 Sent Done Done false
32089 9717 Acked Read 6 Sent Done Done false
32090 9718 Acked Flush 0 Sent Done Done false
32091 9719 Acked Read 10 Sent Done Done false
32092 9720 Acked Read 2 Sent Done Done false
32093 9721 Acked Flush 0 Sent Done Done false
32094 9722 Acked Write 4 Sent Done Done false
32095 9723 Acked Read 4 Sent Done Done false
32096 9724 Acked Read 10 Sent Done Done false
32097 9725 Acked Read 3 Sent Done Done false
32098 9726 Acked Write 5 Sent Done Done false
32099 9727 Acked Read 4 Sent Done Done false
32100 9728 Acked Write 10 Sent Done Done false
32101 9729 Acked Read 2 Sent Done Done false
32102 9730 Acked Read 9 Sent Done Done false
32103 9731 Acked Flush 0 Sent Done Done false
32104 9732 Acked Write 9 Sent Done Done false
32105 9733 Acked Read 9 Sent Done Done false
32106 9734 Acked Read 1 Sent Done Done false
32107 9735 Acked Write 2 Sent Done Done false
32108 9736 Acked Write 4 Sent Done Done false
32109 9737 Acked Write 5 Sent Done Done false
32110 9738 Acked Write 1 Sent Done Done false
32111 9739 Acked Read 7 Sent Done Done false
32112 9740 Acked Read 2 Sent Done Done false
32113 9741 Acked Write 3 Sent Done Done false
32114 9742 Acked Read 4 Sent Done Done false
32115 9743 Acked Write 9 Sent Done Done false
32116 9744 Acked Read 9 Sent Done Done false
32117 9745 Acked Read 7 Sent Done Done false
32118 9746 Acked Write 8 Sent Done Done false
32119 9747 Acked Read 3 Sent Done Done false
32120 9748 Acked Read 1 Sent Done Done false
32121 9749 Acked Write 4 Sent Done Done false
32122 9750 Acked Read 4 Sent Done Done false
32123 9751 Acked Read 5 Sent Done Done false
32124 9752 Acked Write 8 Sent Done Done false
32125 9753 Acked Flush 0 Sent Done Done false
32126 STATES DS:0 DS:1 DS:2 TOTAL
32127 Sent 302 0 0 302
32128 Done 0 302 302 604
32129 Skipped 0 0 0 0
32130 Error 0 0 0 0
32131 Last Flush: 9451 9753 9753
32132 Downstairs last five completed: 9451 9450 9449 9448 9447
32133 Upstairs last five completed: 9753 9752 9751 9750 9749
32134 CLIENT: Up:0 ds:302 act:2
321352025-01-14T16:17:20.788ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
321362025-01-14T16:17:20.788ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:46773) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
321372025-01-14T16:17:20.789ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
321382025-01-14T16:17:20.790ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
321392025-01-14T16:17:20.790ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
321402025-01-14T16:17:20.790ZINFOcrucible: send last flush ID to this DS: 9451 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
321412025-01-14T16:17:20.790ZINFOcrucible: Replied this last flush ID: 9451 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
321422025-01-14T16:17:20.790ZINFOcrucible: [0] client re-new 302 jobs since flush 9451 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
321432025-01-14T16:17:20.790ZINFOcrucible: [0] Replayed 302 jobs since flush: 9451 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
32144 ----------------------------------------------------------------
32145 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
32146 Downstairs last five completed: 9753 9752 9751 9750 9749
32147 Upstairs last five completed: 9753 9752 9751 9750 9749
32148 CLIENT: Up:0 ds:0 act:3
32149 Replay: All jobs finished
32150 Main task has work to do, go find it
32151 got dsc Stop(0)
32152 stop 0
32153 Jan 14 16:17:23.024[8810] Got stop action so:false kr:false
32154 INFO request completed, latency_us: 341, response_code: 204, uri: /stop/cid/0, method: POST, req_id: a9bea743-b21f-47a2-8aa5-5cda1ff095b1, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
321552025-01-14T16:17:23.024ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
321562025-01-14T16:17:23.024ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
321572025-01-14T16:17:23.024ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
32158 Jan 14 16:17:23.024 INFO request completed, latency_us: 137, response_code: 200, uri: /state/cid/0, method: GET, req_id: 1fc5ca4f-d486-4e08-96b5-24853eab0bc4, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
321592025-01-14T16:17:23.024ZINFOcrucible: client task is exiting = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
32160 [8810] Exited with: ExitStatus(unix_wait_status(9))
32161 [8810][0] reports Exit
32162 Jan 14 16:17:27.027 INFO request completed, latency_us: 181, response_code: 200, uri: /state/cid/0, method: GET, req_id: d1a97d60-1e19-497d-bb19-4fa55860be9c, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
32163 001/300 Write block 3653 len 32768 data: 3 3 3 3 4 5 3 4
32164 002/300 Read block 1858 len 4096
32165 003/300 Write block 1981 len 4096 data: 5
32166 004/300 Write block 4362 len 12288 data: 3 3 4
32167 005/300 Write block 4864 len 12288 data: 6 5 4
32168 006/300 Read block 1085 len 36864
32169 007/300 Read block 222 len 20480
32170 008/300 Flush
32171 009/300 Read block 3707 len 40960
32172 010/300 Write block 4805 len 8192 data: 7 7
32173 011/300 Read block 3602 len 20480
32174 012/300 Read block 201 len 32768
32175 013/300 Read block 2737 len 8192
32176 014/300 Write block 3748 len 16384 data: 3 3 3 2
32177 015/300 Write block 2908 len 24576 data: 11 9 9 7 6 4
32178 016/300 Read block 4461 len 12288
32179 017/300 Read block 3403 len 24576
32180 018/300 Read block 4916 len 28672
32181 019/300 Read block 332 len 32768
32182 020/300 Read block 4993 len 8192
32183 021/300 Flush
32184 022/300 Read block 4788 len 40960
32185 023/300 Read block 2558 len 36864
32186 024/300 Write block 829 len 40960 data: 3 2 2 2 3 5 5 7 9 8
32187 025/300 Read block 3691 len 24576
32188 026/300 Read block 2627 len 16384
32189 027/300 Write block 3684 len 4096 data: 7
32190 028/300 Read block 2441 len 16384
32191 029/300 Write block 2220 len 24576 data: 5 5 4 3 3 3
32192 030/300 Read block 3433 len 40960
32193 031/300 Write block 1686 len 28672 data: 6 7 7 6 5 6 6
32194 032/300 Read block 2888 len 40960
32195 033/300 Flush
32196 034/300 Write block 1302 len 4096 data: 6
32197 035/300 Write block 3630 len 16384 data: 8 7 5 8
32198 036/300 Flush
32199 037/300 Write block 2908 len 36864 data: 12 10 10 8 7 5 4 4 4
32200 038/300 Write block 4774 len 28672 data: 7 9 8 8 5 5 4
32201 039/300 Write block 4254 len 32768 data: 4 5 5 4 3 5 4 6
32202 040/300 Flush
32203 041/300 Read block 2086 len 20480
32204 042/300 Write block 662 len 8192 data: 4 4
32205 043/300 Write block 2602 len 40960 data: 7 6 6 7 7 5 6 5 4 6
32206 044/300 Write block 3550 len 28672 data: 4 5 5 5 5 6 5
32207 045/300 Write block 915 len 20480 data: 5 7 7 6 8
32208 046/300 Read block 2325 len 12288
32209 047/300 Read block 4504 len 32768
32210 048/300 Read block 2049 len 40960
32211 049/300 Write block 37 len 12288 data: 6 6 8
32212 050/300 Read block 521 len 16384
32213 051/300 Read block 215 len 16384
32214 052/300 Read block 2297 len 16384
32215 053/300 Read block 1013 len 20480
32216 054/300 Write block 333 len 16384 data: 4 6 7 5
32217 055/300 Read block 3936 len 16384
32218 056/300 Write block 4344 len 32768 data: 6 5 6 6 7 9 11 11
32219 057/300 Read block 386 len 32768
32220 058/300 Read block 2568 len 28672
32221 059/300 Write block 3274 len 36864 data: 6 6 7 6 6 6 7 8 10
32222 060/300 Write block 2372 len 28672 data: 5 5 5 6 8 9 9
32223 061/300 Write block 3146 len 24576 data: 4 5 7 9 10 8
32224 062/300 Write block 725 len 8192 data: 7 6
32225 063/300 Read block 4978 len 20480
32226 064/300 Write block 4595 len 24576 data: 4 3 4 5 5 5
32227 065/300 Write block 1214 len 40960 data: 3 4 4 6 7 8 8 8 8 6
32228 066/300 Write block 1951 len 32768 data: 12 10 8 9 10 9 8 5
32229 067/300 Read block 3785 len 20480
32230 068/300 Read block 1411 len 20480
32231 069/300 Write block 4685 len 32768 data: 3 5 5 5 5 6 6 7
32232 070/300 Write block 981 len 12288 data: 3 4 5
32233 071/300 Write block 4263 len 40960 data: 7 7 7 7 6 5 4 4 4 5
32234 072/300 Read block 4329 len 12288
32235 073/300 Write block 2214 len 20480 data: 6 6 7 8 7
32236 074/300 Flush
32237 075/300 Read block 4021 len 12288
32238 076/300 Read block 1977 len 40960
32239 077/300 Read block 3572 len 36864
32240 078/300 Write block 2208 len 12288 data: 9 8 7
32241 079/300 Flush
32242 080/300 Flush
32243 081/300 Write block 1776 len 16384 data: 4 4 4 3
32244 082/300 Flush
32245 083/300 Read block 3867 len 40960
32246 084/300 Read block 4306 len 24576
32247 085/300 Flush
32248 086/300 Flush
32249 087/300 Read block 2757 len 16384
32250 088/300 Flush
32251 089/300 Write block 411 len 36864 data: 5 4 4 3 4 5 6 7 6
32252 090/300 Read block 3729 len 8192
32253 091/300 Write block 2742 len 8192 data: 3 5
32254 092/300 Write block 432 len 12288 data: 5 5 5
32255 093/300 Write block 3961 len 36864 data: 8 8 6 5 4 5 5 5 5
32256 094/300 Write block 2503 len 12288 data: 5 5 5
32257 095/300 Flush
32258 096/300 Write block 3979 len 40960 data: 5 5 6 4 5 5 6 7 6 7
32259 097/300 Flush
32260 098/300 Flush
32261 099/300 Write block 1046 len 4096 data: 3
32262 100/300 Write block 734 len 36864 data: 3 2 2 2 5 5 5 7 7
32263 101/300 Write block 1958 len 20480 data: 6 5 5 6 6
32264 102/300 Flush
32265 103/300 Read block 411 len 36864
32266 104/300 Flush
32267 105/300 Read block 150 len 28672
32268 106/300 Flush
32269 107/300 Write block 1172 len 4096 data: 5
32270 108/300 Read block 1594 len 40960
32271 109/300 Flush
32272 110/300 Read block 185 len 24576
32273 111/300 Read block 1578 len 4096
32274 112/300 Read block 506 len 32768
32275 113/300 Write block 1783 len 12288 data: 5 5 5
32276 114/300 Read block 3534 len 40960
32277 115/300 Write block 4313 len 28672 data: 4 4 5 5 5 4 4
32278 116/300 Write block 1063 len 24576 data: 8 7 8 8 7 8
32279 117/300 Write block 462 len 32768 data: 8 8 7 6 5 6 4 4
32280 118/300 Write block 2584 len 32768 data: 10 9 9 7 5 4 3 4
32281 119/300 Read block 4129 len 4096
32282 120/300 Flush
32283 121/300 Write block 1763 len 28672 data: 3 4 4 6 6 7 9
32284 122/300 Read block 3770 len 40960
32285 123/300 Read block 3560 len 24576
32286 124/300 Read block 2402 len 24576
32287 125/300 Flush
32288 126/300 Read block 584 len 32768
32289 127/300 Read block 4011 len 20480
32290 128/300 Read block 4781 len 28672
32291 129/300 Read block 3302 len 4096
32292 130/300 Write block 363 len 4096 data: 7
32293 131/300 Read block 1705 len 40960
32294 132/300 Read block 3257 len 16384
32295 133/300 Read block 2541 len 28672
32296 134/300 Read block 1247 len 36864
32297 135/300 Write block 4046 len 16384 data: 11 9 7 8
32298 136/300 Read block 3678 len 12288
32299 137/300 Read block 2674 len 16384
32300 138/300 Write block 3673 len 8192 data: 7 6
32301 139/300 Write block 146 len 8192 data: 11 10
32302 140/300 Read block 3286 len 12288
32303 141/300 Read block 2703 len 20480
32304 142/300 Read block 0 len 12288
32305 143/300 Read block 2785 len 20480
32306 144/300 Write block 4418 len 28672 data: 8 7 8 7 7 8 8
32307 145/300 Read block 3556 len 20480
32308 146/300 Read block 676 len 24576
32309 147/300 Flush
32310 148/300 Read block 989 len 16384
32311 149/300 Read block 991 len 24576
32312 150/300 Read block 1482 len 12288
32313 151/300 Read block 3359 len 28672
32314 152/300 Write block 137 len 8192 data: 8 8
32315 153/300 Read block 4807 len 12288
32316 154/300 Write block 2624 len 24576 data: 9 7 6 5 3 3
32317 155/300 Flush
32318 156/300 Write block 4096 len 16384 data: 4 6 6 7
32319 157/300 Read block 4651 len 32768
32320 158/300 Read block 3766 len 16384
32321 159/300 Write block 4186 len 28672 data: 3 3 5 6 5 4 3
32322 160/300 Read block 2411 len 8192
32323 161/300 Read block 2806 len 32768
32324 162/300 Write block 2216 len 4096 data: 8
32325 163/300 Read block 664 len 16384
32326 164/300 Read block 2063 len 12288
32327 165/300 Read block 446 len 40960
32328 166/300 Read block 2188 len 28672
32329 167/300 Write block 4676 len 8192 data: 6 7
32330 168/300 Flush
32331 169/300 Read block 1810 len 4096
32332 170/300 Read block 3898 len 12288
32333 171/300 Write block 4016 len 8192 data: 6 3
32334 172/300 Read block 1211 len 20480
32335 173/300 Read block 2788 len 36864
32336 174/300 Read block 3923 len 24576
32337 175/300 Flush
32338 176/300 Write block 88 len 20480 data: 6 5 6 4 5
32339 177/300 Read block 166 len 12288
32340 178/300 Write block 3965 len 40960 data: 5 6 6 6 6 4 7 7 9 9
32341 179/300 Write block 1622 len 40960 data: 6 5 5 7 7 5 6 4 4 4
32342 180/300 Read block 2496 len 24576
32343 181/300 Read block 4093 len 4096
32344 182/300 Read block 2882 len 40960
32345 183/300 Read block 4185 len 32768
32346 184/300 Flush
32347 185/300 Read block 2873 len 8192
32348 186/300 Write block 4172 len 20480 data: 7 9 8 8 10
32349 187/300 Read block 3241 len 32768
32350 188/300 Read block 826 len 8192
32351 189/300 Read block 2018 len 40960
32352 190/300 Read block 4581 len 20480
32353 191/300 Write block 3652 len 4096 data: 2
32354 192/300 Write block 65 len 20480 data: 8 7 6 7 5
32355 193/300 Read block 206 len 8192
32356 194/300 Write block 1331 len 8192 data: 7 7
32357 195/300 Write block 3423 len 24576 data: 11 12 11 11 10 8
32358 196/300 Write block 2310 len 4096 data: 8
32359 197/300 Write block 2021 len 36864 data: 7 6 6 7 6 7 7 7 7
32360 198/300 Write block 2330 len 40960 data: 5 5 5 7 6 6 6 6 5 6
32361 199/300 Read block 2098 len 8192
32362 200/300 Write block 771 len 16384 data: 6 6 5 5
32363 201/300 Read block 4351 len 20480
32364 202/300 Read block 3812 len 20480
32365 203/300 Write block 2596 len 24576 data: 5 5 5 5 7 7
32366 204/300 Write block 1735 len 20480 data: 7 6 7 7 7
32367 205/300 Write block 1237 len 28672 data: 8 9 9 8 7 4 4
32368 206/300 Write block 1638 len 40960 data: 5 6 5 5 3 4 3 4 7 7
32369 207/300 Flush
32370 208/300 Write block 2876 len 16384 data: 12 12 13 12
32371 209/300 Write block 2726 len 20480 data: 8 8 9 6 6
32372 210/300 Write block 4249 len 36864 data: 6 8 8 5 4 5 6 6 5
32373 211/300 Write block 3517 len 16384 data: 9 8 6 6
32374 212/300 Read block 383 len 16384
32375 213/300 Read block 3157 len 12288
32376 214/300 Read block 2238 len 28672
32377 215/300 Read block 1435 len 24576
32378 216/300 Read block 423 len 8192
32379 217/300 Write block 427 len 32768 data: 5 6 6 6 5 6 6 6
32380 218/300 Read block 1172 len 12288
32381 219/300 Read block 3341 len 8192
32382 220/300 Read block 3627 len 4096
32383 221/300 Write block 2773 len 24576 data: 6 6 7 6 6 6
32384 222/300 Read block 3411 len 8192
32385 223/300 Read block 459 len 8192
32386 224/300 Flush
32387 225/300 Write block 3781 len 4096 data: 4
32388 226/300 Read block 2892 len 12288
32389 227/300 Read block 1853 len 20480
32390 228/300 Read block 3182 len 24576
32391 229/300 Read block 3154 len 32768
32392 230/300 Write block 631 len 8192 data: 5 5
32393 231/300 Write block 4512 len 24576 data: 5 6 6 5 5 5
32394 232/300 Read block 4611 len 36864
32395 233/300 Read block 4379 len 4096
32396 234/300 Read block 229 len 36864
32397 235/300 Write block 2780 len 12288 data: 6 5 5
32398 236/300 Read block 999 len 12288
32399 237/300 Read block 2640 len 40960
32400 238/300 Read block 2615 len 8192
32401 239/300 Write block 4888 len 20480 data: 4 4 4 4 3
32402 240/300 Read block 623 len 4096
32403 241/300 Write block 1948 len 4096 data: 8
32404 242/300 Flush
32405 243/300 Read block 3809 len 40960
32406 244/300 Write block 3025 len 32768 data: 11 9 7 8 9 8 9 7
32407 245/300 Write block 2742 len 12288 data: 4 6 5
32408 246/300 Write block 2198 len 16384 data: 6 8 7 5
32409 247/300 Read block 2346 len 20480
32410 248/300 Write block 1940 len 32768 data: 8 8 7 7 5 6 5 5
32411 249/300 Read block 1 len 4096
32412 250/300 Write block 4358 len 24576 data: 6 4 2 3 4 4
32413 251/300 Read block 3757 len 40960
32414 252/300 Read block 299 len 36864
32415 253/300 Read block 208 len 12288
32416 254/300 Read block 2943 len 28672
32417 255/300 Read block 4303 len 12288
32418 256/300 Write block 1326 len 36864 data: 7 7 8 7 6 8 8 7 6
32419 257/300 Read block 528 len 8192
32420 258/300 Read block 193 len 24576
32421 259/300 Read block 2188 len 32768
32422 260/300 Write block 3978 len 28672 data: 6 6 6 7 5 6 6
32423 261/300 Read block 4698 len 28672
32424 262/300 Read block 4792 len 12288
32425 263/300 Read block 992 len 28672
32426 264/300 Read block 3863 len 36864
32427 265/300 Flush
32428 266/300 Read block 2777 len 12288
32429 267/300 Read block 664 len 36864
32430 268/300 Read block 3219 len 4096
32431 269/300 Write block 3470 len 36864 data: 4 5 4 4 3 3 3 4 4
32432 270/300 Read block 4476 len 36864
32433 271/300 Write block 3811 len 4096 data: 7
32434 272/300 Flush
32435 273/300 Flush
32436 274/300 Read block 2863 len 16384
32437 275/300 Write block 4617 len 40960 data: 5 4 6 6 7 7 8 9 9 7
32438 276/300 Write block 145 len 8192 data: 12 12
32439 277/300 Read block 4228 len 36864
32440 278/300 Read block 3360 len 24576
32441 279/300 Write block 2173 len 12288 data: 7 8 8
32442 280/300 Read block 3599 len 4096
32443 281/300 Flush
32444 282/300 Read block 3771 len 40960
32445 283/300 Flush
32446 284/300 Read block 1806 len 12288
32447 285/300 Read block 1881 len 4096
32448 286/300 Read block 3209 len 12288
32449 287/300 Flush
32450 288/300 Read block 522 len 24576
32451 289/300 Flush
32452 290/300 Read block 1429 len 8192
32453 291/300 Write block 4406 len 28672 data: 5 5 6 5 6 6 7
32454 292/300 Write block 3095 len 8192 data: 4 4
32455 293/300 Read block 4359 len 8192
32456 294/300 Read block 4826 len 32768
32457 295/300 Read block 1289 len 4096
32458 296/300 Read block 1912 len 28672
32459 297/300 Read block 2594 len 16384
32460 298/300 Flush
32461 299/300 Write block 911 len 28672 data: 10 9 6 5 6 8 8
32462 300/300 Write block 4345 len 16384 data: 6 7 7 8
32463 301/300 Write block 1934 len 36864 data: 3 4 4 5 5 6 9 9 8
32464 Main task has work to do, go find it
32465 got dsc Start(0)
32466 start 0
32467 Jan 14 16:17:27.261 INFO request completed, latency_us: 217, response_code: 204, uri: /start/cid/0, method: POST, req_id: 44ada8e3-a280-4412-b488-bf118f770a39, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
32468 Replay: started 0, returned:Ok(())
32469 ----------------------------------------------------------------
32470 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
32471 States: Offline Active Active
32472 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32473 9754 Acked Write 8 Sent Done Done false
32474 9755 Acked Read 1 Sent Done Done false
32475 9756 Acked Write 1 Sent Done Done false
32476 9757 Acked Write 3 Sent Done Done false
32477 9758 Acked Write 3 Sent Done Done false
32478 9759 Acked Read 9 Sent Done Done false
32479 9760 Acked Read 5 Sent Done Done false
32480 9761 Acked Flush 0 Sent Done Done false
32481 9762 Acked Read 10 Sent Done Done false
32482 9763 Acked Write 2 Sent Done Done false
32483 9764 Acked Read 5 Sent Done Done false
32484 9765 Acked Read 8 Sent Done Done false
32485 9766 Acked Read 2 Sent Done Done false
32486 9767 Acked Write 4 Sent Done Done false
32487 9768 Acked Write 6 Sent Done Done false
32488 9769 Acked Read 3 Sent Done Done false
32489 9770 Acked Read 6 Sent Done Done false
32490 9771 Acked Read 7 Sent Done Done false
32491 9772 Acked Read 8 Sent Done Done false
32492 9773 Acked Read 2 Sent Done Done false
32493 9774 Acked Flush 0 Sent Done Done false
32494 9775 Acked Read 10 Sent Done Done false
32495 9776 Acked Read 9 Sent Done Done false
32496 9777 Acked Write 10 Sent Done Done false
32497 9778 Acked Read 6 Sent Done Done false
32498 9779 Acked Read 4 Sent Done Done false
32499 9780 Acked Write 1 Sent Done Done false
32500 9781 Acked Read 4 Sent Done Done false
32501 9782 Acked Write 6 Sent Done Done false
32502 9783 Acked Read 10 Sent Done Done false
32503 9784 Acked Write 7 Sent Done Done false
32504 9785 Acked Read 10 Sent Done Done false
32505 9786 Acked Flush 0 Sent Done Done false
32506 9787 Acked Write 1 Sent Done Done false
32507 9788 Acked Write 4 Sent Done Done false
32508 9789 Acked Flush 0 Sent Done Done false
32509 9790 Acked Write 9 Sent Done Done false
32510 9791 Acked Write 7 Sent Done Done false
32511 9792 Acked Write 8 Sent Done Done false
32512 9793 Acked Flush 0 Sent Done Done false
32513 9794 Acked Read 5 Sent Done Done false
32514 9795 Acked Write 2 Sent Done Done false
32515 9796 Acked Write 10 Sent Done Done false
32516 9797 Acked Write 7 Sent Done Done false
32517 9798 Acked Write 5 Sent Done Done false
32518 9799 Acked Read 3 Sent Done Done false
32519 9800 Acked Read 8 Sent Done Done false
32520 9801 Acked Read 10 Sent Done Done false
32521 9802 Acked Write 3 Sent Done Done false
32522 9803 Acked Read 4 Sent Done Done false
32523 9804 Acked Read 4 Sent Done Done false
32524 9805 Acked Read 4 Sent Done Done false
32525 9806 Acked Read 5 Sent Done Done false
32526 9807 Acked Write 4 Sent Done Done false
32527 9808 Acked Read 4 Sent Done Done false
32528 9809 Acked Write 8 Sent Done Done false
32529 9810 Acked Read 8 Sent Done Done false
32530 9811 Acked Read 7 Sent Done Done false
32531 9812 Acked Write 9 Sent Done Done false
32532 9813 Acked Write 7 Sent Done Done false
32533 9814 Acked Write 6 Sent Done Done false
32534 9815 Acked Write 2 Sent Done Done false
32535 9816 Acked Read 5 Sent Done Done false
32536 9817 Acked Write 6 Sent Done Done false
32537 9818 Acked Write 10 Sent Done Done false
32538 9819 Acked Write 8 Sent Done Done false
32539 9820 Acked Read 5 Sent Done Done false
32540 9821 Acked Read 5 Sent Done Done false
32541 9822 Acked Write 8 Sent Done Done false
32542 9823 Acked Write 3 Sent Done Done false
32543 9824 Acked Write 10 Sent Done Done false
32544 9825 Acked Read 3 Sent Done Done false
32545 9826 Acked Write 5 Sent Done Done false
32546 9827 Acked Flush 0 Sent Done Done false
32547 9828 Acked Read 3 Sent Done Done false
32548 9829 Acked Read 10 Sent Done Done false
32549 9830 Acked Read 9 Sent Done Done false
32550 9831 Acked Write 3 Sent Done Done false
32551 9832 Acked Flush 0 Sent Done Done false
32552 9833 Acked Flush 0 Sent Done Done false
32553 9834 Acked Write 4 Sent Done Done false
32554 9835 Acked Flush 0 Sent Done Done false
32555 9836 Acked Read 10 Sent Done Done false
32556 9837 Acked Read 6 Sent Done Done false
32557 9838 Acked Flush 0 Sent Done Done false
32558 9839 Acked Flush 0 Sent Done Done false
32559 9840 Acked Read 4 Sent Done Done false
32560 9841 Acked Flush 0 Sent Done Done false
32561 9842 Acked Write 9 Sent Done Done false
32562 9843 Acked Read 2 Sent Done Done false
32563 9844 Acked Write 2 Sent Done Done false
32564 9845 Acked Write 3 Sent Done Done false
32565 9846 Acked Write 9 Sent Done Done false
32566 9847 Acked Write 3 Sent Done Done false
32567 9848 Acked Flush 0 Sent Done Done false
32568 9849 Acked Write 10 Sent Done Done false
32569 9850 Acked Flush 0 Sent Done Done false
32570 9851 Acked Flush 0 Sent Done Done false
32571 9852 Acked Write 1 Sent Done Done false
32572 9853 Acked Write 9 Sent Done Done false
32573 9854 Acked Write 5 Sent Done Done false
32574 9855 Acked Flush 0 Sent Done Done false
32575 9856 Acked Read 9 Sent Done Done false
32576 9857 Acked Flush 0 Sent Done Done false
32577 9858 Acked Read 7 Sent Done Done false
32578 9859 Acked Flush 0 Sent Done Done false
32579 9860 Acked Write 1 Sent Done Done false
32580 9861 Acked Read 10 Sent Done Done false
32581 9862 Acked Flush 0 Sent Done Done false
32582 9863 Acked Read 6 Sent Done Done false
32583 9864 Acked Read 1 Sent Done Done false
32584 9865 Acked Read 8 Sent Done Done false
32585 9866 Acked Write 3 Sent Done Done false
32586 9867 Acked Read 10 Sent Done Done false
32587 9868 Acked Write 7 Sent Done Done false
32588 9869 Acked Write 6 Sent Done Done false
32589 9870 Acked Write 8 Sent Done Done false
32590 9871 Acked Write 8 Sent Done Done false
32591 9872 Acked Read 1 Sent Done Done false
32592 9873 Acked Flush 0 Sent Done Done false
32593 9874 Acked Write 7 Sent Done Done false
32594 9875 Acked Read 10 Sent Done Done false
32595 9876 Acked Read 6 Sent Done Done false
32596 9877 Acked Read 6 Sent Done Done false
32597 9878 Acked Flush 0 Sent Done Done false
32598 9879 Acked Read 8 Sent Done Done false
32599 9880 Acked Read 5 Sent Done Done false
32600 9881 Acked Read 7 Sent Done Done false
32601 9882 Acked Read 1 Sent Done Done false
32602 9883 Acked Write 1 Sent Done Done false
32603 9884 Acked Read 10 Sent Done Done false
32604 9885 Acked Read 4 Sent Done Done false
32605 9886 Acked Read 7 Sent Done Done false
32606 9887 Acked Read 9 Sent Done Done false
32607 9888 Acked Write 4 Sent Done Done false
32608 9889 Acked Read 3 Sent Done Done false
32609 9890 Acked Read 4 Sent Done Done false
32610 9891 Acked Write 2 Sent Done Done false
32611 9892 Acked Write 2 Sent Done Done false
32612 9893 Acked Read 3 Sent Done Done false
32613 9894 Acked Read 5 Sent Done Done false
32614 9895 Acked Read 3 Sent Done Done false
32615 9896 Acked Read 5 Sent Done Done false
32616 9897 Acked Write 7 Sent Done Done false
32617 9898 Acked Read 5 Sent Done Done false
32618 9899 Acked Read 6 Sent Done Done false
32619 9900 Acked Flush 0 Sent Done Done false
32620 9901 Acked Read 4 Sent Done Done false
32621 9902 Acked Read 6 Sent Done Done false
32622 9903 Acked Read 3 Sent Done Done false
32623 9904 Acked Read 7 Sent Done Done false
32624 9905 Acked Write 2 Sent Done Done false
32625 9906 Acked Read 3 Sent Done Done false
32626 9907 Acked Write 6 Sent Done Done false
32627 9908 Acked Flush 0 Sent Done Done false
32628 9909 Acked Write 4 Sent Done Done false
32629 9910 Acked Read 8 Sent Done Done false
32630 9911 Acked Read 4 Sent Done Done false
32631 9912 Acked Write 7 Sent Done Done false
32632 9913 Acked Read 2 Sent Done Done false
32633 9914 Acked Read 8 Sent Done Done false
32634 9915 Acked Write 1 Sent Done Done false
32635 9916 Acked Read 4 Sent Done Done false
32636 9917 Acked Read 3 Sent Done Done false
32637 9918 Acked Read 10 Sent Done Done false
32638 9919 Acked Read 7 Sent Done Done false
32639 9920 Acked Write 2 Sent Done Done false
32640 9921 Acked Flush 0 Sent Done Done false
32641 9922 Acked Read 1 Sent Done Done false
32642 9923 Acked Read 3 Sent Done Done false
32643 9924 Acked Write 2 Sent Done Done false
32644 9925 Acked Read 5 Sent Done Done false
32645 9926 Acked Read 9 Sent Done Done false
32646 9927 Acked Read 6 Sent Done Done false
32647 9928 Acked Flush 0 Sent Done Done false
32648 9929 Acked Write 5 Sent Done Done false
32649 9930 Acked Read 3 Sent Done Done false
32650 9931 Acked Write 10 Sent Done Done false
32651 9932 Acked Write 10 Sent Done Done false
32652 9933 Acked Read 6 Sent Done Done false
32653 9934 Acked Read 1 Sent Done Done false
32654 9935 Acked Read 10 Sent Done Done false
32655 9936 Acked Read 8 Sent Done Done false
32656 9937 Acked Flush 0 Sent Done Done false
32657 9938 Acked Read 2 Sent Done Done false
32658 9939 Acked Write 5 Sent Done Done false
32659 9940 Acked Read 8 Sent Done Done false
32660 9941 Acked Read 2 Sent Done Done false
32661 9942 Acked Read 10 Sent Done Done false
32662 9943 Acked Read 5 Sent Done Done false
32663 9944 Acked Write 1 Sent Done Done false
32664 9945 Acked Write 5 Sent Done Done false
32665 9946 Acked Read 2 Sent Done Done false
32666 9947 Acked Write 2 Sent Done Done false
32667 9948 Acked Write 6 Sent Done Done false
32668 9949 Acked Write 1 Sent Done Done false
32669 9950 Acked Write 9 Sent Done Done false
32670 9951 Acked Write 10 Sent Done Done false
32671 9952 Acked Read 2 Sent Done Done false
32672 9953 Acked Write 4 Sent Done Done false
32673 9954 Acked Read 5 Sent Done Done false
32674 9955 Acked Read 5 Sent Done Done false
32675 9956 Acked Write 6 Sent Done Done false
32676 9957 Acked Write 5 Sent Done Done false
32677 9958 Acked Write 7 Sent Done Done false
32678 9959 Acked Write 10 Sent Done Done false
32679 9960 Acked Flush 0 Sent Done Done false
32680 9961 Acked Write 4 Sent Done Done false
32681 9962 Acked Write 5 Sent Done Done false
32682 9963 Acked Write 9 Sent Done Done false
32683 9964 Acked Write 4 Sent Done Done false
32684 9965 Acked Read 4 Sent Done Done false
32685 9966 Acked Read 3 Sent Done Done false
32686 9967 Acked Read 7 Sent Done Done false
32687 9968 Acked Read 6 Sent Done Done false
32688 9969 Acked Read 2 Sent Done Done false
32689 9970 Acked Write 8 Sent Done Done false
32690 9971 Acked Read 3 Sent Done Done false
32691 9972 Acked Read 2 Sent Done Done false
32692 9973 Acked Read 1 Sent Done Done false
32693 9974 Acked Write 6 Sent Done Done false
32694 9975 Acked Read 2 Sent Done Done false
32695 9976 Acked Read 2 Sent Done Done false
32696 9977 Acked Flush 0 Sent Done Done false
32697 9978 Acked Write 1 Sent Done Done false
32698 9979 Acked Read 3 Sent Done Done false
32699 9980 Acked Read 5 Sent Done Done false
32700 9981 Acked Read 6 Sent Done Done false
32701 9982 Acked Read 8 Sent Done Done false
32702 9983 Acked Write 2 Sent Done Done false
32703 9984 Acked Write 6 Sent Done Done false
32704 9985 Acked Read 9 Sent Done Done false
32705 9986 Acked Read 1 Sent Done Done false
32706 9987 Acked Read 9 Sent Done Done false
32707 9988 Acked Write 3 Sent Done Done false
32708 9989 Acked Read 3 Sent Done Done false
32709 9990 Acked Read 10 Sent Done Done false
32710 9991 Acked Read 2 Sent Done Done false
32711 9992 Acked Write 5 Sent Done Done false
32712 9993 Acked Read 1 Sent Done Done false
32713 9994 Acked Write 1 Sent Done Done false
32714 9995 Acked Flush 0 Sent Done Done false
32715 9996 Acked Read 10 Sent Done Done false
32716 9997 Acked Write 8 Sent Done Done false
32717 9998 Acked Write 3 Sent Done Done false
32718 9999 Acked Write 4 Sent Done Done false
32719 10000 Acked Read 5 Sent Done Done false
32720 10001 Acked Write 8 Sent Done Done false
32721 10002 Acked Read 1 Sent Done Done false
32722 10003 Acked Write 6 Sent Done Done false
32723 10004 Acked Read 10 Sent Done Done false
32724 10005 Acked Read 9 Sent Done Done false
32725 10006 Acked Read 3 Sent Done Done false
32726 10007 Acked Read 7 Sent Done Done false
32727 10008 Acked Read 3 Sent Done Done false
32728 10009 Acked Write 9 Sent Done Done false
32729 10010 Acked Read 2 Sent Done Done false
32730 10011 Acked Read 6 Sent Done Done false
32731 10012 Acked Read 8 Sent Done Done false
32732 10013 Acked Write 7 Sent Done Done false
32733 10014 Acked Read 7 Sent Done Done false
32734 10015 Acked Read 3 Sent Done Done false
32735 10016 Acked Read 7 Sent Done Done false
32736 10017 Acked Read 9 Sent Done Done false
32737 10018 Acked Flush 0 Sent Done Done false
32738 10019 Acked Read 3 Sent Done Done false
32739 10020 Acked Read 9 Sent Done Done false
32740 10021 Acked Read 1 Sent Done Done false
32741 10022 Acked Write 9 Sent Done Done false
32742 10023 Acked Read 9 Sent Done Done false
32743 10024 Acked Write 1 Sent Done Done false
32744 10025 Acked Flush 0 Sent Done Done false
32745 10026 Acked Flush 0 Sent Done Done false
32746 10027 Acked Read 4 Sent Done Done false
32747 10028 Acked Write 10 Sent Done Done false
32748 10029 Acked Write 2 Sent Done Done false
32749 10030 Acked Read 9 Sent Done Done false
32750 10031 Acked Read 6 Sent Done Done false
32751 10032 Acked Write 3 Sent Done Done false
32752 10033 Acked Read 1 Sent Done Done false
32753 10034 Acked Flush 0 Sent Done Done false
32754 10035 Acked Read 10 Sent Done Done false
32755 10036 Acked Flush 0 Sent Done Done false
32756 10037 Acked Read 3 Sent Done Done false
32757 10038 Acked Read 1 Sent Done Done false
32758 10039 Acked Read 3 Sent Done Done false
32759 10040 Acked Flush 0 Sent Done Done false
32760 10041 Acked Read 6 Sent Done Done false
32761 10042 Acked Flush 0 Sent Done Done false
32762 10043 Acked Read 2 Sent Done Done false
32763 10044 Acked Write 7 Sent Done Done false
32764 10045 Acked Write 2 Sent Done Done false
32765 10046 Acked Read 2 Sent Done Done false
32766 10047 Acked Read 8 Sent Done Done false
32767 10048 Acked Read 1 Sent Done Done false
32768 10049 Acked Read 7 Sent Done Done false
32769 10050 Acked Read 4 Sent Done Done false
32770 10051 Acked Flush 0 Sent Done Done false
32771 10052 Acked Write 7 Sent Done Done false
32772 10053 Acked Write 4 Sent Done Done false
32773 10054 Acked Write 9 Sent Done Done false
32774 STATES DS:0 DS:1 DS:2 TOTAL
32775 Sent 301 0 0 301
32776 Done 0 301 301 602
32777 Skipped 0 0 0 0
32778 Error 0 0 0 0
32779 Last Flush: 9753 10051 10051
32780 Downstairs last five completed: 9753 9752 9751 9750 9749
32781 Upstairs last five completed: 10054 10053 10052 10051 10050
32782 CLIENT: Up:0 ds:301 act:2
32783 [8810] got start action
32784 [8810] I am going to restart
32785 Starting downstairs at port 8810
32786 Make output file at "/tmp/dsc/downstairs-8810.txt"
32787 [8810][0] reports Starting
32788 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1232)
32789 [8810][0] reports Running
32790 ----------------------------------------------------------------
32791 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32792 States: Offline Active Active
32793 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32794 9754 Acked Write 8 Sent Done Done false
32795 9755 Acked Read 1 Sent Done Done false
32796 9756 Acked Write 1 Sent Done Done false
32797 9757 Acked Write 3 Sent Done Done false
32798 9758 Acked Write 3 Sent Done Done false
32799 9759 Acked Read 9 Sent Done Done false
32800 9760 Acked Read 5 Sent Done Done false
32801 9761 Acked Flush 0 Sent Done Done false
32802 9762 Acked Read 10 Sent Done Done false
32803 9763 Acked Write 2 Sent Done Done false
32804 9764 Acked Read 5 Sent Done Done false
32805 9765 Acked Read 8 Sent Done Done false
32806 9766 Acked Read 2 Sent Done Done false
32807 9767 Acked Write 4 Sent Done Done false
32808 9768 Acked Write 6 Sent Done Done false
32809 9769 Acked Read 3 Sent Done Done false
32810 9770 Acked Read 6 Sent Done Done false
32811 9771 Acked Read 7 Sent Done Done false
32812 9772 Acked Read 8 Sent Done Done false
32813 9773 Acked Read 2 Sent Done Done false
32814 9774 Acked Flush 0 Sent Done Done false
32815 9775 Acked Read 10 Sent Done Done false
32816 9776 Acked Read 9 Sent Done Done false
32817 9777 Acked Write 10 Sent Done Done false
32818 9778 Acked Read 6 Sent Done Done false
32819 9779 Acked Read 4 Sent Done Done false
32820 9780 Acked Write 1 Sent Done Done false
32821 9781 Acked Read 4 Sent Done Done false
32822 9782 Acked Write 6 Sent Done Done false
32823 9783 Acked Read 10 Sent Done Done false
32824 9784 Acked Write 7 Sent Done Done false
32825 9785 Acked Read 10 Sent Done Done false
32826 9786 Acked Flush 0 Sent Done Done false
32827 9787 Acked Write 1 Sent Done Done false
32828 9788 Acked Write 4 Sent Done Done false
32829 9789 Acked Flush 0 Sent Done Done false
32830 9790 Acked Write 9 Sent Done Done false
32831 9791 Acked Write 7 Sent Done Done false
32832 9792 Acked Write 8 Sent Done Done false
32833 9793 Acked Flush 0 Sent Done Done false
32834 9794 Acked Read 5 Sent Done Done false
32835 9795 Acked Write 2 Sent Done Done false
32836 9796 Acked Write 10 Sent Done Done false
32837 9797 Acked Write 7 Sent Done Done false
32838 9798 Acked Write 5 Sent Done Done false
32839 9799 Acked Read 3 Sent Done Done false
32840 9800 Acked Read 8 Sent Done Done false
32841 9801 Acked Read 10 Sent Done Done false
32842 9802 Acked Write 3 Sent Done Done false
32843 9803 Acked Read 4 Sent Done Done false
32844 9804 Acked Read 4 Sent Done Done false
32845 9805 Acked Read 4 Sent Done Done false
32846 9806 Acked Read 5 Sent Done Done false
32847 9807 Acked Write 4 Sent Done Done false
32848 9808 Acked Read 4 Sent Done Done false
32849 9809 Acked Write 8 Sent Done Done false
32850 9810 Acked Read 8 Sent Done Done false
32851 9811 Acked Read 7 Sent Done Done false
32852 9812 Acked Write 9 Sent Done Done false
32853 9813 Acked Write 7 Sent Done Done false
32854 9814 Acked Write 6 Sent Done Done false
32855 9815 Acked Write 2 Sent Done Done false
32856 9816 Acked Read 5 Sent Done Done false
32857 9817 Acked Write 6 Sent Done Done false
32858 9818 Acked Write 10 Sent Done Done false
32859 9819 Acked Write 8 Sent Done Done false
32860 9820 Acked Read 5 Sent Done Done false
32861 9821 Acked Read 5 Sent Done Done false
32862 9822 Acked Write 8 Sent Done Done false
32863 9823 Acked Write 3 Sent Done Done false
32864 9824 Acked Write 10 Sent Done Done false
32865 9825 Acked Read 3 Sent Done Done false
32866 9826 Acked Write 5 Sent Done Done false
32867 9827 Acked Flush 0 Sent Done Done false
32868 9828 Acked Read 3 Sent Done Done false
32869 9829 Acked Read 10 Sent Done Done false
32870 9830 Acked Read 9 Sent Done Done false
32871 9831 Acked Write 3 Sent Done Done false
32872 9832 Acked Flush 0 Sent Done Done false
32873 9833 Acked Flush 0 Sent Done Done false
32874 9834 Acked Write 4 Sent Done Done false
32875 9835 Acked Flush 0 Sent Done Done false
32876 9836 Acked Read 10 Sent Done Done false
32877 9837 Acked Read 6 Sent Done Done false
32878 9838 Acked Flush 0 Sent Done Done false
32879 9839 Acked Flush 0 Sent Done Done false
32880 9840 Acked Read 4 Sent Done Done false
32881 9841 Acked Flush 0 Sent Done Done false
32882 9842 Acked Write 9 Sent Done Done false
32883 9843 Acked Read 2 Sent Done Done false
32884 9844 Acked Write 2 Sent Done Done false
32885 9845 Acked Write 3 Sent Done Done false
32886 9846 Acked Write 9 Sent Done Done false
32887 9847 Acked Write 3 Sent Done Done false
32888 9848 Acked Flush 0 Sent Done Done false
32889 9849 Acked Write 10 Sent Done Done false
32890 9850 Acked Flush 0 Sent Done Done false
32891 9851 Acked Flush 0 Sent Done Done false
32892 9852 Acked Write 1 Sent Done Done false
32893 9853 Acked Write 9 Sent Done Done false
32894 9854 Acked Write 5 Sent Done Done false
32895 9855 Acked Flush 0 Sent Done Done false
32896 9856 Acked Read 9 Sent Done Done false
32897 9857 Acked Flush 0 Sent Done Done false
32898 9858 Acked Read 7 Sent Done Done false
32899 9859 Acked Flush 0 Sent Done Done false
32900 9860 Acked Write 1 Sent Done Done false
32901 9861 Acked Read 10 Sent Done Done false
32902 9862 Acked Flush 0 Sent Done Done false
32903 9863 Acked Read 6 Sent Done Done false
32904 9864 Acked Read 1 Sent Done Done false
32905 9865 Acked Read 8 Sent Done Done false
32906 9866 Acked Write 3 Sent Done Done false
32907 9867 Acked Read 10 Sent Done Done false
32908 9868 Acked Write 7 Sent Done Done false
32909 9869 Acked Write 6 Sent Done Done false
32910 9870 Acked Write 8 Sent Done Done false
32911 9871 Acked Write 8 Sent Done Done false
32912 9872 Acked Read 1 Sent Done Done false
32913 9873 Acked Flush 0 Sent Done Done false
32914 9874 Acked Write 7 Sent Done Done false
32915 9875 Acked Read 10 Sent Done Done false
32916 9876 Acked Read 6 Sent Done Done false
32917 9877 Acked Read 6 Sent Done Done false
32918 9878 Acked Flush 0 Sent Done Done false
32919 9879 Acked Read 8 Sent Done Done false
32920 9880 Acked Read 5 Sent Done Done false
32921 9881 Acked Read 7 Sent Done Done false
32922 9882 Acked Read 1 Sent Done Done false
32923 9883 Acked Write 1 Sent Done Done false
32924 9884 Acked Read 10 Sent Done Done false
32925 9885 Acked Read 4 Sent Done Done false
32926 9886 Acked Read 7 Sent Done Done false
32927 9887 Acked Read 9 Sent Done Done false
32928 9888 Acked Write 4 Sent Done Done false
32929 9889 Acked Read 3 Sent Done Done false
32930 9890 Acked Read 4 Sent Done Done false
32931 9891 Acked Write 2 Sent Done Done false
32932 9892 Acked Write 2 Sent Done Done false
32933 9893 Acked Read 3 Sent Done Done false
32934 9894 Acked Read 5 Sent Done Done false
32935 9895 Acked Read 3 Sent Done Done false
32936 9896 Acked Read 5 Sent Done Done false
32937 9897 Acked Write 7 Sent Done Done false
32938 9898 Acked Read 5 Sent Done Done false
32939 9899 Acked Read 6 Sent Done Done false
32940 9900 Acked Flush 0 Sent Done Done false
32941 9901 Acked Read 4 Sent Done Done false
32942 9902 Acked Read 6 Sent Done Done false
32943 9903 Acked Read 3 Sent Done Done false
32944 9904 Acked Read 7 Sent Done Done false
32945 9905 Acked Write 2 Sent Done Done false
32946 9906 Acked Read 3 Sent Done Done false
32947 9907 Acked Write 6 Sent Done Done false
32948 9908 Acked Flush 0 Sent Done Done false
32949 9909 Acked Write 4 Sent Done Done false
32950 9910 Acked Read 8 Sent Done Done false
32951 9911 Acked Read 4 Sent Done Done false
32952 9912 Acked Write 7 Sent Done Done false
32953 9913 Acked Read 2 Sent Done Done false
32954 9914 Acked Read 8 Sent Done Done false
32955 9915 Acked Write 1 Sent Done Done false
32956 9916 Acked Read 4 Sent Done Done false
32957 9917 Acked Read 3 Sent Done Done false
32958 9918 Acked Read 10 Sent Done Done false
32959 9919 Acked Read 7 Sent Done Done false
32960 9920 Acked Write 2 Sent Done Done false
32961 9921 Acked Flush 0 Sent Done Done false
32962 9922 Acked Read 1 Sent Done Done false
32963 9923 Acked Read 3 Sent Done Done false
32964 9924 Acked Write 2 Sent Done Done false
32965 9925 Acked Read 5 Sent Done Done false
32966 9926 Acked Read 9 Sent Done Done false
32967 9927 Acked Read 6 Sent Done Done false
32968 9928 Acked Flush 0 Sent Done Done false
32969 9929 Acked Write 5 Sent Done Done false
32970 9930 Acked Read 3 Sent Done Done false
32971 9931 Acked Write 10 Sent Done Done false
32972 9932 Acked Write 10 Sent Done Done false
32973 9933 Acked Read 6 Sent Done Done false
32974 9934 Acked Read 1 Sent Done Done false
32975 9935 Acked Read 10 Sent Done Done false
32976 9936 Acked Read 8 Sent Done Done false
32977 9937 Acked Flush 0 Sent Done Done false
32978 9938 Acked Read 2 Sent Done Done false
32979 9939 Acked Write 5 Sent Done Done false
32980 9940 Acked Read 8 Sent Done Done false
32981 9941 Acked Read 2 Sent Done Done false
32982 9942 Acked Read 10 Sent Done Done false
32983 9943 Acked Read 5 Sent Done Done false
32984 9944 Acked Write 1 Sent Done Done false
32985 9945 Acked Write 5 Sent Done Done false
32986 9946 Acked Read 2 Sent Done Done false
32987 9947 Acked Write 2 Sent Done Done false
32988 9948 Acked Write 6 Sent Done Done false
32989 9949 Acked Write 1 Sent Done Done false
32990 9950 Acked Write 9 Sent Done Done false
32991 9951 Acked Write 10 Sent Done Done false
32992 9952 Acked Read 2 Sent Done Done false
32993 9953 Acked Write 4 Sent Done Done false
32994 9954 Acked Read 5 Sent Done Done false
32995 9955 Acked Read 5 Sent Done Done false
32996 9956 Acked Write 6 Sent Done Done false
32997 9957 Acked Write 5 Sent Done Done false
32998 9958 Acked Write 7 Sent Done Done false
32999 9959 Acked Write 10 Sent Done Done false
33000 9960 Acked Flush 0 Sent Done Done false
33001 9961 Acked Write 4 Sent Done Done false
33002 9962 Acked Write 5 Sent Done Done false
33003 9963 Acked Write 9 Sent Done Done false
33004 9964 Acked Write 4 Sent Done Done false
33005 9965 Acked Read 4 Sent Done Done false
33006 9966 Acked Read 3 Sent Done Done false
33007 9967 Acked Read 7 Sent Done Done false
33008 9968 Acked Read 6 Sent Done Done false
33009 9969 Acked Read 2 Sent Done Done false
33010 9970 Acked Write 8 Sent Done Done false
33011 9971 Acked Read 3 Sent Done Done false
33012 9972 Acked Read 2 Sent Done Done false
33013 9973 Acked Read 1 Sent Done Done false
33014 9974 Acked Write 6 Sent Done Done false
33015 9975 Acked Read 2 Sent Done Done false
33016 9976 Acked Read 2 Sent Done Done false
33017 9977 Acked Flush 0 Sent Done Done false
33018 9978 Acked Write 1 Sent Done Done false
33019 9979 Acked Read 3 Sent Done Done false
33020 9980 Acked Read 5 Sent Done Done false
33021 9981 Acked Read 6 Sent Done Done false
33022 9982 Acked Read 8 Sent Done Done false
33023 9983 Acked Write 2 Sent Done Done false
33024 9984 Acked Write 6 Sent Done Done false
33025 9985 Acked Read 9 Sent Done Done false
33026 9986 Acked Read 1 Sent Done Done false
33027 9987 Acked Read 9 Sent Done Done false
33028 9988 Acked Write 3 Sent Done Done false
33029 9989 Acked Read 3 Sent Done Done false
33030 9990 Acked Read 10 Sent Done Done false
33031 9991 Acked Read 2 Sent Done Done false
33032 9992 Acked Write 5 Sent Done Done false
33033 9993 Acked Read 1 Sent Done Done false
33034 9994 Acked Write 1 Sent Done Done false
33035 9995 Acked Flush 0 Sent Done Done false
33036 9996 Acked Read 10 Sent Done Done false
33037 9997 Acked Write 8 Sent Done Done false
33038 9998 Acked Write 3 Sent Done Done false
33039 9999 Acked Write 4 Sent Done Done false
33040 10000 Acked Read 5 Sent Done Done false
33041 10001 Acked Write 8 Sent Done Done false
33042 10002 Acked Read 1 Sent Done Done false
33043 10003 Acked Write 6 Sent Done Done false
33044 10004 Acked Read 10 Sent Done Done false
33045 10005 Acked Read 9 Sent Done Done false
33046 10006 Acked Read 3 Sent Done Done false
33047 10007 Acked Read 7 Sent Done Done false
33048 10008 Acked Read 3 Sent Done Done false
33049 10009 Acked Write 9 Sent Done Done false
33050 10010 Acked Read 2 Sent Done Done false
33051 10011 Acked Read 6 Sent Done Done false
33052 10012 Acked Read 8 Sent Done Done false
33053 10013 Acked Write 7 Sent Done Done false
33054 10014 Acked Read 7 Sent Done Done false
33055 10015 Acked Read 3 Sent Done Done false
33056 10016 Acked Read 7 Sent Done Done false
33057 10017 Acked Read 9 Sent Done Done false
33058 10018 Acked Flush 0 Sent Done Done false
33059 10019 Acked Read 3 Sent Done Done false
33060 10020 Acked Read 9 Sent Done Done false
33061 10021 Acked Read 1 Sent Done Done false
33062 10022 Acked Write 9 Sent Done Done false
33063 10023 Acked Read 9 Sent Done Done false
33064 10024 Acked Write 1 Sent Done Done false
33065 10025 Acked Flush 0 Sent Done Done false
33066 10026 Acked Flush 0 Sent Done Done false
33067 10027 Acked Read 4 Sent Done Done false
33068 10028 Acked Write 10 Sent Done Done false
33069 10029 Acked Write 2 Sent Done Done false
33070 10030 Acked Read 9 Sent Done Done false
33071 10031 Acked Read 6 Sent Done Done false
33072 10032 Acked Write 3 Sent Done Done false
33073 10033 Acked Read 1 Sent Done Done false
33074 10034 Acked Flush 0 Sent Done Done false
33075 10035 Acked Read 10 Sent Done Done false
33076 10036 Acked Flush 0 Sent Done Done false
33077 10037 Acked Read 3 Sent Done Done false
33078 10038 Acked Read 1 Sent Done Done false
33079 10039 Acked Read 3 Sent Done Done false
33080 10040 Acked Flush 0 Sent Done Done false
33081 10041 Acked Read 6 Sent Done Done false
33082 10042 Acked Flush 0 Sent Done Done false
33083 10043 Acked Read 2 Sent Done Done false
33084 10044 Acked Write 7 Sent Done Done false
33085 10045 Acked Write 2 Sent Done Done false
33086 10046 Acked Read 2 Sent Done Done false
33087 10047 Acked Read 8 Sent Done Done false
33088 10048 Acked Read 1 Sent Done Done false
33089 10049 Acked Read 7 Sent Done Done false
33090 10050 Acked Read 4 Sent Done Done false
33091 10051 Acked Flush 0 Sent Done Done false
33092 10052 Acked Write 7 Sent Done Done false
33093 10053 Acked Write 4 Sent Done Done false
33094 10054 Acked Write 9 Sent Done Done false
33095 10055 Acked Flush 0 Sent Done Done false
33096 STATES DS:0 DS:1 DS:2 TOTAL
33097 Sent 302 0 0 302
33098 Done 0 302 302 604
33099 Skipped 0 0 0 0
33100 Error 0 0 0 0
33101 Last Flush: 9753 10055 10055
33102 Downstairs last five completed: 9753 9752 9751 9750 9749
33103 Upstairs last five completed: 10055 10054 10053 10052 10051
33104 CLIENT: Up:0 ds:302 act:2
331052025-01-14T16:17:33.026ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
331062025-01-14T16:17:33.026ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55608) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
331072025-01-14T16:17:33.028ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
331082025-01-14T16:17:33.028ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
331092025-01-14T16:17:33.028ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
331102025-01-14T16:17:33.028ZINFOcrucible: send last flush ID to this DS: 9753 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
331112025-01-14T16:17:33.028ZINFOcrucible: Replied this last flush ID: 9753 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
331122025-01-14T16:17:33.028ZINFOcrucible: [0] client re-new 302 jobs since flush 9753 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
331132025-01-14T16:17:33.029ZINFOcrucible: [0] Replayed 302 jobs since flush: 9753 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
33114 ----------------------------------------------------------------
33115 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
33116 Downstairs last five completed: 10055 10054 10053 10052 10051
33117 Upstairs last five completed: 10055 10054 10053 10052 10051
33118 CLIENT: Up:0 ds:0 act:3
33119 Replay: All jobs finished
33120 Main task has work to do, go find it
33121 got dsc Stop(0)
33122 stop 0
33123 Jan 14 16:17:35.266 [8810] Got stop action so:false kr:false
33124 INFO request completed, latency_us: 359, response_code: 204, uri: /stop/cid/0, method: POST, req_id: b48cb17d-18cf-4f28-8f28-1572b051ebf2, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
331252025-01-14T16:17:35.266ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
331262025-01-14T16:17:35.266ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
331272025-01-14T16:17:35.266ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = c2423ef0-728e-4895-997a-d9044c50948b
331282025-01-14T16:17:35.267ZINFOcrucible: client task is exiting = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
33129 Jan 14 16:17:35.267 INFO request completed, latency_us: 164, response_code: 200, uri: /state/cid/0, method: GET, req_id: eced60e0-dad6-4880-b5e7-597106e512e6, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
33130 [8810] Exited with: ExitStatus(unix_wait_status(9))
33131 [8810][0] reports Exit
33132 Jan 14 16:17:39.269 INFO request completed, latency_us: 194, response_code: 200, uri: /state/cid/0, method: GET, req_id: 79a6233b-5bbf-41c5-8d17-b5e344e44981, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
33133 001/300 Write block 2844 len 28672 data: 3 4 4 4 5 6 6
33134 002/300 Read block 2257 len 32768
33135 003/300 Read block 625 len 12288
33136 004/300 Flush
33137 005/300 Write block 2855 len 20480 data: 7 8 9 8 7
33138 006/300 Flush
33139 007/300 Read block 258 len 36864
33140 008/300 Write block 3468 len 32768 data: 4 5 5 6 5 5 4 4
33141 009/300 Read block 2441 len 36864
33142 010/300 Read block 3842 len 16384
33143 011/300 Flush
33144 012/300 Read block 1430 len 4096
33145 013/300 Write block 501 len 24576 data: 5 5 5 6 6 6
33146 014/300 Read block 569 len 32768
33147 015/300 Write block 1508 len 12288 data: 5 5 6
33148 016/300 Read block 1674 len 4096
33149 017/300 Flush
33150 018/300 Write block 3838 len 20480 data: 4 4 5 5 7
33151 019/300 Write block 1760 len 4096 data: 5
33152 020/300 Read block 3421 len 16384
33153 021/300 Write block 2655 len 16384 data: 5 4 6 6
33154 022/300 Read block 120 len 20480
33155 023/300 Write block 3125 len 8192 data: 3 2
33156 024/300 Read block 3701 len 32768
33157 025/300 Write block 962 len 8192 data: 3 4
33158 026/300 Write block 853 len 36864 data: 7 6 6 5 5 5 6 6 6
33159 027/300 Read block 1101 len 28672
33160 028/300 Read block 1674 len 4096
33161 029/300 Read block 150 len 32768
33162 030/300 Read block 100 len 24576
33163 031/300 Read block 112 len 20480
33164 032/300 Write block 1119 len 4096 data: 5
33165 033/300 Read block 1257 len 28672
33166 034/300 Read block 3666 len 20480
33167 035/300 Flush
33168 036/300 Write block 376 len 12288 data: 9 5 4
33169 037/300 Write block 1888 len 36864 data: 7 9 10 11 10 9 9 9 7
33170 038/300 Write block 3559 len 16384 data: 6 5 5 8
33171 039/300 Read block 4233 len 20480
33172 040/300 Write block 1203 len 24576 data: 4 5 7 7 7 5
33173 041/300 Write block 3984 len 16384 data: 7 7 8 7
33174 042/300 Write block 3051 len 16384 data: 8 6 5 4
33175 043/300 Read block 4352 len 8192
33176 044/300 Read block 4550 len 16384
33177 045/300 Read block 1215 len 28672
33178 046/300 Read block 761 len 28672
33179 047/300 Read block 2638 len 24576
33180 048/300 Read block 2463 len 12288
33181 049/300 Flush
33182 050/300 Write block 424 len 28672 data: 8 8 6 6 7 7 7
33183 051/300 Read block 3133 len 36864
33184 052/300 Read block 3243 len 32768
33185 053/300 Read block 2559 len 8192
33186 054/300 Read block 2649 len 12288
33187 055/300 Write block 1946 len 24576 data: 6 6 9 9 10 13
33188 056/300 Write block 1589 len 8192 data: 4 4
33189 057/300 Read block 2276 len 8192
33190 058/300 Write block 1439 len 16384 data: 8 8 9 9
33191 059/300 Read block 406 len 12288
33192 060/300 Write block 335 len 4096 data: 8
33193 061/300 Write block 2842 len 40960 data: 7 4 4 5 5 5 6 7 7 6
33194 062/300 Flush
33195 063/300 Read block 4975 len 16384
33196 064/300 Read block 3032 len 4096
33197 065/300 Write block 1323 len 12288 data: 6 5 6
33198 066/300 Flush
33199 067/300 Write block 4056 len 20480 data: 6 8 7 8 6
33200 068/300 Read block 2480 len 36864
33201 069/300 Flush
33202 070/300 Read block 4796 len 4096
33203 071/300 Read block 755 len 8192
33204 072/300 Write block 4817 len 8192 data: 4 4
33205 073/300 Read block 4745 len 8192
33206 074/300 Flush
33207 075/300 Read block 3639 len 32768
33208 076/300 Read block 823 len 12288
33209 077/300 Read block 4704 len 32768
33210 078/300 Write block 3452 len 4096 data: 6
33211 079/300 Read block 1216 len 32768
33212 080/300 Write block 3437 len 28672 data: 5 5 4 4 5 7 7
33213 081/300 Read block 1469 len 16384
33214 082/300 Read block 1953 len 16384
33215 083/300 Read block 4130 len 4096
33216 084/300 Read block 4818 len 24576
33217 085/300 Read block 1129 len 36864
33218 086/300 Write block 1927 len 40960 data: 7 6 6 6 5 5 5 4 5 5
33219 087/300 Write block 4694 len 28672 data: 5 4 3 2 2 3 4
33220 088/300 Write block 4689 len 36864 data: 6 7 7 8 6 6 5 4 3
33221 089/300 Read block 3336 len 8192
33222 090/300 Read block 464 len 36864
33223 091/300 Read block 2022 len 12288
33224 092/300 Write block 1613 len 32768 data: 5 5 6 6 8 8 8 7
33225 093/300 Read block 351 len 36864
33226 094/300 Read block 636 len 12288
33227 095/300 Read block 4848 len 20480
33228 096/300 Read block 2202 len 28672
33229 097/300 Read block 3796 len 4096
33230 098/300 Read block 1121 len 4096
33231 099/300 Write block 3320 len 20480 data: 4 4 5 6 6
33232 100/300 Flush
33233 101/300 Write block 242 len 24576 data: 6 6 7 6 5 4
33234 102/300 Read block 314 len 28672
33235 103/300 Flush
33236 104/300 Read block 3843 len 12288
33237 105/300 Read block 1431 len 8192
33238 106/300 Write block 3946 len 8192 data: 5 3
33239 107/300 Flush
33240 108/300 Read block 2307 len 12288
33241 109/300 Write block 2574 len 20480 data: 6 6 3 4 4
33242 110/300 Read block 4070 len 20480
33243 111/300 Write block 3110 len 24576 data: 4 3 3 3 3 4
33244 112/300 Write block 827 len 16384 data: 5 4 4 3
33245 113/300 Write block 2727 len 20480 data: 9 10 7 7 6
33246 114/300 Read block 1537 len 20480
33247 115/300 Write block 176 len 36864 data: 4 5 6 6 5 6 5 5 5
33248 116/300 Write block 2795 len 12288 data: 5 8 7
33249 117/300 Write block 3106 len 36864 data: 6 7 4 4 5 4 4 4 4
33250 118/300 Write block 735 len 16384 data: 3 3 3 6
33251 119/300 Write block 1999 len 32768 data: 8 6 5 5 5 7 5 5
33252 120/300 Read block 2530 len 20480
33253 121/300 Read block 2223 len 36864
33254 122/300 Write block 2618 len 4096 data: 13
33255 123/300 Write block 4198 len 12288 data: 5 6 6
33256 124/300 Write block 1809 len 24576 data: 7 7 7 4 6 6
33257 125/300 Flush
33258 126/300 Write block 4655 len 4096 data: 6
33259 127/300 Read block 4509 len 20480
33260 128/300 Read block 692 len 16384
33261 129/300 Read block 52 len 4096
33262 130/300 Write block 1077 len 4096 data: 5
33263 131/300 Read block 4096 len 4096
33264 132/300 Write block 723 len 8192 data: 4 5
33265 133/300 Read block 1784 len 20480
33266 134/300 Flush
33267 135/300 Write block 2750 len 28672 data: 5 7 7 9 8 7 5
33268 136/300 Write block 2783 len 28672 data: 5 4 4 4 5 5 5
33269 137/300 Read block 1714 len 40960
33270 138/300 Read block 875 len 28672
33271 139/300 Read block 1382 len 24576
33272 140/300 Write block 3401 len 4096 data: 11
33273 141/300 Read block 3701 len 40960
33274 142/300 Read block 1017 len 36864
33275 143/300 Write block 404 len 36864 data: 9 7 6 6 7 6 6 6 5
33276 144/300 Write block 1993 len 8192 data: 13 10
33277 145/300 Read block 4464 len 32768
33278 146/300 Write block 3071 len 12288 data: 3 3 4
33279 147/300 Read block 2235 len 4096
33280 148/300 Read block 377 len 4096
33281 149/300 Read block 2061 len 12288
33282 150/300 Write block 2406 len 32768 data: 7 6 8 7 9 7 7 7
33283 151/300 Write block 4072 len 16384 data: 4 4 4 3
33284 152/300 Read block 4170 len 32768
33285 153/300 Read block 4470 len 28672
33286 154/300 Read block 2982 len 8192
33287 155/300 Write block 2317 len 12288 data: 11 10 10
33288 156/300 Write block 1677 len 4096 data: 8
33289 157/300 Read block 4915 len 20480
33290 158/300 Read block 1018 len 40960
33291 159/300 Flush
33292 160/300 Flush
33293 161/300 Write block 2388 len 20480 data: 4 6 8 8 9
33294 162/300 Write block 2715 len 12288 data: 8 7 6
33295 163/300 Read block 2795 len 32768
33296 164/300 Read block 3061 len 12288
33297 165/300 Flush
33298 166/300 Flush
33299 167/300 Write block 4631 len 16384 data: 4 4 3 5
33300 168/300 Read block 3867 len 28672
33301 169/300 Read block 2336 len 28672
33302 170/300 Read block 4611 len 28672
33303 171/300 Write block 4524 len 36864 data: 7 6 4 3 2 3 3 3 3
33304 172/300 Write block 353 len 24576 data: 5 6 7 5 6 6
33305 173/300 Read block 3763 len 16384
33306 174/300 Read block 4128 len 28672
33307 175/300 Read block 696 len 32768
33308 176/300 Write block 3682 len 36864 data: 5 6 8 8 7 6 9 9 8
33309 177/300 Write block 1861 len 16384 data: 5 5 4 3
33310 178/300 Flush
33311 179/300 Read block 1182 len 16384
33312 180/300 Write block 2772 len 32768 data: 6 7 7 8 7 7 7 6
33313 181/300 Read block 3482 len 16384
33314 182/300 Write block 1569 len 4096 data: 8
33315 183/300 Flush
33316 184/300 Read block 4067 len 24576
33317 185/300 Read block 533 len 28672
33318 186/300 Write block 1280 len 40960 data: 8 7 7 5 5 3 3 2 3 3
33319 187/300 Write block 3943 len 28672 data: 6 4 3 6 4 4 5
33320 188/300 Write block 4296 len 16384 data: 9 7 8 8
33321 189/300 Read block 155 len 8192
33322 190/300 Flush
33323 191/300 Read block 4947 len 28672
33324 192/300 Read block 3197 len 4096
33325 193/300 Read block 123 len 4096
33326 194/300 Write block 3915 len 32768 data: 12 12 10 8 5 5 4 4
33327 195/300 Read block 4175 len 4096
33328 196/300 Write block 3565 len 8192 data: 10 10
33329 197/300 Read block 2332 len 16384
33330 198/300 Write block 447 len 12288 data: 7 8 10
33331 199/300 Read block 375 len 32768
33332 200/300 Read block 2150 len 28672
33333 201/300 Read block 2918 len 20480
33334 202/300 Read block 3602 len 32768
33335 203/300 Flush
33336 204/300 Flush
33337 205/300 Write block 2712 len 40960 data: 6 6 6 9 8 7 6 7 9 8
33338 206/300 Flush
33339 207/300 Read block 1290 len 24576
33340 208/300 Write block 341 len 32768 data: 4 5 5 6 6 6 6 9
33341 209/300 Write block 4627 len 36864 data: 4 5 4 4 5 5 4 6 6
33342 210/300 Flush
33343 211/300 Read block 4487 len 16384
33344 212/300 Write block 2177 len 12288 data: 5 4 3
33345 213/300 Read block 2336 len 4096
33346 214/300 Write block 3481 len 20480 data: 4 4 5 5 4
33347 215/300 Write block 2223 len 28672 data: 4 4 4 3 2 2 2
33348 216/300 Write block 4568 len 24576 data: 7 5 2 2 2 3
33349 217/300 Write block 3803 len 28672 data: 5 6 5 8 8 7 7
33350 218/300 Write block 2981 len 12288 data: 8 8 7
33351 219/300 Read block 4485 len 24576
33352 220/300 Write block 2286 len 8192 data: 7 6
33353 221/300 Flush
33354 222/300 Write block 2677 len 36864 data: 5 6 7 7 8 7 7 6 5
33355 223/300 Read block 2180 len 8192
33356 224/300 Read block 1763 len 32768
33357 225/300 Flush
33358 226/300 Write block 3683 len 28672 data: 7 9 9 8 7 10 10
33359 227/300 Write block 4106 len 16384 data: 5 4 4 5
33360 228/300 Read block 3544 len 8192
33361 229/300 Read block 1654 len 40960
33362 230/300 Read block 1178 len 36864
33363 231/300 Write block 527 len 36864 data: 7 6 8 7 7 6 4 6 6
33364 232/300 Write block 2438 len 12288 data: 6 4 3
33365 233/300 Write block 3897 len 4096 data: 8
33366 234/300 Read block 456 len 28672
33367 235/300 Read block 583 len 12288
33368 236/300 Write block 4214 len 16384 data: 6 5 4 5
33369 237/300 Read block 1966 len 36864
33370 238/300 Read block 806 len 28672
33371 239/300 Read block 3125 len 40960
33372 240/300 Read block 1162 len 28672
33373 241/300 Write block 2651 len 40960 data: 6 6 6 6 6 5 7 7 4 5
33374 242/300 Read block 2116 len 36864
33375 243/300 Write block 2939 len 20480 data: 7 6 5 5 4
33376 244/300 Read block 4899 len 20480
33377 245/300 Write block 1929 len 24576 data: 7 7 6 6 6 5
33378 246/300 Read block 1140 len 12288
33379 247/300 Read block 366 len 40960
33380 248/300 Read block 854 len 12288
33381 249/300 Read block 614 len 40960
33382 250/300 Read block 4710 len 28672
33383 251/300 Write block 827 len 40960 data: 6 5 5 4 3 3 4 6 6 8
33384 252/300 Write block 4552 len 40960 data: 3 2 2 4 5 7 7 7 8 8
33385 253/300 Flush
33386 254/300 Read block 4774 len 36864
33387 255/300 Write block 244 len 12288 data: 8 7 6
33388 256/300 Write block 3792 len 28672 data: 6 6 8 9 8 8 8
33389 257/300 Read block 4544 len 28672
33390 258/300 Write block 3889 len 4096 data: 6
33391 259/300 Write block 11 len 24576 data: 3 5 6 8 8 8
33392 260/300 Read block 4333 len 36864
33393 261/300 Write block 3215 len 28672 data: 5 4 4 4 4 6 6
33394 262/300 Write block 307 len 12288 data: 7 7 6
33395 263/300 Flush
33396 264/300 Write block 1758 len 16384 data: 7 5 6 4
33397 265/300 Write block 2784 len 32768 data: 5 5 5 6 6 6 4 3
33398 266/300 Read block 4141 len 40960
33399 267/300 Write block 1720 len 20480 data: 5 5 7 6 7
33400 268/300 Write block 1550 len 32768 data: 7 7 8 7 6 6 7 6
33401 269/300 Read block 2837 len 20480
33402 270/300 Write block 273 len 36864 data: 5 5 7 7 8 7 6 4 3
33403 271/300 Read block 2656 len 40960
33404 272/300 Write block 2293 len 32768 data: 4 2 2 2 3 3 3 4
33405 273/300 Write block 568 len 12288 data: 7 7 5
33406 274/300 Read block 264 len 12288
33407 275/300 Read block 2730 len 36864
33408 276/300 Read block 3157 len 8192
33409 277/300 Read block 2670 len 20480
33410 278/300 Read block 3595 len 28672
33411 279/300 Write block 1289 len 32768 data: 4 4 4 6 5 5 5 7
33412 280/300 Read block 137 len 24576
33413 281/300 Write block 2667 len 28672 data: 8 8 5 5 6 5 5
33414 282/300 Read block 90 len 8192
33415 283/300 Write block 1 len 28672 data: 3 5 6 6 6 5 4
33416 284/300 Read block 347 len 16384
33417 285/300 Write block 4606 len 8192 data: 4 3
33418 286/300 Write block 461 len 4096 data: 7
33419 287/300 Read block 4852 len 20480
33420 288/300 Read block 599 len 28672
33421 289/300 Read block 1483 len 40960
33422 290/300 Write block 4650 len 12288 data: 3 4 4
33423 291/300 Write block 2229 len 4096 data: 3
33424 292/300 Write block 3698 len 28672 data: 4 5 5 5 5 6 6
33425 293/300 Write block 1108 len 4096 data: 3
33426 294/300 Read block 1393 len 24576
33427 295/300 Read block 3335 len 12288
33428 296/300 Write block 2818 len 4096 data: 7
33429 297/300 Read block 3568 len 24576
33430 298/300 Read block 2569 len 12288
33431 299/300 Read block 3204 len 8192
33432 300/300 Write block 3908 len 40960 data: 6 6 7 9 10 11 10 13 13 11
33433 301/300 Flush
33434 Main task has work to do, go find it
33435 got dsc Start(0)
33436 start 0
33437 Jan 14 16:17:39.499 INFO request completed, latency_us: 224, response_code: 204, uri: /start/cid/0, method: POST, req_id: 84ed76f6-b241-4f51-bc41-671f597f392b, remote_addr: 127.0.0.1:61942, local_addr: 127.0.0.1:9998
33438 Replay: started 0, returned:Ok(())
33439 ----------------------------------------------------------------
33440 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
33441 States: Offline Active Active
33442 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33443 10056 Acked Write 7 Sent Done Done false
33444 10057 Acked Read 8 Sent Done Done false
33445 10058 Acked Read 3 Sent Done Done false
33446 10059 Acked Flush 0 Sent Done Done false
33447 10060 Acked Write 5 Sent Done Done false
33448 10061 Acked Flush 0 Sent Done Done false
33449 10062 Acked Read 9 Sent Done Done false
33450 10063 Acked Write 8 Sent Done Done false
33451 10064 Acked Read 9 Sent Done Done false
33452 10065 Acked Read 4 Sent Done Done false
33453 10066 Acked Flush 0 Sent Done Done false
33454 10067 Acked Read 1 Sent Done Done false
33455 10068 Acked Write 6 Sent Done Done false
33456 10069 Acked Read 8 Sent Done Done false
33457 10070 Acked Write 3 Sent Done Done false
33458 10071 Acked Read 1 Sent Done Done false
33459 10072 Acked Flush 0 Sent Done Done false
33460 10073 Acked Write 5 Sent Done Done false
33461 10074 Acked Write 1 Sent Done Done false
33462 10075 Acked Read 4 Sent Done Done false
33463 10076 Acked Write 4 Sent Done Done false
33464 10077 Acked Read 5 Sent Done Done false
33465 10078 Acked Write 2 Sent Done Done false
33466 10079 Acked Read 8 Sent Done Done false
33467 10080 Acked Write 2 Sent Done Done false
33468 10081 Acked Write 9 Sent Done Done false
33469 10082 Acked Read 7 Sent Done Done false
33470 10083 Acked Read 1 Sent Done Done false
33471 10084 Acked Read 8 Sent Done Done false
33472 10085 Acked Read 6 Sent Done Done false
33473 10086 Acked Read 5 Sent Done Done false
33474 10087 Acked Write 1 Sent Done Done false
33475 10088 Acked Read 7 Sent Done Done false
33476 10089 Acked Read 5 Sent Done Done false
33477 10090 Acked Flush 0 Sent Done Done false
33478 10091 Acked Write 3 Sent Done Done false
33479 10092 Acked Write 9 Sent Done Done false
33480 10093 Acked Write 4 Sent Done Done false
33481 10094 Acked Read 5 Sent Done Done false
33482 10095 Acked Write 6 Sent Done Done false
33483 10096 Acked Write 4 Sent Done Done false
33484 10097 Acked Write 4 Sent Done Done false
33485 10098 Acked Read 2 Sent Done Done false
33486 10099 Acked Read 4 Sent Done Done false
33487 10100 Acked Read 7 Sent Done Done false
33488 10101 Acked Read 7 Sent Done Done false
33489 10102 Acked Read 6 Sent Done Done false
33490 10103 Acked Read 3 Sent Done Done false
33491 10104 Acked Flush 0 Sent Done Done false
33492 10105 Acked Write 7 Sent Done Done false
33493 10106 Acked Read 9 Sent Done Done false
33494 10107 Acked Read 8 Sent Done Done false
33495 10108 Acked Read 2 Sent Done Done false
33496 10109 Acked Read 3 Sent Done Done false
33497 10110 Acked Write 6 Sent Done Done false
33498 10111 Acked Write 2 Sent Done Done false
33499 10112 Acked Read 2 Sent Done Done false
33500 10113 Acked Write 4 Sent Done Done false
33501 10114 Acked Read 3 Sent Done Done false
33502 10115 Acked Write 1 Sent Done Done false
33503 10116 Acked Write 10 Sent Done Done false
33504 10117 Acked Flush 0 Sent Done Done false
33505 10118 Acked Read 4 Sent Done Done false
33506 10119 Acked Read 1 Sent Done Done false
33507 10120 Acked Write 3 Sent Done Done false
33508 10121 Acked Flush 0 Sent Done Done false
33509 10122 Acked Write 5 Sent Done Done false
33510 10123 Acked Read 9 Sent Done Done false
33511 10124 Acked Flush 0 Sent Done Done false
33512 10125 Acked Read 1 Sent Done Done false
33513 10126 Acked Read 2 Sent Done Done false
33514 10127 Acked Write 2 Sent Done Done false
33515 10128 Acked Read 2 Sent Done Done false
33516 10129 Acked Flush 0 Sent Done Done false
33517 10130 Acked Read 8 Sent Done Done false
33518 10131 Acked Read 3 Sent Done Done false
33519 10132 Acked Read 8 Sent Done Done false
33520 10133 Acked Write 1 Sent Done Done false
33521 10134 Acked Read 8 Sent Done Done false
33522 10135 Acked Write 7 Sent Done Done false
33523 10136 Acked Read 4 Sent Done Done false
33524 10137 Acked Read 4 Sent Done Done false
33525 10138 Acked Read 1 Sent Done Done false
33526 10139 Acked Read 6 Sent Done Done false
33527 10140 Acked Read 9 Sent Done Done false
33528 10141 Acked Write 10 Sent Done Done false
33529 10142 Acked Write 7 Sent Done Done false
33530 10143 Acked Write 9 Sent Done Done false
33531 10144 Acked Read 2 Sent Done Done false
33532 10145 Acked Read 9 Sent Done Done false
33533 10146 Acked Read 3 Sent Done Done false
33534 10147 Acked Write 8 Sent Done Done false
33535 10148 Acked Read 9 Sent Done Done false
33536 10149 Acked Read 3 Sent Done Done false
33537 10150 Acked Read 5 Sent Done Done false
33538 10151 Acked Read 7 Sent Done Done false
33539 10152 Acked Read 1 Sent Done Done false
33540 10153 Acked Read 1 Sent Done Done false
33541 10154 Acked Write 5 Sent Done Done false
33542 10155 Acked Flush 0 Sent Done Done false
33543 10156 Acked Write 6 Sent Done Done false
33544 10157 Acked Read 7 Sent Done Done false
33545 10158 Acked Flush 0 Sent Done Done false
33546 10159 Acked Read 3 Sent Done Done false
33547 10160 Acked Read 2 Sent Done Done false
33548 10161 Acked Write 2 Sent Done Done false
33549 10162 Acked Flush 0 Sent Done Done false
33550 10163 Acked Read 3 Sent Done Done false
33551 10164 Acked Write 5 Sent Done Done false
33552 10165 Acked Read 5 Sent Done Done false
33553 10166 Acked Write 6 Sent Done Done false
33554 10167 Acked Write 4 Sent Done Done false
33555 10168 Acked Write 5 Sent Done Done false
33556 10169 Acked Read 5 Sent Done Done false
33557 10170 Acked Write 9 Sent Done Done false
33558 10171 Acked Write 3 Sent Done Done false
33559 10172 Acked Write 9 Sent Done Done false
33560 10173 Acked Write 4 Sent Done Done false
33561 10174 Acked Write 8 Sent Done Done false
33562 10175 Acked Read 5 Sent Done Done false
33563 10176 Acked Read 9 Sent Done Done false
33564 10177 Acked Write 1 Sent Done Done false
33565 10178 Acked Write 3 Sent Done Done false
33566 10179 Acked Write 6 Sent Done Done false
33567 10180 Acked Flush 0 Sent Done Done false
33568 10181 Acked Write 1 Sent Done Done false
33569 10182 Acked Read 5 Sent Done Done false
33570 10183 Acked Read 4 Sent Done Done false
33571 10184 Acked Read 1 Sent Done Done false
33572 10185 Acked Write 1 Sent Done Done false
33573 10186 Acked Read 1 Sent Done Done false
33574 10187 Acked Write 2 Sent Done Done false
33575 10188 Acked Read 5 Sent Done Done false
33576 10189 Acked Flush 0 Sent Done Done false
33577 10190 Acked Write 7 Sent Done Done false
33578 10191 Acked Write 7 Sent Done Done false
33579 10192 Acked Read 10 Sent Done Done false
33580 10193 Acked Read 7 Sent Done Done false
33581 10194 Acked Read 6 Sent Done Done false
33582 10195 Acked Write 1 Sent Done Done false
33583 10196 Acked Read 10 Sent Done Done false
33584 10197 Acked Read 9 Sent Done Done false
33585 10198 Acked Write 9 Sent Done Done false
33586 10199 Acked Write 2 Sent Done Done false
33587 10200 Acked Read 8 Sent Done Done false
33588 10201 Acked Write 3 Sent Done Done false
33589 10202 Acked Read 1 Sent Done Done false
33590 10203 Acked Read 1 Sent Done Done false
33591 10204 Acked Read 3 Sent Done Done false
33592 10205 Acked Write 8 Sent Done Done false
33593 10206 Acked Write 4 Sent Done Done false
33594 10207 Acked Read 8 Sent Done Done false
33595 10208 Acked Read 7 Sent Done Done false
33596 10209 Acked Read 2 Sent Done Done false
33597 10210 Acked Write 3 Sent Done Done false
33598 10211 Acked Write 1 Sent Done Done false
33599 10212 Acked Read 5 Sent Done Done false
33600 10213 Acked Read 10 Sent Done Done false
33601 10214 Acked Flush 0 Sent Done Done false
33602 10215 Acked Flush 0 Sent Done Done false
33603 10216 Acked Write 5 Sent Done Done false
33604 10217 Acked Write 3 Sent Done Done false
33605 10218 Acked Read 8 Sent Done Done false
33606 10219 Acked Read 3 Sent Done Done false
33607 10220 Acked Flush 0 Sent Done Done false
33608 10221 Acked Flush 0 Sent Done Done false
33609 10222 Acked Write 4 Sent Done Done false
33610 10223 Acked Read 7 Sent Done Done false
33611 10224 Acked Read 7 Sent Done Done false
33612 10225 Acked Read 7 Sent Done Done false
33613 10226 Acked Write 9 Sent Done Done false
33614 10227 Acked Write 6 Sent Done Done false
33615 10228 Acked Read 4 Sent Done Done false
33616 10229 Acked Read 7 Sent Done Done false
33617 10230 Acked Read 8 Sent Done Done false
33618 10231 Acked Write 9 Sent Done Done false
33619 10232 Acked Write 4 Sent Done Done false
33620 10233 Acked Flush 0 Sent Done Done false
33621 10234 Acked Read 4 Sent Done Done false
33622 10235 Acked Write 8 Sent Done Done false
33623 10236 Acked Read 4 Sent Done Done false
33624 10237 Acked Write 1 Sent Done Done false
33625 10238 Acked Flush 0 Sent Done Done false
33626 10239 Acked Read 6 Sent Done Done false
33627 10240 Acked Read 7 Sent Done Done false
33628 10241 Acked Write 10 Sent Done Done false
33629 10242 Acked Write 7 Sent Done Done false
33630 10243 Acked Write 4 Sent Done Done false
33631 10244 Acked Read 2 Sent Done Done false
33632 10245 Acked Flush 0 Sent Done Done false
33633 10246 Acked Read 7 Sent Done Done false
33634 10247 Acked Read 1 Sent Done Done false
33635 10248 Acked Read 1 Sent Done Done false
33636 10249 Acked Write 8 Sent Done Done false
33637 10250 Acked Read 1 Sent Done Done false
33638 10251 Acked Write 2 Sent Done Done false
33639 10252 Acked Read 4 Sent Done Done false
33640 10253 Acked Write 3 Sent Done Done false
33641 10254 Acked Read 8 Sent Done Done false
33642 10255 Acked Read 7 Sent Done Done false
33643 10256 Acked Read 5 Sent Done Done false
33644 10257 Acked Read 8 Sent Done Done false
33645 10258 Acked Flush 0 Sent Done Done false
33646 10259 Acked Flush 0 Sent Done Done false
33647 10260 Acked Write 10 Sent Done Done false
33648 10261 Acked Flush 0 Sent Done Done false
33649 10262 Acked Read 6 Sent Done Done false
33650 10263 Acked Write 8 Sent Done Done false
33651 10264 Acked Write 9 Sent Done Done false
33652 10265 Acked Flush 0 Sent Done Done false
33653 10266 Acked Read 4 Sent Done Done false
33654 10267 Acked Write 3 Sent Done Done false
33655 10268 Acked Read 1 Sent Done Done false
33656 10269 Acked Write 5 Sent Done Done false
33657 10270 Acked Write 7 Sent Done Done false
33658 10271 Acked Write 6 Sent Done Done false
33659 10272 Acked Write 7 Sent Done Done false
33660 10273 Acked Write 3 Sent Done Done false
33661 10274 Acked Read 6 Sent Done Done false
33662 10275 Acked Write 2 Sent Done Done false
33663 10276 Acked Flush 0 Sent Done Done false
33664 10277 Acked Write 9 Sent Done Done false
33665 10278 Acked Read 2 Sent Done Done false
33666 10279 Acked Read 8 Sent Done Done false
33667 10280 Acked Flush 0 Sent Done Done false
33668 10281 Acked Write 7 Sent Done Done false
33669 10282 Acked Write 4 Sent Done Done false
33670 10283 Acked Read 2 Sent Done Done false
33671 10284 Acked Read 10 Sent Done Done false
33672 10285 Acked Read 9 Sent Done Done false
33673 10286 Acked Write 9 Sent Done Done false
33674 10287 Acked Write 3 Sent Done Done false
33675 10288 Acked Write 1 Sent Done Done false
33676 10289 Acked Read 7 Sent Done Done false
33677 10290 Acked Read 3 Sent Done Done false
33678 10291 Acked Write 4 Sent Done Done false
33679 10292 Acked Read 9 Sent Done Done false
33680 10293 Acked Read 7 Sent Done Done false
33681 10294 Acked Read 10 Sent Done Done false
33682 10295 Acked Read 7 Sent Done Done false
33683 10296 Acked Write 10 Sent Done Done false
33684 10297 Acked Read 9 Sent Done Done false
33685 10298 Acked Write 5 Sent Done Done false
33686 10299 Acked Read 5 Sent Done Done false
33687 10300 Acked Write 6 Sent Done Done false
33688 10301 Acked Read 3 Sent Done Done false
33689 10302 Acked Read 10 Sent Done Done false
33690 10303 Acked Read 3 Sent Done Done false
33691 10304 Acked Read 10 Sent Done Done false
33692 10305 Acked Read 7 Sent Done Done false
33693 10306 Acked Write 10 Sent Done Done false
33694 10307 Acked Write 10 Sent Done Done false
33695 10308 Acked Flush 0 Sent Done Done false
33696 10309 Acked Read 9 Sent Done Done false
33697 10310 Acked Write 3 Sent Done Done false
33698 10311 Acked Write 7 Sent Done Done false
33699 10312 Acked Read 7 Sent Done Done false
33700 10313 Acked Write 1 Sent Done Done false
33701 10314 Acked Write 6 Sent Done Done false
33702 10315 Acked Read 9 Sent Done Done false
33703 10316 Acked Write 7 Sent Done Done false
33704 10317 Acked Write 3 Sent Done Done false
33705 10318 Acked Flush 0 Sent Done Done false
33706 10319 Acked Write 4 Sent Done Done false
33707 10320 Acked Write 8 Sent Done Done false
33708 10321 Acked Read 10 Sent Done Done false
33709 10322 Acked Write 5 Sent Done Done false
33710 10323 Acked Write 8 Sent Done Done false
33711 10324 Acked Read 5 Sent Done Done false
33712 10325 Acked Write 9 Sent Done Done false
33713 10326 Acked Read 10 Sent Done Done false
33714 10327 Acked Write 8 Sent Done Done false
33715 10328 Acked Write 3 Sent Done Done false
33716 10329 Acked Read 3 Sent Done Done false
33717 10330 Acked Read 9 Sent Done Done false
33718 10331 Acked Read 2 Sent Done Done false
33719 10332 Acked Read 5 Sent Done Done false
33720 10333 Acked Read 7 Sent Done Done false
33721 10334 Acked Write 8 Sent Done Done false
33722 10335 Acked Read 6 Sent Done Done false
33723 10336 Acked Write 7 Sent Done Done false
33724 10337 Acked Read 2 Sent Done Done false
33725 10338 Acked Write 7 Sent Done Done false
33726 10339 Acked Read 4 Sent Done Done false
33727 10340 Acked Write 2 Sent Done Done false
33728 10341 Acked Write 1 Sent Done Done false
33729 10342 Acked Read 5 Sent Done Done false
33730 10343 Acked Read 7 Sent Done Done false
33731 10344 Acked Read 10 Sent Done Done false
33732 10345 Acked Write 3 Sent Done Done false
33733 10346 Acked Write 1 Sent Done Done false
33734 10347 Acked Write 7 Sent Done Done false
33735 10348 Acked Write 1 Sent Done Done false
33736 10349 Acked Read 6 Sent Done Done false
33737 10350 Acked Read 3 Sent Done Done false
33738 10351 Acked Write 1 Sent Done Done false
33739 10352 Acked Read 6 Sent Done Done false
33740 10353 Acked Read 3 Sent Done Done false
33741 10354 Acked Read 2 Sent Done Done false
33742 10355 Acked Write 10 Sent Done Done false
33743 10356 Acked Flush 0 Sent Done Done false
33744 STATES DS:0 DS:1 DS:2 TOTAL
33745 Sent 301 0 0 301
33746 Done 0 301 301 602
33747 Skipped 0 0 0 0
33748 Error 0 0 0 0
33749 Last Flush: 10055 10356 10356
33750 Downstairs last five completed: 10055 10054 10053 10052 10051
33751 Upstairs last five completed: 10356 10355 10354 10353 10352
33752 CLIENT: Up:0 ds:301 act:2
33753 [8810] got start action
33754 [8810] I am going to restart
33755 Starting downstairs at port 8810
33756 Make output file at "/tmp/dsc/downstairs-8810.txt"
33757 [8810][0] reports Starting
33758 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1233)
33759 [8810][0] reports Running
33760 ----------------------------------------------------------------
33761 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
33762 States: Offline Active Active
33763 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33764 10056 Acked Write 7 Sent Done Done false
33765 10057 Acked Read 8 Sent Done Done false
33766 10058 Acked Read 3 Sent Done Done false
33767 10059 Acked Flush 0 Sent Done Done false
33768 10060 Acked Write 5 Sent Done Done false
33769 10061 Acked Flush 0 Sent Done Done false
33770 10062 Acked Read 9 Sent Done Done false
33771 10063 Acked Write 8 Sent Done Done false
33772 10064 Acked Read 9 Sent Done Done false
33773 10065 Acked Read 4 Sent Done Done false
33774 10066 Acked Flush 0 Sent Done Done false
33775 10067 Acked Read 1 Sent Done Done false
33776 10068 Acked Write 6 Sent Done Done false
33777 10069 Acked Read 8 Sent Done Done false
33778 10070 Acked Write 3 Sent Done Done false
33779 10071 Acked Read 1 Sent Done Done false
33780 10072 Acked Flush 0 Sent Done Done false
33781 10073 Acked Write 5 Sent Done Done false
33782 10074 Acked Write 1 Sent Done Done false
33783 10075 Acked Read 4 Sent Done Done false
33784 10076 Acked Write 4 Sent Done Done false
33785 10077 Acked Read 5 Sent Done Done false
33786 10078 Acked Write 2 Sent Done Done false
33787 10079 Acked Read 8 Sent Done Done false
33788 10080 Acked Write 2 Sent Done Done false
33789 10081 Acked Write 9 Sent Done Done false
33790 10082 Acked Read 7 Sent Done Done false
33791 10083 Acked Read 1 Sent Done Done false
33792 10084 Acked Read 8 Sent Done Done false
33793 10085 Acked Read 6 Sent Done Done false
33794 10086 Acked Read 5 Sent Done Done false
33795 10087 Acked Write 1 Sent Done Done false
33796 10088 Acked Read 7 Sent Done Done false
33797 10089 Acked Read 5 Sent Done Done false
33798 10090 Acked Flush 0 Sent Done Done false
33799 10091 Acked Write 3 Sent Done Done false
33800 10092 Acked Write 9 Sent Done Done false
33801 10093 Acked Write 4 Sent Done Done false
33802 10094 Acked Read 5 Sent Done Done false
33803 10095 Acked Write 6 Sent Done Done false
33804 10096 Acked Write 4 Sent Done Done false
33805 10097 Acked Write 4 Sent Done Done false
33806 10098 Acked Read 2 Sent Done Done false
33807 10099 Acked Read 4 Sent Done Done false
33808 10100 Acked Read 7 Sent Done Done false
33809 10101 Acked Read 7 Sent Done Done false
33810 10102 Acked Read 6 Sent Done Done false
33811 10103 Acked Read 3 Sent Done Done false
33812 10104 Acked Flush 0 Sent Done Done false
33813 10105 Acked Write 7 Sent Done Done false
33814 10106 Acked Read 9 Sent Done Done false
33815 10107 Acked Read 8 Sent Done Done false
33816 10108 Acked Read 2 Sent Done Done false
33817 10109 Acked Read 3 Sent Done Done false
33818 10110 Acked Write 6 Sent Done Done false
33819 10111 Acked Write 2 Sent Done Done false
33820 10112 Acked Read 2 Sent Done Done false
33821 10113 Acked Write 4 Sent Done Done false
33822 10114 Acked Read 3 Sent Done Done false
33823 10115 Acked Write 1 Sent Done Done false
33824 10116 Acked Write 10 Sent Done Done false
33825 10117 Acked Flush 0 Sent Done Done false
33826 10118 Acked Read 4 Sent Done Done false
33827 10119 Acked Read 1 Sent Done Done false
33828 10120 Acked Write 3 Sent Done Done false
33829 10121 Acked Flush 0 Sent Done Done false
33830 10122 Acked Write 5 Sent Done Done false
33831 10123 Acked Read 9 Sent Done Done false
33832 10124 Acked Flush 0 Sent Done Done false
33833 10125 Acked Read 1 Sent Done Done false
33834 10126 Acked Read 2 Sent Done Done false
33835 10127 Acked Write 2 Sent Done Done false
33836 10128 Acked Read 2 Sent Done Done false
33837 10129 Acked Flush 0 Sent Done Done false
33838 10130 Acked Read 8 Sent Done Done false
33839 10131 Acked Read 3 Sent Done Done false
33840 10132 Acked Read 8 Sent Done Done false
33841 10133 Acked Write 1 Sent Done Done false
33842 10134 Acked Read 8 Sent Done Done false
33843 10135 Acked Write 7 Sent Done Done false
33844 10136 Acked Read 4 Sent Done Done false
33845 10137 Acked Read 4 Sent Done Done false
33846 10138 Acked Read 1 Sent Done Done false
33847 10139 Acked Read 6 Sent Done Done false
33848 10140 Acked Read 9 Sent Done Done false
33849 10141 Acked Write 10 Sent Done Done false
33850 10142 Acked Write 7 Sent Done Done false
33851 10143 Acked Write 9 Sent Done Done false
33852 10144 Acked Read 2 Sent Done Done false
33853 10145 Acked Read 9 Sent Done Done false
33854 10146 Acked Read 3 Sent Done Done false
33855 10147 Acked Write 8 Sent Done Done false
33856 10148 Acked Read 9 Sent Done Done false
33857 10149 Acked Read 3 Sent Done Done false
33858 10150 Acked Read 5 Sent Done Done false
33859 10151 Acked Read 7 Sent Done Done false
33860 10152 Acked Read 1 Sent Done Done false
33861 10153 Acked Read 1 Sent Done Done false
33862 10154 Acked Write 5 Sent Done Done false
33863 10155 Acked Flush 0 Sent Done Done false
33864 10156 Acked Write 6 Sent Done Done false
33865 10157 Acked Read 7 Sent Done Done false
33866 10158 Acked Flush 0 Sent Done Done false
33867 10159 Acked Read 3 Sent Done Done false
33868 10160 Acked Read 2 Sent Done Done false
33869 10161 Acked Write 2 Sent Done Done false
33870 10162 Acked Flush 0 Sent Done Done false
33871 10163 Acked Read 3 Sent Done Done false
33872 10164 Acked Write 5 Sent Done Done false
33873 10165 Acked Read 5 Sent Done Done false
33874 10166 Acked Write 6 Sent Done Done false
33875 10167 Acked Write 4 Sent Done Done false
33876 10168 Acked Write 5 Sent Done Done false
33877 10169 Acked Read 5 Sent Done Done false
33878 10170 Acked Write 9 Sent Done Done false
33879 10171 Acked Write 3 Sent Done Done false
33880 10172 Acked Write 9 Sent Done Done false
33881 10173 Acked Write 4 Sent Done Done false
33882 10174 Acked Write 8 Sent Done Done false
33883 10175 Acked Read 5 Sent Done Done false
33884 10176 Acked Read 9 Sent Done Done false
33885 10177 Acked Write 1 Sent Done Done false
33886 10178 Acked Write 3 Sent Done Done false
33887 10179 Acked Write 6 Sent Done Done false
33888 10180 Acked Flush 0 Sent Done Done false
33889 10181 Acked Write 1 Sent Done Done false
33890 10182 Acked Read 5 Sent Done Done false
33891 10183 Acked Read 4 Sent Done Done false
33892 10184 Acked Read 1 Sent Done Done false
33893 10185 Acked Write 1 Sent Done Done false
33894 10186 Acked Read 1 Sent Done Done false
33895 10187 Acked Write 2 Sent Done Done false
33896 10188 Acked Read 5 Sent Done Done false
33897 10189 Acked Flush 0 Sent Done Done false
33898 10190 Acked Write 7 Sent Done Done false
33899 10191 Acked Write 7 Sent Done Done false
33900 10192 Acked Read 10 Sent Done Done false
33901 10193 Acked Read 7 Sent Done Done false
33902 10194 Acked Read 6 Sent Done Done false
33903 10195 Acked Write 1 Sent Done Done false
33904 10196 Acked Read 10 Sent Done Done false
33905 10197 Acked Read 9 Sent Done Done false
33906 10198 Acked Write 9 Sent Done Done false
33907 10199 Acked Write 2 Sent Done Done false
33908 10200 Acked Read 8 Sent Done Done false
33909 10201 Acked Write 3 Sent Done Done false
33910 10202 Acked Read 1 Sent Done Done false
33911 10203 Acked Read 1 Sent Done Done false
33912 10204 Acked Read 3 Sent Done Done false
33913 10205 Acked Write 8 Sent Done Done false
33914 10206 Acked Write 4 Sent Done Done false
33915 10207 Acked Read 8 Sent Done Done false
33916 10208 Acked Read 7 Sent Done Done false
33917 10209 Acked Read 2 Sent Done Done false
33918 10210 Acked Write 3 Sent Done Done false
33919 10211 Acked Write 1 Sent Done Done false
33920 10212 Acked Read 5 Sent Done Done false
33921 10213 Acked Read 10 Sent Done Done false
33922 10214 Acked Flush 0 Sent Done Done false
33923 10215 Acked Flush 0 Sent Done Done false
33924 10216 Acked Write 5 Sent Done Done false
33925 10217 Acked Write 3 Sent Done Done false
33926 10218 Acked Read 8 Sent Done Done false
33927 10219 Acked Read 3 Sent Done Done false
33928 10220 Acked Flush 0 Sent Done Done false
33929 10221 Acked Flush 0 Sent Done Done false
33930 10222 Acked Write 4 Sent Done Done false
33931 10223 Acked Read 7 Sent Done Done false
33932 10224 Acked Read 7 Sent Done Done false
33933 10225 Acked Read 7 Sent Done Done false
33934 10226 Acked Write 9 Sent Done Done false
33935 10227 Acked Write 6 Sent Done Done false
33936 10228 Acked Read 4 Sent Done Done false
33937 10229 Acked Read 7 Sent Done Done false
33938 10230 Acked Read 8 Sent Done Done false
33939 10231 Acked Write 9 Sent Done Done false
33940 10232 Acked Write 4 Sent Done Done false
33941 10233 Acked Flush 0 Sent Done Done false
33942 10234 Acked Read 4 Sent Done Done false
33943 10235 Acked Write 8 Sent Done Done false
33944 10236 Acked Read 4 Sent Done Done false
33945 10237 Acked Write 1 Sent Done Done false
33946 10238 Acked Flush 0 Sent Done Done false
33947 10239 Acked Read 6 Sent Done Done false
33948 10240 Acked Read 7 Sent Done Done false
33949 10241 Acked Write 10 Sent Done Done false
33950 10242 Acked Write 7 Sent Done Done false
33951 10243 Acked Write 4 Sent Done Done false
33952 10244 Acked Read 2 Sent Done Done false
33953 10245 Acked Flush 0 Sent Done Done false
33954 10246 Acked Read 7 Sent Done Done false
33955 10247 Acked Read 1 Sent Done Done false
33956 10248 Acked Read 1 Sent Done Done false
33957 10249 Acked Write 8 Sent Done Done false
33958 10250 Acked Read 1 Sent Done Done false
33959 10251 Acked Write 2 Sent Done Done false
33960 10252 Acked Read 4 Sent Done Done false
33961 10253 Acked Write 3 Sent Done Done false
33962 10254 Acked Read 8 Sent Done Done false
33963 10255 Acked Read 7 Sent Done Done false
33964 10256 Acked Read 5 Sent Done Done false
33965 10257 Acked Read 8 Sent Done Done false
33966 10258 Acked Flush 0 Sent Done Done false
33967 10259 Acked Flush 0 Sent Done Done false
33968 10260 Acked Write 10 Sent Done Done false
33969 10261 Acked Flush 0 Sent Done Done false
33970 10262 Acked Read 6 Sent Done Done false
33971 10263 Acked Write 8 Sent Done Done false
33972 10264 Acked Write 9 Sent Done Done false
33973 10265 Acked Flush 0 Sent Done Done false
33974 10266 Acked Read 4 Sent Done Done false
33975 10267 Acked Write 3 Sent Done Done false
33976 10268 Acked Read 1 Sent Done Done false
33977 10269 Acked Write 5 Sent Done Done false
33978 10270 Acked Write 7 Sent Done Done false
33979 10271 Acked Write 6 Sent Done Done false
33980 10272 Acked Write 7 Sent Done Done false
33981 10273 Acked Write 3 Sent Done Done false
33982 10274 Acked Read 6 Sent Done Done false
33983 10275 Acked Write 2 Sent Done Done false
33984 10276 Acked Flush 0 Sent Done Done false
33985 10277 Acked Write 9 Sent Done Done false
33986 10278 Acked Read 2 Sent Done Done false
33987 10279 Acked Read 8 Sent Done Done false
33988 10280 Acked Flush 0 Sent Done Done false
33989 10281 Acked Write 7 Sent Done Done false
33990 10282 Acked Write 4 Sent Done Done false
33991 10283 Acked Read 2 Sent Done Done false
33992 10284 Acked Read 10 Sent Done Done false
33993 10285 Acked Read 9 Sent Done Done false
33994 10286 Acked Write 9 Sent Done Done false
33995 10287 Acked Write 3 Sent Done Done false
33996 10288 Acked Write 1 Sent Done Done false
33997 10289 Acked Read 7 Sent Done Done false
33998 10290 Acked Read 3 Sent Done Done false
33999 10291 Acked Write 4 Sent Done Done false
34000 10292 Acked Read 9 Sent Done Done false
34001 10293 Acked Read 7 Sent Done Done false
34002 10294 Acked Read 10 Sent Done Done false
34003 10295 Acked Read 7 Sent Done Done false
34004 10296 Acked Write 10 Sent Done Done false
34005 10297 Acked Read 9 Sent Done Done false
34006 10298 Acked Write 5 Sent Done Done false
34007 10299 Acked Read 5 Sent Done Done false
34008 10300 Acked Write 6 Sent Done Done false
34009 10301 Acked Read 3 Sent Done Done false
34010 10302 Acked Read 10 Sent Done Done false
34011 10303 Acked Read 3 Sent Done Done false
34012 10304 Acked Read 10 Sent Done Done false
34013 10305 Acked Read 7 Sent Done Done false
34014 10306 Acked Write 10 Sent Done Done false
34015 10307 Acked Write 10 Sent Done Done false
34016 10308 Acked Flush 0 Sent Done Done false
34017 10309 Acked Read 9 Sent Done Done false
34018 10310 Acked Write 3 Sent Done Done false
34019 10311 Acked Write 7 Sent Done Done false
34020 10312 Acked Read 7 Sent Done Done false
34021 10313 Acked Write 1 Sent Done Done false
34022 10314 Acked Write 6 Sent Done Done false
34023 10315 Acked Read 9 Sent Done Done false
34024 10316 Acked Write 7 Sent Done Done false
34025 10317 Acked Write 3 Sent Done Done false
34026 10318 Acked Flush 0 Sent Done Done false
34027 10319 Acked Write 4 Sent Done Done false
34028 10320 Acked Write 8 Sent Done Done false
34029 10321 Acked Read 10 Sent Done Done false
34030 10322 Acked Write 5 Sent Done Done false
34031 10323 Acked Write 8 Sent Done Done false
34032 10324 Acked Read 5 Sent Done Done false
34033 10325 Acked Write 9 Sent Done Done false
34034 10326 Acked Read 10 Sent Done Done false
34035 10327 Acked Write 8 Sent Done Done false
34036 10328 Acked Write 3 Sent Done Done false
34037 10329 Acked Read 3 Sent Done Done false
34038 10330 Acked Read 9 Sent Done Done false
34039 10331 Acked Read 2 Sent Done Done false
34040 10332 Acked Read 5 Sent Done Done false
34041 10333 Acked Read 7 Sent Done Done false
34042 10334 Acked Write 8 Sent Done Done false
34043 10335 Acked Read 6 Sent Done Done false
34044 10336 Acked Write 7 Sent Done Done false
34045 10337 Acked Read 2 Sent Done Done false
34046 10338 Acked Write 7 Sent Done Done false
34047 10339 Acked Read 4 Sent Done Done false
34048 10340 Acked Write 2 Sent Done Done false
34049 10341 Acked Write 1 Sent Done Done false
34050 10342 Acked Read 5 Sent Done Done false
34051 10343 Acked Read 7 Sent Done Done false
34052 10344 Acked Read 10 Sent Done Done false
34053 10345 Acked Write 3 Sent Done Done false
34054 10346 Acked Write 1 Sent Done Done false
34055 10347 Acked Write 7 Sent Done Done false
34056 10348 Acked Write 1 Sent Done Done false
34057 10349 Acked Read 6 Sent Done Done false
34058 10350 Acked Read 3 Sent Done Done false
34059 10351 Acked Write 1 Sent Done Done false
34060 10352 Acked Read 6 Sent Done Done false
34061 10353 Acked Read 3 Sent Done Done false
34062 10354 Acked Read 2 Sent Done Done false
34063 10355 Acked Write 10 Sent Done Done false
34064 10356 Acked Flush 0 Sent Done Done false
34065 STATES DS:0 DS:1 DS:2 TOTAL
34066 Sent 301 0 0 301
34067 Done 0 301 301 602
34068 Skipped 0 0 0 0
34069 Error 0 0 0 0
34070 Last Flush: 10055 10356 10356
34071 Downstairs last five completed: 10055 10054 10053 10052 10051
34072 Upstairs last five completed: 10356 10355 10354 10353 10352
34073 CLIENT: Up:0 ds:301 act:2
340742025-01-14T16:17:45.268ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
340752025-01-14T16:17:45.268ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59711) = io task client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
340762025-01-14T16:17:45.269ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
340772025-01-14T16:17:45.269ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
340782025-01-14T16:17:45.269ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
340792025-01-14T16:17:45.269ZINFOcrucible: send last flush ID to this DS: 10055 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
340802025-01-14T16:17:45.270ZINFOcrucible: Replied this last flush ID: 10055 = downstairs client = 0 session_id = c2423ef0-728e-4895-997a-d9044c50948b
340812025-01-14T16:17:45.270ZINFOcrucible: [0] client re-new 301 jobs since flush 10055 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
340822025-01-14T16:17:45.270ZINFOcrucible: [0] Replayed 301 jobs since flush: 10055 = downstairs session_id = c2423ef0-728e-4895-997a-d9044c50948b
34083 ----------------------------------------------------------------
34084 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
34085 Downstairs last five completed: 10356 10355 10354 10353 10352
34086 Upstairs last five completed: 10356 10355 10354 10353 10352
34087 CLIENT: Up:0 ds:0 act:3
34088 Replay: All jobs finished
34089 Test replay has completed
34090 Wrote out file "/tmp/test_replay_verify.log"
34091 CLIENT: Tests done. All submitted work has been ACK'd
34092 ----------------------------------------------------------------
34093 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
34094 Downstairs last five completed: 10356 10355 10354 10353 10352
34095 Upstairs last five completed: 10356 10355 10354 10353 10352
34096 CLIENT: Up:0 ds:0 act:3
34097 CLIENT: All jobs finished, all DS active.
340982025-01-14T16:17:47.506ZWARNcrucible: Guest handle has been dropped
34099 Replays:30 time: 7:08
34100 Do final verify
34101 Jan 14 16:17:47.578 INFO accepted connection, remote_addr: 127.0.0.1:38533, local_addr: 127.0.0.1:9998
34102 Jan 14 16:17:47.579 INFO request completed, latency_us: 218, response_code: 200, uri: /regioninfo, method: GET, req_id: f65f2304-f10b-4ee2-83dc-d450653f7860, remote_addr: 127.0.0.1:38533, local_addr: 127.0.0.1:9998
341032025-01-14T16:17:47.580ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
34104 Jan 14 16:17:47.580 INFO request completed, latency_us: 83, response_code: 200, uri: /regioncount, method: GET, req_id: af355baa-0b42-4753-9cf3-4a35ed0f40dc, remote_addr: 127.0.0.1:38533, local_addr: 127.0.0.1:9998
341052025-01-14T16:17:47.580ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
34106 Jan 14 16:17:47.581 INFO request completed, latency_us: 113, response_code: 200, uri: /port/cid/0, method: GET, req_id: 63982cf4-76a3-433d-bb05-6f613a35e23f, remote_addr: 127.0.0.1:38533, local_addr: 127.0.0.1:9998
34107 Jan 14 16:17:47.581 INFO request completed, latency_us: 91, response_code: 200, uri: /port/cid/1, method: GET, req_id: 62ad2406-c63d-4b3e-940d-835a219fbf4a, remote_addr: 127.0.0.1:38533, local_addr: 127.0.0.1:9998
34108 Jan 14 16:17:47.582 INFO request completed, latency_us: 102, response_code: 200, uri: /port/cid/2, method: GET, req_id: a54cabd9-363c-45df-adc8-1e5e2437c088, remote_addr: 127.0.0.1:38533, local_addr: 127.0.0.1:9998
341092025-01-14T16:17:47.582ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
341102025-01-14T16:17:47.582ZINFOcrucible: Upstairs starts
341112025-01-14T16:17:47.582ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "42687c1346c9f5be1faf41362247db1201c426a8", git_commit_timestamp: "2025-01-14T15:54:39.000000000Z", git_branch: "mkeeter/separate-notify-task", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
341122025-01-14T16:17:47.582ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
341132025-01-14T16:17:47.582ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
341142025-01-14T16:17:47.583ZINFOcrucible: Crucible 97a2a4f6-6ec7-487c-8515-17c6c15d44cf has session id: 9081d82d-34a4-44db-a33d-a861e47967a1 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341152025-01-14T16:17:47.583ZINFOcrucible: Upstairs opts: Upstairs UUID: 97a2a4f6-6ec7-487c-8515-17c6c15d44cf, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341162025-01-14T16:17:47.583ZINFOcrucible: Crucible stats registered with UUID: 97a2a4f6-6ec7-487c-8515-17c6c15d44cf session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341172025-01-14T16:17:47.583ZWARNcrucible: could not find Downstairs address for Nexus session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341182025-01-14T16:17:47.584ZINFOcrucible: Downstairs in volume = 3 task = crutest
341192025-01-14T16:17:47.584ZINFOcrucible: The guest has requested activation with gen:3
341202025-01-14T16:17:47.584ZINFOcrucible: 97a2a4f6-6ec7-487c-8515-17c6c15d44cf active request set session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341212025-01-14T16:17:47.584ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341222025-01-14T16:17:47.584ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341232025-01-14T16:17:47.584ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341242025-01-14T16:17:47.584ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341252025-01-14T16:17:47.584ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341262025-01-14T16:17:47.584ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341272025-01-14T16:17:47.584ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341282025-01-14T16:17:47.584ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341292025-01-14T16:17:47.584ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341302025-01-14T16:17:47.584ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50102) = io task client = 1 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341312025-01-14T16:17:47.584ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59485) = io task client = 0 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341322025-01-14T16:17:47.584ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:44861) = io task client = 2 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341332025-01-14T16:17:47.585ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341342025-01-14T16:17:47.585ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341352025-01-14T16:17:47.585ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341362025-01-14T16:17:47.586ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341372025-01-14T16:17:47.586ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341382025-01-14T16:17:47.586ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341392025-01-14T16:17:47.586ZINFOcrucible: Waiting for more clients to be ready session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341402025-01-14T16:17:47.586ZINFOcrucible: Waiting for more clients to be ready session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341412025-01-14T16:17:47.586ZINFOcrucible: [0]R flush_numbers[0..12]: [967, 950, 967, 966, 967, 967, 927, 951, 965, 941, 951, 967] = downstairs session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341422025-01-14T16:17:47.587ZINFOcrucible: [0]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341432025-01-14T16:17:47.587ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341442025-01-14T16:17:47.587ZINFOcrucible: [1]R flush_numbers[0..12]: [967, 950, 967, 966, 967, 967, 927, 951, 965, 941, 951, 967] = downstairs session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341452025-01-14T16:17:47.587ZINFOcrucible: [1]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341462025-01-14T16:17:47.587ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341472025-01-14T16:17:47.587ZINFOcrucible: [2]R flush_numbers[0..12]: [967, 950, 967, 966, 967, 967, 927, 951, 965, 941, 951, 967] = downstairs session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341482025-01-14T16:17:47.587ZINFOcrucible: [2]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341492025-01-14T16:17:47.587ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341502025-01-14T16:17:47.587ZINFOcrucible: Max found gen is 3 = downstairs session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341512025-01-14T16:17:47.587ZINFOcrucible: Generation requested: 3 >= found:3 = downstairs session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341522025-01-14T16:17:47.587ZINFOcrucible: Next flush: 968 = downstairs session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
34153 {"msg":"All extents match","v":0,"name":"crucible","level":30Wait for a query_work_queue command to finish before sending IO
34154 ,"time":"2025-01-14T16:17:47.587163918Z","hostname":"w-01JHJS759ASXS54ZHYQM6RSJD4","pid":1238,"":"downstairs","":"downstairs","session_id":"9081d82d-34a4-44db-a33d-a861e47967a1"}
341552025-01-14T16:17:47.587ZINFOcrucible: No downstairs reconciliation required session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341562025-01-14T16:17:47.587ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341572025-01-14T16:17:47.587ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341582025-01-14T16:17:47.587ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341592025-01-14T16:17:47.587ZINFOcrucible: All required reconciliation work is completed session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341602025-01-14T16:17:47.587ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341612025-01-14T16:17:47.587ZINFOcrucible: 97a2a4f6-6ec7-487c-8515-17c6c15d44cf is now active with session: 9081d82d-34a4-44db-a33d-a861e47967a1 session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341622025-01-14T16:17:47.587ZINFOcrucible: Set Active after no reconciliation session_id = 9081d82d-34a4-44db-a33d-a861e47967a1
341632025-01-14T16:17:47.587ZINFOcrucible: The guest has finished waiting for activation with:3
34164 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
34165 Loading write count information from file "/tmp/test_replay_verify.log"
34166 Read and Verify all blocks (0..5000 range:false)
34167 Wrote out file "/tmp/test_replay_verify.log"
34168 Verify test completed
34169 Wrote out file "/tmp/test_replay_verify.log"
34170 CLIENT: Tests done. All submitted work has been ACK'd
34171 ----------------------------------------------------------------
34172 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:20
34173 States: Active Active Active
34174 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34175 1000 Acked Read 256 Done Done Done false
34176 1001 Acked Read 256 Done Done Done false
34177 1002 Acked Read 256 Done Done Done false
34178 1003 Acked Read 256 Done Done Done false
34179 1004 Acked Read 256 Done Done Done false
34180 1005 Acked Read 256 Done Done Done false
34181 1006 Acked Read 256 Done Done Done false
34182 1007 Acked Read 256 Done Done Done false
34183 1008 Acked Read 256 Done Done Done false
34184 1009 Acked Read 256 Done Done Done false
34185 1010 Acked Read 256 Done Done Done false
34186 1011 Acked Read 256 Done Done Done false
34187 1012 Acked Read 256 Done Done Done false
34188 1013 Acked Read 256 Done Done Done false
34189 1014 Acked Read 256 Done Done Done false
34190 1015 Acked Read 256 Done Done Done false
34191 1016 Acked Read 256 Done Done Done false
34192 1017 Acked Read 256 Done Done Done false
34193 1018 Acked Read 256 Done Done Done false
34194 1019 Acked Read 136 Done Done Done false
34195 STATES DS:0 DS:1 DS:2 TOTAL
34196 Sent 0 0 0 0
34197 Done 20 20 20 60
34198 Skipped 0 0 0 0
34199 Error 0 0 0 0
34200 Last Flush: 0 0 0
34201 Downstairs last five completed:
34202 Upstairs last five completed: 1019 1018 1017 1016 1015
34203 CLIENT: Up:0 ds:20 act:3
34204 ----------------------------------------------------------------
34205 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:0
34206 Downstairs last five completed: 1020 1019 1018 1017 1016
34207 Upstairs last five completed: 1020 1019 1018 1017 1016
34208 CLIENT: Up:0 ds:0 act:3
34209 CLIENT: All crucible jobs finished, exiting program
342102025-01-14T16:17:51.979ZWARNcrucible: Guest handle has been dropped
34211 Jan 14 16:17:52.044 INFO accepted connection, remote_addr: 127.0.0.1:56354, local_addr: 127.0.0.1:9998
34212 Main task has work to do, go find it
34213 got dsc Shutdown
34214 Shutdown
34215 Jan 14 16:17:52.044 INFO request completed, latency_us: 146, response_code: 204, uri: /shutdown, method: POST, req_id: 15e61afd-5023-42f3-9400-956220a7ceec, remote_addr: 127.0.0.1:56354[8810] Disable keep_running
34216 , [8810] Got stop action so:false kr:false
34217 local_addr: 127.0.0.1:9998
34218 Shut it down
34219 [8820] Disable keep_running
34220 [8820] Got stop action so:false kr:false
34221 [8830] Disable keep_running
34222 [8830] Got stop action so:false kr:false
34223 [8830] Exited with: ExitStatus(unix_wait_status(9))
34224 [8820] Exited with: ExitStatus(unix_wait_status(9))
34225 [8830][2] reports Exit
34226 [8820][1] reports Exit
34227 [8810] Exited with: ExitStatus(unix_wait_status(9))
34228 [8810][0] reports Exit
34229 January 14, 2025 at 04:17:57 PM UTC Test ends with 0