Max level shown:
1 starting January 13, 2025 at 03:51:40 PM UTC
2 Creating 3 downstairs regions
3 Creating region directory at: /var/tmp/dsc/region
4 Creating dsc directory at: /tmp/dsc
5 Downstairs region 0 created at /var/tmp/dsc/region/8810 in 0.263445
6 Downstairs region 1 created at /var/tmp/dsc/region/8820 in 0.020419247
7 Downstairs region 2 created at /var/tmp/dsc/region/8830 in 0.019203598
8 Created 3 regions
9 Starting 3 downstairs
10 Using existing output directory "/tmp/dsc"
11 Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 })
12 start ds: 8810
13 start ds: 8820
14 start access at:127.0.0.1:9998
15 start ds: 8830
16 Starting downstairs at port 8820
17 Starting downstairs at port 8810
18 Starting downstairs at port 8830
19 Make output file at "/tmp/dsc/downstairs-8820.txt"
20 Make output file at "/tmp/dsc/downstairs-8830.txt"
21 Make output file at "/tmp/dsc/downstairs-8810.txt"
22 [8820][1] initial start wait reports Starting
23 [8810][0] initial start wait reports Starting
24 [8830][2] initial start wait reports Starting
25 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1192)
26 [8820][1] initial start wait reports Running
27 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1193)
28 [8810][0] initial start wait reports Running
29 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1194)
30 [8830][2] initial start wait reports Running
31 All downstairs are running
32 Jan 13 15:51:41.222 INFO listening, local_addr: 127.0.0.1:9998
33 Control access at:127.0.0.1:9998
34 Running initial fill
35 Jan 13 15:51:46.290 INFO accepted connection, remote_addr: 127.0.0.1:54112, local_addr: 127.0.0.1:9998
36 Jan 13 15:51:46.292 INFO request completed, latency_us: 479, response_code: 200, uri: /regioninfo, method: GET, req_id: a02156b5-4229-4ce0-a3d3-3fbf0c56e2a6, remote_addr: 127.0.0.1:54112, local_addr: 127.0.0.1:9998
37 Jan 13 15:51:46.293 INFO request completed, latency_us: 118, response_code: 200, uri: /regioncount, method: GET, req_id: 6c28656e-4f36-4902-a0ca-b0d320a1b416, remote_addr: 127.0.0.1:54112, local_addr: 127.0.0.1:9998
382025-01-13T15:51:46.293ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
392025-01-13T15:51:46.294ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
40 Jan 13 15:51:46.294 INFO request completed, latency_us: 164, response_code: 200, uri: /port/cid/0, method: GET, req_id: b0d97010-97c1-4636-b810-05c7fa736768, remote_addr: 127.0.0.1:54112, local_addr: 127.0.0.1:9998
41 Jan 13 15:51:46.295 INFO request completed, latency_us: 93, response_code: 200, uri: /port/cid/1, method: GET, req_id: 9a40d7d6-5e75-4640-9b11-b464eb170bf8, remote_addr: 127.0.0.1:54112, local_addr: 127.0.0.1:9998
42 Jan 13 15:51:46.295 INFO request completed, latency_us: 96, response_code: 200, uri: /port/cid/2, method: GET, req_id: c33635a3-5299-4dcd-9d36-4581746ec1b7, remote_addr: 127.0.0.1:54112, local_addr: 127.0.0.1:9998
432025-01-13T15:51:46.295ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
442025-01-13T15:51:46.296ZINFOcrucible: Upstairs starts
452025-01-13T15:51:46.296ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "12c38e1c2ca92eb8e00f82331ef94fdbf7d3f8af", git_commit_timestamp: "2025-01-13T15:35:09.000000000Z", git_branch: "mkeeter/lean-mean-state-machine", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
462025-01-13T15:51:46.296ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
472025-01-13T15:51:46.296ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
482025-01-13T15:51:46.297ZINFOcrucible: Crucible 0ae85eae-c69d-4bc4-83cf-63e7293cced5 has session id: 7a2ffe81-67a3-4652-b277-e57a1ba4af69 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
492025-01-13T15:51:46.297ZINFOcrucible: Upstairs opts: Upstairs UUID: 0ae85eae-c69d-4bc4-83cf-63e7293cced5, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
502025-01-13T15:51:46.297ZINFOcrucible: Crucible stats registered with UUID: 0ae85eae-c69d-4bc4-83cf-63e7293cced5 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
512025-01-13T15:51:46.332ZINFOcrucible: Downstairs in volume = 3 task = crutest
522025-01-13T15:51:46.332ZINFOcrucible: The guest has requested activation with gen:1
532025-01-13T15:51:46.332ZINFOcrucible: 0ae85eae-c69d-4bc4-83cf-63e7293cced5 active request set session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
542025-01-13T15:51:46.332ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
552025-01-13T15:51:46.332ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
562025-01-13T15:51:46.332ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
572025-01-13T15:51:46.332ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
582025-01-13T15:51:46.332ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
592025-01-13T15:51:46.332ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
602025-01-13T15:51:46.332ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
612025-01-13T15:51:46.332ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
622025-01-13T15:51:46.332ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
632025-01-13T15:51:46.332ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:33066) = io task client = 2 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
642025-01-13T15:51:46.332ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:47941) = io task client = 0 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
652025-01-13T15:51:46.332ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59023) = io task client = 1 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
662025-01-13T15:51:46.334ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
672025-01-13T15:51:46.334ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
682025-01-13T15:51:46.335ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
692025-01-13T15:51:46.336ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
702025-01-13T15:51:46.336ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
712025-01-13T15:51:46.336ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
722025-01-13T15:51:46.336ZINFOcrucible: Waiting for more clients to be ready session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
732025-01-13T15:51:46.337ZINFOcrucible: Waiting for more clients to be ready session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
742025-01-13T15:51:46.337ZINFOcrucible: [0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
752025-01-13T15:51:46.337ZINFOcrucible: [0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
762025-01-13T15:51:46.337ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
772025-01-13T15:51:46.337ZINFOcrucible: [1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
782025-01-13T15:51:46.337ZINFOcrucible: [1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
792025-01-13T15:51:46.337ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
802025-01-13T15:51:46.337ZINFOcrucible: [2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
812025-01-13T15:51:46.337ZINFOcrucible: [2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
822025-01-13T15:51:46.337ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
832025-01-13T15:51:46.337ZINFOcrucible: Max found gen is 1 = downstairs session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
842025-01-13T15:51:46.337ZINFOcrucible: Generation requested: 1 >= found:1 = downstairs session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
852025-01-13T15:51:46.337ZINFOcrucible: Next flush: 1 = downstairs session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
862025-01-13T15:51:46.337ZINFOcrucible: All extents match = downstairs session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
872025-01-13T15:51:46.337ZINFOcrucible: No downstairs reconciliation required session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
882025-01-13T15:51:46.337ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
892025-01-13T15:51:46.337ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
902025-01-13T15:51:46.337ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
912025-01-13T15:51:46.337ZINFOcrucible: All required reconciliation work is completed session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
922025-01-13T15:51:46.337ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
932025-01-13T15:51:46.337ZINFOcrucible: 0ae85eae-c69d-4bc4-83cf-63e7293cced5 is now active with session: 7a2ffe81-67a3-4652-b277-e57a1ba4af69 session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
942025-01-13T15:51:46.337ZINFOcrucible: Set Active after no reconciliation session_id = 7a2ffe81-67a3-4652-b277-e57a1ba4af69
952025-01-13T15:51:46.337ZINFOcrucible: The guest has finished waiting for activation with:1
96 Activate successful
97 Wait for a query_work_queue command to finish before sending IO
98 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
99 Fill test
100 Read and Verify all blocks (0..5000 range:false)
101 Wrote out file "/tmp/test_replay_verify.log"
102 CLIENT: Tests done. All submitted work has been ACK'd
103 ----------------------------------------------------------------
104 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:20
105 States: Active Active Active
106 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
107 1021 Acked Read 256 Done Done Done false
108 1022 Acked Read 256 Done Done Done false
109 1023 Acked Read 256 Done Done Done false
110 1024 Acked Read 256 Done Done Done false
111 1025 Acked Read 256 Done Done Done false
112 1026 Acked Read 256 Done Done Done false
113 1027 Acked Read 256 Done Done Done false
114 1028 Acked Read 256 Done Done Done false
115 1029 Acked Read 256 Done Done Done false
116 1030 Acked Read 256 Done Done Done false
117 1031 Acked Read 256 Done Done Done false
118 1032 Acked Read 256 Done Done Done false
119 1033 Acked Read 256 Done Done Done false
120 1034 Acked Read 256 Done Done Done false
121 1035 Acked Read 256 Done Done Done false
122 1036 Acked Read 256 Done Done Done false
123 1037 Acked Read 256 Done Done Done false
124 1038 Acked Read 256 Done Done Done false
125 1039 Acked Read 256 Done Done Done false
126 1040 Acked Read 136 Done Done Done false
127 STATES DS:0 DS:1 DS:2 TOTAL
128 Sent 0 0 0 0
129 Done 20 20 20 60
130 Skipped 0 0 0 0
131 Error 0 0 0 0
132 Last Flush: 1020 1020 1020
133 Downstairs last five completed: 1020 1019 1018 1017 1016
134 Upstairs last five completed: 1040 1039 1038 1037 1036
135 CLIENT: Up:0 ds:20 act:3
136 ----------------------------------------------------------------
137 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0
138 Downstairs last five completed: 1041 1040 1039 1038 1037
139 Upstairs last five completed: 1041 1040 1039 1038 1037
140 CLIENT: Up:0 ds:0 act:3
141 CLIENT: All crucible jobs finished, exiting program
1422025-01-13T15:51:50.886ZWARNcrucible: Guest handle has been dropped
143 Replay loop starts now January 13, 2025 at 03:51:50 PM UTC
144 Jan 13 15:51:50.959 INFO accepted connection, remote_addr: 127.0.0.1:62151, local_addr: 127.0.0.1:9998
145 Jan 13 15:51:50.960 INFO request completed, latency_us: 177, response_code: 200, uri: /regioninfo, method: GET, req_id: e911e683-8ba8-470b-b03f-fcdb8603b50b, remote_addr: 127.0.0.1:62151, local_addr: 127.0.0.1:9998
1462025-01-13T15:51:50.960ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
147 Jan 13 15:51:50.961 INFO request completed, latency_us: 97, response_code: 200, uri: /regioncount, method: GET, req_id: 8928f3d0-f98e-4f43-bcbe-46b1b1ee7f80, remote_addr: 127.0.0.1:62151, local_addr: 127.0.0.1:9998
1482025-01-13T15:51:50.961ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
149 Jan 13 15:51:50.961 INFO request completed, latency_us: 115, response_code: 200, uri: /port/cid/0, method: GET, req_id: a8580063-c859-4c07-86ef-e196732ab867, remote_addr: 127.0.0.1:62151, local_addr: 127.0.0.1:9998
150 Jan 13 15:51:50.962 INFO request completed, latency_us: 96, response_code: 200, uri: /port/cid/1, method: GET, req_id: 1cb6329a-a1dc-4c03-97e5-547ba09b4b49, remote_addr: 127.0.0.1:62151, local_addr: 127.0.0.1:9998
151 Jan 13 15:51:50.963 INFO request completed, latency_us: 96, response_code: 200, uri: /port/cid/2, method: GET, req_id: 53bd1e33-97de-45d1-9c33-a01cbc4d17f6, remote_addr: 127.0.0.1:62151, local_addr: 127.0.0.1:9998
1522025-01-13T15:51:50.963ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
1532025-01-13T15:51:50.963ZINFOcrucible: Upstairs starts
1542025-01-13T15:51:50.963ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "12c38e1c2ca92eb8e00f82331ef94fdbf7d3f8af", git_commit_timestamp: "2025-01-13T15:35:09.000000000Z", git_branch: "mkeeter/lean-mean-state-machine", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
1552025-01-13T15:51:50.963ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
1562025-01-13T15:51:50.963ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
1572025-01-13T15:51:50.964ZINFOcrucible: Crucible e4e90995-7f90-410f-969b-ac8b15d308f1 has session id: cd58429c-c491-4a1e-aa01-02db1d6a3d79 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1582025-01-13T15:51:50.964ZINFOcrucible: Upstairs opts: Upstairs UUID: e4e90995-7f90-410f-969b-ac8b15d308f1, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1592025-01-13T15:51:50.964ZINFOcrucible: Crucible stats registered with UUID: e4e90995-7f90-410f-969b-ac8b15d308f1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1602025-01-13T15:51:50.997ZINFOcrucible: Downstairs in volume = 3 task = crutest
1612025-01-13T15:51:50.997ZINFOcrucible: The guest has requested activation with gen:2
1622025-01-13T15:51:50.997ZINFOcrucible: e4e90995-7f90-410f-969b-ac8b15d308f1 active request set session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1632025-01-13T15:51:50.998ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1642025-01-13T15:51:50.998ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1652025-01-13T15:51:50.998ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1662025-01-13T15:51:50.998ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1672025-01-13T15:51:50.998ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1682025-01-13T15:51:50.998ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1692025-01-13T15:51:50.998ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1702025-01-13T15:51:50.998ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1712025-01-13T15:51:50.998ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1722025-01-13T15:51:50.998ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61932) = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1732025-01-13T15:51:50.998ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57378) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1742025-01-13T15:51:50.998ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:44630) = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1752025-01-13T15:51:50.999ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1762025-01-13T15:51:50.999ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1772025-01-13T15:51:50.999ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1782025-01-13T15:51:50.999ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1792025-01-13T15:51:50.999ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1802025-01-13T15:51:50.999ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1812025-01-13T15:51:51.000ZINFOcrucible: Waiting for more clients to be ready session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1822025-01-13T15:51:51.000ZINFOcrucible: Waiting for more clients to be ready session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1832025-01-13T15:51:51.000ZINFOcrucible: [0]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1842025-01-13T15:51:51.000ZINFOcrucible: [0]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1852025-01-13T15:51:51.000ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1862025-01-13T15:51:51.000ZINFOcrucible: [1]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1872025-01-13T15:51:51.000ZINFOcrucible: [1]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1882025-01-13T15:51:51.000ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1892025-01-13T15:51:51.000ZINFOcrucible: [2]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1902025-01-13T15:51:51.000ZINFOcrucible: [2]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1912025-01-13T15:51:51.000ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1922025-01-13T15:51:51.000ZINFOcrucible: Max found gen is 2 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1932025-01-13T15:51:51.000ZINFOcrucible: Generation requested: 2 >= found:2 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1942025-01-13T15:51:51.000ZINFOcrucible: Next flush: 2 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1952025-01-13T15:51:51.000ZINFOcrucible: All extents match = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1962025-01-13T15:51:51.000ZINFOcrucible: No downstairs reconciliation required session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1972025-01-13T15:51:51.001ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1982025-01-13T15:51:51.001ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1992025-01-13T15:51:51.001ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
2002025-01-13T15:51:51.001ZINFOcrucible: All required reconciliation work is completed session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
2012025-01-13T15:51:51.001ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
2022025-01-13T15:51:51.001ZINFOcrucible: e4e90995-7f90-410f-969b-ac8b15d308f1 is now active with session: cd58429c-c491-4a1e-aa01-02db1d6a3d79 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
2032025-01-13T15:51:51.001ZINFOcrucible: Set Active after no reconciliation session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
2042025-01-13T15:51:51.001ZINFOcrucible: The guest has finished waiting for activation with:2
205 Activate successful
206 Wait for a query_work_queue command to finish before sending IO
207 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
208 Loading write count information from file "/tmp/test_replay_verify.log"
209 Jan 13 15:51:51.035 INFO accepted connection, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
210 Main task has work to do, go find it
211 got dsc Stop(1)
212 stop 1
213 Jan 13 15:51:51.036 INFO request completed, latency_us: 230, response_code: 204, uri: /stop/cid/1[8820] Got stop action so:false kr:false
214 , method: POST, req_id: 5ba4de93-c791-4921-9443-f84ca59ef0f6, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
2152025-01-13T15:51:51.036ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
2162025-01-13T15:51:51.037ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
2172025-01-13T15:51:51.037ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
218 Jan 13 15:51:51.037 INFO request completed, latency_us: 159, response_code: 200, uri: /state/cid/1, method: GET, req_id: ab08d58c-b278-459c-876e-4afeab8e116e, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
2192025-01-13T15:51:51.037ZINFOcrucible: client task is exiting = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
220 [8820] Exited with: ExitStatus(unix_wait_status(9))
221 [8820][1] reports Exit
222 Jan 13 15:51:55.040 INFO request completed, latency_us: 336, response_code: 200, uri: /state/cid/1, method: GET, req_id: a6a0b166-03ce-4bb4-8a7f-00b0f7859a88, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
223 001/300 Write block 2027 len 20480 data: 2 2 2 2 2
224 002/300 Read block 3235 len 20480
225 003/300 Write block 3378 len 36864 data: 2 2 2 2 2 2 2 2 2
226 004/300 Flush
227 005/300 Flush
228 006/300 Read block 2323 len 32768
229 007/300 Write block 474 len 24576 data: 2 2 2 2 2 2
230 008/300 Read block 2847 len 24576
231 009/300 Flush
232 010/300 Read block 4093 len 16384
233 011/300 Read block 1382 len 40960
234 012/300 Write block 3646 len 8192 data: 2 2
235 013/300 Flush
236 014/300 Write block 3959 len 28672 data: 2 2 2 2 2 2 2
237 015/300 Read block 2941 len 32768
238 016/300 Read block 3440 len 16384
239 017/300 Flush
240 018/300 Write block 522 len 16384 data: 2 2 2 2
241 019/300 Read block 2961 len 28672
242 020/300 Write block 398 len 24576 data: 2 2 2 2 2 2
243 021/300 Write block 2792 len 36864 data: 2 2 2 2 2 2 2 2 2
244 022/300 Write block 3283 len 4096 data: 2
245 023/300 Read block 4644 len 12288
246 024/300 Write block 1582 len 40960 data: 2 2 2 2 2 2 2 2 2 2
247 025/300 Write block 1247 len 12288 data: 2 2 2
248 026/300 Flush
249 027/300 Read block 1912 len 16384
250 028/300 Flush
251 029/300 Read block 4774 len 8192
252 030/300 Write block 4642 len 16384 data: 2 2 2 2
253 031/300 Read block 2014 len 16384
254 032/300 Read block 1343 len 36864
255 033/300 Read block 3450 len 36864
256 034/300 Read block 4552 len 4096
257 035/300 Read block 2098 len 36864
258 036/300 Write block 4157 len 36864 data: 2 2 2 2 2 2 2 2 2
259 037/300 Read block 3254 len 36864
260 038/300 Write block 115 len 8192 data: 2 2
261 039/300 Read block 4226 len 4096
262 040/300 Read block 146 len 20480
263 041/300 Read block 1775 len 32768
264 042/300 Write block 4980 len 40960 data: 2 2 2 2 2 2 2 2 2 2
265 043/300 Read block 1871 len 40960
266 044/300 Write block 660 len 28672 data: 2 2 2 2 2 2 2
267 045/300 Write block 1426 len 20480 data: 2 2 2 2 2
268 046/300 Read block 3891 len 32768
269 047/300 Flush
270 048/300 Flush
271 049/300 Flush
272 050/300 Write block 860 len 32768 data: 2 2 2 2 2 2 2 2
273 051/300 Read block 4306 len 36864
274 052/300 Write block 3196 len 28672 data: 2 2 2 2 2 2 2
275 053/300 Flush
276 054/300 Read block 1613 len 40960
277 055/300 Read block 1182 len 8192
278 056/300 Read block 573 len 36864
279 057/300 Write block 3076 len 24576 data: 2 2 2 2 2 2
280 058/300 Read block 2260 len 24576
281 059/300 Write block 2221 len 28672 data: 2 2 2 2 2 2 2
282 060/300 Write block 1426 len 20480 data: 3 3 3 3 3
283 061/300 Flush
284 062/300 Write block 2706 len 12288 data: 2 2 2
285 063/300 Flush
286 064/300 Flush
287 065/300 Read block 1911 len 16384
288 066/300 Write block 2602 len 40960 data: 2 2 2 2 2 2 2 2 2 2
289 067/300 Write block 1428 len 40960 data: 4 4 4 2 2 2 2 2 2 2
290 068/300 Write block 1758 len 36864 data: 2 2 2 2 2 2 2 2 2
291 069/300 Read block 2677 len 12288
292 070/300 Read block 3546 len 36864
293 071/300 Read block 257 len 20480
294 072/300 Write block 4225 len 28672 data: 2 2 2 2 2 2 2
295 073/300 Write block 1971 len 4096 data: 2
296 074/300 Read block 4934 len 12288
297 075/300 Read block 3379 len 16384
298 076/300 Write block 270 len 24576 data: 2 2 2 2 2 2
299 077/300 Read block 3631 len 20480
300 078/300 Write block 4579 len 12288 data: 2 2 2
301 079/300 Write block 1004 len 36864 data: 2 2 2 2 2 2 2 2 2
302 080/300 Read block 606 len 4096
303 081/300 Read block 4485 len 12288
304 082/300 Write block 3093 len 36864 data: 2 2 2 2 2 2 2 2 2
305 083/300 Read block 269 len 32768
306 084/300 Write block 4200 len 32768 data: 2 2 2 2 2 2 2 2
307 085/300 Read block 3720 len 16384
308 086/300 Write block 1823 len 24576 data: 2 2 2 2 2 2
309 087/300 Read block 4529 len 4096
310 088/300 Read block 4787 len 36864
311 089/300 Write block 2093 len 40960 data: 2 2 2 2 2 2 2 2 2 2
312 090/300 Read block 2645 len 28672
313 091/300 Read block 3317 len 12288
314 092/300 Write block 3944 len 28672 data: 2 2 2 2 2 2 2
315 093/300 Write block 3486 len 28672 data: 2 2 2 2 2 2 2
316 094/300 Read block 2229 len 16384
317 095/300 Write block 117 len 16384 data: 2 2 2 2
318 096/300 Read block 3080 len 32768
319 097/300 Write block 4324 len 20480 data: 2 2 2 2 2
320 098/300 Read block 1157 len 4096
321 099/300 Read block 201 len 12288
322 100/300 Read block 377 len 8192
323 101/300 Read block 3227 len 12288
324 102/300 Read block 347 len 24576
325 103/300 Read block 3243 len 12288
326 104/300 Read block 2261 len 20480
327 105/300 Flush
328 106/300 Read block 1400 len 16384
329 107/300 Read block 1513 len 16384
330 108/300 Write block 1739 len 28672 data: 2 2 2 2 2 2 2
331 109/300 Read block 4316 len 8192
332 110/300 Read block 4947 len 4096
333 111/300 Write block 4106 len 12288 data: 2 2 2
334 112/300 Write block 4193 len 12288 data: 2 2 2
335 113/300 Write block 1052 len 40960 data: 2 2 2 2 2 2 2 2 2 2
336 114/300 Write block 538 len 36864 data: 2 2 2 2 2 2 2 2 2
337 115/300 Flush
338 116/300 Write block 1568 len 12288 data: 2 2 2
339 117/300 Read block 4464 len 16384
340 118/300 Read block 1139 len 24576
341 119/300 Write block 2782 len 40960 data: 2 2 2 2 2 2 2 2 2 2
342 120/300 Read block 453 len 28672
343 121/300 Read block 4464 len 8192
344 122/300 Read block 2733 len 24576
345 123/300 Read block 2485 len 20480
346 124/300 Write block 1604 len 32768 data: 2 2 2 2 2 2 2 2
347 125/300 Write block 1143 len 40960 data: 2 2 2 2 2 2 2 2 2 2
348 126/300 Read block 2851 len 28672
349 127/300 Write block 2262 len 40960 data: 2 2 2 2 2 2 2 2 2 2
350 128/300 Write block 1939 len 24576 data: 2 2 2 2 2 2
351 129/300 Flush
352 130/300 Read block 3972 len 40960
353 131/300 Read block 2018 len 8192
354 132/300 Read block 606 len 20480
355 133/300 Read block 1034 len 40960
356 134/300 Write block 576 len 40960 data: 2 2 2 2 2 2 2 2 2 2
357 135/300 Read block 4953 len 8192
358 136/300 Write block 1579 len 16384 data: 2 2 2 3
359 137/300 Read block 1507 len 12288
360 138/300 Write block 4316 len 20480 data: 2 2 2 2 2
361 139/300 Write block 3887 len 12288 data: 2 2 2
362 140/300 Flush
363 141/300 Read block 220 len 40960
364 142/300 Write block 2497 len 28672 data: 2 2 2 2 2 2 2
365 143/300 Read block 1025 len 12288
366 144/300 Write block 4862 len 36864 data: 2 2 2 2 2 2 2 2 2
367 145/300 Read block 1955 len 40960
368 146/300 Read block 3637 len 36864
369 147/300 Read block 1596 len 40960
370 148/300 Write block 4888 len 32768 data: 2 2 2 2 2 2 2 2
371 149/300 Read block 3022 len 20480
372 150/300 Read block 773 len 28672
373 151/300 Read block 2031 len 40960
374 152/300 Read block 206 len 24576
375 153/300 Write block 771 len 36864 data: 2 2 2 2 2 2 2 2 2
376 154/300 Write block 1789 len 32768 data: 2 2 2 2 2 2 2 2
377 155/300 Read block 2996 len 12288
378 156/300 Read block 3614 len 12288
379 157/300 Write block 574 len 20480 data: 2 2 3 3 3
380 158/300 Flush
381 159/300 Flush
382 160/300 Read block 1915 len 12288
383 161/300 Flush
384 162/300 Read block 2332 len 16384
385 163/300 Read block 558 len 4096
386 164/300 Read block 2565 len 4096
387 165/300 Write block 445 len 40960 data: 2 2 2 2 2 2 2 2 2 2
388 166/300 Write block 3997 len 32768 data: 2 2 2 2 2 2 2 2
389 167/300 Read block 4504 len 32768
390 168/300 Read block 2832 len 40960
391 169/300 Write block 3009 len 32768 data: 2 2 2 2 2 2 2 2
392 170/300 Read block 4097 len 4096
393 171/300 Read block 715 len 16384
394 172/300 Write block 4962 len 40960 data: 2 2 2 2 2 2 2 2 2 2
395 173/300 Flush
396 174/300 Write block 1014 len 4096 data: 2
397 175/300 Flush
398 176/300 Write block 3977 len 32768 data: 2 2 2 2 2 2 2 2
399 177/300 Read block 2550 len 4096
400 178/300 Read block 2732 len 8192
401 179/300 Flush
402 180/300 Write block 1293 len 16384 data: 2 2 2 2
403 181/300 Flush
404 182/300 Read block 1205 len 28672
405 183/300 Read block 4076 len 36864
406 184/300 Write block 3682 len 12288 data: 2 2 2
407 185/300 Read block 2419 len 32768
408 186/300 Read block 3210 len 12288
409 187/300 Write block 4019 len 12288 data: 2 2 2
410 188/300 Write block 2193 len 4096 data: 2
411 189/300 Write block 4008 len 24576 data: 2 2 2 2 2 2
412 190/300 Write block 2912 len 28672 data: 2 2 2 2 2 2 2
413 191/300 Read block 2513 len 40960
414 192/300 Write block 142 len 4096 data: 2
415 193/300 Write block 1743 len 4096 data: 3
416 194/300 Read block 3278 len 8192
417 195/300 Write block 3264 len 8192 data: 2 2
418 196/300 Write block 2790 len 20480 data: 3 3 3 3 3
419 197/300 Flush
420 198/300 Read block 2243 len 32768
421 199/300 Flush
422 200/300 Read block 2162 len 36864
423 201/300 Write block 2467 len 36864 data: 2 2 2 2 2 2 2 2 2
424 202/300 Read block 1551 len 40960
425 203/300 Flush
426 204/300 Read block 1463 len 28672
427 205/300 Write block 1173 len 4096 data: 2
428 206/300 Read block 3885 len 32768
429 207/300 Write block 2747 len 20480 data: 2 2 2 2 2
430 208/300 Read block 4018 len 8192
431 209/300 Read block 4639 len 40960
432 210/300 Read block 1469 len 12288
433 211/300 Write block 2117 len 24576 data: 2 2 2 2 2 2
434 212/300 Flush
435 213/300 Read block 1624 len 40960
436 214/300 Read block 3953 len 20480
437 215/300 Write block 4477 len 24576 data: 2 2 2 2 2 2
438 216/300 Write block 2947 len 8192 data: 2 2
439 217/300 Read block 1488 len 40960
440 218/300 Write block 1313 len 40960 data: 2 2 2 2 2 2 2 2 2 2
441 219/300 Write block 1680 len 16384 data: 2 2 2 2
442 220/300 Flush
443 221/300 Read block 2291 len 8192
444 222/300 Write block 1241 len 36864 data: 2 2 2 2 2 2 3 3 3
445 223/300 Read block 434 len 32768
446 224/300 Write block 2018 len 8192 data: 2 2
447 225/300 Write block 662 len 32768 data: 3 3 3 3 3 2 2 2
448 226/300 Write block 3935 len 36864 data: 2 2 2 2 2 2 2 2 2
449 227/300 Read block 4963 len 28672
450 228/300 Read block 3942 len 40960
451 229/300 Read block 236 len 20480
452 230/300 Flush
453 231/300 Write block 1527 len 20480 data: 2 2 2 2 2
454 232/300 Read block 4316 len 8192
455 233/300 Read block 2571 len 28672
456 234/300 Write block 2542 len 32768 data: 2 2 2 2 2 2 2 2
457 235/300 Read block 953 len 36864
458 236/300 Read block 1624 len 40960
459 237/300 Read block 2658 len 24576
460 238/300 Write block 838 len 24576 data: 2 2 2 2 2 2
461 239/300 Flush
462 240/300 Read block 4536 len 20480
463 241/300 Read block 3145 len 20480
464 242/300 Read block 4028 len 40960
465 243/300 Read block 3178 len 20480
466 244/300 Read block 4448 len 4096
467 245/300 Flush
468 246/300 Write block 802 len 4096 data: 2
469 247/300 Write block 3020 len 24576 data: 2 2 2 2 2 2
470 248/300 Flush
471 249/300 Read block 4877 len 24576
472 250/300 Write block 4519 len 36864 data: 2 2 2 2 2 2 2 2 2
473 251/300 Read block 2029 len 16384
474 252/300 Write block 801 len 24576 data: 2 3 2 2 2 2
475 253/300 Write block 4187 len 8192 data: 2 2
476 254/300 Read block 1622 len 24576
477 255/300 Read block 4689 len 20480
478 256/300 Read block 457 len 16384
479 257/300 Read block 4031 len 40960
480 258/300 Read block 3527 len 12288
481 259/300 Read block 3062 len 4096
482 260/300 Read block 1847 len 4096
483 261/300 Read block 672 len 40960
484 262/300 Write block 579 len 12288 data: 3 3 3
485 263/300 Write block 30 len 36864 data: 2 2 2 2 2 2 2 2 2
486 264/300 Write block 1177 len 12288 data: 2 2 2
487 265/300 Read block 4968 len 20480
488 266/300 Write block 3159 len 16384 data: 2 2 2 2
489 267/300 Flush
490 268/300 Write block 4329 len 8192 data: 2 2
491 269/300 Read block 4324 len 20480
492 270/300 Read block 2055 len 28672
493 271/300 Read block 1867 len 20480
494 272/300 Write block 1334 len 28672 data: 2 2 2 2 2 2 2
495 273/300 Read block 4942 len 32768
496 274/300 Write block 2209 len 4096 data: 2
497 275/300 Write block 4800 len 24576 data: 2 2 2 2 2 2
498 276/300 Read block 2851 len 32768
499 277/300 Read block 2291 len 4096
500 278/300 Write block 3167 len 28672 data: 2 2 2 2 2 2 2
501 279/300 Read block 4813 len 16384
502 280/300 Read block 2402 len 20480
503 281/300 Read block 4784 len 12288
504 282/300 Write block 3027 len 12288 data: 2 2 2
505 283/300 Read block 4661 len 40960
506 284/300 Write block 2171 len 12288 data: 2 2 2
507 285/300 Read block 425 len 16384
508 286/300 Flush
509 287/300 Read block 814 len 8192
510 288/300 Read block 3956 len 36864
511 289/300 Write block 2358 len 4096 data: 2
512 290/300 Write block 3784 len 36864 data: 2 2 2 2 2 2 2 2 2
513 291/300 Write block 2058 len 40960 data: 2 2 2 2 2 2 2 2 2 2
514 292/300 Write block 3389 len 36864 data: 2 2 2 2 2 2 2 2 2
515 293/300 Flush
516 294/300 Read block 2133 len 20480
517 295/300 Write block 858 len 20480 data: 2 2 3 3 3
518 296/300 Write block 1668 len 20480 data: 2 2 2 2 2
519 297/300 Read block 2750 len 32768
520 298/300 Flush
521 299/300 Read block 2870 len 20480
522 300/300 Read block 2012 len 12288
523 301/300 Write block 4 len 8192 data: 2 2
524 Main task has work to do, go find it
525 got dsc Start(1)
526 start 1
527 Jan 13 15:51:55.286 INFO request completed, latency_us: 186, response_code: 204, uri: /start/cid/1, method: POST, req_id: 3853989e-1309-49c9-b609-9541dd8bd004, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
528 Replay: started 1, returned:Ok(())
529 ----------------------------------------------------------------
530 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
531 States: Active Offline Active
532 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
533 1000 Acked Write 5 Done Sent Done false
534 1001 Acked Read 5 Done Sent Done false
535 1002 Acked Write 9 Done Sent Done false
536 1003 Acked Flush 0 Done Sent Done false
537 1004 Acked Flush 0 Done Sent Done false
538 1005 Acked Read 8 Done Sent Done false
539 1006 Acked Write 6 Done Sent Done false
540 1007 Acked Read 6 Done Sent Done false
541 1008 Acked Flush 0 Done Sent Done false
542 1009 Acked Read 4 Done Sent Done false
543 1010 Acked Read 10 Done Sent Done false
544 1011 Acked Write 2 Done Sent Done false
545 1012 Acked Flush 0 Done Sent Done false
546 1013 Acked Write 7 Done Sent Done false
547 1014 Acked Read 8 Done Sent Done false
548 1015 Acked Read 4 Done Sent Done false
549 1016 Acked Flush 0 Done Sent Done false
550 1017 Acked Write 4 Done Sent Done false
551 1018 Acked Read 7 Done Sent Done false
552 1019 Acked Write 6 Done Sent Done false
553 1020 Acked Write 9 Done Sent Done false
554 1021 Acked Write 1 Done Sent Done false
555 1022 Acked Read 3 Done Sent Done false
556 1023 Acked Write 10 Done Sent Done false
557 1024 Acked Write 3 Done Sent Done false
558 1025 Acked Flush 0 Done Sent Done false
559 1026 Acked Read 4 Done Sent Done false
560 1027 Acked Flush 0 Done Sent Done false
561 1028 Acked Read 2 Done Sent Done false
562 1029 Acked Write 4 Done Sent Done false
563 1030 Acked Read 4 Done Sent Done false
564 1031 Acked Read 9 Done Sent Done false
565 1032 Acked Read 9 Done Sent Done false
566 1033 Acked Read 1 Done Sent Done false
567 1034 Acked Read 9 Done Sent Done false
568 1035 Acked Write 9 Done Sent Done false
569 1036 Acked Read 9 Done Sent Done false
570 1037 Acked Write 2 Done Sent Done false
571 1038 Acked Read 1 Done Sent Done false
572 1039 Acked Read 5 Done Sent Done false
573 1040 Acked Read 8 Done Sent Done false
574 1041 Acked Write 10 Done Sent Done false
575 1042 Acked Read 10 Done Sent Done false
576 1043 Acked Write 7 Done Sent Done false
577 1044 Acked Write 5 Done Sent Done false
578 1045 Acked Read 8 Done Sent Done false
579 1046 Acked Flush 0 Done Sent Done false
580 1047 Acked Flush 0 Done Sent Done false
581 1048 Acked Flush 0 Done Sent Done false
582 1049 Acked Write 8 Done Sent Done false
583 1050 Acked Read 9 Done Sent Done false
584 1051 Acked Write 7 Done Sent Done false
585 1052 Acked Flush 0 Done Sent Done false
586 1053 Acked Read 10 Done Sent Done false
587 1054 Acked Read 2 Done Sent Done false
588 1055 Acked Read 9 Done Sent Done false
589 1056 Acked Write 6 Done Sent Done false
590 1057 Acked Read 6 Done Sent Done false
591 1058 Acked Write 7 Done Sent Done false
592 1059 Acked Write 5 Done Sent Done false
593 1060 Acked Flush 0 Done Sent Done false
594 1061 Acked Write 3 Done Sent Done false
595 1062 Acked Flush 0 Done Sent Done false
596 1063 Acked Flush 0 Done Sent Done false
597 1064 Acked Read 4 Done Sent Done false
598 1065 Acked Write 10 Done Sent Done false
599 1066 Acked Write 10 Done Sent Done false
600 1067 Acked Write 9 Done Sent Done false
601 1068 Acked Read 3 Done Sent Done false
602 1069 Acked Read 9 Done Sent Done false
603 1070 Acked Read 5 Done Sent Done false
604 1071 Acked Write 7 Done Sent Done false
605 1072 Acked Write 1 Done Sent Done false
606 1073 Acked Read 3 Done Sent Done false
607 1074 Acked Read 4 Done Sent Done false
608 1075 Acked Write 6 Done Sent Done false
609 1076 Acked Read 5 Done Sent Done false
610 1077 Acked Write 3 Done Sent Done false
611 1078 Acked Write 9 Done Sent Done false
612 1079 Acked Read 1 Done Sent Done false
613 1080 Acked Read 3 Done Sent Done false
614 1081 Acked Write 9 Done Sent Done false
615 1082 Acked Read 8 Done Sent Done false
616 1083 Acked Write 8 Done Sent Done false
617 1084 Acked Read 4 Done Sent Done false
618 1085 Acked Write 6 Done Sent Done false
619 1086 Acked Read 1 Done Sent Done false
620 1087 Acked Read 9 Done Sent Done false
621 1088 Acked Write 10 Done Sent Done false
622 1089 Acked Read 7 Done Sent Done false
623 1090 Acked Read 3 Done Sent Done false
624 1091 Acked Write 7 Done Sent Done false
625 1092 Acked Write 7 Done Sent Done false
626 1093 Acked Read 4 Done Sent Done false
627 1094 Acked Write 4 Done Sent Done false
628 1095 Acked Read 8 Done Sent Done false
629 1096 Acked Write 5 Done Sent Done false
630 1097 Acked Read 1 Done Sent Done false
631 1098 Acked Read 3 Done Sent Done false
632 1099 Acked Read 2 Done Sent Done false
633 1100 Acked Read 3 Done Sent Done false
634 1101 Acked Read 6 Done Sent Done false
635 1102 Acked Read 3 Done Sent Done false
636 1103 Acked Read 5 Done Sent Done false
637 1104 Acked Flush 0 Done Sent Done false
638 1105 Acked Read 4 Done Sent Done false
639 1106 Acked Read 4 Done Sent Done false
640 1107 Acked Write 7 Done Sent Done false
641 1108 Acked Read 2 Done Sent Done false
642 1109 Acked Read 1 Done Sent Done false
643 1110 Acked Write 3 Done Sent Done false
644 1111 Acked Write 3 Done Sent Done false
645 1112 Acked Write 10 Done Sent Done false
646 1113 Acked Write 9 Done Sent Done false
647 1114 Acked Flush 0 Done Sent Done false
648 1115 Acked Write 3 Done Sent Done false
649 1116 Acked Read 4 Done Sent Done false
650 1117 Acked Read 6 Done Sent Done false
651 1118 Acked Write 10 Done Sent Done false
652 1119 Acked Read 7 Done Sent Done false
653 1120 Acked Read 2 Done Sent Done false
654 1121 Acked Read 6 Done Sent Done false
655 1122 Acked Read 5 Done Sent Done false
656 1123 Acked Write 8 Done Sent Done false
657 1124 Acked Write 10 Done Sent Done false
658 1125 Acked Read 7 Done Sent Done false
659 1126 Acked Write 10 Done Sent Done false
660 1127 Acked Write 6 Done Sent Done false
661 1128 Acked Flush 0 Done Sent Done false
662 1129 Acked Read 10 Done Sent Done false
663 1130 Acked Read 2 Done Sent Done false
664 1131 Acked Read 5 Done Sent Done false
665 1132 Acked Read 10 Done Sent Done false
666 1133 Acked Write 10 Done Sent Done false
667 1134 Acked Read 2 Done Sent Done false
668 1135 Acked Write 4 Done Sent Done false
669 1136 Acked Read 3 Done Sent Done false
670 1137 Acked Write 5 Done Sent Done false
671 1138 Acked Write 3 Done Sent Done false
672 1139 Acked Flush 0 Done Sent Done false
673 1140 Acked Read 10 Done Sent Done false
674 1141 Acked Write 7 Done Sent Done false
675 1142 Acked Read 3 Done Sent Done false
676 1143 Acked Write 9 Done Sent Done false
677 1144 Acked Read 10 Done Sent Done false
678 1145 Acked Read 9 Done Sent Done false
679 1146 Acked Read 10 Done Sent Done false
680 1147 Acked Write 8 Done Sent Done false
681 1148 Acked Read 5 Done Sent Done false
682 1149 Acked Read 7 Done Sent Done false
683 1150 Acked Read 10 Done Sent Done false
684 1151 Acked Read 6 Done Sent Done false
685 1152 Acked Write 9 Done Sent Done false
686 1153 Acked Write 8 Done Sent Done false
687 1154 Acked Read 3 Done Sent Done false
688 1155 Acked Read 3 Done Sent Done false
689 1156 Acked Write 5 Done Sent Done false
690 1157 Acked Flush 0 Done Sent Done false
691 1158 Acked Flush 0 Done Sent Done false
692 1159 Acked Read 3 Done Sent Done false
693 1160 Acked Flush 0 Done Sent Done false
694 1161 Acked Read 4 Done Sent Done false
695 1162 Acked Read 1 Done Sent Done false
696 1163 Acked Read 1 Done Sent Done false
697 1164 Acked Write 10 Done Sent Done false
698 1165 Acked Write 8 Done Sent Done false
699 1166 Acked Read 8 Done Sent Done false
700 1167 Acked Read 10 Done Sent Done false
701 1168 Acked Write 8 Done Sent Done false
702 1169 Acked Read 1 Done Sent Done false
703 1170 Acked Read 4 Done Sent Done false
704 1171 Acked Write 10 Done Sent Done false
705 1172 Acked Flush 0 Done Sent Done false
706 1173 Acked Write 1 Done Sent Done false
707 1174 Acked Flush 0 Done Sent Done false
708 1175 Acked Write 8 Done Sent Done false
709 1176 Acked Read 1 Done Sent Done false
710 1177 Acked Read 2 Done Sent Done false
711 1178 Acked Flush 0 Done Sent Done false
712 1179 Acked Write 4 Done Sent Done false
713 1180 Acked Flush 0 Done Sent Done false
714 1181 Acked Read 7 Done Sent Done false
715 1182 Acked Read 9 Done Sent Done false
716 1183 Acked Write 3 Done Sent Done false
717 1184 Acked Read 8 Done Sent Done false
718 1185 Acked Read 3 Done Sent Done false
719 1186 Acked Write 3 Done Sent Done false
720 1187 Acked Write 1 Done Sent Done false
721 1188 Acked Write 6 Done Sent Done false
722 1189 Acked Write 7 Done Sent Done false
723 1190 Acked Read 10 Done Sent Done false
724 1191 Acked Write 1 Done Sent Done false
725 1192 Acked Write 1 Done Sent Done false
726 1193 Acked Read 2 Done Sent Done false
727 1194 Acked Write 2 Done Sent Done false
728 1195 Acked Write 5 Done Sent Done false
729 1196 Acked Flush 0 Done Sent Done false
730 1197 Acked Read 8 Done Sent Done false
731 1198 Acked Flush 0 Done Sent Done false
732 1199 Acked Read 9 Done Sent Done false
733 1200 Acked Write 9 Done Sent Done false
734 1201 Acked Read 10 Done Sent Done false
735 1202 Acked Flush 0 Done Sent Done false
736 1203 Acked Read 7 Done Sent Done false
737 1204 Acked Write 1 Done Sent Done false
738 1205 Acked Read 8 Done Sent Done false
739 1206 Acked Write 5 Done Sent Done false
740 1207 Acked Read 2 Done Sent Done false
741 1208 Acked Read 10 Done Sent Done false
742 1209 Acked Read 3 Done Sent Done false
743 1210 Acked Write 6 Done Sent Done false
744 1211 Acked Flush 0 Done Sent Done false
745 1212 Acked Read 10 Done Sent Done false
746 1213 Acked Read 5 Done Sent Done false
747 1214 Acked Write 6 Done Sent Done false
748 1215 Acked Write 2 Done Sent Done false
749 1216 Acked Read 10 Done Sent Done false
750 1217 Acked Write 10 Done Sent Done false
751 1218 Acked Write 4 Done Sent Done false
752 1219 Acked Flush 0 Done Sent Done false
753 1220 Acked Read 2 Done Sent Done false
754 1221 Acked Write 9 Done Sent Done false
755 1222 Acked Read 8 Done Sent Done false
756 1223 Acked Write 2 Done Sent Done false
757 1224 Acked Write 8 Done Sent Done false
758 1225 Acked Write 9 Done Sent Done false
759 1226 Acked Read 7 Done Sent Done false
760 1227 Acked Read 10 Done Sent Done false
761 1228 Acked Read 5 Done Sent Done false
762 1229 Acked Flush 0 Done Sent Done false
763 1230 Acked Write 5 Done Sent Done false
764 1231 Acked Read 2 Done Sent Done false
765 1232 Acked Read 7 Done Sent Done false
766 1233 Acked Write 8 Done Sent Done false
767 1234 Acked Read 9 Done Sent Done false
768 1235 Acked Read 10 Done Sent Done false
769 1236 Acked Read 6 Done Sent Done false
770 1237 Acked Write 6 Done Sent Done false
771 1238 Acked Flush 0 Done Sent Done false
772 1239 Acked Read 5 Done Sent Done false
773 1240 Acked Read 5 Done Sent Done false
774 1241 Acked Read 10 Done Sent Done false
775 1242 Acked Read 5 Done Sent Done false
776 1243 Acked Read 1 Done Sent Done false
777 1244 Acked Flush 0 Done Sent Done false
778 1245 Acked Write 1 Done Sent Done false
779 1246 Acked Write 6 Done Sent Done false
780 1247 Acked Flush 0 Done Sent Done false
781 1248 Acked Read 6 Done Sent Done false
782 1249 Acked Write 9 Done Sent Done false
783 1250 Acked Read 4 Done Sent Done false
784 1251 Acked Write 6 Done Sent Done false
785 1252 Acked Write 2 Done Sent Done false
786 1253 Acked Read 6 Done Sent Done false
787 1254 Acked Read 5 Done Sent Done false
788 1255 Acked Read 4 Done Sent Done false
789 1256 Acked Read 10 Done Sent Done false
790 1257 Acked Read 3 Done Sent Done false
791 1258 Acked Read 1 Done Sent Done false
792 1259 Acked Read 1 Done Sent Done false
793 1260 Acked Read 10 Done Sent Done false
794 1261 Acked Write 3 Done Sent Done false
795 1262 Acked Write 9 Done Sent Done false
796 1263 Acked Write 3 Done Sent Done false
797 1264 Acked Read 5 Done Sent Done false
798 1265 Acked Write 4 Done Sent Done false
799 1266 Acked Flush 0 Done Sent Done false
800 1267 Acked Write 2 Done Sent Done false
801 1268 Acked Read 5 Done Sent Done false
802 1269 Acked Read 7 Done Sent Done false
803 1270 Acked Read 5 Done Sent Done false
804 1271 Acked Write 7 Done Sent Done false
805 1272 Acked Read 8 Done Sent Done false
806 1273 Acked Write 1 Done Sent Done false
807 1274 Acked Write 6 Done Sent Done false
808 1275 Acked Read 8 Done Sent Done false
809 1276 Acked Read 1 Done Sent Done false
810 1277 Acked Write 7 Done Sent Done false
811 1278 Acked Read 4 Done Sent Done false
812 1279 Acked Read 5 Done Sent Done false
813 1280 Acked Read 3 Done Sent Done false
814 1281 Acked Write 3 Done Sent Done false
815 1282 Acked Read 10 Done Sent Done false
816 1283 Acked Write 3 Done Sent Done false
817 1284 Acked Read 4 Done Sent Done false
818 1285 Acked Flush 0 Done Sent Done false
819 1286 Acked Read 2 Done Sent Done false
820 1287 Acked Read 9 Done Sent Done false
821 1288 Acked Write 1 Done Sent Done false
822 1289 Acked Write 9 Done Sent Done false
823 1290 Acked Write 10 Done Sent Done false
824 1291 Acked Write 9 Done Sent Done false
825 1292 Acked Flush 0 Done Sent Done false
826 1293 Acked Read 5 Done Sent Done false
827 1294 Acked Write 5 Done Sent Done false
828 1295 Acked Write 5 Done Sent Done false
829 1296 Acked Read 8 Done Sent Done false
830 1297 Acked Flush 0 Done Sent Done false
831 1298 Acked Read 5 Done Sent Done false
832 1299 Acked Read 3 Done Sent Done false
833 1300 Acked Write 2 Done Sent Done false
834 STATES DS:0 DS:1 DS:2 TOTAL
835 Sent 0 301 0 301
836 Done 301 0 301 602
837 Skipped 0 0 0 0
838 Error 0 0 0 0
839 Last Flush: 1297 0 1297
840 Downstairs last five completed:
841 Upstairs last five completed: 1300 1299 1298 1297 1296
842 CLIENT: Up:0 ds:301 act:2
843 [8820] got start action
844 [8820] I am going to restart
845 Starting downstairs at port 8820
846 Make output file at "/tmp/dsc/downstairs-8820.txt"
847 [8820][1] reports Starting
848 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1203)
849 [8820][1] reports Running
850 ----------------------------------------------------------------
851 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
852 States: Active Offline Active
853 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
854 1000 Acked Write 5 Done Sent Done false
855 1001 Acked Read 5 Done Sent Done false
856 1002 Acked Write 9 Done Sent Done false
857 1003 Acked Flush 0 Done Sent Done false
858 1004 Acked Flush 0 Done Sent Done false
859 1005 Acked Read 8 Done Sent Done false
860 1006 Acked Write 6 Done Sent Done false
861 1007 Acked Read 6 Done Sent Done false
862 1008 Acked Flush 0 Done Sent Done false
863 1009 Acked Read 4 Done Sent Done false
864 1010 Acked Read 10 Done Sent Done false
865 1011 Acked Write 2 Done Sent Done false
866 1012 Acked Flush 0 Done Sent Done false
867 1013 Acked Write 7 Done Sent Done false
868 1014 Acked Read 8 Done Sent Done false
869 1015 Acked Read 4 Done Sent Done false
870 1016 Acked Flush 0 Done Sent Done false
871 1017 Acked Write 4 Done Sent Done false
872 1018 Acked Read 7 Done Sent Done false
873 1019 Acked Write 6 Done Sent Done false
874 1020 Acked Write 9 Done Sent Done false
875 1021 Acked Write 1 Done Sent Done false
876 1022 Acked Read 3 Done Sent Done false
877 1023 Acked Write 10 Done Sent Done false
878 1024 Acked Write 3 Done Sent Done false
879 1025 Acked Flush 0 Done Sent Done false
880 1026 Acked Read 4 Done Sent Done false
881 1027 Acked Flush 0 Done Sent Done false
882 1028 Acked Read 2 Done Sent Done false
883 1029 Acked Write 4 Done Sent Done false
884 1030 Acked Read 4 Done Sent Done false
885 1031 Acked Read 9 Done Sent Done false
886 1032 Acked Read 9 Done Sent Done false
887 1033 Acked Read 1 Done Sent Done false
888 1034 Acked Read 9 Done Sent Done false
889 1035 Acked Write 9 Done Sent Done false
890 1036 Acked Read 9 Done Sent Done false
891 1037 Acked Write 2 Done Sent Done false
892 1038 Acked Read 1 Done Sent Done false
893 1039 Acked Read 5 Done Sent Done false
894 1040 Acked Read 8 Done Sent Done false
895 1041 Acked Write 10 Done Sent Done false
896 1042 Acked Read 10 Done Sent Done false
897 1043 Acked Write 7 Done Sent Done false
898 1044 Acked Write 5 Done Sent Done false
899 1045 Acked Read 8 Done Sent Done false
900 1046 Acked Flush 0 Done Sent Done false
901 1047 Acked Flush 0 Done Sent Done false
902 1048 Acked Flush 0 Done Sent Done false
903 1049 Acked Write 8 Done Sent Done false
904 1050 Acked Read 9 Done Sent Done false
905 1051 Acked Write 7 Done Sent Done false
906 1052 Acked Flush 0 Done Sent Done false
907 1053 Acked Read 10 Done Sent Done false
908 1054 Acked Read 2 Done Sent Done false
909 1055 Acked Read 9 Done Sent Done false
910 1056 Acked Write 6 Done Sent Done false
911 1057 Acked Read 6 Done Sent Done false
912 1058 Acked Write 7 Done Sent Done false
913 1059 Acked Write 5 Done Sent Done false
914 1060 Acked Flush 0 Done Sent Done false
915 1061 Acked Write 3 Done Sent Done false
916 1062 Acked Flush 0 Done Sent Done false
917 1063 Acked Flush 0 Done Sent Done false
918 1064 Acked Read 4 Done Sent Done false
919 1065 Acked Write 10 Done Sent Done false
920 1066 Acked Write 10 Done Sent Done false
921 1067 Acked Write 9 Done Sent Done false
922 1068 Acked Read 3 Done Sent Done false
923 1069 Acked Read 9 Done Sent Done false
924 1070 Acked Read 5 Done Sent Done false
925 1071 Acked Write 7 Done Sent Done false
926 1072 Acked Write 1 Done Sent Done false
927 1073 Acked Read 3 Done Sent Done false
928 1074 Acked Read 4 Done Sent Done false
929 1075 Acked Write 6 Done Sent Done false
930 1076 Acked Read 5 Done Sent Done false
931 1077 Acked Write 3 Done Sent Done false
932 1078 Acked Write 9 Done Sent Done false
933 1079 Acked Read 1 Done Sent Done false
934 1080 Acked Read 3 Done Sent Done false
935 1081 Acked Write 9 Done Sent Done false
936 1082 Acked Read 8 Done Sent Done false
937 1083 Acked Write 8 Done Sent Done false
938 1084 Acked Read 4 Done Sent Done false
939 1085 Acked Write 6 Done Sent Done false
940 1086 Acked Read 1 Done Sent Done false
941 1087 Acked Read 9 Done Sent Done false
942 1088 Acked Write 10 Done Sent Done false
943 1089 Acked Read 7 Done Sent Done false
944 1090 Acked Read 3 Done Sent Done false
945 1091 Acked Write 7 Done Sent Done false
946 1092 Acked Write 7 Done Sent Done false
947 1093 Acked Read 4 Done Sent Done false
948 1094 Acked Write 4 Done Sent Done false
949 1095 Acked Read 8 Done Sent Done false
950 1096 Acked Write 5 Done Sent Done false
951 1097 Acked Read 1 Done Sent Done false
952 1098 Acked Read 3 Done Sent Done false
953 1099 Acked Read 2 Done Sent Done false
954 1100 Acked Read 3 Done Sent Done false
955 1101 Acked Read 6 Done Sent Done false
956 1102 Acked Read 3 Done Sent Done false
957 1103 Acked Read 5 Done Sent Done false
958 1104 Acked Flush 0 Done Sent Done false
959 1105 Acked Read 4 Done Sent Done false
960 1106 Acked Read 4 Done Sent Done false
961 1107 Acked Write 7 Done Sent Done false
962 1108 Acked Read 2 Done Sent Done false
963 1109 Acked Read 1 Done Sent Done false
964 1110 Acked Write 3 Done Sent Done false
965 1111 Acked Write 3 Done Sent Done false
966 1112 Acked Write 10 Done Sent Done false
967 1113 Acked Write 9 Done Sent Done false
968 1114 Acked Flush 0 Done Sent Done false
969 1115 Acked Write 3 Done Sent Done false
970 1116 Acked Read 4 Done Sent Done false
971 1117 Acked Read 6 Done Sent Done false
972 1118 Acked Write 10 Done Sent Done false
973 1119 Acked Read 7 Done Sent Done false
974 1120 Acked Read 2 Done Sent Done false
975 1121 Acked Read 6 Done Sent Done false
976 1122 Acked Read 5 Done Sent Done false
977 1123 Acked Write 8 Done Sent Done false
978 1124 Acked Write 10 Done Sent Done false
979 1125 Acked Read 7 Done Sent Done false
980 1126 Acked Write 10 Done Sent Done false
981 1127 Acked Write 6 Done Sent Done false
982 1128 Acked Flush 0 Done Sent Done false
983 1129 Acked Read 10 Done Sent Done false
984 1130 Acked Read 2 Done Sent Done false
985 1131 Acked Read 5 Done Sent Done false
986 1132 Acked Read 10 Done Sent Done false
987 1133 Acked Write 10 Done Sent Done false
988 1134 Acked Read 2 Done Sent Done false
989 1135 Acked Write 4 Done Sent Done false
990 1136 Acked Read 3 Done Sent Done false
991 1137 Acked Write 5 Done Sent Done false
992 1138 Acked Write 3 Done Sent Done false
993 1139 Acked Flush 0 Done Sent Done false
994 1140 Acked Read 10 Done Sent Done false
995 1141 Acked Write 7 Done Sent Done false
996 1142 Acked Read 3 Done Sent Done false
997 1143 Acked Write 9 Done Sent Done false
998 1144 Acked Read 10 Done Sent Done false
999 1145 Acked Read 9 Done Sent Done false
1000 1146 Acked Read 10 Done Sent Done false
1001 1147 Acked Write 8 Done Sent Done false
1002 1148 Acked Read 5 Done Sent Done false
1003 1149 Acked Read 7 Done Sent Done false
1004 1150 Acked Read 10 Done Sent Done false
1005 1151 Acked Read 6 Done Sent Done false
1006 1152 Acked Write 9 Done Sent Done false
1007 1153 Acked Write 8 Done Sent Done false
1008 1154 Acked Read 3 Done Sent Done false
1009 1155 Acked Read 3 Done Sent Done false
1010 1156 Acked Write 5 Done Sent Done false
1011 1157 Acked Flush 0 Done Sent Done false
1012 1158 Acked Flush 0 Done Sent Done false
1013 1159 Acked Read 3 Done Sent Done false
1014 1160 Acked Flush 0 Done Sent Done false
1015 1161 Acked Read 4 Done Sent Done false
1016 1162 Acked Read 1 Done Sent Done false
1017 1163 Acked Read 1 Done Sent Done false
1018 1164 Acked Write 10 Done Sent Done false
1019 1165 Acked Write 8 Done Sent Done false
1020 1166 Acked Read 8 Done Sent Done false
1021 1167 Acked Read 10 Done Sent Done false
1022 1168 Acked Write 8 Done Sent Done false
1023 1169 Acked Read 1 Done Sent Done false
1024 1170 Acked Read 4 Done Sent Done false
1025 1171 Acked Write 10 Done Sent Done false
1026 1172 Acked Flush 0 Done Sent Done false
1027 1173 Acked Write 1 Done Sent Done false
1028 1174 Acked Flush 0 Done Sent Done false
1029 1175 Acked Write 8 Done Sent Done false
1030 1176 Acked Read 1 Done Sent Done false
1031 1177 Acked Read 2 Done Sent Done false
1032 1178 Acked Flush 0 Done Sent Done false
1033 1179 Acked Write 4 Done Sent Done false
1034 1180 Acked Flush 0 Done Sent Done false
1035 1181 Acked Read 7 Done Sent Done false
1036 1182 Acked Read 9 Done Sent Done false
1037 1183 Acked Write 3 Done Sent Done false
1038 1184 Acked Read 8 Done Sent Done false
1039 1185 Acked Read 3 Done Sent Done false
1040 1186 Acked Write 3 Done Sent Done false
1041 1187 Acked Write 1 Done Sent Done false
1042 1188 Acked Write 6 Done Sent Done false
1043 1189 Acked Write 7 Done Sent Done false
1044 1190 Acked Read 10 Done Sent Done false
1045 1191 Acked Write 1 Done Sent Done false
1046 1192 Acked Write 1 Done Sent Done false
1047 1193 Acked Read 2 Done Sent Done false
1048 1194 Acked Write 2 Done Sent Done false
1049 1195 Acked Write 5 Done Sent Done false
1050 1196 Acked Flush 0 Done Sent Done false
1051 1197 Acked Read 8 Done Sent Done false
1052 1198 Acked Flush 0 Done Sent Done false
1053 1199 Acked Read 9 Done Sent Done false
1054 1200 Acked Write 9 Done Sent Done false
1055 1201 Acked Read 10 Done Sent Done false
1056 1202 Acked Flush 0 Done Sent Done false
1057 1203 Acked Read 7 Done Sent Done false
1058 1204 Acked Write 1 Done Sent Done false
1059 1205 Acked Read 8 Done Sent Done false
1060 1206 Acked Write 5 Done Sent Done false
1061 1207 Acked Read 2 Done Sent Done false
1062 1208 Acked Read 10 Done Sent Done false
1063 1209 Acked Read 3 Done Sent Done false
1064 1210 Acked Write 6 Done Sent Done false
1065 1211 Acked Flush 0 Done Sent Done false
1066 1212 Acked Read 10 Done Sent Done false
1067 1213 Acked Read 5 Done Sent Done false
1068 1214 Acked Write 6 Done Sent Done false
1069 1215 Acked Write 2 Done Sent Done false
1070 1216 Acked Read 10 Done Sent Done false
1071 1217 Acked Write 10 Done Sent Done false
1072 1218 Acked Write 4 Done Sent Done false
1073 1219 Acked Flush 0 Done Sent Done false
1074 1220 Acked Read 2 Done Sent Done false
1075 1221 Acked Write 9 Done Sent Done false
1076 1222 Acked Read 8 Done Sent Done false
1077 1223 Acked Write 2 Done Sent Done false
1078 1224 Acked Write 8 Done Sent Done false
1079 1225 Acked Write 9 Done Sent Done false
1080 1226 Acked Read 7 Done Sent Done false
1081 1227 Acked Read 10 Done Sent Done false
1082 1228 Acked Read 5 Done Sent Done false
1083 1229 Acked Flush 0 Done Sent Done false
1084 1230 Acked Write 5 Done Sent Done false
1085 1231 Acked Read 2 Done Sent Done false
1086 1232 Acked Read 7 Done Sent Done false
1087 1233 Acked Write 8 Done Sent Done false
1088 1234 Acked Read 9 Done Sent Done false
1089 1235 Acked Read 10 Done Sent Done false
1090 1236 Acked Read 6 Done Sent Done false
1091 1237 Acked Write 6 Done Sent Done false
1092 1238 Acked Flush 0 Done Sent Done false
1093 1239 Acked Read 5 Done Sent Done false
1094 1240 Acked Read 5 Done Sent Done false
1095 1241 Acked Read 10 Done Sent Done false
1096 1242 Acked Read 5 Done Sent Done false
1097 1243 Acked Read 1 Done Sent Done false
1098 1244 Acked Flush 0 Done Sent Done false
1099 1245 Acked Write 1 Done Sent Done false
1100 1246 Acked Write 6 Done Sent Done false
1101 1247 Acked Flush 0 Done Sent Done false
1102 1248 Acked Read 6 Done Sent Done false
1103 1249 Acked Write 9 Done Sent Done false
1104 1250 Acked Read 4 Done Sent Done false
1105 1251 Acked Write 6 Done Sent Done false
1106 1252 Acked Write 2 Done Sent Done false
1107 1253 Acked Read 6 Done Sent Done false
1108 1254 Acked Read 5 Done Sent Done false
1109 1255 Acked Read 4 Done Sent Done false
1110 1256 Acked Read 10 Done Sent Done false
1111 1257 Acked Read 3 Done Sent Done false
1112 1258 Acked Read 1 Done Sent Done false
1113 1259 Acked Read 1 Done Sent Done false
1114 1260 Acked Read 10 Done Sent Done false
1115 1261 Acked Write 3 Done Sent Done false
1116 1262 Acked Write 9 Done Sent Done false
1117 1263 Acked Write 3 Done Sent Done false
1118 1264 Acked Read 5 Done Sent Done false
1119 1265 Acked Write 4 Done Sent Done false
1120 1266 Acked Flush 0 Done Sent Done false
1121 1267 Acked Write 2 Done Sent Done false
1122 1268 Acked Read 5 Done Sent Done false
1123 1269 Acked Read 7 Done Sent Done false
1124 1270 Acked Read 5 Done Sent Done false
1125 1271 Acked Write 7 Done Sent Done false
1126 1272 Acked Read 8 Done Sent Done false
1127 1273 Acked Write 1 Done Sent Done false
1128 1274 Acked Write 6 Done Sent Done false
1129 1275 Acked Read 8 Done Sent Done false
1130 1276 Acked Read 1 Done Sent Done false
1131 1277 Acked Write 7 Done Sent Done false
1132 1278 Acked Read 4 Done Sent Done false
1133 1279 Acked Read 5 Done Sent Done false
1134 1280 Acked Read 3 Done Sent Done false
1135 1281 Acked Write 3 Done Sent Done false
1136 1282 Acked Read 10 Done Sent Done false
1137 1283 Acked Write 3 Done Sent Done false
1138 1284 Acked Read 4 Done Sent Done false
1139 1285 Acked Flush 0 Done Sent Done false
1140 1286 Acked Read 2 Done Sent Done false
1141 1287 Acked Read 9 Done Sent Done false
1142 1288 Acked Write 1 Done Sent Done false
1143 1289 Acked Write 9 Done Sent Done false
1144 1290 Acked Write 10 Done Sent Done false
1145 1291 Acked Write 9 Done Sent Done false
1146 1292 Acked Flush 0 Done Sent Done false
1147 1293 Acked Read 5 Done Sent Done false
1148 1294 Acked Write 5 Done Sent Done false
1149 1295 Acked Write 5 Done Sent Done false
1150 1296 Acked Read 8 Done Sent Done false
1151 1297 Acked Flush 0 Done Sent Done false
1152 1298 Acked Read 5 Done Sent Done false
1153 1299 Acked Read 3 Done Sent Done false
1154 1300 Acked Write 2 Done Sent Done false
1155 1301 Acked Flush 0 Done Sent Done false
1156 STATES DS:0 DS:1 DS:2 TOTAL
1157 Sent 0 302 0 302
1158 Done 302 0 302 604
1159 Skipped 0 0 0 0
1160 Error 0 0 0 0
1161 Last Flush: 1301 0 1301
1162 Downstairs last five completed:
1163 Upstairs last five completed: 1301 1300 1299 1298 1297
1164 CLIENT: Up:0 ds:302 act:2
11652025-01-13T15:52:01.037ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
11662025-01-13T15:52:01.037ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:63638) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
11672025-01-13T15:52:01.039ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
11682025-01-13T15:52:01.039ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
11692025-01-13T15:52:01.039ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
11702025-01-13T15:52:01.039ZINFOcrucible: send last flush ID to this DS: 0 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
11712025-01-13T15:52:01.039ZINFOcrucible: Replied this last flush ID: 0 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
11722025-01-13T15:52:01.039ZINFOcrucible: [1] client re-new 302 jobs since flush 0 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
11732025-01-13T15:52:01.040ZINFOcrucible: [1] Replayed 302 jobs since flush: 0 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1174 ----------------------------------------------------------------
1175 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
1176 Downstairs last five completed: 1301 1300 1299 1298 1297
1177 Upstairs last five completed: 1301 1300 1299 1298 1297
1178 CLIENT: Up:0 ds:0 act:3
1179 Replay: All jobs finished
1180 Main task has work to do, go find it
1181 got dsc Stop(0)
1182 stop 0
1183 Jan 13 15:52:03.291 INFO request completed, latency_us: 581, response_code: 204[8810] Got stop action so:false kr:false
1184 , uri: /stop/cid/0, method: POST, req_id: a4b5d75b-50c7-456a-b99e-93efa87e9bee, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
11852025-01-13T15:52:03.292ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1186 Jan 13 15:52:03.292 INFO request completed, latency_us: 122, response_code: 200, uri: /state/cid/0, method: GET, req_id: aee159f6-6e7f-48d9-a7c7-fca124ff6739, remote_addr: 127.0.0.1:52422, local_addr: 127.0.{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:52:03.292281167Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
1187 0.1:9998
11882025-01-13T15:52:03.292ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
11892025-01-13T15:52:03.292ZINFOcrucible: client task is exiting = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
1190 [8810] Exited with: ExitStatus(unix_wait_status(9))
1191 [8810][0] reports Exit
1192 Jan 13 15:52:07.295 INFO request completed, latency_us: 428, response_code: 200, uri: /state/cid/0, method: GET, req_id: feea5f29-11d7-4c01-9997-48b1e14af1d3, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
1193 001/300 Flush
1194 002/300 Write block 615 len 20480 data: 2 2 2 2 2
1195 003/300 Flush
1196 004/300 Write block 456 len 20480 data: 2 2 2 2 2
1197 005/300 Write block 1837 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1198 006/300 Write block 642 len 20480 data: 2 2 2 2 2
1199 007/300 Read block 3329 len 8192
1200 008/300 Write block 3480 len 8192 data: 2 2
1201 009/300 Read block 676 len 40960
1202 010/300 Read block 1971 len 32768
1203 011/300 Write block 2820 len 32768 data: 2 2 2 2 2 2 2 2
1204 012/300 Read block 4154 len 24576
1205 013/300 Read block 861 len 4096
1206 014/300 Write block 4619 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1207 015/300 Write block 2496 len 12288 data: 2 3 3
1208 016/300 Flush
1209 017/300 Write block 4363 len 4096 data: 2
1210 018/300 Read block 4039 len 24576
1211 019/300 Flush
1212 020/300 Read block 3499 len 24576
1213 021/300 Flush
1214 022/300 Write block 3283 len 32768 data: 3 2 2 2 2 2 2 2
1215 023/300 Write block 841 len 32768 data: 3 3 3 2 2 2 2 2
1216 024/300 Read block 999 len 16384
1217 025/300 Write block 1048 len 40960 data: 2 2 2 2 3 3 3 3 3 3
1218 026/300 Write block 4895 len 4096 data: 3
1219 027/300 Read block 4285 len 28672
1220 028/300 Write block 4594 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1221 029/300 Read block 4539 len 24576
1222 030/300 Write block 920 len 36864 data: 2 2 2 2 2 2 2 2 2
1223 031/300 Write block 1872 len 24576 data: 2 2 2 2 2 2
1224 032/300 Read block 1951 len 8192
1225 033/300 Read block 2318 len 36864
1226 034/300 Write block 3589 len 16384 data: 2 2 2 2
1227 035/300 Read block 1721 len 32768
1228 036/300 Write block 4170 len 24576 data: 2 2 2 2 2 2
1229 037/300 Read block 4841 len 32768
1230 038/300 Write block 2013 len 8192 data: 2 2
1231 039/300 Read block 382 len 32768
1232 040/300 Read block 4199 len 24576
1233 041/300 Write block 1354 len 12288 data: 2 2 2
1234 042/300 Read block 2025 len 40960
1235 043/300 Write block 181 len 32768 data: 2 2 2 2 2 2 2 2
1236 044/300 Read block 1034 len 20480
1237 045/300 Read block 954 len 36864
1238 046/300 Read block 3165 len 36864
1239 047/300 Read block 2735 len 8192
1240 048/300 Write block 2847 len 28672 data: 2 2 2 2 2 2 2
1241 049/300 Write block 2843 len 24576 data: 2 2 2 2 3 3
1242 050/300 Flush
1243 051/300 Read block 3699 len 28672
1244 052/300 Write block 4693 len 28672 data: 2 2 2 2 2 2 2
1245 053/300 Read block 2753 len 32768
1246 054/300 Flush
1247 055/300 Flush
1248 056/300 Write block 4332 len 4096 data: 2
1249 057/300 Read block 3904 len 20480
1250 058/300 Read block 2389 len 40960
1251 059/300 Write block 100 len 28672 data: 2 2 2 2 2 2 2
1252 060/300 Flush
1253 061/300 Read block 2057 len 40960
1254 062/300 Read block 3675 len 40960
1255 063/300 Read block 4002 len 8192
1256 064/300 Read block 1091 len 8192
1257 065/300 Read block 4462 len 20480
1258 066/300 Read block 2022 len 32768
1259 067/300 Read block 4036 len 32768
1260 068/300 Write block 3875 len 32768 data: 2 2 2 2 2 2 2 2
1261 069/300 Write block 4330 len 4096 data: 3
1262 070/300 Read block 4567 len 4096
1263 071/300 Write block 3791 len 28672 data: 3 3 2 2 2 2 2
1264 072/300 Read block 4365 len 16384
1265 073/300 Write block 1281 len 4096 data: 2
1266 074/300 Read block 2185 len 12288
1267 075/300 Read block 2377 len 8192
1268 076/300 Read block 2511 len 12288
1269 077/300 Read block 1830 len 12288
1270 078/300 Write block 4510 len 40960 data: 2 2 2 2 2 2 2 2 2 3
1271 079/300 Read block 3981 len 16384
1272 080/300 Read block 1286 len 24576
1273 081/300 Write block 1881 len 12288 data: 2 2 2
1274 082/300 Write block 197 len 36864 data: 2 2 2 2 2 2 2 2 2
1275 083/300 Read block 563 len 32768
1276 084/300 Write block 1056 len 32768 data: 4 4 3 3 3 3 2 2
1277 085/300 Write block 2477 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1278 086/300 Read block 167 len 36864
1279 087/300 Read block 4754 len 12288
1280 088/300 Write block 3355 len 28672 data: 2 2 2 2 2 2 2
1281 089/300 Write block 1142 len 28672 data: 2 3 3 3 3 3 3
1282 090/300 Write block 4807 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1283 091/300 Write block 3454 len 16384 data: 2 2 2 2
1284 092/300 Read block 1664 len 20480
1285 093/300 Read block 2544 len 12288
1286 094/300 Read block 3196 len 8192
1287 095/300 Write block 500 len 16384 data: 2 2 2 2
1288 096/300 Write block 202 len 24576 data: 3 3 3 3 2 2
1289 097/300 Write block 1292 len 20480 data: 2 3 3 3 3
1290 098/300 Flush
1291 099/300 Read block 4227 len 40960
1292 100/300 Write block 4112 len 12288 data: 2 2 2
1293 101/300 Read block 1681 len 40960
1294 102/300 Write block 3568 len 12288 data: 2 2 2
1295 103/300 Write block 2383 len 32768 data: 2 2 2 2 2 2 2 2
1296 104/300 Write block 1764 len 24576 data: 3 3 3 2 2 2
1297 105/300 Write block 4345 len 4096 data: 2
1298 106/300 Read block 1886 len 20480
1299 107/300 Read block 1134 len 28672
1300 108/300 Write block 1817 len 8192 data: 2 2
1301 109/300 Flush
1302 110/300 Read block 1231 len 40960
1303 111/300 Read block 2299 len 16384
1304 112/300 Write block 3033 len 36864 data: 2 2 2 2 2 2 2 2 2
1305 113/300 Read block 3735 len 20480
1306 114/300 Write block 1353 len 32768 data: 2 3 3 3 2 2 2 2
1307 115/300 Read block 3489 len 4096
1308 116/300 Read block 1873 len 36864
1309 117/300 Write block 3620 len 8192 data: 2 2
1310 118/300 Flush
1311 119/300 Write block 2949 len 28672 data: 2 2 2 2 2 2 2
1312 120/300 Write block 343 len 24576 data: 2 2 2 2 2 2
1313 121/300 Read block 442 len 28672
1314 122/300 Read block 3236 len 12288
1315 123/300 Read block 1027 len 28672
1316 124/300 Write block 3221 len 8192 data: 2 2
1317 125/300 Read block 1661 len 36864
1318 126/300 Write block 3400 len 8192 data: 2 2
1319 127/300 Read block 3513 len 4096
1320 128/300 Read block 1047 len 4096
1321 129/300 Write block 1695 len 8192 data: 2 2
1322 130/300 Write block 4941 len 16384 data: 2 2 2 2
1323 131/300 Read block 785 len 8192
1324 132/300 Read block 4641 len 32768
1325 133/300 Read block 2758 len 20480
1326 134/300 Read block 1892 len 4096
1327 135/300 Read block 1822 len 36864
1328 136/300 Write block 4223 len 16384 data: 2 2 3 3
1329 137/300 Flush
1330 138/300 Write block 633 len 4096 data: 2
1331 139/300 Read block 3435 len 8192
1332 140/300 Write block 4849 len 4096 data: 2
1333 141/300 Read block 341 len 16384
1334 142/300 Read block 604 len 28672
1335 143/300 Write block 4591 len 40960 data: 2 2 2 3 3 3 3 3 3 3
1336 144/300 Read block 994 len 28672
1337 145/300 Read block 63 len 4096
1338 146/300 Read block 4979 len 32768
1339 147/300 Read block 1425 len 12288
1340 148/300 Read block 32 len 40960
1341 149/300 Flush
1342 150/300 Write block 386 len 32768 data: 2 2 2 2 2 2 2 2
1343 151/300 Read block 908 len 36864
1344 152/300 Write block 511 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1345 153/300 Write block 728 len 24576 data: 2 2 2 2 2 2
1346 154/300 Read block 925 len 40960
1347 155/300 Write block 1148 len 16384 data: 4 3 3 3
1348 156/300 Read block 2762 len 40960
1349 157/300 Write block 3635 len 4096 data: 2
1350 158/300 Flush
1351 159/300 Write block 3482 len 24576 data: 2 2 2 2 3 3
1352 160/300 Read block 4821 len 40960
1353 161/300 Read block 647 len 40960
1354 162/300 Write block 2223 len 28672 data: 3 3 3 3 3 2 2
1355 163/300 Read block 4020 len 24576
1356 164/300 Flush
1357 165/300 Read block 4552 len 20480
1358 166/300 Read block 4531 len 36864
1359 167/300 Flush
1360 168/300 Write block 2341 len 20480 data: 2 2 2 2 2
1361 169/300 Read block 2848 len 16384
1362 170/300 Read block 3417 len 28672
1363 171/300 Write block 4886 len 36864 data: 2 2 3 3 3 3 3 3 3
1364 172/300 Write block 3609 len 16384 data: 2 2 2 2
1365 173/300 Read block 3487 len 4096
1366 174/300 Read block 2154 len 4096
1367 175/300 Read block 4520 len 28672
1368 176/300 Flush
1369 177/300 Write block 1088 len 12288 data: 2 2 2
1370 178/300 Read block 3827 len 36864
1371 179/300 Write block 933 len 12288 data: 2 2 2
1372 180/300 Write block 24 len 24576 data: 2 2 2 2 2 2
1373 181/300 Write block 4161 len 4096 data: 3
1374 182/300 Flush
1375 183/300 Write block 4282 len 24576 data: 2 2 2 2 2 2
1376 184/300 Read block 1600 len 24576
1377 185/300 Flush
1378 186/300 Read block 1076 len 40960
1379 187/300 Read block 2254 len 16384
1380 188/300 Write block 4461 len 32768 data: 2 2 2 2 2 2 2 2
1381 189/300 Read block 1112 len 24576
1382 190/300 Read block 3939 len 8192
1383 191/300 Read block 1062 len 32768
1384 192/300 Write block 1645 len 28672 data: 2 2 2 2 2 2 2
1385 193/300 Write block 4268 len 36864 data: 2 2 2 2 2 2 2 2 2
1386 194/300 Flush
1387 195/300 Write block 2246 len 8192 data: 2 2
1388 196/300 Read block 3142 len 24576
1389 197/300 Read block 4178 len 24576
1390 198/300 Read block 2273 len 20480
1391 199/300 Write block 2747 len 24576 data: 3 3 3 3 3 2
1392 200/300 Write block 4734 len 28672 data: 2 2 2 2 2 2 2
1393 201/300 Read block 1163 len 24576
1394 202/300 Write block 3401 len 16384 data: 3 2 2 2
1395 203/300 Write block 2028 len 32768 data: 3 3 3 3 2 2 2 2
1396 204/300 Flush
1397 205/300 Read block 393 len 40960
1398 206/300 Read block 3285 len 28672
1399 207/300 Read block 3013 len 20480
1400 208/300 Read block 527 len 24576
1401 209/300 Write block 3893 len 28672 data: 2 2 2 2 2 2 2
1402 210/300 Write block 1779 len 8192 data: 2 2
1403 211/300 Read block 1179 len 40960
1404 212/300 Read block 494 len 8192
1405 213/300 Read block 4793 len 40960
1406 214/300 Read block 2625 len 28672
1407 215/300 Write block 78 len 20480 data: 2 2 2 2 2
1408 216/300 Read block 1210 len 12288
1409 217/300 Flush
1410 218/300 Read block 3336 len 28672
1411 219/300 Write block 4077 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1412 220/300 Read block 3766 len 8192
1413 221/300 Flush
1414 222/300 Read block 4517 len 28672
1415 223/300 Read block 2684 len 8192
1416 224/300 Write block 1727 len 28672 data: 2 2 2 2 2 2 2
1417 225/300 Read block 197 len 8192
1418 226/300 Read block 2287 len 36864
1419 227/300 Read block 4432 len 4096
1420 228/300 Read block 1880 len 28672
1421 229/300 Write block 123 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1422 230/300 Flush
1423 231/300 Read block 4561 len 8192
1424 232/300 Write block 3092 len 4096 data: 2
1425 233/300 Read block 2069 len 24576
1426 234/300 Read block 1406 len 28672
1427 235/300 Write block 2938 len 12288 data: 2 2 2
1428 236/300 Read block 1885 len 40960
1429 237/300 Write block 4577 len 12288 data: 2 2 3
1430 238/300 Read block 4821 len 36864
1431 239/300 Write block 3024 len 4096 data: 3
1432 240/300 Write block 2096 len 8192 data: 3 3
1433 241/300 Read block 4713 len 12288
1434 242/300 Write block 3614 len 20480 data: 2 2 2 2 2
1435 243/300 Read block 1173 len 24576
1436 244/300 Write block 1705 len 12288 data: 2 2 2
1437 245/300 Write block 1440 len 20480 data: 2 2 2 2 2
1438 246/300 Read block 768 len 28672
1439 247/300 Read block 3208 len 28672
1440 248/300 Read block 1952 len 20480
1441 249/300 Read block 3198 len 40960
1442 250/300 Flush
1443 251/300 Read block 506 len 4096
1444 252/300 Write block 3710 len 8192 data: 2 2
1445 253/300 Read block 2811 len 20480
1446 254/300 Read block 3806 len 24576
1447 255/300 Write block 2975 len 16384 data: 2 2 2 2
1448 256/300 Write block 723 len 4096 data: 2
1449 257/300 Write block 4370 len 12288 data: 2 2 2
1450 258/300 Write block 2516 len 28672 data: 2 2 2 2 2 2 2
1451 259/300 Write block 202 len 24576 data: 4 4 4 4 3 3
1452 260/300 Read block 3371 len 28672
1453 261/300 Read block 3202 len 32768
1454 262/300 Write block 1819 len 12288 data: 2 2 2
1455 263/300 Write block 1171 len 12288 data: 2 2 3
1456 264/300 Write block 664 len 32768 data: 4 4 4 3 3 3 2 2
1457 265/300 Flush
1458 266/300 Read block 4419 len 16384
1459 267/300 Write block 2103 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1460 268/300 Write block 1486 len 20480 data: 2 2 2 2 2
1461 269/300 Write block 2907 len 12288 data: 2 2 2
1462 270/300 Read block 3701 len 8192
1463 271/300 Read block 4334 len 16384
1464 272/300 Read block 2089 len 20480
1465 273/300 Write block 4531 len 8192 data: 2 2
1466 274/300 Read block 846 len 16384
1467 275/300 Flush
1468 276/300 Flush
1469 277/300 Read block 1411 len 12288
1470 278/300 Read block 3647 len 12288
1471 279/300 Write block 3916 len 4096 data: 2
1472 280/300 Read block 2471 len 36864
1473 281/300 Read block 537 len 28672
1474 282/300 Flush
1475 283/300 Flush
1476 284/300 Read block 349 len 4096
1477 285/300 Read block 70 len 12288
1478 286/300 Flush
1479 287/300 Flush
1480 288/300 Write block 1418 len 28672 data: 2 2 2 2 2 2 2
1481 289/300 Read block 3890 len 20480
1482 290/300 Write block 2592 len 32768 data: 2 2 2 2 2 2 2 2
1483 291/300 Read block 1205 len 20480
1484 292/300 Read block 3844 len 8192
1485 293/300 Read block 4839 len 12288
1486 294/300 Read block 1762 len 24576
1487 295/300 Flush
1488 296/300 Read block 4586 len 24576
1489 297/300 Write block 3065 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1490 298/300 Read block 1573 len 8192
1491 299/300 Write block 160 len 12288 data: 2 2 2
1492 300/300 Flush
1493 301/300 Read block 300 len 4096
1494 Main task has work to do, go find it
1495 got dsc Start(0)
1496 start 0
1497 Jan 13 15:52:07.534 INFO request completed, latency_us: 173, response_code: 204, uri: /start/cid/0, method: POST, req_id: fbbeff44-650b-4498-bc05-1da2fb6aac80, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
1498 Replay: started 0, returned:Ok(())
1499 ----------------------------------------------------------------
1500 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
1501 States: Offline Active Active
1502 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1503 1302 Acked Flush 0 Sent Done Done false
1504 1303 Acked Write 5 Sent Done Done false
1505 1304 Acked Flush 0 Sent Done Done false
1506 1305 Acked Write 5 Sent Done Done false
1507 1306 Acked Write 10 Sent Done Done false
1508 1307 Acked Write 5 Sent Done Done false
1509 1308 Acked Read 2 Sent Done Done false
1510 1309 Acked Write 2 Sent Done Done false
1511 1310 Acked Read 10 Sent Done Done false
1512 1311 Acked Read 8 Sent Done Done false
1513 1312 Acked Write 8 Sent Done Done false
1514 1313 Acked Read 6 Sent Done Done false
1515 1314 Acked Read 1 Sent Done Done false
1516 1315 Acked Write 10 Sent Done Done false
1517 1316 Acked Write 3 Sent Done Done false
1518 1317 Acked Flush 0 Sent Done Done false
1519 1318 Acked Write 1 Sent Done Done false
1520 1319 Acked Read 6 Sent Done Done false
1521 1320 Acked Flush 0 Sent Done Done false
1522 1321 Acked Read 6 Sent Done Done false
1523 1322 Acked Flush 0 Sent Done Done false
1524 1323 Acked Write 8 Sent Done Done false
1525 1324 Acked Write 8 Sent Done Done false
1526 1325 Acked Read 4 Sent Done Done false
1527 1326 Acked Write 10 Sent Done Done false
1528 1327 Acked Write 1 Sent Done Done false
1529 1328 Acked Read 7 Sent Done Done false
1530 1329 Acked Write 10 Sent Done Done false
1531 1330 Acked Read 6 Sent Done Done false
1532 1331 Acked Write 9 Sent Done Done false
1533 1332 Acked Write 6 Sent Done Done false
1534 1333 Acked Read 2 Sent Done Done false
1535 1334 Acked Read 9 Sent Done Done false
1536 1335 Acked Write 4 Sent Done Done false
1537 1336 Acked Read 8 Sent Done Done false
1538 1337 Acked Write 6 Sent Done Done false
1539 1338 Acked Read 8 Sent Done Done false
1540 1339 Acked Write 2 Sent Done Done false
1541 1340 Acked Read 8 Sent Done Done false
1542 1341 Acked Read 6 Sent Done Done false
1543 1342 Acked Write 3 Sent Done Done false
1544 1343 Acked Read 10 Sent Done Done false
1545 1344 Acked Write 8 Sent Done Done false
1546 1345 Acked Read 5 Sent Done Done false
1547 1346 Acked Read 9 Sent Done Done false
1548 1347 Acked Read 9 Sent Done Done false
1549 1348 Acked Read 2 Sent Done Done false
1550 1349 Acked Write 7 Sent Done Done false
1551 1350 Acked Write 6 Sent Done Done false
1552 1351 Acked Flush 0 Sent Done Done false
1553 1352 Acked Read 7 Sent Done Done false
1554 1353 Acked Write 7 Sent Done Done false
1555 1354 Acked Read 8 Sent Done Done false
1556 1355 Acked Flush 0 Sent Done Done false
1557 1356 Acked Flush 0 Sent Done Done false
1558 1357 Acked Write 1 Sent Done Done false
1559 1358 Acked Read 5 Sent Done Done false
1560 1359 Acked Read 10 Sent Done Done false
1561 1360 Acked Write 7 Sent Done Done false
1562 1361 Acked Flush 0 Sent Done Done false
1563 1362 Acked Read 10 Sent Done Done false
1564 1363 Acked Read 10 Sent Done Done false
1565 1364 Acked Read 2 Sent Done Done false
1566 1365 Acked Read 2 Sent Done Done false
1567 1366 Acked Read 5 Sent Done Done false
1568 1367 Acked Read 8 Sent Done Done false
1569 1368 Acked Read 8 Sent Done Done false
1570 1369 Acked Write 8 Sent Done Done false
1571 1370 Acked Write 1 Sent Done Done false
1572 1371 Acked Read 1 Sent Done Done false
1573 1372 Acked Write 7 Sent Done Done false
1574 1373 Acked Read 4 Sent Done Done false
1575 1374 Acked Write 1 Sent Done Done false
1576 1375 Acked Read 3 Sent Done Done false
1577 1376 Acked Read 2 Sent Done Done false
1578 1377 Acked Read 3 Sent Done Done false
1579 1378 Acked Read 3 Sent Done Done false
1580 1379 Acked Write 10 Sent Done Done false
1581 1380 Acked Read 4 Sent Done Done false
1582 1381 Acked Read 6 Sent Done Done false
1583 1382 Acked Write 3 Sent Done Done false
1584 1383 Acked Write 9 Sent Done Done false
1585 1384 Acked Read 8 Sent Done Done false
1586 1385 Acked Write 8 Sent Done Done false
1587 1386 Acked Write 10 Sent Done Done false
1588 1387 Acked Read 9 Sent Done Done false
1589 1388 Acked Read 3 Sent Done Done false
1590 1389 Acked Write 7 Sent Done Done false
1591 1390 Acked Write 7 Sent Done Done false
1592 1391 Acked Write 10 Sent Done Done false
1593 1392 Acked Write 4 Sent Done Done false
1594 1393 Acked Read 5 Sent Done Done false
1595 1394 Acked Read 3 Sent Done Done false
1596 1395 Acked Read 2 Sent Done Done false
1597 1396 Acked Write 4 Sent Done Done false
1598 1397 Acked Write 6 Sent Done Done false
1599 1398 Acked Write 5 Sent Done Done false
1600 1399 Acked Flush 0 Sent Done Done false
1601 1400 Acked Read 10 Sent Done Done false
1602 1401 Acked Write 3 Sent Done Done false
1603 1402 Acked Read 10 Sent Done Done false
1604 1403 Acked Write 3 Sent Done Done false
1605 1404 Acked Write 8 Sent Done Done false
1606 1405 Acked Write 6 Sent Done Done false
1607 1406 Acked Write 1 Sent Done Done false
1608 1407 Acked Read 5 Sent Done Done false
1609 1408 Acked Read 7 Sent Done Done false
1610 1409 Acked Write 2 Sent Done Done false
1611 1410 Acked Flush 0 Sent Done Done false
1612 1411 Acked Read 10 Sent Done Done false
1613 1412 Acked Read 4 Sent Done Done false
1614 1413 Acked Write 9 Sent Done Done false
1615 1414 Acked Read 5 Sent Done Done false
1616 1415 Acked Write 8 Sent Done Done false
1617 1416 Acked Read 1 Sent Done Done false
1618 1417 Acked Read 9 Sent Done Done false
1619 1418 Acked Write 2 Sent Done Done false
1620 1419 Acked Flush 0 Sent Done Done false
1621 1420 Acked Write 7 Sent Done Done false
1622 1421 Acked Write 6 Sent Done Done false
1623 1422 Acked Read 7 Sent Done Done false
1624 1423 Acked Read 3 Sent Done Done false
1625 1424 Acked Read 7 Sent Done Done false
1626 1425 Acked Write 2 Sent Done Done false
1627 1426 Acked Read 9 Sent Done Done false
1628 1427 Acked Write 2 Sent Done Done false
1629 1428 Acked Read 1 Sent Done Done false
1630 1429 Acked Read 1 Sent Done Done false
1631 1430 Acked Write 2 Sent Done Done false
1632 1431 Acked Write 4 Sent Done Done false
1633 1432 Acked Read 2 Sent Done Done false
1634 1433 Acked Read 8 Sent Done Done false
1635 1434 Acked Read 5 Sent Done Done false
1636 1435 Acked Read 1 Sent Done Done false
1637 1436 Acked Read 9 Sent Done Done false
1638 1437 Acked Write 4 Sent Done Done false
1639 1438 Acked Flush 0 Sent Done Done false
1640 1439 Acked Write 1 Sent Done Done false
1641 1440 Acked Read 2 Sent Done Done false
1642 1441 Acked Write 1 Sent Done Done false
1643 1442 Acked Read 4 Sent Done Done false
1644 1443 Acked Read 7 Sent Done Done false
1645 1444 Acked Write 10 Sent Done Done false
1646 1445 Acked Read 7 Sent Done Done false
1647 1446 Acked Read 1 Sent Done Done false
1648 1447 Acked Read 8 Sent Done Done false
1649 1448 Acked Read 3 Sent Done Done false
1650 1449 Acked Read 10 Sent Done Done false
1651 1450 Acked Flush 0 Sent Done Done false
1652 1451 Acked Write 8 Sent Done Done false
1653 1452 Acked Read 9 Sent Done Done false
1654 1453 Acked Write 10 Sent Done Done false
1655 1454 Acked Write 6 Sent Done Done false
1656 1455 Acked Read 10 Sent Done Done false
1657 1456 Acked Write 4 Sent Done Done false
1658 1457 Acked Read 10 Sent Done Done false
1659 1458 Acked Write 1 Sent Done Done false
1660 1459 Acked Flush 0 Sent Done Done false
1661 1460 Acked Write 6 Sent Done Done false
1662 1461 Acked Read 10 Sent Done Done false
1663 1462 Acked Read 10 Sent Done Done false
1664 1463 Acked Write 7 Sent Done Done false
1665 1464 Acked Read 6 Sent Done Done false
1666 1465 Acked Flush 0 Sent Done Done false
1667 1466 Acked Read 5 Sent Done Done false
1668 1467 Acked Read 9 Sent Done Done false
1669 1468 Acked Flush 0 Sent Done Done false
1670 1469 Acked Write 5 Sent Done Done false
1671 1470 Acked Read 4 Sent Done Done false
1672 1471 Acked Read 7 Sent Done Done false
1673 1472 Acked Write 9 Sent Done Done false
1674 1473 Acked Write 4 Sent Done Done false
1675 1474 Acked Read 1 Sent Done Done false
1676 1475 Acked Read 1 Sent Done Done false
1677 1476 Acked Read 7 Sent Done Done false
1678 1477 Acked Flush 0 Sent Done Done false
1679 1478 Acked Write 3 Sent Done Done false
1680 1479 Acked Read 9 Sent Done Done false
1681 1480 Acked Write 3 Sent Done Done false
1682 1481 Acked Write 6 Sent Done Done false
1683 1482 Acked Write 1 Sent Done Done false
1684 1483 Acked Flush 0 Sent Done Done false
1685 1484 Acked Write 6 Sent Done Done false
1686 1485 Acked Read 6 Sent Done Done false
1687 1486 Acked Flush 0 Sent Done Done false
1688 1487 Acked Read 10 Sent Done Done false
1689 1488 Acked Read 4 Sent Done Done false
1690 1489 Acked Write 8 Sent Done Done false
1691 1490 Acked Read 6 Sent Done Done false
1692 1491 Acked Read 2 Sent Done Done false
1693 1492 Acked Read 8 Sent Done Done false
1694 1493 Acked Write 7 Sent Done Done false
1695 1494 Acked Write 9 Sent Done Done false
1696 1495 Acked Flush 0 Sent Done Done false
1697 1496 Acked Write 2 Sent Done Done false
1698 1497 Acked Read 6 Sent Done Done false
1699 1498 Acked Read 6 Sent Done Done false
1700 1499 Acked Read 5 Sent Done Done false
1701 1500 Acked Write 6 Sent Done Done false
1702 1501 Acked Write 7 Sent Done Done false
1703 1502 Acked Read 6 Sent Done Done false
1704 1503 Acked Write 4 Sent Done Done false
1705 1504 Acked Write 8 Sent Done Done false
1706 1505 Acked Flush 0 Sent Done Done false
1707 1506 Acked Read 10 Sent Done Done false
1708 1507 Acked Read 7 Sent Done Done false
1709 1508 Acked Read 5 Sent Done Done false
1710 1509 Acked Read 6 Sent Done Done false
1711 1510 Acked Write 7 Sent Done Done false
1712 1511 Acked Write 2 Sent Done Done false
1713 1512 Acked Read 10 Sent Done Done false
1714 1513 Acked Read 2 Sent Done Done false
1715 1514 Acked Read 10 Sent Done Done false
1716 1515 Acked Read 7 Sent Done Done false
1717 1516 Acked Write 5 Sent Done Done false
1718 1517 Acked Read 3 Sent Done Done false
1719 1518 Acked Flush 0 Sent Done Done false
1720 1519 Acked Read 7 Sent Done Done false
1721 1520 Acked Write 10 Sent Done Done false
1722 1521 Acked Read 2 Sent Done Done false
1723 1522 Acked Flush 0 Sent Done Done false
1724 1523 Acked Read 7 Sent Done Done false
1725 1524 Acked Read 2 Sent Done Done false
1726 1525 Acked Write 7 Sent Done Done false
1727 1526 Acked Read 2 Sent Done Done false
1728 1527 Acked Read 9 Sent Done Done false
1729 1528 Acked Read 1 Sent Done Done false
1730 1529 Acked Read 7 Sent Done Done false
1731 1530 Acked Write 10 Sent Done Done false
1732 1531 Acked Flush 0 Sent Done Done false
1733 1532 Acked Read 2 Sent Done Done false
1734 1533 Acked Write 1 Sent Done Done false
1735 1534 Acked Read 6 Sent Done Done false
1736 1535 Acked Read 7 Sent Done Done false
1737 1536 Acked Write 3 Sent Done Done false
1738 1537 Acked Read 10 Sent Done Done false
1739 1538 Acked Write 3 Sent Done Done false
1740 1539 Acked Read 9 Sent Done Done false
1741 1540 Acked Write 1 Sent Done Done false
1742 1541 Acked Write 2 Sent Done Done false
1743 1542 Acked Read 3 Sent Done Done false
1744 1543 Acked Write 5 Sent Done Done false
1745 1544 Acked Read 6 Sent Done Done false
1746 1545 Acked Write 3 Sent Done Done false
1747 1546 Acked Write 5 Sent Done Done false
1748 1547 Acked Read 7 Sent Done Done false
1749 1548 Acked Read 7 Sent Done Done false
1750 1549 Acked Read 5 Sent Done Done false
1751 1550 Acked Read 10 Sent Done Done false
1752 1551 Acked Flush 0 Sent Done Done false
1753 1552 Acked Read 1 Sent Done Done false
1754 1553 Acked Write 2 Sent Done Done false
1755 1554 Acked Read 5 Sent Done Done false
1756 1555 Acked Read 6 Sent Done Done false
1757 1556 Acked Write 4 Sent Done Done false
1758 1557 Acked Write 1 Sent Done Done false
1759 1558 Acked Write 3 Sent Done Done false
1760 1559 Acked Write 7 Sent Done Done false
1761 1560 Acked Write 6 Sent Done Done false
1762 1561 Acked Read 7 Sent Done Done false
1763 1562 Acked Read 8 Sent Done Done false
1764 1563 Acked Write 3 Sent Done Done false
1765 1564 Acked Write 3 Sent Done Done false
1766 1565 Acked Write 8 Sent Done Done false
1767 1566 Acked Flush 0 Sent Done Done false
1768 1567 Acked Read 4 Sent Done Done false
1769 1568 Acked Write 10 Sent Done Done false
1770 1569 Acked Write 5 Sent Done Done false
1771 1570 Acked Write 3 Sent Done Done false
1772 1571 Acked Read 2 Sent Done Done false
1773 1572 Acked Read 4 Sent Done Done false
1774 1573 Acked Read 5 Sent Done Done false
1775 1574 Acked Write 2 Sent Done Done false
1776 1575 Acked Read 4 Sent Done Done false
1777 1576 Acked Flush 0 Sent Done Done false
1778 1577 Acked Flush 0 Sent Done Done false
1779 1578 Acked Read 3 Sent Done Done false
1780 1579 Acked Read 3 Sent Done Done false
1781 1580 Acked Write 1 Sent Done Done false
1782 1581 Acked Read 9 Sent Done Done false
1783 1582 Acked Read 7 Sent Done Done false
1784 1583 Acked Flush 0 Sent Done Done false
1785 1584 Acked Flush 0 Sent Done Done false
1786 1585 Acked Read 1 Sent Done Done false
1787 1586 Acked Read 3 Sent Done Done false
1788 1587 Acked Flush 0 Sent Done Done false
1789 1588 Acked Flush 0 Sent Done Done false
1790 1589 Acked Write 7 Sent Done Done false
1791 1590 Acked Read 5 Sent Done Done false
1792 1591 Acked Write 8 Sent Done Done false
1793 1592 Acked Read 5 Sent Done Done false
1794 1593 Acked Read 2 Sent Done Done false
1795 1594 Acked Read 3 Sent Done Done false
1796 1595 Acked Read 6 Sent Done Done false
1797 1596 Acked Flush 0 Sent Done Done false
1798 1597 Acked Read 6 Sent Done Done false
1799 1598 Acked Write 10 Sent Done Done false
1800 1599 Acked Read 2 Sent Done Done false
1801 1600 Acked Write 3 Sent Done Done false
1802 1601 Acked Flush 0 Sent Done Done false
1803 1602 Acked Read 1 Sent Done Done false
1804 STATES DS:0 DS:1 DS:2 TOTAL
1805 Sent 301 0 0 301
1806 Done 0 301 301 602
1807 Skipped 0 0 0 0
1808 Error 0 0 0 0
1809 Last Flush: 1301 1601 1601
1810 Downstairs last five completed: 1301 1300 1299 1298 1297
1811 Upstairs last five completed: 1602 1601 1600 1599 1598
1812 CLIENT: Up:0 ds:301 act:2
1813 [8810] got start action
1814 [8810] I am going to restart
1815 Starting downstairs at port 8810
1816 Make output file at "/tmp/dsc/downstairs-8810.txt"
1817 [8810][0] reports Starting
1818 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1204)
1819 [8810][0] reports Running
1820 ----------------------------------------------------------------
1821 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
1822 States: Offline Active Active
1823 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1824 1302 Acked Flush 0 Sent Done Done false
1825 1303 Acked Write 5 Sent Done Done false
1826 1304 Acked Flush 0 Sent Done Done false
1827 1305 Acked Write 5 Sent Done Done false
1828 1306 Acked Write 10 Sent Done Done false
1829 1307 Acked Write 5 Sent Done Done false
1830 1308 Acked Read 2 Sent Done Done false
1831 1309 Acked Write 2 Sent Done Done false
1832 1310 Acked Read 10 Sent Done Done false
1833 1311 Acked Read 8 Sent Done Done false
1834 1312 Acked Write 8 Sent Done Done false
1835 1313 Acked Read 6 Sent Done Done false
1836 1314 Acked Read 1 Sent Done Done false
1837 1315 Acked Write 10 Sent Done Done false
1838 1316 Acked Write 3 Sent Done Done false
1839 1317 Acked Flush 0 Sent Done Done false
1840 1318 Acked Write 1 Sent Done Done false
1841 1319 Acked Read 6 Sent Done Done false
1842 1320 Acked Flush 0 Sent Done Done false
1843 1321 Acked Read 6 Sent Done Done false
1844 1322 Acked Flush 0 Sent Done Done false
1845 1323 Acked Write 8 Sent Done Done false
1846 1324 Acked Write 8 Sent Done Done false
1847 1325 Acked Read 4 Sent Done Done false
1848 1326 Acked Write 10 Sent Done Done false
1849 1327 Acked Write 1 Sent Done Done false
1850 1328 Acked Read 7 Sent Done Done false
1851 1329 Acked Write 10 Sent Done Done false
1852 1330 Acked Read 6 Sent Done Done false
1853 1331 Acked Write 9 Sent Done Done false
1854 1332 Acked Write 6 Sent Done Done false
1855 1333 Acked Read 2 Sent Done Done false
1856 1334 Acked Read 9 Sent Done Done false
1857 1335 Acked Write 4 Sent Done Done false
1858 1336 Acked Read 8 Sent Done Done false
1859 1337 Acked Write 6 Sent Done Done false
1860 1338 Acked Read 8 Sent Done Done false
1861 1339 Acked Write 2 Sent Done Done false
1862 1340 Acked Read 8 Sent Done Done false
1863 1341 Acked Read 6 Sent Done Done false
1864 1342 Acked Write 3 Sent Done Done false
1865 1343 Acked Read 10 Sent Done Done false
1866 1344 Acked Write 8 Sent Done Done false
1867 1345 Acked Read 5 Sent Done Done false
1868 1346 Acked Read 9 Sent Done Done false
1869 1347 Acked Read 9 Sent Done Done false
1870 1348 Acked Read 2 Sent Done Done false
1871 1349 Acked Write 7 Sent Done Done false
1872 1350 Acked Write 6 Sent Done Done false
1873 1351 Acked Flush 0 Sent Done Done false
1874 1352 Acked Read 7 Sent Done Done false
1875 1353 Acked Write 7 Sent Done Done false
1876 1354 Acked Read 8 Sent Done Done false
1877 1355 Acked Flush 0 Sent Done Done false
1878 1356 Acked Flush 0 Sent Done Done false
1879 1357 Acked Write 1 Sent Done Done false
1880 1358 Acked Read 5 Sent Done Done false
1881 1359 Acked Read 10 Sent Done Done false
1882 1360 Acked Write 7 Sent Done Done false
1883 1361 Acked Flush 0 Sent Done Done false
1884 1362 Acked Read 10 Sent Done Done false
1885 1363 Acked Read 10 Sent Done Done false
1886 1364 Acked Read 2 Sent Done Done false
1887 1365 Acked Read 2 Sent Done Done false
1888 1366 Acked Read 5 Sent Done Done false
1889 1367 Acked Read 8 Sent Done Done false
1890 1368 Acked Read 8 Sent Done Done false
1891 1369 Acked Write 8 Sent Done Done false
1892 1370 Acked Write 1 Sent Done Done false
1893 1371 Acked Read 1 Sent Done Done false
1894 1372 Acked Write 7 Sent Done Done false
1895 1373 Acked Read 4 Sent Done Done false
1896 1374 Acked Write 1 Sent Done Done false
1897 1375 Acked Read 3 Sent Done Done false
1898 1376 Acked Read 2 Sent Done Done false
1899 1377 Acked Read 3 Sent Done Done false
1900 1378 Acked Read 3 Sent Done Done false
1901 1379 Acked Write 10 Sent Done Done false
1902 1380 Acked Read 4 Sent Done Done false
1903 1381 Acked Read 6 Sent Done Done false
1904 1382 Acked Write 3 Sent Done Done false
1905 1383 Acked Write 9 Sent Done Done false
1906 1384 Acked Read 8 Sent Done Done false
1907 1385 Acked Write 8 Sent Done Done false
1908 1386 Acked Write 10 Sent Done Done false
1909 1387 Acked Read 9 Sent Done Done false
1910 1388 Acked Read 3 Sent Done Done false
1911 1389 Acked Write 7 Sent Done Done false
1912 1390 Acked Write 7 Sent Done Done false
1913 1391 Acked Write 10 Sent Done Done false
1914 1392 Acked Write 4 Sent Done Done false
1915 1393 Acked Read 5 Sent Done Done false
1916 1394 Acked Read 3 Sent Done Done false
1917 1395 Acked Read 2 Sent Done Done false
1918 1396 Acked Write 4 Sent Done Done false
1919 1397 Acked Write 6 Sent Done Done false
1920 1398 Acked Write 5 Sent Done Done false
1921 1399 Acked Flush 0 Sent Done Done false
1922 1400 Acked Read 10 Sent Done Done false
1923 1401 Acked Write 3 Sent Done Done false
1924 1402 Acked Read 10 Sent Done Done false
1925 1403 Acked Write 3 Sent Done Done false
1926 1404 Acked Write 8 Sent Done Done false
1927 1405 Acked Write 6 Sent Done Done false
1928 1406 Acked Write 1 Sent Done Done false
1929 1407 Acked Read 5 Sent Done Done false
1930 1408 Acked Read 7 Sent Done Done false
1931 1409 Acked Write 2 Sent Done Done false
1932 1410 Acked Flush 0 Sent Done Done false
1933 1411 Acked Read 10 Sent Done Done false
1934 1412 Acked Read 4 Sent Done Done false
1935 1413 Acked Write 9 Sent Done Done false
1936 1414 Acked Read 5 Sent Done Done false
1937 1415 Acked Write 8 Sent Done Done false
1938 1416 Acked Read 1 Sent Done Done false
1939 1417 Acked Read 9 Sent Done Done false
1940 1418 Acked Write 2 Sent Done Done false
1941 1419 Acked Flush 0 Sent Done Done false
1942 1420 Acked Write 7 Sent Done Done false
1943 1421 Acked Write 6 Sent Done Done false
1944 1422 Acked Read 7 Sent Done Done false
1945 1423 Acked Read 3 Sent Done Done false
1946 1424 Acked Read 7 Sent Done Done false
1947 1425 Acked Write 2 Sent Done Done false
1948 1426 Acked Read 9 Sent Done Done false
1949 1427 Acked Write 2 Sent Done Done false
1950 1428 Acked Read 1 Sent Done Done false
1951 1429 Acked Read 1 Sent Done Done false
1952 1430 Acked Write 2 Sent Done Done false
1953 1431 Acked Write 4 Sent Done Done false
1954 1432 Acked Read 2 Sent Done Done false
1955 1433 Acked Read 8 Sent Done Done false
1956 1434 Acked Read 5 Sent Done Done false
1957 1435 Acked Read 1 Sent Done Done false
1958 1436 Acked Read 9 Sent Done Done false
1959 1437 Acked Write 4 Sent Done Done false
1960 1438 Acked Flush 0 Sent Done Done false
1961 1439 Acked Write 1 Sent Done Done false
1962 1440 Acked Read 2 Sent Done Done false
1963 1441 Acked Write 1 Sent Done Done false
1964 1442 Acked Read 4 Sent Done Done false
1965 1443 Acked Read 7 Sent Done Done false
1966 1444 Acked Write 10 Sent Done Done false
1967 1445 Acked Read 7 Sent Done Done false
1968 1446 Acked Read 1 Sent Done Done false
1969 1447 Acked Read 8 Sent Done Done false
1970 1448 Acked Read 3 Sent Done Done false
1971 1449 Acked Read 10 Sent Done Done false
1972 1450 Acked Flush 0 Sent Done Done false
1973 1451 Acked Write 8 Sent Done Done false
1974 1452 Acked Read 9 Sent Done Done false
1975 1453 Acked Write 10 Sent Done Done false
1976 1454 Acked Write 6 Sent Done Done false
1977 1455 Acked Read 10 Sent Done Done false
1978 1456 Acked Write 4 Sent Done Done false
1979 1457 Acked Read 10 Sent Done Done false
1980 1458 Acked Write 1 Sent Done Done false
1981 1459 Acked Flush 0 Sent Done Done false
1982 1460 Acked Write 6 Sent Done Done false
1983 1461 Acked Read 10 Sent Done Done false
1984 1462 Acked Read 10 Sent Done Done false
1985 1463 Acked Write 7 Sent Done Done false
1986 1464 Acked Read 6 Sent Done Done false
1987 1465 Acked Flush 0 Sent Done Done false
1988 1466 Acked Read 5 Sent Done Done false
1989 1467 Acked Read 9 Sent Done Done false
1990 1468 Acked Flush 0 Sent Done Done false
1991 1469 Acked Write 5 Sent Done Done false
1992 1470 Acked Read 4 Sent Done Done false
1993 1471 Acked Read 7 Sent Done Done false
1994 1472 Acked Write 9 Sent Done Done false
1995 1473 Acked Write 4 Sent Done Done false
1996 1474 Acked Read 1 Sent Done Done false
1997 1475 Acked Read 1 Sent Done Done false
1998 1476 Acked Read 7 Sent Done Done false
1999 1477 Acked Flush 0 Sent Done Done false
2000 1478 Acked Write 3 Sent Done Done false
2001 1479 Acked Read 9 Sent Done Done false
2002 1480 Acked Write 3 Sent Done Done false
2003 1481 Acked Write 6 Sent Done Done false
2004 1482 Acked Write 1 Sent Done Done false
2005 1483 Acked Flush 0 Sent Done Done false
2006 1484 Acked Write 6 Sent Done Done false
2007 1485 Acked Read 6 Sent Done Done false
2008 1486 Acked Flush 0 Sent Done Done false
2009 1487 Acked Read 10 Sent Done Done false
2010 1488 Acked Read 4 Sent Done Done false
2011 1489 Acked Write 8 Sent Done Done false
2012 1490 Acked Read 6 Sent Done Done false
2013 1491 Acked Read 2 Sent Done Done false
2014 1492 Acked Read 8 Sent Done Done false
2015 1493 Acked Write 7 Sent Done Done false
2016 1494 Acked Write 9 Sent Done Done false
2017 1495 Acked Flush 0 Sent Done Done false
2018 1496 Acked Write 2 Sent Done Done false
2019 1497 Acked Read 6 Sent Done Done false
2020 1498 Acked Read 6 Sent Done Done false
2021 1499 Acked Read 5 Sent Done Done false
2022 1500 Acked Write 6 Sent Done Done false
2023 1501 Acked Write 7 Sent Done Done false
2024 1502 Acked Read 6 Sent Done Done false
2025 1503 Acked Write 4 Sent Done Done false
2026 1504 Acked Write 8 Sent Done Done false
2027 1505 Acked Flush 0 Sent Done Done false
2028 1506 Acked Read 10 Sent Done Done false
2029 1507 Acked Read 7 Sent Done Done false
2030 1508 Acked Read 5 Sent Done Done false
2031 1509 Acked Read 6 Sent Done Done false
2032 1510 Acked Write 7 Sent Done Done false
2033 1511 Acked Write 2 Sent Done Done false
2034 1512 Acked Read 10 Sent Done Done false
2035 1513 Acked Read 2 Sent Done Done false
2036 1514 Acked Read 10 Sent Done Done false
2037 1515 Acked Read 7 Sent Done Done false
2038 1516 Acked Write 5 Sent Done Done false
2039 1517 Acked Read 3 Sent Done Done false
2040 1518 Acked Flush 0 Sent Done Done false
2041 1519 Acked Read 7 Sent Done Done false
2042 1520 Acked Write 10 Sent Done Done false
2043 1521 Acked Read 2 Sent Done Done false
2044 1522 Acked Flush 0 Sent Done Done false
2045 1523 Acked Read 7 Sent Done Done false
2046 1524 Acked Read 2 Sent Done Done false
2047 1525 Acked Write 7 Sent Done Done false
2048 1526 Acked Read 2 Sent Done Done false
2049 1527 Acked Read 9 Sent Done Done false
2050 1528 Acked Read 1 Sent Done Done false
2051 1529 Acked Read 7 Sent Done Done false
2052 1530 Acked Write 10 Sent Done Done false
2053 1531 Acked Flush 0 Sent Done Done false
2054 1532 Acked Read 2 Sent Done Done false
2055 1533 Acked Write 1 Sent Done Done false
2056 1534 Acked Read 6 Sent Done Done false
2057 1535 Acked Read 7 Sent Done Done false
2058 1536 Acked Write 3 Sent Done Done false
2059 1537 Acked Read 10 Sent Done Done false
2060 1538 Acked Write 3 Sent Done Done false
2061 1539 Acked Read 9 Sent Done Done false
2062 1540 Acked Write 1 Sent Done Done false
2063 1541 Acked Write 2 Sent Done Done false
2064 1542 Acked Read 3 Sent Done Done false
2065 1543 Acked Write 5 Sent Done Done false
2066 1544 Acked Read 6 Sent Done Done false
2067 1545 Acked Write 3 Sent Done Done false
2068 1546 Acked Write 5 Sent Done Done false
2069 1547 Acked Read 7 Sent Done Done false
2070 1548 Acked Read 7 Sent Done Done false
2071 1549 Acked Read 5 Sent Done Done false
2072 1550 Acked Read 10 Sent Done Done false
2073 1551 Acked Flush 0 Sent Done Done false
2074 1552 Acked Read 1 Sent Done Done false
2075 1553 Acked Write 2 Sent Done Done false
2076 1554 Acked Read 5 Sent Done Done false
2077 1555 Acked Read 6 Sent Done Done false
2078 1556 Acked Write 4 Sent Done Done false
2079 1557 Acked Write 1 Sent Done Done false
2080 1558 Acked Write 3 Sent Done Done false
2081 1559 Acked Write 7 Sent Done Done false
2082 1560 Acked Write 6 Sent Done Done false
2083 1561 Acked Read 7 Sent Done Done false
2084 1562 Acked Read 8 Sent Done Done false
2085 1563 Acked Write 3 Sent Done Done false
2086 1564 Acked Write 3 Sent Done Done false
2087 1565 Acked Write 8 Sent Done Done false
2088 1566 Acked Flush 0 Sent Done Done false
2089 1567 Acked Read 4 Sent Done Done false
2090 1568 Acked Write 10 Sent Done Done false
2091 1569 Acked Write 5 Sent Done Done false
2092 1570 Acked Write 3 Sent Done Done false
2093 1571 Acked Read 2 Sent Done Done false
2094 1572 Acked Read 4 Sent Done Done false
2095 1573 Acked Read 5 Sent Done Done false
2096 1574 Acked Write 2 Sent Done Done false
2097 1575 Acked Read 4 Sent Done Done false
2098 1576 Acked Flush 0 Sent Done Done false
2099 1577 Acked Flush 0 Sent Done Done false
2100 1578 Acked Read 3 Sent Done Done false
2101 1579 Acked Read 3 Sent Done Done false
2102 1580 Acked Write 1 Sent Done Done false
2103 1581 Acked Read 9 Sent Done Done false
2104 1582 Acked Read 7 Sent Done Done false
2105 1583 Acked Flush 0 Sent Done Done false
2106 1584 Acked Flush 0 Sent Done Done false
2107 1585 Acked Read 1 Sent Done Done false
2108 1586 Acked Read 3 Sent Done Done false
2109 1587 Acked Flush 0 Sent Done Done false
2110 1588 Acked Flush 0 Sent Done Done false
2111 1589 Acked Write 7 Sent Done Done false
2112 1590 Acked Read 5 Sent Done Done false
2113 1591 Acked Write 8 Sent Done Done false
2114 1592 Acked Read 5 Sent Done Done false
2115 1593 Acked Read 2 Sent Done Done false
2116 1594 Acked Read 3 Sent Done Done false
2117 1595 Acked Read 6 Sent Done Done false
2118 1596 Acked Flush 0 Sent Done Done false
2119 1597 Acked Read 6 Sent Done Done false
2120 1598 Acked Write 10 Sent Done Done false
2121 1599 Acked Read 2 Sent Done Done false
2122 1600 Acked Write 3 Sent Done Done false
2123 1601 Acked Flush 0 Sent Done Done false
2124 1602 Acked Read 1 Sent Done Done false
2125 1603 Acked Flush 0 Sent Done Done false
2126 STATES DS:0 DS:1 DS:2 TOTAL
2127 Sent 302 0 0 302
2128 Done 0 302 302 604
2129 Skipped 0 0 0 0
2130 Error 0 0 0 0
2131 Last Flush: 1301 1603 1603
2132 Downstairs last five completed: 1301 1300 1299 1298 1297
2133 Upstairs last five completed: 1603 1602 1601 1600 1599
2134 CLIENT: Up:0 ds:302 act:2
21352025-01-13T15:52:13.293ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
21362025-01-13T15:52:13.293ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50821) = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
21372025-01-13T15:52:13.294ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
21382025-01-13T15:52:13.295ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
21392025-01-13T15:52:13.295ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
21402025-01-13T15:52:13.295ZINFOcrucible: send last flush ID to this DS: 1301 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
21412025-01-13T15:52:13.295ZINFOcrucible: Replied this last flush ID: 1301 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
21422025-01-13T15:52:13.295ZINFOcrucible: [0] client re-new 302 jobs since flush 1301 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
21432025-01-13T15:52:13.296ZINFOcrucible: [0] Replayed 302 jobs since flush: 1301 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
2144 ----------------------------------------------------------------
2145 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
2146 Downstairs last five completed: 1603 1602 1601 1600 1599
2147 Upstairs last five completed: 1603 1602 1601 1600 1599
2148 CLIENT: Up:0 ds:0 act:3
2149 Replay: All jobs finished
2150 Main task has work to do, go find it
2151 got dsc Stop(0)
2152 stop 0
2153 Jan 13 15:52:15.540 INFO request completed, latency_us: 364, response_code: 204[8810] Got stop action so:false kr:false
2154 , uri: /stop/cid/0, method: POST, req_id: eb519300-8127-439f-8893-534d6b9b2ee6, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
2155 Jan 13 15:52:15.540 INFO request completed, latency_us: 137, response_code: 200, uri: /state/cid/0, method: GET, req_id: ac0e66dd-f5ed-41ef-8f25-1aa67d60be3f, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
21562025-01-13T15:52:15.541ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
21572025-01-13T15:52:15.541ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
21582025-01-13T15:52:15.541ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
21592025-01-13T15:52:15.541ZINFOcrucible: client task is exiting = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
2160 [8810] Exited with: ExitStatus(unix_wait_status(9))
2161 [8810][0] reports Exit
2162 Jan 13 15:52:19.543 INFO request completed, latency_us: 260, response_code: 200, uri: /state/cid/0, method: GET, req_id: 23ac8b56-ec84-4886-a101-debbc4fc152f, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
2163 001/300 Flush
2164 002/300 Flush
2165 003/300 Read block 3129 len 4096
2166 004/300 Read block 1479 len 16384
2167 005/300 Read block 2434 len 8192
2168 006/300 Read block 3889 len 4096
2169 007/300 Write block 1772 len 8192 data: 2 2
2170 008/300 Write block 1646 len 20480 data: 3 3 3 3 3
2171 009/300 Write block 4392 len 32768 data: 2 2 2 2 2 2 2 2
2172 010/300 Read block 4921 len 28672
2173 011/300 Read block 3630 len 12288
2174 012/300 Read block 1132 len 12288
2175 013/300 Write block 502 len 8192 data: 3 3
2176 014/300 Read block 4017 len 40960
2177 015/300 Read block 2079 len 32768
2178 016/300 Write block 2590 len 32768 data: 2 2 3 3 3 3 3 3
2179 017/300 Write block 1570 len 8192 data: 3 2
2180 018/300 Write block 3387 len 16384 data: 2 2 3 3
2181 019/300 Read block 1502 len 28672
2182 020/300 Read block 4222 len 12288
2183 021/300 Read block 4721 len 12288
2184 022/300 Read block 4669 len 32768
2185 023/300 Write block 3290 len 20480 data: 3 2 2 2 2
2186 024/300 Flush
2187 025/300 Read block 430 len 4096
2188 026/300 Write block 838 len 20480 data: 3 3 3 4 4
2189 027/300 Read block 2704 len 28672
2190 028/300 Write block 229 len 8192 data: 2 2
2191 029/300 Read block 4943 len 36864
2192 030/300 Write block 4343 len 24576 data: 2 2 3 2 2 2
2193 031/300 Read block 2287 len 28672
2194 032/300 Read block 1885 len 8192
2195 033/300 Flush
2196 034/300 Write block 4334 len 16384 data: 2 2 2 2
2197 035/300 Write block 1514 len 16384 data: 2 2 2 2
2198 036/300 Read block 1939 len 16384
2199 037/300 Write block 211 len 32768 data: 2 2 2 2 2 2 2 2
2200 038/300 Write block 4439 len 20480 data: 2 2 2 2 2
2201 039/300 Read block 125 len 36864
2202 040/300 Read block 1719 len 32768
2203 041/300 Read block 2288 len 20480
2204 042/300 Write block 1632 len 24576 data: 2 2 2 2 2 2
2205 043/300 Read block 2202 len 20480
2206 044/300 Write block 2216 len 32768 data: 2 2 2 2 2 3 3 4
2207 045/300 Read block 3578 len 24576
2208 046/300 Write block 1094 len 16384 data: 2 2 2 2
2209 047/300 Write block 3235 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2210 048/300 Write block 142 len 32768 data: 3 2 2 2 2 2 2 2
2211 049/300 Write block 3644 len 16384 data: 2 2 3 3
2212 050/300 Read block 4162 len 40960
2213 051/300 Write block 4448 len 36864 data: 2 2 2 2 2 2 2 2 2
2214 052/300 Write block 942 len 32768 data: 2 2 2 2 2 2 2 2
2215 053/300 Read block 2743 len 20480
2216 054/300 Read block 451 len 28672
2217 055/300 Write block 961 len 4096 data: 2
2218 056/300 Write block 987 len 12288 data: 2 2 2
2219 057/300 Read block 1107 len 36864
2220 058/300 Read block 1182 len 28672
2221 059/300 Read block 1609 len 4096
2222 060/300 Write block 1217 len 32768 data: 2 2 2 2 2 2 2 2
2223 061/300 Write block 2711 len 36864 data: 2 2 2 2 2 2 2 2 2
2224 062/300 Read block 2601 len 12288
2225 063/300 Write block 1811 len 36864 data: 2 2 2 2 2 2 3 3 3
2226 064/300 Write block 2437 len 24576 data: 2 2 2 2 2 2
2227 065/300 Write block 3393 len 16384 data: 3 3 3 3
2228 066/300 Write block 3582 len 20480 data: 2 2 2 2 2
2229 067/300 Read block 2175 len 16384
2230 068/300 Write block 3195 len 32768 data: 2 3 3 3 3 3 3 3
2231 069/300 Write block 2697 len 12288 data: 2 2 2
2232 070/300 Read block 2305 len 16384
2233 071/300 Write block 2479 len 32768 data: 3 3 3 3 3 3 3 3
2234 072/300 Read block 2705 len 40960
2235 073/300 Write block 2089 len 24576 data: 2 2 2 2 3 3
2236 074/300 Write block 3664 len 24576 data: 2 2 2 2 2 2
2237 075/300 Write block 3185 len 32768 data: 2 2 2 2 2 2 2 2
2238 076/300 Read block 2380 len 20480
2239 077/300 Read block 4843 len 40960
2240 078/300 Write block 2847 len 8192 data: 4 4
2241 079/300 Read block 173 len 8192
2242 080/300 Write block 546 len 20480 data: 3 2 2 2 2
2243 081/300 Read block 1505 len 24576
2244 082/300 Write block 4447 len 4096 data: 2
2245 083/300 Write block 1668 len 8192 data: 3 3
2246 084/300 Write block 3971 len 16384 data: 2 2 2 2
2247 085/300 Read block 3836 len 12288
2248 086/300 Read block 2649 len 28672
2249 087/300 Write block 108 len 32768 data: 2 2 2 2 2 2 2 3
2250 088/300 Read block 230 len 28672
2251 089/300 Read block 4312 len 4096
2252 090/300 Read block 4038 len 12288
2253 091/300 Write block 743 len 32768 data: 2 2 2 2 2 2 2 2
2254 092/300 Write block 3666 len 36864 data: 3 3 3 3 2 2 2 2 2
2255 093/300 Read block 4886 len 20480
2256 094/300 Flush
2257 095/300 Read block 2246 len 20480
2258 096/300 Write block 4379 len 12288 data: 2 2 2
2259 097/300 Read block 3722 len 40960
2260 098/300 Read block 3048 len 36864
2261 099/300 Write block 4684 len 20480 data: 2 2 2 2 2
2262 100/300 Read block 4088 len 28672
2263 101/300 Write block 3721 len 36864 data: 2 2 2 2 2 2 2 2 2
2264 102/300 Read block 1636 len 40960
2265 103/300 Read block 2058 len 36864
2266 104/300 Read block 1291 len 28672
2267 105/300 Read block 3015 len 36864
2268 106/300 Read block 3222 len 24576
2269 107/300 Read block 3136 len 32768
2270 108/300 Write block 2277 len 20480 data: 2 2 2 2 2
2271 109/300 Read block 1455 len 12288
2272 110/300 Write block 542 len 40960 data: 3 3 3 3 4 3 3 3 3 2
2273 111/300 Read block 2479 len 4096
2274 112/300 Write block 1745 len 36864 data: 3 2 2 2 2 2 2 2 2
2275 113/300 Read block 3719 len 40960
2276 114/300 Write block 4301 len 12288 data: 2 2 2
2277 115/300 Read block 2243 len 24576
2278 116/300 Read block 906 len 12288
2279 117/300 Write block 4632 len 20480 data: 2 2 2 2 2
2280 118/300 Read block 1125 len 8192
2281 119/300 Read block 3590 len 28672
2282 120/300 Flush
2283 121/300 Read block 4577 len 32768
2284 122/300 Write block 3095 len 32768 data: 3 3 3 3 3 3 3 2
2285 123/300 Read block 1638 len 24576
2286 124/300 Read block 4013 len 16384
2287 125/300 Write block 3003 len 40960 data: 2 2 2 2 2 2 3 3 3 3
2288 126/300 Write block 474 len 8192 data: 3 3
2289 127/300 Flush
2290 128/300 Write block 339 len 12288 data: 2 2 2
2291 129/300 Read block 4304 len 8192
2292 130/300 Read block 3777 len 28672
2293 131/300 Read block 3369 len 4096
2294 132/300 Write block 1590 len 12288 data: 3 3 2
2295 133/300 Flush
2296 134/300 Read block 3011 len 28672
2297 135/300 Read block 4283 len 20480
2298 136/300 Read block 3902 len 8192
2299 137/300 Read block 2251 len 40960
2300 138/300 Read block 2791 len 12288
2301 139/300 Read block 3359 len 4096
2302 140/300 Write block 1044 len 24576 data: 2 2 2 2 3 3
2303 141/300 Read block 517 len 12288
2304 142/300 Read block 1406 len 20480
2305 143/300 Write block 4749 len 32768 data: 2 2 2 2 2 2 2 2
2306 144/300 Read block 3839 len 4096
2307 145/300 Write block 288 len 16384 data: 2 2 2 2
2308 146/300 Read block 2902 len 24576
2309 147/300 Read block 907 len 36864
2310 148/300 Read block 4784 len 28672
2311 149/300 Read block 4073 len 28672
2312 150/300 Write block 4211 len 24576 data: 2 2 2 2 2 2
2313 151/300 Read block 2685 len 36864
2314 152/300 Write block 2791 len 8192 data: 4 4
2315 153/300 Write block 3389 len 8192 data: 4 4
2316 154/300 Write block 2814 len 8192 data: 2 2
2317 155/300 Read block 702 len 36864
2318 156/300 Flush
2319 157/300 Read block 861 len 12288
2320 158/300 Write block 2805 len 24576 data: 2 2 2 2 2 2
2321 159/300 Read block 3016 len 32768
2322 160/300 Write block 4180 len 24576 data: 2 2 2 2 2 2
2323 161/300 Write block 3084 len 24576 data: 2 2 2 2 2 2
2324 162/300 Write block 512 len 32768 data: 3 3 3 3 3 3 3 3
2325 163/300 Read block 4316 len 24576
2326 164/300 Read block 1338 len 8192
2327 165/300 Write block 4860 len 16384 data: 2 2 3 3
2328 166/300 Write block 1115 len 16384 data: 2 2 2 2
2329 167/300 Flush
2330 168/300 Read block 2684 len 40960
2331 169/300 Read block 3444 len 24576
2332 170/300 Read block 2470 len 8192
2333 171/300 Write block 4930 len 32768 data: 2 2 2 2 2 2 2 2
2334 172/300 Write block 2553 len 12288 data: 2 2 2
2335 173/300 Read block 4517 len 12288
2336 174/300 Write block 77 len 32768 data: 2 3 3 3 3 3 2 2
2337 175/300 Write block 3223 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2338 176/300 Read block 2373 len 16384
2339 177/300 Read block 1910 len 28672
2340 178/300 Write block 2543 len 36864 data: 3 3 3 3 3 3 3 2 2
2341 179/300 Read block 521 len 40960
2342 180/300 Read block 3548 len 4096
2343 181/300 Flush
2344 182/300 Read block 4279 len 24576
2345 183/300 Read block 3521 len 12288
2346 184/300 Write block 1372 len 24576 data: 2 2 2 2 2 2
2347 185/300 Read block 3094 len 8192
2348 186/300 Write block 527 len 24576 data: 2 2 2 2 2 2
2349 187/300 Write block 1523 len 36864 data: 2 2 2 2 3 3 3 3 3
2350 188/300 Write block 1161 len 36864 data: 2 2 2 2 2 2 2 2 2
2351 189/300 Write block 3609 len 36864 data: 3 3 3 3 2 3 3 3 3
2352 190/300 Read block 4856 len 16384
2353 191/300 Read block 2354 len 16384
2354 192/300 Write block 3499 len 28672 data: 2 2 2 2 2 2 2
2355 193/300 Write block 3883 len 24576 data: 2 2 2 2 3 3
2356 194/300 Read block 2949 len 8192
2357 195/300 Read block 3293 len 4096
2358 196/300 Read block 1807 len 16384
2359 197/300 Write block 3487 len 40960 data: 4 3 3 3 3 3 2 2 2 2
2360 198/300 Write block 2228 len 16384 data: 3 3 2 2
2361 199/300 Read block 727 len 20480
2362 200/300 Write block 3421 len 24576 data: 2 2 2 2 2 2
2363 201/300 Read block 3911 len 16384
2364 202/300 Read block 1922 len 20480
2365 203/300 Write block 3937 len 32768 data: 3 3 3 3 3 3 3 3
2366 204/300 Write block 1739 len 36864 data: 3 3 3 3 4 3 4 3 3
2367 205/300 Read block 1800 len 16384
2368 206/300 Read block 360 len 8192
2369 207/300 Write block 954 len 12288 data: 2 2 2
2370 208/300 Read block 3922 len 40960
2371 209/300 Read block 2413 len 32768
2372 210/300 Write block 3406 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2373 211/300 Write block 1813 len 28672 data: 3 3 3 3 4 4 4
2374 212/300 Read block 2163 len 16384
2375 213/300 Flush
2376 214/300 Read block 3266 len 4096
2377 215/300 Read block 16 len 12288
2378 216/300 Write block 2433 len 16384 data: 2 2 2 2
2379 217/300 Read block 1977 len 8192
2380 218/300 Read block 4130 len 20480
2381 219/300 Write block 4181 len 4096 data: 3
2382 220/300 Read block 1173 len 12288
2383 221/300 Write block 2443 len 8192 data: 2 2
2384 222/300 Read block 30 len 24576
2385 223/300 Write block 3592 len 32768 data: 3 2 2 2 2 2 2 2
2386 224/300 Write block 2248 len 36864 data: 2 2 2 2 2 2 2 2 2
2387 225/300 Write block 1688 len 8192 data: 2 2
2388 226/300 Flush
2389 227/300 Read block 3274 len 4096
2390 228/300 Read block 2488 len 40960
2391 229/300 Write block 2064 len 4096 data: 3
2392 230/300 Write block 4436 len 28672 data: 2 2 2 3 3 3 3
2393 231/300 Read block 3067 len 28672
2394 232/300 Write block 1336 len 8192 data: 3 3
2395 233/300 Write block 1335 len 36864 data: 3 4 4 3 3 3 2 2 2
2396 234/300 Read block 871 len 16384
2397 235/300 Write block 2937 len 40960 data: 2 3 3 3 2 2 2 2 2 2
2398 236/300 Flush
2399 237/300 Write block 4604 len 16384 data: 2 2 2 2
2400 238/300 Write block 944 len 36864 data: 3 3 3 3 3 3 2 2 2
2401 239/300 Write block 3911 len 12288 data: 2 2 2
2402 240/300 Write block 1492 len 8192 data: 2 2
2403 241/300 Read block 276 len 40960
2404 242/300 Read block 2129 len 32768
2405 243/300 Read block 3607 len 20480
2406 244/300 Write block 2341 len 36864 data: 3 3 3 3 3 2 2 2 2
2407 245/300 Write block 1972 len 12288 data: 2 2 2
2408 246/300 Read block 506 len 16384
2409 247/300 Read block 84 len 20480
2410 248/300 Read block 1583 len 32768
2411 249/300 Write block 2591 len 36864 data: 3 4 4 4 4 4 4 3 3
2412 250/300 Read block 3901 len 16384
2413 251/300 Write block 2798 len 24576 data: 3 3 3 2 2 2
2414 252/300 Read block 2416 len 28672
2415 253/300 Write block 4073 len 20480 data: 2 2 2 2 3
2416 254/300 Read block 3924 len 32768
2417 255/300 Write block 588 len 4096 data: 2
2418 256/300 Flush
2419 257/300 Read block 117 len 16384
2420 258/300 Read block 837 len 24576
2421 259/300 Write block 3063 len 8192 data: 2 2
2422 260/300 Write block 3956 len 8192 data: 2 2
2423 261/300 Write block 2697 len 36864 data: 3 3 3 2 2 2 2 2 2
2424 262/300 Write block 4974 len 16384 data: 2 2 2 2
2425 263/300 Flush
2426 264/300 Write block 3321 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2427 265/300 Read block 2295 len 4096
2428 266/300 Write block 3459 len 36864 data: 2 2 2 2 2 2 2 2 2
2429 267/300 Write block 3800 len 24576 data: 2 2 2 2 2 2
2430 268/300 Write block 1186 len 8192 data: 2 2
2431 269/300 Read block 3731 len 20480
2432 270/300 Write block 1757 len 36864 data: 2 3 3 3 3 3 3 4 4
2433 271/300 Flush
2434 272/300 Read block 2377 len 20480
2435 273/300 Read block 1112 len 16384
2436 274/300 Flush
2437 275/300 Read block 2994 len 16384
2438 276/300 Write block 1410 len 12288 data: 2 2 2
2439 277/300 Flush
2440 278/300 Read block 2104 len 8192
2441 279/300 Read block 2406 len 40960
2442 280/300 Write block 3065 len 4096 data: 3
2443 281/300 Write block 271 len 12288 data: 3 3 3
2444 282/300 Write block 1650 len 8192 data: 4 3
2445 283/300 Read block 983 len 40960
2446 284/300 Write block 2077 len 12288 data: 2 2 2
2447 285/300 Read block 658 len 24576
2448 286/300 Write block 168 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2449 287/300 Write block 4926 len 28672 data: 2 2 2 2 3 3 3
2450 288/300 Read block 3130 len 12288
2451 289/300 Read block 3518 len 24576
2452 290/300 Write block 433 len 28672 data: 2 2 2 2 2 2 2
2453 291/300 Read block 4952 len 20480
2454 292/300 Read block 1279 len 4096
2455 293/300 Write block 3987 len 8192 data: 2 2
2456 294/300 Read block 865 len 32768
2457 295/300 Read block 3054 len 40960
2458 296/300 Read block 2024 len 4096
2459 297/300 Flush
2460 298/300 Read block 467 len 20480
2461 299/300 Write block 2661 len 36864 data: 2 2 2 2 2 2 2 2 2
2462 300/300 Flush
2463 301/300 Write block 2770 len 8192 data: 2 2
2464 Main task has work to do, go find it
2465 got dsc Start(0)
2466 start 0
2467 Jan 13 15:52:19.781 INFO request completed, latency_us: 199, response_code: 204, uri: /start/cid/0, method: POST, req_id: 08a31553-0019-4b3f-83c0-259bdef2348d, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
2468 Replay: started 0, returned:Ok(())
2469 ----------------------------------------------------------------
2470 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
2471 States: Offline Active Active
2472 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2473 1604 Acked Flush 0 Sent Done Done false
2474 1605 Acked Flush 0 Sent Done Done false
2475 1606 Acked Read 1 Sent Done Done false
2476 1607 Acked Read 4 Sent Done Done false
2477 1608 Acked Read 2 Sent Done Done false
2478 1609 Acked Read 1 Sent Done Done false
2479 1610 Acked Write 2 Sent Done Done false
2480 1611 Acked Write 5 Sent Done Done false
2481 1612 Acked Write 8 Sent Done Done false
2482 1613 Acked Read 7 Sent Done Done false
2483 1614 Acked Read 3 Sent Done Done false
2484 1615 Acked Read 3 Sent Done Done false
2485 1616 Acked Write 2 Sent Done Done false
2486 1617 Acked Read 10 Sent Done Done false
2487 1618 Acked Read 8 Sent Done Done false
2488 1619 Acked Write 8 Sent Done Done false
2489 1620 Acked Write 2 Sent Done Done false
2490 1621 Acked Write 4 Sent Done Done false
2491 1622 Acked Read 7 Sent Done Done false
2492 1623 Acked Read 3 Sent Done Done false
2493 1624 Acked Read 3 Sent Done Done false
2494 1625 Acked Read 8 Sent Done Done false
2495 1626 Acked Write 5 Sent Done Done false
2496 1627 Acked Flush 0 Sent Done Done false
2497 1628 Acked Read 1 Sent Done Done false
2498 1629 Acked Write 5 Sent Done Done false
2499 1630 Acked Read 7 Sent Done Done false
2500 1631 Acked Write 2 Sent Done Done false
2501 1632 Acked Read 9 Sent Done Done false
2502 1633 Acked Write 6 Sent Done Done false
2503 1634 Acked Read 7 Sent Done Done false
2504 1635 Acked Read 2 Sent Done Done false
2505 1636 Acked Flush 0 Sent Done Done false
2506 1637 Acked Write 4 Sent Done Done false
2507 1638 Acked Write 4 Sent Done Done false
2508 1639 Acked Read 4 Sent Done Done false
2509 1640 Acked Write 8 Sent Done Done false
2510 1641 Acked Write 5 Sent Done Done false
2511 1642 Acked Read 9 Sent Done Done false
2512 1643 Acked Read 8 Sent Done Done false
2513 1644 Acked Read 5 Sent Done Done false
2514 1645 Acked Write 6 Sent Done Done false
2515 1646 Acked Read 5 Sent Done Done false
2516 1647 Acked Write 8 Sent Done Done false
2517 1648 Acked Read 6 Sent Done Done false
2518 1649 Acked Write 4 Sent Done Done false
2519 1650 Acked Write 10 Sent Done Done false
2520 1651 Acked Write 8 Sent Done Done false
2521 1652 Acked Write 4 Sent Done Done false
2522 1653 Acked Read 10 Sent Done Done false
2523 1654 Acked Write 9 Sent Done Done false
2524 1655 Acked Write 8 Sent Done Done false
2525 1656 Acked Read 5 Sent Done Done false
2526 1657 Acked Read 7 Sent Done Done false
2527 1658 Acked Write 1 Sent Done Done false
2528 1659 Acked Write 3 Sent Done Done false
2529 1660 Acked Read 9 Sent Done Done false
2530 1661 Acked Read 7 Sent Done Done false
2531 1662 Acked Read 1 Sent Done Done false
2532 1663 Acked Write 8 Sent Done Done false
2533 1664 Acked Write 9 Sent Done Done false
2534 1665 Acked Read 3 Sent Done Done false
2535 1666 Acked Write 9 Sent Done Done false
2536 1667 Acked Write 6 Sent Done Done false
2537 1668 Acked Write 4 Sent Done Done false
2538 1669 Acked Write 5 Sent Done Done false
2539 1670 Acked Read 4 Sent Done Done false
2540 1671 Acked Write 8 Sent Done Done false
2541 1672 Acked Write 3 Sent Done Done false
2542 1673 Acked Read 4 Sent Done Done false
2543 1674 Acked Write 8 Sent Done Done false
2544 1675 Acked Read 10 Sent Done Done false
2545 1676 Acked Write 6 Sent Done Done false
2546 1677 Acked Write 6 Sent Done Done false
2547 1678 Acked Write 8 Sent Done Done false
2548 1679 Acked Read 5 Sent Done Done false
2549 1680 Acked Read 10 Sent Done Done false
2550 1681 Acked Write 2 Sent Done Done false
2551 1682 Acked Read 2 Sent Done Done false
2552 1683 Acked Write 5 Sent Done Done false
2553 1684 Acked Read 6 Sent Done Done false
2554 1685 Acked Write 1 Sent Done Done false
2555 1686 Acked Write 2 Sent Done Done false
2556 1687 Acked Write 4 Sent Done Done false
2557 1688 Acked Read 3 Sent Done Done false
2558 1689 Acked Read 7 Sent Done Done false
2559 1690 Acked Write 8 Sent Done Done false
2560 1691 Acked Read 7 Sent Done Done false
2561 1692 Acked Read 1 Sent Done Done false
2562 1693 Acked Read 3 Sent Done Done false
2563 1694 Acked Write 8 Sent Done Done false
2564 1695 Acked Write 9 Sent Done Done false
2565 1696 Acked Read 5 Sent Done Done false
2566 1697 Acked Flush 0 Sent Done Done false
2567 1698 Acked Read 5 Sent Done Done false
2568 1699 Acked Write 3 Sent Done Done false
2569 1700 Acked Read 10 Sent Done Done false
2570 1701 Acked Read 9 Sent Done Done false
2571 1702 Acked Write 5 Sent Done Done false
2572 1703 Acked Read 7 Sent Done Done false
2573 1704 Acked Write 9 Sent Done Done false
2574 1705 Acked Read 10 Sent Done Done false
2575 1706 Acked Read 9 Sent Done Done false
2576 1707 Acked Read 7 Sent Done Done false
2577 1708 Acked Read 9 Sent Done Done false
2578 1709 Acked Read 6 Sent Done Done false
2579 1710 Acked Read 8 Sent Done Done false
2580 1711 Acked Write 5 Sent Done Done false
2581 1712 Acked Read 3 Sent Done Done false
2582 1713 Acked Write 10 Sent Done Done false
2583 1714 Acked Read 1 Sent Done Done false
2584 1715 Acked Write 9 Sent Done Done false
2585 1716 Acked Read 10 Sent Done Done false
2586 1717 Acked Write 3 Sent Done Done false
2587 1718 Acked Read 6 Sent Done Done false
2588 1719 Acked Read 3 Sent Done Done false
2589 1720 Acked Write 5 Sent Done Done false
2590 1721 Acked Read 2 Sent Done Done false
2591 1722 Acked Read 7 Sent Done Done false
2592 1723 Acked Flush 0 Sent Done Done false
2593 1724 Acked Read 8 Sent Done Done false
2594 1725 Acked Write 8 Sent Done Done false
2595 1726 Acked Read 6 Sent Done Done false
2596 1727 Acked Read 4 Sent Done Done false
2597 1728 Acked Write 10 Sent Done Done false
2598 1729 Acked Write 2 Sent Done Done false
2599 1730 Acked Flush 0 Sent Done Done false
2600 1731 Acked Write 3 Sent Done Done false
2601 1732 Acked Read 2 Sent Done Done false
2602 1733 Acked Read 7 Sent Done Done false
2603 1734 Acked Read 1 Sent Done Done false
2604 1735 Acked Write 3 Sent Done Done false
2605 1736 Acked Flush 0 Sent Done Done false
2606 1737 Acked Read 7 Sent Done Done false
2607 1738 Acked Read 5 Sent Done Done false
2608 1739 Acked Read 2 Sent Done Done false
2609 1740 Acked Read 10 Sent Done Done false
2610 1741 Acked Read 3 Sent Done Done false
2611 1742 Acked Read 1 Sent Done Done false
2612 1743 Acked Write 6 Sent Done Done false
2613 1744 Acked Read 3 Sent Done Done false
2614 1745 Acked Read 5 Sent Done Done false
2615 1746 Acked Write 8 Sent Done Done false
2616 1747 Acked Read 1 Sent Done Done false
2617 1748 Acked Write 4 Sent Done Done false
2618 1749 Acked Read 6 Sent Done Done false
2619 1750 Acked Read 9 Sent Done Done false
2620 1751 Acked Read 7 Sent Done Done false
2621 1752 Acked Read 7 Sent Done Done false
2622 1753 Acked Write 6 Sent Done Done false
2623 1754 Acked Read 9 Sent Done Done false
2624 1755 Acked Write 2 Sent Done Done false
2625 1756 Acked Write 2 Sent Done Done false
2626 1757 Acked Write 2 Sent Done Done false
2627 1758 Acked Read 9 Sent Done Done false
2628 1759 Acked Flush 0 Sent Done Done false
2629 1760 Acked Read 3 Sent Done Done false
2630 1761 Acked Write 6 Sent Done Done false
2631 1762 Acked Read 8 Sent Done Done false
2632 1763 Acked Write 6 Sent Done Done false
2633 1764 Acked Write 6 Sent Done Done false
2634 1765 Acked Write 8 Sent Done Done false
2635 1766 Acked Read 6 Sent Done Done false
2636 1767 Acked Read 2 Sent Done Done false
2637 1768 Acked Write 4 Sent Done Done false
2638 1769 Acked Write 4 Sent Done Done false
2639 1770 Acked Flush 0 Sent Done Done false
2640 1771 Acked Read 10 Sent Done Done false
2641 1772 Acked Read 6 Sent Done Done false
2642 1773 Acked Read 2 Sent Done Done false
2643 1774 Acked Write 8 Sent Done Done false
2644 1775 Acked Write 3 Sent Done Done false
2645 1776 Acked Read 3 Sent Done Done false
2646 1777 Acked Write 8 Sent Done Done false
2647 1778 Acked Write 10 Sent Done Done false
2648 1779 Acked Read 4 Sent Done Done false
2649 1780 Acked Read 7 Sent Done Done false
2650 1781 Acked Write 9 Sent Done Done false
2651 1782 Acked Read 10 Sent Done Done false
2652 1783 Acked Read 1 Sent Done Done false
2653 1784 Acked Flush 0 Sent Done Done false
2654 1785 Acked Read 6 Sent Done Done false
2655 1786 Acked Read 3 Sent Done Done false
2656 1787 Acked Write 6 Sent Done Done false
2657 1788 Acked Read 2 Sent Done Done false
2658 1789 Acked Write 6 Sent Done Done false
2659 1790 Acked Write 9 Sent Done Done false
2660 1791 Acked Write 9 Sent Done Done false
2661 1792 Acked Write 9 Sent Done Done false
2662 1793 Acked Read 4 Sent Done Done false
2663 1794 Acked Read 4 Sent Done Done false
2664 1795 Acked Write 7 Sent Done Done false
2665 1796 Acked Write 6 Sent Done Done false
2666 1797 Acked Read 2 Sent Done Done false
2667 1798 Acked Read 1 Sent Done Done false
2668 1799 Acked Read 4 Sent Done Done false
2669 1800 Acked Write 10 Sent Done Done false
2670 1801 Acked Write 4 Sent Done Done false
2671 1802 Acked Read 5 Sent Done Done false
2672 1803 Acked Write 6 Sent Done Done false
2673 1804 Acked Read 4 Sent Done Done false
2674 1805 Acked Read 5 Sent Done Done false
2675 1806 Acked Write 8 Sent Done Done false
2676 1807 Acked Write 9 Sent Done Done false
2677 1808 Acked Read 4 Sent Done Done false
2678 1809 Acked Read 2 Sent Done Done false
2679 1810 Acked Write 3 Sent Done Done false
2680 1811 Acked Read 10 Sent Done Done false
2681 1812 Acked Read 8 Sent Done Done false
2682 1813 Acked Write 10 Sent Done Done false
2683 1814 Acked Write 7 Sent Done Done false
2684 1815 Acked Read 4 Sent Done Done false
2685 1816 Acked Flush 0 Sent Done Done false
2686 1817 Acked Read 1 Sent Done Done false
2687 1818 Acked Read 3 Sent Done Done false
2688 1819 Acked Write 4 Sent Done Done false
2689 1820 Acked Read 2 Sent Done Done false
2690 1821 Acked Read 5 Sent Done Done false
2691 1822 Acked Write 1 Sent Done Done false
2692 1823 Acked Read 3 Sent Done Done false
2693 1824 Acked Write 2 Sent Done Done false
2694 1825 Acked Read 6 Sent Done Done false
2695 1826 Acked Write 8 Sent Done Done false
2696 1827 Acked Write 9 Sent Done Done false
2697 1828 Acked Write 2 Sent Done Done false
2698 1829 Acked Flush 0 Sent Done Done false
2699 1830 Acked Read 1 Sent Done Done false
2700 1831 Acked Read 10 Sent Done Done false
2701 1832 Acked Write 1 Sent Done Done false
2702 1833 Acked Write 7 Sent Done Done false
2703 1834 Acked Read 7 Sent Done Done false
2704 1835 Acked Write 2 Sent Done Done false
2705 1836 Acked Write 9 Sent Done Done false
2706 1837 Acked Read 4 Sent Done Done false
2707 1838 Acked Write 10 Sent Done Done false
2708 1839 Acked Flush 0 Sent Done Done false
2709 1840 Acked Write 4 Sent Done Done false
2710 1841 Acked Write 9 Sent Done Done false
2711 1842 Acked Write 3 Sent Done Done false
2712 1843 Acked Write 2 Sent Done Done false
2713 1844 Acked Read 10 Sent Done Done false
2714 1845 Acked Read 8 Sent Done Done false
2715 1846 Acked Read 5 Sent Done Done false
2716 1847 Acked Write 9 Sent Done Done false
2717 1848 Acked Write 3 Sent Done Done false
2718 1849 Acked Read 4 Sent Done Done false
2719 1850 Acked Read 5 Sent Done Done false
2720 1851 Acked Read 8 Sent Done Done false
2721 1852 Acked Write 9 Sent Done Done false
2722 1853 Acked Read 4 Sent Done Done false
2723 1854 Acked Write 6 Sent Done Done false
2724 1855 Acked Read 7 Sent Done Done false
2725 1856 Acked Write 5 Sent Done Done false
2726 1857 Acked Read 8 Sent Done Done false
2727 1858 Acked Write 1 Sent Done Done false
2728 1859 Acked Flush 0 Sent Done Done false
2729 1860 Acked Read 4 Sent Done Done false
2730 1861 Acked Read 6 Sent Done Done false
2731 1862 Acked Write 2 Sent Done Done false
2732 1863 Acked Write 2 Sent Done Done false
2733 1864 Acked Write 9 Sent Done Done false
2734 1865 Acked Write 4 Sent Done Done false
2735 1866 Acked Flush 0 Sent Done Done false
2736 1867 Acked Write 10 Sent Done Done false
2737 1868 Acked Read 1 Sent Done Done false
2738 1869 Acked Write 9 Sent Done Done false
2739 1870 Acked Write 6 Sent Done Done false
2740 1871 Acked Write 2 Sent Done Done false
2741 1872 Acked Read 5 Sent Done Done false
2742 1873 Acked Write 9 Sent Done Done false
2743 1874 Acked Flush 0 Sent Done Done false
2744 1875 Acked Read 5 Sent Done Done false
2745 1876 Acked Read 4 Sent Done Done false
2746 1877 Acked Flush 0 Sent Done Done false
2747 1878 Acked Read 4 Sent Done Done false
2748 1879 Acked Write 3 Sent Done Done false
2749 1880 Acked Flush 0 Sent Done Done false
2750 1881 Acked Read 2 Sent Done Done false
2751 1882 Acked Read 10 Sent Done Done false
2752 1883 Acked Write 1 Sent Done Done false
2753 1884 Acked Write 3 Sent Done Done false
2754 1885 Acked Write 2 Sent Done Done false
2755 1886 Acked Read 10 Sent Done Done false
2756 1887 Acked Write 3 Sent Done Done false
2757 1888 Acked Read 6 Sent Done Done false
2758 1889 Acked Write 10 Sent Done Done false
2759 1890 Acked Write 7 Sent Done Done false
2760 1891 Acked Read 3 Sent Done Done false
2761 1892 Acked Read 6 Sent Done Done false
2762 1893 Acked Write 7 Sent Done Done false
2763 1894 Acked Read 5 Sent Done Done false
2764 1895 Acked Read 1 Sent Done Done false
2765 1896 Acked Write 2 Sent Done Done false
2766 1897 Acked Read 8 Sent Done Done false
2767 1898 Acked Read 10 Sent Done Done false
2768 1899 Acked Read 1 Sent Done Done false
2769 1900 Acked Flush 0 Sent Done Done false
2770 1901 Acked Read 5 Sent Done Done false
2771 1902 Acked Write 9 Sent Done Done false
2772 1903 Acked Flush 0 Sent Done Done false
2773 1904 Acked Write 2 Sent Done Done false
2774 STATES DS:0 DS:1 DS:2 TOTAL
2775 Sent 301 0 0 301
2776 Done 0 301 301 602
2777 Skipped 0 0 0 0
2778 Error 0 0 0 0
2779 Last Flush: 1603 1903 1903
2780 Downstairs last five completed: 1603 1602 1601 1600 1599
2781 Upstairs last five completed: 1904 1903 1902 1901 1900
2782 CLIENT: Up:0 ds:301 act:2
2783 [8810] got start action
2784 [8810] I am going to restart
2785 Starting downstairs at port 8810
2786 Make output file at "/tmp/dsc/downstairs-8810.txt"
2787 [8810][0] reports Starting
2788 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1205)
2789 [8810][0] reports Running
2790 ----------------------------------------------------------------
2791 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
2792 States: Offline Active Active
2793 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2794 1604 Acked Flush 0 Sent Done Done false
2795 1605 Acked Flush 0 Sent Done Done false
2796 1606 Acked Read 1 Sent Done Done false
2797 1607 Acked Read 4 Sent Done Done false
2798 1608 Acked Read 2 Sent Done Done false
2799 1609 Acked Read 1 Sent Done Done false
2800 1610 Acked Write 2 Sent Done Done false
2801 1611 Acked Write 5 Sent Done Done false
2802 1612 Acked Write 8 Sent Done Done false
2803 1613 Acked Read 7 Sent Done Done false
2804 1614 Acked Read 3 Sent Done Done false
2805 1615 Acked Read 3 Sent Done Done false
2806 1616 Acked Write 2 Sent Done Done false
2807 1617 Acked Read 10 Sent Done Done false
2808 1618 Acked Read 8 Sent Done Done false
2809 1619 Acked Write 8 Sent Done Done false
2810 1620 Acked Write 2 Sent Done Done false
2811 1621 Acked Write 4 Sent Done Done false
2812 1622 Acked Read 7 Sent Done Done false
2813 1623 Acked Read 3 Sent Done Done false
2814 1624 Acked Read 3 Sent Done Done false
2815 1625 Acked Read 8 Sent Done Done false
2816 1626 Acked Write 5 Sent Done Done false
2817 1627 Acked Flush 0 Sent Done Done false
2818 1628 Acked Read 1 Sent Done Done false
2819 1629 Acked Write 5 Sent Done Done false
2820 1630 Acked Read 7 Sent Done Done false
2821 1631 Acked Write 2 Sent Done Done false
2822 1632 Acked Read 9 Sent Done Done false
2823 1633 Acked Write 6 Sent Done Done false
2824 1634 Acked Read 7 Sent Done Done false
2825 1635 Acked Read 2 Sent Done Done false
2826 1636 Acked Flush 0 Sent Done Done false
2827 1637 Acked Write 4 Sent Done Done false
2828 1638 Acked Write 4 Sent Done Done false
2829 1639 Acked Read 4 Sent Done Done false
2830 1640 Acked Write 8 Sent Done Done false
2831 1641 Acked Write 5 Sent Done Done false
2832 1642 Acked Read 9 Sent Done Done false
2833 1643 Acked Read 8 Sent Done Done false
2834 1644 Acked Read 5 Sent Done Done false
2835 1645 Acked Write 6 Sent Done Done false
2836 1646 Acked Read 5 Sent Done Done false
2837 1647 Acked Write 8 Sent Done Done false
2838 1648 Acked Read 6 Sent Done Done false
2839 1649 Acked Write 4 Sent Done Done false
2840 1650 Acked Write 10 Sent Done Done false
2841 1651 Acked Write 8 Sent Done Done false
2842 1652 Acked Write 4 Sent Done Done false
2843 1653 Acked Read 10 Sent Done Done false
2844 1654 Acked Write 9 Sent Done Done false
2845 1655 Acked Write 8 Sent Done Done false
2846 1656 Acked Read 5 Sent Done Done false
2847 1657 Acked Read 7 Sent Done Done false
2848 1658 Acked Write 1 Sent Done Done false
2849 1659 Acked Write 3 Sent Done Done false
2850 1660 Acked Read 9 Sent Done Done false
2851 1661 Acked Read 7 Sent Done Done false
2852 1662 Acked Read 1 Sent Done Done false
2853 1663 Acked Write 8 Sent Done Done false
2854 1664 Acked Write 9 Sent Done Done false
2855 1665 Acked Read 3 Sent Done Done false
2856 1666 Acked Write 9 Sent Done Done false
2857 1667 Acked Write 6 Sent Done Done false
2858 1668 Acked Write 4 Sent Done Done false
2859 1669 Acked Write 5 Sent Done Done false
2860 1670 Acked Read 4 Sent Done Done false
2861 1671 Acked Write 8 Sent Done Done false
2862 1672 Acked Write 3 Sent Done Done false
2863 1673 Acked Read 4 Sent Done Done false
2864 1674 Acked Write 8 Sent Done Done false
2865 1675 Acked Read 10 Sent Done Done false
2866 1676 Acked Write 6 Sent Done Done false
2867 1677 Acked Write 6 Sent Done Done false
2868 1678 Acked Write 8 Sent Done Done false
2869 1679 Acked Read 5 Sent Done Done false
2870 1680 Acked Read 10 Sent Done Done false
2871 1681 Acked Write 2 Sent Done Done false
2872 1682 Acked Read 2 Sent Done Done false
2873 1683 Acked Write 5 Sent Done Done false
2874 1684 Acked Read 6 Sent Done Done false
2875 1685 Acked Write 1 Sent Done Done false
2876 1686 Acked Write 2 Sent Done Done false
2877 1687 Acked Write 4 Sent Done Done false
2878 1688 Acked Read 3 Sent Done Done false
2879 1689 Acked Read 7 Sent Done Done false
2880 1690 Acked Write 8 Sent Done Done false
2881 1691 Acked Read 7 Sent Done Done false
2882 1692 Acked Read 1 Sent Done Done false
2883 1693 Acked Read 3 Sent Done Done false
2884 1694 Acked Write 8 Sent Done Done false
2885 1695 Acked Write 9 Sent Done Done false
2886 1696 Acked Read 5 Sent Done Done false
2887 1697 Acked Flush 0 Sent Done Done false
2888 1698 Acked Read 5 Sent Done Done false
2889 1699 Acked Write 3 Sent Done Done false
2890 1700 Acked Read 10 Sent Done Done false
2891 1701 Acked Read 9 Sent Done Done false
2892 1702 Acked Write 5 Sent Done Done false
2893 1703 Acked Read 7 Sent Done Done false
2894 1704 Acked Write 9 Sent Done Done false
2895 1705 Acked Read 10 Sent Done Done false
2896 1706 Acked Read 9 Sent Done Done false
2897 1707 Acked Read 7 Sent Done Done false
2898 1708 Acked Read 9 Sent Done Done false
2899 1709 Acked Read 6 Sent Done Done false
2900 1710 Acked Read 8 Sent Done Done false
2901 1711 Acked Write 5 Sent Done Done false
2902 1712 Acked Read 3 Sent Done Done false
2903 1713 Acked Write 10 Sent Done Done false
2904 1714 Acked Read 1 Sent Done Done false
2905 1715 Acked Write 9 Sent Done Done false
2906 1716 Acked Read 10 Sent Done Done false
2907 1717 Acked Write 3 Sent Done Done false
2908 1718 Acked Read 6 Sent Done Done false
2909 1719 Acked Read 3 Sent Done Done false
2910 1720 Acked Write 5 Sent Done Done false
2911 1721 Acked Read 2 Sent Done Done false
2912 1722 Acked Read 7 Sent Done Done false
2913 1723 Acked Flush 0 Sent Done Done false
2914 1724 Acked Read 8 Sent Done Done false
2915 1725 Acked Write 8 Sent Done Done false
2916 1726 Acked Read 6 Sent Done Done false
2917 1727 Acked Read 4 Sent Done Done false
2918 1728 Acked Write 10 Sent Done Done false
2919 1729 Acked Write 2 Sent Done Done false
2920 1730 Acked Flush 0 Sent Done Done false
2921 1731 Acked Write 3 Sent Done Done false
2922 1732 Acked Read 2 Sent Done Done false
2923 1733 Acked Read 7 Sent Done Done false
2924 1734 Acked Read 1 Sent Done Done false
2925 1735 Acked Write 3 Sent Done Done false
2926 1736 Acked Flush 0 Sent Done Done false
2927 1737 Acked Read 7 Sent Done Done false
2928 1738 Acked Read 5 Sent Done Done false
2929 1739 Acked Read 2 Sent Done Done false
2930 1740 Acked Read 10 Sent Done Done false
2931 1741 Acked Read 3 Sent Done Done false
2932 1742 Acked Read 1 Sent Done Done false
2933 1743 Acked Write 6 Sent Done Done false
2934 1744 Acked Read 3 Sent Done Done false
2935 1745 Acked Read 5 Sent Done Done false
2936 1746 Acked Write 8 Sent Done Done false
2937 1747 Acked Read 1 Sent Done Done false
2938 1748 Acked Write 4 Sent Done Done false
2939 1749 Acked Read 6 Sent Done Done false
2940 1750 Acked Read 9 Sent Done Done false
2941 1751 Acked Read 7 Sent Done Done false
2942 1752 Acked Read 7 Sent Done Done false
2943 1753 Acked Write 6 Sent Done Done false
2944 1754 Acked Read 9 Sent Done Done false
2945 1755 Acked Write 2 Sent Done Done false
2946 1756 Acked Write 2 Sent Done Done false
2947 1757 Acked Write 2 Sent Done Done false
2948 1758 Acked Read 9 Sent Done Done false
2949 1759 Acked Flush 0 Sent Done Done false
2950 1760 Acked Read 3 Sent Done Done false
2951 1761 Acked Write 6 Sent Done Done false
2952 1762 Acked Read 8 Sent Done Done false
2953 1763 Acked Write 6 Sent Done Done false
2954 1764 Acked Write 6 Sent Done Done false
2955 1765 Acked Write 8 Sent Done Done false
2956 1766 Acked Read 6 Sent Done Done false
2957 1767 Acked Read 2 Sent Done Done false
2958 1768 Acked Write 4 Sent Done Done false
2959 1769 Acked Write 4 Sent Done Done false
2960 1770 Acked Flush 0 Sent Done Done false
2961 1771 Acked Read 10 Sent Done Done false
2962 1772 Acked Read 6 Sent Done Done false
2963 1773 Acked Read 2 Sent Done Done false
2964 1774 Acked Write 8 Sent Done Done false
2965 1775 Acked Write 3 Sent Done Done false
2966 1776 Acked Read 3 Sent Done Done false
2967 1777 Acked Write 8 Sent Done Done false
2968 1778 Acked Write 10 Sent Done Done false
2969 1779 Acked Read 4 Sent Done Done false
2970 1780 Acked Read 7 Sent Done Done false
2971 1781 Acked Write 9 Sent Done Done false
2972 1782 Acked Read 10 Sent Done Done false
2973 1783 Acked Read 1 Sent Done Done false
2974 1784 Acked Flush 0 Sent Done Done false
2975 1785 Acked Read 6 Sent Done Done false
2976 1786 Acked Read 3 Sent Done Done false
2977 1787 Acked Write 6 Sent Done Done false
2978 1788 Acked Read 2 Sent Done Done false
2979 1789 Acked Write 6 Sent Done Done false
2980 1790 Acked Write 9 Sent Done Done false
2981 1791 Acked Write 9 Sent Done Done false
2982 1792 Acked Write 9 Sent Done Done false
2983 1793 Acked Read 4 Sent Done Done false
2984 1794 Acked Read 4 Sent Done Done false
2985 1795 Acked Write 7 Sent Done Done false
2986 1796 Acked Write 6 Sent Done Done false
2987 1797 Acked Read 2 Sent Done Done false
2988 1798 Acked Read 1 Sent Done Done false
2989 1799 Acked Read 4 Sent Done Done false
2990 1800 Acked Write 10 Sent Done Done false
2991 1801 Acked Write 4 Sent Done Done false
2992 1802 Acked Read 5 Sent Done Done false
2993 1803 Acked Write 6 Sent Done Done false
2994 1804 Acked Read 4 Sent Done Done false
2995 1805 Acked Read 5 Sent Done Done false
2996 1806 Acked Write 8 Sent Done Done false
2997 1807 Acked Write 9 Sent Done Done false
2998 1808 Acked Read 4 Sent Done Done false
2999 1809 Acked Read 2 Sent Done Done false
3000 1810 Acked Write 3 Sent Done Done false
3001 1811 Acked Read 10 Sent Done Done false
3002 1812 Acked Read 8 Sent Done Done false
3003 1813 Acked Write 10 Sent Done Done false
3004 1814 Acked Write 7 Sent Done Done false
3005 1815 Acked Read 4 Sent Done Done false
3006 1816 Acked Flush 0 Sent Done Done false
3007 1817 Acked Read 1 Sent Done Done false
3008 1818 Acked Read 3 Sent Done Done false
3009 1819 Acked Write 4 Sent Done Done false
3010 1820 Acked Read 2 Sent Done Done false
3011 1821 Acked Read 5 Sent Done Done false
3012 1822 Acked Write 1 Sent Done Done false
3013 1823 Acked Read 3 Sent Done Done false
3014 1824 Acked Write 2 Sent Done Done false
3015 1825 Acked Read 6 Sent Done Done false
3016 1826 Acked Write 8 Sent Done Done false
3017 1827 Acked Write 9 Sent Done Done false
3018 1828 Acked Write 2 Sent Done Done false
3019 1829 Acked Flush 0 Sent Done Done false
3020 1830 Acked Read 1 Sent Done Done false
3021 1831 Acked Read 10 Sent Done Done false
3022 1832 Acked Write 1 Sent Done Done false
3023 1833 Acked Write 7 Sent Done Done false
3024 1834 Acked Read 7 Sent Done Done false
3025 1835 Acked Write 2 Sent Done Done false
3026 1836 Acked Write 9 Sent Done Done false
3027 1837 Acked Read 4 Sent Done Done false
3028 1838 Acked Write 10 Sent Done Done false
3029 1839 Acked Flush 0 Sent Done Done false
3030 1840 Acked Write 4 Sent Done Done false
3031 1841 Acked Write 9 Sent Done Done false
3032 1842 Acked Write 3 Sent Done Done false
3033 1843 Acked Write 2 Sent Done Done false
3034 1844 Acked Read 10 Sent Done Done false
3035 1845 Acked Read 8 Sent Done Done false
3036 1846 Acked Read 5 Sent Done Done false
3037 1847 Acked Write 9 Sent Done Done false
3038 1848 Acked Write 3 Sent Done Done false
3039 1849 Acked Read 4 Sent Done Done false
3040 1850 Acked Read 5 Sent Done Done false
3041 1851 Acked Read 8 Sent Done Done false
3042 1852 Acked Write 9 Sent Done Done false
3043 1853 Acked Read 4 Sent Done Done false
3044 1854 Acked Write 6 Sent Done Done false
3045 1855 Acked Read 7 Sent Done Done false
3046 1856 Acked Write 5 Sent Done Done false
3047 1857 Acked Read 8 Sent Done Done false
3048 1858 Acked Write 1 Sent Done Done false
3049 1859 Acked Flush 0 Sent Done Done false
3050 1860 Acked Read 4 Sent Done Done false
3051 1861 Acked Read 6 Sent Done Done false
3052 1862 Acked Write 2 Sent Done Done false
3053 1863 Acked Write 2 Sent Done Done false
3054 1864 Acked Write 9 Sent Done Done false
3055 1865 Acked Write 4 Sent Done Done false
3056 1866 Acked Flush 0 Sent Done Done false
3057 1867 Acked Write 10 Sent Done Done false
3058 1868 Acked Read 1 Sent Done Done false
3059 1869 Acked Write 9 Sent Done Done false
3060 1870 Acked Write 6 Sent Done Done false
3061 1871 Acked Write 2 Sent Done Done false
3062 1872 Acked Read 5 Sent Done Done false
3063 1873 Acked Write 9 Sent Done Done false
3064 1874 Acked Flush 0 Sent Done Done false
3065 1875 Acked Read 5 Sent Done Done false
3066 1876 Acked Read 4 Sent Done Done false
3067 1877 Acked Flush 0 Sent Done Done false
3068 1878 Acked Read 4 Sent Done Done false
3069 1879 Acked Write 3 Sent Done Done false
3070 1880 Acked Flush 0 Sent Done Done false
3071 1881 Acked Read 2 Sent Done Done false
3072 1882 Acked Read 10 Sent Done Done false
3073 1883 Acked Write 1 Sent Done Done false
3074 1884 Acked Write 3 Sent Done Done false
3075 1885 Acked Write 2 Sent Done Done false
3076 1886 Acked Read 10 Sent Done Done false
3077 1887 Acked Write 3 Sent Done Done false
3078 1888 Acked Read 6 Sent Done Done false
3079 1889 Acked Write 10 Sent Done Done false
3080 1890 Acked Write 7 Sent Done Done false
3081 1891 Acked Read 3 Sent Done Done false
3082 1892 Acked Read 6 Sent Done Done false
3083 1893 Acked Write 7 Sent Done Done false
3084 1894 Acked Read 5 Sent Done Done false
3085 1895 Acked Read 1 Sent Done Done false
3086 1896 Acked Write 2 Sent Done Done false
3087 1897 Acked Read 8 Sent Done Done false
3088 1898 Acked Read 10 Sent Done Done false
3089 1899 Acked Read 1 Sent Done Done false
3090 1900 Acked Flush 0 Sent Done Done false
3091 1901 Acked Read 5 Sent Done Done false
3092 1902 Acked Write 9 Sent Done Done false
3093 1903 Acked Flush 0 Sent Done Done false
3094 1904 Acked Write 2 Sent Done Done false
3095 1905 Acked Flush 0 Sent Done Done false
3096 STATES DS:0 DS:1 DS:2 TOTAL
3097 Sent 302 0 0 302
3098 Done 0 302 302 604
3099 Skipped 0 0 0 0
3100 Error 0 0 0 0
3101 Last Flush: 1603 1905 1905
3102 Downstairs last five completed: 1603 1602 1601 1600 1599
3103 Upstairs last five completed: 1905 1904 1903 1902 1901
3104 CLIENT: Up:0 ds:302 act:2
31052025-01-13T15:52:25.541ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
31062025-01-13T15:52:25.542ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61879) = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
31072025-01-13T15:52:25.543ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
31082025-01-13T15:52:25.543ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
31092025-01-13T15:52:25.543ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
31102025-01-13T15:52:25.543ZINFOcrucible: send last flush ID to this DS: 1603 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
31112025-01-13T15:52:25.544ZINFOcrucible: Replied this last flush ID: 1603 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
31122025-01-13T15:52:25.544ZINFOcrucible: [0] client re-new 302 jobs since flush 1603 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
31132025-01-13T15:52:25.544ZINFOcrucible: [0] Replayed 302 jobs since flush: 1603 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
3114 ----------------------------------------------------------------
3115 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
3116 Downstairs last five completed: 1905 1904 1903 1902 1901
3117 Upstairs last five completed: 1905 1904 1903 1902 1901
3118 CLIENT: Up:0 ds:0 act:3
3119 Replay: All jobs finished
3120 Main task has work to do, go find it
3121 got dsc Stop(2)
3122 stop 2
3123 Jan 13 15:52:27.787 INFO request completed, latency_us[8830] Got stop action so:false kr:false
3124 : 358, response_code: 204, uri: /stop/cid/2, method: POST, req_id: feba2fa5-0041-40d5-bca7-db827133d8eb, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
31252025-01-13T15:52:27.787ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
31262025-01-13T15:52:27.788ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
31272025-01-13T15:52:27.788ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
31282025-01-13T15:52:27.788ZINFOcrucible: client task is exiting = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
3129 Jan 13 15:52:27.788 INFO request completed, latency_us: 165, response_code: 200, uri: /state/cid/2, method: GET, req_id: 45b9b224-eef4-4f10-aaa2-fe0f7d632876, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
3130 [8830] Exited with: ExitStatus(unix_wait_status(9))
3131 [8830][2] reports Exit
3132 Jan 13 15:52:31.790 INFO request completed, latency_us: 397, response_code: 200, uri: /state/cid/2, method: GET, req_id: 2e5bd26f-bcb1-44ea-b068-effbe596f08d, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
3133 001/300 Read block 1156 len 8192
3134 002/300 Read block 1051 len 36864
3135 003/300 Write block 4532 len 20480 data: 3 2 2 2 2
3136 004/300 Write block 2570 len 12288 data: 2 2 2
3137 005/300 Read block 2321 len 36864
3138 006/300 Write block 4470 len 40960 data: 2 2 2 2 2 2 2 3 3 3
3139 007/300 Write block 1176 len 12288 data: 2 3 3
3140 008/300 Read block 475 len 16384
3141 009/300 Write block 295 len 4096 data: 2
3142 010/300 Write block 1658 len 32768 data: 2 2 2 2 2 2 2 2
3143 011/300 Read block 865 len 20480
3144 012/300 Write block 3121 len 32768 data: 2 2 2 2 2 2 2 2
3145 013/300 Read block 2388 len 16384
3146 014/300 Read block 3353 len 28672
3147 015/300 Write block 3227 len 4096 data: 3
3148 016/300 Write block 2546 len 24576 data: 4 4 4 4 3 3
3149 017/300 Read block 4789 len 28672
3150 018/300 Write block 200 len 28672 data: 3 3 5 5 5 5 4
3151 019/300 Read block 1281 len 32768
3152 020/300 Read block 2128 len 4096
3153 021/300 Flush
3154 022/300 Read block 3565 len 24576
3155 023/300 Read block 3139 len 32768
3156 024/300 Write block 2636 len 8192 data: 2 2
3157 025/300 Read block 4146 len 36864
3158 026/300 Read block 3279 len 12288
3159 027/300 Flush
3160 028/300 Flush
3161 029/300 Write block 3273 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3162 030/300 Read block 4847 len 28672
3163 031/300 Write block 213 len 12288 data: 3 3 3
3164 032/300 Read block 1198 len 12288
3165 033/300 Read block 1658 len 8192
3166 034/300 Read block 263 len 12288
3167 035/300 Read block 1067 len 32768
3168 036/300 Write block 4681 len 20480 data: 2 2 2 3 3
3169 037/300 Read block 4381 len 12288
3170 038/300 Read block 219 len 36864
3171 039/300 Write block 670 len 4096 data: 3
3172 040/300 Write block 4105 len 32768 data: 2 3 3 3 2 2 2 3
3173 041/300 Write block 834 len 4096 data: 2
3174 042/300 Write block 3075 len 24576 data: 2 3 3 3 3 3
3175 043/300 Read block 4116 len 4096
3176 044/300 Write block 2181 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3177 045/300 Write block 3136 len 4096 data: 2
3178 046/300 Read block 1059 len 40960
3179 047/300 Write block 137 len 8192 data: 2 2
3180 048/300 Read block 2322 len 8192
3181 049/300 Read block 680 len 16384
3182 050/300 Read block 2936 len 4096
3183 051/300 Read block 3561 len 20480
3184 052/300 Write block 4241 len 24576 data: 2 2 2 2 2 2
3185 053/300 Read block 971 len 12288
3186 054/300 Write block 520 len 32768 data: 3 2 3 3 3 3 2 3
3187 055/300 Write block 1297 len 32768 data: 2 2 2 2 2 2 2 2
3188 056/300 Read block 3842 len 32768
3189 057/300 Flush
3190 058/300 Write block 4378 len 4096 data: 2
3191 059/300 Read block 4600 len 16384
3192 060/300 Read block 963 len 36864
3193 061/300 Flush
3194 062/300 Write block 942 len 40960 data: 3 3 4 4 4 4 4 4 3 3
3195 063/300 Write block 880 len 12288 data: 2 2 2
3196 064/300 Write block 3390 len 36864 data: 5 3 3 4 4 4 4 3 2
3197 065/300 Read block 2639 len 28672
3198 066/300 Read block 736 len 36864
3199 067/300 Flush
3200 068/300 Read block 1594 len 24576
3201 069/300 Write block 398 len 12288 data: 3 3 3
3202 070/300 Flush
3203 071/300 Write block 2271 len 32768 data: 3 2 2 2 2 2 3 3
3204 072/300 Write block 4624 len 16384 data: 3 3 3 3
3205 073/300 Write block 4514 len 24576 data: 3 3 3 3 3 4
3206 074/300 Read block 1424 len 36864
3207 075/300 Read block 3299 len 20480
3208 076/300 Read block 4290 len 12288
3209 077/300 Write block 3896 len 36864 data: 3 3 3 3 2 2 2 2 2
3210 078/300 Read block 1354 len 36864
3211 079/300 Read block 2239 len 40960
3212 080/300 Read block 1809 len 12288
3213 081/300 Read block 3620 len 36864
3214 082/300 Write block 4749 len 36864 data: 3 3 3 3 3 3 3 3 2
3215 083/300 Flush
3216 084/300 Read block 4106 len 16384
3217 085/300 Read block 202 len 20480
3218 086/300 Write block 477 len 4096 data: 3
3219 087/300 Read block 3420 len 36864
3220 088/300 Write block 1504 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3221 089/300 Flush
3222 090/300 Read block 4456 len 32768
3223 091/300 Read block 4262 len 12288
3224 092/300 Read block 2235 len 16384
3225 093/300 Write block 4841 len 28672 data: 2 2 2 2 2 2 2
3226 094/300 Read block 1612 len 16384
3227 095/300 Read block 4642 len 24576
3228 096/300 Read block 4576 len 12288
3229 097/300 Write block 2120 len 36864 data: 3 3 3 2 2 2 2 2 2
3230 098/300 Read block 3778 len 28672
3231 099/300 Read block 3437 len 4096
3232 100/300 Read block 1382 len 16384
3233 101/300 Read block 4438 len 8192
3234 102/300 Flush
3235 103/300 Read block 4690 len 32768
3236 104/300 Read block 454 len 36864
3237 105/300 Read block 2632 len 4096
3238 106/300 Write block 4506 len 28672 data: 2 2 2 2 3 3 3
3239 107/300 Flush
3240 108/300 Read block 2054 len 36864
3241 109/300 Write block 1039 len 28672 data: 2 2 2 2 2 3 3
3242 110/300 Flush
3243 111/300 Read block 1218 len 28672
3244 112/300 Write block 1293 len 32768 data: 4 4 4 4 3 3 3 3
3245 113/300 Write block 285 len 12288 data: 2 2 2
3246 114/300 Write block 3462 len 8192 data: 3 3
3247 115/300 Write block 510 len 28672 data: 2 3 4 4 4 4 4
3248 116/300 Write block 328 len 12288 data: 2 2 2
3249 117/300 Read block 4906 len 28672
3250 118/300 Read block 2502 len 36864
3251 119/300 Write block 668 len 8192 data: 4 4
3252 120/300 Read block 172 len 16384
3253 121/300 Read block 437 len 24576
3254 122/300 Read block 345 len 28672
3255 123/300 Write block 1522 len 36864 data: 2 3 3 3 3 4 4 4 4
3256 124/300 Read block 417 len 4096
3257 125/300 Write block 977 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3258 126/300 Flush
3259 127/300 Write block 2443 len 24576 data: 3 3 2 2 2 2
3260 128/300 Read block 2579 len 20480
3261 129/300 Read block 1529 len 24576
3262 130/300 Read block 1381 len 36864
3263 131/300 Read block 49 len 40960
3264 132/300 Write block 1933 len 4096 data: 2
3265 133/300 Write block 2214 len 28672 data: 2 2 3 3 3 3 3
3266 134/300 Write block 2908 len 16384 data: 3 3 2 2
3267 135/300 Write block 2101 len 4096 data: 3
3268 136/300 Read block 3963 len 8192
3269 137/300 Write block 2413 len 4096 data: 2
3270 138/300 Write block 1749 len 16384 data: 3 3 3 3
3271 139/300 Write block 1498 len 28672 data: 2 2 2 2 2 2 3
3272 140/300 Write block 874 len 12288 data: 2 2 2
3273 141/300 Read block 2972 len 40960
3274 142/300 Read block 4515 len 16384
3275 143/300 Read block 1141 len 24576
3276 144/300 Read block 229 len 4096
3277 145/300 Write block 3326 len 16384 data: 3 3 3 3
3278 146/300 Write block 3439 len 20480 data: 2 2 2 2 2
3279 147/300 Read block 3298 len 28672
3280 148/300 Read block 3375 len 12288
3281 149/300 Read block 1764 len 16384
3282 150/300 Read block 3730 len 36864
3283 151/300 Write block 661 len 40960 data: 3 4 4 5 5 5 4 5 5 4
3284 152/300 Read block 3583 len 36864
3285 153/300 Flush
3286 154/300 Read block 754 len 40960
3287 155/300 Read block 3272 len 40960
3288 156/300 Write block 1984 len 20480 data: 2 2 2 2 2
3289 157/300 Read block 413 len 8192
3290 158/300 Write block 962 len 16384 data: 2 2 2 2
3291 159/300 Read block 1997 len 28672
3292 160/300 Write block 127 len 40960 data: 3 3 3 3 3 3 2 2 2 2
3293 161/300 Read block 4863 len 24576
3294 162/300 Read block 1378 len 20480
3295 163/300 Write block 3810 len 16384 data: 2 2 2 2
3296 164/300 Read block 3811 len 8192
3297 165/300 Read block 1842 len 12288
3298 166/300 Read block 4715 len 16384
3299 167/300 Read block 3604 len 28672
3300 168/300 Read block 4144 len 28672
3301 169/300 Read block 2011 len 16384
3302 170/300 Write block 4364 len 24576 data: 2 2 2 2 2 2
3303 171/300 Write block 397 len 36864 data: 2 4 4 4 3 3 3 2 2
3304 172/300 Write block 3917 len 12288 data: 2 2 2
3305 173/300 Write block 4751 len 8192 data: 4 4
3306 174/300 Write block 3067 len 40960 data: 3 3 3 3 3 3 3 3 3 4
3307 175/300 Write block 1538 len 12288 data: 2 2 2
3308 176/300 Write block 1616 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3309 177/300 Write block 4215 len 12288 data: 3 3 2
3310 178/300 Flush
3311 179/300 Read block 4084 len 40960
3312 180/300 Read block 3412 len 20480
3313 181/300 Write block 3921 len 16384 data: 2 2 2 2
3314 182/300 Read block 681 len 32768
3315 183/300 Read block 880 len 36864
3316 184/300 Flush
3317 185/300 Write block 765 len 28672 data: 2 2 2 2 2 2 3
3318 186/300 Read block 92 len 16384
3319 187/300 Write block 4213 len 36864 data: 3 3 4 4 3 2 2 2 2
3320 188/300 Write block 2973 len 20480 data: 2 2 3 3 3
3321 189/300 Write block 3219 len 16384 data: 2 2 3 3
3322 190/300 Flush
3323 191/300 Read block 2590 len 16384
3324 192/300 Read block 4966 len 20480
3325 193/300 Write block 4343 len 32768 data: 3 3 4 3 3 3 2 2
3326 194/300 Read block 4821 len 8192
3327 195/300 Read block 3258 len 28672
3328 196/300 Read block 345 len 36864
3329 197/300 Read block 3865 len 40960
3330 198/300 Read block 1600 len 40960
3331 199/300 Write block 1079 len 40960 data: 2 2 2 2 2 2 2 2 2 3
3332 200/300 Write block 3516 len 16384 data: 2 2 2 2
3333 201/300 Write block 869 len 16384 data: 2 2 2 2
3334 202/300 Read block 490 len 20480
3335 203/300 Read block 1390 len 4096
3336 204/300 Read block 3836 len 8192
3337 205/300 Read block 1416 len 16384
3338 206/300 Write block 2901 len 8192 data: 2 2
3339 207/300 Read block 2090 len 20480
3340 208/300 Write block 3702 len 4096 data: 2
3341 209/300 Write block 3539 len 4096 data: 2
3342 210/300 Write block 2046 len 36864 data: 2 2 2 2 2 2 2 2 2
3343 211/300 Write block 2981 len 16384 data: 2 2 2 2
3344 212/300 Read block 4276 len 32768
3345 213/300 Write block 532 len 20480 data: 3 2 2 2 2
3346 214/300 Read block 2413 len 20480
3347 215/300 Read block 3348 len 36864
3348 216/300 Flush
3349 217/300 Read block 1349 len 24576
3350 218/300 Write block 2908 len 12288 data: 4 4 3
3351 219/300 Write block 2002 len 36864 data: 2 2 2 2 2 2 2 2 2
3352 220/300 Write block 2800 len 20480 data: 4 3 3 3 2
3353 221/300 Read block 2140 len 20480
3354 222/300 Read block 2541 len 4096
3355 223/300 Read block 1799 len 4096
3356 224/300 Read block 4127 len 20480
3357 225/300 Write block 4835 len 20480 data: 2 2 2 2 2
3358 226/300 Read block 1319 len 36864
3359 227/300 Write block 3815 len 20480 data: 2 2 2 2 2
3360 228/300 Write block 2342 len 32768 data: 4 4 4 4 3 3 3 3
3361 229/300 Write block 2026 len 12288 data: 2 3 4
3362 230/300 Write block 2949 len 4096 data: 3
3363 231/300 Read block 4150 len 28672
3364 232/300 Read block 3336 len 20480
3365 233/300 Flush
3366 234/300 Write block 2480 len 20480 data: 4 4 4 4 4
3367 235/300 Write block 3406 len 24576 data: 3 3 3 3 3 3
3368 236/300 Read block 4416 len 20480
3369 237/300 Write block 1574 len 24576 data: 2 2 2 2 2 3
3370 238/300 Read block 4468 len 32768
3371 239/300 Write block 3809 len 24576 data: 2 3 3 3 3 2
3372 240/300 Read block 2237 len 20480
3373 241/300 Write block 2405 len 12288 data: 2 2 2
3374 242/300 Read block 4296 len 40960
3375 243/300 Write block 693 len 20480 data: 2 2 2 2 2
3376 244/300 Read block 1030 len 20480
3377 245/300 Read block 3999 len 32768
3378 246/300 Write block 2701 len 24576 data: 3 3 3 3 3 3
3379 247/300 Read block 47 len 4096
3380 248/300 Read block 1054 len 32768
3381 249/300 Read block 2089 len 16384
3382 250/300 Write block 3514 len 20480 data: 2 2 3 3 3
3383 251/300 Read block 3320 len 40960
3384 252/300 Read block 1638 len 20480
3385 253/300 Write block 2210 len 4096 data: 2
3386 254/300 Read block 948 len 12288
3387 255/300 Read block 3622 len 12288
3388 256/300 Write block 4302 len 28672 data: 3 3 2 2 2 2 2
3389 257/300 Read block 3175 len 8192
3390 258/300 Write block 1022 len 28672 data: 2 2 2 2 2 2 2
3391 259/300 Read block 868 len 36864
3392 260/300 Read block 3146 len 20480
3393 261/300 Write block 2171 len 28672 data: 3 3 3 2 2 2 2
3394 262/300 Write block 3708 len 8192 data: 2 2
3395 263/300 Read block 751 len 4096
3396 264/300 Write block 1742 len 20480 data: 4 5 4 5 4
3397 265/300 Read block 1796 len 12288
3398 266/300 Read block 1512 len 16384
3399 267/300 Read block 141 len 40960
3400 268/300 Read block 419 len 8192
3401 269/300 Read block 1324 len 4096
3402 270/300 Read block 1058 len 16384
3403 271/300 Write block 1797 len 8192 data: 2 2
3404 272/300 Write block 2026 len 36864 data: 3 4 5 4 4 4 3 3 3
3405 273/300 Write block 733 len 40960 data: 3 2 2 2 2 2 2 2 2 2
3406 274/300 Flush
3407 275/300 Write block 3209 len 24576 data: 2 2 2 2 2 2
3408 276/300 Read block 4256 len 12288
3409 277/300 Write block 649 len 16384 data: 2 2 2 2
3410 278/300 Read block 242 len 28672
3411 279/300 Read block 173 len 28672
3412 280/300 Flush
3413 281/300 Read block 105 len 20480
3414 282/300 Read block 4351 len 4096
3415 283/300 Read block 209 len 4096
3416 284/300 Write block 2966 len 28672 data: 2 2 2 2 2 2 2
3417 285/300 Read block 2676 len 8192
3418 286/300 Read block 4385 len 8192
3419 287/300 Read block 1774 len 8192
3420 288/300 Write block 1234 len 28672 data: 2 2 2 2 2 2 2
3421 289/300 Write block 2363 len 32768 data: 2 2 2 2 2 2 2 2
3422 290/300 Read block 4237 len 16384
3423 291/300 Read block 2315 len 8192
3424 292/300 Write block 4107 len 32768 data: 4 4 3 3 3 4 3 3
3425 293/300 Read block 619 len 36864
3426 294/300 Write block 4448 len 8192 data: 3 3
3427 295/300 Read block 1619 len 24576
3428 296/300 Write block 2138 len 24576 data: 2 2 2 2 2 2
3429 297/300 Write block 4495 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3430 298/300 Read block 4545 len 20480
3431 299/300 Read block 1534 len 4096
3432 300/300 Read block 4054 len 24576
3433 301/300 Write block 1332 len 28672 data: 2 2 3 4 5 5 4
3434 Main task has work to do, go find it
3435 got dsc Start(2)
3436 start 2
3437 Jan 13 15:52:32.034 INFO request completed, latency_us: 273, response_code: 204, uri: /start/cid/2, method: POST, req_id: 3d3377b3-847c-49ec-aadd-29c48b0a53c8, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
3438 Replay: started 2, returned:Ok(())
3439 ----------------------------------------------------------------
3440 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
3441 States: Active Active Offline
3442 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3443 1906 Acked Read 2 Done Done Sent false
3444 1907 Acked Read 9 Done Done Sent false
3445 1908 Acked Write 5 Done Done Sent false
3446 1909 Acked Write 3 Done Done Sent false
3447 1910 Acked Read 9 Done Done Sent false
3448 1911 Acked Write 10 Done Done Sent false
3449 1912 Acked Write 3 Done Done Sent false
3450 1913 Acked Read 4 Done Done Sent false
3451 1914 Acked Write 1 Done Done Sent false
3452 1915 Acked Write 8 Done Done Sent false
3453 1916 Acked Read 5 Done Done Sent false
3454 1917 Acked Write 8 Done Done Sent false
3455 1918 Acked Read 4 Done Done Sent false
3456 1919 Acked Read 7 Done Done Sent false
3457 1920 Acked Write 1 Done Done Sent false
3458 1921 Acked Write 6 Done Done Sent false
3459 1922 Acked Read 7 Done Done Sent false
3460 1923 Acked Write 7 Done Done Sent false
3461 1924 Acked Read 8 Done Done Sent false
3462 1925 Acked Read 1 Done Done Sent false
3463 1926 Acked Flush 0 Done Done Sent false
3464 1927 Acked Read 6 Done Done Sent false
3465 1928 Acked Read 8 Done Done Sent false
3466 1929 Acked Write 2 Done Done Sent false
3467 1930 Acked Read 9 Done Done Sent false
3468 1931 Acked Read 3 Done Done Sent false
3469 1932 Acked Flush 0 Done Done Sent false
3470 1933 Acked Flush 0 Done Done Sent false
3471 1934 Acked Write 10 Done Done Sent false
3472 1935 Acked Read 7 Done Done Sent false
3473 1936 Acked Write 3 Done Done Sent false
3474 1937 Acked Read 3 Done Done Sent false
3475 1938 Acked Read 2 Done Done Sent false
3476 1939 Acked Read 3 Done Done Sent false
3477 1940 Acked Read 8 Done Done Sent false
3478 1941 Acked Write 5 Done Done Sent false
3479 1942 Acked Read 3 Done Done Sent false
3480 1943 Acked Read 9 Done Done Sent false
3481 1944 Acked Write 1 Done Done Sent false
3482 1945 Acked Write 8 Done Done Sent false
3483 1946 Acked Write 1 Done Done Sent false
3484 1947 Acked Write 6 Done Done Sent false
3485 1948 Acked Read 1 Done Done Sent false
3486 1949 Acked Write 10 Done Done Sent false
3487 1950 Acked Write 1 Done Done Sent false
3488 1951 Acked Read 10 Done Done Sent false
3489 1952 Acked Write 2 Done Done Sent false
3490 1953 Acked Read 2 Done Done Sent false
3491 1954 Acked Read 4 Done Done Sent false
3492 1955 Acked Read 1 Done Done Sent false
3493 1956 Acked Read 5 Done Done Sent false
3494 1957 Acked Write 6 Done Done Sent false
3495 1958 Acked Read 3 Done Done Sent false
3496 1959 Acked Write 8 Done Done Sent false
3497 1960 Acked Write 8 Done Done Sent false
3498 1961 Acked Read 8 Done Done Sent false
3499 1962 Acked Flush 0 Done Done Sent false
3500 1963 Acked Write 1 Done Done Sent false
3501 1964 Acked Read 4 Done Done Sent false
3502 1965 Acked Read 9 Done Done Sent false
3503 1966 Acked Flush 0 Done Done Sent false
3504 1967 Acked Write 10 Done Done Sent false
3505 1968 Acked Write 3 Done Done Sent false
3506 1969 Acked Write 9 Done Done Sent false
3507 1970 Acked Read 7 Done Done Sent false
3508 1971 Acked Read 9 Done Done Sent false
3509 1972 Acked Flush 0 Done Done Sent false
3510 1973 Acked Read 6 Done Done Sent false
3511 1974 Acked Write 3 Done Done Sent false
3512 1975 Acked Flush 0 Done Done Sent false
3513 1976 Acked Write 8 Done Done Sent false
3514 1977 Acked Write 4 Done Done Sent false
3515 1978 Acked Write 6 Done Done Sent false
3516 1979 Acked Read 9 Done Done Sent false
3517 1980 Acked Read 5 Done Done Sent false
3518 1981 Acked Read 3 Done Done Sent false
3519 1982 Acked Write 9 Done Done Sent false
3520 1983 Acked Read 9 Done Done Sent false
3521 1984 Acked Read 10 Done Done Sent false
3522 1985 Acked Read 3 Done Done Sent false
3523 1986 Acked Read 9 Done Done Sent false
3524 1987 Acked Write 9 Done Done Sent false
3525 1988 Acked Flush 0 Done Done Sent false
3526 1989 Acked Read 4 Done Done Sent false
3527 1990 Acked Read 5 Done Done Sent false
3528 1991 Acked Write 1 Done Done Sent false
3529 1992 Acked Read 9 Done Done Sent false
3530 1993 Acked Write 10 Done Done Sent false
3531 1994 Acked Flush 0 Done Done Sent false
3532 1995 Acked Read 8 Done Done Sent false
3533 1996 Acked Read 3 Done Done Sent false
3534 1997 Acked Read 4 Done Done Sent false
3535 1998 Acked Write 7 Done Done Sent false
3536 1999 Acked Read 4 Done Done Sent false
3537 2000 Acked Read 6 Done Done Sent false
3538 2001 Acked Read 3 Done Done Sent false
3539 2002 Acked Write 9 Done Done Sent false
3540 2003 Acked Read 7 Done Done Sent false
3541 2004 Acked Read 1 Done Done Sent false
3542 2005 Acked Read 4 Done Done Sent false
3543 2006 Acked Read 2 Done Done Sent false
3544 2007 Acked Flush 0 Done Done Sent false
3545 2008 Acked Read 8 Done Done Sent false
3546 2009 Acked Read 9 Done Done Sent false
3547 2010 Acked Read 1 Done Done Sent false
3548 2011 Acked Write 7 Done Done Sent false
3549 2012 Acked Flush 0 Done Done Sent false
3550 2013 Acked Read 9 Done Done Sent false
3551 2014 Acked Write 7 Done Done Sent false
3552 2015 Acked Flush 0 Done Done Sent false
3553 2016 Acked Read 7 Done Done Sent false
3554 2017 Acked Write 8 Done Done Sent false
3555 2018 Acked Write 3 Done Done Sent false
3556 2019 Acked Write 2 Done Done Sent false
3557 2020 Acked Write 7 Done Done Sent false
3558 2021 Acked Write 3 Done Done Sent false
3559 2022 Acked Read 7 Done Done Sent false
3560 2023 Acked Read 9 Done Done Sent false
3561 2024 Acked Write 2 Done Done Sent false
3562 2025 Acked Read 4 Done Done Sent false
3563 2026 Acked Read 6 Done Done Sent false
3564 2027 Acked Read 7 Done Done Sent false
3565 2028 Acked Write 9 Done Done Sent false
3566 2029 Acked Read 1 Done Done Sent false
3567 2030 Acked Write 10 Done Done Sent false
3568 2031 Acked Flush 0 Done Done Sent false
3569 2032 Acked Write 6 Done Done Sent false
3570 2033 Acked Read 5 Done Done Sent false
3571 2034 Acked Read 6 Done Done Sent false
3572 2035 Acked Read 9 Done Done Sent false
3573 2036 Acked Read 10 Done Done Sent false
3574 2037 Acked Write 1 Done Done Sent false
3575 2038 Acked Write 7 Done Done Sent false
3576 2039 Acked Write 4 Done Done Sent false
3577 2040 Acked Write 1 Done Done Sent false
3578 2041 Acked Read 2 Done Done Sent false
3579 2042 Acked Write 1 Done Done Sent false
3580 2043 Acked Write 4 Done Done Sent false
3581 2044 Acked Write 7 Done Done Sent false
3582 2045 Acked Write 3 Done Done Sent false
3583 2046 Acked Read 10 Done Done Sent false
3584 2047 Acked Read 4 Done Done Sent false
3585 2048 Acked Read 6 Done Done Sent false
3586 2049 Acked Read 1 Done Done Sent false
3587 2050 Acked Write 4 Done Done Sent false
3588 2051 Acked Write 5 Done Done Sent false
3589 2052 Acked Read 7 Done Done Sent false
3590 2053 Acked Read 3 Done Done Sent false
3591 2054 Acked Read 4 Done Done Sent false
3592 2055 Acked Read 9 Done Done Sent false
3593 2056 Acked Write 10 Done Done Sent false
3594 2057 Acked Read 9 Done Done Sent false
3595 2058 Acked Flush 0 Done Done Sent false
3596 2059 Acked Read 10 Done Done Sent false
3597 2060 Acked Read 10 Done Done Sent false
3598 2061 Acked Write 5 Done Done Sent false
3599 2062 Acked Read 2 Done Done Sent false
3600 2063 Acked Write 4 Done Done Sent false
3601 2064 Acked Read 7 Done Done Sent false
3602 2065 Acked Write 10 Done Done Sent false
3603 2066 Acked Read 6 Done Done Sent false
3604 2067 Acked Read 5 Done Done Sent false
3605 2068 Acked Write 4 Done Done Sent false
3606 2069 Acked Read 2 Done Done Sent false
3607 2070 Acked Read 3 Done Done Sent false
3608 2071 Acked Read 4 Done Done Sent false
3609 2072 Acked Read 7 Done Done Sent false
3610 2073 Acked Read 7 Done Done Sent false
3611 2074 Acked Read 4 Done Done Sent false
3612 2075 Acked Write 6 Done Done Sent false
3613 2076 Acked Write 9 Done Done Sent false
3614 2077 Acked Write 3 Done Done Sent false
3615 2078 Acked Write 2 Done Done Sent false
3616 2079 Acked Write 10 Done Done Sent false
3617 2080 Acked Write 3 Done Done Sent false
3618 2081 Acked Write 10 Done Done Sent false
3619 2082 Acked Write 3 Done Done Sent false
3620 2083 Acked Flush 0 Done Done Sent false
3621 2084 Acked Read 10 Done Done Sent false
3622 2085 Acked Read 5 Done Done Sent false
3623 2086 Acked Write 4 Done Done Sent false
3624 2087 Acked Read 8 Done Done Sent false
3625 2088 Acked Read 9 Done Done Sent false
3626 2089 Acked Flush 0 Done Done Sent false
3627 2090 Acked Write 7 Done Done Sent false
3628 2091 Acked Read 4 Done Done Sent false
3629 2092 Acked Write 9 Done Done Sent false
3630 2093 Acked Write 5 Done Done Sent false
3631 2094 Acked Write 4 Done Done Sent false
3632 2095 Acked Flush 0 Done Done Sent false
3633 2096 Acked Read 4 Done Done Sent false
3634 2097 Acked Read 5 Done Done Sent false
3635 2098 Acked Write 8 Done Done Sent false
3636 2099 Acked Read 2 Done Done Sent false
3637 2100 Acked Read 7 Done Done Sent false
3638 2101 Acked Read 9 Done Done Sent false
3639 2102 Acked Read 10 Done Done Sent false
3640 2103 Acked Read 10 Done Done Sent false
3641 2104 Acked Write 10 Done Done Sent false
3642 2105 Acked Write 4 Done Done Sent false
3643 2106 Acked Write 4 Done Done Sent false
3644 2107 Acked Read 5 Done Done Sent false
3645 2108 Acked Read 1 Done Done Sent false
3646 2109 Acked Read 2 Done Done Sent false
3647 2110 Acked Read 4 Done Done Sent false
3648 2111 Acked Write 2 Done Done Sent false
3649 2112 Acked Read 5 Done Done Sent false
3650 2113 Acked Write 1 Done Done Sent false
3651 2114 Acked Write 1 Done Done Sent false
3652 2115 Acked Write 9 Done Done Sent false
3653 2116 Acked Write 4 Done Done Sent false
3654 2117 Acked Read 8 Done Done Sent false
3655 2118 Acked Write 5 Done Done Sent false
3656 2119 Acked Read 5 Done Done Sent false
3657 2120 Acked Read 9 Done Done Sent false
3658 2121 Acked Flush 0 Done Done Sent false
3659 2122 Acked Read 6 Done Done Sent false
3660 2123 Acked Write 3 Done Done Sent false
3661 2124 Acked Write 9 Done Done Sent false
3662 2125 Acked Write 5 Done Done Sent false
3663 2126 Acked Read 5 Done Done Sent false
3664 2127 Acked Read 1 Done Done Sent false
3665 2128 Acked Read 1 Done Done Sent false
3666 2129 Acked Read 5 Done Done Sent false
3667 2130 Acked Write 5 Done Done Sent false
3668 2131 Acked Read 9 Done Done Sent false
3669 2132 Acked Write 5 Done Done Sent false
3670 2133 Acked Write 8 Done Done Sent false
3671 2134 Acked Write 3 Done Done Sent false
3672 2135 Acked Write 1 Done Done Sent false
3673 2136 Acked Read 7 Done Done Sent false
3674 2137 Acked Read 5 Done Done Sent false
3675 2138 Acked Flush 0 Done Done Sent false
3676 2139 Acked Write 5 Done Done Sent false
3677 2140 Acked Write 6 Done Done Sent false
3678 2141 Acked Read 5 Done Done Sent false
3679 2142 Acked Write 6 Done Done Sent false
3680 2143 Acked Read 8 Done Done Sent false
3681 2144 Acked Write 6 Done Done Sent false
3682 2145 Acked Read 5 Done Done Sent false
3683 2146 Acked Write 3 Done Done Sent false
3684 2147 Acked Read 10 Done Done Sent false
3685 2148 Acked Write 5 Done Done Sent false
3686 2149 Acked Read 5 Done Done Sent false
3687 2150 Acked Read 8 Done Done Sent false
3688 2151 Acked Write 6 Done Done Sent false
3689 2152 Acked Read 1 Done Done Sent false
3690 2153 Acked Read 8 Done Done Sent false
3691 2154 Acked Read 4 Done Done Sent false
3692 2155 Acked Write 5 Done Done Sent false
3693 2156 Acked Read 10 Done Done Sent false
3694 2157 Acked Read 5 Done Done Sent false
3695 2158 Acked Write 1 Done Done Sent false
3696 2159 Acked Read 3 Done Done Sent false
3697 2160 Acked Read 3 Done Done Sent false
3698 2161 Acked Write 7 Done Done Sent false
3699 2162 Acked Read 2 Done Done Sent false
3700 2163 Acked Write 7 Done Done Sent false
3701 2164 Acked Read 9 Done Done Sent false
3702 2165 Acked Read 5 Done Done Sent false
3703 2166 Acked Write 7 Done Done Sent false
3704 2167 Acked Write 2 Done Done Sent false
3705 2168 Acked Read 1 Done Done Sent false
3706 2169 Acked Write 5 Done Done Sent false
3707 2170 Acked Read 3 Done Done Sent false
3708 2171 Acked Read 4 Done Done Sent false
3709 2172 Acked Read 10 Done Done Sent false
3710 2173 Acked Read 2 Done Done Sent false
3711 2174 Acked Read 1 Done Done Sent false
3712 2175 Acked Read 4 Done Done Sent false
3713 2176 Acked Write 2 Done Done Sent false
3714 2177 Acked Write 9 Done Done Sent false
3715 2178 Acked Write 10 Done Done Sent false
3716 2179 Acked Flush 0 Done Done Sent false
3717 2180 Acked Write 6 Done Done Sent false
3718 2181 Acked Read 3 Done Done Sent false
3719 2182 Acked Write 4 Done Done Sent false
3720 2183 Acked Read 7 Done Done Sent false
3721 2184 Acked Read 7 Done Done Sent false
3722 2185 Acked Flush 0 Done Done Sent false
3723 2186 Acked Read 5 Done Done Sent false
3724 2187 Acked Read 1 Done Done Sent false
3725 2188 Acked Read 1 Done Done Sent false
3726 2189 Acked Write 7 Done Done Sent false
3727 2190 Acked Read 2 Done Done Sent false
3728 2191 Acked Read 2 Done Done Sent false
3729 2192 Acked Read 2 Done Done Sent false
3730 2193 Acked Write 7 Done Done Sent false
3731 2194 Acked Write 8 Done Done Sent false
3732 2195 Acked Read 4 Done Done Sent false
3733 2196 Acked Read 2 Done Done Sent false
3734 2197 Acked Write 8 Done Done Sent false
3735 2198 Acked Read 9 Done Done Sent false
3736 2199 Acked Write 2 Done Done Sent false
3737 2200 Acked Read 6 Done Done Sent false
3738 2201 Acked Write 6 Done Done Sent false
3739 2202 Acked Write 10 Done Done Sent false
3740 2203 Acked Read 5 Done Done Sent false
3741 2204 Acked Read 1 Done Done Sent false
3742 2205 Acked Read 6 Done Done Sent false
3743 2206 Acked Write 7 Done Done Sent false
3744 STATES DS:0 DS:1 DS:2 TOTAL
3745 Sent 0 0 301 301
3746 Done 301 301 0 602
3747 Skipped 0 0 0 0
3748 Error 0 0 0 0
3749 Last Flush: 2185 2185 1905
3750 Downstairs last five completed: 1905 1904 1903 1902 1901
3751 Upstairs last five completed: 2206 2205 2204 2203 2202
3752 CLIENT: Up:0 ds:301 act:2
3753 [8830] got start action
3754 [8830] I am going to restart
3755 Starting downstairs at port 8830
3756 Make output file at "/tmp/dsc/downstairs-8830.txt"
3757 [8830][2] reports Starting
3758 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1206)
3759 [8830][2] reports Running
3760 ----------------------------------------------------------------
3761 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
3762 States: Active Active Offline
3763 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3764 1906 Acked Read 2 Done Done Sent false
3765 1907 Acked Read 9 Done Done Sent false
3766 1908 Acked Write 5 Done Done Sent false
3767 1909 Acked Write 3 Done Done Sent false
3768 1910 Acked Read 9 Done Done Sent false
3769 1911 Acked Write 10 Done Done Sent false
3770 1912 Acked Write 3 Done Done Sent false
3771 1913 Acked Read 4 Done Done Sent false
3772 1914 Acked Write 1 Done Done Sent false
3773 1915 Acked Write 8 Done Done Sent false
3774 1916 Acked Read 5 Done Done Sent false
3775 1917 Acked Write 8 Done Done Sent false
3776 1918 Acked Read 4 Done Done Sent false
3777 1919 Acked Read 7 Done Done Sent false
3778 1920 Acked Write 1 Done Done Sent false
3779 1921 Acked Write 6 Done Done Sent false
3780 1922 Acked Read 7 Done Done Sent false
3781 1923 Acked Write 7 Done Done Sent false
3782 1924 Acked Read 8 Done Done Sent false
3783 1925 Acked Read 1 Done Done Sent false
3784 1926 Acked Flush 0 Done Done Sent false
3785 1927 Acked Read 6 Done Done Sent false
3786 1928 Acked Read 8 Done Done Sent false
3787 1929 Acked Write 2 Done Done Sent false
3788 1930 Acked Read 9 Done Done Sent false
3789 1931 Acked Read 3 Done Done Sent false
3790 1932 Acked Flush 0 Done Done Sent false
3791 1933 Acked Flush 0 Done Done Sent false
3792 1934 Acked Write 10 Done Done Sent false
3793 1935 Acked Read 7 Done Done Sent false
3794 1936 Acked Write 3 Done Done Sent false
3795 1937 Acked Read 3 Done Done Sent false
3796 1938 Acked Read 2 Done Done Sent false
3797 1939 Acked Read 3 Done Done Sent false
3798 1940 Acked Read 8 Done Done Sent false
3799 1941 Acked Write 5 Done Done Sent false
3800 1942 Acked Read 3 Done Done Sent false
3801 1943 Acked Read 9 Done Done Sent false
3802 1944 Acked Write 1 Done Done Sent false
3803 1945 Acked Write 8 Done Done Sent false
3804 1946 Acked Write 1 Done Done Sent false
3805 1947 Acked Write 6 Done Done Sent false
3806 1948 Acked Read 1 Done Done Sent false
3807 1949 Acked Write 10 Done Done Sent false
3808 1950 Acked Write 1 Done Done Sent false
3809 1951 Acked Read 10 Done Done Sent false
3810 1952 Acked Write 2 Done Done Sent false
3811 1953 Acked Read 2 Done Done Sent false
3812 1954 Acked Read 4 Done Done Sent false
3813 1955 Acked Read 1 Done Done Sent false
3814 1956 Acked Read 5 Done Done Sent false
3815 1957 Acked Write 6 Done Done Sent false
3816 1958 Acked Read 3 Done Done Sent false
3817 1959 Acked Write 8 Done Done Sent false
3818 1960 Acked Write 8 Done Done Sent false
3819 1961 Acked Read 8 Done Done Sent false
3820 1962 Acked Flush 0 Done Done Sent false
3821 1963 Acked Write 1 Done Done Sent false
3822 1964 Acked Read 4 Done Done Sent false
3823 1965 Acked Read 9 Done Done Sent false
3824 1966 Acked Flush 0 Done Done Sent false
3825 1967 Acked Write 10 Done Done Sent false
3826 1968 Acked Write 3 Done Done Sent false
3827 1969 Acked Write 9 Done Done Sent false
3828 1970 Acked Read 7 Done Done Sent false
3829 1971 Acked Read 9 Done Done Sent false
3830 1972 Acked Flush 0 Done Done Sent false
3831 1973 Acked Read 6 Done Done Sent false
3832 1974 Acked Write 3 Done Done Sent false
3833 1975 Acked Flush 0 Done Done Sent false
3834 1976 Acked Write 8 Done Done Sent false
3835 1977 Acked Write 4 Done Done Sent false
3836 1978 Acked Write 6 Done Done Sent false
3837 1979 Acked Read 9 Done Done Sent false
3838 1980 Acked Read 5 Done Done Sent false
3839 1981 Acked Read 3 Done Done Sent false
3840 1982 Acked Write 9 Done Done Sent false
3841 1983 Acked Read 9 Done Done Sent false
3842 1984 Acked Read 10 Done Done Sent false
3843 1985 Acked Read 3 Done Done Sent false
3844 1986 Acked Read 9 Done Done Sent false
3845 1987 Acked Write 9 Done Done Sent false
3846 1988 Acked Flush 0 Done Done Sent false
3847 1989 Acked Read 4 Done Done Sent false
3848 1990 Acked Read 5 Done Done Sent false
3849 1991 Acked Write 1 Done Done Sent false
3850 1992 Acked Read 9 Done Done Sent false
3851 1993 Acked Write 10 Done Done Sent false
3852 1994 Acked Flush 0 Done Done Sent false
3853 1995 Acked Read 8 Done Done Sent false
3854 1996 Acked Read 3 Done Done Sent false
3855 1997 Acked Read 4 Done Done Sent false
3856 1998 Acked Write 7 Done Done Sent false
3857 1999 Acked Read 4 Done Done Sent false
3858 2000 Acked Read 6 Done Done Sent false
3859 2001 Acked Read 3 Done Done Sent false
3860 2002 Acked Write 9 Done Done Sent false
3861 2003 Acked Read 7 Done Done Sent false
3862 2004 Acked Read 1 Done Done Sent false
3863 2005 Acked Read 4 Done Done Sent false
3864 2006 Acked Read 2 Done Done Sent false
3865 2007 Acked Flush 0 Done Done Sent false
3866 2008 Acked Read 8 Done Done Sent false
3867 2009 Acked Read 9 Done Done Sent false
3868 2010 Acked Read 1 Done Done Sent false
3869 2011 Acked Write 7 Done Done Sent false
3870 2012 Acked Flush 0 Done Done Sent false
3871 2013 Acked Read 9 Done Done Sent false
3872 2014 Acked Write 7 Done Done Sent false
3873 2015 Acked Flush 0 Done Done Sent false
3874 2016 Acked Read 7 Done Done Sent false
3875 2017 Acked Write 8 Done Done Sent false
3876 2018 Acked Write 3 Done Done Sent false
3877 2019 Acked Write 2 Done Done Sent false
3878 2020 Acked Write 7 Done Done Sent false
3879 2021 Acked Write 3 Done Done Sent false
3880 2022 Acked Read 7 Done Done Sent false
3881 2023 Acked Read 9 Done Done Sent false
3882 2024 Acked Write 2 Done Done Sent false
3883 2025 Acked Read 4 Done Done Sent false
3884 2026 Acked Read 6 Done Done Sent false
3885 2027 Acked Read 7 Done Done Sent false
3886 2028 Acked Write 9 Done Done Sent false
3887 2029 Acked Read 1 Done Done Sent false
3888 2030 Acked Write 10 Done Done Sent false
3889 2031 Acked Flush 0 Done Done Sent false
3890 2032 Acked Write 6 Done Done Sent false
3891 2033 Acked Read 5 Done Done Sent false
3892 2034 Acked Read 6 Done Done Sent false
3893 2035 Acked Read 9 Done Done Sent false
3894 2036 Acked Read 10 Done Done Sent false
3895 2037 Acked Write 1 Done Done Sent false
3896 2038 Acked Write 7 Done Done Sent false
3897 2039 Acked Write 4 Done Done Sent false
3898 2040 Acked Write 1 Done Done Sent false
3899 2041 Acked Read 2 Done Done Sent false
3900 2042 Acked Write 1 Done Done Sent false
3901 2043 Acked Write 4 Done Done Sent false
3902 2044 Acked Write 7 Done Done Sent false
3903 2045 Acked Write 3 Done Done Sent false
3904 2046 Acked Read 10 Done Done Sent false
3905 2047 Acked Read 4 Done Done Sent false
3906 2048 Acked Read 6 Done Done Sent false
3907 2049 Acked Read 1 Done Done Sent false
3908 2050 Acked Write 4 Done Done Sent false
3909 2051 Acked Write 5 Done Done Sent false
3910 2052 Acked Read 7 Done Done Sent false
3911 2053 Acked Read 3 Done Done Sent false
3912 2054 Acked Read 4 Done Done Sent false
3913 2055 Acked Read 9 Done Done Sent false
3914 2056 Acked Write 10 Done Done Sent false
3915 2057 Acked Read 9 Done Done Sent false
3916 2058 Acked Flush 0 Done Done Sent false
3917 2059 Acked Read 10 Done Done Sent false
3918 2060 Acked Read 10 Done Done Sent false
3919 2061 Acked Write 5 Done Done Sent false
3920 2062 Acked Read 2 Done Done Sent false
3921 2063 Acked Write 4 Done Done Sent false
3922 2064 Acked Read 7 Done Done Sent false
3923 2065 Acked Write 10 Done Done Sent false
3924 2066 Acked Read 6 Done Done Sent false
3925 2067 Acked Read 5 Done Done Sent false
3926 2068 Acked Write 4 Done Done Sent false
3927 2069 Acked Read 2 Done Done Sent false
3928 2070 Acked Read 3 Done Done Sent false
3929 2071 Acked Read 4 Done Done Sent false
3930 2072 Acked Read 7 Done Done Sent false
3931 2073 Acked Read 7 Done Done Sent false
3932 2074 Acked Read 4 Done Done Sent false
3933 2075 Acked Write 6 Done Done Sent false
3934 2076 Acked Write 9 Done Done Sent false
3935 2077 Acked Write 3 Done Done Sent false
3936 2078 Acked Write 2 Done Done Sent false
3937 2079 Acked Write 10 Done Done Sent false
3938 2080 Acked Write 3 Done Done Sent false
3939 2081 Acked Write 10 Done Done Sent false
3940 2082 Acked Write 3 Done Done Sent false
3941 2083 Acked Flush 0 Done Done Sent false
3942 2084 Acked Read 10 Done Done Sent false
3943 2085 Acked Read 5 Done Done Sent false
3944 2086 Acked Write 4 Done Done Sent false
3945 2087 Acked Read 8 Done Done Sent false
3946 2088 Acked Read 9 Done Done Sent false
3947 2089 Acked Flush 0 Done Done Sent false
3948 2090 Acked Write 7 Done Done Sent false
3949 2091 Acked Read 4 Done Done Sent false
3950 2092 Acked Write 9 Done Done Sent false
3951 2093 Acked Write 5 Done Done Sent false
3952 2094 Acked Write 4 Done Done Sent false
3953 2095 Acked Flush 0 Done Done Sent false
3954 2096 Acked Read 4 Done Done Sent false
3955 2097 Acked Read 5 Done Done Sent false
3956 2098 Acked Write 8 Done Done Sent false
3957 2099 Acked Read 2 Done Done Sent false
3958 2100 Acked Read 7 Done Done Sent false
3959 2101 Acked Read 9 Done Done Sent false
3960 2102 Acked Read 10 Done Done Sent false
3961 2103 Acked Read 10 Done Done Sent false
3962 2104 Acked Write 10 Done Done Sent false
3963 2105 Acked Write 4 Done Done Sent false
3964 2106 Acked Write 4 Done Done Sent false
3965 2107 Acked Read 5 Done Done Sent false
3966 2108 Acked Read 1 Done Done Sent false
3967 2109 Acked Read 2 Done Done Sent false
3968 2110 Acked Read 4 Done Done Sent false
3969 2111 Acked Write 2 Done Done Sent false
3970 2112 Acked Read 5 Done Done Sent false
3971 2113 Acked Write 1 Done Done Sent false
3972 2114 Acked Write 1 Done Done Sent false
3973 2115 Acked Write 9 Done Done Sent false
3974 2116 Acked Write 4 Done Done Sent false
3975 2117 Acked Read 8 Done Done Sent false
3976 2118 Acked Write 5 Done Done Sent false
3977 2119 Acked Read 5 Done Done Sent false
3978 2120 Acked Read 9 Done Done Sent false
3979 2121 Acked Flush 0 Done Done Sent false
3980 2122 Acked Read 6 Done Done Sent false
3981 2123 Acked Write 3 Done Done Sent false
3982 2124 Acked Write 9 Done Done Sent false
3983 2125 Acked Write 5 Done Done Sent false
3984 2126 Acked Read 5 Done Done Sent false
3985 2127 Acked Read 1 Done Done Sent false
3986 2128 Acked Read 1 Done Done Sent false
3987 2129 Acked Read 5 Done Done Sent false
3988 2130 Acked Write 5 Done Done Sent false
3989 2131 Acked Read 9 Done Done Sent false
3990 2132 Acked Write 5 Done Done Sent false
3991 2133 Acked Write 8 Done Done Sent false
3992 2134 Acked Write 3 Done Done Sent false
3993 2135 Acked Write 1 Done Done Sent false
3994 2136 Acked Read 7 Done Done Sent false
3995 2137 Acked Read 5 Done Done Sent false
3996 2138 Acked Flush 0 Done Done Sent false
3997 2139 Acked Write 5 Done Done Sent false
3998 2140 Acked Write 6 Done Done Sent false
3999 2141 Acked Read 5 Done Done Sent false
4000 2142 Acked Write 6 Done Done Sent false
4001 2143 Acked Read 8 Done Done Sent false
4002 2144 Acked Write 6 Done Done Sent false
4003 2145 Acked Read 5 Done Done Sent false
4004 2146 Acked Write 3 Done Done Sent false
4005 2147 Acked Read 10 Done Done Sent false
4006 2148 Acked Write 5 Done Done Sent false
4007 2149 Acked Read 5 Done Done Sent false
4008 2150 Acked Read 8 Done Done Sent false
4009 2151 Acked Write 6 Done Done Sent false
4010 2152 Acked Read 1 Done Done Sent false
4011 2153 Acked Read 8 Done Done Sent false
4012 2154 Acked Read 4 Done Done Sent false
4013 2155 Acked Write 5 Done Done Sent false
4014 2156 Acked Read 10 Done Done Sent false
4015 2157 Acked Read 5 Done Done Sent false
4016 2158 Acked Write 1 Done Done Sent false
4017 2159 Acked Read 3 Done Done Sent false
4018 2160 Acked Read 3 Done Done Sent false
4019 2161 Acked Write 7 Done Done Sent false
4020 2162 Acked Read 2 Done Done Sent false
4021 2163 Acked Write 7 Done Done Sent false
4022 2164 Acked Read 9 Done Done Sent false
4023 2165 Acked Read 5 Done Done Sent false
4024 2166 Acked Write 7 Done Done Sent false
4025 2167 Acked Write 2 Done Done Sent false
4026 2168 Acked Read 1 Done Done Sent false
4027 2169 Acked Write 5 Done Done Sent false
4028 2170 Acked Read 3 Done Done Sent false
4029 2171 Acked Read 4 Done Done Sent false
4030 2172 Acked Read 10 Done Done Sent false
4031 2173 Acked Read 2 Done Done Sent false
4032 2174 Acked Read 1 Done Done Sent false
4033 2175 Acked Read 4 Done Done Sent false
4034 2176 Acked Write 2 Done Done Sent false
4035 2177 Acked Write 9 Done Done Sent false
4036 2178 Acked Write 10 Done Done Sent false
4037 2179 Acked Flush 0 Done Done Sent false
4038 2180 Acked Write 6 Done Done Sent false
4039 2181 Acked Read 3 Done Done Sent false
4040 2182 Acked Write 4 Done Done Sent false
4041 2183 Acked Read 7 Done Done Sent false
4042 2184 Acked Read 7 Done Done Sent false
4043 2185 Acked Flush 0 Done Done Sent false
4044 2186 Acked Read 5 Done Done Sent false
4045 2187 Acked Read 1 Done Done Sent false
4046 2188 Acked Read 1 Done Done Sent false
4047 2189 Acked Write 7 Done Done Sent false
4048 2190 Acked Read 2 Done Done Sent false
4049 2191 Acked Read 2 Done Done Sent false
4050 2192 Acked Read 2 Done Done Sent false
4051 2193 Acked Write 7 Done Done Sent false
4052 2194 Acked Write 8 Done Done Sent false
4053 2195 Acked Read 4 Done Done Sent false
4054 2196 Acked Read 2 Done Done Sent false
4055 2197 Acked Write 8 Done Done Sent false
4056 2198 Acked Read 9 Done Done Sent false
4057 2199 Acked Write 2 Done Done Sent false
4058 2200 Acked Read 6 Done Done Sent false
4059 2201 Acked Write 6 Done Done Sent false
4060 2202 Acked Write 10 Done Done Sent false
4061 2203 Acked Read 5 Done Done Sent false
4062 2204 Acked Read 1 Done Done Sent false
4063 2205 Acked Read 6 Done Done Sent false
4064 2206 Acked Write 7 Done Done Sent false
4065 2207 Acked Flush 0 Done Done Sent false
4066 STATES DS:0 DS:1 DS:2 TOTAL
4067 Sent 0 0 302 302
4068 Done 302 302 0 604
4069 Skipped 0 0 0 0
4070 Error 0 0 0 0
4071 Last Flush: 2207 2207 1905
4072 Downstairs last five completed: 1905 1904 1903 1902 1901
4073 Upstairs last five completed: 2207 2206 2205 2204 2203
4074 CLIENT: Up:0 ds:302 act:2
40752025-01-13T15:52:37.788ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
40762025-01-13T15:52:37.788ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61546) = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
40772025-01-13T15:52:37.790ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
40782025-01-13T15:52:37.790ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
40792025-01-13T15:52:37.790ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
40802025-01-13T15:52:37.790ZINFOcrucible: send last flush ID to this DS: 1905 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
40812025-01-13T15:52:37.791ZINFOcrucible: Replied this last flush ID: 1905 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
40822025-01-13T15:52:37.791ZINFOcrucible: [2] client re-new 302 jobs since flush 1905 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
40832025-01-13T15:52:37.791ZINFOcrucible: [2] Replayed 302 jobs since flush: 1905 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
4084 ----------------------------------------------------------------
4085 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
4086 Downstairs last five completed: 2207 2206 2205 2204 2203
4087 Upstairs last five completed: 2207 2206 2205 2204 2203
4088 CLIENT: Up:0 ds:0 act:3
4089 Replay: All jobs finished
4090 Main task has work to do, go find it
4091 got dsc Stop(1)
4092 stop 1
4093 Jan 13 15:52:40.041 INFO request completed, latency_us[8820] Got stop action so:false kr:false
4094 : 361, response_code: 204, uri: /stop/cid/1, method: POST, req_id: cab3ee61-dc10-467a-83ed-4af313855d03, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
4095 Jan 13 15:52:40.041 INFO request completed, latency_us: 111, response_code: 200, uri: /state/cid/1, method: GET{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:52:40.041734564Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
4096 , req_id: eb94a605-ef26-4396-979a-b006a045f395, remote_addr: 127.0.0.1:52422, local_addr{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:52:40.041844547Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
4097 : 127.0.0.1:9998
40982025-01-13T15:52:40.041ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
40992025-01-13T15:52:40.041ZINFOcrucible: client task is exiting = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
4100 [8820] Exited with: ExitStatus(unix_wait_status(9))
4101 [8820][1] reports Exit
4102 Jan 13 15:52:44.043 INFO request completed, latency_us: 253, response_code: 200, uri: /state/cid/1, method: GET, req_id: fe9059e5-2287-4850-8165-91ebce1d5fb1, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
4103 001/300 Write block 1448 len 8192 data: 2 2
4104 002/300 Read block 4146 len 8192
4105 003/300 Write block 2618 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4106 004/300 Write block 4542 len 32768 data: 2 2 2 2 2 2 2 2
4107 005/300 Flush
4108 006/300 Write block 2019 len 32768 data: 3 2 2 2 2 2 2 4
4109 007/300 Read block 4811 len 4096
4110 008/300 Write block 808 len 16384 data: 2 2 2 2
4111 009/300 Read block 418 len 28672
4112 010/300 Read block 818 len 4096
4113 011/300 Flush
4114 012/300 Read block 4817 len 32768
4115 013/300 Write block 1183 len 40960 data: 2 2 2 3 3 2 2 2 2 2
4116 014/300 Read block 4842 len 32768
4117 015/300 Read block 4013 len 8192
4118 016/300 Read block 152 len 36864
4119 017/300 Read block 4505 len 8192
4120 018/300 Read block 1750 len 40960
4121 019/300 Write block 1915 len 36864 data: 2 2 2 2 2 2 2 2 2
4122 020/300 Flush
4123 021/300 Read block 3140 len 40960
4124 022/300 Write block 3759 len 12288 data: 2 2 2
4125 023/300 Read block 3659 len 40960
4126 024/300 Read block 4131 len 36864
4127 025/300 Write block 1026 len 12288 data: 3 3 3
4128 026/300 Flush
4129 027/300 Read block 4755 len 40960
4130 028/300 Write block 2264 len 16384 data: 3 3 3 3
4131 029/300 Write block 1169 len 36864 data: 3 2 3 3 4 2 2 3 4
4132 030/300 Write block 1897 len 24576 data: 2 2 2 2 2 2
4133 031/300 Read block 2022 len 4096
4134 032/300 Read block 2891 len 24576
4135 033/300 Read block 3046 len 12288
4136 034/300 Flush
4137 035/300 Read block 1175 len 20480
4138 036/300 Read block 419 len 4096
4139 037/300 Read block 3708 len 20480
4140 038/300 Read block 4883 len 8192
4141 039/300 Read block 635 len 28672
4142 040/300 Read block 4525 len 32768
4143 041/300 Write block 1613 len 36864 data: 2 2 2 3 3 3 3 3 3
4144 042/300 Write block 2146 len 28672 data: 2 2 2 2 2 2 2
4145 043/300 Read block 1489 len 16384
4146 044/300 Read block 3207 len 24576
4147 045/300 Write block 1605 len 12288 data: 3 3 3
4148 046/300 Flush
4149 047/300 Write block 936 len 4096 data: 2
4150 048/300 Write block 3351 len 4096 data: 2
4151 049/300 Write block 4131 len 16384 data: 2 2 2 2
4152 050/300 Read block 2342 len 8192
4153 051/300 Write block 1165 len 28672 data: 3 3 3 3 4 3 4
4154 052/300 Write block 128 len 32768 data: 4 4 4 4 4 3 3 3
4155 053/300 Read block 4497 len 36864
4156 054/300 Write block 1499 len 4096 data: 3
4157 055/300 Read block 2891 len 20480
4158 056/300 Flush
4159 057/300 Write block 3347 len 12288 data: 2 2 2
4160 058/300 Write block 1838 len 4096 data: 3
4161 059/300 Write block 4204 len 8192 data: 3 3
4162 060/300 Write block 141 len 28672 data: 2 4 3 3 3 3 3
4163 061/300 Write block 3398 len 32768 data: 3 2 3 4 3 3 3 2
4164 062/300 Read block 3762 len 36864
4165 063/300 Write block 100 len 24576 data: 3 3 3 3 3 3
4166 064/300 Write block 1082 len 32768 data: 3 3 3 3 3 3 4 3
4167 065/300 Write block 749 len 40960 data: 3 3 2 2 2 2 2 2 2 2
4168 066/300 Read block 1363 len 16384
4169 067/300 Write block 1775 len 12288 data: 2 2 2
4170 068/300 Write block 3958 len 40960 data: 2 3 3 3 3 3 3 3 2 2
4171 069/300 Flush
4172 070/300 Read block 2195 len 16384
4173 071/300 Write block 2883 len 16384 data: 2 2 2 2
4174 072/300 Write block 1328 len 8192 data: 2 2
4175 073/300 Read block 1380 len 36864
4176 074/300 Read block 2987 len 20480
4177 075/300 Write block 2171 len 40960 data: 4 4 4 3 3 3 3 2 2 2
4178 076/300 Read block 218 len 32768
4179 077/300 Read block 1161 len 36864
4180 078/300 Read block 1060 len 8192
4181 079/300 Write block 4928 len 40960 data: 3 3 4 4 4 3 3 3 3 3
4182 080/300 Write block 4987 len 20480 data: 3 3 3 2 2
4183 081/300 Read block 4774 len 16384
4184 082/300 Flush
4185 083/300 Write block 3509 len 28672 data: 2 2 2 2 2 3 3
4186 084/300 Read block 552 len 8192
4187 085/300 Read block 4259 len 4096
4188 086/300 Write block 4865 len 8192 data: 3 3
4189 087/300 Read block 3679 len 24576
4190 088/300 Read block 1500 len 32768
4191 089/300 Read block 3319 len 16384
4192 090/300 Write block 866 len 16384 data: 3 3 2 3
4193 091/300 Write block 215 len 36864 data: 4 3 3 3 2 2 2 2 2
4194 092/300 Read block 3127 len 24576
4195 093/300 Read block 4278 len 24576
4196 094/300 Read block 479 len 16384
4197 095/300 Read block 3610 len 32768
4198 096/300 Write block 4728 len 40960 data: 2 2 2 2 2 2 3 3 3 3
4199 097/300 Flush
4200 098/300 Read block 4586 len 12288
4201 099/300 Write block 3598 len 36864 data: 3 3 2 2 2 2 2 2 2
4202 100/300 Read block 2029 len 32768
4203 101/300 Read block 3547 len 32768
4204 102/300 Read block 2605 len 20480
4205 103/300 Read block 1056 len 4096
4206 104/300 Flush
4207 105/300 Flush
4208 106/300 Read block 3590 len 24576
4209 107/300 Write block 4624 len 28672 data: 4 4 4 4 3 2 2
4210 108/300 Read block 4334 len 28672
4211 109/300 Write block 4855 len 24576 data: 2 2 2 2 2 3
4212 110/300 Flush
4213 111/300 Read block 2807 len 16384
4214 112/300 Read block 549 len 36864
4215 113/300 Read block 566 len 28672
4216 114/300 Read block 2984 len 4096
4217 115/300 Flush
4218 116/300 Flush
4219 117/300 Read block 2458 len 4096
4220 118/300 Read block 4093 len 16384
4221 119/300 Read block 4733 len 12288
4222 120/300 Write block 3305 len 36864 data: 2 2 2 2 2 2 2 2 2
4223 121/300 Read block 4421 len 4096
4224 122/300 Read block 2462 len 4096
4225 123/300 Write block 4931 len 32768 data: 5 5 4 4 4 4 4 2
4226 124/300 Write block 3072 len 8192 data: 4 4
4227 125/300 Read block 4659 len 24576
4228 126/300 Write block 529 len 12288 data: 3 3 3
4229 127/300 Write block 3721 len 16384 data: 3 3 3 3
4230 128/300 Read block 273 len 24576
4231 129/300 Write block 4347 len 36864 data: 4 4 3 3 2 2 2 2 2
4232 130/300 Read block 2858 len 32768
4233 131/300 Write block 4637 len 28672 data: 2 2 2 2 2 3 3
4234 132/300 Write block 4958 len 12288 data: 2 2 2
4235 133/300 Read block 3685 len 36864
4236 134/300 Read block 2259 len 28672
4237 135/300 Read block 3549 len 24576
4238 136/300 Write block 1975 len 20480 data: 2 2 2 2 2
4239 137/300 Write block 1699 len 4096 data: 2
4240 138/300 Write block 2141 len 12288 data: 3 3 3
4241 139/300 Flush
4242 140/300 Write block 4600 len 24576 data: 4 3 3 3 3 3
4243 141/300 Read block 4395 len 28672
4244 142/300 Read block 2664 len 36864
4245 143/300 Flush
4246 144/300 Read block 1727 len 24576
4247 145/300 Read block 2773 len 40960
4248 146/300 Read block 4710 len 12288
4249 147/300 Write block 55 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4250 148/300 Write block 4109 len 12288 data: 4 4 4
4251 149/300 Write block 3043 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4252 150/300 Write block 4518 len 28672 data: 4 5 3 3 3 3 3
4253 151/300 Read block 1220 len 20480
4254 152/300 Read block 2346 len 12288
4255 153/300 Read block 3575 len 4096
4256 154/300 Read block 1959 len 12288
4257 155/300 Write block 2189 len 40960 data: 3 3 2 2 3 2 2 2 2 2
4258 156/300 Write block 407 len 16384 data: 2 2 2 2
4259 157/300 Read block 2951 len 32768
4260 158/300 Write block 4035 len 12288 data: 2 2 2
4261 159/300 Read block 3302 len 20480
4262 160/300 Write block 1277 len 24576 data: 2 2 2 2 3 2
4263 161/300 Write block 960 len 12288 data: 2 3 3
4264 162/300 Read block 1692 len 28672
4265 163/300 Read block 2481 len 36864
4266 164/300 Write block 2591 len 36864 data: 4 5 5 5 5 5 5 4 4
4267 165/300 Read block 2905 len 28672
4268 166/300 Read block 4429 len 20480
4269 167/300 Write block 850 len 12288 data: 2 2 2
4270 168/300 Read block 3026 len 32768
4271 169/300 Write block 2777 len 28672 data: 2 2 2 2 2 3 3
4272 170/300 Read block 553 len 24576
4273 171/300 Write block 4654 len 32768 data: 2 2 2 2 2 2 2 2
4274 172/300 Write block 4886 len 16384 data: 3 3 4 4
4275 173/300 Flush
4276 174/300 Write block 4730 len 24576 data: 3 3 3 3 4 4
4277 175/300 Read block 2167 len 16384
4278 176/300 Read block 841 len 40960
4279 177/300 Read block 14 len 28672
4280 178/300 Read block 2650 len 4096
4281 179/300 Read block 3853 len 36864
4282 180/300 Write block 4023 len 28672 data: 2 2 2 2 2 2 2
4283 181/300 Write block 1776 len 36864 data: 3 3 2 3 3 2 2 2 2
4284 182/300 Write block 2274 len 16384 data: 3 3 3 4
4285 183/300 Write block 3899 len 16384 data: 4 3 3 3
4286 184/300 Write block 3414 len 32768 data: 3 3 2 2 2 2 2 3
4287 185/300 Flush
4288 186/300 Flush
4289 187/300 Read block 1769 len 36864
4290 188/300 Write block 3666 len 16384 data: 4 4 4 4
4291 189/300 Write block 4298 len 12288 data: 2 2 2
4292 190/300 Read block 3682 len 20480
4293 191/300 Flush
4294 192/300 Read block 4468 len 8192
4295 193/300 Write block 2524 len 20480 data: 2 2 2 2 2
4296 194/300 Write block 4663 len 16384 data: 2 2 2 2
4297 195/300 Write block 2024 len 16384 data: 3 3 5 5
4298 196/300 Read block 4782 len 20480
4299 197/300 Read block 4856 len 40960
4300 198/300 Read block 1444 len 20480
4301 199/300 Read block 4134 len 32768
4302 200/300 Read block 4930 len 16384
4303 201/300 Write block 3443 len 36864 data: 3 2 2 2 2 2 2 2 2
4304 202/300 Read block 3245 len 40960
4305 203/300 Read block 1763 len 36864
4306 204/300 Read block 1640 len 40960
4307 205/300 Write block 3776 len 32768 data: 2 2 2 2 2 2 2 2
4308 206/300 Write block 2687 len 16384 data: 2 2 2 2
4309 207/300 Write block 2810 len 4096 data: 3
4310 208/300 Read block 4943 len 8192
4311 209/300 Flush
4312 210/300 Write block 1694 len 20480 data: 2 3 3 2 2
4313 211/300 Write block 1020 len 12288 data: 2 2 3
4314 212/300 Write block 4223 len 12288 data: 3 3 4
4315 213/300 Write block 2926 len 28672 data: 2 2 2 2 2 2 2
4316 214/300 Write block 4058 len 4096 data: 2
4317 215/300 Read block 1283 len 36864
4318 216/300 Read block 241 len 36864
4319 217/300 Read block 1802 len 8192
4320 218/300 Read block 3286 len 32768
4321 219/300 Read block 4401 len 24576
4322 220/300 Flush
4323 221/300 Read block 3291 len 16384
4324 222/300 Write block 3230 len 20480 data: 3 3 3 2 2
4325 223/300 Write block 2023 len 28672 data: 3 4 4 6 6 6 5
4326 224/300 Read block 311 len 8192
4327 225/300 Flush
4328 226/300 Read block 1499 len 40960
4329 227/300 Write block 1222 len 20480 data: 3 3 3 2 2
4330 228/300 Read block 222 len 16384
4331 229/300 Write block 939 len 36864 data: 2 2 2 4 4 5 5 5 5
4332 230/300 Write block 987 len 24576 data: 3 3 3 2 2 2
4333 231/300 Write block 2875 len 8192 data: 2 2
4334 232/300 Write block 4205 len 36864 data: 4 3 3 2 2 2 3 3 4
4335 233/300 Write block 2638 len 24576 data: 2 2 2 2 2 2
4336 234/300 Read block 3684 len 28672
4337 235/300 Flush
4338 236/300 Read block 3117 len 24576
4339 237/300 Read block 4033 len 28672
4340 238/300 Write block 4572 len 32768 data: 2 2 2 2 2 3 3 4
4341 239/300 Write block 4051 len 32768 data: 2 2 2 2 2 2 2 3
4342 240/300 Flush
4343 241/300 Write block 3617 len 4096 data: 4
4344 242/300 Read block 156 len 40960
4345 243/300 Read block 4555 len 28672
4346 244/300 Read block 2408 len 20480
4347 245/300 Read block 616 len 16384
4348 246/300 Write block 781 len 16384 data: 2 2 2 2
4349 247/300 Read block 2034 len 40960
4350 248/300 Write block 3640 len 28672 data: 2 2 2 2 3 3 4
4351 249/300 Write block 1753 len 8192 data: 3 2
4352 250/300 Write block 2120 len 32768 data: 4 4 4 3 3 3 3 3
4353 251/300 Read block 3390 len 4096
4354 252/300 Write block 2293 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4355 253/300 Write block 383 len 20480 data: 2 2 2 3 3
4356 254/300 Read block 2385 len 16384
4357 255/300 Read block 4334 len 24576
4358 256/300 Read block 2 len 8192
4359 257/300 Write block 3271 len 20480 data: 2 2 3 3 3
4360 258/300 Read block 1202 len 16384
4361 259/300 Read block 4886 len 36864
4362 260/300 Read block 4730 len 40960
4363 261/300 Read block 1253 len 40960
4364 262/300 Write block 2683 len 8192 data: 2 2
4365 263/300 Flush
4366 264/300 Write block 1836 len 16384 data: 2 3 4 3
4367 265/300 Write block 1312 len 4096 data: 2
4368 266/300 Read block 156 len 8192
4369 267/300 Read block 2507 len 24576
4370 268/300 Read block 1251 len 16384
4371 269/300 Flush
4372 270/300 Write block 3040 len 36864 data: 3 3 2 3 3 3 3 3 3
4373 271/300 Read block 1464 len 28672
4374 272/300 Write block 1034 len 24576 data: 2 2 2 2 2 3
4375 273/300 Write block 3087 len 12288 data: 3 3 3
4376 274/300 Write block 3293 len 8192 data: 3 3
4377 275/300 Read block 2689 len 36864
4378 276/300 Write block 2590 len 32768 data: 3 5 6 6 6 6 6 6
4379 277/300 Write block 3994 len 16384 data: 2 2 2 3
4380 278/300 Read block 899 len 16384
4381 279/300 Write block 186 len 12288 data: 3 3 3
4382 280/300 Flush
4383 281/300 Write block 3437 len 20480 data: 2 2 3 3 3
4384 282/300 Flush
4385 283/300 Flush
4386 284/300 Read block 50 len 4096
4387 285/300 Read block 3701 len 32768
4388 286/300 Write block 3523 len 32768 data: 2 2 2 2 2 2 2 2
4389 287/300 Write block 3871 len 32768 data: 2 2 2 2 3 3 3 3
4390 288/300 Write block 783 len 16384 data: 3 3 2 2
4391 289/300 Write block 3395 len 12288 data: 5 5 4
4392 290/300 Read block 2714 len 4096
4393 291/300 Read block 2019 len 16384
4394 292/300 Write block 214 len 24576 data: 4 5 4 4 4 3
4395 293/300 Flush
4396 294/300 Write block 818 len 36864 data: 2 2 2 2 2 2 2 2 2
4397 295/300 Read block 1166 len 20480
4398 296/300 Write block 1665 len 16384 data: 3 2 2 4
4399 297/300 Read block 2782 len 32768
4400 298/300 Write block 2650 len 28672 data: 2 2 2 2 2 2 2
4401 299/300 Write block 1377 len 28672 data: 3 2 2 2 2 2 2
4402 300/300 Write block 4940 len 32768 data: 2 3 3 3 3 2 2 2
4403 301/300 Write block 3273 len 28672 data: 4 4 4 3 3 3 3
4404 Main task has work to do, go find it
4405 got dsc Start(1)
4406 start 1
4407 Jan 13 15:52:44.278 INFO request completed, latency_us: 205, response_code: 204, uri: /start/cid/1, method: POST, req_id: f475bbd7-4be3-40bc-8264-47aa36d20bd2, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
4408 Replay: started 1, returned:Ok(())
4409 ----------------------------------------------------------------
4410 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
4411 States: Active Offline Active
4412 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4413 2208 Acked Write 2 Done Sent Done false
4414 2209 Acked Read 2 Done Sent Done false
4415 2210 Acked Write 10 Done Sent Done false
4416 2211 Acked Write 8 Done Sent Done false
4417 2212 Acked Flush 0 Done Sent Done false
4418 2213 Acked Write 8 Done Sent Done false
4419 2214 Acked Read 1 Done Sent Done false
4420 2215 Acked Write 4 Done Sent Done false
4421 2216 Acked Read 7 Done Sent Done false
4422 2217 Acked Read 1 Done Sent Done false
4423 2218 Acked Flush 0 Done Sent Done false
4424 2219 Acked Read 8 Done Sent Done false
4425 2220 Acked Write 10 Done Sent Done false
4426 2221 Acked Read 8 Done Sent Done false
4427 2222 Acked Read 2 Done Sent Done false
4428 2223 Acked Read 9 Done Sent Done false
4429 2224 Acked Read 2 Done Sent Done false
4430 2225 Acked Read 10 Done Sent Done false
4431 2226 Acked Write 9 Done Sent Done false
4432 2227 Acked Flush 0 Done Sent Done false
4433 2228 Acked Read 10 Done Sent Done false
4434 2229 Acked Write 3 Done Sent Done false
4435 2230 Acked Read 10 Done Sent Done false
4436 2231 Acked Read 9 Done Sent Done false
4437 2232 Acked Write 3 Done Sent Done false
4438 2233 Acked Flush 0 Done Sent Done false
4439 2234 Acked Read 10 Done Sent Done false
4440 2235 Acked Write 4 Done Sent Done false
4441 2236 Acked Write 9 Done Sent Done false
4442 2237 Acked Write 6 Done Sent Done false
4443 2238 Acked Read 1 Done Sent Done false
4444 2239 Acked Read 6 Done Sent Done false
4445 2240 Acked Read 3 Done Sent Done false
4446 2241 Acked Flush 0 Done Sent Done false
4447 2242 Acked Read 5 Done Sent Done false
4448 2243 Acked Read 1 Done Sent Done false
4449 2244 Acked Read 5 Done Sent Done false
4450 2245 Acked Read 2 Done Sent Done false
4451 2246 Acked Read 7 Done Sent Done false
4452 2247 Acked Read 8 Done Sent Done false
4453 2248 Acked Write 9 Done Sent Done false
4454 2249 Acked Write 7 Done Sent Done false
4455 2250 Acked Read 4 Done Sent Done false
4456 2251 Acked Read 6 Done Sent Done false
4457 2252 Acked Write 3 Done Sent Done false
4458 2253 Acked Flush 0 Done Sent Done false
4459 2254 Acked Write 1 Done Sent Done false
4460 2255 Acked Write 1 Done Sent Done false
4461 2256 Acked Write 4 Done Sent Done false
4462 2257 Acked Read 2 Done Sent Done false
4463 2258 Acked Write 7 Done Sent Done false
4464 2259 Acked Write 8 Done Sent Done false
4465 2260 Acked Read 9 Done Sent Done false
4466 2261 Acked Write 1 Done Sent Done false
4467 2262 Acked Read 5 Done Sent Done false
4468 2263 Acked Flush 0 Done Sent Done false
4469 2264 Acked Write 3 Done Sent Done false
4470 2265 Acked Write 1 Done Sent Done false
4471 2266 Acked Write 2 Done Sent Done false
4472 2267 Acked Write 7 Done Sent Done false
4473 2268 Acked Write 8 Done Sent Done false
4474 2269 Acked Read 9 Done Sent Done false
4475 2270 Acked Write 6 Done Sent Done false
4476 2271 Acked Write 8 Done Sent Done false
4477 2272 Acked Write 10 Done Sent Done false
4478 2273 Acked Read 4 Done Sent Done false
4479 2274 Acked Write 3 Done Sent Done false
4480 2275 Acked Write 10 Done Sent Done false
4481 2276 Acked Flush 0 Done Sent Done false
4482 2277 Acked Read 4 Done Sent Done false
4483 2278 Acked Write 4 Done Sent Done false
4484 2279 Acked Write 2 Done Sent Done false
4485 2280 Acked Read 9 Done Sent Done false
4486 2281 Acked Read 5 Done Sent Done false
4487 2282 Acked Write 10 Done Sent Done false
4488 2283 Acked Read 8 Done Sent Done false
4489 2284 Acked Read 9 Done Sent Done false
4490 2285 Acked Read 2 Done Sent Done false
4491 2286 Acked Write 10 Done Sent Done false
4492 2287 Acked Write 5 Done Sent Done false
4493 2288 Acked Read 4 Done Sent Done false
4494 2289 Acked Flush 0 Done Sent Done false
4495 2290 Acked Write 7 Done Sent Done false
4496 2291 Acked Read 2 Done Sent Done false
4497 2292 Acked Read 1 Done Sent Done false
4498 2293 Acked Write 2 Done Sent Done false
4499 2294 Acked Read 6 Done Sent Done false
4500 2295 Acked Read 8 Done Sent Done false
4501 2296 Acked Read 4 Done Sent Done false
4502 2297 Acked Write 4 Done Sent Done false
4503 2298 Acked Write 9 Done Sent Done false
4504 2299 Acked Read 6 Done Sent Done false
4505 2300 Acked Read 6 Done Sent Done false
4506 2301 Acked Read 4 Done Sent Done false
4507 2302 Acked Read 8 Done Sent Done false
4508 2303 Acked Write 10 Done Sent Done false
4509 2304 Acked Flush 0 Done Sent Done false
4510 2305 Acked Read 3 Done Sent Done false
4511 2306 Acked Write 9 Done Sent Done false
4512 2307 Acked Read 8 Done Sent Done false
4513 2308 Acked Read 8 Done Sent Done false
4514 2309 Acked Read 5 Done Sent Done false
4515 2310 Acked Read 1 Done Sent Done false
4516 2311 Acked Flush 0 Done Sent Done false
4517 2312 Acked Flush 0 Done Sent Done false
4518 2313 Acked Read 6 Done Sent Done false
4519 2314 Acked Write 7 Done Sent Done false
4520 2315 Acked Read 7 Done Sent Done false
4521 2316 Acked Write 6 Done Sent Done false
4522 2317 Acked Flush 0 Done Sent Done false
4523 2318 Acked Read 4 Done Sent Done false
4524 2319 Acked Read 9 Done Sent Done false
4525 2320 Acked Read 7 Done Sent Done false
4526 2321 Acked Read 1 Done Sent Done false
4527 2322 Acked Flush 0 Done Sent Done false
4528 2323 Acked Flush 0 Done Sent Done false
4529 2324 Acked Read 1 Done Sent Done false
4530 2325 Acked Read 4 Done Sent Done false
4531 2326 Acked Read 3 Done Sent Done false
4532 2327 Acked Write 9 Done Sent Done false
4533 2328 Acked Read 1 Done Sent Done false
4534 2329 Acked Read 1 Done Sent Done false
4535 2330 Acked Write 8 Done Sent Done false
4536 2331 Acked Write 2 Done Sent Done false
4537 2332 Acked Read 6 Done Sent Done false
4538 2333 Acked Write 3 Done Sent Done false
4539 2334 Acked Write 4 Done Sent Done false
4540 2335 Acked Read 6 Done Sent Done false
4541 2336 Acked Write 9 Done Sent Done false
4542 2337 Acked Read 8 Done Sent Done false
4543 2338 Acked Write 7 Done Sent Done false
4544 2339 Acked Write 3 Done Sent Done false
4545 2340 Acked Read 9 Done Sent Done false
4546 2341 Acked Read 7 Done Sent Done false
4547 2342 Acked Read 6 Done Sent Done false
4548 2343 Acked Write 5 Done Sent Done false
4549 2344 Acked Write 1 Done Sent Done false
4550 2345 Acked Write 3 Done Sent Done false
4551 2346 Acked Flush 0 Done Sent Done false
4552 2347 Acked Write 6 Done Sent Done false
4553 2348 Acked Read 7 Done Sent Done false
4554 2349 Acked Read 9 Done Sent Done false
4555 2350 Acked Flush 0 Done Sent Done false
4556 2351 Acked Read 6 Done Sent Done false
4557 2352 Acked Read 10 Done Sent Done false
4558 2353 Acked Read 3 Done Sent Done false
4559 2354 Acked Write 10 Done Sent Done false
4560 2355 Acked Write 3 Done Sent Done false
4561 2356 Acked Write 10 Done Sent Done false
4562 2357 Acked Write 7 Done Sent Done false
4563 2358 Acked Read 5 Done Sent Done false
4564 2359 Acked Read 3 Done Sent Done false
4565 2360 Acked Read 1 Done Sent Done false
4566 2361 Acked Read 3 Done Sent Done false
4567 2362 Acked Write 10 Done Sent Done false
4568 2363 Acked Write 4 Done Sent Done false
4569 2364 Acked Read 8 Done Sent Done false
4570 2365 Acked Write 3 Done Sent Done false
4571 2366 Acked Read 5 Done Sent Done false
4572 2367 Acked Write 6 Done Sent Done false
4573 2368 Acked Write 3 Done Sent Done false
4574 2369 Acked Read 7 Done Sent Done false
4575 2370 Acked Read 9 Done Sent Done false
4576 2371 Acked Write 9 Done Sent Done false
4577 2372 Acked Read 7 Done Sent Done false
4578 2373 Acked Read 5 Done Sent Done false
4579 2374 Acked Write 3 Done Sent Done false
4580 2375 Acked Read 8 Done Sent Done false
4581 2376 Acked Write 7 Done Sent Done false
4582 2377 Acked Read 6 Done Sent Done false
4583 2378 Acked Write 8 Done Sent Done false
4584 2379 Acked Write 4 Done Sent Done false
4585 2380 Acked Flush 0 Done Sent Done false
4586 2381 Acked Write 6 Done Sent Done false
4587 2382 Acked Read 4 Done Sent Done false
4588 2383 Acked Read 10 Done Sent Done false
4589 2384 Acked Read 7 Done Sent Done false
4590 2385 Acked Read 1 Done Sent Done false
4591 2386 Acked Read 9 Done Sent Done false
4592 2387 Acked Write 7 Done Sent Done false
4593 2388 Acked Write 9 Done Sent Done false
4594 2389 Acked Write 4 Done Sent Done false
4595 2390 Acked Write 4 Done Sent Done false
4596 2391 Acked Write 8 Done Sent Done false
4597 2392 Acked Flush 0 Done Sent Done false
4598 2393 Acked Flush 0 Done Sent Done false
4599 2394 Acked Read 9 Done Sent Done false
4600 2395 Acked Write 4 Done Sent Done false
4601 2396 Acked Write 3 Done Sent Done false
4602 2397 Acked Read 5 Done Sent Done false
4603 2398 Acked Flush 0 Done Sent Done false
4604 2399 Acked Read 2 Done Sent Done false
4605 2400 Acked Write 5 Done Sent Done false
4606 2401 Acked Write 4 Done Sent Done false
4607 2402 Acked Write 4 Done Sent Done false
4608 2403 Acked Read 5 Done Sent Done false
4609 2404 Acked Read 10 Done Sent Done false
4610 2405 Acked Read 5 Done Sent Done false
4611 2406 Acked Read 8 Done Sent Done false
4612 2407 Acked Read 4 Done Sent Done false
4613 2408 Acked Write 9 Done Sent Done false
4614 2409 Acked Read 10 Done Sent Done false
4615 2410 Acked Read 9 Done Sent Done false
4616 2411 Acked Read 10 Done Sent Done false
4617 2412 Acked Write 8 Done Sent Done false
4618 2413 Acked Write 4 Done Sent Done false
4619 2414 Acked Write 1 Done Sent Done false
4620 2415 Acked Read 2 Done Sent Done false
4621 2416 Acked Flush 0 Done Sent Done false
4622 2417 Acked Write 5 Done Sent Done false
4623 2418 Acked Write 3 Done Sent Done false
4624 2419 Acked Write 3 Done Sent Done false
4625 2420 Acked Write 7 Done Sent Done false
4626 2421 Acked Write 1 Done Sent Done false
4627 2422 Acked Read 9 Done Sent Done false
4628 2423 Acked Read 9 Done Sent Done false
4629 2424 Acked Read 2 Done Sent Done false
4630 2425 Acked Read 8 Done Sent Done false
4631 2426 Acked Read 6 Done Sent Done false
4632 2427 Acked Flush 0 Done Sent Done false
4633 2428 Acked Read 4 Done Sent Done false
4634 2429 Acked Write 5 Done Sent Done false
4635 2430 Acked Write 7 Done Sent Done false
4636 2431 Acked Read 2 Done Sent Done false
4637 2432 Acked Flush 0 Done Sent Done false
4638 2433 Acked Read 10 Done Sent Done false
4639 2434 Acked Write 5 Done Sent Done false
4640 2435 Acked Read 4 Done Sent Done false
4641 2436 Acked Write 9 Done Sent Done false
4642 2437 Acked Write 6 Done Sent Done false
4643 2438 Acked Write 2 Done Sent Done false
4644 2439 Acked Write 9 Done Sent Done false
4645 2440 Acked Write 6 Done Sent Done false
4646 2441 Acked Read 7 Done Sent Done false
4647 2442 Acked Flush 0 Done Sent Done false
4648 2443 Acked Read 6 Done Sent Done false
4649 2444 Acked Read 7 Done Sent Done false
4650 2445 Acked Write 8 Done Sent Done false
4651 2446 Acked Write 8 Done Sent Done false
4652 2447 Acked Flush 0 Done Sent Done false
4653 2448 Acked Write 1 Done Sent Done false
4654 2449 Acked Read 10 Done Sent Done false
4655 2450 Acked Read 7 Done Sent Done false
4656 2451 Acked Read 5 Done Sent Done false
4657 2452 Acked Read 4 Done Sent Done false
4658 2453 Acked Write 4 Done Sent Done false
4659 2454 Acked Read 10 Done Sent Done false
4660 2455 Acked Write 7 Done Sent Done false
4661 2456 Acked Write 2 Done Sent Done false
4662 2457 Acked Write 8 Done Sent Done false
4663 2458 Acked Read 1 Done Sent Done false
4664 2459 Acked Write 10 Done Sent Done false
4665 2460 Acked Write 5 Done Sent Done false
4666 2461 Acked Read 4 Done Sent Done false
4667 2462 Acked Read 6 Done Sent Done false
4668 2463 Acked Read 2 Done Sent Done false
4669 2464 Acked Write 5 Done Sent Done false
4670 2465 Acked Read 4 Done Sent Done false
4671 2466 Acked Read 9 Done Sent Done false
4672 2467 Acked Read 10 Done Sent Done false
4673 2468 Acked Read 10 Done Sent Done false
4674 2469 Acked Write 2 Done Sent Done false
4675 2470 Acked Flush 0 Done Sent Done false
4676 2471 Acked Write 4 Done Sent Done false
4677 2472 Acked Write 1 Done Sent Done false
4678 2473 Acked Read 2 Done Sent Done false
4679 2474 Acked Read 6 Done Sent Done false
4680 2475 Acked Read 4 Done Sent Done false
4681 2476 Acked Flush 0 Done Sent Done false
4682 2477 Acked Write 9 Done Sent Done false
4683 2478 Acked Read 7 Done Sent Done false
4684 2479 Acked Write 6 Done Sent Done false
4685 2480 Acked Write 3 Done Sent Done false
4686 2481 Acked Write 2 Done Sent Done false
4687 2482 Acked Read 9 Done Sent Done false
4688 2483 Acked Write 8 Done Sent Done false
4689 2484 Acked Write 4 Done Sent Done false
4690 2485 Acked Read 4 Done Sent Done false
4691 2486 Acked Write 3 Done Sent Done false
4692 2487 Acked Flush 0 Done Sent Done false
4693 2488 Acked Write 5 Done Sent Done false
4694 2489 Acked Flush 0 Done Sent Done false
4695 2490 Acked Flush 0 Done Sent Done false
4696 2491 Acked Read 1 Done Sent Done false
4697 2492 Acked Read 8 Done Sent Done false
4698 2493 Acked Write 8 Done Sent Done false
4699 2494 Acked Write 8 Done Sent Done false
4700 2495 Acked Write 4 Done Sent Done false
4701 2496 Acked Write 3 Done Sent Done false
4702 2497 Acked Read 1 Done Sent Done false
4703 2498 Acked Read 4 Done Sent Done false
4704 2499 Acked Write 6 Done Sent Done false
4705 2500 Acked Flush 0 Done Sent Done false
4706 2501 Acked Write 9 Done Sent Done false
4707 2502 Acked Read 5 Done Sent Done false
4708 2503 Acked Write 4 Done Sent Done false
4709 2504 Acked Read 8 Done Sent Done false
4710 2505 Acked Write 7 Done Sent Done false
4711 2506 Acked Write 7 Done Sent Done false
4712 2507 Acked Write 8 Done Sent Done false
4713 2508 Acked Write 7 Done Sent Sent false
4714 STATES DS:0 DS:1 DS:2 TOTAL
4715 Sent 0 301 1 302
4716 Done 301 0 300 601
4717 Skipped 0 0 0 0
4718 Error 0 0 0 0
4719 Last Flush: 2500 2207 2500
4720 Downstairs last five completed: 2207 2206 2205 2204 2203
4721 Upstairs last five completed: 2508 2507 2506 2505 2504
4722 CLIENT: Up:0 ds:301 act:2
4723 ----------------------------------------------------------------
4724 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
4725 States: Active Offline Active
4726 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4727 2208 Acked Write 2 Done Sent Done false
4728 2209 Acked Read 2 Done Sent Done false
4729 2210 Acked Write 10 Done Sent Done false
4730 2211 Acked Write 8 Done Sent Done false
4731 2212 Acked Flush 0 Done Sent Done false
4732 2213 Acked Write 8 Done Sent Done false
4733 2214 Acked Read 1 Done Sent Done false
4734 2215 Acked Write 4 Done Sent Done false
4735 2216 Acked Read 7 Done Sent Done false
4736 2217 Acked Read 1 Done Sent Done false
4737 2218 Acked Flush 0 Done Sent Done false
4738 2219 Acked Read 8 Done Sent Done false
4739 2220 Acked Write 10 Done Sent Done false
4740 2221 Acked Read 8 Done Sent Done false
4741 2222 Acked Read 2 Done Sent Done false
4742 2223 Acked Read 9 Done Sent Done false
4743 2224 Acked Read 2 Done Sent Done false
4744 2225 Acked Read 10 Done Sent Done false
4745 2226 Acked Write 9 Done Sent Done false
4746 2227 Acked Flush 0 Done Sent Done false
4747 2228 Acked Read 10 Done Sent Done false
4748 2229 Acked Write 3 Done Sent Done false
4749 2230 Acked Read 10 Done Sent Done false
4750 2231 Acked Read 9 Done Sent Done false
4751 2232 Acked Write 3 Done Sent Done false
4752 2233 Acked Flush 0 Done Sent Done false
4753 2234 Acked Read 10 Done Sent Done false
4754 2235 Acked Write 4 Done Sent Done false
4755 2236 Acked Write 9 Done Sent Done false
4756 2237 Acked Write 6 Done Sent Done false
4757 2238 Acked Read 1 Done Sent Done false
4758 2239 Acked Read 6 Done Sent Done false
4759 2240 Acked Read 3 Done Sent Done false
4760 2241 Acked Flush 0 Done Sent Done false
4761 2242 Acked Read 5 Done Sent Done false
4762 2243 Acked Read 1 Done Sent Done false
4763 2244 Acked Read 5 Done Sent Done false
4764 2245 Acked Read 2 Done Sent Done false
4765 2246 Acked Read 7 Done Sent Done false
4766 2247 Acked Read 8 Done Sent Done false
4767 2248 Acked Write 9 Done Sent Done false
4768 2249 Acked Write 7 Done Sent Done false
4769 2250 Acked Read 4 Done Sent Done false
4770 2251 Acked Read 6 Done Sent Done false
4771 2252 Acked Write 3 Done Sent Done false
4772 2253 Acked Flush 0 Done Sent Done false
4773 2254 Acked Write 1 Done Sent Done false
4774 2255 Acked Write 1 Done Sent Done false
4775 2256 Acked Write 4 Done Sent Done false
4776 2257 Acked Read 2 Done Sent Done false
4777 2258 Acked Write 7 Done Sent Done false
4778 2259 Acked Write 8 Done Sent Done false
4779 2260 Acked Read 9 Done Sent Done false
4780 2261 Acked Write 1 Done Sent Done false
4781 2262 Acked Read 5 Done Sent Done false
4782 2263 Acked Flush 0 Done Sent Done false
4783 2264 Acked Write 3 Done Sent Done false
4784 2265 Acked Write 1 Done Sent Done false
4785 2266 Acked Write 2 Done Sent Done false
4786 2267 Acked Write 7 Done Sent Done false
4787 2268 Acked Write 8 Done Sent Done false
4788 2269 Acked Read 9 Done Sent Done false
4789 2270 Acked Write 6 Done Sent Done false
4790 2271 Acked Write 8 Done Sent Done false
4791 2272 Acked Write 10 Done Sent Done false
4792 2273 Acked Read 4 Done Sent Done false
4793 2274 Acked Write 3 Done Sent Done false
4794 2275 Acked Write 10 Done Sent Done false
4795 2276 Acked Flush 0 Done Sent Done false
4796 2277 Acked Read 4 Done Sent Done false
4797 2278 Acked Write 4 Done Sent Done false
4798 2279 Acked Write 2 Done Sent Done false
4799 2280 Acked Read 9 Done Sent Done false
4800 2281 Acked Read 5 Done Sent Done false
4801 2282 Acked Write 10 Done Sent Done false
4802 2283 Acked Read 8 Done Sent Done false
4803 2284 Acked Read 9 Done Sent Done false
4804 2285 Acked Read 2 Done Sent Done false
4805 2286 Acked Write 10 Done Sent Done false
4806 2287 Acked Write 5 Done Sent Done false
4807 2288 Acked Read 4 Done Sent Done false
4808 2289 Acked Flush 0 Done Sent Done false
4809 2290 Acked Write 7 Done Sent Done false
4810 2291 Acked Read 2 Done Sent Done false
4811 2292 Acked Read 1 Done Sent Done false
4812 2293 Acked Write 2 Done Sent Done false
4813 2294 Acked Read 6 Done Sent Done false
4814 2295 Acked Read 8 Done Sent Done false
4815 2296 Acked Read 4 Done Sent Done false
4816 2297 Acked Write 4 Done Sent Done false
4817 2298 Acked Write 9 Done Sent Done false
4818 2299 Acked Read 6 Done Sent Done false
4819 2300 Acked Read 6 Done Sent Done false
4820 2301 Acked Read 4 Done Sent Done false
4821 2302 Acked Read 8 Done Sent Done false
4822 2303 Acked Write 10 Done Sent Done false
4823 2304 Acked Flush 0 Done Sent Done false
4824 2305 Acked Read 3 Done Sent Done false
4825 2306 Acked Write 9 Done Sent Done false
4826 2307 Acked Read 8 Done Sent Done false
4827 2308 Acked Read 8 Done Sent Done false
4828 2309 Acked Read 5 Done Sent Done false
4829 2310 Acked Read 1 Done Sent Done false
4830 2311 Acked Flush 0 Done Sent Done false
4831 2312 Acked Flush 0 Done Sent Done false
4832 2313 Acked Read 6 Done Sent Done false
4833 2314 Acked Write 7 Done Sent Done false
4834 2315 Acked Read 7 Done Sent Done false
4835 2316 Acked Write 6 Done Sent Done false
4836 2317 Acked Flush 0 Done Sent Done false
4837 2318 Acked Read 4 Done Sent Done false
4838 2319 Acked Read 9 Done Sent Done false
4839 2320 Acked Read 7 Done Sent Done false
4840 2321 Acked Read 1 Done Sent Done false
4841 2322 Acked Flush 0 Done Sent Done false
4842 2323 Acked Flush 0 Done Sent Done false
4843 2324 Acked Read 1 Done Sent Done false
4844 2325 Acked Read 4 Done Sent Done false
4845 2326 Acked Read 3 Done Sent Done false
4846 2327 Acked Write 9 Done Sent Done false
4847 2328 Acked Read 1 Done Sent Done false
4848 2329 Acked Read 1 Done Sent Done false
4849 2330 Acked Write 8 Done Sent Done false
4850 2331 Acked Write 2 Done Sent Done false
4851 2332 Acked Read 6 Done Sent Done false
4852 2333 Acked Write 3 Done Sent Done false
4853 2334 Acked Write 4 Done Sent Done false
4854 2335 Acked Read 6 Done Sent Done false
4855 2336 Acked Write 9 Done Sent Done false
4856 2337 Acked Read 8 Done Sent Done false
4857 2338 Acked Write 7 Done Sent Done false
4858 2339 Acked Write 3 Done Sent Done false
4859 2340 Acked Read 9 Done Sent Done false
4860 2341 Acked Read 7 Done Sent Done false
4861 2342 Acked Read 6 Done Sent Done false
4862 2343 Acked Write 5 Done Sent Done false
4863 2344 Acked Write 1 Done Sent Done false
4864 2345 Acked Write 3 Done Sent Done false
4865 2346 Acked Flush 0 Done Sent Done false
4866 2347 Acked Write 6 Done Sent Done false
4867 2348 Acked Read 7 Done Sent Done false
4868 2349 Acked Read 9 Done Sent Done false
4869 2350 Acked Flush 0 Done Sent Done false
4870 2351 Acked Read 6 Done Sent Done false
4871 2352 Acked Read 10 Done Sent Done false
4872 2353 Acked Read 3 Done Sent Done false
4873 2354 Acked Write 10 Done Sent Done false
4874 2355 Acked Write 3 Done Sent Done false
4875 2356 Acked Write 10 Done Sent Done false
4876 2357 Acked Write 7 Done Sent Done false
4877 2358 Acked Read 5 Done Sent Done false
4878 2359 Acked Read 3 Done Sent Done false
4879 2360 Acked Read 1 Done Sent Done false
4880 2361 Acked Read 3 Done Sent Done false
4881 2362 Acked Write 10 Done Sent Done false
4882 2363 Acked Write 4 Done Sent Done false
4883 2364 Acked Read 8 Done Sent Done false
4884 2365 Acked Write 3 Done Sent Done false
4885 2366 Acked Read 5 Done Sent Done false
4886 2367 Acked Write 6 Done Sent Done false
4887 2368 Acked Write 3 Done Sent Done false
4888 2369 Acked Read 7 Done Sent Done false
4889 2370 Acked Read 9 Done Sent Done false
4890 2371 Acked Write 9 Done Sent Done false
4891 2372 Acked Read 7 Done Sent Done false
4892 2373 Acked Read 5 Done Sent Done false
4893 2374 Acked Write 3 Done Sent Done false
4894 2375 Acked Read 8 Done Sent Done false
4895 2376 Acked Write 7 Done Sent Done false
4896 2377 Acked Read 6 Done Sent Done false
4897 2378 Acked Write 8 Done Sent Done false
4898 2379 Acked Write 4 Done Sent Done false
4899 2380 Acked Flush 0 Done Sent Done false
4900 2381 Acked Write 6 Done Sent Done false
4901 2382 Acked Read 4 Done Sent Done false
4902 2383 Acked Read 10 Done Sent Done false
4903 2384 Acked Read 7 Done Sent Done false
4904 2385 Acked Read 1 Done Sent Done false
4905 2386 Acked Read 9 Done Sent Done false
4906 2387 Acked Write 7 Done Sent Done false
4907 2388 Acked Write 9 Done Sent Done false
4908 2389 Acked Write 4 Done Sent Done false
4909 2390 Acked Write 4 Done Sent Done false
4910 2391 Acked Write 8 Done Sent Done false
4911 2392 Acked Flush 0 Done Sent Done false
4912 2393 Acked Flush 0 Done Sent Done false
4913 2394 Acked Read 9 Done Sent Done false
4914 2395 Acked Write 4 Done Sent Done false
4915 2396 Acked Write 3 Done Sent Done false
4916 2397 Acked Read 5 Done Sent Done false
4917 2398 Acked Flush 0 Done Sent Done false
4918 2399 Acked Read 2 Done Sent Done false
4919 2400 Acked Write 5 Done Sent Done false
4920 2401 Acked Write 4 Done Sent Done false
4921 2402 Acked Write 4 Done Sent Done false
4922 2403 Acked Read 5 Done Sent Done false
4923 2404 Acked Read 10 Done Sent Done false
4924 2405 Acked Read 5 Done Sent Done false
4925 2406 Acked Read 8 Done Sent Done false
4926 2407 Acked Read 4 Done Sent Done false
4927 2408 Acked Write 9 Done Sent Done false
4928 2409 Acked Read 10 Done Sent Done false
4929 2410 Acked Read 9 Done Sent Done false
4930 2411 Acked Read 10 Done Sent Done false
4931 2412 Acked Write 8 Done Sent Done false
4932 2413 Acked Write 4 Done Sent Done false
4933 2414 Acked Write 1 Done Sent Done false
4934 2415 Acked Read 2 Done Sent Done false
4935 2416 Acked Flush 0 Done Sent Done false
4936 2417 Acked Write 5 Done Sent Done false
4937 2418 Acked Write 3 Done Sent Done false
4938 2419 Acked Write 3 Done Sent Done false
4939 2420 Acked Write 7 Done Sent Done false
4940 2421 Acked Write 1 Done Sent Done false
4941 2422 Acked Read 9 Done Sent Done false
4942 2423 Acked Read 9 Done Sent Done false
4943 2424 Acked Read 2 Done Sent Done false
4944 2425 Acked Read 8 Done Sent Done false
4945 2426 Acked Read 6 Done Sent Done false
4946 2427 Acked Flush 0 Done Sent Done false
4947 2428 Acked Read 4 Done Sent Done false
4948 2429 Acked Write 5 Done Sent Done false
4949 2430 Acked Write 7 Done Sent Done false
4950 2431 Acked Read 2 Done Sent Done false
4951 2432 Acked Flush 0 Done Sent Done false
4952 2433 Acked Read 10 Done Sent Done false
4953 2434 Acked Write 5 Done Sent Done false
4954 2435 Acked Read 4 Done Sent Done false
4955 2436 Acked Write 9 Done Sent Done false
4956 2437 Acked Write 6 Done Sent Done false
4957 2438 Acked Write 2 Done Sent Done false
4958 2439 Acked Write 9 Done Sent Done false
4959 2440 Acked Write 6 Done Sent Done false
4960 2441 Acked Read 7 Done Sent Done false
4961 2442 Acked Flush 0 Done Sent Done false
4962 2443 Acked Read 6 Done Sent Done false
4963 2444 Acked Read 7 Done Sent Done false
4964 2445 Acked Write 8 Done Sent Done false
4965 2446 Acked Write 8 Done Sent Done false
4966 2447 Acked Flush 0 Done Sent Done false
4967 2448 Acked Write 1 Done Sent Done false
4968 2449 Acked Read 10 Done Sent Done false
4969 2450 Acked Read 7 Done Sent Done false
4970 2451 Acked Read 5 Done Sent Done false
4971 2452 Acked Read 4 Done Sent Done false
4972 2453 Acked Write 4 Done Sent Done false
4973 2454 Acked Read 10 Done Sent Done false
4974 2455 Acked Write 7 Done Sent Done false
4975 2456 Acked Write 2 Done Sent Done false
4976 2457 Acked Write 8 Done Sent Done false
4977 2458 Acked Read 1 Done Sent Done false
4978 2459 Acked Write 10 Done Sent Done false
4979 2460 Acked Write 5 Done Sent Done false
4980 2461 Acked Read 4 Done Sent Done false
4981 2462 Acked Read 6 Done Sent Done false
4982 2463 Acked Read 2 Done Sent Done false
4983 2464 Acked Write 5 Done Sent Done false
4984 2465 Acked Read 4 Done Sent Done false
4985 2466 Acked Read 9 Done Sent Done false
4986 2467 Acked Read 10 Done Sent Done false
4987 2468 Acked Read 10 Done Sent Done false
4988 2469 Acked Write 2 Done Sent Done false
4989 2470 Acked Flush 0 Done Sent Done false
4990 2471 Acked Write 4 Done Sent Done false
4991 2472 Acked Write 1 Done Sent Done false
4992 2473 Acked Read 2 Done Sent Done false
4993 2474 Acked Read 6 Done Sent Done false
4994 2475 Acked Read 4 Done Sent Done false
4995 2476 Acked Flush 0 Done Sent Done false
4996 2477 Acked Write 9 Done Sent Done false
4997 2478 Acked Read 7 Done Sent Done false
4998 2479 Acked Write 6 Done Sent Done false
4999 2480 Acked Write 3 Done Sent Done false
5000 2481 Acked Write 2 Done Sent Done false
5001 2482 Acked Read 9 Done Sent Done false
5002 2483 Acked Write 8 Done Sent Done false
5003 2484 Acked Write 4 Done Sent Done false
5004 2485 Acked Read 4 Done Sent Done false
5005 2486 Acked Write 3 Done Sent Done false
5006 2487 Acked Flush 0 Done Sent Done false
5007 2488 Acked Write 5 Done Sent Done false
5008 2489 Acked Flush 0 Done Sent Done false
5009 2490 Acked Flush 0 Done Sent Done false
5010 2491 Acked Read 1 Done Sent Done false
5011 2492 Acked Read 8 Done Sent Done false
5012 2493 Acked Write 8 Done Sent Done false
5013 2494 Acked Write 8 Done Sent Done false
5014 2495 Acked Write 4 Done Sent Done false
5015 2496 Acked Write 3 Done Sent Done false
5016 2497 Acked Read 1 Done Sent Done false
5017 2498 Acked Read 4 Done Sent Done false
5018 2499 Acked Write 6 Done Sent Done false
5019 2500 Acked Flush 0 Done Sent Done false
5020 2501 Acked Write 9 Done Sent Done false
5021 2502 Acked Read 5 Done Sent Done false
5022 2503 Acked Write 4 Done Sent Done false
5023 2504 Acked Read 8 Done Sent Done false
5024 2505 Acked Write 7 Done Sent Done false
5025 2506 Acked Write 7 Done Sent Done false
5026 2507 Acked Write 8 Done Sent Done false
5027 2508 Acked Write 7 Done Sent Done false
5028 2509 Acked Flush 0 Done Sent Done false
5029 STATES DS:0 DS:1 DS:2 TOTAL
5030 Sent 0 302 0 302
5031 Done 302 0 302 604
5032 Skipped 0 0 0 0
5033 Error 0 0 0 0
5034 Last Flush: 2509 2207 2509
5035 Downstairs last five completed: 2207 2206 2205 2204 2203
5036 Upstairs last five completed: 2509 2508 2507 2506 2505
5037 CLIENT: Up:0 ds:302 act:2
50382025-01-13T15:52:50.042ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
50392025-01-13T15:52:50.042ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
50402025-01-13T15:52:50.042ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
50412025-01-13T15:52:50.042ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
50422025-01-13T15:52:50.043ZINFOcrucible: client task is exiting = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
5043 [8820] got start action
5044 [8820] I am going to restart
5045 Starting downstairs at port 8820
5046 Make output file at "/tmp/dsc/downstairs-8820.txt"
5047 [8820][1] reports Starting
5048 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1207)
5049 [8820][1] reports Running
5050 ----------------------------------------------------------------
5051 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
5052 States: Active Offline Active
5053 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5054 2208 Acked Write 2 Done Sent Done false
5055 2209 Acked Read 2 Done Sent Done false
5056 2210 Acked Write 10 Done Sent Done false
5057 2211 Acked Write 8 Done Sent Done false
5058 2212 Acked Flush 0 Done Sent Done false
5059 2213 Acked Write 8 Done Sent Done false
5060 2214 Acked Read 1 Done Sent Done false
5061 2215 Acked Write 4 Done Sent Done false
5062 2216 Acked Read 7 Done Sent Done false
5063 2217 Acked Read 1 Done Sent Done false
5064 2218 Acked Flush 0 Done Sent Done false
5065 2219 Acked Read 8 Done Sent Done false
5066 2220 Acked Write 10 Done Sent Done false
5067 2221 Acked Read 8 Done Sent Done false
5068 2222 Acked Read 2 Done Sent Done false
5069 2223 Acked Read 9 Done Sent Done false
5070 2224 Acked Read 2 Done Sent Done false
5071 2225 Acked Read 10 Done Sent Done false
5072 2226 Acked Write 9 Done Sent Done false
5073 2227 Acked Flush 0 Done Sent Done false
5074 2228 Acked Read 10 Done Sent Done false
5075 2229 Acked Write 3 Done Sent Done false
5076 2230 Acked Read 10 Done Sent Done false
5077 2231 Acked Read 9 Done Sent Done false
5078 2232 Acked Write 3 Done Sent Done false
5079 2233 Acked Flush 0 Done Sent Done false
5080 2234 Acked Read 10 Done Sent Done false
5081 2235 Acked Write 4 Done Sent Done false
5082 2236 Acked Write 9 Done Sent Done false
5083 2237 Acked Write 6 Done Sent Done false
5084 2238 Acked Read 1 Done Sent Done false
5085 2239 Acked Read 6 Done Sent Done false
5086 2240 Acked Read 3 Done Sent Done false
5087 2241 Acked Flush 0 Done Sent Done false
5088 2242 Acked Read 5 Done Sent Done false
5089 2243 Acked Read 1 Done Sent Done false
5090 2244 Acked Read 5 Done Sent Done false
5091 2245 Acked Read 2 Done Sent Done false
5092 2246 Acked Read 7 Done Sent Done false
5093 2247 Acked Read 8 Done Sent Done false
5094 2248 Acked Write 9 Done Sent Done false
5095 2249 Acked Write 7 Done Sent Done false
5096 2250 Acked Read 4 Done Sent Done false
5097 2251 Acked Read 6 Done Sent Done false
5098 2252 Acked Write 3 Done Sent Done false
5099 2253 Acked Flush 0 Done Sent Done false
5100 2254 Acked Write 1 Done Sent Done false
5101 2255 Acked Write 1 Done Sent Done false
5102 2256 Acked Write 4 Done Sent Done false
5103 2257 Acked Read 2 Done Sent Done false
5104 2258 Acked Write 7 Done Sent Done false
5105 2259 Acked Write 8 Done Sent Done false
5106 2260 Acked Read 9 Done Sent Done false
5107 2261 Acked Write 1 Done Sent Done false
5108 2262 Acked Read 5 Done Sent Done false
5109 2263 Acked Flush 0 Done Sent Done false
5110 2264 Acked Write 3 Done Sent Done false
5111 2265 Acked Write 1 Done Sent Done false
5112 2266 Acked Write 2 Done Sent Done false
5113 2267 Acked Write 7 Done Sent Done false
5114 2268 Acked Write 8 Done Sent Done false
5115 2269 Acked Read 9 Done Sent Done false
5116 2270 Acked Write 6 Done Sent Done false
5117 2271 Acked Write 8 Done Sent Done false
5118 2272 Acked Write 10 Done Sent Done false
5119 2273 Acked Read 4 Done Sent Done false
5120 2274 Acked Write 3 Done Sent Done false
5121 2275 Acked Write 10 Done Sent Done false
5122 2276 Acked Flush 0 Done Sent Done false
5123 2277 Acked Read 4 Done Sent Done false
5124 2278 Acked Write 4 Done Sent Done false
5125 2279 Acked Write 2 Done Sent Done false
5126 2280 Acked Read 9 Done Sent Done false
5127 2281 Acked Read 5 Done Sent Done false
5128 2282 Acked Write 10 Done Sent Done false
5129 2283 Acked Read 8 Done Sent Done false
5130 2284 Acked Read 9 Done Sent Done false
5131 2285 Acked Read 2 Done Sent Done false
5132 2286 Acked Write 10 Done Sent Done false
5133 2287 Acked Write 5 Done Sent Done false
5134 2288 Acked Read 4 Done Sent Done false
5135 2289 Acked Flush 0 Done Sent Done false
5136 2290 Acked Write 7 Done Sent Done false
5137 2291 Acked Read 2 Done Sent Done false
5138 2292 Acked Read 1 Done Sent Done false
5139 2293 Acked Write 2 Done Sent Done false
5140 2294 Acked Read 6 Done Sent Done false
5141 2295 Acked Read 8 Done Sent Done false
5142 2296 Acked Read 4 Done Sent Done false
5143 2297 Acked Write 4 Done Sent Done false
5144 2298 Acked Write 9 Done Sent Done false
5145 2299 Acked Read 6 Done Sent Done false
5146 2300 Acked Read 6 Done Sent Done false
5147 2301 Acked Read 4 Done Sent Done false
5148 2302 Acked Read 8 Done Sent Done false
5149 2303 Acked Write 10 Done Sent Done false
5150 2304 Acked Flush 0 Done Sent Done false
5151 2305 Acked Read 3 Done Sent Done false
5152 2306 Acked Write 9 Done Sent Done false
5153 2307 Acked Read 8 Done Sent Done false
5154 2308 Acked Read 8 Done Sent Done false
5155 2309 Acked Read 5 Done Sent Done false
5156 2310 Acked Read 1 Done Sent Done false
5157 2311 Acked Flush 0 Done Sent Done false
5158 2312 Acked Flush 0 Done Sent Done false
5159 2313 Acked Read 6 Done Sent Done false
5160 2314 Acked Write 7 Done Sent Done false
5161 2315 Acked Read 7 Done Sent Done false
5162 2316 Acked Write 6 Done Sent Done false
5163 2317 Acked Flush 0 Done Sent Done false
5164 2318 Acked Read 4 Done Sent Done false
5165 2319 Acked Read 9 Done Sent Done false
5166 2320 Acked Read 7 Done Sent Done false
5167 2321 Acked Read 1 Done Sent Done false
5168 2322 Acked Flush 0 Done Sent Done false
5169 2323 Acked Flush 0 Done Sent Done false
5170 2324 Acked Read 1 Done Sent Done false
5171 2325 Acked Read 4 Done Sent Done false
5172 2326 Acked Read 3 Done Sent Done false
5173 2327 Acked Write 9 Done Sent Done false
5174 2328 Acked Read 1 Done Sent Done false
5175 2329 Acked Read 1 Done Sent Done false
5176 2330 Acked Write 8 Done Sent Done false
5177 2331 Acked Write 2 Done Sent Done false
5178 2332 Acked Read 6 Done Sent Done false
5179 2333 Acked Write 3 Done Sent Done false
5180 2334 Acked Write 4 Done Sent Done false
5181 2335 Acked Read 6 Done Sent Done false
5182 2336 Acked Write 9 Done Sent Done false
5183 2337 Acked Read 8 Done Sent Done false
5184 2338 Acked Write 7 Done Sent Done false
5185 2339 Acked Write 3 Done Sent Done false
5186 2340 Acked Read 9 Done Sent Done false
5187 2341 Acked Read 7 Done Sent Done false
5188 2342 Acked Read 6 Done Sent Done false
5189 2343 Acked Write 5 Done Sent Done false
5190 2344 Acked Write 1 Done Sent Done false
5191 2345 Acked Write 3 Done Sent Done false
5192 2346 Acked Flush 0 Done Sent Done false
5193 2347 Acked Write 6 Done Sent Done false
5194 2348 Acked Read 7 Done Sent Done false
5195 2349 Acked Read 9 Done Sent Done false
5196 2350 Acked Flush 0 Done Sent Done false
5197 2351 Acked Read 6 Done Sent Done false
5198 2352 Acked Read 10 Done Sent Done false
5199 2353 Acked Read 3 Done Sent Done false
5200 2354 Acked Write 10 Done Sent Done false
5201 2355 Acked Write 3 Done Sent Done false
5202 2356 Acked Write 10 Done Sent Done false
5203 2357 Acked Write 7 Done Sent Done false
5204 2358 Acked Read 5 Done Sent Done false
5205 2359 Acked Read 3 Done Sent Done false
5206 2360 Acked Read 1 Done Sent Done false
5207 2361 Acked Read 3 Done Sent Done false
5208 2362 Acked Write 10 Done Sent Done false
5209 2363 Acked Write 4 Done Sent Done false
5210 2364 Acked Read 8 Done Sent Done false
5211 2365 Acked Write 3 Done Sent Done false
5212 2366 Acked Read 5 Done Sent Done false
5213 2367 Acked Write 6 Done Sent Done false
5214 2368 Acked Write 3 Done Sent Done false
5215 2369 Acked Read 7 Done Sent Done false
5216 2370 Acked Read 9 Done Sent Done false
5217 2371 Acked Write 9 Done Sent Done false
5218 2372 Acked Read 7 Done Sent Done false
5219 2373 Acked Read 5 Done Sent Done false
5220 2374 Acked Write 3 Done Sent Done false
5221 2375 Acked Read 8 Done Sent Done false
5222 2376 Acked Write 7 Done Sent Done false
5223 2377 Acked Read 6 Done Sent Done false
5224 2378 Acked Write 8 Done Sent Done false
5225 2379 Acked Write 4 Done Sent Done false
5226 2380 Acked Flush 0 Done Sent Done false
5227 2381 Acked Write 6 Done Sent Done false
5228 2382 Acked Read 4 Done Sent Done false
5229 2383 Acked Read 10 Done Sent Done false
5230 2384 Acked Read 7 Done Sent Done false
5231 2385 Acked Read 1 Done Sent Done false
5232 2386 Acked Read 9 Done Sent Done false
5233 2387 Acked Write 7 Done Sent Done false
5234 2388 Acked Write 9 Done Sent Done false
5235 2389 Acked Write 4 Done Sent Done false
5236 2390 Acked Write 4 Done Sent Done false
5237 2391 Acked Write 8 Done Sent Done false
5238 2392 Acked Flush 0 Done Sent Done false
5239 2393 Acked Flush 0 Done Sent Done false
5240 2394 Acked Read 9 Done Sent Done false
5241 2395 Acked Write 4 Done Sent Done false
5242 2396 Acked Write 3 Done Sent Done false
5243 2397 Acked Read 5 Done Sent Done false
5244 2398 Acked Flush 0 Done Sent Done false
5245 2399 Acked Read 2 Done Sent Done false
5246 2400 Acked Write 5 Done Sent Done false
5247 2401 Acked Write 4 Done Sent Done false
5248 2402 Acked Write 4 Done Sent Done false
5249 2403 Acked Read 5 Done Sent Done false
5250 2404 Acked Read 10 Done Sent Done false
5251 2405 Acked Read 5 Done Sent Done false
5252 2406 Acked Read 8 Done Sent Done false
5253 2407 Acked Read 4 Done Sent Done false
5254 2408 Acked Write 9 Done Sent Done false
5255 2409 Acked Read 10 Done Sent Done false
5256 2410 Acked Read 9 Done Sent Done false
5257 2411 Acked Read 10 Done Sent Done false
5258 2412 Acked Write 8 Done Sent Done false
5259 2413 Acked Write 4 Done Sent Done false
5260 2414 Acked Write 1 Done Sent Done false
5261 2415 Acked Read 2 Done Sent Done false
5262 2416 Acked Flush 0 Done Sent Done false
5263 2417 Acked Write 5 Done Sent Done false
5264 2418 Acked Write 3 Done Sent Done false
5265 2419 Acked Write 3 Done Sent Done false
5266 2420 Acked Write 7 Done Sent Done false
5267 2421 Acked Write 1 Done Sent Done false
5268 2422 Acked Read 9 Done Sent Done false
5269 2423 Acked Read 9 Done Sent Done false
5270 2424 Acked Read 2 Done Sent Done false
5271 2425 Acked Read 8 Done Sent Done false
5272 2426 Acked Read 6 Done Sent Done false
5273 2427 Acked Flush 0 Done Sent Done false
5274 2428 Acked Read 4 Done Sent Done false
5275 2429 Acked Write 5 Done Sent Done false
5276 2430 Acked Write 7 Done Sent Done false
5277 2431 Acked Read 2 Done Sent Done false
5278 2432 Acked Flush 0 Done Sent Done false
5279 2433 Acked Read 10 Done Sent Done false
5280 2434 Acked Write 5 Done Sent Done false
5281 2435 Acked Read 4 Done Sent Done false
5282 2436 Acked Write 9 Done Sent Done false
5283 2437 Acked Write 6 Done Sent Done false
5284 2438 Acked Write 2 Done Sent Done false
5285 2439 Acked Write 9 Done Sent Done false
5286 2440 Acked Write 6 Done Sent Done false
5287 2441 Acked Read 7 Done Sent Done false
5288 2442 Acked Flush 0 Done Sent Done false
5289 2443 Acked Read 6 Done Sent Done false
5290 2444 Acked Read 7 Done Sent Done false
5291 2445 Acked Write 8 Done Sent Done false
5292 2446 Acked Write 8 Done Sent Done false
5293 2447 Acked Flush 0 Done Sent Done false
5294 2448 Acked Write 1 Done Sent Done false
5295 2449 Acked Read 10 Done Sent Done false
5296 2450 Acked Read 7 Done Sent Done false
5297 2451 Acked Read 5 Done Sent Done false
5298 2452 Acked Read 4 Done Sent Done false
5299 2453 Acked Write 4 Done Sent Done false
5300 2454 Acked Read 10 Done Sent Done false
5301 2455 Acked Write 7 Done Sent Done false
5302 2456 Acked Write 2 Done Sent Done false
5303 2457 Acked Write 8 Done Sent Done false
5304 2458 Acked Read 1 Done Sent Done false
5305 2459 Acked Write 10 Done Sent Done false
5306 2460 Acked Write 5 Done Sent Done false
5307 2461 Acked Read 4 Done Sent Done false
5308 2462 Acked Read 6 Done Sent Done false
5309 2463 Acked Read 2 Done Sent Done false
5310 2464 Acked Write 5 Done Sent Done false
5311 2465 Acked Read 4 Done Sent Done false
5312 2466 Acked Read 9 Done Sent Done false
5313 2467 Acked Read 10 Done Sent Done false
5314 2468 Acked Read 10 Done Sent Done false
5315 2469 Acked Write 2 Done Sent Done false
5316 2470 Acked Flush 0 Done Sent Done false
5317 2471 Acked Write 4 Done Sent Done false
5318 2472 Acked Write 1 Done Sent Done false
5319 2473 Acked Read 2 Done Sent Done false
5320 2474 Acked Read 6 Done Sent Done false
5321 2475 Acked Read 4 Done Sent Done false
5322 2476 Acked Flush 0 Done Sent Done false
5323 2477 Acked Write 9 Done Sent Done false
5324 2478 Acked Read 7 Done Sent Done false
5325 2479 Acked Write 6 Done Sent Done false
5326 2480 Acked Write 3 Done Sent Done false
5327 2481 Acked Write 2 Done Sent Done false
5328 2482 Acked Read 9 Done Sent Done false
5329 2483 Acked Write 8 Done Sent Done false
5330 2484 Acked Write 4 Done Sent Done false
5331 2485 Acked Read 4 Done Sent Done false
5332 2486 Acked Write 3 Done Sent Done false
5333 2487 Acked Flush 0 Done Sent Done false
5334 2488 Acked Write 5 Done Sent Done false
5335 2489 Acked Flush 0 Done Sent Done false
5336 2490 Acked Flush 0 Done Sent Done false
5337 2491 Acked Read 1 Done Sent Done false
5338 2492 Acked Read 8 Done Sent Done false
5339 2493 Acked Write 8 Done Sent Done false
5340 2494 Acked Write 8 Done Sent Done false
5341 2495 Acked Write 4 Done Sent Done false
5342 2496 Acked Write 3 Done Sent Done false
5343 2497 Acked Read 1 Done Sent Done false
5344 2498 Acked Read 4 Done Sent Done false
5345 2499 Acked Write 6 Done Sent Done false
5346 2500 Acked Flush 0 Done Sent Done false
5347 2501 Acked Write 9 Done Sent Done false
5348 2502 Acked Read 5 Done Sent Done false
5349 2503 Acked Write 4 Done Sent Done false
5350 2504 Acked Read 8 Done Sent Done false
5351 2505 Acked Write 7 Done Sent Done false
5352 2506 Acked Write 7 Done Sent Done false
5353 2507 Acked Write 8 Done Sent Done false
5354 2508 Acked Write 7 Done Sent Done false
5355 2509 Acked Flush 0 Done Sent Done false
5356 STATES DS:0 DS:1 DS:2 TOTAL
5357 Sent 0 302 0 302
5358 Done 302 0 302 604
5359 Skipped 0 0 0 0
5360 Error 0 0 0 0
5361 Last Flush: 2509 2207 2509
5362 Downstairs last five completed: 2207 2206 2205 2204 2203
5363 Upstairs last five completed: 2509 2508 2507 2506 2505
5364 CLIENT: Up:0 ds:302 act:2
5365 ----------------------------------------------------------------
5366 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
5367 States: Active Offline Active
5368 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5369 2208 Acked Write 2 Done Sent Done false
5370 2209 Acked Read 2 Done Sent Done false
5371 2210 Acked Write 10 Done Sent Done false
5372 2211 Acked Write 8 Done Sent Done false
5373 2212 Acked Flush 0 Done Sent Done false
5374 2213 Acked Write 8 Done Sent Done false
5375 2214 Acked Read 1 Done Sent Done false
5376 2215 Acked Write 4 Done Sent Done false
5377 2216 Acked Read 7 Done Sent Done false
5378 2217 Acked Read 1 Done Sent Done false
5379 2218 Acked Flush 0 Done Sent Done false
5380 2219 Acked Read 8 Done Sent Done false
5381 2220 Acked Write 10 Done Sent Done false
5382 2221 Acked Read 8 Done Sent Done false
5383 2222 Acked Read 2 Done Sent Done false
5384 2223 Acked Read 9 Done Sent Done false
5385 2224 Acked Read 2 Done Sent Done false
5386 2225 Acked Read 10 Done Sent Done false
5387 2226 Acked Write 9 Done Sent Done false
5388 2227 Acked Flush 0 Done Sent Done false
5389 2228 Acked Read 10 Done Sent Done false
5390 2229 Acked Write 3 Done Sent Done false
5391 2230 Acked Read 10 Done Sent Done false
5392 2231 Acked Read 9 Done Sent Done false
5393 2232 Acked Write 3 Done Sent Done false
5394 2233 Acked Flush 0 Done Sent Done false
5395 2234 Acked Read 10 Done Sent Done false
5396 2235 Acked Write 4 Done Sent Done false
5397 2236 Acked Write 9 Done Sent Done false
5398 2237 Acked Write 6 Done Sent Done false
5399 2238 Acked Read 1 Done Sent Done false
5400 2239 Acked Read 6 Done Sent Done false
5401 2240 Acked Read 3 Done Sent Done false
5402 2241 Acked Flush 0 Done Sent Done false
5403 2242 Acked Read 5 Done Sent Done false
5404 2243 Acked Read 1 Done Sent Done false
5405 2244 Acked Read 5 Done Sent Done false
5406 2245 Acked Read 2 Done Sent Done false
5407 2246 Acked Read 7 Done Sent Done false
5408 2247 Acked Read 8 Done Sent Done false
5409 2248 Acked Write 9 Done Sent Done false
5410 2249 Acked Write 7 Done Sent Done false
5411 2250 Acked Read 4 Done Sent Done false
5412 2251 Acked Read 6 Done Sent Done false
5413 2252 Acked Write 3 Done Sent Done false
5414 2253 Acked Flush 0 Done Sent Done false
5415 2254 Acked Write 1 Done Sent Done false
5416 2255 Acked Write 1 Done Sent Done false
5417 2256 Acked Write 4 Done Sent Done false
5418 2257 Acked Read 2 Done Sent Done false
5419 2258 Acked Write 7 Done Sent Done false
5420 2259 Acked Write 8 Done Sent Done false
5421 2260 Acked Read 9 Done Sent Done false
5422 2261 Acked Write 1 Done Sent Done false
5423 2262 Acked Read 5 Done Sent Done false
5424 2263 Acked Flush 0 Done Sent Done false
5425 2264 Acked Write 3 Done Sent Done false
5426 2265 Acked Write 1 Done Sent Done false
5427 2266 Acked Write 2 Done Sent Done false
5428 2267 Acked Write 7 Done Sent Done false
5429 2268 Acked Write 8 Done Sent Done false
5430 2269 Acked Read 9 Done Sent Done false
5431 2270 Acked Write 6 Done Sent Done false
5432 2271 Acked Write 8 Done Sent Done false
5433 2272 Acked Write 10 Done Sent Done false
5434 2273 Acked Read 4 Done Sent Done false
5435 2274 Acked Write 3 Done Sent Done false
5436 2275 Acked Write 10 Done Sent Done false
5437 2276 Acked Flush 0 Done Sent Done false
5438 2277 Acked Read 4 Done Sent Done false
5439 2278 Acked Write 4 Done Sent Done false
5440 2279 Acked Write 2 Done Sent Done false
5441 2280 Acked Read 9 Done Sent Done false
5442 2281 Acked Read 5 Done Sent Done false
5443 2282 Acked Write 10 Done Sent Done false
5444 2283 Acked Read 8 Done Sent Done false
5445 2284 Acked Read 9 Done Sent Done false
5446 2285 Acked Read 2 Done Sent Done false
5447 2286 Acked Write 10 Done Sent Done false
5448 2287 Acked Write 5 Done Sent Done false
5449 2288 Acked Read 4 Done Sent Done false
5450 2289 Acked Flush 0 Done Sent Done false
5451 2290 Acked Write 7 Done Sent Done false
5452 2291 Acked Read 2 Done Sent Done false
5453 2292 Acked Read 1 Done Sent Done false
5454 2293 Acked Write 2 Done Sent Done false
5455 2294 Acked Read 6 Done Sent Done false
5456 2295 Acked Read 8 Done Sent Done false
5457 2296 Acked Read 4 Done Sent Done false
5458 2297 Acked Write 4 Done Sent Done false
5459 2298 Acked Write 9 Done Sent Done false
5460 2299 Acked Read 6 Done Sent Done false
5461 2300 Acked Read 6 Done Sent Done false
5462 2301 Acked Read 4 Done Sent Done false
5463 2302 Acked Read 8 Done Sent Done false
5464 2303 Acked Write 10 Done Sent Done false
5465 2304 Acked Flush 0 Done Sent Done false
5466 2305 Acked Read 3 Done Sent Done false
5467 2306 Acked Write 9 Done Sent Done false
5468 2307 Acked Read 8 Done Sent Done false
5469 2308 Acked Read 8 Done Sent Done false
5470 2309 Acked Read 5 Done Sent Done false
5471 2310 Acked Read 1 Done Sent Done false
5472 2311 Acked Flush 0 Done Sent Done false
5473 2312 Acked Flush 0 Done Sent Done false
5474 2313 Acked Read 6 Done Sent Done false
5475 2314 Acked Write 7 Done Sent Done false
5476 2315 Acked Read 7 Done Sent Done false
5477 2316 Acked Write 6 Done Sent Done false
5478 2317 Acked Flush 0 Done Sent Done false
5479 2318 Acked Read 4 Done Sent Done false
5480 2319 Acked Read 9 Done Sent Done false
5481 2320 Acked Read 7 Done Sent Done false
5482 2321 Acked Read 1 Done Sent Done false
5483 2322 Acked Flush 0 Done Sent Done false
5484 2323 Acked Flush 0 Done Sent Done false
5485 2324 Acked Read 1 Done Sent Done false
5486 2325 Acked Read 4 Done Sent Done false
5487 2326 Acked Read 3 Done Sent Done false
5488 2327 Acked Write 9 Done Sent Done false
5489 2328 Acked Read 1 Done Sent Done false
5490 2329 Acked Read 1 Done Sent Done false
5491 2330 Acked Write 8 Done Sent Done false
5492 2331 Acked Write 2 Done Sent Done false
5493 2332 Acked Read 6 Done Sent Done false
5494 2333 Acked Write 3 Done Sent Done false
5495 2334 Acked Write 4 Done Sent Done false
5496 2335 Acked Read 6 Done Sent Done false
5497 2336 Acked Write 9 Done Sent Done false
5498 2337 Acked Read 8 Done Sent Done false
5499 2338 Acked Write 7 Done Sent Done false
5500 2339 Acked Write 3 Done Sent Done false
5501 2340 Acked Read 9 Done Sent Done false
5502 2341 Acked Read 7 Done Sent Done false
5503 2342 Acked Read 6 Done Sent Done false
5504 2343 Acked Write 5 Done Sent Done false
5505 2344 Acked Write 1 Done Sent Done false
5506 2345 Acked Write 3 Done Sent Done false
5507 2346 Acked Flush 0 Done Sent Done false
5508 2347 Acked Write 6 Done Sent Done false
5509 2348 Acked Read 7 Done Sent Done false
5510 2349 Acked Read 9 Done Sent Done false
5511 2350 Acked Flush 0 Done Sent Done false
5512 2351 Acked Read 6 Done Sent Done false
5513 2352 Acked Read 10 Done Sent Done false
5514 2353 Acked Read 3 Done Sent Done false
5515 2354 Acked Write 10 Done Sent Done false
5516 2355 Acked Write 3 Done Sent Done false
5517 2356 Acked Write 10 Done Sent Done false
5518 2357 Acked Write 7 Done Sent Done false
5519 2358 Acked Read 5 Done Sent Done false
5520 2359 Acked Read 3 Done Sent Done false
5521 2360 Acked Read 1 Done Sent Done false
5522 2361 Acked Read 3 Done Sent Done false
5523 2362 Acked Write 10 Done Sent Done false
5524 2363 Acked Write 4 Done Sent Done false
5525 2364 Acked Read 8 Done Sent Done false
5526 2365 Acked Write 3 Done Sent Done false
5527 2366 Acked Read 5 Done Sent Done false
5528 2367 Acked Write 6 Done Sent Done false
5529 2368 Acked Write 3 Done Sent Done false
5530 2369 Acked Read 7 Done Sent Done false
5531 2370 Acked Read 9 Done Sent Done false
5532 2371 Acked Write 9 Done Sent Done false
5533 2372 Acked Read 7 Done Sent Done false
5534 2373 Acked Read 5 Done Sent Done false
5535 2374 Acked Write 3 Done Sent Done false
5536 2375 Acked Read 8 Done Sent Done false
5537 2376 Acked Write 7 Done Sent Done false
5538 2377 Acked Read 6 Done Sent Done false
5539 2378 Acked Write 8 Done Sent Done false
5540 2379 Acked Write 4 Done Sent Done false
5541 2380 Acked Flush 0 Done Sent Done false
5542 2381 Acked Write 6 Done Sent Done false
5543 2382 Acked Read 4 Done Sent Done false
5544 2383 Acked Read 10 Done Sent Done false
5545 2384 Acked Read 7 Done Sent Done false
5546 2385 Acked Read 1 Done Sent Done false
5547 2386 Acked Read 9 Done Sent Done false
5548 2387 Acked Write 7 Done Sent Done false
5549 2388 Acked Write 9 Done Sent Done false
5550 2389 Acked Write 4 Done Sent Done false
5551 2390 Acked Write 4 Done Sent Done false
5552 2391 Acked Write 8 Done Sent Done false
5553 2392 Acked Flush 0 Done Sent Done false
5554 2393 Acked Flush 0 Done Sent Done false
5555 2394 Acked Read 9 Done Sent Done false
5556 2395 Acked Write 4 Done Sent Done false
5557 2396 Acked Write 3 Done Sent Done false
5558 2397 Acked Read 5 Done Sent Done false
5559 2398 Acked Flush 0 Done Sent Done false
5560 2399 Acked Read 2 Done Sent Done false
5561 2400 Acked Write 5 Done Sent Done false
5562 2401 Acked Write 4 Done Sent Done false
5563 2402 Acked Write 4 Done Sent Done false
5564 2403 Acked Read 5 Done Sent Done false
5565 2404 Acked Read 10 Done Sent Done false
5566 2405 Acked Read 5 Done Sent Done false
5567 2406 Acked Read 8 Done Sent Done false
5568 2407 Acked Read 4 Done Sent Done false
5569 2408 Acked Write 9 Done Sent Done false
5570 2409 Acked Read 10 Done Sent Done false
5571 2410 Acked Read 9 Done Sent Done false
5572 2411 Acked Read 10 Done Sent Done false
5573 2412 Acked Write 8 Done Sent Done false
5574 2413 Acked Write 4 Done Sent Done false
5575 2414 Acked Write 1 Done Sent Done false
5576 2415 Acked Read 2 Done Sent Done false
5577 2416 Acked Flush 0 Done Sent Done false
5578 2417 Acked Write 5 Done Sent Done false
5579 2418 Acked Write 3 Done Sent Done false
5580 2419 Acked Write 3 Done Sent Done false
5581 2420 Acked Write 7 Done Sent Done false
5582 2421 Acked Write 1 Done Sent Done false
5583 2422 Acked Read 9 Done Sent Done false
5584 2423 Acked Read 9 Done Sent Done false
5585 2424 Acked Read 2 Done Sent Done false
5586 2425 Acked Read 8 Done Sent Done false
5587 2426 Acked Read 6 Done Sent Done false
5588 2427 Acked Flush 0 Done Sent Done false
5589 2428 Acked Read 4 Done Sent Done false
5590 2429 Acked Write 5 Done Sent Done false
5591 2430 Acked Write 7 Done Sent Done false
5592 2431 Acked Read 2 Done Sent Done false
5593 2432 Acked Flush 0 Done Sent Done false
5594 2433 Acked Read 10 Done Sent Done false
5595 2434 Acked Write 5 Done Sent Done false
5596 2435 Acked Read 4 Done Sent Done false
5597 2436 Acked Write 9 Done Sent Done false
5598 2437 Acked Write 6 Done Sent Done false
5599 2438 Acked Write 2 Done Sent Done false
5600 2439 Acked Write 9 Done Sent Done false
5601 2440 Acked Write 6 Done Sent Done false
5602 2441 Acked Read 7 Done Sent Done false
5603 2442 Acked Flush 0 Done Sent Done false
5604 2443 Acked Read 6 Done Sent Done false
5605 2444 Acked Read 7 Done Sent Done false
5606 2445 Acked Write 8 Done Sent Done false
5607 2446 Acked Write 8 Done Sent Done false
5608 2447 Acked Flush 0 Done Sent Done false
5609 2448 Acked Write 1 Done Sent Done false
5610 2449 Acked Read 10 Done Sent Done false
5611 2450 Acked Read 7 Done Sent Done false
5612 2451 Acked Read 5 Done Sent Done false
5613 2452 Acked Read 4 Done Sent Done false
5614 2453 Acked Write 4 Done Sent Done false
5615 2454 Acked Read 10 Done Sent Done false
5616 2455 Acked Write 7 Done Sent Done false
5617 2456 Acked Write 2 Done Sent Done false
5618 2457 Acked Write 8 Done Sent Done false
5619 2458 Acked Read 1 Done Sent Done false
5620 2459 Acked Write 10 Done Sent Done false
5621 2460 Acked Write 5 Done Sent Done false
5622 2461 Acked Read 4 Done Sent Done false
5623 2462 Acked Read 6 Done Sent Done false
5624 2463 Acked Read 2 Done Sent Done false
5625 2464 Acked Write 5 Done Sent Done false
5626 2465 Acked Read 4 Done Sent Done false
5627 2466 Acked Read 9 Done Sent Done false
5628 2467 Acked Read 10 Done Sent Done false
5629 2468 Acked Read 10 Done Sent Done false
5630 2469 Acked Write 2 Done Sent Done false
5631 2470 Acked Flush 0 Done Sent Done false
5632 2471 Acked Write 4 Done Sent Done false
5633 2472 Acked Write 1 Done Sent Done false
5634 2473 Acked Read 2 Done Sent Done false
5635 2474 Acked Read 6 Done Sent Done false
5636 2475 Acked Read 4 Done Sent Done false
5637 2476 Acked Flush 0 Done Sent Done false
5638 2477 Acked Write 9 Done Sent Done false
5639 2478 Acked Read 7 Done Sent Done false
5640 2479 Acked Write 6 Done Sent Done false
5641 2480 Acked Write 3 Done Sent Done false
5642 2481 Acked Write 2 Done Sent Done false
5643 2482 Acked Read 9 Done Sent Done false
5644 2483 Acked Write 8 Done Sent Done false
5645 2484 Acked Write 4 Done Sent Done false
5646 2485 Acked Read 4 Done Sent Done false
5647 2486 Acked Write 3 Done Sent Done false
5648 2487 Acked Flush 0 Done Sent Done false
5649 2488 Acked Write 5 Done Sent Done false
5650 2489 Acked Flush 0 Done Sent Done false
5651 2490 Acked Flush 0 Done Sent Done false
5652 2491 Acked Read 1 Done Sent Done false
5653 2492 Acked Read 8 Done Sent Done false
5654 2493 Acked Write 8 Done Sent Done false
5655 2494 Acked Write 8 Done Sent Done false
5656 2495 Acked Write 4 Done Sent Done false
5657 2496 Acked Write 3 Done Sent Done false
5658 2497 Acked Read 1 Done Sent Done false
5659 2498 Acked Read 4 Done Sent Done false
5660 2499 Acked Write 6 Done Sent Done false
5661 2500 Acked Flush 0 Done Sent Done false
5662 2501 Acked Write 9 Done Sent Done false
5663 2502 Acked Read 5 Done Sent Done false
5664 2503 Acked Write 4 Done Sent Done false
5665 2504 Acked Read 8 Done Sent Done false
5666 2505 Acked Write 7 Done Sent Done false
5667 2506 Acked Write 7 Done Sent Done false
5668 2507 Acked Write 8 Done Sent Done false
5669 2508 Acked Write 7 Done Sent Done false
5670 2509 Acked Flush 0 Done Sent Done false
5671 STATES DS:0 DS:1 DS:2 TOTAL
5672 Sent 0 302 0 302
5673 Done 302 0 302 604
5674 Skipped 0 0 0 0
5675 Error 0 0 0 0
5676 Last Flush: 2509 2207 2509
5677 Downstairs last five completed: 2207 2206 2205 2204 2203
5678 Upstairs last five completed: 2509 2508 2507 2506 2505
5679 CLIENT: Up:0 ds:302 act:2
56802025-01-13T15:53:00.044ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
56812025-01-13T15:53:00.044ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39482) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
56822025-01-13T15:53:00.046ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
56832025-01-13T15:53:00.046ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
56842025-01-13T15:53:00.046ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
56852025-01-13T15:53:00.046ZINFOcrucible: send last flush ID to this DS: 2207 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
56862025-01-13T15:53:00.046ZINFOcrucible: Replied this last flush ID: 2207 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
56872025-01-13T15:53:00.046ZINFOcrucible: [1] client re-new 302 jobs since flush 2207 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
56882025-01-13T15:53:00.047ZINFOcrucible: [1] Replayed 302 jobs since flush: 2207 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
5689 ----------------------------------------------------------------
5690 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
5691 Downstairs last five completed: 2509 2508 2507 2506 2505
5692 Upstairs last five completed: 2509 2508 2507 2506 2505
5693 CLIENT: Up:0 ds:0 act:3
5694 Replay: All jobs finished
5695 Main task has work to do, go find it
5696 got dsc Stop(2)
5697 stop 2
5698 Jan 13 15:53:00.289 INFO request completed, latency_us[8830] Got stop action so:false kr:false
5699 : 334, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 95cb102e-5de9-4248-ac6a-9d5589b83f81, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
57002025-01-13T15:53:00.290ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
5701 Jan 13 15:53:00.290 INFO request completed, latency_us: 114, response_code: 200, uri: /state/cid/2, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:53:00.29058217Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
5702 method: GET, req_id: {"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:53:00.290614725Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
5703 f0a0d59a-3b44-4ba8-b64d-2e5d9b36344b, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
57042025-01-13T15:53:00.290ZINFOcrucible: client task is exiting = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
5705 [8830] Exited with: ExitStatus(unix_wait_status(9))
5706 [8830][2] reports Exit
5707 Jan 13 15:53:04.293 INFO request completed, latency_us: 326, response_code: 200, uri: /state/cid/2, method: GET, req_id: b0cd9e7a-100d-4ec7-af11-09dff84f1035, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
5708 001/300 Read block 3830 len 28672
5709 002/300 Write block 1744 len 12288 data: 5 6 5
5710 003/300 Write block 4836 len 40960 data: 3 3 3 3 2 3 3 3 3 3
5711 004/300 Write block 4070 len 20480 data: 2 2 2 3 3
5712 005/300 Write block 3189 len 24576 data: 3 3 3 3 2 2
5713 006/300 Read block 1019 len 20480
5714 007/300 Read block 1151 len 24576
5715 008/300 Write block 2197 len 36864 data: 3 3 2 2 2 2 2 2 2
5716 009/300 Read block 3750 len 40960
5717 010/300 Flush
5718 011/300 Read block 18 len 40960
5719 012/300 Read block 762 len 40960
5720 013/300 Write block 4156 len 4096 data: 2
5721 014/300 Write block 1560 len 12288 data: 2 2 2
5722 015/300 Flush
5723 016/300 Read block 1305 len 12288
5724 017/300 Read block 27 len 20480
5725 018/300 Flush
5726 019/300 Flush
5727 020/300 Write block 2823 len 20480 data: 3 3 3 3 3
5728 021/300 Read block 1472 len 40960
5729 022/300 Write block 4721 len 8192 data: 2 2
5730 023/300 Read block 2670 len 12288
5731 024/300 Read block 2272 len 32768
5732 025/300 Flush
5733 026/300 Read block 2626 len 40960
5734 027/300 Flush
5735 028/300 Write block 773 len 12288 data: 3 3 3
5736 029/300 Write block 3151 len 40960 data: 2 2 2 2 2 2 2 2 3 3
5737 030/300 Write block 3041 len 20480 data: 4 3 4 4 4
5738 031/300 Write block 375 len 28672 data: 2 2 2 2 2 2 2
5739 032/300 Read block 4618 len 20480
5740 033/300 Write block 2644 len 32768 data: 2 2 2 2 2 2 3 3
5741 034/300 Read block 631 len 8192
5742 035/300 Write block 1373 len 8192 data: 3 3
5743 036/300 Write block 567 len 32768 data: 2 2 2 2 2 2 2 3
5744 037/300 Read block 3034 len 16384
5745 038/300 Flush
5746 039/300 Write block 2148 len 16384 data: 3 3 3 3
5747 040/300 Read block 4986 len 12288
5748 041/300 Write block 3024 len 8192 data: 4 3
5749 042/300 Write block 959 len 40960 data: 2 3 4 4 3 3 3 2 2 2
5750 043/300 Read block 71 len 36864
5751 044/300 Write block 1111 len 36864 data: 2 2 2 2 3 3 3 3 2
5752 045/300 Write block 3142 len 28672 data: 2 2 2 2 2 2 2
5753 046/300 Read block 2686 len 36864
5754 047/300 Flush
5755 048/300 Write block 3154 len 8192 data: 3 3
5756 049/300 Read block 1833 len 20480
5757 050/300 Write block 2812 len 40960 data: 2 2 3 3 2 2 2 2 3 3
5758 051/300 Read block 1539 len 32768
5759 052/300 Write block 367 len 8192 data: 2 2
5760 053/300 Read block 2087 len 40960
5761 054/300 Read block 4866 len 12288
5762 055/300 Read block 2459 len 28672
5763 056/300 Write block 1424 len 24576 data: 3 2 4 4 5 5
5764 057/300 Write block 3953 len 24576 data: 2 2 2 3 3 3
5765 058/300 Read block 4558 len 16384
5766 059/300 Flush
5767 060/300 Write block 3985 len 40960 data: 2 2 3 3 2 2 2 2 2 3
5768 061/300 Read block 1233 len 24576
5769 062/300 Write block 4959 len 8192 data: 3 3
5770 063/300 Read block 3891 len 28672
5771 064/300 Write block 2871 len 24576 data: 2 2 2 2 3 3
5772 065/300 Write block 664 len 20480 data: 6 6 6 5 6
5773 066/300 Read block 1830 len 20480
5774 067/300 Read block 1197 len 20480
5775 068/300 Read block 870 len 40960
5776 069/300 Write block 2730 len 28672 data: 2 2 2 2 2 2 2
5777 070/300 Read block 763 len 20480
5778 071/300 Read block 1385 len 40960
5779 072/300 Write block 1382 len 28672 data: 3 3 2 2 2 2 2
5780 073/300 Read block 1181 len 36864
5781 074/300 Read block 1309 len 24576
5782 075/300 Flush
5783 076/300 Read block 3306 len 16384
5784 077/300 Read block 1766 len 28672
5785 078/300 Write block 653 len 36864 data: 2 2 2 2 2 2 2 3 4
5786 079/300 Flush
5787 080/300 Write block 3188 len 16384 data: 3 4 4 4
5788 081/300 Read block 132 len 36864
5789 082/300 Read block 1255 len 20480
5790 083/300 Read block 2549 len 16384
5791 084/300 Read block 2015 len 4096
5792 085/300 Read block 805 len 4096
5793 086/300 Read block 1449 len 8192
5794 087/300 Read block 391 len 28672
5795 088/300 Read block 615 len 20480
5796 089/300 Read block 3183 len 8192
5797 090/300 Read block 3037 len 40960
5798 091/300 Write block 764 len 28672 data: 2 3 3 3 3 3 3
5799 092/300 Write block 1377 len 40960 data: 4 3 3 3 3 4 4 3 3 3
5800 093/300 Write block 3519 len 4096 data: 3
5801 094/300 Write block 3625 len 24576 data: 2 2 2 2 2 2
5802 095/300 Read block 3775 len 40960
5803 096/300 Read block 3938 len 16384
5804 097/300 Read block 333 len 4096
5805 098/300 Flush
5806 099/300 Read block 4367 len 16384
5807 100/300 Write block 3727 len 40960 data: 3 3 3 2 2 2 2 2 2 2
5808 101/300 Read block 2556 len 32768
5809 102/300 Read block 3475 len 16384
5810 103/300 Read block 3501 len 20480
5811 104/300 Read block 4979 len 16384
5812 105/300 Flush
5813 106/300 Read block 3816 len 16384
5814 107/300 Write block 1966 len 28672 data: 2 2 2 2 2 3 3
5815 108/300 Write block 2766 len 16384 data: 2 2 2 2
5816 109/300 Flush
5817 110/300 Write block 4227 len 40960 data: 3 3 3 3 3 2 2 2 2 2
5818 111/300 Read block 1833 len 20480
5819 112/300 Read block 1517 len 40960
5820 113/300 Read block 515 len 16384
5821 114/300 Write block 3501 len 24576 data: 3 3 3 3 3 2
5822 115/300 Read block 3898 len 24576
5823 116/300 Flush
5824 117/300 Read block 2370 len 8192
5825 118/300 Read block 1635 len 20480
5826 119/300 Flush
5827 120/300 Read block 159 len 40960
5828 121/300 Write block 907 len 20480 data: 2 2 2 2 2
5829 122/300 Write block 1035 len 4096 data: 3
5830 123/300 Read block 2829 len 12288
5831 124/300 Write block 4249 len 28672 data: 2 2 2 2 2 2 2
5832 125/300 Write block 3826 len 28672 data: 2 2 2 2 2 2 2
5833 126/300 Write block 3118 len 8192 data: 2 2
5834 127/300 Write block 4003 len 24576 data: 3 3 2 2 2 3
5835 128/300 Read block 3706 len 12288
5836 129/300 Read block 2973 len 24576
5837 130/300 Write block 4217 len 4096 data: 4
5838 131/300 Flush
5839 132/300 Flush
5840 133/300 Read block 576 len 20480
5841 134/300 Write block 119 len 16384 data: 3 3 2 2
5842 135/300 Read block 507 len 40960
5843 136/300 Write block 1607 len 24576 data: 4 3 3 3 3 2
5844 137/300 Read block 2547 len 4096
5845 138/300 Write block 1263 len 32768 data: 2 2 2 2 2 2 2 2
5846 139/300 Read block 4704 len 32768
5847 140/300 Read block 1224 len 24576
5848 141/300 Write block 4797 len 16384 data: 2 2 2 3
5849 142/300 Read block 4470 len 36864
5850 143/300 Read block 377 len 28672
5851 144/300 Write block 4817 len 24576 data: 2 2 2 2 2 2
5852 145/300 Read block 3354 len 24576
5853 146/300 Read block 606 len 8192
5854 147/300 Flush
5855 148/300 Write block 1637 len 20480 data: 3 2 2 2 2
5856 149/300 Read block 4339 len 24576
5857 150/300 Read block 4082 len 20480
5858 151/300 Write block 4884 len 24576 data: 2 2 4 4 5 5
5859 152/300 Read block 145 len 24576
5860 153/300 Read block 4249 len 24576
5861 154/300 Read block 2350 len 40960
5862 155/300 Write block 3788 len 40960 data: 3 3 3 4 4 3 3 3 3 3
5863 156/300 Read block 2860 len 36864
5864 157/300 Read block 2685 len 32768
5865 158/300 Write block 479 len 12288 data: 3 2 2
5866 159/300 Read block 1827 len 20480
5867 160/300 Read block 3022 len 12288
5868 161/300 Read block 2576 len 24576
5869 162/300 Write block 4681 len 4096 data: 3
5870 163/300 Write block 973 len 24576 data: 2 2 2 2 3 3
5871 164/300 Write block 4774 len 28672 data: 2 2 2 2 2 2 2
5872 165/300 Write block 2272 len 32768 data: 3 3 4 4 4 5 4 3
5873 166/300 Read block 558 len 8192
5874 167/300 Read block 2778 len 12288
5875 168/300 Read block 1849 len 16384
5876 169/300 Write block 1862 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5877 170/300 Read block 1980 len 40960
5878 171/300 Read block 2321 len 40960
5879 172/300 Read block 1014 len 8192
5880 173/300 Write block 1856 len 12288 data: 2 2 2
5881 174/300 Read block 1369 len 16384
5882 175/300 Read block 801 len 4096
5883 176/300 Read block 4479 len 28672
5884 177/300 Write block 4370 len 4096 data: 3
5885 178/300 Write block 1409 len 4096 data: 2
5886 179/300 Write block 1326 len 20480 data: 2 2 3 3 2
5887 180/300 Read block 422 len 36864
5888 181/300 Write block 984 len 32768 data: 3 3 3 4 4 4 3 3
5889 182/300 Read block 190 len 4096
5890 183/300 Read block 3669 len 12288
5891 184/300 Write block 3770 len 16384 data: 2 2 2 2
5892 185/300 Read block 3346 len 20480
5893 186/300 Read block 2455 len 32768
5894 187/300 Read block 4720 len 20480
5895 188/300 Write block 4284 len 20480 data: 3 3 3 3 2
5896 189/300 Flush
5897 190/300 Write block 557 len 32768 data: 2 2 2 2 2 2 2 2
5898 191/300 Write block 3827 len 28672 data: 3 3 3 3 3 3 2
5899 192/300 Flush
5900 193/300 Read block 1732 len 24576
5901 194/300 Read block 3392 len 8192
5902 195/300 Write block 3297 len 8192 data: 2 2
5903 196/300 Read block 3855 len 8192
5904 197/300 Read block 2697 len 28672
5905 198/300 Read block 2124 len 12288
5906 199/300 Read block 2618 len 32768
5907 200/300 Read block 2746 len 40960
5908 201/300 Read block 4842 len 16384
5909 202/300 Write block 2043 len 40960 data: 2 2 2 3 3 3 3 3 3 3
5910 203/300 Flush
5911 204/300 Read block 917 len 20480
5912 205/300 Write block 497 len 8192 data: 2 2
5913 206/300 Write block 1968 len 32768 data: 3 3 3 4 4 3 3 3
5914 207/300 Flush
5915 208/300 Flush
5916 209/300 Write block 1 len 28672 data: 2 2 2 3 3 2 2
5917 210/300 Flush
5918 211/300 Read block 4714 len 24576
5919 212/300 Read block 3137 len 28672
5920 213/300 Read block 3708 len 12288
5921 214/300 Read block 80 len 4096
5922 215/300 Read block 759 len 28672
5923 216/300 Write block 2763 len 8192 data: 2 2
5924 217/300 Read block 4064 len 8192
5925 218/300 Read block 958 len 40960
5926 219/300 Read block 1386 len 24576
5927 220/300 Write block 1953 len 20480 data: 2 2 2 2 2
5928 221/300 Flush
5929 222/300 Write block 1440 len 16384 data: 3 3 3 3
5930 223/300 Write block 2422 len 32768 data: 2 2 2 2 2 2 2 2
5931 224/300 Read block 4442 len 12288
5932 225/300 Read block 1014 len 36864
5933 226/300 Flush
5934 227/300 Flush
5935 228/300 Read block 4739 len 20480
5936 229/300 Write block 4447 len 16384 data: 3 4 4 3
5937 230/300 Write block 3601 len 12288 data: 3 3 3
5938 231/300 Write block 4296 len 40960 data: 2 2 3 3 3 3 4 4 3 3
5939 232/300 Write block 130 len 24576 data: 5 5 5 4 4 4
5940 233/300 Write block 752 len 40960 data: 3 3 3 3 3 3 3 2 2 2
5941 234/300 Write block 1624 len 4096 data: 3
5942 235/300 Read block 138 len 24576
5943 236/300 Write block 1980 len 20480 data: 2 2 2 2 3
5944 237/300 Read block 4222 len 40960
5945 238/300 Write block 372 len 28672 data: 2 2 2 3 3 3 3
5946 239/300 Read block 2799 len 28672
5947 240/300 Write block 1653 len 4096 data: 2
5948 241/300 Write block 4520 len 4096 data: 4
5949 242/300 Read block 283 len 8192
5950 243/300 Read block 396 len 20480
5951 244/300 Write block 1491 len 40960 data: 2 3 3 2 2 2 2 3 4 3
5952 245/300 Write block 4426 len 32768 data: 2 2 2 2 2 2 2 2
5953 246/300 Write block 1906 len 16384 data: 2 2 2 2
5954 247/300 Write block 3165 len 20480 data: 2 2 3 3 3
5955 248/300 Read block 4489 len 20480
5956 249/300 Write block 645 len 36864 data: 3 3 2 2 3 3 3 3 3
5957 250/300 Write block 4597 len 4096 data: 4
5958 251/300 Read block 1920 len 40960
5959 252/300 Write block 623 len 24576 data: 2 2 2 2 2 2
5960 253/300 Write block 240 len 8192 data: 2 2
5961 254/300 Read block 264 len 8192
5962 255/300 Read block 3263 len 4096
5963 256/300 Write block 4297 len 20480 data: 3 4 4 4 4
5964 257/300 Read block 3950 len 8192
5965 258/300 Read block 3726 len 36864
5966 259/300 Write block 1333 len 40960 data: 3 4 5 6 6 5 4 4 3 3
5967 260/300 Write block 2077 len 28672 data: 3 3 3 2 2 2 2
5968 261/300 Read block 3770 len 28672
5969 262/300 Read block 1110 len 8192
5970 263/300 Write block 2491 len 36864 data: 2 2 2 2 2 3 4 4 3
5971 264/300 Write block 428 len 32768 data: 2 2 2 2 2 3 3 3
5972 265/300 Write block 2263 len 28672 data: 3 4 4 4 4 3 3
5973 266/300 Write block 154 len 28672 data: 2 2 2 2 2 2 3
5974 267/300 Read block 2293 len 20480
5975 268/300 Write block 4967 len 8192 data: 3 3
5976 269/300 Flush
5977 270/300 Read block 765 len 40960
5978 271/300 Write block 3084 len 4096 data: 3
5979 272/300 Flush
5980 273/300 Read block 1055 len 4096
5981 274/300 Read block 514 len 20480
5982 275/300 Read block 2509 len 24576
5983 276/300 Read block 3121 len 20480
5984 277/300 Write block 4744 len 4096 data: 2
5985 278/300 Write block 876 len 36864 data: 3 2 2 2 3 3 3 2 2
5986 279/300 Write block 4136 len 4096 data: 2
5987 280/300 Write block 398 len 8192 data: 5 5
5988 281/300 Read block 1108 len 32768
5989 282/300 Read block 4673 len 20480
5990 283/300 Write block 3330 len 16384 data: 3 2 2 2
5991 284/300 Read block 1135 len 16384
5992 285/300 Write block 4497 len 32768 data: 3 3 3 3 3 3 3 3
5993 286/300 Read block 728 len 32768
5994 287/300 Read block 1670 len 28672
5995 288/300 Read block 2109 len 40960
5996 289/300 Read block 3534 len 20480
5997 290/300 Read block 4072 len 16384
5998 291/300 Write block 1490 len 20480 data: 3 3 4 4 3
5999 292/300 Write block 2303 len 36864 data: 2 2 2 2 2 2 2 2 2
6000 293/300 Read block 2306 len 4096
6001 294/300 Read block 2930 len 8192
6002 295/300 Write block 2643 len 32768 data: 3 3 3 3 3 3 3 4
6003 296/300 Flush
6004 297/300 Write block 485 len 12288 data: 2 2 2
6005 298/300 Read block 2705 len 20480
6006 299/300 Read block 143 len 40960
6007 300/300 Read block 4562 len 4096
6008 301/300 Write block 2486 len 4096 data: 4
6009 Main task has work to do, go find it
6010 got dsc Start(2)
6011 start 2
6012 Jan 13 15:53:04.536 INFO request completed, latency_us: 223, response_code: 204, uri: /start/cid/2, method: POST, req_id: 239bbdb8-c019-46b3-96d2-bbb6be516472, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
6013 Replay: started 2, returned:Ok(())
6014 ----------------------------------------------------------------
6015 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6016 States: Active Active Offline
6017 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6018 2510 Acked Read 7 Done Done Sent false
6019 2511 Acked Write 3 Done Done Sent false
6020 2512 Acked Write 10 Done Done Sent false
6021 2513 Acked Write 5 Done Done Sent false
6022 2514 Acked Write 6 Done Done Sent false
6023 2515 Acked Read 5 Done Done Sent false
6024 2516 Acked Read 6 Done Done Sent false
6025 2517 Acked Write 9 Done Done Sent false
6026 2518 Acked Read 10 Done Done Sent false
6027 2519 Acked Flush 0 Done Done Sent false
6028 2520 Acked Read 10 Done Done Sent false
6029 2521 Acked Read 10 Done Done Sent false
6030 2522 Acked Write 1 Done Done Sent false
6031 2523 Acked Write 3 Done Done Sent false
6032 2524 Acked Flush 0 Done Done Sent false
6033 2525 Acked Read 3 Done Done Sent false
6034 2526 Acked Read 5 Done Done Sent false
6035 2527 Acked Flush 0 Done Done Sent false
6036 2528 Acked Flush 0 Done Done Sent false
6037 2529 Acked Write 5 Done Done Sent false
6038 2530 Acked Read 10 Done Done Sent false
6039 2531 Acked Write 2 Done Done Sent false
6040 2532 Acked Read 3 Done Done Sent false
6041 2533 Acked Read 8 Done Done Sent false
6042 2534 Acked Flush 0 Done Done Sent false
6043 2535 Acked Read 10 Done Done Sent false
6044 2536 Acked Flush 0 Done Done Sent false
6045 2537 Acked Write 3 Done Done Sent false
6046 2538 Acked Write 10 Done Done Sent false
6047 2539 Acked Write 5 Done Done Sent false
6048 2540 Acked Write 7 Done Done Sent false
6049 2541 Acked Read 5 Done Done Sent false
6050 2542 Acked Write 8 Done Done Sent false
6051 2543 Acked Read 2 Done Done Sent false
6052 2544 Acked Write 2 Done Done Sent false
6053 2545 Acked Write 8 Done Done Sent false
6054 2546 Acked Read 4 Done Done Sent false
6055 2547 Acked Flush 0 Done Done Sent false
6056 2548 Acked Write 4 Done Done Sent false
6057 2549 Acked Read 3 Done Done Sent false
6058 2550 Acked Write 2 Done Done Sent false
6059 2551 Acked Write 10 Done Done Sent false
6060 2552 Acked Read 9 Done Done Sent false
6061 2553 Acked Write 9 Done Done Sent false
6062 2554 Acked Write 7 Done Done Sent false
6063 2555 Acked Read 9 Done Done Sent false
6064 2556 Acked Flush 0 Done Done Sent false
6065 2557 Acked Write 2 Done Done Sent false
6066 2558 Acked Read 5 Done Done Sent false
6067 2559 Acked Write 10 Done Done Sent false
6068 2560 Acked Read 8 Done Done Sent false
6069 2561 Acked Write 2 Done Done Sent false
6070 2562 Acked Read 10 Done Done Sent false
6071 2563 Acked Read 3 Done Done Sent false
6072 2564 Acked Read 7 Done Done Sent false
6073 2565 Acked Write 6 Done Done Sent false
6074 2566 Acked Write 6 Done Done Sent false
6075 2567 Acked Read 4 Done Done Sent false
6076 2568 Acked Flush 0 Done Done Sent false
6077 2569 Acked Write 10 Done Done Sent false
6078 2570 Acked Read 6 Done Done Sent false
6079 2571 Acked Write 2 Done Done Sent false
6080 2572 Acked Read 7 Done Done Sent false
6081 2573 Acked Write 6 Done Done Sent false
6082 2574 Acked Write 5 Done Done Sent false
6083 2575 Acked Read 5 Done Done Sent false
6084 2576 Acked Read 5 Done Done Sent false
6085 2577 Acked Read 10 Done Done Sent false
6086 2578 Acked Write 7 Done Done Sent false
6087 2579 Acked Read 5 Done Done Sent false
6088 2580 Acked Read 10 Done Done Sent false
6089 2581 Acked Write 7 Done Done Sent false
6090 2582 Acked Read 9 Done Done Sent false
6091 2583 Acked Read 6 Done Done Sent false
6092 2584 Acked Flush 0 Done Done Sent false
6093 2585 Acked Read 4 Done Done Sent false
6094 2586 Acked Read 7 Done Done Sent false
6095 2587 Acked Write 9 Done Done Sent false
6096 2588 Acked Flush 0 Done Done Sent false
6097 2589 Acked Write 4 Done Done Sent false
6098 2590 Acked Read 9 Done Done Sent false
6099 2591 Acked Read 5 Done Done Sent false
6100 2592 Acked Read 4 Done Done Sent false
6101 2593 Acked Read 1 Done Done Sent false
6102 2594 Acked Read 1 Done Done Sent false
6103 2595 Acked Read 2 Done Done Sent false
6104 2596 Acked Read 7 Done Done Sent false
6105 2597 Acked Read 5 Done Done Sent false
6106 2598 Acked Read 2 Done Done Sent false
6107 2599 Acked Read 10 Done Done Sent false
6108 2600 Acked Write 7 Done Done Sent false
6109 2601 Acked Write 10 Done Done Sent false
6110 2602 Acked Write 1 Done Done Sent false
6111 2603 Acked Write 6 Done Done Sent false
6112 2604 Acked Read 10 Done Done Sent false
6113 2605 Acked Read 4 Done Done Sent false
6114 2606 Acked Read 1 Done Done Sent false
6115 2607 Acked Flush 0 Done Done Sent false
6116 2608 Acked Read 4 Done Done Sent false
6117 2609 Acked Write 10 Done Done Sent false
6118 2610 Acked Read 8 Done Done Sent false
6119 2611 Acked Read 4 Done Done Sent false
6120 2612 Acked Read 5 Done Done Sent false
6121 2613 Acked Read 4 Done Done Sent false
6122 2614 Acked Flush 0 Done Done Sent false
6123 2615 Acked Read 4 Done Done Sent false
6124 2616 Acked Write 7 Done Done Sent false
6125 2617 Acked Write 4 Done Done Sent false
6126 2618 Acked Flush 0 Done Done Sent false
6127 2619 Acked Write 10 Done Done Sent false
6128 2620 Acked Read 5 Done Done Sent false
6129 2621 Acked Read 10 Done Done Sent false
6130 2622 Acked Read 4 Done Done Sent false
6131 2623 Acked Write 6 Done Done Sent false
6132 2624 Acked Read 6 Done Done Sent false
6133 2625 Acked Flush 0 Done Done Sent false
6134 2626 Acked Read 2 Done Done Sent false
6135 2627 Acked Read 5 Done Done Sent false
6136 2628 Acked Flush 0 Done Done Sent false
6137 2629 Acked Read 10 Done Done Sent false
6138 2630 Acked Write 5 Done Done Sent false
6139 2631 Acked Write 1 Done Done Sent false
6140 2632 Acked Read 3 Done Done Sent false
6141 2633 Acked Write 7 Done Done Sent false
6142 2634 Acked Write 7 Done Done Sent false
6143 2635 Acked Write 2 Done Done Sent false
6144 2636 Acked Write 6 Done Done Sent false
6145 2637 Acked Read 3 Done Done Sent false
6146 2638 Acked Read 6 Done Done Sent false
6147 2639 Acked Write 1 Done Done Sent false
6148 2640 Acked Flush 0 Done Done Sent false
6149 2641 Acked Flush 0 Done Done Sent false
6150 2642 Acked Read 5 Done Done Sent false
6151 2643 Acked Write 4 Done Done Sent false
6152 2644 Acked Read 10 Done Done Sent false
6153 2645 Acked Write 6 Done Done Sent false
6154 2646 Acked Read 1 Done Done Sent false
6155 2647 Acked Write 8 Done Done Sent false
6156 2648 Acked Read 8 Done Done Sent false
6157 2649 Acked Read 6 Done Done Sent false
6158 2650 Acked Write 4 Done Done Sent false
6159 2651 Acked Read 9 Done Done Sent false
6160 2652 Acked Read 7 Done Done Sent false
6161 2653 Acked Write 6 Done Done Sent false
6162 2654 Acked Read 6 Done Done Sent false
6163 2655 Acked Read 2 Done Done Sent false
6164 2656 Acked Flush 0 Done Done Sent false
6165 2657 Acked Write 5 Done Done Sent false
6166 2658 Acked Read 6 Done Done Sent false
6167 2659 Acked Read 5 Done Done Sent false
6168 2660 Acked Write 6 Done Done Sent false
6169 2661 Acked Read 6 Done Done Sent false
6170 2662 Acked Read 6 Done Done Sent false
6171 2663 Acked Read 10 Done Done Sent false
6172 2664 Acked Write 10 Done Done Sent false
6173 2665 Acked Read 9 Done Done Sent false
6174 2666 Acked Read 8 Done Done Sent false
6175 2667 Acked Write 3 Done Done Sent false
6176 2668 Acked Read 5 Done Done Sent false
6177 2669 Acked Read 3 Done Done Sent false
6178 2670 Acked Read 6 Done Done Sent false
6179 2671 Acked Write 1 Done Done Sent false
6180 2672 Acked Write 6 Done Done Sent false
6181 2673 Acked Write 7 Done Done Sent false
6182 2674 Acked Write 8 Done Done Sent false
6183 2675 Acked Read 2 Done Done Sent false
6184 2676 Acked Read 3 Done Done Sent false
6185 2677 Acked Read 4 Done Done Sent false
6186 2678 Acked Write 10 Done Done Sent false
6187 2679 Acked Read 10 Done Done Sent false
6188 2680 Acked Read 10 Done Done Sent false
6189 2681 Acked Read 2 Done Done Sent false
6190 2682 Acked Write 3 Done Done Sent false
6191 2683 Acked Read 4 Done Done Sent false
6192 2684 Acked Read 1 Done Done Sent false
6193 2685 Acked Read 7 Done Done Sent false
6194 2686 Acked Write 1 Done Done Sent false
6195 2687 Acked Write 1 Done Done Sent false
6196 2688 Acked Write 5 Done Done Sent false
6197 2689 Acked Read 9 Done Done Sent false
6198 2690 Acked Write 8 Done Done Sent false
6199 2691 Acked Read 1 Done Done Sent false
6200 2692 Acked Read 3 Done Done Sent false
6201 2693 Acked Write 4 Done Done Sent false
6202 2694 Acked Read 5 Done Done Sent false
6203 2695 Acked Read 8 Done Done Sent false
6204 2696 Acked Read 5 Done Done Sent false
6205 2697 Acked Write 5 Done Done Sent false
6206 2698 Acked Flush 0 Done Done Sent false
6207 2699 Acked Write 8 Done Done Sent false
6208 2700 Acked Write 7 Done Done Sent false
6209 2701 Acked Flush 0 Done Done Sent false
6210 2702 Acked Read 6 Done Done Sent false
6211 2703 Acked Read 2 Done Done Sent false
6212 2704 Acked Write 2 Done Done Sent false
6213 2705 Acked Read 2 Done Done Sent false
6214 2706 Acked Read 7 Done Done Sent false
6215 2707 Acked Read 3 Done Done Sent false
6216 2708 Acked Read 8 Done Done Sent false
6217 2709 Acked Read 10 Done Done Sent false
6218 2710 Acked Read 4 Done Done Sent false
6219 2711 Acked Write 10 Done Done Sent false
6220 2712 Acked Flush 0 Done Done Sent false
6221 2713 Acked Read 5 Done Done Sent false
6222 2714 Acked Write 2 Done Done Sent false
6223 2715 Acked Write 8 Done Done Sent false
6224 2716 Acked Flush 0 Done Done Sent false
6225 2717 Acked Flush 0 Done Done Sent false
6226 2718 Acked Write 7 Done Done Sent false
6227 2719 Acked Flush 0 Done Done Sent false
6228 2720 Acked Read 6 Done Done Sent false
6229 2721 Acked Read 7 Done Done Sent false
6230 2722 Acked Read 3 Done Done Sent false
6231 2723 Acked Read 1 Done Done Sent false
6232 2724 Acked Read 7 Done Done Sent false
6233 2725 Acked Write 2 Done Done Sent false
6234 2726 Acked Read 2 Done Done Sent false
6235 2727 Acked Read 10 Done Done Sent false
6236 2728 Acked Read 6 Done Done Sent false
6237 2729 Acked Write 5 Done Done Sent false
6238 2730 Acked Flush 0 Done Done Sent false
6239 2731 Acked Write 4 Done Done Sent false
6240 2732 Acked Write 8 Done Done Sent false
6241 2733 Acked Read 3 Done Done Sent false
6242 2734 Acked Read 9 Done Done Sent false
6243 2735 Acked Flush 0 Done Done Sent false
6244 2736 Acked Flush 0 Done Done Sent false
6245 2737 Acked Read 5 Done Done Sent false
6246 2738 Acked Write 4 Done Done Sent false
6247 2739 Acked Write 3 Done Done Sent false
6248 2740 Acked Write 10 Done Done Sent false
6249 2741 Acked Write 6 Done Done Sent false
6250 2742 Acked Write 10 Done Done Sent false
6251 2743 Acked Write 1 Done Done Sent false
6252 2744 Acked Read 6 Done Done Sent false
6253 2745 Acked Write 5 Done Done Sent false
6254 2746 Acked Read 10 Done Done Sent false
6255 2747 Acked Write 7 Done Done Sent false
6256 2748 Acked Read 7 Done Done Sent false
6257 2749 Acked Write 1 Done Done Sent false
6258 2750 Acked Write 1 Done Done Sent false
6259 2751 Acked Read 2 Done Done Sent false
6260 2752 Acked Read 5 Done Done Sent false
6261 2753 Acked Write 10 Done Done Sent false
6262 2754 Acked Write 8 Done Done Sent false
6263 2755 Acked Write 4 Done Done Sent false
6264 2756 Acked Write 5 Done Done Sent false
6265 2757 Acked Read 5 Done Done Sent false
6266 2758 Acked Write 9 Done Done Sent false
6267 2759 Acked Write 1 Done Done Sent false
6268 2760 Acked Read 10 Done Done Sent false
6269 2761 Acked Write 6 Done Done Sent false
6270 2762 Acked Write 2 Done Done Sent false
6271 2763 Acked Read 2 Done Done Sent false
6272 2764 Acked Read 1 Done Done Sent false
6273 2765 Acked Write 5 Done Done Sent false
6274 2766 Acked Read 2 Done Done Sent false
6275 2767 Acked Read 9 Done Done Sent false
6276 2768 Acked Write 10 Done Done Sent false
6277 2769 Acked Write 7 Done Done Sent false
6278 2770 Acked Read 7 Done Done Sent false
6279 2771 Acked Read 2 Done Done Sent false
6280 2772 Acked Write 9 Done Done Sent false
6281 2773 Acked Write 8 Done Done Sent false
6282 2774 Acked Write 7 Done Done Sent false
6283 2775 Acked Write 7 Done Done Sent false
6284 2776 Acked Read 5 Done Done Sent false
6285 2777 Acked Write 2 Done Done Sent false
6286 2778 Acked Flush 0 Done Done Sent false
6287 2779 Acked Read 10 Done Done Sent false
6288 2780 Acked Write 1 Done Done Sent false
6289 2781 Acked Flush 0 Done Done Sent false
6290 2782 Acked Read 1 Done Done Sent false
6291 2783 Acked Read 5 Done Done Sent false
6292 2784 Acked Read 6 Done Done Sent false
6293 2785 Acked Read 5 Done Done Sent false
6294 2786 Acked Write 1 Done Done Sent false
6295 2787 Acked Write 9 Done Done Sent false
6296 2788 Acked Write 1 Done Done Sent false
6297 2789 Acked Write 2 Done Done Sent false
6298 2790 Acked Read 8 Done Done Sent false
6299 2791 Acked Read 5 Done Done Sent false
6300 2792 Acked Write 4 Done Done Sent false
6301 2793 Acked Read 4 Done Done Sent false
6302 2794 Acked Write 8 Done Done Sent false
6303 2795 Acked Read 8 Done Done Sent false
6304 2796 Acked Read 7 Done Done Sent false
6305 2797 Acked Read 10 Done Done Sent false
6306 2798 Acked Read 5 Done Done Sent false
6307 2799 Acked Read 4 Done Done Sent false
6308 2800 Acked Write 5 Done Done Sent false
6309 2801 Acked Write 9 Done Done Sent false
6310 2802 Acked Read 1 Done Done Sent false
6311 2803 Acked Read 2 Done Done Sent false
6312 2804 Acked Write 8 Done Done Sent false
6313 2805 Acked Flush 0 Done Done Sent false
6314 2806 Acked Write 3 Done Done Sent false
6315 2807 Acked Read 5 Done Done Sent false
6316 2808 Acked Read 10 Done Done Sent false
6317 2809 Acked Read 1 Done Done Sent false
6318 2810 Acked Write 1 Done Done Sent false
6319 STATES DS:0 DS:1 DS:2 TOTAL
6320 Sent 0 0 301 301
6321 Done 301 301 0 602
6322 Skipped 0 0 0 0
6323 Error 0 0 0 0
6324 Last Flush: 2805 2805 2509
6325 Downstairs last five completed: 2509 2508 2507 2506 2505
6326 Upstairs last five completed: 2810 2809 2808 2807 2806
6327 CLIENT: Up:0 ds:301 act:2
6328 ----------------------------------------------------------------
6329 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6330 States: Active Active Offline
6331 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6332 2510 Acked Read 7 Done Done Sent false
6333 2511 Acked Write 3 Done Done Sent false
6334 2512 Acked Write 10 Done Done Sent false
6335 2513 Acked Write 5 Done Done Sent false
6336 2514 Acked Write 6 Done Done Sent false
6337 2515 Acked Read 5 Done Done Sent false
6338 2516 Acked Read 6 Done Done Sent false
6339 2517 Acked Write 9 Done Done Sent false
6340 2518 Acked Read 10 Done Done Sent false
6341 2519 Acked Flush 0 Done Done Sent false
6342 2520 Acked Read 10 Done Done Sent false
6343 2521 Acked Read 10 Done Done Sent false
6344 2522 Acked Write 1 Done Done Sent false
6345 2523 Acked Write 3 Done Done Sent false
6346 2524 Acked Flush 0 Done Done Sent false
6347 2525 Acked Read 3 Done Done Sent false
6348 2526 Acked Read 5 Done Done Sent false
6349 2527 Acked Flush 0 Done Done Sent false
6350 2528 Acked Flush 0 Done Done Sent false
6351 2529 Acked Write 5 Done Done Sent false
6352 2530 Acked Read 10 Done Done Sent false
6353 2531 Acked Write 2 Done Done Sent false
6354 2532 Acked Read 3 Done Done Sent false
6355 2533 Acked Read 8 Done Done Sent false
6356 2534 Acked Flush 0 Done Done Sent false
6357 2535 Acked Read 10 Done Done Sent false
6358 2536 Acked Flush 0 Done Done Sent false
6359 2537 Acked Write 3 Done Done Sent false
6360 2538 Acked Write 10 Done Done Sent false
6361 2539 Acked Write 5 Done Done Sent false
6362 2540 Acked Write 7 Done Done Sent false
6363 2541 Acked Read 5 Done Done Sent false
6364 2542 Acked Write 8 Done Done Sent false
6365 2543 Acked Read 2 Done Done Sent false
6366 2544 Acked Write 2 Done Done Sent false
6367 2545 Acked Write 8 Done Done Sent false
6368 2546 Acked Read 4 Done Done Sent false
6369 2547 Acked Flush 0 Done Done Sent false
6370 2548 Acked Write 4 Done Done Sent false
6371 2549 Acked Read 3 Done Done Sent false
6372 2550 Acked Write 2 Done Done Sent false
6373 2551 Acked Write 10 Done Done Sent false
6374 2552 Acked Read 9 Done Done Sent false
6375 2553 Acked Write 9 Done Done Sent false
6376 2554 Acked Write 7 Done Done Sent false
6377 2555 Acked Read 9 Done Done Sent false
6378 2556 Acked Flush 0 Done Done Sent false
6379 2557 Acked Write 2 Done Done Sent false
6380 2558 Acked Read 5 Done Done Sent false
6381 2559 Acked Write 10 Done Done Sent false
6382 2560 Acked Read 8 Done Done Sent false
6383 2561 Acked Write 2 Done Done Sent false
6384 2562 Acked Read 10 Done Done Sent false
6385 2563 Acked Read 3 Done Done Sent false
6386 2564 Acked Read 7 Done Done Sent false
6387 2565 Acked Write 6 Done Done Sent false
6388 2566 Acked Write 6 Done Done Sent false
6389 2567 Acked Read 4 Done Done Sent false
6390 2568 Acked Flush 0 Done Done Sent false
6391 2569 Acked Write 10 Done Done Sent false
6392 2570 Acked Read 6 Done Done Sent false
6393 2571 Acked Write 2 Done Done Sent false
6394 2572 Acked Read 7 Done Done Sent false
6395 2573 Acked Write 6 Done Done Sent false
6396 2574 Acked Write 5 Done Done Sent false
6397 2575 Acked Read 5 Done Done Sent false
6398 2576 Acked Read 5 Done Done Sent false
6399 2577 Acked Read 10 Done Done Sent false
6400 2578 Acked Write 7 Done Done Sent false
6401 2579 Acked Read 5 Done Done Sent false
6402 2580 Acked Read 10 Done Done Sent false
6403 2581 Acked Write 7 Done Done Sent false
6404 2582 Acked Read 9 Done Done Sent false
6405 2583 Acked Read 6 Done Done Sent false
6406 2584 Acked Flush 0 Done Done Sent false
6407 2585 Acked Read 4 Done Done Sent false
6408 2586 Acked Read 7 Done Done Sent false
6409 2587 Acked Write 9 Done Done Sent false
6410 2588 Acked Flush 0 Done Done Sent false
6411 2589 Acked Write 4 Done Done Sent false
6412 2590 Acked Read 9 Done Done Sent false
6413 2591 Acked Read 5 Done Done Sent false
6414 2592 Acked Read 4 Done Done Sent false
6415 2593 Acked Read 1 Done Done Sent false
6416 2594 Acked Read 1 Done Done Sent false
6417 2595 Acked Read 2 Done Done Sent false
6418 2596 Acked Read 7 Done Done Sent false
6419 2597 Acked Read 5 Done Done Sent false
6420 2598 Acked Read 2 Done Done Sent false
6421 2599 Acked Read 10 Done Done Sent false
6422 2600 Acked Write 7 Done Done Sent false
6423 2601 Acked Write 10 Done Done Sent false
6424 2602 Acked Write 1 Done Done Sent false
6425 2603 Acked Write 6 Done Done Sent false
6426 2604 Acked Read 10 Done Done Sent false
6427 2605 Acked Read 4 Done Done Sent false
6428 2606 Acked Read 1 Done Done Sent false
6429 2607 Acked Flush 0 Done Done Sent false
6430 2608 Acked Read 4 Done Done Sent false
6431 2609 Acked Write 10 Done Done Sent false
6432 2610 Acked Read 8 Done Done Sent false
6433 2611 Acked Read 4 Done Done Sent false
6434 2612 Acked Read 5 Done Done Sent false
6435 2613 Acked Read 4 Done Done Sent false
6436 2614 Acked Flush 0 Done Done Sent false
6437 2615 Acked Read 4 Done Done Sent false
6438 2616 Acked Write 7 Done Done Sent false
6439 2617 Acked Write 4 Done Done Sent false
6440 2618 Acked Flush 0 Done Done Sent false
6441 2619 Acked Write 10 Done Done Sent false
6442 2620 Acked Read 5 Done Done Sent false
6443 2621 Acked Read 10 Done Done Sent false
6444 2622 Acked Read 4 Done Done Sent false
6445 2623 Acked Write 6 Done Done Sent false
6446 2624 Acked Read 6 Done Done Sent false
6447 2625 Acked Flush 0 Done Done Sent false
6448 2626 Acked Read 2 Done Done Sent false
6449 2627 Acked Read 5 Done Done Sent false
6450 2628 Acked Flush 0 Done Done Sent false
6451 2629 Acked Read 10 Done Done Sent false
6452 2630 Acked Write 5 Done Done Sent false
6453 2631 Acked Write 1 Done Done Sent false
6454 2632 Acked Read 3 Done Done Sent false
6455 2633 Acked Write 7 Done Done Sent false
6456 2634 Acked Write 7 Done Done Sent false
6457 2635 Acked Write 2 Done Done Sent false
6458 2636 Acked Write 6 Done Done Sent false
6459 2637 Acked Read 3 Done Done Sent false
6460 2638 Acked Read 6 Done Done Sent false
6461 2639 Acked Write 1 Done Done Sent false
6462 2640 Acked Flush 0 Done Done Sent false
6463 2641 Acked Flush 0 Done Done Sent false
6464 2642 Acked Read 5 Done Done Sent false
6465 2643 Acked Write 4 Done Done Sent false
6466 2644 Acked Read 10 Done Done Sent false
6467 2645 Acked Write 6 Done Done Sent false
6468 2646 Acked Read 1 Done Done Sent false
6469 2647 Acked Write 8 Done Done Sent false
6470 2648 Acked Read 8 Done Done Sent false
6471 2649 Acked Read 6 Done Done Sent false
6472 2650 Acked Write 4 Done Done Sent false
6473 2651 Acked Read 9 Done Done Sent false
6474 2652 Acked Read 7 Done Done Sent false
6475 2653 Acked Write 6 Done Done Sent false
6476 2654 Acked Read 6 Done Done Sent false
6477 2655 Acked Read 2 Done Done Sent false
6478 2656 Acked Flush 0 Done Done Sent false
6479 2657 Acked Write 5 Done Done Sent false
6480 2658 Acked Read 6 Done Done Sent false
6481 2659 Acked Read 5 Done Done Sent false
6482 2660 Acked Write 6 Done Done Sent false
6483 2661 Acked Read 6 Done Done Sent false
6484 2662 Acked Read 6 Done Done Sent false
6485 2663 Acked Read 10 Done Done Sent false
6486 2664 Acked Write 10 Done Done Sent false
6487 2665 Acked Read 9 Done Done Sent false
6488 2666 Acked Read 8 Done Done Sent false
6489 2667 Acked Write 3 Done Done Sent false
6490 2668 Acked Read 5 Done Done Sent false
6491 2669 Acked Read 3 Done Done Sent false
6492 2670 Acked Read 6 Done Done Sent false
6493 2671 Acked Write 1 Done Done Sent false
6494 2672 Acked Write 6 Done Done Sent false
6495 2673 Acked Write 7 Done Done Sent false
6496 2674 Acked Write 8 Done Done Sent false
6497 2675 Acked Read 2 Done Done Sent false
6498 2676 Acked Read 3 Done Done Sent false
6499 2677 Acked Read 4 Done Done Sent false
6500 2678 Acked Write 10 Done Done Sent false
6501 2679 Acked Read 10 Done Done Sent false
6502 2680 Acked Read 10 Done Done Sent false
6503 2681 Acked Read 2 Done Done Sent false
6504 2682 Acked Write 3 Done Done Sent false
6505 2683 Acked Read 4 Done Done Sent false
6506 2684 Acked Read 1 Done Done Sent false
6507 2685 Acked Read 7 Done Done Sent false
6508 2686 Acked Write 1 Done Done Sent false
6509 2687 Acked Write 1 Done Done Sent false
6510 2688 Acked Write 5 Done Done Sent false
6511 2689 Acked Read 9 Done Done Sent false
6512 2690 Acked Write 8 Done Done Sent false
6513 2691 Acked Read 1 Done Done Sent false
6514 2692 Acked Read 3 Done Done Sent false
6515 2693 Acked Write 4 Done Done Sent false
6516 2694 Acked Read 5 Done Done Sent false
6517 2695 Acked Read 8 Done Done Sent false
6518 2696 Acked Read 5 Done Done Sent false
6519 2697 Acked Write 5 Done Done Sent false
6520 2698 Acked Flush 0 Done Done Sent false
6521 2699 Acked Write 8 Done Done Sent false
6522 2700 Acked Write 7 Done Done Sent false
6523 2701 Acked Flush 0 Done Done Sent false
6524 2702 Acked Read 6 Done Done Sent false
6525 2703 Acked Read 2 Done Done Sent false
6526 2704 Acked Write 2 Done Done Sent false
6527 2705 Acked Read 2 Done Done Sent false
6528 2706 Acked Read 7 Done Done Sent false
6529 2707 Acked Read 3 Done Done Sent false
6530 2708 Acked Read 8 Done Done Sent false
6531 2709 Acked Read 10 Done Done Sent false
6532 2710 Acked Read 4 Done Done Sent false
6533 2711 Acked Write 10 Done Done Sent false
6534 2712 Acked Flush 0 Done Done Sent false
6535 2713 Acked Read 5 Done Done Sent false
6536 2714 Acked Write 2 Done Done Sent false
6537 2715 Acked Write 8 Done Done Sent false
6538 2716 Acked Flush 0 Done Done Sent false
6539 2717 Acked Flush 0 Done Done Sent false
6540 2718 Acked Write 7 Done Done Sent false
6541 2719 Acked Flush 0 Done Done Sent false
6542 2720 Acked Read 6 Done Done Sent false
6543 2721 Acked Read 7 Done Done Sent false
6544 2722 Acked Read 3 Done Done Sent false
6545 2723 Acked Read 1 Done Done Sent false
6546 2724 Acked Read 7 Done Done Sent false
6547 2725 Acked Write 2 Done Done Sent false
6548 2726 Acked Read 2 Done Done Sent false
6549 2727 Acked Read 10 Done Done Sent false
6550 2728 Acked Read 6 Done Done Sent false
6551 2729 Acked Write 5 Done Done Sent false
6552 2730 Acked Flush 0 Done Done Sent false
6553 2731 Acked Write 4 Done Done Sent false
6554 2732 Acked Write 8 Done Done Sent false
6555 2733 Acked Read 3 Done Done Sent false
6556 2734 Acked Read 9 Done Done Sent false
6557 2735 Acked Flush 0 Done Done Sent false
6558 2736 Acked Flush 0 Done Done Sent false
6559 2737 Acked Read 5 Done Done Sent false
6560 2738 Acked Write 4 Done Done Sent false
6561 2739 Acked Write 3 Done Done Sent false
6562 2740 Acked Write 10 Done Done Sent false
6563 2741 Acked Write 6 Done Done Sent false
6564 2742 Acked Write 10 Done Done Sent false
6565 2743 Acked Write 1 Done Done Sent false
6566 2744 Acked Read 6 Done Done Sent false
6567 2745 Acked Write 5 Done Done Sent false
6568 2746 Acked Read 10 Done Done Sent false
6569 2747 Acked Write 7 Done Done Sent false
6570 2748 Acked Read 7 Done Done Sent false
6571 2749 Acked Write 1 Done Done Sent false
6572 2750 Acked Write 1 Done Done Sent false
6573 2751 Acked Read 2 Done Done Sent false
6574 2752 Acked Read 5 Done Done Sent false
6575 2753 Acked Write 10 Done Done Sent false
6576 2754 Acked Write 8 Done Done Sent false
6577 2755 Acked Write 4 Done Done Sent false
6578 2756 Acked Write 5 Done Done Sent false
6579 2757 Acked Read 5 Done Done Sent false
6580 2758 Acked Write 9 Done Done Sent false
6581 2759 Acked Write 1 Done Done Sent false
6582 2760 Acked Read 10 Done Done Sent false
6583 2761 Acked Write 6 Done Done Sent false
6584 2762 Acked Write 2 Done Done Sent false
6585 2763 Acked Read 2 Done Done Sent false
6586 2764 Acked Read 1 Done Done Sent false
6587 2765 Acked Write 5 Done Done Sent false
6588 2766 Acked Read 2 Done Done Sent false
6589 2767 Acked Read 9 Done Done Sent false
6590 2768 Acked Write 10 Done Done Sent false
6591 2769 Acked Write 7 Done Done Sent false
6592 2770 Acked Read 7 Done Done Sent false
6593 2771 Acked Read 2 Done Done Sent false
6594 2772 Acked Write 9 Done Done Sent false
6595 2773 Acked Write 8 Done Done Sent false
6596 2774 Acked Write 7 Done Done Sent false
6597 2775 Acked Write 7 Done Done Sent false
6598 2776 Acked Read 5 Done Done Sent false
6599 2777 Acked Write 2 Done Done Sent false
6600 2778 Acked Flush 0 Done Done Sent false
6601 2779 Acked Read 10 Done Done Sent false
6602 2780 Acked Write 1 Done Done Sent false
6603 2781 Acked Flush 0 Done Done Sent false
6604 2782 Acked Read 1 Done Done Sent false
6605 2783 Acked Read 5 Done Done Sent false
6606 2784 Acked Read 6 Done Done Sent false
6607 2785 Acked Read 5 Done Done Sent false
6608 2786 Acked Write 1 Done Done Sent false
6609 2787 Acked Write 9 Done Done Sent false
6610 2788 Acked Write 1 Done Done Sent false
6611 2789 Acked Write 2 Done Done Sent false
6612 2790 Acked Read 8 Done Done Sent false
6613 2791 Acked Read 5 Done Done Sent false
6614 2792 Acked Write 4 Done Done Sent false
6615 2793 Acked Read 4 Done Done Sent false
6616 2794 Acked Write 8 Done Done Sent false
6617 2795 Acked Read 8 Done Done Sent false
6618 2796 Acked Read 7 Done Done Sent false
6619 2797 Acked Read 10 Done Done Sent false
6620 2798 Acked Read 5 Done Done Sent false
6621 2799 Acked Read 4 Done Done Sent false
6622 2800 Acked Write 5 Done Done Sent false
6623 2801 Acked Write 9 Done Done Sent false
6624 2802 Acked Read 1 Done Done Sent false
6625 2803 Acked Read 2 Done Done Sent false
6626 2804 Acked Write 8 Done Done Sent false
6627 2805 Acked Flush 0 Done Done Sent false
6628 2806 Acked Write 3 Done Done Sent false
6629 2807 Acked Read 5 Done Done Sent false
6630 2808 Acked Read 10 Done Done Sent false
6631 2809 Acked Read 1 Done Done Sent false
6632 2810 Acked Write 1 Done Done Sent false
6633 2811 Acked Flush 0 Done Done Sent false
6634 STATES DS:0 DS:1 DS:2 TOTAL
6635 Sent 0 0 302 302
6636 Done 302 302 0 604
6637 Skipped 0 0 0 0
6638 Error 0 0 0 0
6639 Last Flush: 2811 2811 2509
6640 Downstairs last five completed: 2509 2508 2507 2506 2505
6641 Upstairs last five completed: 2811 2810 2809 2808 2807
6642 CLIENT: Up:0 ds:302 act:2
66432025-01-13T15:53:10.292ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
66442025-01-13T15:53:10.292ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
66452025-01-13T15:53:10.292ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
66462025-01-13T15:53:10.292ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
66472025-01-13T15:53:10.292ZINFOcrucible: client task is exiting = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
6648 [8830] got start action
6649 [8830] I am going to restart
6650 Starting downstairs at port 8830
6651 Make output file at "/tmp/dsc/downstairs-8830.txt"
6652 [8830][2] reports Starting
6653 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1208)
6654 [8830][2] reports Running
6655 ----------------------------------------------------------------
6656 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6657 States: Active Active Offline
6658 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6659 2510 Acked Read 7 Done Done Sent false
6660 2511 Acked Write 3 Done Done Sent false
6661 2512 Acked Write 10 Done Done Sent false
6662 2513 Acked Write 5 Done Done Sent false
6663 2514 Acked Write 6 Done Done Sent false
6664 2515 Acked Read 5 Done Done Sent false
6665 2516 Acked Read 6 Done Done Sent false
6666 2517 Acked Write 9 Done Done Sent false
6667 2518 Acked Read 10 Done Done Sent false
6668 2519 Acked Flush 0 Done Done Sent false
6669 2520 Acked Read 10 Done Done Sent false
6670 2521 Acked Read 10 Done Done Sent false
6671 2522 Acked Write 1 Done Done Sent false
6672 2523 Acked Write 3 Done Done Sent false
6673 2524 Acked Flush 0 Done Done Sent false
6674 2525 Acked Read 3 Done Done Sent false
6675 2526 Acked Read 5 Done Done Sent false
6676 2527 Acked Flush 0 Done Done Sent false
6677 2528 Acked Flush 0 Done Done Sent false
6678 2529 Acked Write 5 Done Done Sent false
6679 2530 Acked Read 10 Done Done Sent false
6680 2531 Acked Write 2 Done Done Sent false
6681 2532 Acked Read 3 Done Done Sent false
6682 2533 Acked Read 8 Done Done Sent false
6683 2534 Acked Flush 0 Done Done Sent false
6684 2535 Acked Read 10 Done Done Sent false
6685 2536 Acked Flush 0 Done Done Sent false
6686 2537 Acked Write 3 Done Done Sent false
6687 2538 Acked Write 10 Done Done Sent false
6688 2539 Acked Write 5 Done Done Sent false
6689 2540 Acked Write 7 Done Done Sent false
6690 2541 Acked Read 5 Done Done Sent false
6691 2542 Acked Write 8 Done Done Sent false
6692 2543 Acked Read 2 Done Done Sent false
6693 2544 Acked Write 2 Done Done Sent false
6694 2545 Acked Write 8 Done Done Sent false
6695 2546 Acked Read 4 Done Done Sent false
6696 2547 Acked Flush 0 Done Done Sent false
6697 2548 Acked Write 4 Done Done Sent false
6698 2549 Acked Read 3 Done Done Sent false
6699 2550 Acked Write 2 Done Done Sent false
6700 2551 Acked Write 10 Done Done Sent false
6701 2552 Acked Read 9 Done Done Sent false
6702 2553 Acked Write 9 Done Done Sent false
6703 2554 Acked Write 7 Done Done Sent false
6704 2555 Acked Read 9 Done Done Sent false
6705 2556 Acked Flush 0 Done Done Sent false
6706 2557 Acked Write 2 Done Done Sent false
6707 2558 Acked Read 5 Done Done Sent false
6708 2559 Acked Write 10 Done Done Sent false
6709 2560 Acked Read 8 Done Done Sent false
6710 2561 Acked Write 2 Done Done Sent false
6711 2562 Acked Read 10 Done Done Sent false
6712 2563 Acked Read 3 Done Done Sent false
6713 2564 Acked Read 7 Done Done Sent false
6714 2565 Acked Write 6 Done Done Sent false
6715 2566 Acked Write 6 Done Done Sent false
6716 2567 Acked Read 4 Done Done Sent false
6717 2568 Acked Flush 0 Done Done Sent false
6718 2569 Acked Write 10 Done Done Sent false
6719 2570 Acked Read 6 Done Done Sent false
6720 2571 Acked Write 2 Done Done Sent false
6721 2572 Acked Read 7 Done Done Sent false
6722 2573 Acked Write 6 Done Done Sent false
6723 2574 Acked Write 5 Done Done Sent false
6724 2575 Acked Read 5 Done Done Sent false
6725 2576 Acked Read 5 Done Done Sent false
6726 2577 Acked Read 10 Done Done Sent false
6727 2578 Acked Write 7 Done Done Sent false
6728 2579 Acked Read 5 Done Done Sent false
6729 2580 Acked Read 10 Done Done Sent false
6730 2581 Acked Write 7 Done Done Sent false
6731 2582 Acked Read 9 Done Done Sent false
6732 2583 Acked Read 6 Done Done Sent false
6733 2584 Acked Flush 0 Done Done Sent false
6734 2585 Acked Read 4 Done Done Sent false
6735 2586 Acked Read 7 Done Done Sent false
6736 2587 Acked Write 9 Done Done Sent false
6737 2588 Acked Flush 0 Done Done Sent false
6738 2589 Acked Write 4 Done Done Sent false
6739 2590 Acked Read 9 Done Done Sent false
6740 2591 Acked Read 5 Done Done Sent false
6741 2592 Acked Read 4 Done Done Sent false
6742 2593 Acked Read 1 Done Done Sent false
6743 2594 Acked Read 1 Done Done Sent false
6744 2595 Acked Read 2 Done Done Sent false
6745 2596 Acked Read 7 Done Done Sent false
6746 2597 Acked Read 5 Done Done Sent false
6747 2598 Acked Read 2 Done Done Sent false
6748 2599 Acked Read 10 Done Done Sent false
6749 2600 Acked Write 7 Done Done Sent false
6750 2601 Acked Write 10 Done Done Sent false
6751 2602 Acked Write 1 Done Done Sent false
6752 2603 Acked Write 6 Done Done Sent false
6753 2604 Acked Read 10 Done Done Sent false
6754 2605 Acked Read 4 Done Done Sent false
6755 2606 Acked Read 1 Done Done Sent false
6756 2607 Acked Flush 0 Done Done Sent false
6757 2608 Acked Read 4 Done Done Sent false
6758 2609 Acked Write 10 Done Done Sent false
6759 2610 Acked Read 8 Done Done Sent false
6760 2611 Acked Read 4 Done Done Sent false
6761 2612 Acked Read 5 Done Done Sent false
6762 2613 Acked Read 4 Done Done Sent false
6763 2614 Acked Flush 0 Done Done Sent false
6764 2615 Acked Read 4 Done Done Sent false
6765 2616 Acked Write 7 Done Done Sent false
6766 2617 Acked Write 4 Done Done Sent false
6767 2618 Acked Flush 0 Done Done Sent false
6768 2619 Acked Write 10 Done Done Sent false
6769 2620 Acked Read 5 Done Done Sent false
6770 2621 Acked Read 10 Done Done Sent false
6771 2622 Acked Read 4 Done Done Sent false
6772 2623 Acked Write 6 Done Done Sent false
6773 2624 Acked Read 6 Done Done Sent false
6774 2625 Acked Flush 0 Done Done Sent false
6775 2626 Acked Read 2 Done Done Sent false
6776 2627 Acked Read 5 Done Done Sent false
6777 2628 Acked Flush 0 Done Done Sent false
6778 2629 Acked Read 10 Done Done Sent false
6779 2630 Acked Write 5 Done Done Sent false
6780 2631 Acked Write 1 Done Done Sent false
6781 2632 Acked Read 3 Done Done Sent false
6782 2633 Acked Write 7 Done Done Sent false
6783 2634 Acked Write 7 Done Done Sent false
6784 2635 Acked Write 2 Done Done Sent false
6785 2636 Acked Write 6 Done Done Sent false
6786 2637 Acked Read 3 Done Done Sent false
6787 2638 Acked Read 6 Done Done Sent false
6788 2639 Acked Write 1 Done Done Sent false
6789 2640 Acked Flush 0 Done Done Sent false
6790 2641 Acked Flush 0 Done Done Sent false
6791 2642 Acked Read 5 Done Done Sent false
6792 2643 Acked Write 4 Done Done Sent false
6793 2644 Acked Read 10 Done Done Sent false
6794 2645 Acked Write 6 Done Done Sent false
6795 2646 Acked Read 1 Done Done Sent false
6796 2647 Acked Write 8 Done Done Sent false
6797 2648 Acked Read 8 Done Done Sent false
6798 2649 Acked Read 6 Done Done Sent false
6799 2650 Acked Write 4 Done Done Sent false
6800 2651 Acked Read 9 Done Done Sent false
6801 2652 Acked Read 7 Done Done Sent false
6802 2653 Acked Write 6 Done Done Sent false
6803 2654 Acked Read 6 Done Done Sent false
6804 2655 Acked Read 2 Done Done Sent false
6805 2656 Acked Flush 0 Done Done Sent false
6806 2657 Acked Write 5 Done Done Sent false
6807 2658 Acked Read 6 Done Done Sent false
6808 2659 Acked Read 5 Done Done Sent false
6809 2660 Acked Write 6 Done Done Sent false
6810 2661 Acked Read 6 Done Done Sent false
6811 2662 Acked Read 6 Done Done Sent false
6812 2663 Acked Read 10 Done Done Sent false
6813 2664 Acked Write 10 Done Done Sent false
6814 2665 Acked Read 9 Done Done Sent false
6815 2666 Acked Read 8 Done Done Sent false
6816 2667 Acked Write 3 Done Done Sent false
6817 2668 Acked Read 5 Done Done Sent false
6818 2669 Acked Read 3 Done Done Sent false
6819 2670 Acked Read 6 Done Done Sent false
6820 2671 Acked Write 1 Done Done Sent false
6821 2672 Acked Write 6 Done Done Sent false
6822 2673 Acked Write 7 Done Done Sent false
6823 2674 Acked Write 8 Done Done Sent false
6824 2675 Acked Read 2 Done Done Sent false
6825 2676 Acked Read 3 Done Done Sent false
6826 2677 Acked Read 4 Done Done Sent false
6827 2678 Acked Write 10 Done Done Sent false
6828 2679 Acked Read 10 Done Done Sent false
6829 2680 Acked Read 10 Done Done Sent false
6830 2681 Acked Read 2 Done Done Sent false
6831 2682 Acked Write 3 Done Done Sent false
6832 2683 Acked Read 4 Done Done Sent false
6833 2684 Acked Read 1 Done Done Sent false
6834 2685 Acked Read 7 Done Done Sent false
6835 2686 Acked Write 1 Done Done Sent false
6836 2687 Acked Write 1 Done Done Sent false
6837 2688 Acked Write 5 Done Done Sent false
6838 2689 Acked Read 9 Done Done Sent false
6839 2690 Acked Write 8 Done Done Sent false
6840 2691 Acked Read 1 Done Done Sent false
6841 2692 Acked Read 3 Done Done Sent false
6842 2693 Acked Write 4 Done Done Sent false
6843 2694 Acked Read 5 Done Done Sent false
6844 2695 Acked Read 8 Done Done Sent false
6845 2696 Acked Read 5 Done Done Sent false
6846 2697 Acked Write 5 Done Done Sent false
6847 2698 Acked Flush 0 Done Done Sent false
6848 2699 Acked Write 8 Done Done Sent false
6849 2700 Acked Write 7 Done Done Sent false
6850 2701 Acked Flush 0 Done Done Sent false
6851 2702 Acked Read 6 Done Done Sent false
6852 2703 Acked Read 2 Done Done Sent false
6853 2704 Acked Write 2 Done Done Sent false
6854 2705 Acked Read 2 Done Done Sent false
6855 2706 Acked Read 7 Done Done Sent false
6856 2707 Acked Read 3 Done Done Sent false
6857 2708 Acked Read 8 Done Done Sent false
6858 2709 Acked Read 10 Done Done Sent false
6859 2710 Acked Read 4 Done Done Sent false
6860 2711 Acked Write 10 Done Done Sent false
6861 2712 Acked Flush 0 Done Done Sent false
6862 2713 Acked Read 5 Done Done Sent false
6863 2714 Acked Write 2 Done Done Sent false
6864 2715 Acked Write 8 Done Done Sent false
6865 2716 Acked Flush 0 Done Done Sent false
6866 2717 Acked Flush 0 Done Done Sent false
6867 2718 Acked Write 7 Done Done Sent false
6868 2719 Acked Flush 0 Done Done Sent false
6869 2720 Acked Read 6 Done Done Sent false
6870 2721 Acked Read 7 Done Done Sent false
6871 2722 Acked Read 3 Done Done Sent false
6872 2723 Acked Read 1 Done Done Sent false
6873 2724 Acked Read 7 Done Done Sent false
6874 2725 Acked Write 2 Done Done Sent false
6875 2726 Acked Read 2 Done Done Sent false
6876 2727 Acked Read 10 Done Done Sent false
6877 2728 Acked Read 6 Done Done Sent false
6878 2729 Acked Write 5 Done Done Sent false
6879 2730 Acked Flush 0 Done Done Sent false
6880 2731 Acked Write 4 Done Done Sent false
6881 2732 Acked Write 8 Done Done Sent false
6882 2733 Acked Read 3 Done Done Sent false
6883 2734 Acked Read 9 Done Done Sent false
6884 2735 Acked Flush 0 Done Done Sent false
6885 2736 Acked Flush 0 Done Done Sent false
6886 2737 Acked Read 5 Done Done Sent false
6887 2738 Acked Write 4 Done Done Sent false
6888 2739 Acked Write 3 Done Done Sent false
6889 2740 Acked Write 10 Done Done Sent false
6890 2741 Acked Write 6 Done Done Sent false
6891 2742 Acked Write 10 Done Done Sent false
6892 2743 Acked Write 1 Done Done Sent false
6893 2744 Acked Read 6 Done Done Sent false
6894 2745 Acked Write 5 Done Done Sent false
6895 2746 Acked Read 10 Done Done Sent false
6896 2747 Acked Write 7 Done Done Sent false
6897 2748 Acked Read 7 Done Done Sent false
6898 2749 Acked Write 1 Done Done Sent false
6899 2750 Acked Write 1 Done Done Sent false
6900 2751 Acked Read 2 Done Done Sent false
6901 2752 Acked Read 5 Done Done Sent false
6902 2753 Acked Write 10 Done Done Sent false
6903 2754 Acked Write 8 Done Done Sent false
6904 2755 Acked Write 4 Done Done Sent false
6905 2756 Acked Write 5 Done Done Sent false
6906 2757 Acked Read 5 Done Done Sent false
6907 2758 Acked Write 9 Done Done Sent false
6908 2759 Acked Write 1 Done Done Sent false
6909 2760 Acked Read 10 Done Done Sent false
6910 2761 Acked Write 6 Done Done Sent false
6911 2762 Acked Write 2 Done Done Sent false
6912 2763 Acked Read 2 Done Done Sent false
6913 2764 Acked Read 1 Done Done Sent false
6914 2765 Acked Write 5 Done Done Sent false
6915 2766 Acked Read 2 Done Done Sent false
6916 2767 Acked Read 9 Done Done Sent false
6917 2768 Acked Write 10 Done Done Sent false
6918 2769 Acked Write 7 Done Done Sent false
6919 2770 Acked Read 7 Done Done Sent false
6920 2771 Acked Read 2 Done Done Sent false
6921 2772 Acked Write 9 Done Done Sent false
6922 2773 Acked Write 8 Done Done Sent false
6923 2774 Acked Write 7 Done Done Sent false
6924 2775 Acked Write 7 Done Done Sent false
6925 2776 Acked Read 5 Done Done Sent false
6926 2777 Acked Write 2 Done Done Sent false
6927 2778 Acked Flush 0 Done Done Sent false
6928 2779 Acked Read 10 Done Done Sent false
6929 2780 Acked Write 1 Done Done Sent false
6930 2781 Acked Flush 0 Done Done Sent false
6931 2782 Acked Read 1 Done Done Sent false
6932 2783 Acked Read 5 Done Done Sent false
6933 2784 Acked Read 6 Done Done Sent false
6934 2785 Acked Read 5 Done Done Sent false
6935 2786 Acked Write 1 Done Done Sent false
6936 2787 Acked Write 9 Done Done Sent false
6937 2788 Acked Write 1 Done Done Sent false
6938 2789 Acked Write 2 Done Done Sent false
6939 2790 Acked Read 8 Done Done Sent false
6940 2791 Acked Read 5 Done Done Sent false
6941 2792 Acked Write 4 Done Done Sent false
6942 2793 Acked Read 4 Done Done Sent false
6943 2794 Acked Write 8 Done Done Sent false
6944 2795 Acked Read 8 Done Done Sent false
6945 2796 Acked Read 7 Done Done Sent false
6946 2797 Acked Read 10 Done Done Sent false
6947 2798 Acked Read 5 Done Done Sent false
6948 2799 Acked Read 4 Done Done Sent false
6949 2800 Acked Write 5 Done Done Sent false
6950 2801 Acked Write 9 Done Done Sent false
6951 2802 Acked Read 1 Done Done Sent false
6952 2803 Acked Read 2 Done Done Sent false
6953 2804 Acked Write 8 Done Done Sent false
6954 2805 Acked Flush 0 Done Done Sent false
6955 2806 Acked Write 3 Done Done Sent false
6956 2807 Acked Read 5 Done Done Sent false
6957 2808 Acked Read 10 Done Done Sent false
6958 2809 Acked Read 1 Done Done Sent false
6959 2810 Acked Write 1 Done Done Sent false
6960 2811 Acked Flush 0 Done Done Sent false
6961 STATES DS:0 DS:1 DS:2 TOTAL
6962 Sent 0 0 302 302
6963 Done 302 302 0 604
6964 Skipped 0 0 0 0
6965 Error 0 0 0 0
6966 Last Flush: 2811 2811 2509
6967 Downstairs last five completed: 2509 2508 2507 2506 2505
6968 Upstairs last five completed: 2811 2810 2809 2808 2807
6969 CLIENT: Up:0 ds:302 act:2
6970 ----------------------------------------------------------------
6971 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6972 States: Active Active Offline
6973 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6974 2510 Acked Read 7 Done Done Sent false
6975 2511 Acked Write 3 Done Done Sent false
6976 2512 Acked Write 10 Done Done Sent false
6977 2513 Acked Write 5 Done Done Sent false
6978 2514 Acked Write 6 Done Done Sent false
6979 2515 Acked Read 5 Done Done Sent false
6980 2516 Acked Read 6 Done Done Sent false
6981 2517 Acked Write 9 Done Done Sent false
6982 2518 Acked Read 10 Done Done Sent false
6983 2519 Acked Flush 0 Done Done Sent false
6984 2520 Acked Read 10 Done Done Sent false
6985 2521 Acked Read 10 Done Done Sent false
6986 2522 Acked Write 1 Done Done Sent false
6987 2523 Acked Write 3 Done Done Sent false
6988 2524 Acked Flush 0 Done Done Sent false
6989 2525 Acked Read 3 Done Done Sent false
6990 2526 Acked Read 5 Done Done Sent false
6991 2527 Acked Flush 0 Done Done Sent false
6992 2528 Acked Flush 0 Done Done Sent false
6993 2529 Acked Write 5 Done Done Sent false
6994 2530 Acked Read 10 Done Done Sent false
6995 2531 Acked Write 2 Done Done Sent false
6996 2532 Acked Read 3 Done Done Sent false
6997 2533 Acked Read 8 Done Done Sent false
6998 2534 Acked Flush 0 Done Done Sent false
6999 2535 Acked Read 10 Done Done Sent false
7000 2536 Acked Flush 0 Done Done Sent false
7001 2537 Acked Write 3 Done Done Sent false
7002 2538 Acked Write 10 Done Done Sent false
7003 2539 Acked Write 5 Done Done Sent false
7004 2540 Acked Write 7 Done Done Sent false
7005 2541 Acked Read 5 Done Done Sent false
7006 2542 Acked Write 8 Done Done Sent false
7007 2543 Acked Read 2 Done Done Sent false
7008 2544 Acked Write 2 Done Done Sent false
7009 2545 Acked Write 8 Done Done Sent false
7010 2546 Acked Read 4 Done Done Sent false
7011 2547 Acked Flush 0 Done Done Sent false
7012 2548 Acked Write 4 Done Done Sent false
7013 2549 Acked Read 3 Done Done Sent false
7014 2550 Acked Write 2 Done Done Sent false
7015 2551 Acked Write 10 Done Done Sent false
7016 2552 Acked Read 9 Done Done Sent false
7017 2553 Acked Write 9 Done Done Sent false
7018 2554 Acked Write 7 Done Done Sent false
7019 2555 Acked Read 9 Done Done Sent false
7020 2556 Acked Flush 0 Done Done Sent false
7021 2557 Acked Write 2 Done Done Sent false
7022 2558 Acked Read 5 Done Done Sent false
7023 2559 Acked Write 10 Done Done Sent false
7024 2560 Acked Read 8 Done Done Sent false
7025 2561 Acked Write 2 Done Done Sent false
7026 2562 Acked Read 10 Done Done Sent false
7027 2563 Acked Read 3 Done Done Sent false
7028 2564 Acked Read 7 Done Done Sent false
7029 2565 Acked Write 6 Done Done Sent false
7030 2566 Acked Write 6 Done Done Sent false
7031 2567 Acked Read 4 Done Done Sent false
7032 2568 Acked Flush 0 Done Done Sent false
7033 2569 Acked Write 10 Done Done Sent false
7034 2570 Acked Read 6 Done Done Sent false
7035 2571 Acked Write 2 Done Done Sent false
7036 2572 Acked Read 7 Done Done Sent false
7037 2573 Acked Write 6 Done Done Sent false
7038 2574 Acked Write 5 Done Done Sent false
7039 2575 Acked Read 5 Done Done Sent false
7040 2576 Acked Read 5 Done Done Sent false
7041 2577 Acked Read 10 Done Done Sent false
7042 2578 Acked Write 7 Done Done Sent false
7043 2579 Acked Read 5 Done Done Sent false
7044 2580 Acked Read 10 Done Done Sent false
7045 2581 Acked Write 7 Done Done Sent false
7046 2582 Acked Read 9 Done Done Sent false
7047 2583 Acked Read 6 Done Done Sent false
7048 2584 Acked Flush 0 Done Done Sent false
7049 2585 Acked Read 4 Done Done Sent false
7050 2586 Acked Read 7 Done Done Sent false
7051 2587 Acked Write 9 Done Done Sent false
7052 2588 Acked Flush 0 Done Done Sent false
7053 2589 Acked Write 4 Done Done Sent false
7054 2590 Acked Read 9 Done Done Sent false
7055 2591 Acked Read 5 Done Done Sent false
7056 2592 Acked Read 4 Done Done Sent false
7057 2593 Acked Read 1 Done Done Sent false
7058 2594 Acked Read 1 Done Done Sent false
7059 2595 Acked Read 2 Done Done Sent false
7060 2596 Acked Read 7 Done Done Sent false
7061 2597 Acked Read 5 Done Done Sent false
7062 2598 Acked Read 2 Done Done Sent false
7063 2599 Acked Read 10 Done Done Sent false
7064 2600 Acked Write 7 Done Done Sent false
7065 2601 Acked Write 10 Done Done Sent false
7066 2602 Acked Write 1 Done Done Sent false
7067 2603 Acked Write 6 Done Done Sent false
7068 2604 Acked Read 10 Done Done Sent false
7069 2605 Acked Read 4 Done Done Sent false
7070 2606 Acked Read 1 Done Done Sent false
7071 2607 Acked Flush 0 Done Done Sent false
7072 2608 Acked Read 4 Done Done Sent false
7073 2609 Acked Write 10 Done Done Sent false
7074 2610 Acked Read 8 Done Done Sent false
7075 2611 Acked Read 4 Done Done Sent false
7076 2612 Acked Read 5 Done Done Sent false
7077 2613 Acked Read 4 Done Done Sent false
7078 2614 Acked Flush 0 Done Done Sent false
7079 2615 Acked Read 4 Done Done Sent false
7080 2616 Acked Write 7 Done Done Sent false
7081 2617 Acked Write 4 Done Done Sent false
7082 2618 Acked Flush 0 Done Done Sent false
7083 2619 Acked Write 10 Done Done Sent false
7084 2620 Acked Read 5 Done Done Sent false
7085 2621 Acked Read 10 Done Done Sent false
7086 2622 Acked Read 4 Done Done Sent false
7087 2623 Acked Write 6 Done Done Sent false
7088 2624 Acked Read 6 Done Done Sent false
7089 2625 Acked Flush 0 Done Done Sent false
7090 2626 Acked Read 2 Done Done Sent false
7091 2627 Acked Read 5 Done Done Sent false
7092 2628 Acked Flush 0 Done Done Sent false
7093 2629 Acked Read 10 Done Done Sent false
7094 2630 Acked Write 5 Done Done Sent false
7095 2631 Acked Write 1 Done Done Sent false
7096 2632 Acked Read 3 Done Done Sent false
7097 2633 Acked Write 7 Done Done Sent false
7098 2634 Acked Write 7 Done Done Sent false
7099 2635 Acked Write 2 Done Done Sent false
7100 2636 Acked Write 6 Done Done Sent false
7101 2637 Acked Read 3 Done Done Sent false
7102 2638 Acked Read 6 Done Done Sent false
7103 2639 Acked Write 1 Done Done Sent false
7104 2640 Acked Flush 0 Done Done Sent false
7105 2641 Acked Flush 0 Done Done Sent false
7106 2642 Acked Read 5 Done Done Sent false
7107 2643 Acked Write 4 Done Done Sent false
7108 2644 Acked Read 10 Done Done Sent false
7109 2645 Acked Write 6 Done Done Sent false
7110 2646 Acked Read 1 Done Done Sent false
7111 2647 Acked Write 8 Done Done Sent false
7112 2648 Acked Read 8 Done Done Sent false
7113 2649 Acked Read 6 Done Done Sent false
7114 2650 Acked Write 4 Done Done Sent false
7115 2651 Acked Read 9 Done Done Sent false
7116 2652 Acked Read 7 Done Done Sent false
7117 2653 Acked Write 6 Done Done Sent false
7118 2654 Acked Read 6 Done Done Sent false
7119 2655 Acked Read 2 Done Done Sent false
7120 2656 Acked Flush 0 Done Done Sent false
7121 2657 Acked Write 5 Done Done Sent false
7122 2658 Acked Read 6 Done Done Sent false
7123 2659 Acked Read 5 Done Done Sent false
7124 2660 Acked Write 6 Done Done Sent false
7125 2661 Acked Read 6 Done Done Sent false
7126 2662 Acked Read 6 Done Done Sent false
7127 2663 Acked Read 10 Done Done Sent false
7128 2664 Acked Write 10 Done Done Sent false
7129 2665 Acked Read 9 Done Done Sent false
7130 2666 Acked Read 8 Done Done Sent false
7131 2667 Acked Write 3 Done Done Sent false
7132 2668 Acked Read 5 Done Done Sent false
7133 2669 Acked Read 3 Done Done Sent false
7134 2670 Acked Read 6 Done Done Sent false
7135 2671 Acked Write 1 Done Done Sent false
7136 2672 Acked Write 6 Done Done Sent false
7137 2673 Acked Write 7 Done Done Sent false
7138 2674 Acked Write 8 Done Done Sent false
7139 2675 Acked Read 2 Done Done Sent false
7140 2676 Acked Read 3 Done Done Sent false
7141 2677 Acked Read 4 Done Done Sent false
7142 2678 Acked Write 10 Done Done Sent false
7143 2679 Acked Read 10 Done Done Sent false
7144 2680 Acked Read 10 Done Done Sent false
7145 2681 Acked Read 2 Done Done Sent false
7146 2682 Acked Write 3 Done Done Sent false
7147 2683 Acked Read 4 Done Done Sent false
7148 2684 Acked Read 1 Done Done Sent false
7149 2685 Acked Read 7 Done Done Sent false
7150 2686 Acked Write 1 Done Done Sent false
7151 2687 Acked Write 1 Done Done Sent false
7152 2688 Acked Write 5 Done Done Sent false
7153 2689 Acked Read 9 Done Done Sent false
7154 2690 Acked Write 8 Done Done Sent false
7155 2691 Acked Read 1 Done Done Sent false
7156 2692 Acked Read 3 Done Done Sent false
7157 2693 Acked Write 4 Done Done Sent false
7158 2694 Acked Read 5 Done Done Sent false
7159 2695 Acked Read 8 Done Done Sent false
7160 2696 Acked Read 5 Done Done Sent false
7161 2697 Acked Write 5 Done Done Sent false
7162 2698 Acked Flush 0 Done Done Sent false
7163 2699 Acked Write 8 Done Done Sent false
7164 2700 Acked Write 7 Done Done Sent false
7165 2701 Acked Flush 0 Done Done Sent false
7166 2702 Acked Read 6 Done Done Sent false
7167 2703 Acked Read 2 Done Done Sent false
7168 2704 Acked Write 2 Done Done Sent false
7169 2705 Acked Read 2 Done Done Sent false
7170 2706 Acked Read 7 Done Done Sent false
7171 2707 Acked Read 3 Done Done Sent false
7172 2708 Acked Read 8 Done Done Sent false
7173 2709 Acked Read 10 Done Done Sent false
7174 2710 Acked Read 4 Done Done Sent false
7175 2711 Acked Write 10 Done Done Sent false
7176 2712 Acked Flush 0 Done Done Sent false
7177 2713 Acked Read 5 Done Done Sent false
7178 2714 Acked Write 2 Done Done Sent false
7179 2715 Acked Write 8 Done Done Sent false
7180 2716 Acked Flush 0 Done Done Sent false
7181 2717 Acked Flush 0 Done Done Sent false
7182 2718 Acked Write 7 Done Done Sent false
7183 2719 Acked Flush 0 Done Done Sent false
7184 2720 Acked Read 6 Done Done Sent false
7185 2721 Acked Read 7 Done Done Sent false
7186 2722 Acked Read 3 Done Done Sent false
7187 2723 Acked Read 1 Done Done Sent false
7188 2724 Acked Read 7 Done Done Sent false
7189 2725 Acked Write 2 Done Done Sent false
7190 2726 Acked Read 2 Done Done Sent false
7191 2727 Acked Read 10 Done Done Sent false
7192 2728 Acked Read 6 Done Done Sent false
7193 2729 Acked Write 5 Done Done Sent false
7194 2730 Acked Flush 0 Done Done Sent false
7195 2731 Acked Write 4 Done Done Sent false
7196 2732 Acked Write 8 Done Done Sent false
7197 2733 Acked Read 3 Done Done Sent false
7198 2734 Acked Read 9 Done Done Sent false
7199 2735 Acked Flush 0 Done Done Sent false
7200 2736 Acked Flush 0 Done Done Sent false
7201 2737 Acked Read 5 Done Done Sent false
7202 2738 Acked Write 4 Done Done Sent false
7203 2739 Acked Write 3 Done Done Sent false
7204 2740 Acked Write 10 Done Done Sent false
7205 2741 Acked Write 6 Done Done Sent false
7206 2742 Acked Write 10 Done Done Sent false
7207 2743 Acked Write 1 Done Done Sent false
7208 2744 Acked Read 6 Done Done Sent false
7209 2745 Acked Write 5 Done Done Sent false
7210 2746 Acked Read 10 Done Done Sent false
7211 2747 Acked Write 7 Done Done Sent false
7212 2748 Acked Read 7 Done Done Sent false
7213 2749 Acked Write 1 Done Done Sent false
7214 2750 Acked Write 1 Done Done Sent false
7215 2751 Acked Read 2 Done Done Sent false
7216 2752 Acked Read 5 Done Done Sent false
7217 2753 Acked Write 10 Done Done Sent false
7218 2754 Acked Write 8 Done Done Sent false
7219 2755 Acked Write 4 Done Done Sent false
7220 2756 Acked Write 5 Done Done Sent false
7221 2757 Acked Read 5 Done Done Sent false
7222 2758 Acked Write 9 Done Done Sent false
7223 2759 Acked Write 1 Done Done Sent false
7224 2760 Acked Read 10 Done Done Sent false
7225 2761 Acked Write 6 Done Done Sent false
7226 2762 Acked Write 2 Done Done Sent false
7227 2763 Acked Read 2 Done Done Sent false
7228 2764 Acked Read 1 Done Done Sent false
7229 2765 Acked Write 5 Done Done Sent false
7230 2766 Acked Read 2 Done Done Sent false
7231 2767 Acked Read 9 Done Done Sent false
7232 2768 Acked Write 10 Done Done Sent false
7233 2769 Acked Write 7 Done Done Sent false
7234 2770 Acked Read 7 Done Done Sent false
7235 2771 Acked Read 2 Done Done Sent false
7236 2772 Acked Write 9 Done Done Sent false
7237 2773 Acked Write 8 Done Done Sent false
7238 2774 Acked Write 7 Done Done Sent false
7239 2775 Acked Write 7 Done Done Sent false
7240 2776 Acked Read 5 Done Done Sent false
7241 2777 Acked Write 2 Done Done Sent false
7242 2778 Acked Flush 0 Done Done Sent false
7243 2779 Acked Read 10 Done Done Sent false
7244 2780 Acked Write 1 Done Done Sent false
7245 2781 Acked Flush 0 Done Done Sent false
7246 2782 Acked Read 1 Done Done Sent false
7247 2783 Acked Read 5 Done Done Sent false
7248 2784 Acked Read 6 Done Done Sent false
7249 2785 Acked Read 5 Done Done Sent false
7250 2786 Acked Write 1 Done Done Sent false
7251 2787 Acked Write 9 Done Done Sent false
7252 2788 Acked Write 1 Done Done Sent false
7253 2789 Acked Write 2 Done Done Sent false
7254 2790 Acked Read 8 Done Done Sent false
7255 2791 Acked Read 5 Done Done Sent false
7256 2792 Acked Write 4 Done Done Sent false
7257 2793 Acked Read 4 Done Done Sent false
7258 2794 Acked Write 8 Done Done Sent false
7259 2795 Acked Read 8 Done Done Sent false
7260 2796 Acked Read 7 Done Done Sent false
7261 2797 Acked Read 10 Done Done Sent false
7262 2798 Acked Read 5 Done Done Sent false
7263 2799 Acked Read 4 Done Done Sent false
7264 2800 Acked Write 5 Done Done Sent false
7265 2801 Acked Write 9 Done Done Sent false
7266 2802 Acked Read 1 Done Done Sent false
7267 2803 Acked Read 2 Done Done Sent false
7268 2804 Acked Write 8 Done Done Sent false
7269 2805 Acked Flush 0 Done Done Sent false
7270 2806 Acked Write 3 Done Done Sent false
7271 2807 Acked Read 5 Done Done Sent false
7272 2808 Acked Read 10 Done Done Sent false
7273 2809 Acked Read 1 Done Done Sent false
7274 2810 Acked Write 1 Done Done Sent false
7275 2811 Acked Flush 0 Done Done Sent false
7276 STATES DS:0 DS:1 DS:2 TOTAL
7277 Sent 0 0 302 302
7278 Done 302 302 0 604
7279 Skipped 0 0 0 0
7280 Error 0 0 0 0
7281 Last Flush: 2811 2811 2509
7282 Downstairs last five completed: 2509 2508 2507 2506 2505
7283 Upstairs last five completed: 2811 2810 2809 2808 2807
7284 CLIENT: Up:0 ds:302 act:2
72852025-01-13T15:53:20.294ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
72862025-01-13T15:53:20.294ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64999) = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
72872025-01-13T15:53:20.295ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
72882025-01-13T15:53:20.295ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
72892025-01-13T15:53:20.295ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
72902025-01-13T15:53:20.295ZINFOcrucible: send last flush ID to this DS: 2509 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
72912025-01-13T15:53:20.296ZINFOcrucible: Replied this last flush ID: 2509 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
72922025-01-13T15:53:20.296ZINFOcrucible: [2] client re-new 302 jobs since flush 2509 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
72932025-01-13T15:53:20.296ZINFOcrucible: [2] Replayed 302 jobs since flush: 2509 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
7294 ----------------------------------------------------------------
7295 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
7296 Downstairs last five completed: 2811 2810 2809 2808 2807
7297 Upstairs last five completed: 2811 2810 2809 2808 2807
7298 CLIENT: Up:0 ds:0 act:3
7299 Replay: All jobs finished
7300 Main task has work to do, go find it
7301 got dsc Stop(1)
7302 stop 1
7303 Jan 13 15:53:20.546 INFO request completed, latency_us: 337[8820] Got stop action so:false kr:false
7304 , response_code: 204, uri: /stop/cid/1, method: POST, req_id: 9aa32a35-74cc-4970-ba8c-9a69dc29eb7c, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
73052025-01-13T15:53:20.547ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
73062025-01-13T15:53:20.547ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
73072025-01-13T15:53:20.547ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
73082025-01-13T15:53:20.547ZINFOcrucible: client task is exiting = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
7309 Jan 13 15:53:20.548 INFO request completed, latency_us: 295, response_code: 200, uri: /state/cid/1, method: GET, req_id: 999e06f8-3b69-4972-a820-36130b46abaf, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
7310 [8820] Exited with: ExitStatus(unix_wait_status(9))
7311 [8820][1] reports Exit
7312 Jan 13 15:53:24.550 INFO request completed, latency_us: 381, response_code: 200, uri: /state/cid/1, method: GET, req_id: b52e0289-2e02-464d-8a41-5e8735e7f588, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
7313 001/300 Read block 4809 len 32768
7314 002/300 Read block 710 len 8192
7315 003/300 Read block 2788 len 8192
7316 004/300 Read block 2511 len 36864
7317 005/300 Read block 4583 len 20480
7318 006/300 Write block 3482 len 32768 data: 3 3 3 3 4 5 4 4
7319 007/300 Read block 2141 len 40960
7320 008/300 Write block 1383 len 32768 data: 5 4 4 4 3 3 2 2
7321 009/300 Write block 897 len 16384 data: 2 2 2 2
7322 010/300 Write block 1686 len 12288 data: 2 2 3
7323 011/300 Read block 1846 len 16384
7324 012/300 Write block 3804 len 4096 data: 3
7325 013/300 Read block 608 len 28672
7326 014/300 Read block 4069 len 16384
7327 015/300 Write block 3138 len 20480 data: 2 2 2 2 3
7328 016/300 Read block 2656 len 16384
7329 017/300 Read block 4006 len 32768
7330 018/300 Read block 3389 len 8192
7331 019/300 Write block 3858 len 24576 data: 2 2 2 2 2 2
7332 020/300 Write block 2226 len 36864 data: 4 4 4 4 3 3 2 2 2
7333 021/300 Write block 4215 len 40960 data: 5 5 5 3 3 3 3 2 4 4
7334 022/300 Write block 3035 len 12288 data: 3 3 3
7335 023/300 Write block 1518 len 20480 data: 2 2 2 2 3
7336 024/300 Write block 4660 len 24576 data: 3 3 2 3 3 3
7337 025/300 Read block 4187 len 12288
7338 026/300 Read block 1632 len 36864
7339 027/300 Write block 3262 len 36864 data: 2 2 3 3 2 2 2 2 2
7340 028/300 Read block 557 len 24576
7341 029/300 Write block 4975 len 4096 data: 3
7342 030/300 Flush
7343 031/300 Read block 2835 len 24576
7344 032/300 Read block 935 len 24576
7345 033/300 Read block 4920 len 4096
7346 034/300 Read block 2131 len 36864
7347 035/300 Read block 3195 len 32768
7348 036/300 Read block 1420 len 12288
7349 037/300 Read block 3674 len 12288
7350 038/300 Write block 3779 len 4096 data: 3
7351 039/300 Write block 1577 len 40960 data: 3 3 4 3 3 4 3 3 3 3
7352 040/300 Read block 3940 len 16384
7353 041/300 Read block 1250 len 32768
7354 042/300 Read block 1354 len 36864
7355 043/300 Write block 23 len 20480 data: 2 3 3 3 3
7356 044/300 Write block 1797 len 12288 data: 3 3 2
7357 045/300 Write block 2045 len 28672 data: 3 4 4 4 4 4 4
7358 046/300 Read block 2588 len 8192
7359 047/300 Write block 1442 len 8192 data: 4 4
7360 048/300 Read block 4115 len 36864
7361 049/300 Read block 444 len 32768
7362 050/300 Read block 1180 len 4096
7363 051/300 Write block 743 len 4096 data: 3
7364 052/300 Write block 4733 len 36864 data: 4 5 5 4 4 3 3 3 2
7365 053/300 Write block 2391 len 24576 data: 2 2 2 2 2 2
7366 054/300 Read block 4712 len 12288
7367 055/300 Read block 3982 len 28672
7368 056/300 Read block 4165 len 16384
7369 057/300 Write block 317 len 8192 data: 2 2
7370 058/300 Write block 793 len 8192 data: 2 2
7371 059/300 Read block 111 len 32768
7372 060/300 Read block 2927 len 20480
7373 061/300 Flush
7374 062/300 Read block 4190 len 16384
7375 063/300 Read block 1049 len 28672
7376 064/300 Read block 1726 len 8192
7377 065/300 Write block 4050 len 36864 data: 2 3 3 3 3 3 3 3 4
7378 066/300 Flush
7379 067/300 Read block 4426 len 8192
7380 068/300 Write block 754 len 12288 data: 4 4 4
7381 069/300 Read block 3598 len 20480
7382 070/300 Read block 2452 len 16384
7383 071/300 Flush
7384 072/300 Read block 4906 len 28672
7385 073/300 Write block 3983 len 40960 data: 3 3 3 3 4 4 3 3 3 3
7386 074/300 Write block 4172 len 8192 data: 3 3
7387 075/300 Read block 3673 len 40960
7388 076/300 Read block 1627 len 20480
7389 077/300 Write block 4116 len 8192 data: 2 2
7390 078/300 Read block 1580 len 8192
7391 079/300 Write block 4303 len 12288 data: 5 4 4
7392 080/300 Read block 4312 len 40960
7393 081/300 Read block 2955 len 20480
7394 082/300 Flush
7395 083/300 Write block 3082 len 40960 data: 2 2 4 3 3 4 4 4 2 2
7396 084/300 Write block 4929 len 24576 data: 4 5 6 6 5 5
7397 085/300 Write block 4218 len 16384 data: 4 4 4 4
7398 086/300 Read block 3370 len 12288
7399 087/300 Flush
7400 088/300 Read block 2438 len 4096
7401 089/300 Read block 4626 len 12288
7402 090/300 Read block 4949 len 36864
7403 091/300 Read block 2107 len 36864
7404 092/300 Write block 1571 len 28672 data: 3 2 2 3 3 3 4
7405 093/300 Write block 123 len 20480 data: 3 3 3 3 4
7406 094/300 Write block 4950 len 36864 data: 2 2 2 2 2 2 2 2 3
7407 095/300 Read block 4306 len 20480
7408 096/300 Read block 906 len 28672
7409 097/300 Read block 3013 len 20480
7410 098/300 Write block 3727 len 12288 data: 4 4 4
7411 099/300 Read block 789 len 40960
7412 100/300 Read block 1718 len 12288
7413 101/300 Read block 1911 len 12288
7414 102/300 Write block 4396 len 20480 data: 3 3 3 3 2
7415 103/300 Write block 3186 len 32768 data: 3 3 4 5 5 5 4 3
7416 104/300 Write block 2580 len 12288 data: 2 2 2
7417 105/300 Write block 3817 len 12288 data: 3 3 3
7418 106/300 Flush
7419 107/300 Read block 2739 len 28672
7420 108/300 Read block 1794 len 32768
7421 109/300 Read block 3051 len 16384
7422 110/300 Read block 2927 len 32768
7423 111/300 Read block 4768 len 28672
7424 112/300 Read block 1020 len 8192
7425 113/300 Read block 652 len 28672
7426 114/300 Write block 1003 len 20480 data: 2 3 3 3 3
7427 115/300 Read block 524 len 4096
7428 116/300 Flush
7429 117/300 Write block 1203 len 4096 data: 2
7430 118/300 Write block 967 len 8192 data: 3 3
7431 119/300 Read block 3828 len 20480
7432 120/300 Read block 4747 len 24576
7433 121/300 Read block 2619 len 8192
7434 122/300 Read block 1705 len 28672
7435 123/300 Read block 2010 len 12288
7436 124/300 Write block 1516 len 12288 data: 3 3 3
7437 125/300 Flush
7438 126/300 Read block 1601 len 24576
7439 127/300 Write block 1069 len 12288 data: 2 2 2
7440 128/300 Write block 4331 len 4096 data: 2
7441 129/300 Read block 3271 len 36864
7442 130/300 Write block 755 len 36864 data: 5 5 4 4 3 3 3 2 2
7443 131/300 Write block 3442 len 36864 data: 3 4 3 3 3 3 3 3 3
7444 132/300 Read block 4174 len 24576
7445 133/300 Read block 2501 len 4096
7446 134/300 Write block 4221 len 40960 data: 5 3 5 5 5 4 4 4 4 4
7447 135/300 Write block 2926 len 28672 data: 3 3 3 3 3 3 3
7448 136/300 Read block 3994 len 28672
7449 137/300 Read block 3895 len 20480
7450 138/300 Read block 3310 len 32768
7451 139/300 Read block 3994 len 16384
7452 140/300 Flush
7453 141/300 Write block 3517 len 4096 data: 4
7454 142/300 Read block 4405 len 32768
7455 143/300 Read block 1699 len 16384
7456 144/300 Write block 1213 len 8192 data: 2 2
7457 145/300 Write block 3019 len 24576 data: 2 3 3 3 3 5
7458 146/300 Read block 1237 len 24576
7459 147/300 Write block 2635 len 20480 data: 2 3 3 3 3
7460 148/300 Write block 4373 len 28672 data: 2 2 2 2 2 3 3
7461 149/300 Read block 1793 len 8192
7462 150/300 Read block 1432 len 8192
7463 151/300 Write block 4968 len 12288 data: 4 3 3
7464 152/300 Write block 341 len 4096 data: 3
7465 153/300 Flush
7466 154/300 Read block 4681 len 4096
7467 155/300 Read block 1564 len 24576
7468 156/300 Flush
7469 157/300 Read block 1192 len 40960
7470 158/300 Write block 3324 len 16384 data: 3 3 4 4
7471 159/300 Read block 3774 len 28672
7472 160/300 Read block 1502 len 40960
7473 161/300 Write block 1750 len 20480 data: 4 4 4 4 3
7474 162/300 Read block 3367 len 28672
7475 163/300 Write block 4266 len 40960 data: 2 2 3 3 3 3 3 3 3 3
7476 164/300 Write block 4956 len 28672 data: 3 3 4 4 4 2 3
7477 165/300 Read block 515 len 16384
7478 166/300 Read block 381 len 16384
7479 167/300 Read block 9 len 12288
7480 168/300 Read block 4969 len 40960
7481 169/300 Write block 269 len 24576 data: 2 3 4 4 4 3
7482 170/300 Write block 3 len 28672 data: 3 4 4 3 3 2 2
7483 171/300 Flush
7484 172/300 Read block 802 len 8192
7485 173/300 Read block 4747 len 40960
7486 174/300 Flush
7487 175/300 Write block 2610 len 28672 data: 3 3 2 2 2 2 2
7488 176/300 Read block 2909 len 12288
7489 177/300 Read block 4898 len 12288
7490 178/300 Read block 355 len 16384
7491 179/300 Read block 2372 len 20480
7492 180/300 Write block 3798 len 28672 data: 2 2 3 3 3 3 4
7493 181/300 Read block 1490 len 28672
7494 182/300 Write block 4782 len 20480 data: 2 2 2 2 2
7495 183/300 Read block 4417 len 20480
7496 184/300 Flush
7497 185/300 Read block 4315 len 32768
7498 186/300 Write block 2075 len 40960 data: 2 2 4 4 4 3 3 3 3 2
7499 187/300 Read block 4652 len 8192
7500 188/300 Write block 2309 len 12288 data: 3 3 3
7501 189/300 Read block 4171 len 32768
7502 190/300 Write block 114 len 36864 data: 3 4 3 3 3 4 4 3 3
7503 191/300 Read block 36 len 40960
7504 192/300 Read block 181 len 28672
7505 193/300 Read block 4772 len 28672
7506 194/300 Read block 4347 len 36864
7507 195/300 Read block 3960 len 8192
7508 196/300 Write block 879 len 36864 data: 3 4 4 4 3 3 2 2 2
7509 197/300 Write block 2886 len 40960 data: 3 2 2 2 2 2 2 2 2 2
7510 198/300 Flush
7511 199/300 Write block 86 len 8192 data: 2 2
7512 200/300 Read block 3767 len 4096
7513 201/300 Read block 952 len 16384
7514 202/300 Write block 2964 len 40960 data: 2 2 3 3 3 3 3 3 3 3
7515 203/300 Write block 4731 len 8192 data: 4 4
7516 204/300 Read block 920 len 28672
7517 205/300 Write block 4250 len 16384 data: 3 3 3 3
7518 206/300 Write block 284 len 20480 data: 2 3 3 3 3
7519 207/300 Flush
7520 208/300 Write block 3445 len 36864 data: 4 4 4 4 4 4 3 2 2
7521 209/300 Write block 3838 len 36864 data: 2 2 2 2 2 2 2 2 2
7522 210/300 Write block 4684 len 4096 data: 4
7523 211/300 Write block 3915 len 8192 data: 2 3
7524 212/300 Write block 2679 len 4096 data: 2
7525 213/300 Read block 1496 len 32768
7526 214/300 Read block 2052 len 40960
7527 215/300 Write block 2985 len 8192 data: 2 2
7528 216/300 Write block 4596 len 28672 data: 4 5 4 4 5 4 4
7529 217/300 Write block 4669 len 24576 data: 2 2 2 2 2 2
7530 218/300 Write block 698 len 40960 data: 2 2 2 2 2 2 2 2 2 2
7531 219/300 Read block 3135 len 4096
7532 220/300 Flush
7533 221/300 Read block 1334 len 4096
7534 222/300 Write block 2484 len 4096 data: 5
7535 223/300 Read block 394 len 20480
7536 224/300 Write block 4541 len 4096 data: 2
7537 225/300 Write block 574 len 28672 data: 4 3 4 4 4 4 4
7538 226/300 Read block 2604 len 4096
7539 227/300 Read block 1493 len 16384
7540 228/300 Write block 3348 len 40960 data: 3 3 2 3 2 2 2 3 3 3
7541 229/300 Read block 4783 len 20480
7542 230/300 Write block 4079 len 16384 data: 3 3 3 3
7543 231/300 Flush
7544 232/300 Read block 2789 len 8192
7545 233/300 Write block 4861 len 4096 data: 3
7546 234/300 Write block 2224 len 32768 data: 4 4 5 5 5 5 4 4
7547 235/300 Read block 3338 len 28672
7548 236/300 Write block 1621 len 32768 data: 4 3 3 4 3 2 2 2
7549 237/300 Read block 4148 len 4096
7550 238/300 Read block 2044 len 36864
7551 239/300 Write block 476 len 12288 data: 3 4 3
7552 240/300 Write block 4609 len 4096 data: 2
7553 241/300 Read block 1813 len 8192
7554 242/300 Write block 1030 len 4096 data: 2
7555 243/300 Write block 4518 len 36864 data: 5 6 5 4 4 4 4 3 3
7556 244/300 Read block 4749 len 16384
7557 245/300 Write block 3356 len 32768 data: 4 4 3 3 3 3 2 2
7558 246/300 Read block 1916 len 40960
7559 247/300 Write block 678 len 16384 data: 2 2 2 2
7560 248/300 Write block 1855 len 32768 data: 2 3 3 3 2 2 2 3
7561 249/300 Write block 640 len 32768 data: 2 2 3 3 3 4 4 3
7562 250/300 Flush
7563 251/300 Read block 4778 len 32768
7564 252/300 Write block 4189 len 12288 data: 2 2 2
7565 253/300 Read block 776 len 32768
7566 254/300 Flush
7567 255/300 Flush
7568 256/300 Read block 1200 len 40960
7569 257/300 Write block 3726 len 16384 data: 3 5 5 5
7570 258/300 Read block 245 len 28672
7571 259/300 Read block 767 len 40960
7572 260/300 Write block 402 len 12288 data: 4 4 3
7573 261/300 Write block 1352 len 20480 data: 2 3 4 4 4
7574 262/300 Write block 2240 len 32768 data: 2 2 2 2 2 2 3 3
7575 263/300 Flush
7576 264/300 Read block 2732 len 20480
7577 265/300 Read block 2749 len 16384
7578 266/300 Flush
7579 267/300 Read block 3319 len 24576
7580 268/300 Write block 4297 len 40960 data: 4 5 5 5 5 5 6 5 5 3
7581 269/300 Read block 2303 len 36864
7582 270/300 Write block 4729 len 4096 data: 3
7583 271/300 Write block 4782 len 40960 data: 3 3 3 3 3 2 2 2 2 2
7584 272/300 Read block 1849 len 12288
7585 273/300 Read block 1197 len 12288
7586 274/300 Read block 2642 len 32768
7587 275/300 Flush
7588 276/300 Read block 4592 len 12288
7589 277/300 Flush
7590 278/300 Read block 3302 len 12288
7591 279/300 Flush
7592 280/300 Write block 148 len 12288 data: 3 3 2
7593 281/300 Read block 1752 len 24576
7594 282/300 Write block 3226 len 12288 data: 3 4 3
7595 283/300 Read block 929 len 16384
7596 284/300 Write block 1042 len 16384 data: 3 3 4 4
7597 285/300 Write block 4366 len 12288 data: 3 3 3
7598 286/300 Read block 3461 len 36864
7599 287/300 Read block 2496 len 24576
7600 288/300 Write block 3726 len 20480 data: 4 6 6 6 3
7601 289/300 Write block 2380 len 16384 data: 2 2 2 3
7602 290/300 Write block 483 len 16384 data: 2 2 3 3
7603 291/300 Read block 11 len 4096
7604 292/300 Read block 3114 len 12288
7605 293/300 Read block 3423 len 36864
7606 294/300 Write block 831 len 8192 data: 2 2
7607 295/300 Flush
7608 296/300 Read block 1878 len 36864
7609 297/300 Read block 3250 len 8192
7610 298/300 Write block 2745 len 28672 data: 2 2 4 4 4 4 4
7611 299/300 Write block 1263 len 12288 data: 3 3 3
7612 300/300 Flush
7613 301/300 Read block 2122 len 4096
7614 Main task has work to do, go find it
7615 got dsc Start(1)
7616 start 1
7617 Jan 13 15:53:24.786 INFO request completed, latency_us: 207, response_code: 204, uri: /start/cid/1, method: POST, req_id: b416777d-5e8d-44ae-91c2-d34bc66d6f87, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
7618 Replay: started 1, returned:Ok(())
7619 ----------------------------------------------------------------
7620 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
7621 States: Active Offline Active
7622 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7623 2812 Acked Read 8 Done Sent Done false
7624 2813 Acked Read 2 Done Sent Done false
7625 2814 Acked Read 2 Done Sent Done false
7626 2815 Acked Read 9 Done Sent Done false
7627 2816 Acked Read 5 Done Sent Done false
7628 2817 Acked Write 8 Done Sent Done false
7629 2818 Acked Read 10 Done Sent Done false
7630 2819 Acked Write 8 Done Sent Done false
7631 2820 Acked Write 4 Done Sent Done false
7632 2821 Acked Write 3 Done Sent Done false
7633 2822 Acked Read 4 Done Sent Done false
7634 2823 Acked Write 1 Done Sent Done false
7635 2824 Acked Read 7 Done Sent Done false
7636 2825 Acked Read 4 Done Sent Done false
7637 2826 Acked Write 5 Done Sent Done false
7638 2827 Acked Read 4 Done Sent Done false
7639 2828 Acked Read 8 Done Sent Done false
7640 2829 Acked Read 2 Done Sent Done false
7641 2830 Acked Write 6 Done Sent Done false
7642 2831 Acked Write 9 Done Sent Done false
7643 2832 Acked Write 10 Done Sent Done false
7644 2833 Acked Write 3 Done Sent Done false
7645 2834 Acked Write 5 Done Sent Done false
7646 2835 Acked Write 6 Done Sent Done false
7647 2836 Acked Read 3 Done Sent Done false
7648 2837 Acked Read 9 Done Sent Done false
7649 2838 Acked Write 9 Done Sent Done false
7650 2839 Acked Read 6 Done Sent Done false
7651 2840 Acked Write 1 Done Sent Done false
7652 2841 Acked Flush 0 Done Sent Done false
7653 2842 Acked Read 6 Done Sent Done false
7654 2843 Acked Read 6 Done Sent Done false
7655 2844 Acked Read 1 Done Sent Done false
7656 2845 Acked Read 9 Done Sent Done false
7657 2846 Acked Read 8 Done Sent Done false
7658 2847 Acked Read 3 Done Sent Done false
7659 2848 Acked Read 3 Done Sent Done false
7660 2849 Acked Write 1 Done Sent Done false
7661 2850 Acked Write 10 Done Sent Done false
7662 2851 Acked Read 4 Done Sent Done false
7663 2852 Acked Read 8 Done Sent Done false
7664 2853 Acked Read 9 Done Sent Done false
7665 2854 Acked Write 5 Done Sent Done false
7666 2855 Acked Write 3 Done Sent Done false
7667 2856 Acked Write 7 Done Sent Done false
7668 2857 Acked Read 2 Done Sent Done false
7669 2858 Acked Write 2 Done Sent Done false
7670 2859 Acked Read 9 Done Sent Done false
7671 2860 Acked Read 8 Done Sent Done false
7672 2861 Acked Read 1 Done Sent Done false
7673 2862 Acked Write 1 Done Sent Done false
7674 2863 Acked Write 9 Done Sent Done false
7675 2864 Acked Write 6 Done Sent Done false
7676 2865 Acked Read 3 Done Sent Done false
7677 2866 Acked Read 7 Done Sent Done false
7678 2867 Acked Read 4 Done Sent Done false
7679 2868 Acked Write 2 Done Sent Done false
7680 2869 Acked Write 2 Done Sent Done false
7681 2870 Acked Read 8 Done Sent Done false
7682 2871 Acked Read 5 Done Sent Done false
7683 2872 Acked Flush 0 Done Sent Done false
7684 2873 Acked Read 4 Done Sent Done false
7685 2874 Acked Read 7 Done Sent Done false
7686 2875 Acked Read 2 Done Sent Done false
7687 2876 Acked Write 9 Done Sent Done false
7688 2877 Acked Flush 0 Done Sent Done false
7689 2878 Acked Read 2 Done Sent Done false
7690 2879 Acked Write 3 Done Sent Done false
7691 2880 Acked Read 5 Done Sent Done false
7692 2881 Acked Read 4 Done Sent Done false
7693 2882 Acked Flush 0 Done Sent Done false
7694 2883 Acked Read 7 Done Sent Done false
7695 2884 Acked Write 10 Done Sent Done false
7696 2885 Acked Write 2 Done Sent Done false
7697 2886 Acked Read 10 Done Sent Done false
7698 2887 Acked Read 5 Done Sent Done false
7699 2888 Acked Write 2 Done Sent Done false
7700 2889 Acked Read 2 Done Sent Done false
7701 2890 Acked Write 3 Done Sent Done false
7702 2891 Acked Read 10 Done Sent Done false
7703 2892 Acked Read 5 Done Sent Done false
7704 2893 Acked Flush 0 Done Sent Done false
7705 2894 Acked Write 10 Done Sent Done false
7706 2895 Acked Write 6 Done Sent Done false
7707 2896 Acked Write 4 Done Sent Done false
7708 2897 Acked Read 3 Done Sent Done false
7709 2898 Acked Flush 0 Done Sent Done false
7710 2899 Acked Read 1 Done Sent Done false
7711 2900 Acked Read 3 Done Sent Done false
7712 2901 Acked Read 9 Done Sent Done false
7713 2902 Acked Read 9 Done Sent Done false
7714 2903 Acked Write 7 Done Sent Done false
7715 2904 Acked Write 5 Done Sent Done false
7716 2905 Acked Write 9 Done Sent Done false
7717 2906 Acked Read 5 Done Sent Done false
7718 2907 Acked Read 7 Done Sent Done false
7719 2908 Acked Read 5 Done Sent Done false
7720 2909 Acked Write 3 Done Sent Done false
7721 2910 Acked Read 10 Done Sent Done false
7722 2911 Acked Read 3 Done Sent Done false
7723 2912 Acked Read 3 Done Sent Done false
7724 2913 Acked Write 5 Done Sent Done false
7725 2914 Acked Write 8 Done Sent Done false
7726 2915 Acked Write 3 Done Sent Done false
7727 2916 Acked Write 3 Done Sent Done false
7728 2917 Acked Flush 0 Done Sent Done false
7729 2918 Acked Read 7 Done Sent Done false
7730 2919 Acked Read 8 Done Sent Done false
7731 2920 Acked Read 4 Done Sent Done false
7732 2921 Acked Read 8 Done Sent Done false
7733 2922 Acked Read 7 Done Sent Done false
7734 2923 Acked Read 2 Done Sent Done false
7735 2924 Acked Read 7 Done Sent Done false
7736 2925 Acked Write 5 Done Sent Done false
7737 2926 Acked Read 1 Done Sent Done false
7738 2927 Acked Flush 0 Done Sent Done false
7739 2928 Acked Write 1 Done Sent Done false
7740 2929 Acked Write 2 Done Sent Done false
7741 2930 Acked Read 5 Done Sent Done false
7742 2931 Acked Read 6 Done Sent Done false
7743 2932 Acked Read 2 Done Sent Done false
7744 2933 Acked Read 7 Done Sent Done false
7745 2934 Acked Read 3 Done Sent Done false
7746 2935 Acked Write 3 Done Sent Done false
7747 2936 Acked Flush 0 Done Sent Done false
7748 2937 Acked Read 6 Done Sent Done false
7749 2938 Acked Write 3 Done Sent Done false
7750 2939 Acked Write 1 Done Sent Done false
7751 2940 Acked Read 9 Done Sent Done false
7752 2941 Acked Write 9 Done Sent Done false
7753 2942 Acked Write 9 Done Sent Done false
7754 2943 Acked Read 6 Done Sent Done false
7755 2944 Acked Read 1 Done Sent Done false
7756 2945 Acked Write 10 Done Sent Done false
7757 2946 Acked Write 7 Done Sent Done false
7758 2947 Acked Read 7 Done Sent Done false
7759 2948 Acked Read 5 Done Sent Done false
7760 2949 Acked Read 8 Done Sent Done false
7761 2950 Acked Read 4 Done Sent Done false
7762 2951 Acked Flush 0 Done Sent Done false
7763 2952 Acked Write 1 Done Sent Done false
7764 2953 Acked Read 8 Done Sent Done false
7765 2954 Acked Read 4 Done Sent Done false
7766 2955 Acked Write 2 Done Sent Done false
7767 2956 Acked Write 6 Done Sent Done false
7768 2957 Acked Read 6 Done Sent Done false
7769 2958 Acked Write 5 Done Sent Done false
7770 2959 Acked Write 7 Done Sent Done false
7771 2960 Acked Read 2 Done Sent Done false
7772 2961 Acked Read 2 Done Sent Done false
7773 2962 Acked Write 3 Done Sent Done false
7774 2963 Acked Write 1 Done Sent Done false
7775 2964 Acked Flush 0 Done Sent Done false
7776 2965 Acked Read 1 Done Sent Done false
7777 2966 Acked Read 6 Done Sent Done false
7778 2967 Acked Flush 0 Done Sent Done false
7779 2968 Acked Read 10 Done Sent Done false
7780 2969 Acked Write 4 Done Sent Done false
7781 2970 Acked Read 7 Done Sent Done false
7782 2971 Acked Read 10 Done Sent Done false
7783 2972 Acked Write 5 Done Sent Done false
7784 2973 Acked Read 7 Done Sent Done false
7785 2974 Acked Write 10 Done Sent Done false
7786 2975 Acked Write 7 Done Sent Done false
7787 2976 Acked Read 4 Done Sent Done false
7788 2977 Acked Read 4 Done Sent Done false
7789 2978 Acked Read 3 Done Sent Done false
7790 2979 Acked Read 10 Done Sent Done false
7791 2980 Acked Write 6 Done Sent Done false
7792 2981 Acked Write 7 Done Sent Done false
7793 2982 Acked Flush 0 Done Sent Done false
7794 2983 Acked Read 2 Done Sent Done false
7795 2984 Acked Read 10 Done Sent Done false
7796 2985 Acked Flush 0 Done Sent Done false
7797 2986 Acked Write 7 Done Sent Done false
7798 2987 Acked Read 3 Done Sent Done false
7799 2988 Acked Read 3 Done Sent Done false
7800 2989 Acked Read 4 Done Sent Done false
7801 2990 Acked Read 5 Done Sent Done false
7802 2991 Acked Write 7 Done Sent Done false
7803 2992 Acked Read 7 Done Sent Done false
7804 2993 Acked Write 5 Done Sent Done false
7805 2994 Acked Read 5 Done Sent Done false
7806 2995 Acked Flush 0 Done Sent Done false
7807 2996 Acked Read 8 Done Sent Done false
7808 2997 Acked Write 10 Done Sent Done false
7809 2998 Acked Read 2 Done Sent Done false
7810 2999 Acked Write 3 Done Sent Done false
7811 3000 Acked Read 8 Done Sent Done false
7812 3001 Acked Write 9 Done Sent Done false
7813 3002 Acked Read 10 Done Sent Done false
7814 3003 Acked Read 7 Done Sent Done false
7815 3004 Acked Read 7 Done Sent Done false
7816 3005 Acked Read 9 Done Sent Done false
7817 3006 Acked Read 2 Done Sent Done false
7818 3007 Acked Write 9 Done Sent Done false
7819 3008 Acked Write 10 Done Sent Done false
7820 3009 Acked Flush 0 Done Sent Done false
7821 3010 Acked Write 2 Done Sent Done false
7822 3011 Acked Read 1 Done Sent Done false
7823 3012 Acked Read 4 Done Sent Done false
7824 3013 Acked Write 10 Done Sent Done false
7825 3014 Acked Write 2 Done Sent Done false
7826 3015 Acked Read 7 Done Sent Done false
7827 3016 Acked Write 4 Done Sent Done false
7828 3017 Acked Write 5 Done Sent Done false
7829 3018 Acked Flush 0 Done Sent Done false
7830 3019 Acked Write 9 Done Sent Done false
7831 3020 Acked Write 9 Done Sent Done false
7832 3021 Acked Write 1 Done Sent Done false
7833 3022 Acked Write 2 Done Sent Done false
7834 3023 Acked Write 1 Done Sent Done false
7835 3024 Acked Read 8 Done Sent Done false
7836 3025 Acked Read 10 Done Sent Done false
7837 3026 Acked Write 2 Done Sent Done false
7838 3027 Acked Write 7 Done Sent Done false
7839 3028 Acked Write 6 Done Sent Done false
7840 3029 Acked Write 10 Done Sent Done false
7841 3030 Acked Read 1 Done Sent Done false
7842 3031 Acked Flush 0 Done Sent Done false
7843 3032 Acked Read 1 Done Sent Done false
7844 3033 Acked Write 1 Done Sent Done false
7845 3034 Acked Read 5 Done Sent Done false
7846 3035 Acked Write 1 Done Sent Done false
7847 3036 Acked Write 7 Done Sent Done false
7848 3037 Acked Read 1 Done Sent Done false
7849 3038 Acked Read 4 Done Sent Done false
7850 3039 Acked Write 10 Done Sent Done false
7851 3040 Acked Read 5 Done Sent Done false
7852 3041 Acked Write 4 Done Sent Done false
7853 3042 Acked Flush 0 Done Sent Done false
7854 3043 Acked Read 2 Done Sent Done false
7855 3044 Acked Write 1 Done Sent Done false
7856 3045 Acked Write 8 Done Sent Done false
7857 3046 Acked Read 7 Done Sent Done false
7858 3047 Acked Write 8 Done Sent Done false
7859 3048 Acked Read 1 Done Sent Done false
7860 3049 Acked Read 9 Done Sent Done false
7861 3050 Acked Write 3 Done Sent Done false
7862 3051 Acked Write 1 Done Sent Done false
7863 3052 Acked Read 2 Done Sent Done false
7864 3053 Acked Write 1 Done Sent Done false
7865 3054 Acked Write 9 Done Sent Done false
7866 3055 Acked Read 4 Done Sent Done false
7867 3056 Acked Write 8 Done Sent Done false
7868 3057 Acked Read 10 Done Sent Done false
7869 3058 Acked Write 4 Done Sent Done false
7870 3059 Acked Write 8 Done Sent Done false
7871 3060 Acked Write 8 Done Sent Done false
7872 3061 Acked Flush 0 Done Sent Done false
7873 3062 Acked Read 8 Done Sent Done false
7874 3063 Acked Write 3 Done Sent Done false
7875 3064 Acked Read 8 Done Sent Done false
7876 3065 Acked Flush 0 Done Sent Done false
7877 3066 Acked Flush 0 Done Sent Done false
7878 3067 Acked Read 10 Done Sent Done false
7879 3068 Acked Write 4 Done Sent Done false
7880 3069 Acked Read 7 Done Sent Done false
7881 3070 Acked Read 10 Done Sent Done false
7882 3071 Acked Write 3 Done Sent Done false
7883 3072 Acked Write 5 Done Sent Done false
7884 3073 Acked Write 8 Done Sent Done false
7885 3074 Acked Flush 0 Done Sent Done false
7886 3075 Acked Read 5 Done Sent Done false
7887 3076 Acked Read 4 Done Sent Done false
7888 3077 Acked Flush 0 Done Sent Done false
7889 3078 Acked Read 6 Done Sent Done false
7890 3079 Acked Write 10 Done Sent Done false
7891 3080 Acked Read 9 Done Sent Done false
7892 3081 Acked Write 1 Done Sent Done false
7893 3082 Acked Write 10 Done Sent Done false
7894 3083 Acked Read 3 Done Sent Done false
7895 3084 Acked Read 3 Done Sent Done false
7896 3085 Acked Read 8 Done Sent Done false
7897 3086 Acked Flush 0 Done Sent Done false
7898 3087 Acked Read 3 Done Sent Done false
7899 3088 Acked Flush 0 Done Sent Done false
7900 3089 Acked Read 3 Done Sent Done false
7901 3090 Acked Flush 0 Done Sent Done false
7902 3091 Acked Write 3 Done Sent Done false
7903 3092 Acked Read 6 Done Sent Done false
7904 3093 Acked Write 3 Done Sent Done false
7905 3094 Acked Read 4 Done Sent Done false
7906 3095 Acked Write 4 Done Sent Done false
7907 3096 Acked Write 3 Done Sent Done false
7908 3097 Acked Read 9 Done Sent Done false
7909 3098 Acked Read 6 Done Sent Done false
7910 3099 Acked Write 5 Done Sent Done false
7911 3100 Acked Write 4 Done Sent Done false
7912 3101 Acked Write 4 Done Sent Done false
7913 3102 Acked Read 1 Done Sent Done false
7914 3103 Acked Read 3 Done Sent Done false
7915 3104 Acked Read 9 Done Sent Done false
7916 3105 Acked Write 2 Done Sent Done false
7917 3106 Acked Flush 0 Done Sent Done false
7918 3107 Acked Read 9 Done Sent Done false
7919 3108 Acked Read 2 Done Sent Done false
7920 3109 Acked Write 7 Done Sent Done false
7921 3110 Acked Write 3 Done Sent Done false
7922 3111 Acked Flush 0 Done Sent Done false
7923 3112 Acked Read 1 Done Sent Done false
7924 STATES DS:0 DS:1 DS:2 TOTAL
7925 Sent 0 301 0 301
7926 Done 301 0 301 602
7927 Skipped 0 0 0 0
7928 Error 0 0 0 0
7929 Last Flush: 3111 2811 3111
7930 Downstairs last five completed: 2811 2810 2809 2808 2807
7931 Upstairs last five completed: 3112 3111 3110 3109 3108
7932 CLIENT: Up:0 ds:301 act:2
7933 [8820] got start action
7934 [8820] I am going to restart
7935 Starting downstairs at port 8820
7936 Make output file at "/tmp/dsc/downstairs-8820.txt"
7937 [8820][1] reports Starting
7938 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1209)
7939 [8820][1] reports Running
7940 ----------------------------------------------------------------
7941 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7942 States: Active Offline Active
7943 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7944 2812 Acked Read 8 Done Sent Done false
7945 2813 Acked Read 2 Done Sent Done false
7946 2814 Acked Read 2 Done Sent Done false
7947 2815 Acked Read 9 Done Sent Done false
7948 2816 Acked Read 5 Done Sent Done false
7949 2817 Acked Write 8 Done Sent Done false
7950 2818 Acked Read 10 Done Sent Done false
7951 2819 Acked Write 8 Done Sent Done false
7952 2820 Acked Write 4 Done Sent Done false
7953 2821 Acked Write 3 Done Sent Done false
7954 2822 Acked Read 4 Done Sent Done false
7955 2823 Acked Write 1 Done Sent Done false
7956 2824 Acked Read 7 Done Sent Done false
7957 2825 Acked Read 4 Done Sent Done false
7958 2826 Acked Write 5 Done Sent Done false
7959 2827 Acked Read 4 Done Sent Done false
7960 2828 Acked Read 8 Done Sent Done false
7961 2829 Acked Read 2 Done Sent Done false
7962 2830 Acked Write 6 Done Sent Done false
7963 2831 Acked Write 9 Done Sent Done false
7964 2832 Acked Write 10 Done Sent Done false
7965 2833 Acked Write 3 Done Sent Done false
7966 2834 Acked Write 5 Done Sent Done false
7967 2835 Acked Write 6 Done Sent Done false
7968 2836 Acked Read 3 Done Sent Done false
7969 2837 Acked Read 9 Done Sent Done false
7970 2838 Acked Write 9 Done Sent Done false
7971 2839 Acked Read 6 Done Sent Done false
7972 2840 Acked Write 1 Done Sent Done false
7973 2841 Acked Flush 0 Done Sent Done false
7974 2842 Acked Read 6 Done Sent Done false
7975 2843 Acked Read 6 Done Sent Done false
7976 2844 Acked Read 1 Done Sent Done false
7977 2845 Acked Read 9 Done Sent Done false
7978 2846 Acked Read 8 Done Sent Done false
7979 2847 Acked Read 3 Done Sent Done false
7980 2848 Acked Read 3 Done Sent Done false
7981 2849 Acked Write 1 Done Sent Done false
7982 2850 Acked Write 10 Done Sent Done false
7983 2851 Acked Read 4 Done Sent Done false
7984 2852 Acked Read 8 Done Sent Done false
7985 2853 Acked Read 9 Done Sent Done false
7986 2854 Acked Write 5 Done Sent Done false
7987 2855 Acked Write 3 Done Sent Done false
7988 2856 Acked Write 7 Done Sent Done false
7989 2857 Acked Read 2 Done Sent Done false
7990 2858 Acked Write 2 Done Sent Done false
7991 2859 Acked Read 9 Done Sent Done false
7992 2860 Acked Read 8 Done Sent Done false
7993 2861 Acked Read 1 Done Sent Done false
7994 2862 Acked Write 1 Done Sent Done false
7995 2863 Acked Write 9 Done Sent Done false
7996 2864 Acked Write 6 Done Sent Done false
7997 2865 Acked Read 3 Done Sent Done false
7998 2866 Acked Read 7 Done Sent Done false
7999 2867 Acked Read 4 Done Sent Done false
8000 2868 Acked Write 2 Done Sent Done false
8001 2869 Acked Write 2 Done Sent Done false
8002 2870 Acked Read 8 Done Sent Done false
8003 2871 Acked Read 5 Done Sent Done false
8004 2872 Acked Flush 0 Done Sent Done false
8005 2873 Acked Read 4 Done Sent Done false
8006 2874 Acked Read 7 Done Sent Done false
8007 2875 Acked Read 2 Done Sent Done false
8008 2876 Acked Write 9 Done Sent Done false
8009 2877 Acked Flush 0 Done Sent Done false
8010 2878 Acked Read 2 Done Sent Done false
8011 2879 Acked Write 3 Done Sent Done false
8012 2880 Acked Read 5 Done Sent Done false
8013 2881 Acked Read 4 Done Sent Done false
8014 2882 Acked Flush 0 Done Sent Done false
8015 2883 Acked Read 7 Done Sent Done false
8016 2884 Acked Write 10 Done Sent Done false
8017 2885 Acked Write 2 Done Sent Done false
8018 2886 Acked Read 10 Done Sent Done false
8019 2887 Acked Read 5 Done Sent Done false
8020 2888 Acked Write 2 Done Sent Done false
8021 2889 Acked Read 2 Done Sent Done false
8022 2890 Acked Write 3 Done Sent Done false
8023 2891 Acked Read 10 Done Sent Done false
8024 2892 Acked Read 5 Done Sent Done false
8025 2893 Acked Flush 0 Done Sent Done false
8026 2894 Acked Write 10 Done Sent Done false
8027 2895 Acked Write 6 Done Sent Done false
8028 2896 Acked Write 4 Done Sent Done false
8029 2897 Acked Read 3 Done Sent Done false
8030 2898 Acked Flush 0 Done Sent Done false
8031 2899 Acked Read 1 Done Sent Done false
8032 2900 Acked Read 3 Done Sent Done false
8033 2901 Acked Read 9 Done Sent Done false
8034 2902 Acked Read 9 Done Sent Done false
8035 2903 Acked Write 7 Done Sent Done false
8036 2904 Acked Write 5 Done Sent Done false
8037 2905 Acked Write 9 Done Sent Done false
8038 2906 Acked Read 5 Done Sent Done false
8039 2907 Acked Read 7 Done Sent Done false
8040 2908 Acked Read 5 Done Sent Done false
8041 2909 Acked Write 3 Done Sent Done false
8042 2910 Acked Read 10 Done Sent Done false
8043 2911 Acked Read 3 Done Sent Done false
8044 2912 Acked Read 3 Done Sent Done false
8045 2913 Acked Write 5 Done Sent Done false
8046 2914 Acked Write 8 Done Sent Done false
8047 2915 Acked Write 3 Done Sent Done false
8048 2916 Acked Write 3 Done Sent Done false
8049 2917 Acked Flush 0 Done Sent Done false
8050 2918 Acked Read 7 Done Sent Done false
8051 2919 Acked Read 8 Done Sent Done false
8052 2920 Acked Read 4 Done Sent Done false
8053 2921 Acked Read 8 Done Sent Done false
8054 2922 Acked Read 7 Done Sent Done false
8055 2923 Acked Read 2 Done Sent Done false
8056 2924 Acked Read 7 Done Sent Done false
8057 2925 Acked Write 5 Done Sent Done false
8058 2926 Acked Read 1 Done Sent Done false
8059 2927 Acked Flush 0 Done Sent Done false
8060 2928 Acked Write 1 Done Sent Done false
8061 2929 Acked Write 2 Done Sent Done false
8062 2930 Acked Read 5 Done Sent Done false
8063 2931 Acked Read 6 Done Sent Done false
8064 2932 Acked Read 2 Done Sent Done false
8065 2933 Acked Read 7 Done Sent Done false
8066 2934 Acked Read 3 Done Sent Done false
8067 2935 Acked Write 3 Done Sent Done false
8068 2936 Acked Flush 0 Done Sent Done false
8069 2937 Acked Read 6 Done Sent Done false
8070 2938 Acked Write 3 Done Sent Done false
8071 2939 Acked Write 1 Done Sent Done false
8072 2940 Acked Read 9 Done Sent Done false
8073 2941 Acked Write 9 Done Sent Done false
8074 2942 Acked Write 9 Done Sent Done false
8075 2943 Acked Read 6 Done Sent Done false
8076 2944 Acked Read 1 Done Sent Done false
8077 2945 Acked Write 10 Done Sent Done false
8078 2946 Acked Write 7 Done Sent Done false
8079 2947 Acked Read 7 Done Sent Done false
8080 2948 Acked Read 5 Done Sent Done false
8081 2949 Acked Read 8 Done Sent Done false
8082 2950 Acked Read 4 Done Sent Done false
8083 2951 Acked Flush 0 Done Sent Done false
8084 2952 Acked Write 1 Done Sent Done false
8085 2953 Acked Read 8 Done Sent Done false
8086 2954 Acked Read 4 Done Sent Done false
8087 2955 Acked Write 2 Done Sent Done false
8088 2956 Acked Write 6 Done Sent Done false
8089 2957 Acked Read 6 Done Sent Done false
8090 2958 Acked Write 5 Done Sent Done false
8091 2959 Acked Write 7 Done Sent Done false
8092 2960 Acked Read 2 Done Sent Done false
8093 2961 Acked Read 2 Done Sent Done false
8094 2962 Acked Write 3 Done Sent Done false
8095 2963 Acked Write 1 Done Sent Done false
8096 2964 Acked Flush 0 Done Sent Done false
8097 2965 Acked Read 1 Done Sent Done false
8098 2966 Acked Read 6 Done Sent Done false
8099 2967 Acked Flush 0 Done Sent Done false
8100 2968 Acked Read 10 Done Sent Done false
8101 2969 Acked Write 4 Done Sent Done false
8102 2970 Acked Read 7 Done Sent Done false
8103 2971 Acked Read 10 Done Sent Done false
8104 2972 Acked Write 5 Done Sent Done false
8105 2973 Acked Read 7 Done Sent Done false
8106 2974 Acked Write 10 Done Sent Done false
8107 2975 Acked Write 7 Done Sent Done false
8108 2976 Acked Read 4 Done Sent Done false
8109 2977 Acked Read 4 Done Sent Done false
8110 2978 Acked Read 3 Done Sent Done false
8111 2979 Acked Read 10 Done Sent Done false
8112 2980 Acked Write 6 Done Sent Done false
8113 2981 Acked Write 7 Done Sent Done false
8114 2982 Acked Flush 0 Done Sent Done false
8115 2983 Acked Read 2 Done Sent Done false
8116 2984 Acked Read 10 Done Sent Done false
8117 2985 Acked Flush 0 Done Sent Done false
8118 2986 Acked Write 7 Done Sent Done false
8119 2987 Acked Read 3 Done Sent Done false
8120 2988 Acked Read 3 Done Sent Done false
8121 2989 Acked Read 4 Done Sent Done false
8122 2990 Acked Read 5 Done Sent Done false
8123 2991 Acked Write 7 Done Sent Done false
8124 2992 Acked Read 7 Done Sent Done false
8125 2993 Acked Write 5 Done Sent Done false
8126 2994 Acked Read 5 Done Sent Done false
8127 2995 Acked Flush 0 Done Sent Done false
8128 2996 Acked Read 8 Done Sent Done false
8129 2997 Acked Write 10 Done Sent Done false
8130 2998 Acked Read 2 Done Sent Done false
8131 2999 Acked Write 3 Done Sent Done false
8132 3000 Acked Read 8 Done Sent Done false
8133 3001 Acked Write 9 Done Sent Done false
8134 3002 Acked Read 10 Done Sent Done false
8135 3003 Acked Read 7 Done Sent Done false
8136 3004 Acked Read 7 Done Sent Done false
8137 3005 Acked Read 9 Done Sent Done false
8138 3006 Acked Read 2 Done Sent Done false
8139 3007 Acked Write 9 Done Sent Done false
8140 3008 Acked Write 10 Done Sent Done false
8141 3009 Acked Flush 0 Done Sent Done false
8142 3010 Acked Write 2 Done Sent Done false
8143 3011 Acked Read 1 Done Sent Done false
8144 3012 Acked Read 4 Done Sent Done false
8145 3013 Acked Write 10 Done Sent Done false
8146 3014 Acked Write 2 Done Sent Done false
8147 3015 Acked Read 7 Done Sent Done false
8148 3016 Acked Write 4 Done Sent Done false
8149 3017 Acked Write 5 Done Sent Done false
8150 3018 Acked Flush 0 Done Sent Done false
8151 3019 Acked Write 9 Done Sent Done false
8152 3020 Acked Write 9 Done Sent Done false
8153 3021 Acked Write 1 Done Sent Done false
8154 3022 Acked Write 2 Done Sent Done false
8155 3023 Acked Write 1 Done Sent Done false
8156 3024 Acked Read 8 Done Sent Done false
8157 3025 Acked Read 10 Done Sent Done false
8158 3026 Acked Write 2 Done Sent Done false
8159 3027 Acked Write 7 Done Sent Done false
8160 3028 Acked Write 6 Done Sent Done false
8161 3029 Acked Write 10 Done Sent Done false
8162 3030 Acked Read 1 Done Sent Done false
8163 3031 Acked Flush 0 Done Sent Done false
8164 3032 Acked Read 1 Done Sent Done false
8165 3033 Acked Write 1 Done Sent Done false
8166 3034 Acked Read 5 Done Sent Done false
8167 3035 Acked Write 1 Done Sent Done false
8168 3036 Acked Write 7 Done Sent Done false
8169 3037 Acked Read 1 Done Sent Done false
8170 3038 Acked Read 4 Done Sent Done false
8171 3039 Acked Write 10 Done Sent Done false
8172 3040 Acked Read 5 Done Sent Done false
8173 3041 Acked Write 4 Done Sent Done false
8174 3042 Acked Flush 0 Done Sent Done false
8175 3043 Acked Read 2 Done Sent Done false
8176 3044 Acked Write 1 Done Sent Done false
8177 3045 Acked Write 8 Done Sent Done false
8178 3046 Acked Read 7 Done Sent Done false
8179 3047 Acked Write 8 Done Sent Done false
8180 3048 Acked Read 1 Done Sent Done false
8181 3049 Acked Read 9 Done Sent Done false
8182 3050 Acked Write 3 Done Sent Done false
8183 3051 Acked Write 1 Done Sent Done false
8184 3052 Acked Read 2 Done Sent Done false
8185 3053 Acked Write 1 Done Sent Done false
8186 3054 Acked Write 9 Done Sent Done false
8187 3055 Acked Read 4 Done Sent Done false
8188 3056 Acked Write 8 Done Sent Done false
8189 3057 Acked Read 10 Done Sent Done false
8190 3058 Acked Write 4 Done Sent Done false
8191 3059 Acked Write 8 Done Sent Done false
8192 3060 Acked Write 8 Done Sent Done false
8193 3061 Acked Flush 0 Done Sent Done false
8194 3062 Acked Read 8 Done Sent Done false
8195 3063 Acked Write 3 Done Sent Done false
8196 3064 Acked Read 8 Done Sent Done false
8197 3065 Acked Flush 0 Done Sent Done false
8198 3066 Acked Flush 0 Done Sent Done false
8199 3067 Acked Read 10 Done Sent Done false
8200 3068 Acked Write 4 Done Sent Done false
8201 3069 Acked Read 7 Done Sent Done false
8202 3070 Acked Read 10 Done Sent Done false
8203 3071 Acked Write 3 Done Sent Done false
8204 3072 Acked Write 5 Done Sent Done false
8205 3073 Acked Write 8 Done Sent Done false
8206 3074 Acked Flush 0 Done Sent Done false
8207 3075 Acked Read 5 Done Sent Done false
8208 3076 Acked Read 4 Done Sent Done false
8209 3077 Acked Flush 0 Done Sent Done false
8210 3078 Acked Read 6 Done Sent Done false
8211 3079 Acked Write 10 Done Sent Done false
8212 3080 Acked Read 9 Done Sent Done false
8213 3081 Acked Write 1 Done Sent Done false
8214 3082 Acked Write 10 Done Sent Done false
8215 3083 Acked Read 3 Done Sent Done false
8216 3084 Acked Read 3 Done Sent Done false
8217 3085 Acked Read 8 Done Sent Done false
8218 3086 Acked Flush 0 Done Sent Done false
8219 3087 Acked Read 3 Done Sent Done false
8220 3088 Acked Flush 0 Done Sent Done false
8221 3089 Acked Read 3 Done Sent Done false
8222 3090 Acked Flush 0 Done Sent Done false
8223 3091 Acked Write 3 Done Sent Done false
8224 3092 Acked Read 6 Done Sent Done false
8225 3093 Acked Write 3 Done Sent Done false
8226 3094 Acked Read 4 Done Sent Done false
8227 3095 Acked Write 4 Done Sent Done false
8228 3096 Acked Write 3 Done Sent Done false
8229 3097 Acked Read 9 Done Sent Done false
8230 3098 Acked Read 6 Done Sent Done false
8231 3099 Acked Write 5 Done Sent Done false
8232 3100 Acked Write 4 Done Sent Done false
8233 3101 Acked Write 4 Done Sent Done false
8234 3102 Acked Read 1 Done Sent Done false
8235 3103 Acked Read 3 Done Sent Done false
8236 3104 Acked Read 9 Done Sent Done false
8237 3105 Acked Write 2 Done Sent Done false
8238 3106 Acked Flush 0 Done Sent Done false
8239 3107 Acked Read 9 Done Sent Done false
8240 3108 Acked Read 2 Done Sent Done false
8241 3109 Acked Write 7 Done Sent Done false
8242 3110 Acked Write 3 Done Sent Done false
8243 3111 Acked Flush 0 Done Sent Done false
8244 3112 Acked Read 1 Done Sent Done false
8245 3113 Acked Flush 0 Done Sent Done false
8246 STATES DS:0 DS:1 DS:2 TOTAL
8247 Sent 0 302 0 302
8248 Done 302 0 302 604
8249 Skipped 0 0 0 0
8250 Error 0 0 0 0
8251 Last Flush: 3113 2811 3113
8252 Downstairs last five completed: 2811 2810 2809 2808 2807
8253 Upstairs last five completed: 3113 3112 3111 3110 3109
8254 CLIENT: Up:0 ds:302 act:2
82552025-01-13T15:53:30.548ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
82562025-01-13T15:53:30.548ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:65110) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
82572025-01-13T15:53:30.550ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
82582025-01-13T15:53:30.550ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
82592025-01-13T15:53:30.550ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
82602025-01-13T15:53:30.550ZINFOcrucible: send last flush ID to this DS: 2811 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
82612025-01-13T15:53:30.550ZINFOcrucible: Replied this last flush ID: 2811 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
82622025-01-13T15:53:30.550ZINFOcrucible: [1] client re-new 302 jobs since flush 2811 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
82632025-01-13T15:53:30.551ZINFOcrucible: [1] Replayed 302 jobs since flush: 2811 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
8264 ----------------------------------------------------------------
8265 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
8266 Downstairs last five completed: 3113 3112 3111 3110 3109
8267 Upstairs last five completed: 3113 3112 3111 3110 3109
8268 CLIENT: Up:0 ds:0 act:3
8269 Replay: All jobs finished
8270 Main task has work to do, go find it
8271 got dsc Stop(1)
8272 stop 1
8273 Jan 13 15:53:32.793 INFO request completed[8820] Got stop action so:false kr:false
8274 , latency_us: 564, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 3ff8b360-e396-450c-9b6a-19097a93a67a, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
82752025-01-13T15:53:32.794ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
82762025-01-13T15:53:32.794ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
82772025-01-13T15:53:32.794ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
8278 Jan 13 15:53:32.794 INFO request completed, latency_us: 133, response_code: 200, uri: /state/cid/1, method: GET, req_id: eadb30c8-3838-4429-ac6a-ca77a6d4174e, remote_addr: 127.0.0.{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-13T15:53:32.794804273Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
8279 1:52422, local_addr: 127.0.0.1:9998
8280 [8820] Exited with: ExitStatus(unix_wait_status(9))
8281 [8820][1] reports Exit
8282 Jan 13 15:53:36.797 INFO request completed, latency_us: 328, response_code: 200, uri: /state/cid/1, method: GET, req_id: 7a1cafa7-2ab2-4a67-b204-e43aebcc3ac6, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
8283 001/300 Read block 540 len 32768
8284 002/300 Read block 184 len 36864
8285 003/300 Read block 238 len 4096
8286 004/300 Read block 4934 len 8192
8287 005/300 Read block 2274 len 32768
8288 006/300 Read block 2628 len 8192
8289 007/300 Read block 2258 len 24576
8290 008/300 Write block 2637 len 40960 data: 4 4 4 3 3 3 4 4 4 4
8291 009/300 Read block 1632 len 24576
8292 010/300 Flush
8293 011/300 Read block 719 len 24576
8294 012/300 Read block 4996 len 16384
8295 013/300 Write block 1494 len 28672 data: 4 3 3 3 4 5 4
8296 014/300 Read block 1205 len 32768
8297 015/300 Read block 218 len 40960
8298 016/300 Read block 2823 len 20480
8299 017/300 Flush
8300 018/300 Read block 2838 len 28672
8301 019/300 Write block 2863 len 36864 data: 2 2 2 2 2 2 2 2 3
8302 020/300 Read block 3703 len 16384
8303 021/300 Read block 792 len 16384
8304 022/300 Read block 4369 len 40960
8305 023/300 Read block 3223 len 28672
8306 024/300 Read block 1736 len 16384
8307 025/300 Write block 2595 len 8192 data: 7 7
8308 026/300 Write block 2593 len 28672 data: 7 7 8 8 7 5 5
8309 027/300 Read block 1771 len 12288
8310 028/300 Read block 2141 len 40960
8311 029/300 Read block 1749 len 32768
8312 030/300 Read block 2739 len 12288
8313 031/300 Write block 287 len 36864 data: 4 4 3 3 3 2 2 2 3
8314 032/300 Read block 1308 len 40960
8315 033/300 Read block 1077 len 4096
8316 034/300 Read block 4242 len 20480
8317 035/300 Write block 4129 len 8192 data: 2 2
8318 036/300 Write block 1563 len 16384 data: 2 2 2 2
8319 037/300 Read block 2384 len 36864
8320 038/300 Write block 2083 len 36864 data: 4 3 2 2 2 2 3 3 3
8321 039/300 Read block 4485 len 8192
8322 040/300 Read block 2801 len 4096
8323 041/300 Write block 3195 len 8192 data: 3 4
8324 042/300 Write block 3225 len 20480 data: 3 4 5 4 3
8325 043/300 Read block 3092 len 28672
8326 044/300 Read block 4701 len 28672
8327 045/300 Write block 704 len 20480 data: 3 3 3 3 2
8328 046/300 Read block 3040 len 8192
8329 047/300 Read block 1519 len 28672
8330 048/300 Flush
8331 049/300 Write block 4931 len 20480 data: 7 7 6 6 5
8332 050/300 Read block 4612 len 28672
8333 051/300 Read block 4652 len 28672
8334 052/300 Write block 285 len 32768 data: 4 4 5 5 4 4 4 3
8335 053/300 Write block 4600 len 4096 data: 6
8336 054/300 Write block 4325 len 12288 data: 3 3 3
8337 055/300 Write block 828 len 40960 data: 2 2 2 3 3 2 3 2 2 2
8338 056/300 Flush
8339 057/300 Write block 3394 len 20480 data: 5 6 6 5 4
8340 058/300 Flush
8341 059/300 Read block 2222 len 12288
8342 060/300 Read block 1544 len 20480
8343 061/300 Read block 2793 len 36864
8344 062/300 Read block 1678 len 32768
8345 063/300 Write block 4115 len 16384 data: 2 3 3 2
8346 064/300 Write block 208 len 20480 data: 2 2 2 3 3
8347 065/300 Write block 3262 len 32768 data: 3 3 4 4 3 3 3 3
8348 066/300 Write block 4735 len 40960 data: 6 5 5 4 4 4 3 2 2 3
8349 067/300 Write block 2002 len 40960 data: 3 3 3 3 3 3 3 3 3 2
8350 068/300 Read block 1142 len 28672
8351 069/300 Read block 2182 len 32768
8352 070/300 Write block 3658 len 20480 data: 2 2 2 2 2
8353 071/300 Read block 4145 len 32768
8354 072/300 Write block 4777 len 8192 data: 3 3
8355 073/300 Read block 179 len 24576
8356 074/300 Read block 1387 len 20480
8357 075/300 Read block 4026 len 40960
8358 076/300 Write block 79 len 24576 data: 4 4 4 4 3 3
8359 077/300 Flush
8360 078/300 Write block 4205 len 12288 data: 5 4 4
8361 079/300 Read block 1073 len 20480
8362 080/300 Write block 4059 len 8192 data: 2 2
8363 081/300 Write block 4313 len 24576 data: 2 2 2 3 3 3
8364 082/300 Write block 1892 len 32768 data: 2 2 2 2 2 3 3 3
8365 083/300 Write block 173 len 16384 data: 3 3 3 3
8366 084/300 Read block 109 len 36864
8367 085/300 Write block 1271 len 32768 data: 2 2 2 2 2 2 3 3
8368 086/300 Read block 2862 len 20480
8369 087/300 Read block 2666 len 20480
8370 088/300 Read block 2757 len 36864
8371 089/300 Read block 3895 len 16384
8372 090/300 Read block 340 len 16384
8373 091/300 Write block 417 len 20480 data: 2 2 2 2 2
8374 092/300 Write block 862 len 36864 data: 4 3 3 3 4 4 3 4 3
8375 093/300 Write block 4107 len 28672 data: 5 5 5 5 5 5 4
8376 094/300 Read block 4872 len 16384
8377 095/300 Write block 4594 len 28672 data: 4 4 5 6 5 5 7
8378 096/300 Read block 2301 len 40960
8379 097/300 Write block 3126 len 20480 data: 3 3 3 2 2
8380 098/300 Write block 3894 len 8192 data: 3 3
8381 099/300 Read block 85 len 16384
8382 100/300 Write block 3449 len 8192 data: 5 5
8383 101/300 Read block 2923 len 4096
8384 102/300 Read block 686 len 40960
8385 103/300 Write block 2481 len 40960 data: 5 5 5 6 4 5 2 2 2 2
8386 104/300 Write block 375 len 32768 data: 4 4 4 4 3 3 3 2
8387 105/300 Read block 1984 len 8192
8388 106/300 Read block 1988 len 12288
8389 107/300 Write block 2171 len 28672 data: 5 5 5 4 4 4 4
8390 108/300 Flush
8391 109/300 Write block 397 len 8192 data: 3 6
8392 110/300 Read block 3881 len 40960
8393 111/300 Write block 4732 len 4096 data: 5
8394 112/300 Read block 1539 len 16384
8395 113/300 Write block 461 len 4096 data: 2
8396 114/300 Write block 1930 len 8192 data: 2 2
8397 115/300 Read block 2866 len 12288
8398 116/300 Write block 23 len 32768 data: 3 4 4 4 4 3 3 3
8399 117/300 Write block 1891 len 24576 data: 2 3 3 3 3 3
8400 118/300 Write block 3513 len 12288 data: 3 4 4
8401 119/300 Write block 3913 len 36864 data: 3 2 3 4 3 3 3 2 3
8402 120/300 Write block 1699 len 8192 data: 3 2
8403 121/300 Flush
8404 122/300 Read block 4517 len 40960
8405 123/300 Flush
8406 124/300 Write block 2550 len 36864 data: 4 4 2 3 3 3 2 2 2
8407 125/300 Write block 3858 len 32768 data: 3 3 3 3 3 3 2 2
8408 126/300 Read block 3563 len 28672
8409 127/300 Read block 3607 len 8192
8410 128/300 Read block 3780 len 40960
8411 129/300 Write block 3975 len 40960 data: 2 2 3 3 3 3 3 3 4 4
8412 130/300 Read block 1138 len 8192
8413 131/300 Read block 916 len 32768
8414 132/300 Write block 2557 len 28672 data: 3 3 2 2 2 2 2
8415 133/300 Read block 1365 len 36864
8416 134/300 Write block 1368 len 8192 data: 2 2
8417 135/300 Flush
8418 136/300 Write block 220 len 28672 data: 3 3 3 3 2 2 2
8419 137/300 Read block 535 len 4096
8420 138/300 Write block 4832 len 24576 data: 2 2 2 3 4 4
8421 139/300 Read block 3158 len 20480
8422 140/300 Write block 4164 len 4096 data: 3
8423 141/300 Read block 1032 len 16384
8424 142/300 Write block 461 len 36864 data: 3 2 2 2 2 2 2 2 2
8425 143/300 Write block 1726 len 32768 data: 2 3 3 3 3 3 3 3
8426 144/300 Write block 4198 len 32768 data: 2 2 3 3 3 3 4 6
8427 145/300 Read block 1213 len 40960
8428 146/300 Read block 163 len 36864
8429 147/300 Read block 1666 len 8192
8430 148/300 Write block 2700 len 28672 data: 3 4 4 4 4 4 4
8431 149/300 Write block 3540 len 4096 data: 2
8432 150/300 Write block 4229 len 4096 data: 5
8433 151/300 Read block 1328 len 16384
8434 152/300 Read block 2199 len 40960
8435 153/300 Read block 667 len 20480
8436 154/300 Read block 4141 len 40960
8437 155/300 Read block 2810 len 20480
8438 156/300 Write block 2767 len 20480 data: 3 3 3 3 3
8439 157/300 Flush
8440 158/300 Read block 2875 len 16384
8441 159/300 Write block 3256 len 4096 data: 2
8442 160/300 Read block 2223 len 4096
8443 161/300 Write block 2963 len 12288 data: 2 3 3
8444 162/300 Read block 1443 len 40960
8445 163/300 Flush
8446 164/300 Read block 2022 len 24576
8447 165/300 Write block 2006 len 20480 data: 4 4 4 4 4
8448 166/300 Write block 520 len 24576 data: 4 3 4 4 4 4
8449 167/300 Write block 1498 len 16384 data: 5 6 5 3
8450 168/300 Write block 1516 len 12288 data: 4 4 4
8451 169/300 Read block 677 len 16384
8452 170/300 Write block 3771 len 12288 data: 3 3 3
8453 171/300 Write block 2734 len 40960 data: 3 3 3 2 2 2 2 2 2 2
8454 172/300 Write block 2834 len 12288 data: 2 2 2
8455 173/300 Read block 4603 len 8192
8456 174/300 Read block 3162 len 16384
8457 175/300 Read block 1800 len 40960
8458 176/300 Write block 4317 len 36864 data: 4 4 3 3 2 2 2 3 4
8459 177/300 Read block 245 len 12288
8460 178/300 Read block 4809 len 4096
8461 179/300 Read block 2201 len 28672
8462 180/300 Read block 3394 len 16384
8463 181/300 Write block 3511 len 16384 data: 3 3 4 5
8464 182/300 Read block 4081 len 24576
8465 183/300 Read block 4335 len 36864
8466 184/300 Write block 2185 len 24576 data: 3 3 3 3 4 4
8467 185/300 Write block 2898 len 24576 data: 2 2 2 3 3 2
8468 186/300 Flush
8469 187/300 Write block 2936 len 32768 data: 2 3 4 4 4 3 3 3
8470 188/300 Write block 3582 len 32768 data: 3 3 3 3 3 2 2 3
8471 189/300 Flush
8472 190/300 Write block 1798 len 16384 data: 4 3 2 2
8473 191/300 Read block 4649 len 20480
8474 192/300 Write block 2548 len 40960 data: 5 5 5 5 3 4 4 4 3 4
8475 193/300 Write block 759 len 24576 data: 4 4 4 3 3 3
8476 194/300 Read block 1335 len 16384
8477 195/300 Write block 3592 len 4096 data: 4
8478 196/300 Read block 1251 len 12288
8479 197/300 Read block 4716 len 16384
8480 198/300 Read block 1553 len 16384
8481 199/300 Write block 2316 len 36864 data: 2 2 2 2 2 2 2 2 2
8482 200/300 Write block 4866 len 40960 data: 4 3 3 3 3 2 2 2 2 2
8483 201/300 Read block 4005 len 8192
8484 202/300 Read block 3659 len 4096
8485 203/300 Read block 3054 len 12288
8486 204/300 Read block 4165 len 4096
8487 205/300 Flush
8488 206/300 Read block 3058 len 12288
8489 207/300 Write block 224 len 32768 data: 3 3 3 2 2 3 3 2
8490 208/300 Flush
8491 209/300 Read block 4056 len 36864
8492 210/300 Write block 4692 len 28672 data: 2 3 3 3 3 3 3
8493 211/300 Write block 3553 len 8192 data: 2 2
8494 212/300 Read block 966 len 24576
8495 213/300 Read block 780 len 12288
8496 214/300 Read block 2802 len 32768
8497 215/300 Read block 470 len 20480
8498 216/300 Read block 1367 len 40960
8499 217/300 Read block 716 len 28672
8500 218/300 Write block 469 len 8192 data: 3 2
8501 219/300 Write block 4817 len 4096 data: 3
8502 220/300 Read block 170 len 4096
8503 221/300 Read block 2577 len 20480
8504 222/300 Write block 3835 len 40960 data: 2 2 2 3 3 3 3 3 3 3
8505 223/300 Read block 445 len 16384
8506 224/300 Read block 3131 len 8192
8507 225/300 Read block 3397 len 8192
8508 226/300 Write block 3107 len 28672 data: 2 2 2 2 2 2 2
8509 227/300 Read block 4792 len 4096
8510 228/300 Write block 2964 len 36864 data: 4 4 4 4 4 4 4 4 4
8511 229/300 Read block 4087 len 4096
8512 230/300 Write block 2262 len 12288 data: 3 4 5
8513 231/300 Flush
8514 232/300 Read block 3434 len 24576
8515 233/300 Read block 2369 len 4096
8516 234/300 Write block 4849 len 4096 data: 3
8517 235/300 Read block 99 len 32768
8518 236/300 Write block 3892 len 12288 data: 2 3 4
8519 237/300 Flush
8520 238/300 Write block 3961 len 40960 data: 4 4 4 4 4 3 3 2 2 2
8521 239/300 Read block 1818 len 8192
8522 240/300 Write block 3590 len 8192 data: 3 3
8523 241/300 Read block 4386 len 24576
8524 242/300 Write block 1080 len 28672 data: 3 3 4 4 4 4 4
8525 243/300 Write block 3172 len 32768 data: 3 3 2 2 2 2 2 2
8526 244/300 Read block 759 len 32768
8527 245/300 Read block 1848 len 8192
8528 246/300 Read block 221 len 16384
8529 247/300 Read block 2136 len 12288
8530 248/300 Read block 2050 len 4096
8531 249/300 Read block 2725 len 40960
8532 250/300 Read block 1938 len 8192
8533 251/300 Write block 3758 len 4096 data: 2
8534 252/300 Read block 2006 len 20480
8535 253/300 Flush
8536 254/300 Write block 391 len 20480 data: 3 3 3 2 2
8537 255/300 Write block 4859 len 36864 data: 3 4 4 4 4 3 4 5 4
8538 256/300 Read block 1453 len 8192
8539 257/300 Read block 4025 len 16384
8540 258/300 Read block 456 len 8192
8541 259/300 Read block 276 len 12288
8542 260/300 Read block 1055 len 32768
8543 261/300 Write block 2022 len 24576 data: 3 4 5 5 7 7
8544 262/300 Read block 2191 len 32768
8545 263/300 Read block 4076 len 40960
8546 264/300 Read block 2052 len 8192
8547 265/300 Write block 797 len 8192 data: 2 2
8548 266/300 Write block 1475 len 8192 data: 2 2
8549 267/300 Flush
8550 268/300 Write block 2698 len 24576 data: 4 4 4 5 5 5
8551 269/300 Flush
8552 270/300 Read block 1600 len 32768
8553 271/300 Write block 1435 len 12288 data: 3 3 3
8554 272/300 Write block 2019 len 24576 data: 4 3 3 4 5 6
8555 273/300 Write block 1107 len 20480 data: 2 2 2 2 3
8556 274/300 Flush
8557 275/300 Read block 1538 len 16384
8558 276/300 Read block 2424 len 24576
8559 277/300 Write block 876 len 20480 data: 4 3 3 4 5
8560 278/300 Read block 4343 len 24576
8561 279/300 Read block 1611 len 4096
8562 280/300 Read block 2176 len 12288
8563 281/300 Read block 3293 len 36864
8564 282/300 Write block 2751 len 16384 data: 5 3 2 2
8565 283/300 Read block 2472 len 16384
8566 284/300 Write block 4366 len 36864 data: 4 4 4 3 4 3 3 3 3
8567 285/300 Read block 1885 len 12288
8568 286/300 Read block 1596 len 20480
8569 287/300 Read block 2758 len 8192
8570 288/300 Read block 1562 len 4096
8571 289/300 Read block 2107 len 12288
8572 290/300 Write block 1029 len 28672 data: 2 3 2 2 2 3 4
8573 291/300 Flush
8574 292/300 Flush
8575 293/300 Write block 148 len 20480 data: 4 4 3 2 2
8576 294/300 Read block 3189 len 16384
8577 295/300 Read block 2022 len 36864
8578 296/300 Read block 3117 len 20480
8579 297/300 Write block 69 len 8192 data: 2 2
8580 298/300 Read block 4161 len 8192
8581 299/300 Write block 4844 len 16384 data: 4 4 3 3
8582 300/300 Write block 2724 len 4096 data: 2
8583 301/300 Read block 870 len 4096
8584 Main task has work to do, go find it
8585 got dsc Start(1)
8586 start 1
8587 Jan 13 15:53:37.036 INFO request completed, latency_us: 224, response_code: 204, uri: /start/cid/1, method: POST, req_id: 313745e0-2cf9-4870-8544-a15940039605, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
8588 Replay: started 1, returned:Ok(())
8589 ----------------------------------------------------------------
8590 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
8591 States: Active Offline Active
8592 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8593 3114 Acked Read 8 Done Sent Done false
8594 3115 Acked Read 9 Done Sent Done false
8595 3116 Acked Read 1 Done Sent Done false
8596 3117 Acked Read 2 Done Sent Done false
8597 3118 Acked Read 8 Done Sent Done false
8598 3119 Acked Read 2 Done Sent Done false
8599 3120 Acked Read 6 Done Sent Done false
8600 3121 Acked Write 10 Done Sent Done false
8601 3122 Acked Read 6 Done Sent Done false
8602 3123 Acked Flush 0 Done Sent Done false
8603 3124 Acked Read 6 Done Sent Done false
8604 3125 Acked Read 4 Done Sent Done false
8605 3126 Acked Write 7 Done Sent Done false
8606 3127 Acked Read 8 Done Sent Done false
8607 3128 Acked Read 10 Done Sent Done false
8608 3129 Acked Read 5 Done Sent Done false
8609 3130 Acked Flush 0 Done Sent Done false
8610 3131 Acked Read 7 Done Sent Done false
8611 3132 Acked Write 9 Done Sent Done false
8612 3133 Acked Read 4 Done Sent Done false
8613 3134 Acked Read 4 Done Sent Done false
8614 3135 Acked Read 10 Done Sent Done false
8615 3136 Acked Read 7 Done Sent Done false
8616 3137 Acked Read 4 Done Sent Done false
8617 3138 Acked Write 2 Done Sent Done false
8618 3139 Acked Write 7 Done Sent Done false
8619 3140 Acked Read 3 Done Sent Done false
8620 3141 Acked Read 10 Done Sent Done false
8621 3142 Acked Read 8 Done Sent Done false
8622 3143 Acked Read 3 Done Sent Done false
8623 3144 Acked Write 9 Done Sent Done false
8624 3145 Acked Read 10 Done Sent Done false
8625 3146 Acked Read 1 Done Sent Done false
8626 3147 Acked Read 5 Done Sent Done false
8627 3148 Acked Write 2 Done Sent Done false
8628 3149 Acked Write 4 Done Sent Done false
8629 3150 Acked Read 9 Done Sent Done false
8630 3151 Acked Write 9 Done Sent Done false
8631 3152 Acked Read 2 Done Sent Done false
8632 3153 Acked Read 1 Done Sent Done false
8633 3154 Acked Write 2 Done Sent Done false
8634 3155 Acked Write 5 Done Sent Done false
8635 3156 Acked Read 7 Done Sent Done false
8636 3157 Acked Read 7 Done Sent Done false
8637 3158 Acked Write 5 Done Sent Done false
8638 3159 Acked Read 2 Done Sent Done false
8639 3160 Acked Read 7 Done Sent Done false
8640 3161 Acked Flush 0 Done Sent Done false
8641 3162 Acked Write 5 Done Sent Done false
8642 3163 Acked Read 7 Done Sent Done false
8643 3164 Acked Read 7 Done Sent Done false
8644 3165 Acked Write 8 Done Sent Done false
8645 3166 Acked Write 1 Done Sent Done false
8646 3167 Acked Write 3 Done Sent Done false
8647 3168 Acked Write 10 Done Sent Done false
8648 3169 Acked Flush 0 Done Sent Done false
8649 3170 Acked Write 5 Done Sent Done false
8650 3171 Acked Flush 0 Done Sent Done false
8651 3172 Acked Read 3 Done Sent Done false
8652 3173 Acked Read 5 Done Sent Done false
8653 3174 Acked Read 9 Done Sent Done false
8654 3175 Acked Read 8 Done Sent Done false
8655 3176 Acked Write 4 Done Sent Done false
8656 3177 Acked Write 5 Done Sent Done false
8657 3178 Acked Write 8 Done Sent Done false
8658 3179 Acked Write 10 Done Sent Done false
8659 3180 Acked Write 10 Done Sent Done false
8660 3181 Acked Read 7 Done Sent Done false
8661 3182 Acked Read 8 Done Sent Done false
8662 3183 Acked Write 5 Done Sent Done false
8663 3184 Acked Read 8 Done Sent Done false
8664 3185 Acked Write 2 Done Sent Done false
8665 3186 Acked Read 6 Done Sent Done false
8666 3187 Acked Read 5 Done Sent Done false
8667 3188 Acked Read 10 Done Sent Done false
8668 3189 Acked Write 6 Done Sent Done false
8669 3190 Acked Flush 0 Done Sent Done false
8670 3191 Acked Write 3 Done Sent Done false
8671 3192 Acked Read 5 Done Sent Done false
8672 3193 Acked Write 2 Done Sent Done false
8673 3194 Acked Write 6 Done Sent Done false
8674 3195 Acked Write 8 Done Sent Done false
8675 3196 Acked Write 4 Done Sent Done false
8676 3197 Acked Read 9 Done Sent Done false
8677 3198 Acked Write 8 Done Sent Done false
8678 3199 Acked Read 5 Done Sent Done false
8679 3200 Acked Read 5 Done Sent Done false
8680 3201 Acked Read 9 Done Sent Done false
8681 3202 Acked Read 4 Done Sent Done false
8682 3203 Acked Read 4 Done Sent Done false
8683 3204 Acked Write 5 Done Sent Done false
8684 3205 Acked Write 9 Done Sent Done false
8685 3206 Acked Write 7 Done Sent Done false
8686 3207 Acked Read 4 Done Sent Done false
8687 3208 Acked Write 7 Done Sent Done false
8688 3209 Acked Read 10 Done Sent Done false
8689 3210 Acked Write 5 Done Sent Done false
8690 3211 Acked Write 2 Done Sent Done false
8691 3212 Acked Read 4 Done Sent Done false
8692 3213 Acked Write 2 Done Sent Done false
8693 3214 Acked Read 1 Done Sent Done false
8694 3215 Acked Read 10 Done Sent Done false
8695 3216 Acked Write 10 Done Sent Done false
8696 3217 Acked Write 8 Done Sent Done false
8697 3218 Acked Read 2 Done Sent Done false
8698 3219 Acked Read 3 Done Sent Done false
8699 3220 Acked Write 7 Done Sent Done false
8700 3221 Acked Flush 0 Done Sent Done false
8701 3222 Acked Write 2 Done Sent Done false
8702 3223 Acked Read 10 Done Sent Done false
8703 3224 Acked Write 1 Done Sent Done false
8704 3225 Acked Read 4 Done Sent Done false
8705 3226 Acked Write 1 Done Sent Done false
8706 3227 Acked Write 2 Done Sent Done false
8707 3228 Acked Read 3 Done Sent Done false
8708 3229 Acked Write 8 Done Sent Done false
8709 3230 Acked Write 6 Done Sent Done false
8710 3231 Acked Write 3 Done Sent Done false
8711 3232 Acked Write 9 Done Sent Done false
8712 3233 Acked Write 2 Done Sent Done false
8713 3234 Acked Flush 0 Done Sent Done false
8714 3235 Acked Read 10 Done Sent Done false
8715 3236 Acked Flush 0 Done Sent Done false
8716 3237 Acked Write 9 Done Sent Done false
8717 3238 Acked Write 8 Done Sent Done false
8718 3239 Acked Read 7 Done Sent Done false
8719 3240 Acked Read 2 Done Sent Done false
8720 3241 Acked Read 10 Done Sent Done false
8721 3242 Acked Write 10 Done Sent Done false
8722 3243 Acked Read 2 Done Sent Done false
8723 3244 Acked Read 8 Done Sent Done false
8724 3245 Acked Write 7 Done Sent Done false
8725 3246 Acked Read 9 Done Sent Done false
8726 3247 Acked Write 2 Done Sent Done false
8727 3248 Acked Flush 0 Done Sent Done false
8728 3249 Acked Write 7 Done Sent Done false
8729 3250 Acked Read 1 Done Sent Done false
8730 3251 Acked Write 6 Done Sent Done false
8731 3252 Acked Read 5 Done Sent Done false
8732 3253 Acked Write 1 Done Sent Done false
8733 3254 Acked Read 4 Done Sent Done false
8734 3255 Acked Write 9 Done Sent Done false
8735 3256 Acked Write 8 Done Sent Done false
8736 3257 Acked Write 8 Done Sent Done false
8737 3258 Acked Read 10 Done Sent Done false
8738 3259 Acked Read 9 Done Sent Done false
8739 3260 Acked Read 2 Done Sent Done false
8740 3261 Acked Write 7 Done Sent Done false
8741 3262 Acked Write 1 Done Sent Done false
8742 3263 Acked Write 1 Done Sent Done false
8743 3264 Acked Read 4 Done Sent Done false
8744 3265 Acked Read 10 Done Sent Done false
8745 3266 Acked Read 5 Done Sent Done false
8746 3267 Acked Read 10 Done Sent Done false
8747 3268 Acked Read 5 Done Sent Done false
8748 3269 Acked Write 5 Done Sent Done false
8749 3270 Acked Flush 0 Done Sent Done false
8750 3271 Acked Read 4 Done Sent Done false
8751 3272 Acked Write 1 Done Sent Done false
8752 3273 Acked Read 1 Done Sent Done false
8753 3274 Acked Write 3 Done Sent Done false
8754 3275 Acked Read 10 Done Sent Done false
8755 3276 Acked Flush 0 Done Sent Done false
8756 3277 Acked Read 6 Done Sent Done false
8757 3278 Acked Write 5 Done Sent Done false
8758 3279 Acked Write 6 Done Sent Done false
8759 3280 Acked Write 4 Done Sent Done false
8760 3281 Acked Write 3 Done Sent Done false
8761 3282 Acked Read 4 Done Sent Done false
8762 3283 Acked Write 3 Done Sent Done false
8763 3284 Acked Write 10 Done Sent Done false
8764 3285 Acked Write 3 Done Sent Done false
8765 3286 Acked Read 2 Done Sent Done false
8766 3287 Acked Read 4 Done Sent Done false
8767 3288 Acked Read 10 Done Sent Done false
8768 3289 Acked Write 9 Done Sent Done false
8769 3290 Acked Read 3 Done Sent Done false
8770 3291 Acked Read 1 Done Sent Done false
8771 3292 Acked Read 7 Done Sent Done false
8772 3293 Acked Read 4 Done Sent Done false
8773 3294 Acked Write 4 Done Sent Done false
8774 3295 Acked Read 6 Done Sent Done false
8775 3296 Acked Read 9 Done Sent Done false
8776 3297 Acked Write 6 Done Sent Done false
8777 3298 Acked Write 6 Done Sent Done false
8778 3299 Acked Flush 0 Done Sent Done false
8779 3300 Acked Write 8 Done Sent Done false
8780 3301 Acked Write 8 Done Sent Done false
8781 3302 Acked Flush 0 Done Sent Done false
8782 3303 Acked Write 4 Done Sent Done false
8783 3304 Acked Read 5 Done Sent Done false
8784 3305 Acked Write 10 Done Sent Done false
8785 3306 Acked Write 6 Done Sent Done false
8786 3307 Acked Read 4 Done Sent Done false
8787 3308 Acked Write 1 Done Sent Done false
8788 3309 Acked Read 3 Done Sent Done false
8789 3310 Acked Read 4 Done Sent Done false
8790 3311 Acked Read 4 Done Sent Done false
8791 3312 Acked Write 9 Done Sent Done false
8792 3313 Acked Write 10 Done Sent Done false
8793 3314 Acked Read 2 Done Sent Done false
8794 3315 Acked Read 1 Done Sent Done false
8795 3316 Acked Read 3 Done Sent Done false
8796 3317 Acked Read 1 Done Sent Done false
8797 3318 Acked Flush 0 Done Sent Done false
8798 3319 Acked Read 3 Done Sent Done false
8799 3320 Acked Write 8 Done Sent Done false
8800 3321 Acked Flush 0 Done Sent Done false
8801 3322 Acked Read 9 Done Sent Done false
8802 3323 Acked Write 7 Done Sent Done false
8803 3324 Acked Write 2 Done Sent Done false
8804 3325 Acked Read 6 Done Sent Done false
8805 3326 Acked Read 3 Done Sent Done false
8806 3327 Acked Read 8 Done Sent Done false
8807 3328 Acked Read 5 Done Sent Done false
8808 3329 Acked Read 10 Done Sent Done false
8809 3330 Acked Read 7 Done Sent Done false
8810 3331 Acked Write 2 Done Sent Done false
8811 3332 Acked Write 1 Done Sent Done false
8812 3333 Acked Read 1 Done Sent Done false
8813 3334 Acked Read 5 Done Sent Done false
8814 3335 Acked Write 10 Done Sent Done false
8815 3336 Acked Read 4 Done Sent Done false
8816 3337 Acked Read 2 Done Sent Done false
8817 3338 Acked Read 2 Done Sent Done false
8818 3339 Acked Write 7 Done Sent Done false
8819 3340 Acked Read 1 Done Sent Done false
8820 3341 Acked Write 9 Done Sent Done false
8821 3342 Acked Read 1 Done Sent Done false
8822 3343 Acked Write 3 Done Sent Done false
8823 3344 Acked Flush 0 Done Sent Done false
8824 3345 Acked Read 6 Done Sent Done false
8825 3346 Acked Read 1 Done Sent Done false
8826 3347 Acked Write 1 Done Sent Done false
8827 3348 Acked Read 8 Done Sent Done false
8828 3349 Acked Write 3 Done Sent Done false
8829 3350 Acked Flush 0 Done Sent Done false
8830 3351 Acked Write 10 Done Sent Done false
8831 3352 Acked Read 2 Done Sent Done false
8832 3353 Acked Write 2 Done Sent Done false
8833 3354 Acked Read 6 Done Sent Done false
8834 3355 Acked Write 7 Done Sent Done false
8835 3356 Acked Write 8 Done Sent Done false
8836 3357 Acked Read 8 Done Sent Done false
8837 3358 Acked Read 2 Done Sent Done false
8838 3359 Acked Read 4 Done Sent Done false
8839 3360 Acked Read 3 Done Sent Done false
8840 3361 Acked Read 1 Done Sent Done false
8841 3362 Acked Read 10 Done Sent Done false
8842 3363 Acked Read 2 Done Sent Done false
8843 3364 Acked Write 1 Done Sent Done false
8844 3365 Acked Read 5 Done Sent Done false
8845 3366 Acked Flush 0 Done Sent Done false
8846 3367 Acked Write 5 Done Sent Done false
8847 3368 Acked Write 9 Done Sent Done false
8848 3369 Acked Read 2 Done Sent Done false
8849 3370 Acked Read 4 Done Sent Done false
8850 3371 Acked Read 2 Done Sent Done false
8851 3372 Acked Read 3 Done Sent Done false
8852 3373 Acked Read 8 Done Sent Done false
8853 3374 Acked Write 6 Done Sent Done false
8854 3375 Acked Read 8 Done Sent Done false
8855 3376 Acked Read 10 Done Sent Done false
8856 3377 Acked Read 2 Done Sent Done false
8857 3378 Acked Write 2 Done Sent Done false
8858 3379 Acked Write 2 Done Sent Done false
8859 3380 Acked Flush 0 Done Sent Done false
8860 3381 Acked Write 6 Done Sent Done false
8861 3382 Acked Flush 0 Done Sent Done false
8862 3383 Acked Read 8 Done Sent Done false
8863 3384 Acked Write 3 Done Sent Done false
8864 3385 Acked Write 6 Done Sent Done false
8865 3386 Acked Write 5 Done Sent Done false
8866 3387 Acked Flush 0 Done Sent Done false
8867 3388 Acked Read 4 Done Sent Done false
8868 3389 Acked Read 6 Done Sent Done false
8869 3390 Acked Write 5 Done Sent Done false
8870 3391 Acked Read 6 Done Sent Done false
8871 3392 Acked Read 1 Done Sent Done false
8872 3393 Acked Read 3 Done Sent Done false
8873 3394 Acked Read 9 Done Sent Done false
8874 3395 Acked Write 4 Done Sent Done false
8875 3396 Acked Read 4 Done Sent Done false
8876 3397 Acked Write 9 Done Sent Done false
8877 3398 Acked Read 3 Done Sent Done false
8878 3399 Acked Read 5 Done Sent Done false
8879 3400 Acked Read 2 Done Sent Done false
8880 3401 Acked Read 1 Done Sent Done false
8881 3402 Acked Read 3 Done Sent Done false
8882 3403 Acked Write 7 Done Sent Done false
8883 3404 Acked Flush 0 Done Sent Done false
8884 3405 Acked Flush 0 Done Sent Done false
8885 3406 Acked Write 5 Done Sent Done false
8886 3407 Acked Read 4 Done Sent Done false
8887 3408 Acked Read 9 Done Sent Done false
8888 3409 Acked Read 5 Done Sent Done false
8889 3410 Acked Write 2 Done Sent Done false
8890 3411 Acked Read 2 Done Sent Done false
8891 3412 Acked Write 4 Done Sent Done false
8892 3413 Acked Write 1 Done Sent Done false
8893 3414 Acked Read 1 Done Sent Done false
8894 STATES DS:0 DS:1 DS:2 TOTAL
8895 Sent 0 301 0 301
8896 Done 301 0 301 602
8897 Skipped 0 0 0 0
8898 Error 0 0 0 0
8899 Last Flush: 3405 3113 3405
8900 Downstairs last five completed: 3113 3112 3111 3110 3109
8901 Upstairs last five completed: 3414 3413 3412 3411 3410
8902 CLIENT: Up:0 ds:301 act:2
8903 [8820] got start action
8904 [8820] I am going to restart
8905 Starting downstairs at port 8820
8906 Make output file at "/tmp/dsc/downstairs-8820.txt"
8907 [8820][1] reports Starting
8908 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1210)
8909 [8820][1] reports Running
8910 ----------------------------------------------------------------
8911 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8912 States: Active Offline Active
8913 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8914 3114 Acked Read 8 Done Sent Done false
8915 3115 Acked Read 9 Done Sent Done false
8916 3116 Acked Read 1 Done Sent Done false
8917 3117 Acked Read 2 Done Sent Done false
8918 3118 Acked Read 8 Done Sent Done false
8919 3119 Acked Read 2 Done Sent Done false
8920 3120 Acked Read 6 Done Sent Done false
8921 3121 Acked Write 10 Done Sent Done false
8922 3122 Acked Read 6 Done Sent Done false
8923 3123 Acked Flush 0 Done Sent Done false
8924 3124 Acked Read 6 Done Sent Done false
8925 3125 Acked Read 4 Done Sent Done false
8926 3126 Acked Write 7 Done Sent Done false
8927 3127 Acked Read 8 Done Sent Done false
8928 3128 Acked Read 10 Done Sent Done false
8929 3129 Acked Read 5 Done Sent Done false
8930 3130 Acked Flush 0 Done Sent Done false
8931 3131 Acked Read 7 Done Sent Done false
8932 3132 Acked Write 9 Done Sent Done false
8933 3133 Acked Read 4 Done Sent Done false
8934 3134 Acked Read 4 Done Sent Done false
8935 3135 Acked Read 10 Done Sent Done false
8936 3136 Acked Read 7 Done Sent Done false
8937 3137 Acked Read 4 Done Sent Done false
8938 3138 Acked Write 2 Done Sent Done false
8939 3139 Acked Write 7 Done Sent Done false
8940 3140 Acked Read 3 Done Sent Done false
8941 3141 Acked Read 10 Done Sent Done false
8942 3142 Acked Read 8 Done Sent Done false
8943 3143 Acked Read 3 Done Sent Done false
8944 3144 Acked Write 9 Done Sent Done false
8945 3145 Acked Read 10 Done Sent Done false
8946 3146 Acked Read 1 Done Sent Done false
8947 3147 Acked Read 5 Done Sent Done false
8948 3148 Acked Write 2 Done Sent Done false
8949 3149 Acked Write 4 Done Sent Done false
8950 3150 Acked Read 9 Done Sent Done false
8951 3151 Acked Write 9 Done Sent Done false
8952 3152 Acked Read 2 Done Sent Done false
8953 3153 Acked Read 1 Done Sent Done false
8954 3154 Acked Write 2 Done Sent Done false
8955 3155 Acked Write 5 Done Sent Done false
8956 3156 Acked Read 7 Done Sent Done false
8957 3157 Acked Read 7 Done Sent Done false
8958 3158 Acked Write 5 Done Sent Done false
8959 3159 Acked Read 2 Done Sent Done false
8960 3160 Acked Read 7 Done Sent Done false
8961 3161 Acked Flush 0 Done Sent Done false
8962 3162 Acked Write 5 Done Sent Done false
8963 3163 Acked Read 7 Done Sent Done false
8964 3164 Acked Read 7 Done Sent Done false
8965 3165 Acked Write 8 Done Sent Done false
8966 3166 Acked Write 1 Done Sent Done false
8967 3167 Acked Write 3 Done Sent Done false
8968 3168 Acked Write 10 Done Sent Done false
8969 3169 Acked Flush 0 Done Sent Done false
8970 3170 Acked Write 5 Done Sent Done false
8971 3171 Acked Flush 0 Done Sent Done false
8972 3172 Acked Read 3 Done Sent Done false
8973 3173 Acked Read 5 Done Sent Done false
8974 3174 Acked Read 9 Done Sent Done false
8975 3175 Acked Read 8 Done Sent Done false
8976 3176 Acked Write 4 Done Sent Done false
8977 3177 Acked Write 5 Done Sent Done false
8978 3178 Acked Write 8 Done Sent Done false
8979 3179 Acked Write 10 Done Sent Done false
8980 3180 Acked Write 10 Done Sent Done false
8981 3181 Acked Read 7 Done Sent Done false
8982 3182 Acked Read 8 Done Sent Done false
8983 3183 Acked Write 5 Done Sent Done false
8984 3184 Acked Read 8 Done Sent Done false
8985 3185 Acked Write 2 Done Sent Done false
8986 3186 Acked Read 6 Done Sent Done false
8987 3187 Acked Read 5 Done Sent Done false
8988 3188 Acked Read 10 Done Sent Done false
8989 3189 Acked Write 6 Done Sent Done false
8990 3190 Acked Flush 0 Done Sent Done false
8991 3191 Acked Write 3 Done Sent Done false
8992 3192 Acked Read 5 Done Sent Done false
8993 3193 Acked Write 2 Done Sent Done false
8994 3194 Acked Write 6 Done Sent Done false
8995 3195 Acked Write 8 Done Sent Done false
8996 3196 Acked Write 4 Done Sent Done false
8997 3197 Acked Read 9 Done Sent Done false
8998 3198 Acked Write 8 Done Sent Done false
8999 3199 Acked Read 5 Done Sent Done false
9000 3200 Acked Read 5 Done Sent Done false
9001 3201 Acked Read 9 Done Sent Done false
9002 3202 Acked Read 4 Done Sent Done false
9003 3203 Acked Read 4 Done Sent Done false
9004 3204 Acked Write 5 Done Sent Done false
9005 3205 Acked Write 9 Done Sent Done false
9006 3206 Acked Write 7 Done Sent Done false
9007 3207 Acked Read 4 Done Sent Done false
9008 3208 Acked Write 7 Done Sent Done false
9009 3209 Acked Read 10 Done Sent Done false
9010 3210 Acked Write 5 Done Sent Done false
9011 3211 Acked Write 2 Done Sent Done false
9012 3212 Acked Read 4 Done Sent Done false
9013 3213 Acked Write 2 Done Sent Done false
9014 3214 Acked Read 1 Done Sent Done false
9015 3215 Acked Read 10 Done Sent Done false
9016 3216 Acked Write 10 Done Sent Done false
9017 3217 Acked Write 8 Done Sent Done false
9018 3218 Acked Read 2 Done Sent Done false
9019 3219 Acked Read 3 Done Sent Done false
9020 3220 Acked Write 7 Done Sent Done false
9021 3221 Acked Flush 0 Done Sent Done false
9022 3222 Acked Write 2 Done Sent Done false
9023 3223 Acked Read 10 Done Sent Done false
9024 3224 Acked Write 1 Done Sent Done false
9025 3225 Acked Read 4 Done Sent Done false
9026 3226 Acked Write 1 Done Sent Done false
9027 3227 Acked Write 2 Done Sent Done false
9028 3228 Acked Read 3 Done Sent Done false
9029 3229 Acked Write 8 Done Sent Done false
9030 3230 Acked Write 6 Done Sent Done false
9031 3231 Acked Write 3 Done Sent Done false
9032 3232 Acked Write 9 Done Sent Done false
9033 3233 Acked Write 2 Done Sent Done false
9034 3234 Acked Flush 0 Done Sent Done false
9035 3235 Acked Read 10 Done Sent Done false
9036 3236 Acked Flush 0 Done Sent Done false
9037 3237 Acked Write 9 Done Sent Done false
9038 3238 Acked Write 8 Done Sent Done false
9039 3239 Acked Read 7 Done Sent Done false
9040 3240 Acked Read 2 Done Sent Done false
9041 3241 Acked Read 10 Done Sent Done false
9042 3242 Acked Write 10 Done Sent Done false
9043 3243 Acked Read 2 Done Sent Done false
9044 3244 Acked Read 8 Done Sent Done false
9045 3245 Acked Write 7 Done Sent Done false
9046 3246 Acked Read 9 Done Sent Done false
9047 3247 Acked Write 2 Done Sent Done false
9048 3248 Acked Flush 0 Done Sent Done false
9049 3249 Acked Write 7 Done Sent Done false
9050 3250 Acked Read 1 Done Sent Done false
9051 3251 Acked Write 6 Done Sent Done false
9052 3252 Acked Read 5 Done Sent Done false
9053 3253 Acked Write 1 Done Sent Done false
9054 3254 Acked Read 4 Done Sent Done false
9055 3255 Acked Write 9 Done Sent Done false
9056 3256 Acked Write 8 Done Sent Done false
9057 3257 Acked Write 8 Done Sent Done false
9058 3258 Acked Read 10 Done Sent Done false
9059 3259 Acked Read 9 Done Sent Done false
9060 3260 Acked Read 2 Done Sent Done false
9061 3261 Acked Write 7 Done Sent Done false
9062 3262 Acked Write 1 Done Sent Done false
9063 3263 Acked Write 1 Done Sent Done false
9064 3264 Acked Read 4 Done Sent Done false
9065 3265 Acked Read 10 Done Sent Done false
9066 3266 Acked Read 5 Done Sent Done false
9067 3267 Acked Read 10 Done Sent Done false
9068 3268 Acked Read 5 Done Sent Done false
9069 3269 Acked Write 5 Done Sent Done false
9070 3270 Acked Flush 0 Done Sent Done false
9071 3271 Acked Read 4 Done Sent Done false
9072 3272 Acked Write 1 Done Sent Done false
9073 3273 Acked Read 1 Done Sent Done false
9074 3274 Acked Write 3 Done Sent Done false
9075 3275 Acked Read 10 Done Sent Done false
9076 3276 Acked Flush 0 Done Sent Done false
9077 3277 Acked Read 6 Done Sent Done false
9078 3278 Acked Write 5 Done Sent Done false
9079 3279 Acked Write 6 Done Sent Done false
9080 3280 Acked Write 4 Done Sent Done false
9081 3281 Acked Write 3 Done Sent Done false
9082 3282 Acked Read 4 Done Sent Done false
9083 3283 Acked Write 3 Done Sent Done false
9084 3284 Acked Write 10 Done Sent Done false
9085 3285 Acked Write 3 Done Sent Done false
9086 3286 Acked Read 2 Done Sent Done false
9087 3287 Acked Read 4 Done Sent Done false
9088 3288 Acked Read 10 Done Sent Done false
9089 3289 Acked Write 9 Done Sent Done false
9090 3290 Acked Read 3 Done Sent Done false
9091 3291 Acked Read 1 Done Sent Done false
9092 3292 Acked Read 7 Done Sent Done false
9093 3293 Acked Read 4 Done Sent Done false
9094 3294 Acked Write 4 Done Sent Done false
9095 3295 Acked Read 6 Done Sent Done false
9096 3296 Acked Read 9 Done Sent Done false
9097 3297 Acked Write 6 Done Sent Done false
9098 3298 Acked Write 6 Done Sent Done false
9099 3299 Acked Flush 0 Done Sent Done false
9100 3300 Acked Write 8 Done Sent Done false
9101 3301 Acked Write 8 Done Sent Done false
9102 3302 Acked Flush 0 Done Sent Done false
9103 3303 Acked Write 4 Done Sent Done false
9104 3304 Acked Read 5 Done Sent Done false
9105 3305 Acked Write 10 Done Sent Done false
9106 3306 Acked Write 6 Done Sent Done false
9107 3307 Acked Read 4 Done Sent Done false
9108 3308 Acked Write 1 Done Sent Done false
9109 3309 Acked Read 3 Done Sent Done false
9110 3310 Acked Read 4 Done Sent Done false
9111 3311 Acked Read 4 Done Sent Done false
9112 3312 Acked Write 9 Done Sent Done false
9113 3313 Acked Write 10 Done Sent Done false
9114 3314 Acked Read 2 Done Sent Done false
9115 3315 Acked Read 1 Done Sent Done false
9116 3316 Acked Read 3 Done Sent Done false
9117 3317 Acked Read 1 Done Sent Done false
9118 3318 Acked Flush 0 Done Sent Done false
9119 3319 Acked Read 3 Done Sent Done false
9120 3320 Acked Write 8 Done Sent Done false
9121 3321 Acked Flush 0 Done Sent Done false
9122 3322 Acked Read 9 Done Sent Done false
9123 3323 Acked Write 7 Done Sent Done false
9124 3324 Acked Write 2 Done Sent Done false
9125 3325 Acked Read 6 Done Sent Done false
9126 3326 Acked Read 3 Done Sent Done false
9127 3327 Acked Read 8 Done Sent Done false
9128 3328 Acked Read 5 Done Sent Done false
9129 3329 Acked Read 10 Done Sent Done false
9130 3330 Acked Read 7 Done Sent Done false
9131 3331 Acked Write 2 Done Sent Done false
9132 3332 Acked Write 1 Done Sent Done false
9133 3333 Acked Read 1 Done Sent Done false
9134 3334 Acked Read 5 Done Sent Done false
9135 3335 Acked Write 10 Done Sent Done false
9136 3336 Acked Read 4 Done Sent Done false
9137 3337 Acked Read 2 Done Sent Done false
9138 3338 Acked Read 2 Done Sent Done false
9139 3339 Acked Write 7 Done Sent Done false
9140 3340 Acked Read 1 Done Sent Done false
9141 3341 Acked Write 9 Done Sent Done false
9142 3342 Acked Read 1 Done Sent Done false
9143 3343 Acked Write 3 Done Sent Done false
9144 3344 Acked Flush 0 Done Sent Done false
9145 3345 Acked Read 6 Done Sent Done false
9146 3346 Acked Read 1 Done Sent Done false
9147 3347 Acked Write 1 Done Sent Done false
9148 3348 Acked Read 8 Done Sent Done false
9149 3349 Acked Write 3 Done Sent Done false
9150 3350 Acked Flush 0 Done Sent Done false
9151 3351 Acked Write 10 Done Sent Done false
9152 3352 Acked Read 2 Done Sent Done false
9153 3353 Acked Write 2 Done Sent Done false
9154 3354 Acked Read 6 Done Sent Done false
9155 3355 Acked Write 7 Done Sent Done false
9156 3356 Acked Write 8 Done Sent Done false
9157 3357 Acked Read 8 Done Sent Done false
9158 3358 Acked Read 2 Done Sent Done false
9159 3359 Acked Read 4 Done Sent Done false
9160 3360 Acked Read 3 Done Sent Done false
9161 3361 Acked Read 1 Done Sent Done false
9162 3362 Acked Read 10 Done Sent Done false
9163 3363 Acked Read 2 Done Sent Done false
9164 3364 Acked Write 1 Done Sent Done false
9165 3365 Acked Read 5 Done Sent Done false
9166 3366 Acked Flush 0 Done Sent Done false
9167 3367 Acked Write 5 Done Sent Done false
9168 3368 Acked Write 9 Done Sent Done false
9169 3369 Acked Read 2 Done Sent Done false
9170 3370 Acked Read 4 Done Sent Done false
9171 3371 Acked Read 2 Done Sent Done false
9172 3372 Acked Read 3 Done Sent Done false
9173 3373 Acked Read 8 Done Sent Done false
9174 3374 Acked Write 6 Done Sent Done false
9175 3375 Acked Read 8 Done Sent Done false
9176 3376 Acked Read 10 Done Sent Done false
9177 3377 Acked Read 2 Done Sent Done false
9178 3378 Acked Write 2 Done Sent Done false
9179 3379 Acked Write 2 Done Sent Done false
9180 3380 Acked Flush 0 Done Sent Done false
9181 3381 Acked Write 6 Done Sent Done false
9182 3382 Acked Flush 0 Done Sent Done false
9183 3383 Acked Read 8 Done Sent Done false
9184 3384 Acked Write 3 Done Sent Done false
9185 3385 Acked Write 6 Done Sent Done false
9186 3386 Acked Write 5 Done Sent Done false
9187 3387 Acked Flush 0 Done Sent Done false
9188 3388 Acked Read 4 Done Sent Done false
9189 3389 Acked Read 6 Done Sent Done false
9190 3390 Acked Write 5 Done Sent Done false
9191 3391 Acked Read 6 Done Sent Done false
9192 3392 Acked Read 1 Done Sent Done false
9193 3393 Acked Read 3 Done Sent Done false
9194 3394 Acked Read 9 Done Sent Done false
9195 3395 Acked Write 4 Done Sent Done false
9196 3396 Acked Read 4 Done Sent Done false
9197 3397 Acked Write 9 Done Sent Done false
9198 3398 Acked Read 3 Done Sent Done false
9199 3399 Acked Read 5 Done Sent Done false
9200 3400 Acked Read 2 Done Sent Done false
9201 3401 Acked Read 1 Done Sent Done false
9202 3402 Acked Read 3 Done Sent Done false
9203 3403 Acked Write 7 Done Sent Done false
9204 3404 Acked Flush 0 Done Sent Done false
9205 3405 Acked Flush 0 Done Sent Done false
9206 3406 Acked Write 5 Done Sent Done false
9207 3407 Acked Read 4 Done Sent Done false
9208 3408 Acked Read 9 Done Sent Done false
9209 3409 Acked Read 5 Done Sent Done false
9210 3410 Acked Write 2 Done Sent Done false
9211 3411 Acked Read 2 Done Sent Done false
9212 3412 Acked Write 4 Done Sent Done false
9213 3413 Acked Write 1 Done Sent Done false
9214 3414 Acked Read 1 Done Sent Done false
9215 3415 Acked Flush 0 Done Sent Done false
9216 STATES DS:0 DS:1 DS:2 TOTAL
9217 Sent 0 302 0 302
9218 Done 302 0 302 604
9219 Skipped 0 0 0 0
9220 Error 0 0 0 0
9221 Last Flush: 3415 3113 3415
9222 Downstairs last five completed: 3113 3112 3111 3110 3109
9223 Upstairs last five completed: 3415 3414 3413 3412 3411
9224 CLIENT: Up:0 ds:302 act:2
92252025-01-13T15:53:42.795ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
92262025-01-13T15:53:42.795ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41065) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
92272025-01-13T15:53:42.796ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
92282025-01-13T15:53:42.797ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
92292025-01-13T15:53:42.797ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
92302025-01-13T15:53:42.797ZINFOcrucible: send last flush ID to this DS: 3113 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
92312025-01-13T15:53:42.797ZINFOcrucible: Replied this last flush ID: 3113 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
92322025-01-13T15:53:42.797ZINFOcrucible: [1] client re-new 302 jobs since flush 3113 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
92332025-01-13T15:53:42.797ZINFOcrucible: [1] Replayed 302 jobs since flush: 3113 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
9234 ----------------------------------------------------------------
9235 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
9236 Downstairs last five completed: 3415 3414 3413 3412 3411
9237 Upstairs last five completed: 3415 3414 3413 3412 3411
9238 CLIENT: Up:0 ds:0 act:3
9239 Replay: All jobs finished
9240 Main task has work to do, go find it
9241 got dsc Stop(0)
9242 stop 0
9243 Jan 13 15:53:45.041 INFO request completed, latency_us:[8810] Got stop action so:false kr:false
9244 381, response_code: 204, uri: /stop/cid/0, method: POST, req_id: fa58e8a8-f157-4d6a-a55f-f3a6192b99bb, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
92452025-01-13T15:53:45.042ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
92462025-01-13T15:53:45.042ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
9247 Jan 13 15:53:45.042 INFO {"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:53:45.042573307Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
9248 request completed, latency_us: 163, response_code: 200, uri: /state/cid/0, method: GET, req_id: a5b8c0d4-bd77-42e3-bf7b-c45a1c0876c8, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
92492025-01-13T15:53:45.042ZINFOcrucible: client task is exiting = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
9250 [8810] Exited with: ExitStatus(unix_wait_status(9))
9251 [8810][0] reports Exit
9252 Jan 13 15:53:49.044 INFO request completed, latency_us: 226, response_code: 200, uri: /state/cid/0, method: GET, req_id: 9a813d33-37bb-471d-8968-6720320d44d7, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
9253 001/300 Write block 351 len 4096 data: 2
9254 002/300 Read block 4581 len 20480
9255 003/300 Read block 4122 len 12288
9256 004/300 Write block 3820 len 32768 data: 2 2 2 2 2 2 3 4
9257 005/300 Read block 2661 len 36864
9258 006/300 Write block 303 len 40960 data: 2 2 2 2 2 2 2 2 2 2
9259 007/300 Write block 2938 len 40960 data: 5 5 5 4 4 4 3 3 3 3
9260 008/300 Read block 3332 len 20480
9261 009/300 Read block 1544 len 28672
9262 010/300 Write block 3657 len 20480 data: 2 3 3 3 3
9263 011/300 Write block 1206 len 28672 data: 2 2 2 2 2 2 2
9264 012/300 Flush
9265 013/300 Write block 1626 len 8192 data: 3 3
9266 014/300 Read block 4241 len 24576
9267 015/300 Read block 109 len 4096
9268 016/300 Read block 4927 len 32768
9269 017/300 Write block 2117 len 4096 data: 3
9270 018/300 Write block 1604 len 12288 data: 3 4 4
9271 019/300 Write block 4275 len 28672 data: 4 3 2 2 2 2 2
9272 020/300 Write block 3485 len 32768 data: 4 5 6 5 5 4 4 4
9273 021/300 Write block 3267 len 28672 data: 4 4 4 3 3 3 5
9274 022/300 Write block 3002 len 36864 data: 2 3 3 3 3 3 3 4 4
9275 023/300 Flush
9276 024/300 Read block 4067 len 16384
9277 025/300 Write block 3026 len 8192 data: 2 3
9278 026/300 Read block 1420 len 32768
9279 027/300 Flush
9280 028/300 Read block 2968 len 32768
9281 029/300 Write block 4947 len 40960 data: 3 2 2 3 3 3 3 3 3 4
9282 030/300 Read block 894 len 24576
9283 031/300 Read block 1665 len 24576
9284 032/300 Read block 3750 len 32768
9285 033/300 Write block 3002 len 24576 data: 3 4 4 4 4 4
9286 034/300 Read block 721 len 16384
9287 035/300 Read block 2969 len 24576
9288 036/300 Read block 2788 len 36864
9289 037/300 Write block 2279 len 20480 data: 4 3 3 2 2
9290 038/300 Read block 1252 len 32768
9291 039/300 Read block 3079 len 4096
9292 040/300 Write block 454 len 16384 data: 3 2 3 3
9293 041/300 Write block 3143 len 12288 data: 3 3 3
9294 042/300 Write block 1283 len 24576 data: 2 2 2 2 2 2
9295 043/300 Flush
9296 044/300 Write block 3559 len 36864 data: 2 2 2 2 2 2 2 2 2
9297 045/300 Read block 1975 len 16384
9298 046/300 Write block 2457 len 28672 data: 2 2 2 2 2 2 2
9299 047/300 Read block 2606 len 40960
9300 048/300 Write block 172 len 32768 data: 3 4 4 4 4 3 2 2
9301 049/300 Read block 4519 len 12288
9302 050/300 Write block 2728 len 28672 data: 2 2 3 3 3 3 4
9303 051/300 Write block 3792 len 36864 data: 5 4 4 4 4 4 3 3 4
9304 052/300 Write block 3217 len 40960 data: 2 2 3 3 4 4 3 3 4 5
9305 053/300 Write block 1607 len 20480 data: 5 4 4 4 4
9306 054/300 Write block 316 len 40960 data: 2 3 3 2 2 2 2 2 2 2
9307 055/300 Write block 3722 len 32768 data: 4 4 4 3 5 7 7 7
9308 056/300 Write block 2730 len 12288 data: 4 4 4
9309 057/300 Write block 4702 len 4096 data: 2
9310 058/300 Write block 4777 len 32768 data: 4 4 3 3 2 4 4 4
9311 059/300 Read block 4219 len 28672
9312 060/300 Read block 102 len 24576
9313 061/300 Flush
9314 062/300 Flush
9315 063/300 Write block 3435 len 16384 data: 2 2 3 3
9316 064/300 Read block 1961 len 36864
9317 065/300 Write block 4007 len 36864 data: 3 4 3 3 3 3 3 2 2
9318 066/300 Read block 3190 len 8192
9319 067/300 Read block 2460 len 16384
9320 068/300 Write block 725 len 12288 data: 2 2 2
9321 069/300 Write block 167 len 40960 data: 2 3 3 3 3 4 5 5 5 5
9322 070/300 Read block 3084 len 16384
9323 071/300 Flush
9324 072/300 Read block 2194 len 32768
9325 073/300 Read block 4320 len 28672
9326 074/300 Write block 1415 len 32768 data: 2 2 2 3 3 3 3 3
9327 075/300 Read block 2195 len 32768
9328 076/300 Read block 4815 len 28672
9329 077/300 Read block 2792 len 24576
9330 078/300 Read block 4343 len 28672
9331 079/300 Read block 1824 len 40960
9332 080/300 Read block 2572 len 20480
9333 081/300 Flush
9334 082/300 Read block 4476 len 20480
9335 083/300 Read block 4743 len 4096
9336 084/300 Read block 2565 len 36864
9337 085/300 Read block 1283 len 24576
9338 086/300 Write block 2839 len 4096 data: 2
9339 087/300 Read block 4817 len 20480
9340 088/300 Flush
9341 089/300 Read block 4894 len 16384
9342 090/300 Write block 2412 len 36864 data: 2 3 2 2 2 2 2 2 2
9343 091/300 Write block 3270 len 32768 data: 4 4 4 6 5 5 4 4
9344 092/300 Read block 120 len 32768
9345 093/300 Read block 2322 len 20480
9346 094/300 Write block 4320 len 32768 data: 4 3 3 3 4 5 4 4
9347 095/300 Write block 595 len 32768 data: 2 2 2 2 2 2 2 2
9348 096/300 Write block 2209 len 36864 data: 3 3 2 2 2 3 3 4 4
9349 097/300 Flush
9350 098/300 Read block 1938 len 8192
9351 099/300 Read block 4319 len 36864
9352 100/300 Write block 373 len 36864 data: 3 3 5 5 5 5 4 4 4
9353 101/300 Read block 1564 len 8192
9354 102/300 Write block 3631 len 32768 data: 2 2 2 2 3 2 2 2
9355 103/300 Read block 1929 len 32768
9356 104/300 Read block 3460 len 24576
9357 105/300 Read block 4873 len 16384
9358 106/300 Read block 4608 len 24576
9359 107/300 Write block 2230 len 16384 data: 5 5 3 3
9360 108/300 Read block 1180 len 12288
9361 109/300 Write block 1271 len 32768 data: 3 3 3 3 3 3 4 4
9362 110/300 Read block 1088 len 16384
9363 111/300 Read block 3046 len 28672
9364 112/300 Write block 3716 len 36864 data: 2 2 2 2 2 4 5 5 5
9365 113/300 Read block 2944 len 20480
9366 114/300 Read block 4549 len 4096
9367 115/300 Read block 1260 len 12288
9368 116/300 Read block 348 len 36864
9369 117/300 Write block 1823 len 40960 data: 3 3 3 3 3 3 2 2 2 2
9370 118/300 Read block 508 len 32768
9371 119/300 Read block 1081 len 36864
9372 120/300 Read block 270 len 4096
9373 121/300 Write block 3867 len 28672 data: 2 2 2 2 3 3 3
9374 122/300 Write block 3802 len 36864 data: 4 4 5 3 2 2 2 3 4
9375 123/300 Read block 1032 len 16384
9376 124/300 Write block 2545 len 12288 data: 4 5 5
9377 125/300 Read block 1838 len 16384
9378 126/300 Flush
9379 127/300 Read block 378 len 8192
9380 128/300 Flush
9381 129/300 Write block 2876 len 8192 data: 4 2
9382 130/300 Write block 1237 len 20480 data: 3 3 3 3 3
9383 131/300 Read block 2158 len 16384
9384 132/300 Write block 144 len 16384 data: 4 4 4 4
9385 133/300 Write block 3370 len 32768 data: 2 2 2 2 2 2 2 2
9386 134/300 Read block 1181 len 12288
9387 135/300 Read block 3263 len 20480
9388 136/300 Read block 2811 len 40960
9389 137/300 Write block 1672 len 28672 data: 3 2 2 2 2 2 2
9390 138/300 Write block 346 len 16384 data: 3 3 3 2
9391 139/300 Write block 383 len 32768 data: 3 3 3 4 4 3 3 3
9392 140/300 Read block 1142 len 36864
9393 141/300 Read block 330 len 12288
9394 142/300 Read block 4677 len 32768
9395 143/300 Flush
9396 144/300 Read block 1549 len 12288
9397 145/300 Write block 1570 len 4096 data: 4
9398 146/300 Flush
9399 147/300 Read block 2069 len 12288
9400 148/300 Read block 3615 len 40960
9401 149/300 Flush
9402 150/300 Write block 1379 len 36864 data: 4 4 4 5 6 5 5 5 4
9403 151/300 Read block 3099 len 24576
9404 152/300 Read block 1220 len 16384
9405 153/300 Read block 3842 len 4096
9406 154/300 Read block 224 len 20480
9407 155/300 Read block 4672 len 16384
9408 156/300 Read block 70 len 20480
9409 157/300 Read block 4328 len 40960
9410 158/300 Write block 766 len 20480 data: 4 4 4 4 4
9411 159/300 Read block 4544 len 36864
9412 160/300 Read block 4658 len 24576
9413 161/300 Write block 2209 len 40960 data: 4 4 3 3 3 4 4 5 5 4
9414 162/300 Read block 3191 len 12288
9415 163/300 Flush
9416 164/300 Read block 983 len 24576
9417 165/300 Write block 2894 len 36864 data: 3 3 2 2 3 3 3 4 4
9418 166/300 Flush
9419 167/300 Read block 2645 len 4096
9420 168/300 Write block 1586 len 40960 data: 4 3 3 3 4 4 3 2 2 2
9421 169/300 Read block 2813 len 28672
9422 170/300 Flush
9423 171/300 Write block 416 len 20480 data: 2 3 3 3 3
9424 172/300 Write block 2263 len 36864 data: 5 6 5 5 5 4 4 3 4
9425 173/300 Read block 2825 len 24576
9426 174/300 Write block 22 len 40960 data: 2 4 5 5 5 5 4 4 4 3
9427 175/300 Write block 1349 len 36864 data: 2 2 2 3 4 5 5 5 3
9428 176/300 Read block 565 len 16384
9429 177/300 Flush
9430 178/300 Read block 3323 len 16384
9431 179/300 Flush
9432 180/300 Read block 4174 len 20480
9433 181/300 Write block 3882 len 40960 data: 3 3 3 3 3 4 4 3 2 2
9434 182/300 Write block 4441 len 32768 data: 4 4 3 2 2 2 4 5
9435 183/300 Read block 3259 len 16384
9436 184/300 Read block 4771 len 20480
9437 185/300 Write block 1050 len 28672 data: 3 3 4 4 4 4 5
9438 186/300 Write block 4496 len 8192 data: 3 4
9439 187/300 Read block 1736 len 40960
9440 188/300 Read block 1716 len 12288
9441 189/300 Read block 2280 len 32768
9442 190/300 Write block 1833 len 12288 data: 2 2 2
9443 191/300 Write block 266 len 20480 data: 2 2 2 3 4
9444 192/300 Read block 86 len 36864
9445 193/300 Write block 1042 len 32768 data: 4 4 5 5 3 3 4 4
9446 194/300 Write block 3048 len 16384 data: 4 3 3 3
9447 195/300 Read block 3096 len 24576
9448 196/300 Flush
9449 197/300 Read block 3187 len 36864
9450 198/300 Write block 410 len 8192 data: 3 2
9451 199/300 Write block 2677 len 16384 data: 2 2 3 2
9452 200/300 Read block 4231 len 32768
9453 201/300 Write block 4468 len 12288 data: 3 2 3
9454 202/300 Read block 1595 len 40960
9455 203/300 Write block 4834 len 8192 data: 3 4
9456 204/300 Read block 172 len 36864
9457 205/300 Read block 1710 len 28672
9458 206/300 Flush
9459 207/300 Read block 4599 len 32768
9460 208/300 Read block 3906 len 16384
9461 209/300 Write block 3772 len 36864 data: 4 4 2 2 3 3 3 4 3
9462 210/300 Flush
9463 211/300 Write block 3824 len 40960 data: 3 3 4 5 4 4 4 4 4 3
9464 212/300 Flush
9465 213/300 Read block 650 len 12288
9466 214/300 Read block 4937 len 32768
9467 215/300 Write block 3469 len 36864 data: 2 2 2 2 2 2 2 2 2
9468 216/300 Read block 2414 len 8192
9469 217/300 Read block 778 len 24576
9470 218/300 Write block 4596 len 4096 data: 6
9471 219/300 Write block 3978 len 16384 data: 4 4 4 4
9472 220/300 Read block 3347 len 8192
9473 221/300 Read block 4092 len 32768
9474 222/300 Read block 1496 len 4096
9475 223/300 Write block 1132 len 20480 data: 2 2 2 2 2
9476 224/300 Write block 4902 len 28672 data: 2 2 2 2 2 2 2
9477 225/300 Flush
9478 226/300 Read block 4138 len 40960
9479 227/300 Read block 3885 len 12288
9480 228/300 Write block 4735 len 40960 data: 7 6 6 5 5 5 4 3 3 4
9481 229/300 Read block 2685 len 24576
9482 230/300 Write block 2816 len 28672 data: 3 3 3 3 4 4 3
9483 231/300 Write block 3134 len 12288 data: 2 2 3
9484 232/300 Read block 800 len 28672
9485 233/300 Read block 4208 len 16384
9486 234/300 Read block 2507 len 28672
9487 235/300 Write block 444 len 24576 data: 2 3 3 3 3 3
9488 236/300 Read block 2174 len 28672
9489 237/300 Write block 2272 len 20480 data: 4 4 5 5 5
9490 238/300 Read block 356 len 8192
9491 239/300 Flush
9492 240/300 Read block 2220 len 32768
9493 241/300 Read block 4200 len 40960
9494 242/300 Write block 3921 len 40960 data: 4 3 3 3 2 2 2 2 2 2
9495 243/300 Flush
9496 244/300 Read block 2748 len 36864
9497 245/300 Flush
9498 246/300 Flush
9499 247/300 Write block 1431 len 12288 data: 3 3 3
9500 248/300 Read block 2131 len 40960
9501 249/300 Read block 2825 len 8192
9502 250/300 Write block 874 len 4096 data: 3
9503 251/300 Write block 4560 len 40960 data: 2 2 2 2 2 2 2 2 2 2
9504 252/300 Read block 3028 len 40960
9505 253/300 Read block 2196 len 32768
9506 254/300 Read block 2268 len 32768
9507 255/300 Read block 1431 len 32768
9508 256/300 Write block 3484 len 8192 data: 4 5
9509 257/300 Write block 4139 len 12288 data: 2 2 2
9510 258/300 Read block 210 len 20480
9511 259/300 Read block 3234 len 24576
9512 260/300 Read block 2308 len 36864
9513 261/300 Write block 3028 len 32768 data: 3 3 2 2 2 3 3 4
9514 262/300 Write block 3688 len 8192 data: 2 2
9515 263/300 Write block 2271 len 40960 data: 5 5 5 6 6 6 6 5 5 4
9516 264/300 Write block 2390 len 36864 data: 3 3 3 3 3 3 3 2 2
9517 265/300 Flush
9518 266/300 Read block 4909 len 40960
9519 267/300 Write block 2178 len 12288 data: 3 3 3
9520 268/300 Flush
9521 269/300 Write block 2392 len 20480 data: 4 4 4 4 4
9522 270/300 Write block 809 len 12288 data: 3 3 3
9523 271/300 Write block 3791 len 20480 data: 5 6 5 5 5
9524 272/300 Write block 2214 len 40960 data: 5 5 6 6 5 4 4 4 4 5
9525 273/300 Write block 4944 len 8192 data: 4 3
9526 274/300 Write block 14 len 20480 data: 2 2 2 2 2
9527 275/300 Read block 4905 len 28672
9528 276/300 Write block 2856 len 40960 data: 2 2 2 2 2 2 2 3 3 3
9529 277/300 Read block 274 len 16384
9530 278/300 Read block 2334 len 16384
9531 279/300 Read block 1002 len 24576
9532 280/300 Write block 56 len 36864 data: 3 3 3 3 3 3 3 3 3
9533 281/300 Flush
9534 282/300 Read block 2361 len 16384
9535 283/300 Read block 62 len 32768
9536 284/300 Read block 3040 len 20480
9537 285/300 Write block 2203 len 24576 data: 3 3 3 2 2 2
9538 286/300 Write block 936 len 8192 data: 3 2
9539 287/300 Read block 1935 len 16384
9540 288/300 Read block 2495 len 20480
9541 289/300 Read block 2196 len 28672
9542 290/300 Write block 4749 len 4096 data: 4
9543 291/300 Flush
9544 292/300 Read block 3794 len 24576
9545 293/300 Read block 455 len 4096
9546 294/300 Read block 2046 len 4096
9547 295/300 Read block 2291 len 32768
9548 296/300 Read block 1330 len 4096
9549 297/300 Write block 4705 len 32768 data: 2 2 2 2 2 2 2 2
9550 298/300 Read block 1131 len 24576
9551 299/300 Read block 820 len 36864
9552 300/300 Write block 2176 len 28672 data: 5 5 4 4 4 3 3
9553 301/300 Write block 4298 len 16384 data: 6 6 6 6
9554 Main task has work to do, go find it
9555 got dsc Start(0)
9556 start 0
9557 Jan 13 15:53:49.293 INFO request completed, latency_us: 202, response_code: 204, uri: /start/cid/0, method: POST, req_id: 28ad68ae-9d3d-4746-9a9b-16f1a066147e, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
9558 Replay: started 0, returned:Ok(())
9559 ----------------------------------------------------------------
9560 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
9561 States: Offline Active Active
9562 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9563 3416 Acked Write 1 Sent Done Done false
9564 3417 Acked Read 5 Sent Done Done false
9565 3418 Acked Read 3 Sent Done Done false
9566 3419 Acked Write 8 Sent Done Done false
9567 3420 Acked Read 9 Sent Done Done false
9568 3421 Acked Write 10 Sent Done Done false
9569 3422 Acked Write 10 Sent Done Done false
9570 3423 Acked Read 5 Sent Done Done false
9571 3424 Acked Read 7 Sent Done Done false
9572 3425 Acked Write 5 Sent Done Done false
9573 3426 Acked Write 7 Sent Done Done false
9574 3427 Acked Flush 0 Sent Done Done false
9575 3428 Acked Write 2 Sent Done Done false
9576 3429 Acked Read 6 Sent Done Done false
9577 3430 Acked Read 1 Sent Done Done false
9578 3431 Acked Read 8 Sent Done Done false
9579 3432 Acked Write 1 Sent Done Done false
9580 3433 Acked Write 3 Sent Done Done false
9581 3434 Acked Write 7 Sent Done Done false
9582 3435 Acked Write 8 Sent Done Done false
9583 3436 Acked Write 7 Sent Done Done false
9584 3437 Acked Write 9 Sent Done Done false
9585 3438 Acked Flush 0 Sent Done Done false
9586 3439 Acked Read 4 Sent Done Done false
9587 3440 Acked Write 2 Sent Done Done false
9588 3441 Acked Read 8 Sent Done Done false
9589 3442 Acked Flush 0 Sent Done Done false
9590 3443 Acked Read 8 Sent Done Done false
9591 3444 Acked Write 10 Sent Done Done false
9592 3445 Acked Read 6 Sent Done Done false
9593 3446 Acked Read 6 Sent Done Done false
9594 3447 Acked Read 8 Sent Done Done false
9595 3448 Acked Write 6 Sent Done Done false
9596 3449 Acked Read 4 Sent Done Done false
9597 3450 Acked Read 6 Sent Done Done false
9598 3451 Acked Read 9 Sent Done Done false
9599 3452 Acked Write 5 Sent Done Done false
9600 3453 Acked Read 8 Sent Done Done false
9601 3454 Acked Read 1 Sent Done Done false
9602 3455 Acked Write 4 Sent Done Done false
9603 3456 Acked Write 3 Sent Done Done false
9604 3457 Acked Write 6 Sent Done Done false
9605 3458 Acked Flush 0 Sent Done Done false
9606 3459 Acked Write 9 Sent Done Done false
9607 3460 Acked Read 4 Sent Done Done false
9608 3461 Acked Write 7 Sent Done Done false
9609 3462 Acked Read 10 Sent Done Done false
9610 3463 Acked Write 8 Sent Done Done false
9611 3464 Acked Read 3 Sent Done Done false
9612 3465 Acked Write 7 Sent Done Done false
9613 3466 Acked Write 9 Sent Done Done false
9614 3467 Acked Write 10 Sent Done Done false
9615 3468 Acked Write 5 Sent Done Done false
9616 3469 Acked Write 10 Sent Done Done false
9617 3470 Acked Write 8 Sent Done Done false
9618 3471 Acked Write 3 Sent Done Done false
9619 3472 Acked Write 1 Sent Done Done false
9620 3473 Acked Write 8 Sent Done Done false
9621 3474 Acked Read 7 Sent Done Done false
9622 3475 Acked Read 6 Sent Done Done false
9623 3476 Acked Flush 0 Sent Done Done false
9624 3477 Acked Flush 0 Sent Done Done false
9625 3478 Acked Write 4 Sent Done Done false
9626 3479 Acked Read 9 Sent Done Done false
9627 3480 Acked Write 9 Sent Done Done false
9628 3481 Acked Read 2 Sent Done Done false
9629 3482 Acked Read 4 Sent Done Done false
9630 3483 Acked Write 3 Sent Done Done false
9631 3484 Acked Write 10 Sent Done Done false
9632 3485 Acked Read 4 Sent Done Done false
9633 3486 Acked Flush 0 Sent Done Done false
9634 3487 Acked Read 8 Sent Done Done false
9635 3488 Acked Read 7 Sent Done Done false
9636 3489 Acked Write 8 Sent Done Done false
9637 3490 Acked Read 8 Sent Done Done false
9638 3491 Acked Read 7 Sent Done Done false
9639 3492 Acked Read 6 Sent Done Done false
9640 3493 Acked Read 7 Sent Done Done false
9641 3494 Acked Read 10 Sent Done Done false
9642 3495 Acked Read 5 Sent Done Done false
9643 3496 Acked Flush 0 Sent Done Done false
9644 3497 Acked Read 5 Sent Done Done false
9645 3498 Acked Read 1 Sent Done Done false
9646 3499 Acked Read 9 Sent Done Done false
9647 3500 Acked Read 6 Sent Done Done false
9648 3501 Acked Write 1 Sent Done Done false
9649 3502 Acked Read 5 Sent Done Done false
9650 3503 Acked Flush 0 Sent Done Done false
9651 3504 Acked Read 4 Sent Done Done false
9652 3505 Acked Write 9 Sent Done Done false
9653 3506 Acked Write 8 Sent Done Done false
9654 3507 Acked Read 8 Sent Done Done false
9655 3508 Acked Read 5 Sent Done Done false
9656 3509 Acked Write 8 Sent Done Done false
9657 3510 Acked Write 8 Sent Done Done false
9658 3511 Acked Write 9 Sent Done Done false
9659 3512 Acked Flush 0 Sent Done Done false
9660 3513 Acked Read 2 Sent Done Done false
9661 3514 Acked Read 9 Sent Done Done false
9662 3515 Acked Write 9 Sent Done Done false
9663 3516 Acked Read 2 Sent Done Done false
9664 3517 Acked Write 8 Sent Done Done false
9665 3518 Acked Read 8 Sent Done Done false
9666 3519 Acked Read 6 Sent Done Done false
9667 3520 Acked Read 4 Sent Done Done false
9668 3521 Acked Read 6 Sent Done Done false
9669 3522 Acked Write 4 Sent Done Done false
9670 3523 Acked Read 3 Sent Done Done false
9671 3524 Acked Write 8 Sent Done Done false
9672 3525 Acked Read 4 Sent Done Done false
9673 3526 Acked Read 7 Sent Done Done false
9674 3527 Acked Write 9 Sent Done Done false
9675 3528 Acked Read 5 Sent Done Done false
9676 3529 Acked Read 1 Sent Done Done false
9677 3530 Acked Read 3 Sent Done Done false
9678 3531 Acked Read 9 Sent Done Done false
9679 3532 Acked Write 10 Sent Done Done false
9680 3533 Acked Read 8 Sent Done Done false
9681 3534 Acked Read 9 Sent Done Done false
9682 3535 Acked Read 1 Sent Done Done false
9683 3536 Acked Write 7 Sent Done Done false
9684 3537 Acked Write 9 Sent Done Done false
9685 3538 Acked Read 4 Sent Done Done false
9686 3539 Acked Write 3 Sent Done Done false
9687 3540 Acked Read 4 Sent Done Done false
9688 3541 Acked Flush 0 Sent Done Done false
9689 3542 Acked Read 2 Sent Done Done false
9690 3543 Acked Flush 0 Sent Done Done false
9691 3544 Acked Write 2 Sent Done Done false
9692 3545 Acked Write 5 Sent Done Done false
9693 3546 Acked Read 4 Sent Done Done false
9694 3547 Acked Write 4 Sent Done Done false
9695 3548 Acked Write 8 Sent Done Done false
9696 3549 Acked Read 3 Sent Done Done false
9697 3550 Acked Read 5 Sent Done Done false
9698 3551 Acked Read 10 Sent Done Done false
9699 3552 Acked Write 7 Sent Done Done false
9700 3553 Acked Write 4 Sent Done Done false
9701 3554 Acked Write 8 Sent Done Done false
9702 3555 Acked Read 9 Sent Done Done false
9703 3556 Acked Read 3 Sent Done Done false
9704 3557 Acked Read 8 Sent Done Done false
9705 3558 Acked Flush 0 Sent Done Done false
9706 3559 Acked Read 3 Sent Done Done false
9707 3560 Acked Write 1 Sent Done Done false
9708 3561 Acked Flush 0 Sent Done Done false
9709 3562 Acked Read 3 Sent Done Done false
9710 3563 Acked Read 10 Sent Done Done false
9711 3564 Acked Flush 0 Sent Done Done false
9712 3565 Acked Write 9 Sent Done Done false
9713 3566 Acked Read 6 Sent Done Done false
9714 3567 Acked Read 4 Sent Done Done false
9715 3568 Acked Read 1 Sent Done Done false
9716 3569 Acked Read 5 Sent Done Done false
9717 3570 Acked Read 4 Sent Done Done false
9718 3571 Acked Read 5 Sent Done Done false
9719 3572 Acked Read 10 Sent Done Done false
9720 3573 Acked Write 5 Sent Done Done false
9721 3574 Acked Read 9 Sent Done Done false
9722 3575 Acked Read 6 Sent Done Done false
9723 3576 Acked Write 10 Sent Done Done false
9724 3577 Acked Read 3 Sent Done Done false
9725 3578 Acked Flush 0 Sent Done Done false
9726 3579 Acked Read 6 Sent Done Done false
9727 3580 Acked Write 9 Sent Done Done false
9728 3581 Acked Flush 0 Sent Done Done false
9729 3582 Acked Read 1 Sent Done Done false
9730 3583 Acked Write 10 Sent Done Done false
9731 3584 Acked Read 7 Sent Done Done false
9732 3585 Acked Flush 0 Sent Done Done false
9733 3586 Acked Write 5 Sent Done Done false
9734 3587 Acked Write 9 Sent Done Done false
9735 3588 Acked Read 6 Sent Done Done false
9736 3589 Acked Write 10 Sent Done Done false
9737 3590 Acked Write 9 Sent Done Done false
9738 3591 Acked Read 4 Sent Done Done false
9739 3592 Acked Flush 0 Sent Done Done false
9740 3593 Acked Read 4 Sent Done Done false
9741 3594 Acked Flush 0 Sent Done Done false
9742 3595 Acked Read 5 Sent Done Done false
9743 3596 Acked Write 10 Sent Done Done false
9744 3597 Acked Write 8 Sent Done Done false
9745 3598 Acked Read 4 Sent Done Done false
9746 3599 Acked Read 5 Sent Done Done false
9747 3600 Acked Write 7 Sent Done Done false
9748 3601 Acked Write 2 Sent Done Done false
9749 3602 Acked Read 10 Sent Done Done false
9750 3603 Acked Read 3 Sent Done Done false
9751 3604 Acked Read 8 Sent Done Done false
9752 3605 Acked Write 3 Sent Done Done false
9753 3606 Acked Write 5 Sent Done Done false
9754 3607 Acked Read 9 Sent Done Done false
9755 3608 Acked Write 8 Sent Done Done false
9756 3609 Acked Write 4 Sent Done Done false
9757 3610 Acked Read 6 Sent Done Done false
9758 3611 Acked Flush 0 Sent Done Done false
9759 3612 Acked Read 9 Sent Done Done false
9760 3613 Acked Write 2 Sent Done Done false
9761 3614 Acked Write 4 Sent Done Done false
9762 3615 Acked Read 8 Sent Done Done false
9763 3616 Acked Write 3 Sent Done Done false
9764 3617 Acked Read 10 Sent Done Done false
9765 3618 Acked Write 2 Sent Done Done false
9766 3619 Acked Read 9 Sent Done Done false
9767 3620 Acked Read 7 Sent Done Done false
9768 3621 Acked Flush 0 Sent Done Done false
9769 3622 Acked Read 8 Sent Done Done false
9770 3623 Acked Read 4 Sent Done Done false
9771 3624 Acked Write 9 Sent Done Done false
9772 3625 Acked Flush 0 Sent Done Done false
9773 3626 Acked Write 10 Sent Done Done false
9774 3627 Acked Flush 0 Sent Done Done false
9775 3628 Acked Read 3 Sent Done Done false
9776 3629 Acked Read 8 Sent Done Done false
9777 3630 Acked Write 9 Sent Done Done false
9778 3631 Acked Read 2 Sent Done Done false
9779 3632 Acked Read 6 Sent Done Done false
9780 3633 Acked Write 1 Sent Done Done false
9781 3634 Acked Write 4 Sent Done Done false
9782 3635 Acked Read 2 Sent Done Done false
9783 3636 Acked Read 8 Sent Done Done false
9784 3637 Acked Read 1 Sent Done Done false
9785 3638 Acked Write 5 Sent Done Done false
9786 3639 Acked Write 7 Sent Done Done false
9787 3640 Acked Flush 0 Sent Done Done false
9788 3641 Acked Read 10 Sent Done Done false
9789 3642 Acked Read 3 Sent Done Done false
9790 3643 Acked Write 10 Sent Done Done false
9791 3644 Acked Read 6 Sent Done Done false
9792 3645 Acked Write 7 Sent Done Done false
9793 3646 Acked Write 3 Sent Done Done false
9794 3647 Acked Read 7 Sent Done Done false
9795 3648 Acked Read 4 Sent Done Done false
9796 3649 Acked Read 7 Sent Done Done false
9797 3650 Acked Write 6 Sent Done Done false
9798 3651 Acked Read 7 Sent Done Done false
9799 3652 Acked Write 5 Sent Done Done false
9800 3653 Acked Read 2 Sent Done Done false
9801 3654 Acked Flush 0 Sent Done Done false
9802 3655 Acked Read 8 Sent Done Done false
9803 3656 Acked Read 10 Sent Done Done false
9804 3657 Acked Write 10 Sent Done Done false
9805 3658 Acked Flush 0 Sent Done Done false
9806 3659 Acked Read 9 Sent Done Done false
9807 3660 Acked Flush 0 Sent Done Done false
9808 3661 Acked Flush 0 Sent Done Done false
9809 3662 Acked Write 3 Sent Done Done false
9810 3663 Acked Read 10 Sent Done Done false
9811 3664 Acked Read 2 Sent Done Done false
9812 3665 Acked Write 1 Sent Done Done false
9813 3666 Acked Write 10 Sent Done Done false
9814 3667 Acked Read 10 Sent Done Done false
9815 3668 Acked Read 8 Sent Done Done false
9816 3669 Acked Read 8 Sent Done Done false
9817 3670 Acked Read 8 Sent Done Done false
9818 3671 Acked Write 2 Sent Done Done false
9819 3672 Acked Write 3 Sent Done Done false
9820 3673 Acked Read 5 Sent Done Done false
9821 3674 Acked Read 6 Sent Done Done false
9822 3675 Acked Read 9 Sent Done Done false
9823 3676 Acked Write 8 Sent Done Done false
9824 3677 Acked Write 2 Sent Done Done false
9825 3678 Acked Write 10 Sent Done Done false
9826 3679 Acked Write 9 Sent Done Done false
9827 3680 Acked Flush 0 Sent Done Done false
9828 3681 Acked Read 10 Sent Done Done false
9829 3682 Acked Write 3 Sent Done Done false
9830 3683 Acked Flush 0 Sent Done Done false
9831 3684 Acked Write 5 Sent Done Done false
9832 3685 Acked Write 3 Sent Done Done false
9833 3686 Acked Write 5 Sent Done Done false
9834 3687 Acked Write 10 Sent Done Done false
9835 3688 Acked Write 2 Sent Done Done false
9836 3689 Acked Write 5 Sent Done Done false
9837 3690 Acked Read 7 Sent Done Done false
9838 3691 Acked Write 10 Sent Done Done false
9839 3692 Acked Read 4 Sent Done Done false
9840 3693 Acked Read 4 Sent Done Done false
9841 3694 Acked Read 6 Sent Done Done false
9842 3695 Acked Write 9 Sent Done Done false
9843 3696 Acked Flush 0 Sent Done Done false
9844 3697 Acked Read 4 Sent Done Done false
9845 3698 Acked Read 8 Sent Done Done false
9846 3699 Acked Read 5 Sent Done Done false
9847 3700 Acked Write 6 Sent Done Done false
9848 3701 Acked Write 2 Sent Done Done false
9849 3702 Acked Read 4 Sent Done Done false
9850 3703 Acked Read 5 Sent Done Done false
9851 3704 Acked Read 7 Sent Done Done false
9852 3705 Acked Write 1 Sent Done Done false
9853 3706 Acked Flush 0 Sent Done Done false
9854 3707 Acked Read 6 Sent Done Done false
9855 3708 Acked Read 1 Sent Done Done false
9856 3709 Acked Read 1 Sent Done Done false
9857 3710 Acked Read 8 Sent Done Done false
9858 3711 Acked Read 1 Sent Done Done false
9859 3712 Acked Write 8 Sent Done Done false
9860 3713 Acked Read 6 Sent Done Done false
9861 3714 Acked Read 9 Sent Done Done false
9862 3715 Acked Write 7 Sent Done Done false
9863 3716 Acked Write 4 Sent Done Sent false
9864 STATES DS:0 DS:1 DS:2 TOTAL
9865 Sent 301 0 1 302
9866 Done 0 301 300 601
9867 Skipped 0 0 0 0
9868 Error 0 0 0 0
9869 Last Flush: 3415 3706 3706
9870 Downstairs last five completed: 3415 3414 3413 3412 3411
9871 Upstairs last five completed: 3716 3715 3714 3713 3712
9872 CLIENT: Up:0 ds:301 act:2
9873 [8810] got start action
9874 [8810] I am going to restart
9875 Starting downstairs at port 8810
9876 Make output file at "/tmp/dsc/downstairs-8810.txt"
9877 [8810][0] reports Starting
9878 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1211)
9879 [8810][0] reports Running
9880 ----------------------------------------------------------------
9881 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
9882 States: Offline Active Active
9883 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9884 3416 Acked Write 1 Sent Done Done false
9885 3417 Acked Read 5 Sent Done Done false
9886 3418 Acked Read 3 Sent Done Done false
9887 3419 Acked Write 8 Sent Done Done false
9888 3420 Acked Read 9 Sent Done Done false
9889 3421 Acked Write 10 Sent Done Done false
9890 3422 Acked Write 10 Sent Done Done false
9891 3423 Acked Read 5 Sent Done Done false
9892 3424 Acked Read 7 Sent Done Done false
9893 3425 Acked Write 5 Sent Done Done false
9894 3426 Acked Write 7 Sent Done Done false
9895 3427 Acked Flush 0 Sent Done Done false
9896 3428 Acked Write 2 Sent Done Done false
9897 3429 Acked Read 6 Sent Done Done false
9898 3430 Acked Read 1 Sent Done Done false
9899 3431 Acked Read 8 Sent Done Done false
9900 3432 Acked Write 1 Sent Done Done false
9901 3433 Acked Write 3 Sent Done Done false
9902 3434 Acked Write 7 Sent Done Done false
9903 3435 Acked Write 8 Sent Done Done false
9904 3436 Acked Write 7 Sent Done Done false
9905 3437 Acked Write 9 Sent Done Done false
9906 3438 Acked Flush 0 Sent Done Done false
9907 3439 Acked Read 4 Sent Done Done false
9908 3440 Acked Write 2 Sent Done Done false
9909 3441 Acked Read 8 Sent Done Done false
9910 3442 Acked Flush 0 Sent Done Done false
9911 3443 Acked Read 8 Sent Done Done false
9912 3444 Acked Write 10 Sent Done Done false
9913 3445 Acked Read 6 Sent Done Done false
9914 3446 Acked Read 6 Sent Done Done false
9915 3447 Acked Read 8 Sent Done Done false
9916 3448 Acked Write 6 Sent Done Done false
9917 3449 Acked Read 4 Sent Done Done false
9918 3450 Acked Read 6 Sent Done Done false
9919 3451 Acked Read 9 Sent Done Done false
9920 3452 Acked Write 5 Sent Done Done false
9921 3453 Acked Read 8 Sent Done Done false
9922 3454 Acked Read 1 Sent Done Done false
9923 3455 Acked Write 4 Sent Done Done false
9924 3456 Acked Write 3 Sent Done Done false
9925 3457 Acked Write 6 Sent Done Done false
9926 3458 Acked Flush 0 Sent Done Done false
9927 3459 Acked Write 9 Sent Done Done false
9928 3460 Acked Read 4 Sent Done Done false
9929 3461 Acked Write 7 Sent Done Done false
9930 3462 Acked Read 10 Sent Done Done false
9931 3463 Acked Write 8 Sent Done Done false
9932 3464 Acked Read 3 Sent Done Done false
9933 3465 Acked Write 7 Sent Done Done false
9934 3466 Acked Write 9 Sent Done Done false
9935 3467 Acked Write 10 Sent Done Done false
9936 3468 Acked Write 5 Sent Done Done false
9937 3469 Acked Write 10 Sent Done Done false
9938 3470 Acked Write 8 Sent Done Done false
9939 3471 Acked Write 3 Sent Done Done false
9940 3472 Acked Write 1 Sent Done Done false
9941 3473 Acked Write 8 Sent Done Done false
9942 3474 Acked Read 7 Sent Done Done false
9943 3475 Acked Read 6 Sent Done Done false
9944 3476 Acked Flush 0 Sent Done Done false
9945 3477 Acked Flush 0 Sent Done Done false
9946 3478 Acked Write 4 Sent Done Done false
9947 3479 Acked Read 9 Sent Done Done false
9948 3480 Acked Write 9 Sent Done Done false
9949 3481 Acked Read 2 Sent Done Done false
9950 3482 Acked Read 4 Sent Done Done false
9951 3483 Acked Write 3 Sent Done Done false
9952 3484 Acked Write 10 Sent Done Done false
9953 3485 Acked Read 4 Sent Done Done false
9954 3486 Acked Flush 0 Sent Done Done false
9955 3487 Acked Read 8 Sent Done Done false
9956 3488 Acked Read 7 Sent Done Done false
9957 3489 Acked Write 8 Sent Done Done false
9958 3490 Acked Read 8 Sent Done Done false
9959 3491 Acked Read 7 Sent Done Done false
9960 3492 Acked Read 6 Sent Done Done false
9961 3493 Acked Read 7 Sent Done Done false
9962 3494 Acked Read 10 Sent Done Done false
9963 3495 Acked Read 5 Sent Done Done false
9964 3496 Acked Flush 0 Sent Done Done false
9965 3497 Acked Read 5 Sent Done Done false
9966 3498 Acked Read 1 Sent Done Done false
9967 3499 Acked Read 9 Sent Done Done false
9968 3500 Acked Read 6 Sent Done Done false
9969 3501 Acked Write 1 Sent Done Done false
9970 3502 Acked Read 5 Sent Done Done false
9971 3503 Acked Flush 0 Sent Done Done false
9972 3504 Acked Read 4 Sent Done Done false
9973 3505 Acked Write 9 Sent Done Done false
9974 3506 Acked Write 8 Sent Done Done false
9975 3507 Acked Read 8 Sent Done Done false
9976 3508 Acked Read 5 Sent Done Done false
9977 3509 Acked Write 8 Sent Done Done false
9978 3510 Acked Write 8 Sent Done Done false
9979 3511 Acked Write 9 Sent Done Done false
9980 3512 Acked Flush 0 Sent Done Done false
9981 3513 Acked Read 2 Sent Done Done false
9982 3514 Acked Read 9 Sent Done Done false
9983 3515 Acked Write 9 Sent Done Done false
9984 3516 Acked Read 2 Sent Done Done false
9985 3517 Acked Write 8 Sent Done Done false
9986 3518 Acked Read 8 Sent Done Done false
9987 3519 Acked Read 6 Sent Done Done false
9988 3520 Acked Read 4 Sent Done Done false
9989 3521 Acked Read 6 Sent Done Done false
9990 3522 Acked Write 4 Sent Done Done false
9991 3523 Acked Read 3 Sent Done Done false
9992 3524 Acked Write 8 Sent Done Done false
9993 3525 Acked Read 4 Sent Done Done false
9994 3526 Acked Read 7 Sent Done Done false
9995 3527 Acked Write 9 Sent Done Done false
9996 3528 Acked Read 5 Sent Done Done false
9997 3529 Acked Read 1 Sent Done Done false
9998 3530 Acked Read 3 Sent Done Done false
9999 3531 Acked Read 9 Sent Done Done false
10000 3532 Acked Write 10 Sent Done Done false
10001 3533 Acked Read 8 Sent Done Done false
10002 3534 Acked Read 9 Sent Done Done false
10003 3535 Acked Read 1 Sent Done Done false
10004 3536 Acked Write 7 Sent Done Done false
10005 3537 Acked Write 9 Sent Done Done false
10006 3538 Acked Read 4 Sent Done Done false
10007 3539 Acked Write 3 Sent Done Done false
10008 3540 Acked Read 4 Sent Done Done false
10009 3541 Acked Flush 0 Sent Done Done false
10010 3542 Acked Read 2 Sent Done Done false
10011 3543 Acked Flush 0 Sent Done Done false
10012 3544 Acked Write 2 Sent Done Done false
10013 3545 Acked Write 5 Sent Done Done false
10014 3546 Acked Read 4 Sent Done Done false
10015 3547 Acked Write 4 Sent Done Done false
10016 3548 Acked Write 8 Sent Done Done false
10017 3549 Acked Read 3 Sent Done Done false
10018 3550 Acked Read 5 Sent Done Done false
10019 3551 Acked Read 10 Sent Done Done false
10020 3552 Acked Write 7 Sent Done Done false
10021 3553 Acked Write 4 Sent Done Done false
10022 3554 Acked Write 8 Sent Done Done false
10023 3555 Acked Read 9 Sent Done Done false
10024 3556 Acked Read 3 Sent Done Done false
10025 3557 Acked Read 8 Sent Done Done false
10026 3558 Acked Flush 0 Sent Done Done false
10027 3559 Acked Read 3 Sent Done Done false
10028 3560 Acked Write 1 Sent Done Done false
10029 3561 Acked Flush 0 Sent Done Done false
10030 3562 Acked Read 3 Sent Done Done false
10031 3563 Acked Read 10 Sent Done Done false
10032 3564 Acked Flush 0 Sent Done Done false
10033 3565 Acked Write 9 Sent Done Done false
10034 3566 Acked Read 6 Sent Done Done false
10035 3567 Acked Read 4 Sent Done Done false
10036 3568 Acked Read 1 Sent Done Done false
10037 3569 Acked Read 5 Sent Done Done false
10038 3570 Acked Read 4 Sent Done Done false
10039 3571 Acked Read 5 Sent Done Done false
10040 3572 Acked Read 10 Sent Done Done false
10041 3573 Acked Write 5 Sent Done Done false
10042 3574 Acked Read 9 Sent Done Done false
10043 3575 Acked Read 6 Sent Done Done false
10044 3576 Acked Write 10 Sent Done Done false
10045 3577 Acked Read 3 Sent Done Done false
10046 3578 Acked Flush 0 Sent Done Done false
10047 3579 Acked Read 6 Sent Done Done false
10048 3580 Acked Write 9 Sent Done Done false
10049 3581 Acked Flush 0 Sent Done Done false
10050 3582 Acked Read 1 Sent Done Done false
10051 3583 Acked Write 10 Sent Done Done false
10052 3584 Acked Read 7 Sent Done Done false
10053 3585 Acked Flush 0 Sent Done Done false
10054 3586 Acked Write 5 Sent Done Done false
10055 3587 Acked Write 9 Sent Done Done false
10056 3588 Acked Read 6 Sent Done Done false
10057 3589 Acked Write 10 Sent Done Done false
10058 3590 Acked Write 9 Sent Done Done false
10059 3591 Acked Read 4 Sent Done Done false
10060 3592 Acked Flush 0 Sent Done Done false
10061 3593 Acked Read 4 Sent Done Done false
10062 3594 Acked Flush 0 Sent Done Done false
10063 3595 Acked Read 5 Sent Done Done false
10064 3596 Acked Write 10 Sent Done Done false
10065 3597 Acked Write 8 Sent Done Done false
10066 3598 Acked Read 4 Sent Done Done false
10067 3599 Acked Read 5 Sent Done Done false
10068 3600 Acked Write 7 Sent Done Done false
10069 3601 Acked Write 2 Sent Done Done false
10070 3602 Acked Read 10 Sent Done Done false
10071 3603 Acked Read 3 Sent Done Done false
10072 3604 Acked Read 8 Sent Done Done false
10073 3605 Acked Write 3 Sent Done Done false
10074 3606 Acked Write 5 Sent Done Done false
10075 3607 Acked Read 9 Sent Done Done false
10076 3608 Acked Write 8 Sent Done Done false
10077 3609 Acked Write 4 Sent Done Done false
10078 3610 Acked Read 6 Sent Done Done false
10079 3611 Acked Flush 0 Sent Done Done false
10080 3612 Acked Read 9 Sent Done Done false
10081 3613 Acked Write 2 Sent Done Done false
10082 3614 Acked Write 4 Sent Done Done false
10083 3615 Acked Read 8 Sent Done Done false
10084 3616 Acked Write 3 Sent Done Done false
10085 3617 Acked Read 10 Sent Done Done false
10086 3618 Acked Write 2 Sent Done Done false
10087 3619 Acked Read 9 Sent Done Done false
10088 3620 Acked Read 7 Sent Done Done false
10089 3621 Acked Flush 0 Sent Done Done false
10090 3622 Acked Read 8 Sent Done Done false
10091 3623 Acked Read 4 Sent Done Done false
10092 3624 Acked Write 9 Sent Done Done false
10093 3625 Acked Flush 0 Sent Done Done false
10094 3626 Acked Write 10 Sent Done Done false
10095 3627 Acked Flush 0 Sent Done Done false
10096 3628 Acked Read 3 Sent Done Done false
10097 3629 Acked Read 8 Sent Done Done false
10098 3630 Acked Write 9 Sent Done Done false
10099 3631 Acked Read 2 Sent Done Done false
10100 3632 Acked Read 6 Sent Done Done false
10101 3633 Acked Write 1 Sent Done Done false
10102 3634 Acked Write 4 Sent Done Done false
10103 3635 Acked Read 2 Sent Done Done false
10104 3636 Acked Read 8 Sent Done Done false
10105 3637 Acked Read 1 Sent Done Done false
10106 3638 Acked Write 5 Sent Done Done false
10107 3639 Acked Write 7 Sent Done Done false
10108 3640 Acked Flush 0 Sent Done Done false
10109 3641 Acked Read 10 Sent Done Done false
10110 3642 Acked Read 3 Sent Done Done false
10111 3643 Acked Write 10 Sent Done Done false
10112 3644 Acked Read 6 Sent Done Done false
10113 3645 Acked Write 7 Sent Done Done false
10114 3646 Acked Write 3 Sent Done Done false
10115 3647 Acked Read 7 Sent Done Done false
10116 3648 Acked Read 4 Sent Done Done false
10117 3649 Acked Read 7 Sent Done Done false
10118 3650 Acked Write 6 Sent Done Done false
10119 3651 Acked Read 7 Sent Done Done false
10120 3652 Acked Write 5 Sent Done Done false
10121 3653 Acked Read 2 Sent Done Done false
10122 3654 Acked Flush 0 Sent Done Done false
10123 3655 Acked Read 8 Sent Done Done false
10124 3656 Acked Read 10 Sent Done Done false
10125 3657 Acked Write 10 Sent Done Done false
10126 3658 Acked Flush 0 Sent Done Done false
10127 3659 Acked Read 9 Sent Done Done false
10128 3660 Acked Flush 0 Sent Done Done false
10129 3661 Acked Flush 0 Sent Done Done false
10130 3662 Acked Write 3 Sent Done Done false
10131 3663 Acked Read 10 Sent Done Done false
10132 3664 Acked Read 2 Sent Done Done false
10133 3665 Acked Write 1 Sent Done Done false
10134 3666 Acked Write 10 Sent Done Done false
10135 3667 Acked Read 10 Sent Done Done false
10136 3668 Acked Read 8 Sent Done Done false
10137 3669 Acked Read 8 Sent Done Done false
10138 3670 Acked Read 8 Sent Done Done false
10139 3671 Acked Write 2 Sent Done Done false
10140 3672 Acked Write 3 Sent Done Done false
10141 3673 Acked Read 5 Sent Done Done false
10142 3674 Acked Read 6 Sent Done Done false
10143 3675 Acked Read 9 Sent Done Done false
10144 3676 Acked Write 8 Sent Done Done false
10145 3677 Acked Write 2 Sent Done Done false
10146 3678 Acked Write 10 Sent Done Done false
10147 3679 Acked Write 9 Sent Done Done false
10148 3680 Acked Flush 0 Sent Done Done false
10149 3681 Acked Read 10 Sent Done Done false
10150 3682 Acked Write 3 Sent Done Done false
10151 3683 Acked Flush 0 Sent Done Done false
10152 3684 Acked Write 5 Sent Done Done false
10153 3685 Acked Write 3 Sent Done Done false
10154 3686 Acked Write 5 Sent Done Done false
10155 3687 Acked Write 10 Sent Done Done false
10156 3688 Acked Write 2 Sent Done Done false
10157 3689 Acked Write 5 Sent Done Done false
10158 3690 Acked Read 7 Sent Done Done false
10159 3691 Acked Write 10 Sent Done Done false
10160 3692 Acked Read 4 Sent Done Done false
10161 3693 Acked Read 4 Sent Done Done false
10162 3694 Acked Read 6 Sent Done Done false
10163 3695 Acked Write 9 Sent Done Done false
10164 3696 Acked Flush 0 Sent Done Done false
10165 3697 Acked Read 4 Sent Done Done false
10166 3698 Acked Read 8 Sent Done Done false
10167 3699 Acked Read 5 Sent Done Done false
10168 3700 Acked Write 6 Sent Done Done false
10169 3701 Acked Write 2 Sent Done Done false
10170 3702 Acked Read 4 Sent Done Done false
10171 3703 Acked Read 5 Sent Done Done false
10172 3704 Acked Read 7 Sent Done Done false
10173 3705 Acked Write 1 Sent Done Done false
10174 3706 Acked Flush 0 Sent Done Done false
10175 3707 Acked Read 6 Sent Done Done false
10176 3708 Acked Read 1 Sent Done Done false
10177 3709 Acked Read 1 Sent Done Done false
10178 3710 Acked Read 8 Sent Done Done false
10179 3711 Acked Read 1 Sent Done Done false
10180 3712 Acked Write 8 Sent Done Done false
10181 3713 Acked Read 6 Sent Done Done false
10182 3714 Acked Read 9 Sent Done Done false
10183 3715 Acked Write 7 Sent Done Done false
10184 3716 Acked Write 4 Sent Done Done false
10185 3717 Acked Flush 0 Sent Done Done false
10186 STATES DS:0 DS:1 DS:2 TOTAL
10187 Sent 302 0 0 302
10188 Done 0 302 302 604
10189 Skipped 0 0 0 0
10190 Error 0 0 0 0
10191 Last Flush: 3415 3717 3717
10192 Downstairs last five completed: 3415 3414 3413 3412 3411
10193 Upstairs last five completed: 3717 3716 3715 3714 3713
10194 CLIENT: Up:0 ds:302 act:2
101952025-01-13T15:53:55.043ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
101962025-01-13T15:53:55.043ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64309) = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
101972025-01-13T15:53:55.045ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
101982025-01-13T15:53:55.045ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
101992025-01-13T15:53:55.045ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
102002025-01-13T15:53:55.045ZINFOcrucible: send last flush ID to this DS: 3415 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
102012025-01-13T15:53:55.046ZINFOcrucible: Replied this last flush ID: 3415 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
102022025-01-13T15:53:55.046ZINFOcrucible: [0] client re-new 302 jobs since flush 3415 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
102032025-01-13T15:53:55.046ZINFOcrucible: [0] Replayed 302 jobs since flush: 3415 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
10204 ----------------------------------------------------------------
10205 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
10206 Downstairs last five completed: 3717 3716 3715 3714 3713
10207 Upstairs last five completed: 3717 3716 3715 3714 3713
10208 CLIENT: Up:0 ds:0 act:3
10209 Replay: All jobs finished
10210 Main task has work to do, go find it
10211 got dsc Stop(2)
10212 stop 2
10213 Jan 13 15:53:57.300 INFO request completed[8830] Got stop action so:false kr:false
10214 , latency_us: 333, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 1c339f85-3ce1-446e-8a66-c38eb4525e75, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
102152025-01-13T15:53:57.301ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
102162025-01-13T15:53:57.301ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
102172025-01-13T15:53:57.301ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
10218 Jan 13 15:53:57.301 INFO request completed, latency_us: 119, response_code: 200, uri: /state/cid/2, method: GET, req_id: 4ed2187b-6779-4f20-b886-45d17e4c4817, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
102192025-01-13T15:53:57.301ZINFOcrucible: client task is exiting = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
10220 [8830] Exited with: ExitStatus(unix_wait_status(9))
10221 [8830][2] reports Exit
10222 Jan 13 15:54:01.304 INFO request completed, latency_us: 297, response_code: 200, uri: /state/cid/2, method: GET, req_id: b648edef-59bc-44ae-a273-19ceaba21f44, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
10223 001/300 Read block 2577 len 40960
10224 002/300 Write block 3927 len 32768 data: 3 3 3 3 2 2 2 2
10225 003/300 Read block 4610 len 24576
10226 004/300 Read block 1038 len 36864
10227 005/300 Write block 2952 len 28672 data: 3 3 3 3 2 2 2
10228 006/300 Read block 2003 len 16384
10229 007/300 Read block 4718 len 40960
10230 008/300 Read block 2515 len 24576
10231 009/300 Write block 4972 len 20480 data: 2 2 3 4 3
10232 010/300 Read block 2692 len 4096
10233 011/300 Write block 4454 len 36864 data: 3 3 3 2 2 2 2 3 3
10234 012/300 Read block 3702 len 12288
10235 013/300 Write block 4585 len 4096 data: 2
10236 014/300 Read block 4450 len 8192
10237 015/300 Write block 1818 len 40960 data: 5 5 3 3 2 4 4 4 4 4
10238 016/300 Read block 244 len 28672
10239 017/300 Read block 2574 len 8192
10240 018/300 Read block 2311 len 12288
10241 019/300 Read block 1522 len 36864
10242 020/300 Flush
10243 021/300 Read block 2825 len 16384
10244 022/300 Read block 4749 len 36864
10245 023/300 Write block 2946 len 40960 data: 4 4 3 4 3 3 4 4 4 4
10246 024/300 Write block 4118 len 16384 data: 3 2 2 2
10247 025/300 Read block 4350 len 28672
10248 026/300 Read block 4078 len 32768
10249 027/300 Write block 1316 len 4096 data: 3
10250 028/300 Read block 1577 len 12288
10251 029/300 Write block 2082 len 8192 data: 4 5
10252 030/300 Flush
10253 031/300 Read block 258 len 36864
10254 032/300 Read block 3869 len 4096
10255 033/300 Write block 1517 len 32768 data: 5 5 3 3 3 4 4 4
10256 034/300 Write block 531 len 4096 data: 4
10257 035/300 Read block 1455 len 4096
10258 036/300 Write block 4334 len 12288 data: 3 3 3
10259 037/300 Write block 3261 len 20480 data: 2 4 4 5 5
10260 038/300 Write block 2794 len 20480 data: 4 3 3 3 4
10261 039/300 Read block 3723 len 24576
10262 040/300 Flush
10263 041/300 Read block 66 len 40960
10264 042/300 Write block 587 len 12288 data: 2 3 2
10265 043/300 Read block 1305 len 36864
10266 044/300 Read block 4385 len 40960
10267 045/300 Read block 3325 len 8192
10268 046/300 Flush
10269 047/300 Read block 1025 len 28672
10270 048/300 Write block 205 len 36864 data: 6 5 4 3 3 3 4 4 4
10271 049/300 Write block 3301 len 8192 data: 2 2
10272 050/300 Read block 1285 len 36864
10273 051/300 Read block 645 len 28672
10274 052/300 Flush
10275 053/300 Write block 1437 len 24576 data: 4 2 2 4 4 5
10276 054/300 Write block 3655 len 36864 data: 2 2 3 4 4 4 4 3 2
10277 055/300 Write block 2022 len 36864 data: 5 6 7 6 8 8 7 6 5
10278 056/300 Read block 1715 len 20480
10279 057/300 Read block 2039 len 40960
10280 058/300 Write block 2776 len 8192 data: 2 3
10281 059/300 Write block 1425 len 36864 data: 3 5 5 6 6 5 4 4 4
10282 060/300 Write block 958 len 24576 data: 2 3 4 5 5 4
10283 061/300 Write block 2129 len 4096 data: 2
10284 062/300 Read block 3069 len 16384
10285 063/300 Read block 4670 len 28672
10286 064/300 Write block 4157 len 8192 data: 3 3
10287 065/300 Read block 4113 len 8192
10288 066/300 Flush
10289 067/300 Flush
10290 068/300 Read block 2301 len 4096
10291 069/300 Write block 1060 len 20480 data: 4 4 3 3 2
10292 070/300 Write block 2626 len 4096 data: 3
10293 071/300 Read block 4710 len 36864
10294 072/300 Write block 2193 len 32768 data: 4 3 3 3 4 4 3 3
10295 073/300 Read block 2274 len 36864
10296 074/300 Write block 3873 len 8192 data: 4 3
10297 075/300 Flush
10298 076/300 Write block 4917 len 20480 data: 2 2 2 2 2
10299 077/300 Write block 3189 len 16384 data: 6 6 6 5
10300 078/300 Read block 4945 len 28672
10301 079/300 Write block 1538 len 4096 data: 3
10302 080/300 Write block 2752 len 8192 data: 4 3
10303 081/300 Write block 4784 len 28672 data: 5 4 4 3 3 3 3
10304 082/300 Read block 595 len 4096
10305 083/300 Read block 3004 len 32768
10306 084/300 Read block 2819 len 20480
10307 085/300 Read block 2543 len 24576
10308 086/300 Write block 2979 len 24576 data: 2 2 3 3 3 3
10309 087/300 Write block 1899 len 12288 data: 4 3 3
10310 088/300 Read block 3875 len 20480
10311 089/300 Read block 463 len 32768
10312 090/300 Write block 4109 len 40960 data: 6 6 6 6 5 4 3 4 4 4
10313 091/300 Flush
10314 092/300 Read block 3778 len 32768
10315 093/300 Write block 3829 len 28672 data: 5 5 5 5 4 2 3
10316 094/300 Read block 4314 len 40960
10317 095/300 Read block 3980 len 40960
10318 096/300 Read block 3586 len 20480
10319 097/300 Read block 4369 len 12288
10320 098/300 Read block 2990 len 32768
10321 099/300 Write block 211 len 16384 data: 5 5 5 5
10322 100/300 Write block 4465 len 16384 data: 3 3 3 4
10323 101/300 Write block 1761 len 4096 data: 4
10324 102/300 Read block 1498 len 40960
10325 103/300 Write block 1419 len 4096 data: 4
10326 104/300 Flush
10327 105/300 Read block 4410 len 24576
10328 106/300 Flush
10329 107/300 Write block 4042 len 8192 data: 2 2
10330 108/300 Read block 1552 len 24576
10331 109/300 Write block 2584 len 4096 data: 2
10332 110/300 Write block 102 len 32768 data: 4 4 4 4 3 2 3 3
10333 111/300 Write block 1549 len 4096 data: 2
10334 112/300 Write block 3995 len 24576 data: 3 3 4 3 3 3
10335 113/300 Read block 2927 len 8192
10336 114/300 Read block 24 len 24576
10337 115/300 Write block 801 len 28672 data: 3 4 3 3 3 3 2
10338 116/300 Flush
10339 117/300 Read block 2501 len 24576
10340 118/300 Write block 4145 len 32768 data: 2 2 2 2 2 2 2 2
10341 119/300 Read block 3818 len 24576
10342 120/300 Write block 888 len 36864 data: 2 2 2 2 2 2 2 2 2
10343 121/300 Read block 1851 len 20480
10344 122/300 Write block 1123 len 12288 data: 2 2 2
10345 123/300 Flush
10346 124/300 Write block 3743 len 36864 data: 2 2 2 2 2 2 2 2 2
10347 125/300 Read block 2122 len 36864
10348 126/300 Write block 2268 len 16384 data: 5 5 4 6
10349 127/300 Write block 4426 len 20480 data: 3 3 3 3 3
10350 128/300 Write block 696 len 32768 data: 3 3 3 3 3 3 3 3
10351 129/300 Write block 4679 len 40960 data: 2 2 4 3 3 5 4 3 3 3
10352 130/300 Write block 49 len 12288 data: 2 2 2
10353 131/300 Write block 2184 len 40960 data: 3 4 4 4 4 5 5 3 3 5
10354 132/300 Write block 238 len 20480 data: 2 2 3 3 2
10355 133/300 Write block 1729 len 32768 data: 4 4 4 4 4 2 2 2
10356 134/300 Flush
10357 135/300 Write block 4674 len 8192 data: 3 2
10358 136/300 Read block 2232 len 28672
10359 137/300 Read block 592 len 36864
10360 138/300 Read block 3026 len 36864
10361 139/300 Read block 677 len 28672
10362 140/300 Read block 510 len 36864
10363 141/300 Read block 2857 len 32768
10364 142/300 Write block 3702 len 8192 data: 3 2
10365 143/300 Flush
10366 144/300 Write block 1090 len 8192 data: 3 2
10367 145/300 Write block 2048 len 36864 data: 5 5 5 5 4 3 3 2 2
10368 146/300 Read block 3722 len 36864
10369 147/300 Write block 2590 len 28672 data: 4 6 7 8 8 9 9
10370 148/300 Read block 3855 len 24576
10371 149/300 Read block 422 len 36864
10372 150/300 Read block 142 len 16384
10373 151/300 Write block 3275 len 28672 data: 6 5 5 4 4 3 3
10374 152/300 Write block 3558 len 32768 data: 2 3 3 3 3 3 3 3
10375 153/300 Read block 4404 len 4096
10376 154/300 Flush
10377 155/300 Read block 3530 len 32768
10378 156/300 Write block 714 len 40960 data: 2 2 2 2 2 2 2 2 2 3
10379 157/300 Write block 4586 len 4096 data: 2
10380 158/300 Flush
10381 159/300 Read block 1812 len 40960
10382 160/300 Write block 4029 len 16384 data: 3 2 2 2
10383 161/300 Write block 3854 len 4096 data: 2
10384 162/300 Write block 2737 len 8192 data: 3 3
10385 163/300 Read block 817 len 28672
10386 164/300 Read block 4263 len 8192
10387 165/300 Write block 3226 len 40960 data: 6 6 5 4 4 4 4 3 3 3
10388 166/300 Flush
10389 167/300 Write block 4185 len 32768 data: 3 2 3 3 3 3 3 2
10390 168/300 Read block 283 len 32768
10391 169/300 Read block 4608 len 20480
10392 170/300 Read block 4188 len 28672
10393 171/300 Write block 712 len 32768 data: 2 2 3 3 3 3 3 3
10394 172/300 Read block 1670 len 24576
10395 173/300 Write block 941 len 24576 data: 3 5 5 6 6 6
10396 174/300 Write block 2210 len 28672 data: 5 4 4 4 6 6 7
10397 175/300 Write block 3568 len 20480 data: 3 3 3 2 2
10398 176/300 Write block 3669 len 16384 data: 5 3 3 3
10399 177/300 Write block 2558 len 32768 data: 4 3 3 3 3 3 2 2
10400 178/300 Read block 4497 len 28672
10401 179/300 Flush
10402 180/300 Read block 4275 len 16384
10403 181/300 Read block 1173 len 32768
10404 182/300 Read block 1511 len 4096
10405 183/300 Read block 847 len 8192
10406 184/300 Flush
10407 185/300 Write block 4928 len 28672 data: 4 5 6 8 8 7 7
10408 186/300 Write block 3019 len 16384 data: 3 4 4 4
10409 187/300 Flush
10410 188/300 Write block 3020 len 36864 data: 5 5 5 4 6 4 3 4 4
10411 189/300 Flush
10412 190/300 Write block 2708 len 8192 data: 3 2
10413 191/300 Read block 3238 len 24576
10414 192/300 Flush
10415 193/300 Write block 1487 len 20480 data: 3 3 3 4 4
10416 194/300 Write block 4397 len 24576 data: 4 4 4 3 2 2
10417 195/300 Read block 3583 len 24576
10418 196/300 Read block 4697 len 12288
10419 197/300 Read block 311 len 16384
10420 198/300 Read block 374 len 12288
10421 199/300 Read block 1756 len 4096
10422 200/300 Read block 2147 len 16384
10423 201/300 Read block 3776 len 28672
10424 202/300 Write block 4603 len 20480 data: 4 4 4 3 3
10425 203/300 Flush
10426 204/300 Write block 1385 len 32768 data: 6 6 5 4 3 3 2 2
10427 205/300 Read block 1051 len 12288
10428 206/300 Write block 1566 len 8192 data: 3 2
10429 207/300 Read block 3097 len 32768
10430 208/300 Read block 837 len 32768
10431 209/300 Flush
10432 210/300 Write block 2600 len 36864 data: 2 2 3 3 3 3 3 3 3
10433 211/300 Flush
10434 212/300 Read block 4749 len 24576
10435 213/300 Write block 435 len 24576 data: 4 3 3 3 3 2
10436 214/300 Write block 3954 len 4096 data: 3
10437 215/300 Read block 854 len 12288
10438 216/300 Read block 4137 len 4096
10439 217/300 Flush
10440 218/300 Write block 2050 len 20480 data: 6 6 5 4 4
10441 219/300 Write block 2134 len 8192 data: 2 2
10442 220/300 Write block 4529 len 36864 data: 2 2 3 4 3 3 3 3 2
10443 221/300 Read block 727 len 40960
10444 222/300 Write block 4688 len 32768 data: 4 2 2 2 3 4 4 4
10445 223/300 Read block 451 len 36864
10446 224/300 Write block 3749 len 4096 data: 3
10447 225/300 Read block 4484 len 8192
10448 226/300 Read block 4379 len 40960
10449 227/300 Read block 4570 len 8192
10450 228/300 Read block 2556 len 4096
10451 229/300 Read block 481 len 20480
10452 230/300 Read block 3976 len 8192
10453 231/300 Flush
10454 232/300 Read block 1877 len 20480
10455 233/300 Read block 11 len 12288
10456 234/300 Write block 3399 len 12288 data: 3 4 5
10457 235/300 Write block 1141 len 28672 data: 2 3 4 4 4 4 4
10458 236/300 Read block 4036 len 16384
10459 237/300 Write block 1666 len 32768 data: 3 3 5 4 3 3 4 3
10460 238/300 Write block 4356 len 16384 data: 2 2 2 2
10461 239/300 Write block 423 len 24576 data: 2 2 2 2 2 3
10462 240/300 Read block 2909 len 20480
10463 241/300 Write block 3686 len 12288 data: 2 2 3
10464 242/300 Write block 2672 len 40960 data: 2 2 2 2 2 3 3 4 3 2
10465 243/300 Read block 4017 len 20480
10466 244/300 Read block 1132 len 24576
10467 245/300 Read block 2419 len 28672
10468 246/300 Read block 2314 len 16384
10469 247/300 Read block 182 len 28672
10470 248/300 Read block 4544 len 36864
10471 249/300 Write block 2166 len 40960 data: 2 2 2 2 2 6 6 6 5 5
10472 250/300 Read block 2205 len 20480
10473 251/300 Read block 2541 len 16384
10474 252/300 Read block 4583 len 28672
10475 253/300 Read block 35 len 12288
10476 254/300 Flush
10477 255/300 Write block 2396 len 36864 data: 5 3 3 2 2 2 2 2 2
10478 256/300 Write block 1412 len 24576 data: 3 2 2 3 3 3
10479 257/300 Read block 3117 len 28672
10480 258/300 Read block 712 len 32768
10481 259/300 Read block 1859 len 24576
10482 260/300 Write block 2761 len 36864 data: 2 2 3 3 2 3 4 4 4
10483 261/300 Read block 2773 len 16384
10484 262/300 Write block 684 len 20480 data: 2 2 2 2 2
10485 263/300 Read block 3906 len 40960
10486 264/300 Write block 3502 len 32768 data: 4 4 4 4 3 2 2 3
10487 265/300 Flush
10488 266/300 Write block 1963 len 28672 data: 2 2 2 3 3 4 4
10489 267/300 Read block 1732 len 12288
10490 268/300 Write block 2411 len 32768 data: 2 3 4 3 3 3 3 3
10491 269/300 Read block 892 len 36864
10492 270/300 Write block 1473 len 8192 data: 2 2
10493 271/300 Read block 3110 len 16384
10494 272/300 Read block 4116 len 24576
10495 273/300 Read block 532 len 32768
10496 274/300 Read block 3138 len 12288
10497 275/300 Write block 2375 len 16384 data: 2 2 2 2
10498 276/300 Write block 1226 len 24576 data: 3 2 2 2 2 2
10499 277/300 Write block 3475 len 4096 data: 3
10500 278/300 Read block 4473 len 4096
10501 279/300 Read block 2144 len 36864
10502 280/300 Read block 2353 len 8192
10503 281/300 Read block 3941 len 40960
10504 282/300 Write block 1420 len 24576 data: 4 4 4 3 4 4
10505 283/300 Write block 3670 len 8192 data: 4 4
10506 284/300 Write block 3984 len 12288 data: 5 4 4
10507 285/300 Write block 4163 len 28672 data: 3 4 3 2 2 2 2
10508 286/300 Read block 1208 len 20480
10509 287/300 Read block 1398 len 12288
10510 288/300 Read block 1964 len 24576
10511 289/300 Write block 140 len 20480 data: 2 3 5 4 5
10512 290/300 Write block 1453 len 8192 data: 2 2
10513 291/300 Read block 2806 len 8192
10514 292/300 Read block 4762 len 36864
10515 293/300 Read block 2278 len 32768
10516 294/300 Read block 3912 len 32768
10517 295/300 Write block 1408 len 32768 data: 2 3 3 3 4 3 3 4
10518 296/300 Write block 1360 len 40960 data: 3 2 2 2 2 2 2 2 3 3
10519 297/300 Write block 3336 len 16384 data: 2 2 2 2
10520 298/300 Write block 2339 len 32768 data: 2 2 4 5 5 5 5 4
10521 299/300 Read block 2542 len 12288
10522 300/300 Flush
10523 301/300 Flush
10524 Main task has work to do, go find it
10525 got dsc Start(2)
10526 start 2
10527 Jan 13 15:54:01.543 INFO request completed, latency_us: 188, response_code: 204, uri: /start/cid/2, method: POST, req_id: 88c65c5d-a800-4bc1-8779-7a84ba306a1e, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
10528 Replay: started 2, returned:Ok(())
10529 ----------------------------------------------------------------
10530 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
10531 States: Active Active Offline
10532 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10533 3718 Acked Read 10 Done Done Sent false
10534 3719 Acked Write 8 Done Done Sent false
10535 3720 Acked Read 6 Done Done Sent false
10536 3721 Acked Read 9 Done Done Sent false
10537 3722 Acked Write 7 Done Done Sent false
10538 3723 Acked Read 4 Done Done Sent false
10539 3724 Acked Read 10 Done Done Sent false
10540 3725 Acked Read 6 Done Done Sent false
10541 3726 Acked Write 5 Done Done Sent false
10542 3727 Acked Read 1 Done Done Sent false
10543 3728 Acked Write 9 Done Done Sent false
10544 3729 Acked Read 3 Done Done Sent false
10545 3730 Acked Write 1 Done Done Sent false
10546 3731 Acked Read 2 Done Done Sent false
10547 3732 Acked Write 10 Done Done Sent false
10548 3733 Acked Read 7 Done Done Sent false
10549 3734 Acked Read 2 Done Done Sent false
10550 3735 Acked Read 3 Done Done Sent false
10551 3736 Acked Read 9 Done Done Sent false
10552 3737 Acked Flush 0 Done Done Sent false
10553 3738 Acked Read 4 Done Done Sent false
10554 3739 Acked Read 9 Done Done Sent false
10555 3740 Acked Write 10 Done Done Sent false
10556 3741 Acked Write 4 Done Done Sent false
10557 3742 Acked Read 7 Done Done Sent false
10558 3743 Acked Read 8 Done Done Sent false
10559 3744 Acked Write 1 Done Done Sent false
10560 3745 Acked Read 3 Done Done Sent false
10561 3746 Acked Write 2 Done Done Sent false
10562 3747 Acked Flush 0 Done Done Sent false
10563 3748 Acked Read 9 Done Done Sent false
10564 3749 Acked Read 1 Done Done Sent false
10565 3750 Acked Write 8 Done Done Sent false
10566 3751 Acked Write 1 Done Done Sent false
10567 3752 Acked Read 1 Done Done Sent false
10568 3753 Acked Write 3 Done Done Sent false
10569 3754 Acked Write 5 Done Done Sent false
10570 3755 Acked Write 5 Done Done Sent false
10571 3756 Acked Read 6 Done Done Sent false
10572 3757 Acked Flush 0 Done Done Sent false
10573 3758 Acked Read 10 Done Done Sent false
10574 3759 Acked Write 3 Done Done Sent false
10575 3760 Acked Read 9 Done Done Sent false
10576 3761 Acked Read 10 Done Done Sent false
10577 3762 Acked Read 2 Done Done Sent false
10578 3763 Acked Flush 0 Done Done Sent false
10579 3764 Acked Read 7 Done Done Sent false
10580 3765 Acked Write 9 Done Done Sent false
10581 3766 Acked Write 2 Done Done Sent false
10582 3767 Acked Read 9 Done Done Sent false
10583 3768 Acked Read 7 Done Done Sent false
10584 3769 Acked Flush 0 Done Done Sent false
10585 3770 Acked Write 6 Done Done Sent false
10586 3771 Acked Write 9 Done Done Sent false
10587 3772 Acked Write 9 Done Done Sent false
10588 3773 Acked Read 5 Done Done Sent false
10589 3774 Acked Read 10 Done Done Sent false
10590 3775 Acked Write 2 Done Done Sent false
10591 3776 Acked Write 9 Done Done Sent false
10592 3777 Acked Write 6 Done Done Sent false
10593 3778 Acked Write 1 Done Done Sent false
10594 3779 Acked Read 4 Done Done Sent false
10595 3780 Acked Read 7 Done Done Sent false
10596 3781 Acked Write 2 Done Done Sent false
10597 3782 Acked Read 2 Done Done Sent false
10598 3783 Acked Flush 0 Done Done Sent false
10599 3784 Acked Flush 0 Done Done Sent false
10600 3785 Acked Read 1 Done Done Sent false
10601 3786 Acked Write 5 Done Done Sent false
10602 3787 Acked Write 1 Done Done Sent false
10603 3788 Acked Read 9 Done Done Sent false
10604 3789 Acked Write 8 Done Done Sent false
10605 3790 Acked Read 9 Done Done Sent false
10606 3791 Acked Write 2 Done Done Sent false
10607 3792 Acked Flush 0 Done Done Sent false
10608 3793 Acked Write 5 Done Done Sent false
10609 3794 Acked Write 4 Done Done Sent false
10610 3795 Acked Read 7 Done Done Sent false
10611 3796 Acked Write 1 Done Done Sent false
10612 3797 Acked Write 2 Done Done Sent false
10613 3798 Acked Write 7 Done Done Sent false
10614 3799 Acked Read 1 Done Done Sent false
10615 3800 Acked Read 8 Done Done Sent false
10616 3801 Acked Read 5 Done Done Sent false
10617 3802 Acked Read 6 Done Done Sent false
10618 3803 Acked Write 6 Done Done Sent false
10619 3804 Acked Write 3 Done Done Sent false
10620 3805 Acked Read 5 Done Done Sent false
10621 3806 Acked Read 8 Done Done Sent false
10622 3807 Acked Write 10 Done Done Sent false
10623 3808 Acked Flush 0 Done Done Sent false
10624 3809 Acked Read 8 Done Done Sent false
10625 3810 Acked Write 7 Done Done Sent false
10626 3811 Acked Read 10 Done Done Sent false
10627 3812 Acked Read 10 Done Done Sent false
10628 3813 Acked Read 5 Done Done Sent false
10629 3814 Acked Read 3 Done Done Sent false
10630 3815 Acked Read 8 Done Done Sent false
10631 3816 Acked Write 4 Done Done Sent false
10632 3817 Acked Write 4 Done Done Sent false
10633 3818 Acked Write 1 Done Done Sent false
10634 3819 Acked Read 10 Done Done Sent false
10635 3820 Acked Write 1 Done Done Sent false
10636 3821 Acked Flush 0 Done Done Sent false
10637 3822 Acked Read 6 Done Done Sent false
10638 3823 Acked Flush 0 Done Done Sent false
10639 3824 Acked Write 2 Done Done Sent false
10640 3825 Acked Read 6 Done Done Sent false
10641 3826 Acked Write 1 Done Done Sent false
10642 3827 Acked Write 8 Done Done Sent false
10643 3828 Acked Write 1 Done Done Sent false
10644 3829 Acked Write 6 Done Done Sent false
10645 3830 Acked Read 2 Done Done Sent false
10646 3831 Acked Read 6 Done Done Sent false
10647 3832 Acked Write 7 Done Done Sent false
10648 3833 Acked Flush 0 Done Done Sent false
10649 3834 Acked Read 6 Done Done Sent false
10650 3835 Acked Write 8 Done Done Sent false
10651 3836 Acked Read 6 Done Done Sent false
10652 3837 Acked Write 9 Done Done Sent false
10653 3838 Acked Read 5 Done Done Sent false
10654 3839 Acked Write 3 Done Done Sent false
10655 3840 Acked Flush 0 Done Done Sent false
10656 3841 Acked Write 9 Done Done Sent false
10657 3842 Acked Read 9 Done Done Sent false
10658 3843 Acked Write 4 Done Done Sent false
10659 3844 Acked Write 5 Done Done Sent false
10660 3845 Acked Write 8 Done Done Sent false
10661 3846 Acked Write 10 Done Done Sent false
10662 3847 Acked Write 3 Done Done Sent false
10663 3848 Acked Write 10 Done Done Sent false
10664 3849 Acked Write 5 Done Done Sent false
10665 3850 Acked Write 8 Done Done Sent false
10666 3851 Acked Flush 0 Done Done Sent false
10667 3852 Acked Write 2 Done Done Sent false
10668 3853 Acked Read 7 Done Done Sent false
10669 3854 Acked Read 9 Done Done Sent false
10670 3855 Acked Read 9 Done Done Sent false
10671 3856 Acked Read 7 Done Done Sent false
10672 3857 Acked Read 9 Done Done Sent false
10673 3858 Acked Read 8 Done Done Sent false
10674 3859 Acked Write 2 Done Done Sent false
10675 3860 Acked Flush 0 Done Done Sent false
10676 3861 Acked Write 2 Done Done Sent false
10677 3862 Acked Write 9 Done Done Sent false
10678 3863 Acked Read 9 Done Done Sent false
10679 3864 Acked Write 7 Done Done Sent false
10680 3865 Acked Read 6 Done Done Sent false
10681 3866 Acked Read 9 Done Done Sent false
10682 3867 Acked Read 4 Done Done Sent false
10683 3868 Acked Write 7 Done Done Sent false
10684 3869 Acked Write 8 Done Done Sent false
10685 3870 Acked Read 1 Done Done Sent false
10686 3871 Acked Flush 0 Done Done Sent false
10687 3872 Acked Read 8 Done Done Sent false
10688 3873 Acked Write 10 Done Done Sent false
10689 3874 Acked Write 1 Done Done Sent false
10690 3875 Acked Flush 0 Done Done Sent false
10691 3876 Acked Read 10 Done Done Sent false
10692 3877 Acked Write 4 Done Done Sent false
10693 3878 Acked Write 1 Done Done Sent false
10694 3879 Acked Write 2 Done Done Sent false
10695 3880 Acked Read 7 Done Done Sent false
10696 3881 Acked Read 2 Done Done Sent false
10697 3882 Acked Write 10 Done Done Sent false
10698 3883 Acked Flush 0 Done Done Sent false
10699 3884 Acked Write 8 Done Done Sent false
10700 3885 Acked Read 8 Done Done Sent false
10701 3886 Acked Read 5 Done Done Sent false
10702 3887 Acked Read 7 Done Done Sent false
10703 3888 Acked Write 8 Done Done Sent false
10704 3889 Acked Read 6 Done Done Sent false
10705 3890 Acked Write 6 Done Done Sent false
10706 3891 Acked Write 7 Done Done Sent false
10707 3892 Acked Write 5 Done Done Sent false
10708 3893 Acked Write 4 Done Done Sent false
10709 3894 Acked Write 8 Done Done Sent false
10710 3895 Acked Read 7 Done Done Sent false
10711 3896 Acked Flush 0 Done Done Sent false
10712 3897 Acked Read 4 Done Done Sent false
10713 3898 Acked Read 8 Done Done Sent false
10714 3899 Acked Read 1 Done Done Sent false
10715 3900 Acked Read 2 Done Done Sent false
10716 3901 Acked Flush 0 Done Done Sent false
10717 3902 Acked Write 7 Done Done Sent false
10718 3903 Acked Write 4 Done Done Sent false
10719 3904 Acked Flush 0 Done Done Sent false
10720 3905 Acked Write 9 Done Done Sent false
10721 3906 Acked Flush 0 Done Done Sent false
10722 3907 Acked Write 2 Done Done Sent false
10723 3908 Acked Read 6 Done Done Sent false
10724 3909 Acked Flush 0 Done Done Sent false
10725 3910 Acked Write 5 Done Done Sent false
10726 3911 Acked Write 6 Done Done Sent false
10727 3912 Acked Read 6 Done Done Sent false
10728 3913 Acked Read 3 Done Done Sent false
10729 3914 Acked Read 4 Done Done Sent false
10730 3915 Acked Read 3 Done Done Sent false
10731 3916 Acked Read 1 Done Done Sent false
10732 3917 Acked Read 4 Done Done Sent false
10733 3918 Acked Read 7 Done Done Sent false
10734 3919 Acked Write 5 Done Done Sent false
10735 3920 Acked Flush 0 Done Done Sent false
10736 3921 Acked Write 8 Done Done Sent false
10737 3922 Acked Read 3 Done Done Sent false
10738 3923 Acked Write 2 Done Done Sent false
10739 3924 Acked Read 8 Done Done Sent false
10740 3925 Acked Read 8 Done Done Sent false
10741 3926 Acked Flush 0 Done Done Sent false
10742 3927 Acked Write 9 Done Done Sent false
10743 3928 Acked Flush 0 Done Done Sent false
10744 3929 Acked Read 6 Done Done Sent false
10745 3930 Acked Write 6 Done Done Sent false
10746 3931 Acked Write 1 Done Done Sent false
10747 3932 Acked Read 3 Done Done Sent false
10748 3933 Acked Read 1 Done Done Sent false
10749 3934 Acked Flush 0 Done Done Sent false
10750 3935 Acked Write 5 Done Done Sent false
10751 3936 Acked Write 2 Done Done Sent false
10752 3937 Acked Write 9 Done Done Sent false
10753 3938 Acked Read 10 Done Done Sent false
10754 3939 Acked Write 8 Done Done Sent false
10755 3940 Acked Read 9 Done Done Sent false
10756 3941 Acked Write 1 Done Done Sent false
10757 3942 Acked Read 2 Done Done Sent false
10758 3943 Acked Read 10 Done Done Sent false
10759 3944 Acked Read 2 Done Done Sent false
10760 3945 Acked Read 1 Done Done Sent false
10761 3946 Acked Read 5 Done Done Sent false
10762 3947 Acked Read 2 Done Done Sent false
10763 3948 Acked Flush 0 Done Done Sent false
10764 3949 Acked Read 5 Done Done Sent false
10765 3950 Acked Read 3 Done Done Sent false
10766 3951 Acked Write 3 Done Done Sent false
10767 3952 Acked Write 7 Done Done Sent false
10768 3953 Acked Read 4 Done Done Sent false
10769 3954 Acked Write 8 Done Done Sent false
10770 3955 Acked Write 4 Done Done Sent false
10771 3956 Acked Write 6 Done Done Sent false
10772 3957 Acked Read 5 Done Done Sent false
10773 3958 Acked Write 3 Done Done Sent false
10774 3959 Acked Write 10 Done Done Sent false
10775 3960 Acked Read 5 Done Done Sent false
10776 3961 Acked Read 6 Done Done Sent false
10777 3962 Acked Read 7 Done Done Sent false
10778 3963 Acked Read 4 Done Done Sent false
10779 3964 Acked Read 7 Done Done Sent false
10780 3965 Acked Read 9 Done Done Sent false
10781 3966 Acked Write 10 Done Done Sent false
10782 3967 Acked Read 5 Done Done Sent false
10783 3968 Acked Read 4 Done Done Sent false
10784 3969 Acked Read 7 Done Done Sent false
10785 3970 Acked Read 3 Done Done Sent false
10786 3971 Acked Flush 0 Done Done Sent false
10787 3972 Acked Write 9 Done Done Sent false
10788 3973 Acked Write 6 Done Done Sent false
10789 3974 Acked Read 7 Done Done Sent false
10790 3975 Acked Read 8 Done Done Sent false
10791 3976 Acked Read 6 Done Done Sent false
10792 3977 Acked Write 9 Done Done Sent false
10793 3978 Acked Read 4 Done Done Sent false
10794 3979 Acked Write 5 Done Done Sent false
10795 3980 Acked Read 10 Done Done Sent false
10796 3981 Acked Write 8 Done Done Sent false
10797 3982 Acked Flush 0 Done Done Sent false
10798 3983 Acked Write 7 Done Done Sent false
10799 3984 Acked Read 3 Done Done Sent false
10800 3985 Acked Write 8 Done Done Sent false
10801 3986 Acked Read 9 Done Done Sent false
10802 3987 Acked Write 2 Done Done Sent false
10803 3988 Acked Read 4 Done Done Sent false
10804 3989 Acked Read 6 Done Done Sent false
10805 3990 Acked Read 8 Done Done Sent false
10806 3991 Acked Read 3 Done Done Sent false
10807 3992 Acked Write 4 Done Done Sent false
10808 3993 Acked Write 6 Done Done Sent false
10809 3994 Acked Write 1 Done Done Sent false
10810 3995 Acked Read 1 Done Done Sent false
10811 3996 Acked Read 9 Done Done Sent false
10812 3997 Acked Read 2 Done Done Sent false
10813 3998 Acked Read 10 Done Done Sent false
10814 3999 Acked Write 6 Done Done Sent false
10815 4000 Acked Write 2 Done Done Sent false
10816 4001 Acked Write 3 Done Done Sent false
10817 4002 Acked Write 7 Done Done Sent false
10818 4003 Acked Read 5 Done Done Sent false
10819 4004 Acked Read 3 Done Done Sent false
10820 4005 Acked Read 6 Done Done Sent false
10821 4006 Acked Write 5 Done Done Sent false
10822 4007 Acked Write 2 Done Done Sent false
10823 4008 Acked Read 2 Done Done Sent false
10824 4009 Acked Read 9 Done Done Sent false
10825 4010 Acked Read 8 Done Done Sent false
10826 4011 Acked Read 8 Done Done Sent false
10827 4012 Acked Write 8 Done Done Sent false
10828 4013 Acked Write 10 Done Done Sent false
10829 4014 Acked Write 4 Done Done Sent false
10830 4015 Acked Write 8 Done Done Sent false
10831 4016 Acked Read 3 Done Done Sent false
10832 4017 Acked Flush 0 Done Done Sent false
10833 4018 Acked Flush 0 Done Done Sent false
10834 STATES DS:0 DS:1 DS:2 TOTAL
10835 Sent 0 0 301 301
10836 Done 301 301 0 602
10837 Skipped 0 0 0 0
10838 Error 0 0 0 0
10839 Last Flush: 4018 4018 3717
10840 Downstairs last five completed: 3717 3716 3715 3714 3713
10841 Upstairs last five completed: 4018 4017 4016 4015 4014
10842 CLIENT: Up:0 ds:301 act:2
10843 [8830] got start action
10844 [8830] I am going to restart
10845 Starting downstairs at port 8830
10846 Make output file at "/tmp/dsc/downstairs-8830.txt"
10847 [8830][2] reports Starting
10848 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1212)
10849 [8830][2] reports Running
10850 ----------------------------------------------------------------
10851 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
10852 States: Active Active Offline
10853 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10854 3718 Acked Read 10 Done Done Sent false
10855 3719 Acked Write 8 Done Done Sent false
10856 3720 Acked Read 6 Done Done Sent false
10857 3721 Acked Read 9 Done Done Sent false
10858 3722 Acked Write 7 Done Done Sent false
10859 3723 Acked Read 4 Done Done Sent false
10860 3724 Acked Read 10 Done Done Sent false
10861 3725 Acked Read 6 Done Done Sent false
10862 3726 Acked Write 5 Done Done Sent false
10863 3727 Acked Read 1 Done Done Sent false
10864 3728 Acked Write 9 Done Done Sent false
10865 3729 Acked Read 3 Done Done Sent false
10866 3730 Acked Write 1 Done Done Sent false
10867 3731 Acked Read 2 Done Done Sent false
10868 3732 Acked Write 10 Done Done Sent false
10869 3733 Acked Read 7 Done Done Sent false
10870 3734 Acked Read 2 Done Done Sent false
10871 3735 Acked Read 3 Done Done Sent false
10872 3736 Acked Read 9 Done Done Sent false
10873 3737 Acked Flush 0 Done Done Sent false
10874 3738 Acked Read 4 Done Done Sent false
10875 3739 Acked Read 9 Done Done Sent false
10876 3740 Acked Write 10 Done Done Sent false
10877 3741 Acked Write 4 Done Done Sent false
10878 3742 Acked Read 7 Done Done Sent false
10879 3743 Acked Read 8 Done Done Sent false
10880 3744 Acked Write 1 Done Done Sent false
10881 3745 Acked Read 3 Done Done Sent false
10882 3746 Acked Write 2 Done Done Sent false
10883 3747 Acked Flush 0 Done Done Sent false
10884 3748 Acked Read 9 Done Done Sent false
10885 3749 Acked Read 1 Done Done Sent false
10886 3750 Acked Write 8 Done Done Sent false
10887 3751 Acked Write 1 Done Done Sent false
10888 3752 Acked Read 1 Done Done Sent false
10889 3753 Acked Write 3 Done Done Sent false
10890 3754 Acked Write 5 Done Done Sent false
10891 3755 Acked Write 5 Done Done Sent false
10892 3756 Acked Read 6 Done Done Sent false
10893 3757 Acked Flush 0 Done Done Sent false
10894 3758 Acked Read 10 Done Done Sent false
10895 3759 Acked Write 3 Done Done Sent false
10896 3760 Acked Read 9 Done Done Sent false
10897 3761 Acked Read 10 Done Done Sent false
10898 3762 Acked Read 2 Done Done Sent false
10899 3763 Acked Flush 0 Done Done Sent false
10900 3764 Acked Read 7 Done Done Sent false
10901 3765 Acked Write 9 Done Done Sent false
10902 3766 Acked Write 2 Done Done Sent false
10903 3767 Acked Read 9 Done Done Sent false
10904 3768 Acked Read 7 Done Done Sent false
10905 3769 Acked Flush 0 Done Done Sent false
10906 3770 Acked Write 6 Done Done Sent false
10907 3771 Acked Write 9 Done Done Sent false
10908 3772 Acked Write 9 Done Done Sent false
10909 3773 Acked Read 5 Done Done Sent false
10910 3774 Acked Read 10 Done Done Sent false
10911 3775 Acked Write 2 Done Done Sent false
10912 3776 Acked Write 9 Done Done Sent false
10913 3777 Acked Write 6 Done Done Sent false
10914 3778 Acked Write 1 Done Done Sent false
10915 3779 Acked Read 4 Done Done Sent false
10916 3780 Acked Read 7 Done Done Sent false
10917 3781 Acked Write 2 Done Done Sent false
10918 3782 Acked Read 2 Done Done Sent false
10919 3783 Acked Flush 0 Done Done Sent false
10920 3784 Acked Flush 0 Done Done Sent false
10921 3785 Acked Read 1 Done Done Sent false
10922 3786 Acked Write 5 Done Done Sent false
10923 3787 Acked Write 1 Done Done Sent false
10924 3788 Acked Read 9 Done Done Sent false
10925 3789 Acked Write 8 Done Done Sent false
10926 3790 Acked Read 9 Done Done Sent false
10927 3791 Acked Write 2 Done Done Sent false
10928 3792 Acked Flush 0 Done Done Sent false
10929 3793 Acked Write 5 Done Done Sent false
10930 3794 Acked Write 4 Done Done Sent false
10931 3795 Acked Read 7 Done Done Sent false
10932 3796 Acked Write 1 Done Done Sent false
10933 3797 Acked Write 2 Done Done Sent false
10934 3798 Acked Write 7 Done Done Sent false
10935 3799 Acked Read 1 Done Done Sent false
10936 3800 Acked Read 8 Done Done Sent false
10937 3801 Acked Read 5 Done Done Sent false
10938 3802 Acked Read 6 Done Done Sent false
10939 3803 Acked Write 6 Done Done Sent false
10940 3804 Acked Write 3 Done Done Sent false
10941 3805 Acked Read 5 Done Done Sent false
10942 3806 Acked Read 8 Done Done Sent false
10943 3807 Acked Write 10 Done Done Sent false
10944 3808 Acked Flush 0 Done Done Sent false
10945 3809 Acked Read 8 Done Done Sent false
10946 3810 Acked Write 7 Done Done Sent false
10947 3811 Acked Read 10 Done Done Sent false
10948 3812 Acked Read 10 Done Done Sent false
10949 3813 Acked Read 5 Done Done Sent false
10950 3814 Acked Read 3 Done Done Sent false
10951 3815 Acked Read 8 Done Done Sent false
10952 3816 Acked Write 4 Done Done Sent false
10953 3817 Acked Write 4 Done Done Sent false
10954 3818 Acked Write 1 Done Done Sent false
10955 3819 Acked Read 10 Done Done Sent false
10956 3820 Acked Write 1 Done Done Sent false
10957 3821 Acked Flush 0 Done Done Sent false
10958 3822 Acked Read 6 Done Done Sent false
10959 3823 Acked Flush 0 Done Done Sent false
10960 3824 Acked Write 2 Done Done Sent false
10961 3825 Acked Read 6 Done Done Sent false
10962 3826 Acked Write 1 Done Done Sent false
10963 3827 Acked Write 8 Done Done Sent false
10964 3828 Acked Write 1 Done Done Sent false
10965 3829 Acked Write 6 Done Done Sent false
10966 3830 Acked Read 2 Done Done Sent false
10967 3831 Acked Read 6 Done Done Sent false
10968 3832 Acked Write 7 Done Done Sent false
10969 3833 Acked Flush 0 Done Done Sent false
10970 3834 Acked Read 6 Done Done Sent false
10971 3835 Acked Write 8 Done Done Sent false
10972 3836 Acked Read 6 Done Done Sent false
10973 3837 Acked Write 9 Done Done Sent false
10974 3838 Acked Read 5 Done Done Sent false
10975 3839 Acked Write 3 Done Done Sent false
10976 3840 Acked Flush 0 Done Done Sent false
10977 3841 Acked Write 9 Done Done Sent false
10978 3842 Acked Read 9 Done Done Sent false
10979 3843 Acked Write 4 Done Done Sent false
10980 3844 Acked Write 5 Done Done Sent false
10981 3845 Acked Write 8 Done Done Sent false
10982 3846 Acked Write 10 Done Done Sent false
10983 3847 Acked Write 3 Done Done Sent false
10984 3848 Acked Write 10 Done Done Sent false
10985 3849 Acked Write 5 Done Done Sent false
10986 3850 Acked Write 8 Done Done Sent false
10987 3851 Acked Flush 0 Done Done Sent false
10988 3852 Acked Write 2 Done Done Sent false
10989 3853 Acked Read 7 Done Done Sent false
10990 3854 Acked Read 9 Done Done Sent false
10991 3855 Acked Read 9 Done Done Sent false
10992 3856 Acked Read 7 Done Done Sent false
10993 3857 Acked Read 9 Done Done Sent false
10994 3858 Acked Read 8 Done Done Sent false
10995 3859 Acked Write 2 Done Done Sent false
10996 3860 Acked Flush 0 Done Done Sent false
10997 3861 Acked Write 2 Done Done Sent false
10998 3862 Acked Write 9 Done Done Sent false
10999 3863 Acked Read 9 Done Done Sent false
11000 3864 Acked Write 7 Done Done Sent false
11001 3865 Acked Read 6 Done Done Sent false
11002 3866 Acked Read 9 Done Done Sent false
11003 3867 Acked Read 4 Done Done Sent false
11004 3868 Acked Write 7 Done Done Sent false
11005 3869 Acked Write 8 Done Done Sent false
11006 3870 Acked Read 1 Done Done Sent false
11007 3871 Acked Flush 0 Done Done Sent false
11008 3872 Acked Read 8 Done Done Sent false
11009 3873 Acked Write 10 Done Done Sent false
11010 3874 Acked Write 1 Done Done Sent false
11011 3875 Acked Flush 0 Done Done Sent false
11012 3876 Acked Read 10 Done Done Sent false
11013 3877 Acked Write 4 Done Done Sent false
11014 3878 Acked Write 1 Done Done Sent false
11015 3879 Acked Write 2 Done Done Sent false
11016 3880 Acked Read 7 Done Done Sent false
11017 3881 Acked Read 2 Done Done Sent false
11018 3882 Acked Write 10 Done Done Sent false
11019 3883 Acked Flush 0 Done Done Sent false
11020 3884 Acked Write 8 Done Done Sent false
11021 3885 Acked Read 8 Done Done Sent false
11022 3886 Acked Read 5 Done Done Sent false
11023 3887 Acked Read 7 Done Done Sent false
11024 3888 Acked Write 8 Done Done Sent false
11025 3889 Acked Read 6 Done Done Sent false
11026 3890 Acked Write 6 Done Done Sent false
11027 3891 Acked Write 7 Done Done Sent false
11028 3892 Acked Write 5 Done Done Sent false
11029 3893 Acked Write 4 Done Done Sent false
11030 3894 Acked Write 8 Done Done Sent false
11031 3895 Acked Read 7 Done Done Sent false
11032 3896 Acked Flush 0 Done Done Sent false
11033 3897 Acked Read 4 Done Done Sent false
11034 3898 Acked Read 8 Done Done Sent false
11035 3899 Acked Read 1 Done Done Sent false
11036 3900 Acked Read 2 Done Done Sent false
11037 3901 Acked Flush 0 Done Done Sent false
11038 3902 Acked Write 7 Done Done Sent false
11039 3903 Acked Write 4 Done Done Sent false
11040 3904 Acked Flush 0 Done Done Sent false
11041 3905 Acked Write 9 Done Done Sent false
11042 3906 Acked Flush 0 Done Done Sent false
11043 3907 Acked Write 2 Done Done Sent false
11044 3908 Acked Read 6 Done Done Sent false
11045 3909 Acked Flush 0 Done Done Sent false
11046 3910 Acked Write 5 Done Done Sent false
11047 3911 Acked Write 6 Done Done Sent false
11048 3912 Acked Read 6 Done Done Sent false
11049 3913 Acked Read 3 Done Done Sent false
11050 3914 Acked Read 4 Done Done Sent false
11051 3915 Acked Read 3 Done Done Sent false
11052 3916 Acked Read 1 Done Done Sent false
11053 3917 Acked Read 4 Done Done Sent false
11054 3918 Acked Read 7 Done Done Sent false
11055 3919 Acked Write 5 Done Done Sent false
11056 3920 Acked Flush 0 Done Done Sent false
11057 3921 Acked Write 8 Done Done Sent false
11058 3922 Acked Read 3 Done Done Sent false
11059 3923 Acked Write 2 Done Done Sent false
11060 3924 Acked Read 8 Done Done Sent false
11061 3925 Acked Read 8 Done Done Sent false
11062 3926 Acked Flush 0 Done Done Sent false
11063 3927 Acked Write 9 Done Done Sent false
11064 3928 Acked Flush 0 Done Done Sent false
11065 3929 Acked Read 6 Done Done Sent false
11066 3930 Acked Write 6 Done Done Sent false
11067 3931 Acked Write 1 Done Done Sent false
11068 3932 Acked Read 3 Done Done Sent false
11069 3933 Acked Read 1 Done Done Sent false
11070 3934 Acked Flush 0 Done Done Sent false
11071 3935 Acked Write 5 Done Done Sent false
11072 3936 Acked Write 2 Done Done Sent false
11073 3937 Acked Write 9 Done Done Sent false
11074 3938 Acked Read 10 Done Done Sent false
11075 3939 Acked Write 8 Done Done Sent false
11076 3940 Acked Read 9 Done Done Sent false
11077 3941 Acked Write 1 Done Done Sent false
11078 3942 Acked Read 2 Done Done Sent false
11079 3943 Acked Read 10 Done Done Sent false
11080 3944 Acked Read 2 Done Done Sent false
11081 3945 Acked Read 1 Done Done Sent false
11082 3946 Acked Read 5 Done Done Sent false
11083 3947 Acked Read 2 Done Done Sent false
11084 3948 Acked Flush 0 Done Done Sent false
11085 3949 Acked Read 5 Done Done Sent false
11086 3950 Acked Read 3 Done Done Sent false
11087 3951 Acked Write 3 Done Done Sent false
11088 3952 Acked Write 7 Done Done Sent false
11089 3953 Acked Read 4 Done Done Sent false
11090 3954 Acked Write 8 Done Done Sent false
11091 3955 Acked Write 4 Done Done Sent false
11092 3956 Acked Write 6 Done Done Sent false
11093 3957 Acked Read 5 Done Done Sent false
11094 3958 Acked Write 3 Done Done Sent false
11095 3959 Acked Write 10 Done Done Sent false
11096 3960 Acked Read 5 Done Done Sent false
11097 3961 Acked Read 6 Done Done Sent false
11098 3962 Acked Read 7 Done Done Sent false
11099 3963 Acked Read 4 Done Done Sent false
11100 3964 Acked Read 7 Done Done Sent false
11101 3965 Acked Read 9 Done Done Sent false
11102 3966 Acked Write 10 Done Done Sent false
11103 3967 Acked Read 5 Done Done Sent false
11104 3968 Acked Read 4 Done Done Sent false
11105 3969 Acked Read 7 Done Done Sent false
11106 3970 Acked Read 3 Done Done Sent false
11107 3971 Acked Flush 0 Done Done Sent false
11108 3972 Acked Write 9 Done Done Sent false
11109 3973 Acked Write 6 Done Done Sent false
11110 3974 Acked Read 7 Done Done Sent false
11111 3975 Acked Read 8 Done Done Sent false
11112 3976 Acked Read 6 Done Done Sent false
11113 3977 Acked Write 9 Done Done Sent false
11114 3978 Acked Read 4 Done Done Sent false
11115 3979 Acked Write 5 Done Done Sent false
11116 3980 Acked Read 10 Done Done Sent false
11117 3981 Acked Write 8 Done Done Sent false
11118 3982 Acked Flush 0 Done Done Sent false
11119 3983 Acked Write 7 Done Done Sent false
11120 3984 Acked Read 3 Done Done Sent false
11121 3985 Acked Write 8 Done Done Sent false
11122 3986 Acked Read 9 Done Done Sent false
11123 3987 Acked Write 2 Done Done Sent false
11124 3988 Acked Read 4 Done Done Sent false
11125 3989 Acked Read 6 Done Done Sent false
11126 3990 Acked Read 8 Done Done Sent false
11127 3991 Acked Read 3 Done Done Sent false
11128 3992 Acked Write 4 Done Done Sent false
11129 3993 Acked Write 6 Done Done Sent false
11130 3994 Acked Write 1 Done Done Sent false
11131 3995 Acked Read 1 Done Done Sent false
11132 3996 Acked Read 9 Done Done Sent false
11133 3997 Acked Read 2 Done Done Sent false
11134 3998 Acked Read 10 Done Done Sent false
11135 3999 Acked Write 6 Done Done Sent false
11136 4000 Acked Write 2 Done Done Sent false
11137 4001 Acked Write 3 Done Done Sent false
11138 4002 Acked Write 7 Done Done Sent false
11139 4003 Acked Read 5 Done Done Sent false
11140 4004 Acked Read 3 Done Done Sent false
11141 4005 Acked Read 6 Done Done Sent false
11142 4006 Acked Write 5 Done Done Sent false
11143 4007 Acked Write 2 Done Done Sent false
11144 4008 Acked Read 2 Done Done Sent false
11145 4009 Acked Read 9 Done Done Sent false
11146 4010 Acked Read 8 Done Done Sent false
11147 4011 Acked Read 8 Done Done Sent false
11148 4012 Acked Write 8 Done Done Sent false
11149 4013 Acked Write 10 Done Done Sent false
11150 4014 Acked Write 4 Done Done Sent false
11151 4015 Acked Write 8 Done Done Sent false
11152 4016 Acked Read 3 Done Done Sent false
11153 4017 Acked Flush 0 Done Done Sent false
11154 4018 Acked Flush 0 Done Done Sent false
11155 STATES DS:0 DS:1 DS:2 TOTAL
11156 Sent 0 0 301 301
11157 Done 301 301 0 602
11158 Skipped 0 0 0 0
11159 Error 0 0 0 0
11160 Last Flush: 4018 4018 3717
11161 Downstairs last five completed: 3717 3716 3715 3714 3713
11162 Upstairs last five completed: 4018 4017 4016 4015 4014
11163 CLIENT: Up:0 ds:301 act:2
111642025-01-13T15:54:07.302ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
111652025-01-13T15:54:07.303ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36872) = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
111662025-01-13T15:54:07.304ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
111672025-01-13T15:54:07.304ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
111682025-01-13T15:54:07.304ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
111692025-01-13T15:54:07.304ZINFOcrucible: send last flush ID to this DS: 3717 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
111702025-01-13T15:54:07.305ZINFOcrucible: Replied this last flush ID: 3717 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
111712025-01-13T15:54:07.305ZINFOcrucible: [2] client re-new 301 jobs since flush 3717 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
111722025-01-13T15:54:07.305ZINFOcrucible: [2] Replayed 301 jobs since flush: 3717 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
11173 ----------------------------------------------------------------
11174 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
11175 Downstairs last five completed: 4018 4017 4016 4015 4014
11176 Upstairs last five completed: 4018 4017 4016 4015 4014
11177 CLIENT: Up:0 ds:0 act:3
11178 Replay: All jobs finished
11179 Main task has work to do, go find it
11180 got dsc Stop(2)
11181 stop 2
11182 Jan 13 15:54:09.550 INFO request completed, latency_us: 372, response_code: 204, uri: /stop/cid/2, method: POST, req_id: a651b90f-a698-475c-9bed-91791fad7bf3, remote_addr: 127.0.0.1:[8830] Got stop action so:false kr:false
11183 52422, local_addr: 127.0.0.1:9998
11184 Jan 13 15:54:09.551{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:54:09.551083105Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
11185 INFO request completed, latency_us: 130, response_code: 200, uri{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:54:09.551162853Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
11186 : /state/cid/2, method: GET{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:54:09.551205516Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
11187 , req_id: 928c10db-5f30-4ea5-be97-a1669237c5ce, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
111882025-01-13T15:54:09.551ZINFOcrucible: client task is exiting = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
11189 [8830] Exited with: ExitStatus(unix_wait_status(9))
11190 [8830][2] reports Exit
11191 Jan 13 15:54:13.554 INFO request completed, latency_us: 418, response_code: 200, uri: /state/cid/2, method: GET, req_id: c5c25990-6f6f-46d9-9965-604fd5adae71, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
11192 001/300 Read block 4011 len 24576
11193 002/300 Read block 16 len 40960
11194 003/300 Write block 2523 len 24576 data: 2 3 3 3 3 3
11195 004/300 Write block 2109 len 16384 data: 3 3 3 3
11196 005/300 Read block 1095 len 32768
11197 006/300 Flush
11198 007/300 Read block 776 len 4096
11199 008/300 Flush
11200 009/300 Read block 4398 len 16384
11201 010/300 Read block 2092 len 16384
11202 011/300 Write block 4508 len 40960 data: 3 3 4 4 4 3 4 4 4 4
11203 012/300 Write block 4337 len 12288 data: 3 2 2
11204 013/300 Read block 3472 len 40960
11205 014/300 Write block 3126 len 4096 data: 4
11206 015/300 Read block 2503 len 32768
11207 016/300 Read block 2637 len 32768
11208 017/300 Write block 290 len 12288 data: 5 5 4
11209 018/300 Write block 2755 len 20480 data: 2 2 2 2 2
11210 019/300 Read block 357 len 12288
11211 020/300 Write block 3070 len 12288 data: 4 4 5
11212 021/300 Write block 4819 len 20480 data: 3 3 3 3 2
11213 022/300 Write block 2155 len 4096 data: 2
11214 023/300 Read block 2469 len 28672
11215 024/300 Write block 4211 len 4096 data: 4
11216 025/300 Write block 30 len 12288 data: 5 4 3
11217 026/300 Write block 4800 len 36864 data: 4 3 3 3 3 3 2 3 3
11218 027/300 Write block 4257 len 8192 data: 2 2
11219 028/300 Read block 1951 len 36864
11220 029/300 Read block 4836 len 20480
11221 030/300 Read block 1364 len 8192
11222 031/300 Read block 3412 len 16384
11223 032/300 Read block 2433 len 20480
11224 033/300 Write block 1979 len 32768 data: 3 3 3 3 3 4 3 3
11225 034/300 Write block 2892 len 12288 data: 3 3 4
11226 035/300 Read block 1308 len 4096
11227 036/300 Read block 2244 len 32768
11228 037/300 Read block 2502 len 40960
11229 038/300 Read block 2494 len 28672
11230 039/300 Flush
11231 040/300 Read block 1354 len 36864
11232 041/300 Write block 4497 len 32768 data: 5 4 4 4 4 4 4 4
11233 042/300 Write block 4566 len 40960 data: 3 3 3 3 2 2 3 3 3 3
11234 043/300 Read block 116 len 32768
11235 044/300 Read block 3713 len 20480
11236 045/300 Read block 3330 len 28672
11237 046/300 Read block 3887 len 20480
11238 047/300 Read block 2264 len 28672
11239 048/300 Write block 224 len 8192 data: 4 4
11240 049/300 Read block 4014 len 4096
11241 050/300 Write block 1960 len 40960 data: 2 2 2 3 3 3 4 4 5 5
11242 051/300 Read block 109 len 12288
11243 052/300 Flush
11244 053/300 Write block 144 len 36864 data: 6 5 5 5 5 5 4 3 3
11245 054/300 Read block 2850 len 36864
11246 055/300 Write block 3784 len 4096 data: 3
11247 056/300 Read block 1907 len 28672
11248 057/300 Read block 4802 len 24576
11249 058/300 Read block 1572 len 24576
11250 059/300 Write block 2057 len 4096 data: 2
11251 060/300 Read block 1804 len 16384
11252 061/300 Read block 161 len 12288
11253 062/300 Write block 2332 len 36864 data: 2 2 2 2 2 2 2 3 3
11254 063/300 Write block 3507 len 36864 data: 3 3 4 3 4 4 5 6 5
11255 064/300 Write block 2475 len 20480 data: 3 2 3 3 4
11256 065/300 Read block 323 len 20480
11257 066/300 Read block 2333 len 40960
11258 067/300 Write block 1584 len 36864 data: 4 4 5 4 4 4 5 5 4
11259 068/300 Read block 1663 len 4096
11260 069/300 Write block 821 len 20480 data: 3 3 3 3 3
11261 070/300 Write block 1616 len 8192 data: 4 4
11262 071/300 Flush
11263 072/300 Read block 4422 len 4096
11264 073/300 Read block 2992 len 32768
11265 074/300 Read block 1326 len 12288
11266 075/300 Read block 1463 len 12288
11267 076/300 Flush
11268 077/300 Flush
11269 078/300 Read block 2637 len 16384
11270 079/300 Read block 4325 len 36864
11271 080/300 Read block 3583 len 32768
11272 081/300 Write block 303 len 12288 data: 3 3 3
11273 082/300 Read block 1806 len 16384
11274 083/300 Write block 3837 len 40960 data: 3 4 4 4 4 4 4 4 3 3
11275 084/300 Read block 2870 len 24576
11276 085/300 Read block 4484 len 8192
11277 086/300 Write block 118 len 24576 data: 4 5 5 4 4 4
11278 087/300 Write block 3200 len 20480 data: 4 4 4 2 2
11279 088/300 Read block 932 len 8192
11280 089/300 Read block 3596 len 32768
11281 090/300 Read block 163 len 16384
11282 091/300 Read block 2172 len 32768
11283 092/300 Write block 4765 len 40960 data: 2 2 2 2 2 2 2 2 2 3
11284 093/300 Read block 975 len 16384
11285 094/300 Read block 4324 len 32768
11286 095/300 Write block 61 len 40960 data: 4 4 4 4 2 2 2 2 3 3
11287 096/300 Read block 1860 len 24576
11288 097/300 Write block 2646 len 12288 data: 5 4 4
11289 098/300 Write block 282 len 24576 data: 2 2 3 5 5 6
11290 099/300 Read block 2450 len 4096
11291 100/300 Write block 571 len 24576 data: 3 3 3 5 4 5
11292 101/300 Write block 777 len 8192 data: 3 3
11293 102/300 Write block 2662 len 24576 data: 3 3 3 3 3 3
11294 103/300 Read block 1840 len 4096
11295 104/300 Read block 946 len 40960
11296 105/300 Read block 3040 len 12288
11297 106/300 Read block 4521 len 24576
11298 107/300 Read block 4887 len 16384
11299 108/300 Read block 1371 len 4096
11300 109/300 Write block 4208 len 16384 data: 3 3 3 5
11301 110/300 Read block 2077 len 12288
11302 111/300 Flush
11303 112/300 Flush
11304 113/300 Read block 283 len 16384
11305 114/300 Read block 2654 len 40960
11306 115/300 Flush
11307 116/300 Read block 3318 len 12288
11308 117/300 Read block 940 len 32768
11309 118/300 Read block 2266 len 8192
11310 119/300 Read block 2040 len 8192
11311 120/300 Read block 144 len 4096
11312 121/300 Write block 3776 len 24576 data: 4 4 4 5 4 3
11313 122/300 Flush
11314 123/300 Read block 1980 len 4096
11315 124/300 Read block 3285 len 36864
11316 125/300 Read block 438 len 36864
11317 126/300 Flush
11318 127/300 Write block 2201 len 24576 data: 3 3 4 4 4 3
11319 128/300 Read block 1842 len 36864
11320 129/300 Write block 1386 len 16384 data: 7 6 5 4
11321 130/300 Read block 1050 len 12288
11322 131/300 Write block 2879 len 32768 data: 2 2 2 2 3 3 3 4
11323 132/300 Read block 4492 len 20480
11324 133/300 Read block 883 len 20480
11325 134/300 Read block 1747 len 8192
11326 135/300 Write block 2710 len 28672 data: 2 3 3 3 3 3 3
11327 136/300 Flush
11328 137/300 Write block 1113 len 8192 data: 3 3
11329 138/300 Read block 508 len 36864
11330 139/300 Write block 1319 len 12288 data: 3 3 3
11331 140/300 Read block 4860 len 12288
11332 141/300 Write block 4069 len 12288 data: 2 3 3
11333 142/300 Read block 4265 len 40960
11334 143/300 Read block 873 len 4096
11335 144/300 Read block 794 len 16384
11336 145/300 Write block 1993 len 8192 data: 2 2
11337 146/300 Write block 2436 len 36864 data: 3 3 3 3 3 3 3 4 4
11338 147/300 Read block 1170 len 40960
11339 148/300 Read block 3701 len 32768
11340 149/300 Read block 542 len 40960
11341 150/300 Write block 3492 len 36864 data: 5 3 3 3 3 2 2 3 3
11342 151/300 Flush
11343 152/300 Flush
11344 153/300 Read block 2492 len 20480
11345 154/300 Read block 2307 len 8192
11346 155/300 Read block 455 len 28672
11347 156/300 Write block 2971 len 28672 data: 5 5 4 3 4 4 4
11348 157/300 Write block 863 len 32768 data: 4 4 4 5 5 4 5 4
11349 158/300 Write block 3450 len 16384 data: 6 4 3 3
11350 159/300 Write block 281 len 24576 data: 2 3 3 4 6 6
11351 160/300 Read block 4249 len 24576
11352 161/300 Write block 1184 len 32768 data: 3 3 4 4 3 3 3 3
11353 162/300 Read block 1153 len 16384
11354 163/300 Flush
11355 164/300 Read block 1493 len 40960
11356 165/300 Write block 981 len 8192 data: 3 3
11357 166/300 Read block 537 len 4096
11358 167/300 Read block 562 len 8192
11359 168/300 Flush
11360 169/300 Read block 540 len 16384
11361 170/300 Flush
11362 171/300 Read block 2921 len 8192
11363 172/300 Read block 4759 len 28672
11364 173/300 Write block 1601 len 4096 data: 2
11365 174/300 Write block 1212 len 4096 data: 3
11366 175/300 Write block 1555 len 24576 data: 2 2 2 2 2 3
11367 176/300 Write block 3510 len 24576 data: 4 5 5 6 7 6
11368 177/300 Read block 3203 len 20480
11369 178/300 Read block 2952 len 36864
11370 179/300 Write block 2880 len 36864 data: 3 3 3 4 4 4 5 3 3
11371 180/300 Flush
11372 181/300 Read block 1174 len 24576
11373 182/300 Read block 551 len 36864
11374 183/300 Read block 850 len 24576
11375 184/300 Flush
11376 185/300 Write block 2083 len 20480 data: 6 4 3 3 3
11377 186/300 Read block 2729 len 32768
11378 187/300 Read block 805 len 16384
11379 188/300 Read block 1049 len 28672
11380 189/300 Write block 3921 len 4096 data: 5
11381 190/300 Flush
11382 191/300 Read block 2714 len 16384
11383 192/300 Read block 648 len 40960
11384 193/300 Read block 513 len 28672
11385 194/300 Read block 4079 len 36864
11386 195/300 Read block 4107 len 40960
11387 196/300 Write block 1667 len 12288 data: 4 6 5
11388 197/300 Read block 4806 len 24576
11389 198/300 Write block 3767 len 24576 data: 2 2 2 3 4 5
11390 199/300 Read block 1290 len 28672
11391 200/300 Read block 3403 len 20480
11392 201/300 Read block 4214 len 4096
11393 202/300 Write block 327 len 20480 data: 2 3 3 3 2
11394 203/300 Write block 3481 len 40960 data: 3 4 4 5 6 6 7 6 6 5
11395 204/300 Flush
11396 205/300 Write block 1049 len 36864 data: 5 4 4 5 5 5 5 6 5
11397 206/300 Read block 2807 len 40960
11398 207/300 Write block 4863 len 28672 data: 5 4 5 6 5 4 4
11399 208/300 Write block 2149 len 28672 data: 4 4 4 3 2 2 3
11400 209/300 Read block 3292 len 4096
11401 210/300 Write block 2992 len 12288 data: 2 2 2
11402 211/300 Read block 1260 len 32768
11403 212/300 Flush
11404 213/300 Write block 577 len 32768 data: 5 5 5 5 4 3 3 3
11405 214/300 Read block 773 len 40960
11406 215/300 Read block 3318 len 20480
11407 216/300 Read block 2425 len 8192
11408 217/300 Read block 1269 len 12288
11409 218/300 Read block 3211 len 8192
11410 219/300 Read block 4332 len 24576
11411 220/300 Read block 4874 len 24576
11412 221/300 Write block 3474 len 4096 data: 3
11413 222/300 Read block 3741 len 12288
11414 223/300 Read block 343 len 28672
11415 224/300 Read block 218 len 16384
11416 225/300 Write block 658 len 28672 data: 3 3 4 5 5 5 7
11417 226/300 Read block 4691 len 4096
11418 227/300 Write block 1613 len 4096 data: 3
11419 228/300 Write block 3122 len 4096 data: 3
11420 229/300 Read block 2882 len 20480
11421 230/300 Write block 1177 len 16384 data: 5 4 3 2
11422 231/300 Read block 2950 len 40960
11423 232/300 Write block 4172 len 24576 data: 4 4 3 3 2 2
11424 233/300 Read block 1492 len 16384
11425 234/300 Read block 440 len 28672
11426 235/300 Flush
11427 236/300 Write block 1591 len 40960 data: 6 5 3 3 3 2 2 2 2 2
11428 237/300 Flush
11429 238/300 Write block 538 len 8192 data: 3 3
11430 239/300 Read block 3473 len 16384
11431 240/300 Write block 727 len 32768 data: 3 3 3 3 3 3 4 3
11432 241/300 Flush
11433 242/300 Read block 4726 len 8192
11434 243/300 Read block 4744 len 28672
11435 244/300 Write block 3715 len 20480 data: 2 3 3 3 3
11436 245/300 Read block 1382 len 4096
11437 246/300 Read block 2948 len 12288
11438 247/300 Write block 4448 len 16384 data: 6 5 4 3
11439 248/300 Read block 1943 len 8192
11440 249/300 Read block 1407 len 4096
11441 250/300 Write block 2194 len 36864 data: 4 4 4 5 5 4 4 4 4
11442 251/300 Flush
11443 252/300 Write block 3857 len 40960 data: 2 4 4 4 4 4 4 3 3 2
11444 253/300 Flush
11445 254/300 Read block 4632 len 36864
11446 255/300 Read block 2470 len 24576
11447 256/300 Read block 2860 len 28672
11448 257/300 Write block 3202 len 40960 data: 5 3 3 2 2 2 2 3 3 3
11449 258/300 Read block 3696 len 32768
11450 259/300 Read block 4687 len 4096
11451 260/300 Write block 1392 len 12288 data: 3 2 2
11452 261/300 Read block 3895 len 24576
11453 262/300 Flush
11454 263/300 Read block 4068 len 32768
11455 264/300 Read block 4303 len 8192
11456 265/300 Read block 453 len 28672
11457 266/300 Read block 4942 len 28672
11458 267/300 Flush
11459 268/300 Write block 3488 len 32768 data: 7 7 6 5 6 4 4 4
11460 269/300 Write block 824 len 12288 data: 4 4 3
11461 270/300 Read block 4691 len 12288
11462 271/300 Write block 3187 len 12288 data: 4 5 7
11463 272/300 Write block 3491 len 12288 data: 6 7 5
11464 273/300 Read block 3575 len 12288
11465 274/300 Read block 2815 len 12288
11466 275/300 Read block 4792 len 4096
11467 276/300 Write block 4445 len 32768 data: 3 3 5 7 6 5 4 3
11468 277/300 Write block 425 len 16384 data: 3 3 3 4
11469 278/300 Flush
11470 279/300 Read block 614 len 28672
11471 280/300 Write block 179 len 20480 data: 3 2 3 3 3
11472 281/300 Read block 856 len 36864
11473 282/300 Read block 871 len 20480
11474 283/300 Read block 1745 len 24576
11475 284/300 Flush
11476 285/300 Write block 1335 len 36864 data: 6 7 7 6 5 5 4 4 3
11477 286/300 Flush
11478 287/300 Read block 2775 len 28672
11479 288/300 Write block 3892 len 40960 data: 3 4 5 4 4 4 4 5 4 4
11480 289/300 Read block 1040 len 28672
11481 290/300 Read block 3273 len 32768
11482 291/300 Read block 4767 len 12288
11483 292/300 Read block 386 len 16384
11484 293/300 Read block 3027 len 24576
11485 294/300 Read block 916 len 24576
11486 295/300 Read block 3041 len 28672
11487 296/300 Read block 643 len 36864
11488 297/300 Read block 2117 len 8192
11489 298/300 Read block 3033 len 28672
11490 299/300 Read block 3924 len 4096
11491 300/300 Read block 1480 len 20480
11492 301/300 Read block 323 len 8192
11493 Main task has work to do, go find it
11494 got dsc Start(2)
11495 start 2
11496 Jan 13 15:54:13.803 INFO request completed, latency_us: 209, response_code: 204, uri: /start/cid/2, method: POST, req_id: b00bdb4d-fc43-400d-b79d-aaf0ba59502c, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
11497 Replay: started 2, returned:Ok(())
11498 ----------------------------------------------------------------
11499 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
11500 States: Active Active Offline
11501 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11502 4019 Acked Read 6 Done Done Sent false
11503 4020 Acked Read 10 Done Done Sent false
11504 4021 Acked Write 6 Done Done Sent false
11505 4022 Acked Write 4 Done Done Sent false
11506 4023 Acked Read 8 Done Done Sent false
11507 4024 Acked Flush 0 Done Done Sent false
11508 4025 Acked Read 1 Done Done Sent false
11509 4026 Acked Flush 0 Done Done Sent false
11510 4027 Acked Read 4 Done Done Sent false
11511 4028 Acked Read 4 Done Done Sent false
11512 4029 Acked Write 10 Done Done Sent false
11513 4030 Acked Write 3 Done Done Sent false
11514 4031 Acked Read 10 Done Done Sent false
11515 4032 Acked Write 1 Done Done Sent false
11516 4033 Acked Read 8 Done Done Sent false
11517 4034 Acked Read 8 Done Done Sent false
11518 4035 Acked Write 3 Done Done Sent false
11519 4036 Acked Write 5 Done Done Sent false
11520 4037 Acked Read 3 Done Done Sent false
11521 4038 Acked Write 3 Done Done Sent false
11522 4039 Acked Write 5 Done Done Sent false
11523 4040 Acked Write 1 Done Done Sent false
11524 4041 Acked Read 7 Done Done Sent false
11525 4042 Acked Write 1 Done Done Sent false
11526 4043 Acked Write 3 Done Done Sent false
11527 4044 Acked Write 9 Done Done Sent false
11528 4045 Acked Write 2 Done Done Sent false
11529 4046 Acked Read 9 Done Done Sent false
11530 4047 Acked Read 5 Done Done Sent false
11531 4048 Acked Read 2 Done Done Sent false
11532 4049 Acked Read 4 Done Done Sent false
11533 4050 Acked Read 5 Done Done Sent false
11534 4051 Acked Write 8 Done Done Sent false
11535 4052 Acked Write 3 Done Done Sent false
11536 4053 Acked Read 1 Done Done Sent false
11537 4054 Acked Read 8 Done Done Sent false
11538 4055 Acked Read 10 Done Done Sent false
11539 4056 Acked Read 7 Done Done Sent false
11540 4057 Acked Flush 0 Done Done Sent false
11541 4058 Acked Read 9 Done Done Sent false
11542 4059 Acked Write 8 Done Done Sent false
11543 4060 Acked Write 10 Done Done Sent false
11544 4061 Acked Read 8 Done Done Sent false
11545 4062 Acked Read 5 Done Done Sent false
11546 4063 Acked Read 7 Done Done Sent false
11547 4064 Acked Read 5 Done Done Sent false
11548 4065 Acked Read 7 Done Done Sent false
11549 4066 Acked Write 2 Done Done Sent false
11550 4067 Acked Read 1 Done Done Sent false
11551 4068 Acked Write 10 Done Done Sent false
11552 4069 Acked Read 3 Done Done Sent false
11553 4070 Acked Flush 0 Done Done Sent false
11554 4071 Acked Write 9 Done Done Sent false
11555 4072 Acked Read 9 Done Done Sent false
11556 4073 Acked Write 1 Done Done Sent false
11557 4074 Acked Read 7 Done Done Sent false
11558 4075 Acked Read 6 Done Done Sent false
11559 4076 Acked Read 6 Done Done Sent false
11560 4077 Acked Write 1 Done Done Sent false
11561 4078 Acked Read 4 Done Done Sent false
11562 4079 Acked Read 3 Done Done Sent false
11563 4080 Acked Write 9 Done Done Sent false
11564 4081 Acked Write 9 Done Done Sent false
11565 4082 Acked Write 5 Done Done Sent false
11566 4083 Acked Read 5 Done Done Sent false
11567 4084 Acked Read 10 Done Done Sent false
11568 4085 Acked Write 9 Done Done Sent false
11569 4086 Acked Read 1 Done Done Sent false
11570 4087 Acked Write 5 Done Done Sent false
11571 4088 Acked Write 2 Done Done Sent false
11572 4089 Acked Flush 0 Done Done Sent false
11573 4090 Acked Read 1 Done Done Sent false
11574 4091 Acked Read 8 Done Done Sent false
11575 4092 Acked Read 3 Done Done Sent false
11576 4093 Acked Read 3 Done Done Sent false
11577 4094 Acked Flush 0 Done Done Sent false
11578 4095 Acked Flush 0 Done Done Sent false
11579 4096 Acked Read 4 Done Done Sent false
11580 4097 Acked Read 9 Done Done Sent false
11581 4098 Acked Read 8 Done Done Sent false
11582 4099 Acked Write 3 Done Done Sent false
11583 4100 Acked Read 4 Done Done Sent false
11584 4101 Acked Write 10 Done Done Sent false
11585 4102 Acked Read 6 Done Done Sent false
11586 4103 Acked Read 2 Done Done Sent false
11587 4104 Acked Write 6 Done Done Sent false
11588 4105 Acked Write 5 Done Done Sent false
11589 4106 Acked Read 2 Done Done Sent false
11590 4107 Acked Read 8 Done Done Sent false
11591 4108 Acked Read 4 Done Done Sent false
11592 4109 Acked Read 8 Done Done Sent false
11593 4110 Acked Write 10 Done Done Sent false
11594 4111 Acked Read 4 Done Done Sent false
11595 4112 Acked Read 8 Done Done Sent false
11596 4113 Acked Write 10 Done Done Sent false
11597 4114 Acked Read 6 Done Done Sent false
11598 4115 Acked Write 3 Done Done Sent false
11599 4116 Acked Write 6 Done Done Sent false
11600 4117 Acked Read 1 Done Done Sent false
11601 4118 Acked Write 6 Done Done Sent false
11602 4119 Acked Write 2 Done Done Sent false
11603 4120 Acked Write 6 Done Done Sent false
11604 4121 Acked Read 1 Done Done Sent false
11605 4122 Acked Read 10 Done Done Sent false
11606 4123 Acked Read 3 Done Done Sent false
11607 4124 Acked Read 6 Done Done Sent false
11608 4125 Acked Read 4 Done Done Sent false
11609 4126 Acked Read 1 Done Done Sent false
11610 4127 Acked Write 4 Done Done Sent false
11611 4128 Acked Read 3 Done Done Sent false
11612 4129 Acked Flush 0 Done Done Sent false
11613 4130 Acked Flush 0 Done Done Sent false
11614 4131 Acked Read 4 Done Done Sent false
11615 4132 Acked Read 10 Done Done Sent false
11616 4133 Acked Flush 0 Done Done Sent false
11617 4134 Acked Read 3 Done Done Sent false
11618 4135 Acked Read 8 Done Done Sent false
11619 4136 Acked Read 2 Done Done Sent false
11620 4137 Acked Read 2 Done Done Sent false
11621 4138 Acked Read 1 Done Done Sent false
11622 4139 Acked Write 6 Done Done Sent false
11623 4140 Acked Flush 0 Done Done Sent false
11624 4141 Acked Read 1 Done Done Sent false
11625 4142 Acked Read 9 Done Done Sent false
11626 4143 Acked Read 9 Done Done Sent false
11627 4144 Acked Flush 0 Done Done Sent false
11628 4145 Acked Write 6 Done Done Sent false
11629 4146 Acked Read 9 Done Done Sent false
11630 4147 Acked Write 4 Done Done Sent false
11631 4148 Acked Read 3 Done Done Sent false
11632 4149 Acked Write 8 Done Done Sent false
11633 4150 Acked Read 5 Done Done Sent false
11634 4151 Acked Read 5 Done Done Sent false
11635 4152 Acked Read 2 Done Done Sent false
11636 4153 Acked Write 7 Done Done Sent false
11637 4154 Acked Flush 0 Done Done Sent false
11638 4155 Acked Write 2 Done Done Sent false
11639 4156 Acked Read 9 Done Done Sent false
11640 4157 Acked Write 3 Done Done Sent false
11641 4158 Acked Read 3 Done Done Sent false
11642 4159 Acked Write 3 Done Done Sent false
11643 4160 Acked Read 10 Done Done Sent false
11644 4161 Acked Read 1 Done Done Sent false
11645 4162 Acked Read 4 Done Done Sent false
11646 4163 Acked Write 2 Done Done Sent false
11647 4164 Acked Write 9 Done Done Sent false
11648 4165 Acked Read 10 Done Done Sent false
11649 4166 Acked Read 8 Done Done Sent false
11650 4167 Acked Read 10 Done Done Sent false
11651 4168 Acked Write 9 Done Done Sent false
11652 4169 Acked Flush 0 Done Done Sent false
11653 4170 Acked Flush 0 Done Done Sent false
11654 4171 Acked Read 5 Done Done Sent false
11655 4172 Acked Read 2 Done Done Sent false
11656 4173 Acked Read 7 Done Done Sent false
11657 4174 Acked Write 7 Done Done Sent false
11658 4175 Acked Write 8 Done Done Sent false
11659 4176 Acked Write 4 Done Done Sent false
11660 4177 Acked Write 6 Done Done Sent false
11661 4178 Acked Read 6 Done Done Sent false
11662 4179 Acked Write 8 Done Done Sent false
11663 4180 Acked Read 4 Done Done Sent false
11664 4181 Acked Flush 0 Done Done Sent false
11665 4182 Acked Read 10 Done Done Sent false
11666 4183 Acked Write 2 Done Done Sent false
11667 4184 Acked Read 1 Done Done Sent false
11668 4185 Acked Read 2 Done Done Sent false
11669 4186 Acked Flush 0 Done Done Sent false
11670 4187 Acked Read 4 Done Done Sent false
11671 4188 Acked Flush 0 Done Done Sent false
11672 4189 Acked Read 2 Done Done Sent false
11673 4190 Acked Read 7 Done Done Sent false
11674 4191 Acked Write 1 Done Done Sent false
11675 4192 Acked Write 1 Done Done Sent false
11676 4193 Acked Write 6 Done Done Sent false
11677 4194 Acked Write 6 Done Done Sent false
11678 4195 Acked Read 5 Done Done Sent false
11679 4196 Acked Read 9 Done Done Sent false
11680 4197 Acked Write 9 Done Done Sent false
11681 4198 Acked Flush 0 Done Done Sent false
11682 4199 Acked Read 6 Done Done Sent false
11683 4200 Acked Read 9 Done Done Sent false
11684 4201 Acked Read 6 Done Done Sent false
11685 4202 Acked Flush 0 Done Done Sent false
11686 4203 Acked Write 5 Done Done Sent false
11687 4204 Acked Read 8 Done Done Sent false
11688 4205 Acked Read 4 Done Done Sent false
11689 4206 Acked Read 7 Done Done Sent false
11690 4207 Acked Write 1 Done Done Sent false
11691 4208 Acked Flush 0 Done Done Sent false
11692 4209 Acked Read 4 Done Done Sent false
11693 4210 Acked Read 10 Done Done Sent false
11694 4211 Acked Read 7 Done Done Sent false
11695 4212 Acked Read 9 Done Done Sent false
11696 4213 Acked Read 10 Done Done Sent false
11697 4214 Acked Write 3 Done Done Sent false
11698 4215 Acked Read 6 Done Done Sent false
11699 4216 Acked Write 6 Done Done Sent false
11700 4217 Acked Read 7 Done Done Sent false
11701 4218 Acked Read 5 Done Done Sent false
11702 4219 Acked Read 1 Done Done Sent false
11703 4220 Acked Write 5 Done Done Sent false
11704 4221 Acked Write 10 Done Done Sent false
11705 4222 Acked Flush 0 Done Done Sent false
11706 4223 Acked Write 9 Done Done Sent false
11707 4224 Acked Read 10 Done Done Sent false
11708 4225 Acked Write 7 Done Done Sent false
11709 4226 Acked Write 7 Done Done Sent false
11710 4227 Acked Read 1 Done Done Sent false
11711 4228 Acked Write 3 Done Done Sent false
11712 4229 Acked Read 8 Done Done Sent false
11713 4230 Acked Flush 0 Done Done Sent false
11714 4231 Acked Write 8 Done Done Sent false
11715 4232 Acked Read 10 Done Done Sent false
11716 4233 Acked Read 5 Done Done Sent false
11717 4234 Acked Read 2 Done Done Sent false
11718 4235 Acked Read 3 Done Done Sent false
11719 4236 Acked Read 2 Done Done Sent false
11720 4237 Acked Read 6 Done Done Sent false
11721 4238 Acked Read 6 Done Done Sent false
11722 4239 Acked Write 1 Done Done Sent false
11723 4240 Acked Read 3 Done Done Sent false
11724 4241 Acked Read 7 Done Done Sent false
11725 4242 Acked Read 4 Done Done Sent false
11726 4243 Acked Write 7 Done Done Sent false
11727 4244 Acked Read 1 Done Done Sent false
11728 4245 Acked Write 1 Done Done Sent false
11729 4246 Acked Write 1 Done Done Sent false
11730 4247 Acked Read 5 Done Done Sent false
11731 4248 Acked Write 4 Done Done Sent false
11732 4249 Acked Read 10 Done Done Sent false
11733 4250 Acked Write 6 Done Done Sent false
11734 4251 Acked Read 4 Done Done Sent false
11735 4252 Acked Read 7 Done Done Sent false
11736 4253 Acked Flush 0 Done Done Sent false
11737 4254 Acked Write 10 Done Done Sent false
11738 4255 Acked Flush 0 Done Done Sent false
11739 4256 Acked Write 2 Done Done Sent false
11740 4257 Acked Read 4 Done Done Sent false
11741 4258 Acked Write 8 Done Done Sent false
11742 4259 Acked Flush 0 Done Done Sent false
11743 4260 Acked Read 2 Done Done Sent false
11744 4261 Acked Read 7 Done Done Sent false
11745 4262 Acked Write 5 Done Done Sent false
11746 4263 Acked Read 1 Done Done Sent false
11747 4264 Acked Read 3 Done Done Sent false
11748 4265 Acked Write 4 Done Done Sent false
11749 4266 Acked Read 2 Done Done Sent false
11750 4267 Acked Read 1 Done Done Sent false
11751 4268 Acked Write 9 Done Done Sent false
11752 4269 Acked Flush 0 Done Done Sent false
11753 4270 Acked Write 10 Done Done Sent false
11754 4271 Acked Flush 0 Done Done Sent false
11755 4272 Acked Read 9 Done Done Sent false
11756 4273 Acked Read 6 Done Done Sent false
11757 4274 Acked Read 7 Done Done Sent false
11758 4275 Acked Write 10 Done Done Sent false
11759 4276 Acked Read 8 Done Done Sent false
11760 4277 Acked Read 1 Done Done Sent false
11761 4278 Acked Write 3 Done Done Sent false
11762 4279 Acked Read 6 Done Done Sent false
11763 4280 Acked Flush 0 Done Done Sent false
11764 4281 Acked Read 8 Done Done Sent false
11765 4282 Acked Read 2 Done Done Sent false
11766 4283 Acked Read 7 Done Done Sent false
11767 4284 Acked Read 7 Done Done Sent false
11768 4285 Acked Flush 0 Done Done Sent false
11769 4286 Acked Write 8 Done Done Sent false
11770 4287 Acked Write 3 Done Done Sent false
11771 4288 Acked Read 3 Done Done Sent false
11772 4289 Acked Write 3 Done Done Sent false
11773 4290 Acked Write 3 Done Done Sent false
11774 4291 Acked Read 3 Done Done Sent false
11775 4292 Acked Read 3 Done Done Sent false
11776 4293 Acked Read 1 Done Done Sent false
11777 4294 Acked Write 8 Done Done Sent false
11778 4295 Acked Write 4 Done Done Sent false
11779 4296 Acked Flush 0 Done Done Sent false
11780 4297 Acked Read 7 Done Done Sent false
11781 4298 Acked Write 5 Done Done Sent false
11782 4299 Acked Read 9 Done Done Sent false
11783 4300 Acked Read 5 Done Done Sent false
11784 4301 Acked Read 6 Done Done Sent false
11785 4302 Acked Flush 0 Done Done Sent false
11786 4303 Acked Write 9 Done Done Sent false
11787 4304 Acked Flush 0 Done Done Sent false
11788 4305 Acked Read 7 Done Done Sent false
11789 4306 Acked Write 10 Done Done Sent false
11790 4307 Acked Read 7 Done Done Sent false
11791 4308 Acked Read 8 Done Done Sent false
11792 4309 Acked Read 3 Done Done Sent false
11793 4310 Acked Read 4 Done Done Sent false
11794 4311 Acked Read 6 Done Done Sent false
11795 4312 Acked Read 6 Done Done Sent false
11796 4313 Acked Read 7 Done Done Sent false
11797 4314 Acked Read 9 Done Done Sent false
11798 4315 Acked Read 2 Done Done Sent false
11799 4316 Acked Read 7 Done Done Sent false
11800 4317 Acked Read 1 Done Done Sent false
11801 4318 Acked Read 5 Done Done Sent false
11802 4319 Acked Read 2 Done Done Sent false
11803 STATES DS:0 DS:1 DS:2 TOTAL
11804 Sent 0 0 301 301
11805 Done 301 301 0 602
11806 Skipped 0 0 0 0
11807 Error 0 0 0 0
11808 Last Flush: 4304 4304 4018
11809 Downstairs last five completed: 4018 4017 4016 4015 4014
11810 Upstairs last five completed: 4319 4318 4317 4316 4315
11811 CLIENT: Up:0 ds:301 act:2
11812 ----------------------------------------------------------------
11813 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11814 States: Active Active Offline
11815 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11816 4019 Acked Read 6 Done Done Sent false
11817 4020 Acked Read 10 Done Done Sent false
11818 4021 Acked Write 6 Done Done Sent false
11819 4022 Acked Write 4 Done Done Sent false
11820 4023 Acked Read 8 Done Done Sent false
11821 4024 Acked Flush 0 Done Done Sent false
11822 4025 Acked Read 1 Done Done Sent false
11823 4026 Acked Flush 0 Done Done Sent false
11824 4027 Acked Read 4 Done Done Sent false
11825 4028 Acked Read 4 Done Done Sent false
11826 4029 Acked Write 10 Done Done Sent false
11827 4030 Acked Write 3 Done Done Sent false
11828 4031 Acked Read 10 Done Done Sent false
11829 4032 Acked Write 1 Done Done Sent false
11830 4033 Acked Read 8 Done Done Sent false
11831 4034 Acked Read 8 Done Done Sent false
11832 4035 Acked Write 3 Done Done Sent false
11833 4036 Acked Write 5 Done Done Sent false
11834 4037 Acked Read 3 Done Done Sent false
11835 4038 Acked Write 3 Done Done Sent false
11836 4039 Acked Write 5 Done Done Sent false
11837 4040 Acked Write 1 Done Done Sent false
11838 4041 Acked Read 7 Done Done Sent false
11839 4042 Acked Write 1 Done Done Sent false
11840 4043 Acked Write 3 Done Done Sent false
11841 4044 Acked Write 9 Done Done Sent false
11842 4045 Acked Write 2 Done Done Sent false
11843 4046 Acked Read 9 Done Done Sent false
11844 4047 Acked Read 5 Done Done Sent false
11845 4048 Acked Read 2 Done Done Sent false
11846 4049 Acked Read 4 Done Done Sent false
11847 4050 Acked Read 5 Done Done Sent false
11848 4051 Acked Write 8 Done Done Sent false
11849 4052 Acked Write 3 Done Done Sent false
11850 4053 Acked Read 1 Done Done Sent false
11851 4054 Acked Read 8 Done Done Sent false
11852 4055 Acked Read 10 Done Done Sent false
11853 4056 Acked Read 7 Done Done Sent false
11854 4057 Acked Flush 0 Done Done Sent false
11855 4058 Acked Read 9 Done Done Sent false
11856 4059 Acked Write 8 Done Done Sent false
11857 4060 Acked Write 10 Done Done Sent false
11858 4061 Acked Read 8 Done Done Sent false
11859 4062 Acked Read 5 Done Done Sent false
11860 4063 Acked Read 7 Done Done Sent false
11861 4064 Acked Read 5 Done Done Sent false
11862 4065 Acked Read 7 Done Done Sent false
11863 4066 Acked Write 2 Done Done Sent false
11864 4067 Acked Read 1 Done Done Sent false
11865 4068 Acked Write 10 Done Done Sent false
11866 4069 Acked Read 3 Done Done Sent false
11867 4070 Acked Flush 0 Done Done Sent false
11868 4071 Acked Write 9 Done Done Sent false
11869 4072 Acked Read 9 Done Done Sent false
11870 4073 Acked Write 1 Done Done Sent false
11871 4074 Acked Read 7 Done Done Sent false
11872 4075 Acked Read 6 Done Done Sent false
11873 4076 Acked Read 6 Done Done Sent false
11874 4077 Acked Write 1 Done Done Sent false
11875 4078 Acked Read 4 Done Done Sent false
11876 4079 Acked Read 3 Done Done Sent false
11877 4080 Acked Write 9 Done Done Sent false
11878 4081 Acked Write 9 Done Done Sent false
11879 4082 Acked Write 5 Done Done Sent false
11880 4083 Acked Read 5 Done Done Sent false
11881 4084 Acked Read 10 Done Done Sent false
11882 4085 Acked Write 9 Done Done Sent false
11883 4086 Acked Read 1 Done Done Sent false
11884 4087 Acked Write 5 Done Done Sent false
11885 4088 Acked Write 2 Done Done Sent false
11886 4089 Acked Flush 0 Done Done Sent false
11887 4090 Acked Read 1 Done Done Sent false
11888 4091 Acked Read 8 Done Done Sent false
11889 4092 Acked Read 3 Done Done Sent false
11890 4093 Acked Read 3 Done Done Sent false
11891 4094 Acked Flush 0 Done Done Sent false
11892 4095 Acked Flush 0 Done Done Sent false
11893 4096 Acked Read 4 Done Done Sent false
11894 4097 Acked Read 9 Done Done Sent false
11895 4098 Acked Read 8 Done Done Sent false
11896 4099 Acked Write 3 Done Done Sent false
11897 4100 Acked Read 4 Done Done Sent false
11898 4101 Acked Write 10 Done Done Sent false
11899 4102 Acked Read 6 Done Done Sent false
11900 4103 Acked Read 2 Done Done Sent false
11901 4104 Acked Write 6 Done Done Sent false
11902 4105 Acked Write 5 Done Done Sent false
11903 4106 Acked Read 2 Done Done Sent false
11904 4107 Acked Read 8 Done Done Sent false
11905 4108 Acked Read 4 Done Done Sent false
11906 4109 Acked Read 8 Done Done Sent false
11907 4110 Acked Write 10 Done Done Sent false
11908 4111 Acked Read 4 Done Done Sent false
11909 4112 Acked Read 8 Done Done Sent false
11910 4113 Acked Write 10 Done Done Sent false
11911 4114 Acked Read 6 Done Done Sent false
11912 4115 Acked Write 3 Done Done Sent false
11913 4116 Acked Write 6 Done Done Sent false
11914 4117 Acked Read 1 Done Done Sent false
11915 4118 Acked Write 6 Done Done Sent false
11916 4119 Acked Write 2 Done Done Sent false
11917 4120 Acked Write 6 Done Done Sent false
11918 4121 Acked Read 1 Done Done Sent false
11919 4122 Acked Read 10 Done Done Sent false
11920 4123 Acked Read 3 Done Done Sent false
11921 4124 Acked Read 6 Done Done Sent false
11922 4125 Acked Read 4 Done Done Sent false
11923 4126 Acked Read 1 Done Done Sent false
11924 4127 Acked Write 4 Done Done Sent false
11925 4128 Acked Read 3 Done Done Sent false
11926 4129 Acked Flush 0 Done Done Sent false
11927 4130 Acked Flush 0 Done Done Sent false
11928 4131 Acked Read 4 Done Done Sent false
11929 4132 Acked Read 10 Done Done Sent false
11930 4133 Acked Flush 0 Done Done Sent false
11931 4134 Acked Read 3 Done Done Sent false
11932 4135 Acked Read 8 Done Done Sent false
11933 4136 Acked Read 2 Done Done Sent false
11934 4137 Acked Read 2 Done Done Sent false
11935 4138 Acked Read 1 Done Done Sent false
11936 4139 Acked Write 6 Done Done Sent false
11937 4140 Acked Flush 0 Done Done Sent false
11938 4141 Acked Read 1 Done Done Sent false
11939 4142 Acked Read 9 Done Done Sent false
11940 4143 Acked Read 9 Done Done Sent false
11941 4144 Acked Flush 0 Done Done Sent false
11942 4145 Acked Write 6 Done Done Sent false
11943 4146 Acked Read 9 Done Done Sent false
11944 4147 Acked Write 4 Done Done Sent false
11945 4148 Acked Read 3 Done Done Sent false
11946 4149 Acked Write 8 Done Done Sent false
11947 4150 Acked Read 5 Done Done Sent false
11948 4151 Acked Read 5 Done Done Sent false
11949 4152 Acked Read 2 Done Done Sent false
11950 4153 Acked Write 7 Done Done Sent false
11951 4154 Acked Flush 0 Done Done Sent false
11952 4155 Acked Write 2 Done Done Sent false
11953 4156 Acked Read 9 Done Done Sent false
11954 4157 Acked Write 3 Done Done Sent false
11955 4158 Acked Read 3 Done Done Sent false
11956 4159 Acked Write 3 Done Done Sent false
11957 4160 Acked Read 10 Done Done Sent false
11958 4161 Acked Read 1 Done Done Sent false
11959 4162 Acked Read 4 Done Done Sent false
11960 4163 Acked Write 2 Done Done Sent false
11961 4164 Acked Write 9 Done Done Sent false
11962 4165 Acked Read 10 Done Done Sent false
11963 4166 Acked Read 8 Done Done Sent false
11964 4167 Acked Read 10 Done Done Sent false
11965 4168 Acked Write 9 Done Done Sent false
11966 4169 Acked Flush 0 Done Done Sent false
11967 4170 Acked Flush 0 Done Done Sent false
11968 4171 Acked Read 5 Done Done Sent false
11969 4172 Acked Read 2 Done Done Sent false
11970 4173 Acked Read 7 Done Done Sent false
11971 4174 Acked Write 7 Done Done Sent false
11972 4175 Acked Write 8 Done Done Sent false
11973 4176 Acked Write 4 Done Done Sent false
11974 4177 Acked Write 6 Done Done Sent false
11975 4178 Acked Read 6 Done Done Sent false
11976 4179 Acked Write 8 Done Done Sent false
11977 4180 Acked Read 4 Done Done Sent false
11978 4181 Acked Flush 0 Done Done Sent false
11979 4182 Acked Read 10 Done Done Sent false
11980 4183 Acked Write 2 Done Done Sent false
11981 4184 Acked Read 1 Done Done Sent false
11982 4185 Acked Read 2 Done Done Sent false
11983 4186 Acked Flush 0 Done Done Sent false
11984 4187 Acked Read 4 Done Done Sent false
11985 4188 Acked Flush 0 Done Done Sent false
11986 4189 Acked Read 2 Done Done Sent false
11987 4190 Acked Read 7 Done Done Sent false
11988 4191 Acked Write 1 Done Done Sent false
11989 4192 Acked Write 1 Done Done Sent false
11990 4193 Acked Write 6 Done Done Sent false
11991 4194 Acked Write 6 Done Done Sent false
11992 4195 Acked Read 5 Done Done Sent false
11993 4196 Acked Read 9 Done Done Sent false
11994 4197 Acked Write 9 Done Done Sent false
11995 4198 Acked Flush 0 Done Done Sent false
11996 4199 Acked Read 6 Done Done Sent false
11997 4200 Acked Read 9 Done Done Sent false
11998 4201 Acked Read 6 Done Done Sent false
11999 4202 Acked Flush 0 Done Done Sent false
12000 4203 Acked Write 5 Done Done Sent false
12001 4204 Acked Read 8 Done Done Sent false
12002 4205 Acked Read 4 Done Done Sent false
12003 4206 Acked Read 7 Done Done Sent false
12004 4207 Acked Write 1 Done Done Sent false
12005 4208 Acked Flush 0 Done Done Sent false
12006 4209 Acked Read 4 Done Done Sent false
12007 4210 Acked Read 10 Done Done Sent false
12008 4211 Acked Read 7 Done Done Sent false
12009 4212 Acked Read 9 Done Done Sent false
12010 4213 Acked Read 10 Done Done Sent false
12011 4214 Acked Write 3 Done Done Sent false
12012 4215 Acked Read 6 Done Done Sent false
12013 4216 Acked Write 6 Done Done Sent false
12014 4217 Acked Read 7 Done Done Sent false
12015 4218 Acked Read 5 Done Done Sent false
12016 4219 Acked Read 1 Done Done Sent false
12017 4220 Acked Write 5 Done Done Sent false
12018 4221 Acked Write 10 Done Done Sent false
12019 4222 Acked Flush 0 Done Done Sent false
12020 4223 Acked Write 9 Done Done Sent false
12021 4224 Acked Read 10 Done Done Sent false
12022 4225 Acked Write 7 Done Done Sent false
12023 4226 Acked Write 7 Done Done Sent false
12024 4227 Acked Read 1 Done Done Sent false
12025 4228 Acked Write 3 Done Done Sent false
12026 4229 Acked Read 8 Done Done Sent false
12027 4230 Acked Flush 0 Done Done Sent false
12028 4231 Acked Write 8 Done Done Sent false
12029 4232 Acked Read 10 Done Done Sent false
12030 4233 Acked Read 5 Done Done Sent false
12031 4234 Acked Read 2 Done Done Sent false
12032 4235 Acked Read 3 Done Done Sent false
12033 4236 Acked Read 2 Done Done Sent false
12034 4237 Acked Read 6 Done Done Sent false
12035 4238 Acked Read 6 Done Done Sent false
12036 4239 Acked Write 1 Done Done Sent false
12037 4240 Acked Read 3 Done Done Sent false
12038 4241 Acked Read 7 Done Done Sent false
12039 4242 Acked Read 4 Done Done Sent false
12040 4243 Acked Write 7 Done Done Sent false
12041 4244 Acked Read 1 Done Done Sent false
12042 4245 Acked Write 1 Done Done Sent false
12043 4246 Acked Write 1 Done Done Sent false
12044 4247 Acked Read 5 Done Done Sent false
12045 4248 Acked Write 4 Done Done Sent false
12046 4249 Acked Read 10 Done Done Sent false
12047 4250 Acked Write 6 Done Done Sent false
12048 4251 Acked Read 4 Done Done Sent false
12049 4252 Acked Read 7 Done Done Sent false
12050 4253 Acked Flush 0 Done Done Sent false
12051 4254 Acked Write 10 Done Done Sent false
12052 4255 Acked Flush 0 Done Done Sent false
12053 4256 Acked Write 2 Done Done Sent false
12054 4257 Acked Read 4 Done Done Sent false
12055 4258 Acked Write 8 Done Done Sent false
12056 4259 Acked Flush 0 Done Done Sent false
12057 4260 Acked Read 2 Done Done Sent false
12058 4261 Acked Read 7 Done Done Sent false
12059 4262 Acked Write 5 Done Done Sent false
12060 4263 Acked Read 1 Done Done Sent false
12061 4264 Acked Read 3 Done Done Sent false
12062 4265 Acked Write 4 Done Done Sent false
12063 4266 Acked Read 2 Done Done Sent false
12064 4267 Acked Read 1 Done Done Sent false
12065 4268 Acked Write 9 Done Done Sent false
12066 4269 Acked Flush 0 Done Done Sent false
12067 4270 Acked Write 10 Done Done Sent false
12068 4271 Acked Flush 0 Done Done Sent false
12069 4272 Acked Read 9 Done Done Sent false
12070 4273 Acked Read 6 Done Done Sent false
12071 4274 Acked Read 7 Done Done Sent false
12072 4275 Acked Write 10 Done Done Sent false
12073 4276 Acked Read 8 Done Done Sent false
12074 4277 Acked Read 1 Done Done Sent false
12075 4278 Acked Write 3 Done Done Sent false
12076 4279 Acked Read 6 Done Done Sent false
12077 4280 Acked Flush 0 Done Done Sent false
12078 4281 Acked Read 8 Done Done Sent false
12079 4282 Acked Read 2 Done Done Sent false
12080 4283 Acked Read 7 Done Done Sent false
12081 4284 Acked Read 7 Done Done Sent false
12082 4285 Acked Flush 0 Done Done Sent false
12083 4286 Acked Write 8 Done Done Sent false
12084 4287 Acked Write 3 Done Done Sent false
12085 4288 Acked Read 3 Done Done Sent false
12086 4289 Acked Write 3 Done Done Sent false
12087 4290 Acked Write 3 Done Done Sent false
12088 4291 Acked Read 3 Done Done Sent false
12089 4292 Acked Read 3 Done Done Sent false
12090 4293 Acked Read 1 Done Done Sent false
12091 4294 Acked Write 8 Done Done Sent false
12092 4295 Acked Write 4 Done Done Sent false
12093 4296 Acked Flush 0 Done Done Sent false
12094 4297 Acked Read 7 Done Done Sent false
12095 4298 Acked Write 5 Done Done Sent false
12096 4299 Acked Read 9 Done Done Sent false
12097 4300 Acked Read 5 Done Done Sent false
12098 4301 Acked Read 6 Done Done Sent false
12099 4302 Acked Flush 0 Done Done Sent false
12100 4303 Acked Write 9 Done Done Sent false
12101 4304 Acked Flush 0 Done Done Sent false
12102 4305 Acked Read 7 Done Done Sent false
12103 4306 Acked Write 10 Done Done Sent false
12104 4307 Acked Read 7 Done Done Sent false
12105 4308 Acked Read 8 Done Done Sent false
12106 4309 Acked Read 3 Done Done Sent false
12107 4310 Acked Read 4 Done Done Sent false
12108 4311 Acked Read 6 Done Done Sent false
12109 4312 Acked Read 6 Done Done Sent false
12110 4313 Acked Read 7 Done Done Sent false
12111 4314 Acked Read 9 Done Done Sent false
12112 4315 Acked Read 2 Done Done Sent false
12113 4316 Acked Read 7 Done Done Sent false
12114 4317 Acked Read 1 Done Done Sent false
12115 4318 Acked Read 5 Done Done Sent false
12116 4319 Acked Read 2 Done Done Sent false
12117 4320 Acked Flush 0 Done Done Sent false
12118 STATES DS:0 DS:1 DS:2 TOTAL
12119 Sent 0 0 302 302
12120 Done 302 302 0 604
12121 Skipped 0 0 0 0
12122 Error 0 0 0 0
12123 Last Flush: 4320 4320 4018
12124 Downstairs last five completed: 4018 4017 4016 4015 4014
12125 Upstairs last five completed: 4320 4319 4318 4317 4316
12126 CLIENT: Up:0 ds:302 act:2
121272025-01-13T15:54:19.552ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
121282025-01-13T15:54:19.552ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
121292025-01-13T15:54:19.552ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
121302025-01-13T15:54:19.552ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
121312025-01-13T15:54:19.552ZINFOcrucible: client task is exiting = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
12132 [8830] got start action
12133 [8830] I am going to restart
12134 Starting downstairs at port 8830
12135 Make output file at "/tmp/dsc/downstairs-8830.txt"
12136 [8830][2] reports Starting
12137 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1213)
12138 [8830][2] reports Running
12139 ----------------------------------------------------------------
12140 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
12141 States: Active Active Offline
12142 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12143 4019 Acked Read 6 Done Done Sent false
12144 4020 Acked Read 10 Done Done Sent false
12145 4021 Acked Write 6 Done Done Sent false
12146 4022 Acked Write 4 Done Done Sent false
12147 4023 Acked Read 8 Done Done Sent false
12148 4024 Acked Flush 0 Done Done Sent false
12149 4025 Acked Read 1 Done Done Sent false
12150 4026 Acked Flush 0 Done Done Sent false
12151 4027 Acked Read 4 Done Done Sent false
12152 4028 Acked Read 4 Done Done Sent false
12153 4029 Acked Write 10 Done Done Sent false
12154 4030 Acked Write 3 Done Done Sent false
12155 4031 Acked Read 10 Done Done Sent false
12156 4032 Acked Write 1 Done Done Sent false
12157 4033 Acked Read 8 Done Done Sent false
12158 4034 Acked Read 8 Done Done Sent false
12159 4035 Acked Write 3 Done Done Sent false
12160 4036 Acked Write 5 Done Done Sent false
12161 4037 Acked Read 3 Done Done Sent false
12162 4038 Acked Write 3 Done Done Sent false
12163 4039 Acked Write 5 Done Done Sent false
12164 4040 Acked Write 1 Done Done Sent false
12165 4041 Acked Read 7 Done Done Sent false
12166 4042 Acked Write 1 Done Done Sent false
12167 4043 Acked Write 3 Done Done Sent false
12168 4044 Acked Write 9 Done Done Sent false
12169 4045 Acked Write 2 Done Done Sent false
12170 4046 Acked Read 9 Done Done Sent false
12171 4047 Acked Read 5 Done Done Sent false
12172 4048 Acked Read 2 Done Done Sent false
12173 4049 Acked Read 4 Done Done Sent false
12174 4050 Acked Read 5 Done Done Sent false
12175 4051 Acked Write 8 Done Done Sent false
12176 4052 Acked Write 3 Done Done Sent false
12177 4053 Acked Read 1 Done Done Sent false
12178 4054 Acked Read 8 Done Done Sent false
12179 4055 Acked Read 10 Done Done Sent false
12180 4056 Acked Read 7 Done Done Sent false
12181 4057 Acked Flush 0 Done Done Sent false
12182 4058 Acked Read 9 Done Done Sent false
12183 4059 Acked Write 8 Done Done Sent false
12184 4060 Acked Write 10 Done Done Sent false
12185 4061 Acked Read 8 Done Done Sent false
12186 4062 Acked Read 5 Done Done Sent false
12187 4063 Acked Read 7 Done Done Sent false
12188 4064 Acked Read 5 Done Done Sent false
12189 4065 Acked Read 7 Done Done Sent false
12190 4066 Acked Write 2 Done Done Sent false
12191 4067 Acked Read 1 Done Done Sent false
12192 4068 Acked Write 10 Done Done Sent false
12193 4069 Acked Read 3 Done Done Sent false
12194 4070 Acked Flush 0 Done Done Sent false
12195 4071 Acked Write 9 Done Done Sent false
12196 4072 Acked Read 9 Done Done Sent false
12197 4073 Acked Write 1 Done Done Sent false
12198 4074 Acked Read 7 Done Done Sent false
12199 4075 Acked Read 6 Done Done Sent false
12200 4076 Acked Read 6 Done Done Sent false
12201 4077 Acked Write 1 Done Done Sent false
12202 4078 Acked Read 4 Done Done Sent false
12203 4079 Acked Read 3 Done Done Sent false
12204 4080 Acked Write 9 Done Done Sent false
12205 4081 Acked Write 9 Done Done Sent false
12206 4082 Acked Write 5 Done Done Sent false
12207 4083 Acked Read 5 Done Done Sent false
12208 4084 Acked Read 10 Done Done Sent false
12209 4085 Acked Write 9 Done Done Sent false
12210 4086 Acked Read 1 Done Done Sent false
12211 4087 Acked Write 5 Done Done Sent false
12212 4088 Acked Write 2 Done Done Sent false
12213 4089 Acked Flush 0 Done Done Sent false
12214 4090 Acked Read 1 Done Done Sent false
12215 4091 Acked Read 8 Done Done Sent false
12216 4092 Acked Read 3 Done Done Sent false
12217 4093 Acked Read 3 Done Done Sent false
12218 4094 Acked Flush 0 Done Done Sent false
12219 4095 Acked Flush 0 Done Done Sent false
12220 4096 Acked Read 4 Done Done Sent false
12221 4097 Acked Read 9 Done Done Sent false
12222 4098 Acked Read 8 Done Done Sent false
12223 4099 Acked Write 3 Done Done Sent false
12224 4100 Acked Read 4 Done Done Sent false
12225 4101 Acked Write 10 Done Done Sent false
12226 4102 Acked Read 6 Done Done Sent false
12227 4103 Acked Read 2 Done Done Sent false
12228 4104 Acked Write 6 Done Done Sent false
12229 4105 Acked Write 5 Done Done Sent false
12230 4106 Acked Read 2 Done Done Sent false
12231 4107 Acked Read 8 Done Done Sent false
12232 4108 Acked Read 4 Done Done Sent false
12233 4109 Acked Read 8 Done Done Sent false
12234 4110 Acked Write 10 Done Done Sent false
12235 4111 Acked Read 4 Done Done Sent false
12236 4112 Acked Read 8 Done Done Sent false
12237 4113 Acked Write 10 Done Done Sent false
12238 4114 Acked Read 6 Done Done Sent false
12239 4115 Acked Write 3 Done Done Sent false
12240 4116 Acked Write 6 Done Done Sent false
12241 4117 Acked Read 1 Done Done Sent false
12242 4118 Acked Write 6 Done Done Sent false
12243 4119 Acked Write 2 Done Done Sent false
12244 4120 Acked Write 6 Done Done Sent false
12245 4121 Acked Read 1 Done Done Sent false
12246 4122 Acked Read 10 Done Done Sent false
12247 4123 Acked Read 3 Done Done Sent false
12248 4124 Acked Read 6 Done Done Sent false
12249 4125 Acked Read 4 Done Done Sent false
12250 4126 Acked Read 1 Done Done Sent false
12251 4127 Acked Write 4 Done Done Sent false
12252 4128 Acked Read 3 Done Done Sent false
12253 4129 Acked Flush 0 Done Done Sent false
12254 4130 Acked Flush 0 Done Done Sent false
12255 4131 Acked Read 4 Done Done Sent false
12256 4132 Acked Read 10 Done Done Sent false
12257 4133 Acked Flush 0 Done Done Sent false
12258 4134 Acked Read 3 Done Done Sent false
12259 4135 Acked Read 8 Done Done Sent false
12260 4136 Acked Read 2 Done Done Sent false
12261 4137 Acked Read 2 Done Done Sent false
12262 4138 Acked Read 1 Done Done Sent false
12263 4139 Acked Write 6 Done Done Sent false
12264 4140 Acked Flush 0 Done Done Sent false
12265 4141 Acked Read 1 Done Done Sent false
12266 4142 Acked Read 9 Done Done Sent false
12267 4143 Acked Read 9 Done Done Sent false
12268 4144 Acked Flush 0 Done Done Sent false
12269 4145 Acked Write 6 Done Done Sent false
12270 4146 Acked Read 9 Done Done Sent false
12271 4147 Acked Write 4 Done Done Sent false
12272 4148 Acked Read 3 Done Done Sent false
12273 4149 Acked Write 8 Done Done Sent false
12274 4150 Acked Read 5 Done Done Sent false
12275 4151 Acked Read 5 Done Done Sent false
12276 4152 Acked Read 2 Done Done Sent false
12277 4153 Acked Write 7 Done Done Sent false
12278 4154 Acked Flush 0 Done Done Sent false
12279 4155 Acked Write 2 Done Done Sent false
12280 4156 Acked Read 9 Done Done Sent false
12281 4157 Acked Write 3 Done Done Sent false
12282 4158 Acked Read 3 Done Done Sent false
12283 4159 Acked Write 3 Done Done Sent false
12284 4160 Acked Read 10 Done Done Sent false
12285 4161 Acked Read 1 Done Done Sent false
12286 4162 Acked Read 4 Done Done Sent false
12287 4163 Acked Write 2 Done Done Sent false
12288 4164 Acked Write 9 Done Done Sent false
12289 4165 Acked Read 10 Done Done Sent false
12290 4166 Acked Read 8 Done Done Sent false
12291 4167 Acked Read 10 Done Done Sent false
12292 4168 Acked Write 9 Done Done Sent false
12293 4169 Acked Flush 0 Done Done Sent false
12294 4170 Acked Flush 0 Done Done Sent false
12295 4171 Acked Read 5 Done Done Sent false
12296 4172 Acked Read 2 Done Done Sent false
12297 4173 Acked Read 7 Done Done Sent false
12298 4174 Acked Write 7 Done Done Sent false
12299 4175 Acked Write 8 Done Done Sent false
12300 4176 Acked Write 4 Done Done Sent false
12301 4177 Acked Write 6 Done Done Sent false
12302 4178 Acked Read 6 Done Done Sent false
12303 4179 Acked Write 8 Done Done Sent false
12304 4180 Acked Read 4 Done Done Sent false
12305 4181 Acked Flush 0 Done Done Sent false
12306 4182 Acked Read 10 Done Done Sent false
12307 4183 Acked Write 2 Done Done Sent false
12308 4184 Acked Read 1 Done Done Sent false
12309 4185 Acked Read 2 Done Done Sent false
12310 4186 Acked Flush 0 Done Done Sent false
12311 4187 Acked Read 4 Done Done Sent false
12312 4188 Acked Flush 0 Done Done Sent false
12313 4189 Acked Read 2 Done Done Sent false
12314 4190 Acked Read 7 Done Done Sent false
12315 4191 Acked Write 1 Done Done Sent false
12316 4192 Acked Write 1 Done Done Sent false
12317 4193 Acked Write 6 Done Done Sent false
12318 4194 Acked Write 6 Done Done Sent false
12319 4195 Acked Read 5 Done Done Sent false
12320 4196 Acked Read 9 Done Done Sent false
12321 4197 Acked Write 9 Done Done Sent false
12322 4198 Acked Flush 0 Done Done Sent false
12323 4199 Acked Read 6 Done Done Sent false
12324 4200 Acked Read 9 Done Done Sent false
12325 4201 Acked Read 6 Done Done Sent false
12326 4202 Acked Flush 0 Done Done Sent false
12327 4203 Acked Write 5 Done Done Sent false
12328 4204 Acked Read 8 Done Done Sent false
12329 4205 Acked Read 4 Done Done Sent false
12330 4206 Acked Read 7 Done Done Sent false
12331 4207 Acked Write 1 Done Done Sent false
12332 4208 Acked Flush 0 Done Done Sent false
12333 4209 Acked Read 4 Done Done Sent false
12334 4210 Acked Read 10 Done Done Sent false
12335 4211 Acked Read 7 Done Done Sent false
12336 4212 Acked Read 9 Done Done Sent false
12337 4213 Acked Read 10 Done Done Sent false
12338 4214 Acked Write 3 Done Done Sent false
12339 4215 Acked Read 6 Done Done Sent false
12340 4216 Acked Write 6 Done Done Sent false
12341 4217 Acked Read 7 Done Done Sent false
12342 4218 Acked Read 5 Done Done Sent false
12343 4219 Acked Read 1 Done Done Sent false
12344 4220 Acked Write 5 Done Done Sent false
12345 4221 Acked Write 10 Done Done Sent false
12346 4222 Acked Flush 0 Done Done Sent false
12347 4223 Acked Write 9 Done Done Sent false
12348 4224 Acked Read 10 Done Done Sent false
12349 4225 Acked Write 7 Done Done Sent false
12350 4226 Acked Write 7 Done Done Sent false
12351 4227 Acked Read 1 Done Done Sent false
12352 4228 Acked Write 3 Done Done Sent false
12353 4229 Acked Read 8 Done Done Sent false
12354 4230 Acked Flush 0 Done Done Sent false
12355 4231 Acked Write 8 Done Done Sent false
12356 4232 Acked Read 10 Done Done Sent false
12357 4233 Acked Read 5 Done Done Sent false
12358 4234 Acked Read 2 Done Done Sent false
12359 4235 Acked Read 3 Done Done Sent false
12360 4236 Acked Read 2 Done Done Sent false
12361 4237 Acked Read 6 Done Done Sent false
12362 4238 Acked Read 6 Done Done Sent false
12363 4239 Acked Write 1 Done Done Sent false
12364 4240 Acked Read 3 Done Done Sent false
12365 4241 Acked Read 7 Done Done Sent false
12366 4242 Acked Read 4 Done Done Sent false
12367 4243 Acked Write 7 Done Done Sent false
12368 4244 Acked Read 1 Done Done Sent false
12369 4245 Acked Write 1 Done Done Sent false
12370 4246 Acked Write 1 Done Done Sent false
12371 4247 Acked Read 5 Done Done Sent false
12372 4248 Acked Write 4 Done Done Sent false
12373 4249 Acked Read 10 Done Done Sent false
12374 4250 Acked Write 6 Done Done Sent false
12375 4251 Acked Read 4 Done Done Sent false
12376 4252 Acked Read 7 Done Done Sent false
12377 4253 Acked Flush 0 Done Done Sent false
12378 4254 Acked Write 10 Done Done Sent false
12379 4255 Acked Flush 0 Done Done Sent false
12380 4256 Acked Write 2 Done Done Sent false
12381 4257 Acked Read 4 Done Done Sent false
12382 4258 Acked Write 8 Done Done Sent false
12383 4259 Acked Flush 0 Done Done Sent false
12384 4260 Acked Read 2 Done Done Sent false
12385 4261 Acked Read 7 Done Done Sent false
12386 4262 Acked Write 5 Done Done Sent false
12387 4263 Acked Read 1 Done Done Sent false
12388 4264 Acked Read 3 Done Done Sent false
12389 4265 Acked Write 4 Done Done Sent false
12390 4266 Acked Read 2 Done Done Sent false
12391 4267 Acked Read 1 Done Done Sent false
12392 4268 Acked Write 9 Done Done Sent false
12393 4269 Acked Flush 0 Done Done Sent false
12394 4270 Acked Write 10 Done Done Sent false
12395 4271 Acked Flush 0 Done Done Sent false
12396 4272 Acked Read 9 Done Done Sent false
12397 4273 Acked Read 6 Done Done Sent false
12398 4274 Acked Read 7 Done Done Sent false
12399 4275 Acked Write 10 Done Done Sent false
12400 4276 Acked Read 8 Done Done Sent false
12401 4277 Acked Read 1 Done Done Sent false
12402 4278 Acked Write 3 Done Done Sent false
12403 4279 Acked Read 6 Done Done Sent false
12404 4280 Acked Flush 0 Done Done Sent false
12405 4281 Acked Read 8 Done Done Sent false
12406 4282 Acked Read 2 Done Done Sent false
12407 4283 Acked Read 7 Done Done Sent false
12408 4284 Acked Read 7 Done Done Sent false
12409 4285 Acked Flush 0 Done Done Sent false
12410 4286 Acked Write 8 Done Done Sent false
12411 4287 Acked Write 3 Done Done Sent false
12412 4288 Acked Read 3 Done Done Sent false
12413 4289 Acked Write 3 Done Done Sent false
12414 4290 Acked Write 3 Done Done Sent false
12415 4291 Acked Read 3 Done Done Sent false
12416 4292 Acked Read 3 Done Done Sent false
12417 4293 Acked Read 1 Done Done Sent false
12418 4294 Acked Write 8 Done Done Sent false
12419 4295 Acked Write 4 Done Done Sent false
12420 4296 Acked Flush 0 Done Done Sent false
12421 4297 Acked Read 7 Done Done Sent false
12422 4298 Acked Write 5 Done Done Sent false
12423 4299 Acked Read 9 Done Done Sent false
12424 4300 Acked Read 5 Done Done Sent false
12425 4301 Acked Read 6 Done Done Sent false
12426 4302 Acked Flush 0 Done Done Sent false
12427 4303 Acked Write 9 Done Done Sent false
12428 4304 Acked Flush 0 Done Done Sent false
12429 4305 Acked Read 7 Done Done Sent false
12430 4306 Acked Write 10 Done Done Sent false
12431 4307 Acked Read 7 Done Done Sent false
12432 4308 Acked Read 8 Done Done Sent false
12433 4309 Acked Read 3 Done Done Sent false
12434 4310 Acked Read 4 Done Done Sent false
12435 4311 Acked Read 6 Done Done Sent false
12436 4312 Acked Read 6 Done Done Sent false
12437 4313 Acked Read 7 Done Done Sent false
12438 4314 Acked Read 9 Done Done Sent false
12439 4315 Acked Read 2 Done Done Sent false
12440 4316 Acked Read 7 Done Done Sent false
12441 4317 Acked Read 1 Done Done Sent false
12442 4318 Acked Read 5 Done Done Sent false
12443 4319 Acked Read 2 Done Done Sent false
12444 4320 Acked Flush 0 Done Done Sent false
12445 STATES DS:0 DS:1 DS:2 TOTAL
12446 Sent 0 0 302 302
12447 Done 302 302 0 604
12448 Skipped 0 0 0 0
12449 Error 0 0 0 0
12450 Last Flush: 4320 4320 4018
12451 Downstairs last five completed: 4018 4017 4016 4015 4014
12452 Upstairs last five completed: 4320 4319 4318 4317 4316
12453 CLIENT: Up:0 ds:302 act:2
12454 ----------------------------------------------------------------
12455 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
12456 States: Active Active Offline
12457 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12458 4019 Acked Read 6 Done Done Sent false
12459 4020 Acked Read 10 Done Done Sent false
12460 4021 Acked Write 6 Done Done Sent false
12461 4022 Acked Write 4 Done Done Sent false
12462 4023 Acked Read 8 Done Done Sent false
12463 4024 Acked Flush 0 Done Done Sent false
12464 4025 Acked Read 1 Done Done Sent false
12465 4026 Acked Flush 0 Done Done Sent false
12466 4027 Acked Read 4 Done Done Sent false
12467 4028 Acked Read 4 Done Done Sent false
12468 4029 Acked Write 10 Done Done Sent false
12469 4030 Acked Write 3 Done Done Sent false
12470 4031 Acked Read 10 Done Done Sent false
12471 4032 Acked Write 1 Done Done Sent false
12472 4033 Acked Read 8 Done Done Sent false
12473 4034 Acked Read 8 Done Done Sent false
12474 4035 Acked Write 3 Done Done Sent false
12475 4036 Acked Write 5 Done Done Sent false
12476 4037 Acked Read 3 Done Done Sent false
12477 4038 Acked Write 3 Done Done Sent false
12478 4039 Acked Write 5 Done Done Sent false
12479 4040 Acked Write 1 Done Done Sent false
12480 4041 Acked Read 7 Done Done Sent false
12481 4042 Acked Write 1 Done Done Sent false
12482 4043 Acked Write 3 Done Done Sent false
12483 4044 Acked Write 9 Done Done Sent false
12484 4045 Acked Write 2 Done Done Sent false
12485 4046 Acked Read 9 Done Done Sent false
12486 4047 Acked Read 5 Done Done Sent false
12487 4048 Acked Read 2 Done Done Sent false
12488 4049 Acked Read 4 Done Done Sent false
12489 4050 Acked Read 5 Done Done Sent false
12490 4051 Acked Write 8 Done Done Sent false
12491 4052 Acked Write 3 Done Done Sent false
12492 4053 Acked Read 1 Done Done Sent false
12493 4054 Acked Read 8 Done Done Sent false
12494 4055 Acked Read 10 Done Done Sent false
12495 4056 Acked Read 7 Done Done Sent false
12496 4057 Acked Flush 0 Done Done Sent false
12497 4058 Acked Read 9 Done Done Sent false
12498 4059 Acked Write 8 Done Done Sent false
12499 4060 Acked Write 10 Done Done Sent false
12500 4061 Acked Read 8 Done Done Sent false
12501 4062 Acked Read 5 Done Done Sent false
12502 4063 Acked Read 7 Done Done Sent false
12503 4064 Acked Read 5 Done Done Sent false
12504 4065 Acked Read 7 Done Done Sent false
12505 4066 Acked Write 2 Done Done Sent false
12506 4067 Acked Read 1 Done Done Sent false
12507 4068 Acked Write 10 Done Done Sent false
12508 4069 Acked Read 3 Done Done Sent false
12509 4070 Acked Flush 0 Done Done Sent false
12510 4071 Acked Write 9 Done Done Sent false
12511 4072 Acked Read 9 Done Done Sent false
12512 4073 Acked Write 1 Done Done Sent false
12513 4074 Acked Read 7 Done Done Sent false
12514 4075 Acked Read 6 Done Done Sent false
12515 4076 Acked Read 6 Done Done Sent false
12516 4077 Acked Write 1 Done Done Sent false
12517 4078 Acked Read 4 Done Done Sent false
12518 4079 Acked Read 3 Done Done Sent false
12519 4080 Acked Write 9 Done Done Sent false
12520 4081 Acked Write 9 Done Done Sent false
12521 4082 Acked Write 5 Done Done Sent false
12522 4083 Acked Read 5 Done Done Sent false
12523 4084 Acked Read 10 Done Done Sent false
12524 4085 Acked Write 9 Done Done Sent false
12525 4086 Acked Read 1 Done Done Sent false
12526 4087 Acked Write 5 Done Done Sent false
12527 4088 Acked Write 2 Done Done Sent false
12528 4089 Acked Flush 0 Done Done Sent false
12529 4090 Acked Read 1 Done Done Sent false
12530 4091 Acked Read 8 Done Done Sent false
12531 4092 Acked Read 3 Done Done Sent false
12532 4093 Acked Read 3 Done Done Sent false
12533 4094 Acked Flush 0 Done Done Sent false
12534 4095 Acked Flush 0 Done Done Sent false
12535 4096 Acked Read 4 Done Done Sent false
12536 4097 Acked Read 9 Done Done Sent false
12537 4098 Acked Read 8 Done Done Sent false
12538 4099 Acked Write 3 Done Done Sent false
12539 4100 Acked Read 4 Done Done Sent false
12540 4101 Acked Write 10 Done Done Sent false
12541 4102 Acked Read 6 Done Done Sent false
12542 4103 Acked Read 2 Done Done Sent false
12543 4104 Acked Write 6 Done Done Sent false
12544 4105 Acked Write 5 Done Done Sent false
12545 4106 Acked Read 2 Done Done Sent false
12546 4107 Acked Read 8 Done Done Sent false
12547 4108 Acked Read 4 Done Done Sent false
12548 4109 Acked Read 8 Done Done Sent false
12549 4110 Acked Write 10 Done Done Sent false
12550 4111 Acked Read 4 Done Done Sent false
12551 4112 Acked Read 8 Done Done Sent false
12552 4113 Acked Write 10 Done Done Sent false
12553 4114 Acked Read 6 Done Done Sent false
12554 4115 Acked Write 3 Done Done Sent false
12555 4116 Acked Write 6 Done Done Sent false
12556 4117 Acked Read 1 Done Done Sent false
12557 4118 Acked Write 6 Done Done Sent false
12558 4119 Acked Write 2 Done Done Sent false
12559 4120 Acked Write 6 Done Done Sent false
12560 4121 Acked Read 1 Done Done Sent false
12561 4122 Acked Read 10 Done Done Sent false
12562 4123 Acked Read 3 Done Done Sent false
12563 4124 Acked Read 6 Done Done Sent false
12564 4125 Acked Read 4 Done Done Sent false
12565 4126 Acked Read 1 Done Done Sent false
12566 4127 Acked Write 4 Done Done Sent false
12567 4128 Acked Read 3 Done Done Sent false
12568 4129 Acked Flush 0 Done Done Sent false
12569 4130 Acked Flush 0 Done Done Sent false
12570 4131 Acked Read 4 Done Done Sent false
12571 4132 Acked Read 10 Done Done Sent false
12572 4133 Acked Flush 0 Done Done Sent false
12573 4134 Acked Read 3 Done Done Sent false
12574 4135 Acked Read 8 Done Done Sent false
12575 4136 Acked Read 2 Done Done Sent false
12576 4137 Acked Read 2 Done Done Sent false
12577 4138 Acked Read 1 Done Done Sent false
12578 4139 Acked Write 6 Done Done Sent false
12579 4140 Acked Flush 0 Done Done Sent false
12580 4141 Acked Read 1 Done Done Sent false
12581 4142 Acked Read 9 Done Done Sent false
12582 4143 Acked Read 9 Done Done Sent false
12583 4144 Acked Flush 0 Done Done Sent false
12584 4145 Acked Write 6 Done Done Sent false
12585 4146 Acked Read 9 Done Done Sent false
12586 4147 Acked Write 4 Done Done Sent false
12587 4148 Acked Read 3 Done Done Sent false
12588 4149 Acked Write 8 Done Done Sent false
12589 4150 Acked Read 5 Done Done Sent false
12590 4151 Acked Read 5 Done Done Sent false
12591 4152 Acked Read 2 Done Done Sent false
12592 4153 Acked Write 7 Done Done Sent false
12593 4154 Acked Flush 0 Done Done Sent false
12594 4155 Acked Write 2 Done Done Sent false
12595 4156 Acked Read 9 Done Done Sent false
12596 4157 Acked Write 3 Done Done Sent false
12597 4158 Acked Read 3 Done Done Sent false
12598 4159 Acked Write 3 Done Done Sent false
12599 4160 Acked Read 10 Done Done Sent false
12600 4161 Acked Read 1 Done Done Sent false
12601 4162 Acked Read 4 Done Done Sent false
12602 4163 Acked Write 2 Done Done Sent false
12603 4164 Acked Write 9 Done Done Sent false
12604 4165 Acked Read 10 Done Done Sent false
12605 4166 Acked Read 8 Done Done Sent false
12606 4167 Acked Read 10 Done Done Sent false
12607 4168 Acked Write 9 Done Done Sent false
12608 4169 Acked Flush 0 Done Done Sent false
12609 4170 Acked Flush 0 Done Done Sent false
12610 4171 Acked Read 5 Done Done Sent false
12611 4172 Acked Read 2 Done Done Sent false
12612 4173 Acked Read 7 Done Done Sent false
12613 4174 Acked Write 7 Done Done Sent false
12614 4175 Acked Write 8 Done Done Sent false
12615 4176 Acked Write 4 Done Done Sent false
12616 4177 Acked Write 6 Done Done Sent false
12617 4178 Acked Read 6 Done Done Sent false
12618 4179 Acked Write 8 Done Done Sent false
12619 4180 Acked Read 4 Done Done Sent false
12620 4181 Acked Flush 0 Done Done Sent false
12621 4182 Acked Read 10 Done Done Sent false
12622 4183 Acked Write 2 Done Done Sent false
12623 4184 Acked Read 1 Done Done Sent false
12624 4185 Acked Read 2 Done Done Sent false
12625 4186 Acked Flush 0 Done Done Sent false
12626 4187 Acked Read 4 Done Done Sent false
12627 4188 Acked Flush 0 Done Done Sent false
12628 4189 Acked Read 2 Done Done Sent false
12629 4190 Acked Read 7 Done Done Sent false
12630 4191 Acked Write 1 Done Done Sent false
12631 4192 Acked Write 1 Done Done Sent false
12632 4193 Acked Write 6 Done Done Sent false
12633 4194 Acked Write 6 Done Done Sent false
12634 4195 Acked Read 5 Done Done Sent false
12635 4196 Acked Read 9 Done Done Sent false
12636 4197 Acked Write 9 Done Done Sent false
12637 4198 Acked Flush 0 Done Done Sent false
12638 4199 Acked Read 6 Done Done Sent false
12639 4200 Acked Read 9 Done Done Sent false
12640 4201 Acked Read 6 Done Done Sent false
12641 4202 Acked Flush 0 Done Done Sent false
12642 4203 Acked Write 5 Done Done Sent false
12643 4204 Acked Read 8 Done Done Sent false
12644 4205 Acked Read 4 Done Done Sent false
12645 4206 Acked Read 7 Done Done Sent false
12646 4207 Acked Write 1 Done Done Sent false
12647 4208 Acked Flush 0 Done Done Sent false
12648 4209 Acked Read 4 Done Done Sent false
12649 4210 Acked Read 10 Done Done Sent false
12650 4211 Acked Read 7 Done Done Sent false
12651 4212 Acked Read 9 Done Done Sent false
12652 4213 Acked Read 10 Done Done Sent false
12653 4214 Acked Write 3 Done Done Sent false
12654 4215 Acked Read 6 Done Done Sent false
12655 4216 Acked Write 6 Done Done Sent false
12656 4217 Acked Read 7 Done Done Sent false
12657 4218 Acked Read 5 Done Done Sent false
12658 4219 Acked Read 1 Done Done Sent false
12659 4220 Acked Write 5 Done Done Sent false
12660 4221 Acked Write 10 Done Done Sent false
12661 4222 Acked Flush 0 Done Done Sent false
12662 4223 Acked Write 9 Done Done Sent false
12663 4224 Acked Read 10 Done Done Sent false
12664 4225 Acked Write 7 Done Done Sent false
12665 4226 Acked Write 7 Done Done Sent false
12666 4227 Acked Read 1 Done Done Sent false
12667 4228 Acked Write 3 Done Done Sent false
12668 4229 Acked Read 8 Done Done Sent false
12669 4230 Acked Flush 0 Done Done Sent false
12670 4231 Acked Write 8 Done Done Sent false
12671 4232 Acked Read 10 Done Done Sent false
12672 4233 Acked Read 5 Done Done Sent false
12673 4234 Acked Read 2 Done Done Sent false
12674 4235 Acked Read 3 Done Done Sent false
12675 4236 Acked Read 2 Done Done Sent false
12676 4237 Acked Read 6 Done Done Sent false
12677 4238 Acked Read 6 Done Done Sent false
12678 4239 Acked Write 1 Done Done Sent false
12679 4240 Acked Read 3 Done Done Sent false
12680 4241 Acked Read 7 Done Done Sent false
12681 4242 Acked Read 4 Done Done Sent false
12682 4243 Acked Write 7 Done Done Sent false
12683 4244 Acked Read 1 Done Done Sent false
12684 4245 Acked Write 1 Done Done Sent false
12685 4246 Acked Write 1 Done Done Sent false
12686 4247 Acked Read 5 Done Done Sent false
12687 4248 Acked Write 4 Done Done Sent false
12688 4249 Acked Read 10 Done Done Sent false
12689 4250 Acked Write 6 Done Done Sent false
12690 4251 Acked Read 4 Done Done Sent false
12691 4252 Acked Read 7 Done Done Sent false
12692 4253 Acked Flush 0 Done Done Sent false
12693 4254 Acked Write 10 Done Done Sent false
12694 4255 Acked Flush 0 Done Done Sent false
12695 4256 Acked Write 2 Done Done Sent false
12696 4257 Acked Read 4 Done Done Sent false
12697 4258 Acked Write 8 Done Done Sent false
12698 4259 Acked Flush 0 Done Done Sent false
12699 4260 Acked Read 2 Done Done Sent false
12700 4261 Acked Read 7 Done Done Sent false
12701 4262 Acked Write 5 Done Done Sent false
12702 4263 Acked Read 1 Done Done Sent false
12703 4264 Acked Read 3 Done Done Sent false
12704 4265 Acked Write 4 Done Done Sent false
12705 4266 Acked Read 2 Done Done Sent false
12706 4267 Acked Read 1 Done Done Sent false
12707 4268 Acked Write 9 Done Done Sent false
12708 4269 Acked Flush 0 Done Done Sent false
12709 4270 Acked Write 10 Done Done Sent false
12710 4271 Acked Flush 0 Done Done Sent false
12711 4272 Acked Read 9 Done Done Sent false
12712 4273 Acked Read 6 Done Done Sent false
12713 4274 Acked Read 7 Done Done Sent false
12714 4275 Acked Write 10 Done Done Sent false
12715 4276 Acked Read 8 Done Done Sent false
12716 4277 Acked Read 1 Done Done Sent false
12717 4278 Acked Write 3 Done Done Sent false
12718 4279 Acked Read 6 Done Done Sent false
12719 4280 Acked Flush 0 Done Done Sent false
12720 4281 Acked Read 8 Done Done Sent false
12721 4282 Acked Read 2 Done Done Sent false
12722 4283 Acked Read 7 Done Done Sent false
12723 4284 Acked Read 7 Done Done Sent false
12724 4285 Acked Flush 0 Done Done Sent false
12725 4286 Acked Write 8 Done Done Sent false
12726 4287 Acked Write 3 Done Done Sent false
12727 4288 Acked Read 3 Done Done Sent false
12728 4289 Acked Write 3 Done Done Sent false
12729 4290 Acked Write 3 Done Done Sent false
12730 4291 Acked Read 3 Done Done Sent false
12731 4292 Acked Read 3 Done Done Sent false
12732 4293 Acked Read 1 Done Done Sent false
12733 4294 Acked Write 8 Done Done Sent false
12734 4295 Acked Write 4 Done Done Sent false
12735 4296 Acked Flush 0 Done Done Sent false
12736 4297 Acked Read 7 Done Done Sent false
12737 4298 Acked Write 5 Done Done Sent false
12738 4299 Acked Read 9 Done Done Sent false
12739 4300 Acked Read 5 Done Done Sent false
12740 4301 Acked Read 6 Done Done Sent false
12741 4302 Acked Flush 0 Done Done Sent false
12742 4303 Acked Write 9 Done Done Sent false
12743 4304 Acked Flush 0 Done Done Sent false
12744 4305 Acked Read 7 Done Done Sent false
12745 4306 Acked Write 10 Done Done Sent false
12746 4307 Acked Read 7 Done Done Sent false
12747 4308 Acked Read 8 Done Done Sent false
12748 4309 Acked Read 3 Done Done Sent false
12749 4310 Acked Read 4 Done Done Sent false
12750 4311 Acked Read 6 Done Done Sent false
12751 4312 Acked Read 6 Done Done Sent false
12752 4313 Acked Read 7 Done Done Sent false
12753 4314 Acked Read 9 Done Done Sent false
12754 4315 Acked Read 2 Done Done Sent false
12755 4316 Acked Read 7 Done Done Sent false
12756 4317 Acked Read 1 Done Done Sent false
12757 4318 Acked Read 5 Done Done Sent false
12758 4319 Acked Read 2 Done Done Sent false
12759 4320 Acked Flush 0 Done Done Sent false
12760 STATES DS:0 DS:1 DS:2 TOTAL
12761 Sent 0 0 302 302
12762 Done 302 302 0 604
12763 Skipped 0 0 0 0
12764 Error 0 0 0 0
12765 Last Flush: 4320 4320 4018
12766 Downstairs last five completed: 4018 4017 4016 4015 4014
12767 Upstairs last five completed: 4320 4319 4318 4317 4316
12768 CLIENT: Up:0 ds:302 act:2
127692025-01-13T15:54:29.554ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
127702025-01-13T15:54:29.554ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59131) = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
127712025-01-13T15:54:29.555ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
127722025-01-13T15:54:29.556ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
127732025-01-13T15:54:29.556ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
127742025-01-13T15:54:29.556ZINFOcrucible: send last flush ID to this DS: 4018 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
127752025-01-13T15:54:29.556ZINFOcrucible: Replied this last flush ID: 4018 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
127762025-01-13T15:54:29.556ZINFOcrucible: [2] client re-new 302 jobs since flush 4018 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
127772025-01-13T15:54:29.556ZINFOcrucible: [2] Replayed 302 jobs since flush: 4018 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
12778 ----------------------------------------------------------------
12779 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
12780 Downstairs last five completed: 4320 4319 4318 4317 4316
12781 Upstairs last five completed: 4320 4319 4318 4317 4316
12782 CLIENT: Up:0 ds:0 act:3
12783 Replay: All jobs finished
12784 Main task has work to do, go find it
12785 got dsc Stop(1)
12786 stop 1
12787 Jan 13 15:54:29.816 INFO request completed, latency_us: 717, response_code: 204, uri: /stop/cid/1, method: [8820] Got stop action so:false kr:false
12788 POST, req_id: ddcd39b9-c67a-4c71-8c79-d301fc08cfca, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
127892025-01-13T15:54:29.817ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
127902025-01-13T15:54:29.817ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
127912025-01-13T15:54:29.817ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
12792 Jan 13 15:54:29.817 INFO request completed, latency_us: 138, response_code: 200, uri: {"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-13T15:54:29.817814655Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
12793 /state/cid/1, method: GET, req_id: 59c04985-84c3-4b9b-843d-fa83d150f431, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
12794 [8820] Exited with: ExitStatus(unix_wait_status(9))
12795 [8820][1] reports Exit
12796 Jan 13 15:54:33.819 INFO request completed, latency_us: 348, response_code: 200, uri: /state/cid/1, method: GET, req_id: bfcf31af-9cc3-4f08-a6a6-8c081c04bc12, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
12797 001/300 Write block 3576 len 4096 data: 2
12798 002/300 Read block 4811 len 20480
12799 003/300 Write block 1376 len 32768 data: 3 5 4 5 5 5 6 7
12800 004/300 Read block 3453 len 4096
12801 005/300 Write block 2465 len 24576 data: 2 2 3 3 3 3
12802 006/300 Read block 3964 len 32768
12803 007/300 Read block 2017 len 8192
12804 008/300 Read block 1106 len 28672
12805 009/300 Write block 534 len 16384 data: 3 3 3 2
12806 010/300 Flush
12807 011/300 Write block 2149 len 8192 data: 5 5
12808 012/300 Write block 501 len 8192 data: 3 4
12809 013/300 Write block 4480 len 20480 data: 3 3 3 2 2
12810 014/300 Read block 4926 len 28672
12811 015/300 Write block 1993 len 8192 data: 3 3
12812 016/300 Write block 4152 len 4096 data: 3
12813 017/300 Read block 3912 len 32768
12814 018/300 Write block 4224 len 8192 data: 6 6
12815 019/300 Write block 2900 len 8192 data: 4 5
12816 020/300 Read block 1771 len 12288
12817 021/300 Read block 2937 len 36864
12818 022/300 Read block 4971 len 20480
12819 023/300 Read block 2028 len 24576
12820 024/300 Write block 897 len 8192 data: 3 3
12821 025/300 Read block 1866 len 4096
12822 026/300 Read block 348 len 36864
12823 027/300 Read block 196 len 16384
12824 028/300 Read block 3766 len 20480
12825 029/300 Write block 4613 len 4096 data: 2
12826 030/300 Read block 311 len 12288
12827 031/300 Write block 2889 len 24576 data: 3 3 3 4 4 5
12828 032/300 Write block 1388 len 32768 data: 6 5 4 3 4 3 3 2
12829 033/300 Write block 730 len 8192 data: 4 4
12830 034/300 Read block 240 len 20480
12831 035/300 Write block 3667 len 4096 data: 5
12832 036/300 Read block 4892 len 12288
12833 037/300 Read block 1790 len 8192
12834 038/300 Write block 18 len 8192 data: 3 2
12835 039/300 Flush
12836 040/300 Write block 1020 len 32768 data: 3 3 4 3 3 3 4 4
12837 041/300 Write block 1203 len 32768 data: 3 2 2 3 3 3 3 3
12838 042/300 Read block 1497 len 28672
12839 043/300 Read block 3309 len 16384
12840 044/300 Read block 4814 len 16384
12841 045/300 Write block 4659 len 40960 data: 3 4 4 3 4 4 4 3 2 2
12842 046/300 Read block 2171 len 24576
12843 047/300 Write block 2339 len 32768 data: 4 4 5 6 6 6 6 5
12844 048/300 Read block 3619 len 24576
12845 049/300 Read block 3878 len 20480
12846 050/300 Flush
12847 051/300 Write block 594 len 16384 data: 2 3 3 3
12848 052/300 Write block 3390 len 20480 data: 6 4 4 5 6
12849 053/300 Read block 4306 len 32768
12850 054/300 Write block 1819 len 16384 data: 6 4 4 3
12851 055/300 Write block 832 len 24576 data: 4 3 4 3 3 3
12852 056/300 Write block 3996 len 12288 data: 4 5 4
12853 057/300 Write block 4250 len 20480 data: 4 4 4 4 3
12854 058/300 Read block 2491 len 20480
12855 059/300 Write block 215 len 16384 data: 6 5 5 5
12856 060/300 Read block 1969 len 28672
12857 061/300 Read block 4216 len 12288
12858 062/300 Write block 3218 len 40960 data: 3 4 4 5 5 4 4 5 7 7
12859 063/300 Read block 356 len 40960
12860 064/300 Read block 409 len 12288
12861 065/300 Write block 4723 len 4096 data: 2
12862 066/300 Read block 2342 len 12288
12863 067/300 Write block 1271 len 4096 data: 4
12864 068/300 Read block 476 len 8192
12865 069/300 Read block 2475 len 32768
12866 070/300 Read block 2657 len 20480
12867 071/300 Write block 1379 len 28672 data: 6 6 6 7 8 6 7
12868 072/300 Read block 835 len 16384
12869 073/300 Read block 2404 len 4096
12870 074/300 Read block 4038 len 28672
12871 075/300 Read block 1091 len 8192
12872 076/300 Write block 3136 len 4096 data: 4
12873 077/300 Read block 1860 len 40960
12874 078/300 Flush
12875 079/300 Write block 1201 len 28672 data: 2 2 4 3 3 4 4
12876 080/300 Read block 484 len 20480
12877 081/300 Write block 825 len 20480 data: 5 4 2 3 3
12878 082/300 Write block 2485 len 28672 data: 5 6 3 3 3 3 3
12879 083/300 Read block 3248 len 8192
12880 084/300 Read block 2661 len 40960
12881 085/300 Read block 4992 len 20480
12882 086/300 Read block 3473 len 4096
12883 087/300 Write block 2122 len 36864 data: 5 4 4 4 4 4 3 3 2
12884 088/300 Read block 3977 len 16384
12885 089/300 Write block 2864 len 36864 data: 4 4 3 3 3 3 3 4 3
12886 090/300 Read block 1634 len 8192
12887 091/300 Write block 1952 len 4096 data: 2
12888 092/300 Read block 4002 len 4096
12889 093/300 Flush
12890 094/300 Read block 4678 len 32768
12891 095/300 Write block 3189 len 28672 data: 8 7 7 6 4 3 4
12892 096/300 Read block 4497 len 40960
12893 097/300 Read block 3151 len 40960
12894 098/300 Read block 2772 len 40960
12895 099/300 Write block 2780 len 16384 data: 3 3 4 4
12896 100/300 Read block 1834 len 12288
12897 101/300 Read block 4495 len 40960
12898 102/300 Read block 4133 len 20480
12899 103/300 Read block 4383 len 12288
12900 104/300 Read block 3632 len 36864
12901 105/300 Read block 4453 len 40960
12902 106/300 Read block 2778 len 36864
12903 107/300 Read block 2309 len 16384
12904 108/300 Write block 2329 len 28672 data: 2 2 2 3 3 3 3
12905 109/300 Write block 3760 len 32768 data: 3 3 2 2 2 2 2 3
12906 110/300 Read block 3218 len 24576
12907 111/300 Write block 3762 len 24576 data: 3 3 3 3 3 4
12908 112/300 Read block 2096 len 4096
12909 113/300 Write block 4657 len 12288 data: 3 3 4
12910 114/300 Read block 2163 len 4096
12911 115/300 Read block 4633 len 28672
12912 116/300 Read block 1912 len 12288
12913 117/300 Read block 1057 len 12288
12914 118/300 Write block 2108 len 16384 data: 3 4 4 4
12915 119/300 Flush
12916 120/300 Read block 911 len 28672
12917 121/300 Read block 3470 len 4096
12918 122/300 Write block 2531 len 36864 data: 2 2 2 2 2 2 2 2 2
12919 123/300 Read block 643 len 36864
12920 124/300 Write block 723 len 40960 data: 4 2 3 3 4 4 4 5 5 4
12921 125/300 Write block 3229 len 40960 data: 5 5 5 5 4 4 4 3 3 3
12922 126/300 Flush
12923 127/300 Read block 4825 len 8192
12924 128/300 Write block 3115 len 20480 data: 2 2 2 3 3
12925 129/300 Flush
12926 130/300 Read block 2267 len 12288
12927 131/300 Flush
12928 132/300 Write block 3073 len 36864 data: 5 4 4 5 4 4 4 4 3
12929 133/300 Flush
12930 134/300 Write block 1519 len 8192 data: 4 4
12931 135/300 Write block 1797 len 24576 data: 4 5 4 3 3 2
12932 136/300 Write block 3913 len 12288 data: 4 3 4
12933 137/300 Write block 1739 len 28672 data: 4 4 4 5 6 6 7
12934 138/300 Write block 1325 len 16384 data: 2 3 3 4
12935 139/300 Write block 2553 len 16384 data: 5 5 5 4
12936 140/300 Read block 4428 len 24576
12937 141/300 Read block 4893 len 16384
12938 142/300 Write block 4784 len 40960 data: 6 5 5 4 4 4 4 3 2 2
12939 143/300 Read block 3169 len 32768
12940 144/300 Write block 3530 len 36864 data: 3 2 2 2 2 2 2 2 2
12941 145/300 Write block 1024 len 28672 data: 4 4 5 5 4 3 4
12942 146/300 Write block 613 len 16384 data: 2 2 3 3
12943 147/300 Write block 27 len 16384 data: 6 5 5 6
12944 148/300 Write block 2036 len 4096 data: 2
12945 149/300 Read block 2533 len 36864
12946 150/300 Flush
12947 151/300 Flush
12948 152/300 Read block 3108 len 4096
12949 153/300 Write block 241 len 24576 data: 4 3 2 2 2 2
12950 154/300 Flush
12951 155/300 Write block 4256 len 12288 data: 2 3 3
12952 156/300 Read block 55 len 24576
12953 157/300 Write block 110 len 4096 data: 3
12954 158/300 Read block 3938 len 32768
12955 159/300 Read block 2375 len 20480
12956 160/300 Read block 1978 len 16384
12957 161/300 Write block 63 len 40960 data: 5 5 3 3 3 3 4 4 2 2
12958 162/300 Read block 1333 len 32768
12959 163/300 Read block 3854 len 8192
12960 164/300 Read block 4592 len 12288
12961 165/300 Read block 1830 len 8192
12962 166/300 Read block 1362 len 36864
12963 167/300 Write block 1336 len 16384 data: 8 8 7 6
12964 168/300 Write block 768 len 32768 data: 5 5 5 4 3 4 4 4
12965 169/300 Read block 2856 len 36864
12966 170/300 Read block 58 len 24576
12967 171/300 Flush
12968 172/300 Read block 4108 len 40960
12969 173/300 Read block 2277 len 8192
12970 174/300 Write block 4966 len 16384 data: 3 4 5 4
12971 175/300 Read block 4320 len 20480
12972 176/300 Read block 874 len 20480
12973 177/300 Flush
12974 178/300 Read block 19 len 4096
12975 179/300 Flush
12976 180/300 Write block 4266 len 28672 data: 3 3 4 4 4 4 4
12977 181/300 Read block 4972 len 4096
12978 182/300 Read block 417 len 16384
12979 183/300 Write block 4044 len 12288 data: 2 2 2
12980 184/300 Write block 58 len 16384 data: 4 4 4 5
12981 185/300 Write block 4176 len 16384 data: 3 3 2 2
12982 186/300 Write block 4986 len 8192 data: 3 4
12983 187/300 Read block 1505 len 8192
12984 188/300 Read block 4173 len 4096
12985 189/300 Read block 2273 len 20480
12986 190/300 Read block 3671 len 40960
12987 191/300 Write block 1047 len 12288 data: 4 5 6
12988 192/300 Read block 413 len 24576
12989 193/300 Write block 2818 len 40960 data: 4 4 5 5 4 4 4 4 4 4
12990 194/300 Write block 3432 len 32768 data: 2 2 2 3 3 4 4 4
12991 195/300 Write block 3181 len 24576 data: 2 2 2 2 3 4
12992 196/300 Write block 4243 len 20480 data: 3 3 3 3 2
12993 197/300 Read block 2436 len 40960
12994 198/300 Flush
12995 199/300 Read block 1174 len 24576
12996 200/300 Read block 1568 len 16384
12997 201/300 Read block 2549 len 32768
12998 202/300 Flush
12999 203/300 Read block 214 len 24576
13000 204/300 Write block 371 len 32768 data: 2 3 4 4 6 6 6 6
13001 205/300 Read block 2150 len 16384
13002 206/300 Write block 1616 len 32768 data: 5 5 4 4 4 5 4 4
13003 207/300 Flush
13004 208/300 Read block 151 len 36864
13005 209/300 Read block 2626 len 28672
13006 210/300 Read block 3695 len 36864
13007 211/300 Write block 3216 len 16384 data: 2 3 4 5
13008 212/300 Read block 2809 len 4096
13009 213/300 Read block 2918 len 20480
13010 214/300 Read block 2686 len 20480
13011 215/300 Read block 2235 len 36864
13012 216/300 Read block 2628 len 16384
13013 217/300 Read block 2303 len 40960
13014 218/300 Read block 2376 len 8192
13015 219/300 Read block 3355 len 16384
13016 220/300 Read block 4001 len 20480
13017 221/300 Read block 4297 len 32768
13018 222/300 Read block 1113 len 36864
13019 223/300 Flush
13020 224/300 Read block 4271 len 36864
13021 225/300 Write block 4189 len 12288 data: 4 4 4
13022 226/300 Read block 4378 len 36864
13023 227/300 Write block 2682 len 32768 data: 2 3 3 2 2 3 3 3
13024 228/300 Read block 4616 len 36864
13025 229/300 Write block 2181 len 8192 data: 4 4
13026 230/300 Write block 3519 len 40960 data: 4 2 2 2 3 3 3 3 3 3
13027 231/300 Read block 2719 len 8192
13028 232/300 Write block 1296 len 4096 data: 5
13029 233/300 Write block 3854 len 36864 data: 3 2 2 3 5 5 5 5 5
13030 234/300 Read block 1610 len 28672
13031 235/300 Read block 4035 len 12288
13032 236/300 Read block 2137 len 12288
13033 237/300 Write block 685 len 20480 data: 3 3 3 3 2
13034 238/300 Read block 462 len 32768
13035 239/300 Write block 4216 len 20480 data: 6 6 5 5 5
13036 240/300 Write block 375 len 24576 data: 7 7 7 7 5 5
13037 241/300 Read block 2123 len 16384
13038 242/300 Read block 27 len 32768
13039 243/300 Read block 1389 len 40960
13040 244/300 Read block 4634 len 32768
13041 245/300 Read block 3590 len 36864
13042 246/300 Write block 4086 len 24576 data: 3 2 2 2 2 2
13043 247/300 Read block 2817 len 40960
13044 248/300 Read block 1896 len 32768
13045 249/300 Read block 118 len 36864
13046 250/300 Read block 3711 len 12288
13047 251/300 Write block 1699 len 12288 data: 4 3 2
13048 252/300 Write block 4829 len 8192 data: 2 2
13049 253/300 Read block 4878 len 28672
13050 254/300 Read block 4125 len 12288
13051 255/300 Read block 810 len 12288
13052 256/300 Write block 4704 len 36864 data: 2 3 3 3 3 3 3 3 3
13053 257/300 Write block 4936 len 36864 data: 5 5 3 2 3 4 4 4 5
13054 258/300 Read block 1758 len 24576
13055 259/300 Write block 2208 len 32768 data: 3 5 6 5 5 5 7 7
13056 260/300 Read block 3617 len 40960
13057 261/300 Write block 1940 len 8192 data: 3 3
13058 262/300 Write block 694 len 12288 data: 3 3 4
13059 263/300 Flush
13060 264/300 Write block 2953 len 4096 data: 5
13061 265/300 Write block 4018 len 12288 data: 2 3 3
13062 266/300 Write block 3349 len 16384 data: 4 3 4 3
13063 267/300 Read block 4636 len 24576
13064 268/300 Read block 117 len 28672
13065 269/300 Read block 3854 len 16384
13066 270/300 Read block 4658 len 24576
13067 271/300 Read block 3341 len 32768
13068 272/300 Write block 2381 len 24576 data: 3 3 4 3 3 3
13069 273/300 Write block 1538 len 28672 data: 4 3 3 2 2 2 2
13070 274/300 Read block 1655 len 12288
13071 275/300 Write block 4301 len 28672 data: 7 6 7 6 6 4 3
13072 276/300 Read block 3029 len 28672
13073 277/300 Flush
13074 278/300 Write block 123 len 16384 data: 5 4 4 4
13075 279/300 Read block 212 len 36864
13076 280/300 Read block 4409 len 36864
13077 281/300 Write block 4327 len 8192 data: 5 3
13078 282/300 Write block 3996 len 8192 data: 5 6
13079 283/300 Flush
13080 284/300 Read block 1635 len 28672
13081 285/300 Read block 410 len 36864
13082 286/300 Read block 281 len 32768
13083 287/300 Read block 276 len 20480
13084 288/300 Write block 4537 len 16384 data: 3 2 2 2
13085 289/300 Write block 3320 len 12288 data: 2 3 3
13086 290/300 Flush
13087 291/300 Write block 1086 len 32768 data: 5 4 5 4 4 3 2 2
13088 292/300 Read block 3688 len 28672
13089 293/300 Write block 1084 len 16384 data: 5 5 6 5
13090 294/300 Write block 471 len 36864 data: 2 2 2 4 4 4 5 4 4
13091 295/300 Flush
13092 296/300 Flush
13093 297/300 Read block 1814 len 16384
13094 298/300 Write block 588 len 16384 data: 4 3 2 2
13095 299/300 Write block 1018 len 20480 data: 2 2 4 4 5
13096 300/300 Write block 2015 len 40960 data: 2 2 2 3 5 4 4 6 7 8
13097 301/300 Read block 3580 len 16384
13098 Main task has work to do, go find it
13099 got dsc Start(1)
13100 start 1
13101 Jan 13 15:54:34.063 INFO request completed, latency_us: 215, response_code: 204, uri: /start/cid/1, method: POST, req_id: ac5db8bb-1f4e-4ebc-b1d1-9ea802e4312b, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
13102 Replay: started 1, returned:Ok(())
13103 ----------------------------------------------------------------
13104 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
13105 States: Active Offline Active
13106 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13107 4321 Acked Write 1 Done Sent Done false
13108 4322 Acked Read 5 Done Sent Done false
13109 4323 Acked Write 8 Done Sent Done false
13110 4324 Acked Read 1 Done Sent Done false
13111 4325 Acked Write 6 Done Sent Done false
13112 4326 Acked Read 8 Done Sent Done false
13113 4327 Acked Read 2 Done Sent Done false
13114 4328 Acked Read 7 Done Sent Done false
13115 4329 Acked Write 4 Done Sent Done false
13116 4330 Acked Flush 0 Done Sent Done false
13117 4331 Acked Write 2 Done Sent Done false
13118 4332 Acked Write 2 Done Sent Done false
13119 4333 Acked Write 5 Done Sent Done false
13120 4334 Acked Read 7 Done Sent Done false
13121 4335 Acked Write 2 Done Sent Done false
13122 4336 Acked Write 1 Done Sent Done false
13123 4337 Acked Read 8 Done Sent Done false
13124 4338 Acked Write 2 Done Sent Done false
13125 4339 Acked Write 2 Done Sent Done false
13126 4340 Acked Read 3 Done Sent Done false
13127 4341 Acked Read 9 Done Sent Done false
13128 4342 Acked Read 5 Done Sent Done false
13129 4343 Acked Read 6 Done Sent Done false
13130 4344 Acked Write 2 Done Sent Done false
13131 4345 Acked Read 1 Done Sent Done false
13132 4346 Acked Read 9 Done Sent Done false
13133 4347 Acked Read 4 Done Sent Done false
13134 4348 Acked Read 5 Done Sent Done false
13135 4349 Acked Write 1 Done Sent Done false
13136 4350 Acked Read 3 Done Sent Done false
13137 4351 Acked Write 6 Done Sent Done false
13138 4352 Acked Write 8 Done Sent Done false
13139 4353 Acked Write 2 Done Sent Done false
13140 4354 Acked Read 5 Done Sent Done false
13141 4355 Acked Write 1 Done Sent Done false
13142 4356 Acked Read 3 Done Sent Done false
13143 4357 Acked Read 2 Done Sent Done false
13144 4358 Acked Write 2 Done Sent Done false
13145 4359 Acked Flush 0 Done Sent Done false
13146 4360 Acked Write 8 Done Sent Done false
13147 4361 Acked Write 8 Done Sent Done false
13148 4362 Acked Read 7 Done Sent Done false
13149 4363 Acked Read 4 Done Sent Done false
13150 4364 Acked Read 4 Done Sent Done false
13151 4365 Acked Write 10 Done Sent Done false
13152 4366 Acked Read 6 Done Sent Done false
13153 4367 Acked Write 8 Done Sent Done false
13154 4368 Acked Read 6 Done Sent Done false
13155 4369 Acked Read 5 Done Sent Done false
13156 4370 Acked Flush 0 Done Sent Done false
13157 4371 Acked Write 4 Done Sent Done false
13158 4372 Acked Write 5 Done Sent Done false
13159 4373 Acked Read 8 Done Sent Done false
13160 4374 Acked Write 4 Done Sent Done false
13161 4375 Acked Write 6 Done Sent Done false
13162 4376 Acked Write 3 Done Sent Done false
13163 4377 Acked Write 5 Done Sent Done false
13164 4378 Acked Read 5 Done Sent Done false
13165 4379 Acked Write 4 Done Sent Done false
13166 4380 Acked Read 7 Done Sent Done false
13167 4381 Acked Read 3 Done Sent Done false
13168 4382 Acked Write 10 Done Sent Done false
13169 4383 Acked Read 10 Done Sent Done false
13170 4384 Acked Read 3 Done Sent Done false
13171 4385 Acked Write 1 Done Sent Done false
13172 4386 Acked Read 3 Done Sent Done false
13173 4387 Acked Write 1 Done Sent Done false
13174 4388 Acked Read 2 Done Sent Done false
13175 4389 Acked Read 8 Done Sent Done false
13176 4390 Acked Read 5 Done Sent Done false
13177 4391 Acked Write 7 Done Sent Done false
13178 4392 Acked Read 4 Done Sent Done false
13179 4393 Acked Read 1 Done Sent Done false
13180 4394 Acked Read 7 Done Sent Done false
13181 4395 Acked Read 2 Done Sent Done false
13182 4396 Acked Write 1 Done Sent Done false
13183 4397 Acked Read 10 Done Sent Done false
13184 4398 Acked Flush 0 Done Sent Done false
13185 4399 Acked Write 7 Done Sent Done false
13186 4400 Acked Read 5 Done Sent Done false
13187 4401 Acked Write 5 Done Sent Done false
13188 4402 Acked Write 7 Done Sent Done false
13189 4403 Acked Read 2 Done Sent Done false
13190 4404 Acked Read 10 Done Sent Done false
13191 4405 Acked Read 5 Done Sent Done false
13192 4406 Acked Read 1 Done Sent Done false
13193 4407 Acked Write 9 Done Sent Done false
13194 4408 Acked Read 4 Done Sent Done false
13195 4409 Acked Write 9 Done Sent Done false
13196 4410 Acked Read 2 Done Sent Done false
13197 4411 Acked Write 1 Done Sent Done false
13198 4412 Acked Read 1 Done Sent Done false
13199 4413 Acked Flush 0 Done Sent Done false
13200 4414 Acked Read 8 Done Sent Done false
13201 4415 Acked Write 7 Done Sent Done false
13202 4416 Acked Read 10 Done Sent Done false
13203 4417 Acked Read 10 Done Sent Done false
13204 4418 Acked Read 10 Done Sent Done false
13205 4419 Acked Write 4 Done Sent Done false
13206 4420 Acked Read 3 Done Sent Done false
13207 4421 Acked Read 10 Done Sent Done false
13208 4422 Acked Read 5 Done Sent Done false
13209 4423 Acked Read 3 Done Sent Done false
13210 4424 Acked Read 9 Done Sent Done false
13211 4425 Acked Read 10 Done Sent Done false
13212 4426 Acked Read 9 Done Sent Done false
13213 4427 Acked Read 4 Done Sent Done false
13214 4428 Acked Write 7 Done Sent Done false
13215 4429 Acked Write 8 Done Sent Done false
13216 4430 Acked Read 6 Done Sent Done false
13217 4431 Acked Write 6 Done Sent Done false
13218 4432 Acked Read 1 Done Sent Done false
13219 4433 Acked Write 3 Done Sent Done false
13220 4434 Acked Read 1 Done Sent Done false
13221 4435 Acked Read 7 Done Sent Done false
13222 4436 Acked Read 3 Done Sent Done false
13223 4437 Acked Read 3 Done Sent Done false
13224 4438 Acked Write 4 Done Sent Done false
13225 4439 Acked Flush 0 Done Sent Done false
13226 4440 Acked Read 7 Done Sent Done false
13227 4441 Acked Read 1 Done Sent Done false
13228 4442 Acked Write 9 Done Sent Done false
13229 4443 Acked Read 9 Done Sent Done false
13230 4444 Acked Write 10 Done Sent Done false
13231 4445 Acked Write 10 Done Sent Done false
13232 4446 Acked Flush 0 Done Sent Done false
13233 4447 Acked Read 2 Done Sent Done false
13234 4448 Acked Write 5 Done Sent Done false
13235 4449 Acked Flush 0 Done Sent Done false
13236 4450 Acked Read 3 Done Sent Done false
13237 4451 Acked Flush 0 Done Sent Done false
13238 4452 Acked Write 9 Done Sent Done false
13239 4453 Acked Flush 0 Done Sent Done false
13240 4454 Acked Write 2 Done Sent Done false
13241 4455 Acked Write 6 Done Sent Done false
13242 4456 Acked Write 3 Done Sent Done false
13243 4457 Acked Write 7 Done Sent Done false
13244 4458 Acked Write 4 Done Sent Done false
13245 4459 Acked Write 4 Done Sent Done false
13246 4460 Acked Read 6 Done Sent Done false
13247 4461 Acked Read 4 Done Sent Done false
13248 4462 Acked Write 10 Done Sent Done false
13249 4463 Acked Read 8 Done Sent Done false
13250 4464 Acked Write 9 Done Sent Done false
13251 4465 Acked Write 7 Done Sent Done false
13252 4466 Acked Write 4 Done Sent Done false
13253 4467 Acked Write 4 Done Sent Done false
13254 4468 Acked Write 1 Done Sent Done false
13255 4469 Acked Read 9 Done Sent Done false
13256 4470 Acked Flush 0 Done Sent Done false
13257 4471 Acked Flush 0 Done Sent Done false
13258 4472 Acked Read 1 Done Sent Done false
13259 4473 Acked Write 6 Done Sent Done false
13260 4474 Acked Flush 0 Done Sent Done false
13261 4475 Acked Write 3 Done Sent Done false
13262 4476 Acked Read 6 Done Sent Done false
13263 4477 Acked Write 1 Done Sent Done false
13264 4478 Acked Read 8 Done Sent Done false
13265 4479 Acked Read 5 Done Sent Done false
13266 4480 Acked Read 4 Done Sent Done false
13267 4481 Acked Write 10 Done Sent Done false
13268 4482 Acked Read 8 Done Sent Done false
13269 4483 Acked Read 2 Done Sent Done false
13270 4484 Acked Read 3 Done Sent Done false
13271 4485 Acked Read 2 Done Sent Done false
13272 4486 Acked Read 9 Done Sent Done false
13273 4487 Acked Write 4 Done Sent Done false
13274 4488 Acked Write 8 Done Sent Done false
13275 4489 Acked Read 9 Done Sent Done false
13276 4490 Acked Read 6 Done Sent Done false
13277 4491 Acked Flush 0 Done Sent Done false
13278 4492 Acked Read 10 Done Sent Done false
13279 4493 Acked Read 2 Done Sent Done false
13280 4494 Acked Write 4 Done Sent Done false
13281 4495 Acked Read 5 Done Sent Done false
13282 4496 Acked Read 5 Done Sent Done false
13283 4497 Acked Flush 0 Done Sent Done false
13284 4498 Acked Read 1 Done Sent Done false
13285 4499 Acked Flush 0 Done Sent Done false
13286 4500 Acked Write 7 Done Sent Done false
13287 4501 Acked Read 1 Done Sent Done false
13288 4502 Acked Read 4 Done Sent Done false
13289 4503 Acked Write 3 Done Sent Done false
13290 4504 Acked Write 4 Done Sent Done false
13291 4505 Acked Write 4 Done Sent Done false
13292 4506 Acked Write 2 Done Sent Done false
13293 4507 Acked Read 2 Done Sent Done false
13294 4508 Acked Read 1 Done Sent Done false
13295 4509 Acked Read 5 Done Sent Done false
13296 4510 Acked Read 10 Done Sent Done false
13297 4511 Acked Write 3 Done Sent Done false
13298 4512 Acked Read 6 Done Sent Done false
13299 4513 Acked Write 10 Done Sent Done false
13300 4514 Acked Write 8 Done Sent Done false
13301 4515 Acked Write 6 Done Sent Done false
13302 4516 Acked Write 5 Done Sent Done false
13303 4517 Acked Read 10 Done Sent Done false
13304 4518 Acked Flush 0 Done Sent Done false
13305 4519 Acked Read 6 Done Sent Done false
13306 4520 Acked Read 4 Done Sent Done false
13307 4521 Acked Read 8 Done Sent Done false
13308 4522 Acked Flush 0 Done Sent Done false
13309 4523 Acked Read 6 Done Sent Done false
13310 4524 Acked Write 8 Done Sent Done false
13311 4525 Acked Read 4 Done Sent Done false
13312 4526 Acked Write 8 Done Sent Done false
13313 4527 Acked Flush 0 Done Sent Done false
13314 4528 Acked Read 9 Done Sent Done false
13315 4529 Acked Read 7 Done Sent Done false
13316 4530 Acked Read 9 Done Sent Done false
13317 4531 Acked Write 4 Done Sent Done false
13318 4532 Acked Read 1 Done Sent Done false
13319 4533 Acked Read 5 Done Sent Done false
13320 4534 Acked Read 5 Done Sent Done false
13321 4535 Acked Read 9 Done Sent Done false
13322 4536 Acked Read 4 Done Sent Done false
13323 4537 Acked Read 10 Done Sent Done false
13324 4538 Acked Read 2 Done Sent Done false
13325 4539 Acked Read 4 Done Sent Done false
13326 4540 Acked Read 5 Done Sent Done false
13327 4541 Acked Read 8 Done Sent Done false
13328 4542 Acked Read 9 Done Sent Done false
13329 4543 Acked Flush 0 Done Sent Done false
13330 4544 Acked Read 9 Done Sent Done false
13331 4545 Acked Write 3 Done Sent Done false
13332 4546 Acked Read 9 Done Sent Done false
13333 4547 Acked Write 8 Done Sent Done false
13334 4548 Acked Read 9 Done Sent Done false
13335 4549 Acked Write 2 Done Sent Done false
13336 4550 Acked Write 10 Done Sent Done false
13337 4551 Acked Read 2 Done Sent Done false
13338 4552 Acked Write 1 Done Sent Done false
13339 4553 Acked Write 9 Done Sent Done false
13340 4554 Acked Read 7 Done Sent Done false
13341 4555 Acked Read 3 Done Sent Done false
13342 4556 Acked Read 3 Done Sent Done false
13343 4557 Acked Write 5 Done Sent Done false
13344 4558 Acked Read 8 Done Sent Done false
13345 4559 Acked Write 5 Done Sent Done false
13346 4560 Acked Write 6 Done Sent Done false
13347 4561 Acked Read 4 Done Sent Done false
13348 4562 Acked Read 8 Done Sent Done false
13349 4563 Acked Read 10 Done Sent Done false
13350 4564 Acked Read 8 Done Sent Done false
13351 4565 Acked Read 9 Done Sent Done false
13352 4566 Acked Write 6 Done Sent Done false
13353 4567 Acked Read 10 Done Sent Done false
13354 4568 Acked Read 8 Done Sent Done false
13355 4569 Acked Read 9 Done Sent Done false
13356 4570 Acked Read 3 Done Sent Done false
13357 4571 Acked Write 3 Done Sent Done false
13358 4572 Acked Write 2 Done Sent Done false
13359 4573 Acked Read 7 Done Sent Done false
13360 4574 Acked Read 3 Done Sent Done false
13361 4575 Acked Read 3 Done Sent Done false
13362 4576 Acked Write 9 Done Sent Done false
13363 4577 Acked Write 9 Done Sent Done false
13364 4578 Acked Read 6 Done Sent Done false
13365 4579 Acked Write 8 Done Sent Done false
13366 4580 Acked Read 10 Done Sent Done false
13367 4581 Acked Write 2 Done Sent Done false
13368 4582 Acked Write 3 Done Sent Done false
13369 4583 Acked Flush 0 Done Sent Done false
13370 4584 Acked Write 1 Done Sent Done false
13371 4585 Acked Write 3 Done Sent Done false
13372 4586 Acked Write 4 Done Sent Done false
13373 4587 Acked Read 6 Done Sent Done false
13374 4588 Acked Read 7 Done Sent Done false
13375 4589 Acked Read 4 Done Sent Done false
13376 4590 Acked Read 6 Done Sent Done false
13377 4591 Acked Read 8 Done Sent Done false
13378 4592 Acked Write 6 Done Sent Done false
13379 4593 Acked Write 7 Done Sent Done false
13380 4594 Acked Read 3 Done Sent Done false
13381 4595 Acked Write 7 Done Sent Done false
13382 4596 Acked Read 7 Done Sent Done false
13383 4597 Acked Flush 0 Done Sent Done false
13384 4598 Acked Write 4 Done Sent Done false
13385 4599 Acked Read 9 Done Sent Done false
13386 4600 Acked Read 9 Done Sent Done false
13387 4601 Acked Write 2 Done Sent Done false
13388 4602 Acked Write 2 Done Sent Done false
13389 4603 Acked Flush 0 Done Sent Done false
13390 4604 Acked Read 7 Done Sent Done false
13391 4605 Acked Read 9 Done Sent Done false
13392 4606 Acked Read 8 Done Sent Done false
13393 4607 Acked Read 5 Done Sent Done false
13394 4608 Acked Write 4 Done Sent Done false
13395 4609 Acked Write 3 Done Sent Done false
13396 4610 Acked Flush 0 Done Sent Done false
13397 4611 Acked Write 8 Done Sent Done false
13398 4612 Acked Read 7 Done Sent Done false
13399 4613 Acked Write 4 Done Sent Done false
13400 4614 Acked Write 9 Done Sent Done false
13401 4615 Acked Flush 0 Done Sent Done false
13402 4616 Acked Flush 0 Done Sent Done false
13403 4617 Acked Read 4 Done Sent Done false
13404 4618 Acked Write 4 Done Sent Done false
13405 4619 Acked Write 5 Done Sent Done false
13406 4620 Acked Write 10 Done Sent Done false
13407 4621 Acked Read 4 Done Sent Done false
13408 STATES DS:0 DS:1 DS:2 TOTAL
13409 Sent 0 301 0 301
13410 Done 301 0 301 602
13411 Skipped 0 0 0 0
13412 Error 0 0 0 0
13413 Last Flush: 4616 4320 4616
13414 Downstairs last five completed: 4320 4319 4318 4317 4316
13415 Upstairs last five completed: 4621 4620 4619 4618 4617
13416 CLIENT: Up:0 ds:301 act:2
13417 ----------------------------------------------------------------
13418 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13419 States: Active Offline Active
13420 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13421 4321 Acked Write 1 Done Sent Done false
13422 4322 Acked Read 5 Done Sent Done false
13423 4323 Acked Write 8 Done Sent Done false
13424 4324 Acked Read 1 Done Sent Done false
13425 4325 Acked Write 6 Done Sent Done false
13426 4326 Acked Read 8 Done Sent Done false
13427 4327 Acked Read 2 Done Sent Done false
13428 4328 Acked Read 7 Done Sent Done false
13429 4329 Acked Write 4 Done Sent Done false
13430 4330 Acked Flush 0 Done Sent Done false
13431 4331 Acked Write 2 Done Sent Done false
13432 4332 Acked Write 2 Done Sent Done false
13433 4333 Acked Write 5 Done Sent Done false
13434 4334 Acked Read 7 Done Sent Done false
13435 4335 Acked Write 2 Done Sent Done false
13436 4336 Acked Write 1 Done Sent Done false
13437 4337 Acked Read 8 Done Sent Done false
13438 4338 Acked Write 2 Done Sent Done false
13439 4339 Acked Write 2 Done Sent Done false
13440 4340 Acked Read 3 Done Sent Done false
13441 4341 Acked Read 9 Done Sent Done false
13442 4342 Acked Read 5 Done Sent Done false
13443 4343 Acked Read 6 Done Sent Done false
13444 4344 Acked Write 2 Done Sent Done false
13445 4345 Acked Read 1 Done Sent Done false
13446 4346 Acked Read 9 Done Sent Done false
13447 4347 Acked Read 4 Done Sent Done false
13448 4348 Acked Read 5 Done Sent Done false
13449 4349 Acked Write 1 Done Sent Done false
13450 4350 Acked Read 3 Done Sent Done false
13451 4351 Acked Write 6 Done Sent Done false
13452 4352 Acked Write 8 Done Sent Done false
13453 4353 Acked Write 2 Done Sent Done false
13454 4354 Acked Read 5 Done Sent Done false
13455 4355 Acked Write 1 Done Sent Done false
13456 4356 Acked Read 3 Done Sent Done false
13457 4357 Acked Read 2 Done Sent Done false
13458 4358 Acked Write 2 Done Sent Done false
13459 4359 Acked Flush 0 Done Sent Done false
13460 4360 Acked Write 8 Done Sent Done false
13461 4361 Acked Write 8 Done Sent Done false
13462 4362 Acked Read 7 Done Sent Done false
13463 4363 Acked Read 4 Done Sent Done false
13464 4364 Acked Read 4 Done Sent Done false
13465 4365 Acked Write 10 Done Sent Done false
13466 4366 Acked Read 6 Done Sent Done false
13467 4367 Acked Write 8 Done Sent Done false
13468 4368 Acked Read 6 Done Sent Done false
13469 4369 Acked Read 5 Done Sent Done false
13470 4370 Acked Flush 0 Done Sent Done false
13471 4371 Acked Write 4 Done Sent Done false
13472 4372 Acked Write 5 Done Sent Done false
13473 4373 Acked Read 8 Done Sent Done false
13474 4374 Acked Write 4 Done Sent Done false
13475 4375 Acked Write 6 Done Sent Done false
13476 4376 Acked Write 3 Done Sent Done false
13477 4377 Acked Write 5 Done Sent Done false
13478 4378 Acked Read 5 Done Sent Done false
13479 4379 Acked Write 4 Done Sent Done false
13480 4380 Acked Read 7 Done Sent Done false
13481 4381 Acked Read 3 Done Sent Done false
13482 4382 Acked Write 10 Done Sent Done false
13483 4383 Acked Read 10 Done Sent Done false
13484 4384 Acked Read 3 Done Sent Done false
13485 4385 Acked Write 1 Done Sent Done false
13486 4386 Acked Read 3 Done Sent Done false
13487 4387 Acked Write 1 Done Sent Done false
13488 4388 Acked Read 2 Done Sent Done false
13489 4389 Acked Read 8 Done Sent Done false
13490 4390 Acked Read 5 Done Sent Done false
13491 4391 Acked Write 7 Done Sent Done false
13492 4392 Acked Read 4 Done Sent Done false
13493 4393 Acked Read 1 Done Sent Done false
13494 4394 Acked Read 7 Done Sent Done false
13495 4395 Acked Read 2 Done Sent Done false
13496 4396 Acked Write 1 Done Sent Done false
13497 4397 Acked Read 10 Done Sent Done false
13498 4398 Acked Flush 0 Done Sent Done false
13499 4399 Acked Write 7 Done Sent Done false
13500 4400 Acked Read 5 Done Sent Done false
13501 4401 Acked Write 5 Done Sent Done false
13502 4402 Acked Write 7 Done Sent Done false
13503 4403 Acked Read 2 Done Sent Done false
13504 4404 Acked Read 10 Done Sent Done false
13505 4405 Acked Read 5 Done Sent Done false
13506 4406 Acked Read 1 Done Sent Done false
13507 4407 Acked Write 9 Done Sent Done false
13508 4408 Acked Read 4 Done Sent Done false
13509 4409 Acked Write 9 Done Sent Done false
13510 4410 Acked Read 2 Done Sent Done false
13511 4411 Acked Write 1 Done Sent Done false
13512 4412 Acked Read 1 Done Sent Done false
13513 4413 Acked Flush 0 Done Sent Done false
13514 4414 Acked Read 8 Done Sent Done false
13515 4415 Acked Write 7 Done Sent Done false
13516 4416 Acked Read 10 Done Sent Done false
13517 4417 Acked Read 10 Done Sent Done false
13518 4418 Acked Read 10 Done Sent Done false
13519 4419 Acked Write 4 Done Sent Done false
13520 4420 Acked Read 3 Done Sent Done false
13521 4421 Acked Read 10 Done Sent Done false
13522 4422 Acked Read 5 Done Sent Done false
13523 4423 Acked Read 3 Done Sent Done false
13524 4424 Acked Read 9 Done Sent Done false
13525 4425 Acked Read 10 Done Sent Done false
13526 4426 Acked Read 9 Done Sent Done false
13527 4427 Acked Read 4 Done Sent Done false
13528 4428 Acked Write 7 Done Sent Done false
13529 4429 Acked Write 8 Done Sent Done false
13530 4430 Acked Read 6 Done Sent Done false
13531 4431 Acked Write 6 Done Sent Done false
13532 4432 Acked Read 1 Done Sent Done false
13533 4433 Acked Write 3 Done Sent Done false
13534 4434 Acked Read 1 Done Sent Done false
13535 4435 Acked Read 7 Done Sent Done false
13536 4436 Acked Read 3 Done Sent Done false
13537 4437 Acked Read 3 Done Sent Done false
13538 4438 Acked Write 4 Done Sent Done false
13539 4439 Acked Flush 0 Done Sent Done false
13540 4440 Acked Read 7 Done Sent Done false
13541 4441 Acked Read 1 Done Sent Done false
13542 4442 Acked Write 9 Done Sent Done false
13543 4443 Acked Read 9 Done Sent Done false
13544 4444 Acked Write 10 Done Sent Done false
13545 4445 Acked Write 10 Done Sent Done false
13546 4446 Acked Flush 0 Done Sent Done false
13547 4447 Acked Read 2 Done Sent Done false
13548 4448 Acked Write 5 Done Sent Done false
13549 4449 Acked Flush 0 Done Sent Done false
13550 4450 Acked Read 3 Done Sent Done false
13551 4451 Acked Flush 0 Done Sent Done false
13552 4452 Acked Write 9 Done Sent Done false
13553 4453 Acked Flush 0 Done Sent Done false
13554 4454 Acked Write 2 Done Sent Done false
13555 4455 Acked Write 6 Done Sent Done false
13556 4456 Acked Write 3 Done Sent Done false
13557 4457 Acked Write 7 Done Sent Done false
13558 4458 Acked Write 4 Done Sent Done false
13559 4459 Acked Write 4 Done Sent Done false
13560 4460 Acked Read 6 Done Sent Done false
13561 4461 Acked Read 4 Done Sent Done false
13562 4462 Acked Write 10 Done Sent Done false
13563 4463 Acked Read 8 Done Sent Done false
13564 4464 Acked Write 9 Done Sent Done false
13565 4465 Acked Write 7 Done Sent Done false
13566 4466 Acked Write 4 Done Sent Done false
13567 4467 Acked Write 4 Done Sent Done false
13568 4468 Acked Write 1 Done Sent Done false
13569 4469 Acked Read 9 Done Sent Done false
13570 4470 Acked Flush 0 Done Sent Done false
13571 4471 Acked Flush 0 Done Sent Done false
13572 4472 Acked Read 1 Done Sent Done false
13573 4473 Acked Write 6 Done Sent Done false
13574 4474 Acked Flush 0 Done Sent Done false
13575 4475 Acked Write 3 Done Sent Done false
13576 4476 Acked Read 6 Done Sent Done false
13577 4477 Acked Write 1 Done Sent Done false
13578 4478 Acked Read 8 Done Sent Done false
13579 4479 Acked Read 5 Done Sent Done false
13580 4480 Acked Read 4 Done Sent Done false
13581 4481 Acked Write 10 Done Sent Done false
13582 4482 Acked Read 8 Done Sent Done false
13583 4483 Acked Read 2 Done Sent Done false
13584 4484 Acked Read 3 Done Sent Done false
13585 4485 Acked Read 2 Done Sent Done false
13586 4486 Acked Read 9 Done Sent Done false
13587 4487 Acked Write 4 Done Sent Done false
13588 4488 Acked Write 8 Done Sent Done false
13589 4489 Acked Read 9 Done Sent Done false
13590 4490 Acked Read 6 Done Sent Done false
13591 4491 Acked Flush 0 Done Sent Done false
13592 4492 Acked Read 10 Done Sent Done false
13593 4493 Acked Read 2 Done Sent Done false
13594 4494 Acked Write 4 Done Sent Done false
13595 4495 Acked Read 5 Done Sent Done false
13596 4496 Acked Read 5 Done Sent Done false
13597 4497 Acked Flush 0 Done Sent Done false
13598 4498 Acked Read 1 Done Sent Done false
13599 4499 Acked Flush 0 Done Sent Done false
13600 4500 Acked Write 7 Done Sent Done false
13601 4501 Acked Read 1 Done Sent Done false
13602 4502 Acked Read 4 Done Sent Done false
13603 4503 Acked Write 3 Done Sent Done false
13604 4504 Acked Write 4 Done Sent Done false
13605 4505 Acked Write 4 Done Sent Done false
13606 4506 Acked Write 2 Done Sent Done false
13607 4507 Acked Read 2 Done Sent Done false
13608 4508 Acked Read 1 Done Sent Done false
13609 4509 Acked Read 5 Done Sent Done false
13610 4510 Acked Read 10 Done Sent Done false
13611 4511 Acked Write 3 Done Sent Done false
13612 4512 Acked Read 6 Done Sent Done false
13613 4513 Acked Write 10 Done Sent Done false
13614 4514 Acked Write 8 Done Sent Done false
13615 4515 Acked Write 6 Done Sent Done false
13616 4516 Acked Write 5 Done Sent Done false
13617 4517 Acked Read 10 Done Sent Done false
13618 4518 Acked Flush 0 Done Sent Done false
13619 4519 Acked Read 6 Done Sent Done false
13620 4520 Acked Read 4 Done Sent Done false
13621 4521 Acked Read 8 Done Sent Done false
13622 4522 Acked Flush 0 Done Sent Done false
13623 4523 Acked Read 6 Done Sent Done false
13624 4524 Acked Write 8 Done Sent Done false
13625 4525 Acked Read 4 Done Sent Done false
13626 4526 Acked Write 8 Done Sent Done false
13627 4527 Acked Flush 0 Done Sent Done false
13628 4528 Acked Read 9 Done Sent Done false
13629 4529 Acked Read 7 Done Sent Done false
13630 4530 Acked Read 9 Done Sent Done false
13631 4531 Acked Write 4 Done Sent Done false
13632 4532 Acked Read 1 Done Sent Done false
13633 4533 Acked Read 5 Done Sent Done false
13634 4534 Acked Read 5 Done Sent Done false
13635 4535 Acked Read 9 Done Sent Done false
13636 4536 Acked Read 4 Done Sent Done false
13637 4537 Acked Read 10 Done Sent Done false
13638 4538 Acked Read 2 Done Sent Done false
13639 4539 Acked Read 4 Done Sent Done false
13640 4540 Acked Read 5 Done Sent Done false
13641 4541 Acked Read 8 Done Sent Done false
13642 4542 Acked Read 9 Done Sent Done false
13643 4543 Acked Flush 0 Done Sent Done false
13644 4544 Acked Read 9 Done Sent Done false
13645 4545 Acked Write 3 Done Sent Done false
13646 4546 Acked Read 9 Done Sent Done false
13647 4547 Acked Write 8 Done Sent Done false
13648 4548 Acked Read 9 Done Sent Done false
13649 4549 Acked Write 2 Done Sent Done false
13650 4550 Acked Write 10 Done Sent Done false
13651 4551 Acked Read 2 Done Sent Done false
13652 4552 Acked Write 1 Done Sent Done false
13653 4553 Acked Write 9 Done Sent Done false
13654 4554 Acked Read 7 Done Sent Done false
13655 4555 Acked Read 3 Done Sent Done false
13656 4556 Acked Read 3 Done Sent Done false
13657 4557 Acked Write 5 Done Sent Done false
13658 4558 Acked Read 8 Done Sent Done false
13659 4559 Acked Write 5 Done Sent Done false
13660 4560 Acked Write 6 Done Sent Done false
13661 4561 Acked Read 4 Done Sent Done false
13662 4562 Acked Read 8 Done Sent Done false
13663 4563 Acked Read 10 Done Sent Done false
13664 4564 Acked Read 8 Done Sent Done false
13665 4565 Acked Read 9 Done Sent Done false
13666 4566 Acked Write 6 Done Sent Done false
13667 4567 Acked Read 10 Done Sent Done false
13668 4568 Acked Read 8 Done Sent Done false
13669 4569 Acked Read 9 Done Sent Done false
13670 4570 Acked Read 3 Done Sent Done false
13671 4571 Acked Write 3 Done Sent Done false
13672 4572 Acked Write 2 Done Sent Done false
13673 4573 Acked Read 7 Done Sent Done false
13674 4574 Acked Read 3 Done Sent Done false
13675 4575 Acked Read 3 Done Sent Done false
13676 4576 Acked Write 9 Done Sent Done false
13677 4577 Acked Write 9 Done Sent Done false
13678 4578 Acked Read 6 Done Sent Done false
13679 4579 Acked Write 8 Done Sent Done false
13680 4580 Acked Read 10 Done Sent Done false
13681 4581 Acked Write 2 Done Sent Done false
13682 4582 Acked Write 3 Done Sent Done false
13683 4583 Acked Flush 0 Done Sent Done false
13684 4584 Acked Write 1 Done Sent Done false
13685 4585 Acked Write 3 Done Sent Done false
13686 4586 Acked Write 4 Done Sent Done false
13687 4587 Acked Read 6 Done Sent Done false
13688 4588 Acked Read 7 Done Sent Done false
13689 4589 Acked Read 4 Done Sent Done false
13690 4590 Acked Read 6 Done Sent Done false
13691 4591 Acked Read 8 Done Sent Done false
13692 4592 Acked Write 6 Done Sent Done false
13693 4593 Acked Write 7 Done Sent Done false
13694 4594 Acked Read 3 Done Sent Done false
13695 4595 Acked Write 7 Done Sent Done false
13696 4596 Acked Read 7 Done Sent Done false
13697 4597 Acked Flush 0 Done Sent Done false
13698 4598 Acked Write 4 Done Sent Done false
13699 4599 Acked Read 9 Done Sent Done false
13700 4600 Acked Read 9 Done Sent Done false
13701 4601 Acked Write 2 Done Sent Done false
13702 4602 Acked Write 2 Done Sent Done false
13703 4603 Acked Flush 0 Done Sent Done false
13704 4604 Acked Read 7 Done Sent Done false
13705 4605 Acked Read 9 Done Sent Done false
13706 4606 Acked Read 8 Done Sent Done false
13707 4607 Acked Read 5 Done Sent Done false
13708 4608 Acked Write 4 Done Sent Done false
13709 4609 Acked Write 3 Done Sent Done false
13710 4610 Acked Flush 0 Done Sent Done false
13711 4611 Acked Write 8 Done Sent Done false
13712 4612 Acked Read 7 Done Sent Done false
13713 4613 Acked Write 4 Done Sent Done false
13714 4614 Acked Write 9 Done Sent Done false
13715 4615 Acked Flush 0 Done Sent Done false
13716 4616 Acked Flush 0 Done Sent Done false
13717 4617 Acked Read 4 Done Sent Done false
13718 4618 Acked Write 4 Done Sent Done false
13719 4619 Acked Write 5 Done Sent Done false
13720 4620 Acked Write 10 Done Sent Done false
13721 4621 Acked Read 4 Done Sent Done false
13722 4622 Acked Flush 0 Done Sent Done false
13723 STATES DS:0 DS:1 DS:2 TOTAL
13724 Sent 0 302 0 302
13725 Done 302 0 302 604
13726 Skipped 0 0 0 0
13727 Error 0 0 0 0
13728 Last Flush: 4622 4320 4622
13729 Downstairs last five completed: 4320 4319 4318 4317 4316
13730 Upstairs last five completed: 4622 4621 4620 4619 4618
13731 CLIENT: Up:0 ds:302 act:2
137322025-01-13T15:54:39.818ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
137332025-01-13T15:54:39.818ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
137342025-01-13T15:54:39.818ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
137352025-01-13T15:54:39.818ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
137362025-01-13T15:54:39.818ZINFOcrucible: client task is exiting = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
13737 [8820] got start action
13738 [8820] I am going to restart
13739 Starting downstairs at port 8820
13740 Make output file at "/tmp/dsc/downstairs-8820.txt"
13741 [8820][1] reports Starting
13742 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1214)
13743 [8820][1] reports Running
13744 ----------------------------------------------------------------
13745 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13746 States: Active Offline Active
13747 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13748 4321 Acked Write 1 Done Sent Done false
13749 4322 Acked Read 5 Done Sent Done false
13750 4323 Acked Write 8 Done Sent Done false
13751 4324 Acked Read 1 Done Sent Done false
13752 4325 Acked Write 6 Done Sent Done false
13753 4326 Acked Read 8 Done Sent Done false
13754 4327 Acked Read 2 Done Sent Done false
13755 4328 Acked Read 7 Done Sent Done false
13756 4329 Acked Write 4 Done Sent Done false
13757 4330 Acked Flush 0 Done Sent Done false
13758 4331 Acked Write 2 Done Sent Done false
13759 4332 Acked Write 2 Done Sent Done false
13760 4333 Acked Write 5 Done Sent Done false
13761 4334 Acked Read 7 Done Sent Done false
13762 4335 Acked Write 2 Done Sent Done false
13763 4336 Acked Write 1 Done Sent Done false
13764 4337 Acked Read 8 Done Sent Done false
13765 4338 Acked Write 2 Done Sent Done false
13766 4339 Acked Write 2 Done Sent Done false
13767 4340 Acked Read 3 Done Sent Done false
13768 4341 Acked Read 9 Done Sent Done false
13769 4342 Acked Read 5 Done Sent Done false
13770 4343 Acked Read 6 Done Sent Done false
13771 4344 Acked Write 2 Done Sent Done false
13772 4345 Acked Read 1 Done Sent Done false
13773 4346 Acked Read 9 Done Sent Done false
13774 4347 Acked Read 4 Done Sent Done false
13775 4348 Acked Read 5 Done Sent Done false
13776 4349 Acked Write 1 Done Sent Done false
13777 4350 Acked Read 3 Done Sent Done false
13778 4351 Acked Write 6 Done Sent Done false
13779 4352 Acked Write 8 Done Sent Done false
13780 4353 Acked Write 2 Done Sent Done false
13781 4354 Acked Read 5 Done Sent Done false
13782 4355 Acked Write 1 Done Sent Done false
13783 4356 Acked Read 3 Done Sent Done false
13784 4357 Acked Read 2 Done Sent Done false
13785 4358 Acked Write 2 Done Sent Done false
13786 4359 Acked Flush 0 Done Sent Done false
13787 4360 Acked Write 8 Done Sent Done false
13788 4361 Acked Write 8 Done Sent Done false
13789 4362 Acked Read 7 Done Sent Done false
13790 4363 Acked Read 4 Done Sent Done false
13791 4364 Acked Read 4 Done Sent Done false
13792 4365 Acked Write 10 Done Sent Done false
13793 4366 Acked Read 6 Done Sent Done false
13794 4367 Acked Write 8 Done Sent Done false
13795 4368 Acked Read 6 Done Sent Done false
13796 4369 Acked Read 5 Done Sent Done false
13797 4370 Acked Flush 0 Done Sent Done false
13798 4371 Acked Write 4 Done Sent Done false
13799 4372 Acked Write 5 Done Sent Done false
13800 4373 Acked Read 8 Done Sent Done false
13801 4374 Acked Write 4 Done Sent Done false
13802 4375 Acked Write 6 Done Sent Done false
13803 4376 Acked Write 3 Done Sent Done false
13804 4377 Acked Write 5 Done Sent Done false
13805 4378 Acked Read 5 Done Sent Done false
13806 4379 Acked Write 4 Done Sent Done false
13807 4380 Acked Read 7 Done Sent Done false
13808 4381 Acked Read 3 Done Sent Done false
13809 4382 Acked Write 10 Done Sent Done false
13810 4383 Acked Read 10 Done Sent Done false
13811 4384 Acked Read 3 Done Sent Done false
13812 4385 Acked Write 1 Done Sent Done false
13813 4386 Acked Read 3 Done Sent Done false
13814 4387 Acked Write 1 Done Sent Done false
13815 4388 Acked Read 2 Done Sent Done false
13816 4389 Acked Read 8 Done Sent Done false
13817 4390 Acked Read 5 Done Sent Done false
13818 4391 Acked Write 7 Done Sent Done false
13819 4392 Acked Read 4 Done Sent Done false
13820 4393 Acked Read 1 Done Sent Done false
13821 4394 Acked Read 7 Done Sent Done false
13822 4395 Acked Read 2 Done Sent Done false
13823 4396 Acked Write 1 Done Sent Done false
13824 4397 Acked Read 10 Done Sent Done false
13825 4398 Acked Flush 0 Done Sent Done false
13826 4399 Acked Write 7 Done Sent Done false
13827 4400 Acked Read 5 Done Sent Done false
13828 4401 Acked Write 5 Done Sent Done false
13829 4402 Acked Write 7 Done Sent Done false
13830 4403 Acked Read 2 Done Sent Done false
13831 4404 Acked Read 10 Done Sent Done false
13832 4405 Acked Read 5 Done Sent Done false
13833 4406 Acked Read 1 Done Sent Done false
13834 4407 Acked Write 9 Done Sent Done false
13835 4408 Acked Read 4 Done Sent Done false
13836 4409 Acked Write 9 Done Sent Done false
13837 4410 Acked Read 2 Done Sent Done false
13838 4411 Acked Write 1 Done Sent Done false
13839 4412 Acked Read 1 Done Sent Done false
13840 4413 Acked Flush 0 Done Sent Done false
13841 4414 Acked Read 8 Done Sent Done false
13842 4415 Acked Write 7 Done Sent Done false
13843 4416 Acked Read 10 Done Sent Done false
13844 4417 Acked Read 10 Done Sent Done false
13845 4418 Acked Read 10 Done Sent Done false
13846 4419 Acked Write 4 Done Sent Done false
13847 4420 Acked Read 3 Done Sent Done false
13848 4421 Acked Read 10 Done Sent Done false
13849 4422 Acked Read 5 Done Sent Done false
13850 4423 Acked Read 3 Done Sent Done false
13851 4424 Acked Read 9 Done Sent Done false
13852 4425 Acked Read 10 Done Sent Done false
13853 4426 Acked Read 9 Done Sent Done false
13854 4427 Acked Read 4 Done Sent Done false
13855 4428 Acked Write 7 Done Sent Done false
13856 4429 Acked Write 8 Done Sent Done false
13857 4430 Acked Read 6 Done Sent Done false
13858 4431 Acked Write 6 Done Sent Done false
13859 4432 Acked Read 1 Done Sent Done false
13860 4433 Acked Write 3 Done Sent Done false
13861 4434 Acked Read 1 Done Sent Done false
13862 4435 Acked Read 7 Done Sent Done false
13863 4436 Acked Read 3 Done Sent Done false
13864 4437 Acked Read 3 Done Sent Done false
13865 4438 Acked Write 4 Done Sent Done false
13866 4439 Acked Flush 0 Done Sent Done false
13867 4440 Acked Read 7 Done Sent Done false
13868 4441 Acked Read 1 Done Sent Done false
13869 4442 Acked Write 9 Done Sent Done false
13870 4443 Acked Read 9 Done Sent Done false
13871 4444 Acked Write 10 Done Sent Done false
13872 4445 Acked Write 10 Done Sent Done false
13873 4446 Acked Flush 0 Done Sent Done false
13874 4447 Acked Read 2 Done Sent Done false
13875 4448 Acked Write 5 Done Sent Done false
13876 4449 Acked Flush 0 Done Sent Done false
13877 4450 Acked Read 3 Done Sent Done false
13878 4451 Acked Flush 0 Done Sent Done false
13879 4452 Acked Write 9 Done Sent Done false
13880 4453 Acked Flush 0 Done Sent Done false
13881 4454 Acked Write 2 Done Sent Done false
13882 4455 Acked Write 6 Done Sent Done false
13883 4456 Acked Write 3 Done Sent Done false
13884 4457 Acked Write 7 Done Sent Done false
13885 4458 Acked Write 4 Done Sent Done false
13886 4459 Acked Write 4 Done Sent Done false
13887 4460 Acked Read 6 Done Sent Done false
13888 4461 Acked Read 4 Done Sent Done false
13889 4462 Acked Write 10 Done Sent Done false
13890 4463 Acked Read 8 Done Sent Done false
13891 4464 Acked Write 9 Done Sent Done false
13892 4465 Acked Write 7 Done Sent Done false
13893 4466 Acked Write 4 Done Sent Done false
13894 4467 Acked Write 4 Done Sent Done false
13895 4468 Acked Write 1 Done Sent Done false
13896 4469 Acked Read 9 Done Sent Done false
13897 4470 Acked Flush 0 Done Sent Done false
13898 4471 Acked Flush 0 Done Sent Done false
13899 4472 Acked Read 1 Done Sent Done false
13900 4473 Acked Write 6 Done Sent Done false
13901 4474 Acked Flush 0 Done Sent Done false
13902 4475 Acked Write 3 Done Sent Done false
13903 4476 Acked Read 6 Done Sent Done false
13904 4477 Acked Write 1 Done Sent Done false
13905 4478 Acked Read 8 Done Sent Done false
13906 4479 Acked Read 5 Done Sent Done false
13907 4480 Acked Read 4 Done Sent Done false
13908 4481 Acked Write 10 Done Sent Done false
13909 4482 Acked Read 8 Done Sent Done false
13910 4483 Acked Read 2 Done Sent Done false
13911 4484 Acked Read 3 Done Sent Done false
13912 4485 Acked Read 2 Done Sent Done false
13913 4486 Acked Read 9 Done Sent Done false
13914 4487 Acked Write 4 Done Sent Done false
13915 4488 Acked Write 8 Done Sent Done false
13916 4489 Acked Read 9 Done Sent Done false
13917 4490 Acked Read 6 Done Sent Done false
13918 4491 Acked Flush 0 Done Sent Done false
13919 4492 Acked Read 10 Done Sent Done false
13920 4493 Acked Read 2 Done Sent Done false
13921 4494 Acked Write 4 Done Sent Done false
13922 4495 Acked Read 5 Done Sent Done false
13923 4496 Acked Read 5 Done Sent Done false
13924 4497 Acked Flush 0 Done Sent Done false
13925 4498 Acked Read 1 Done Sent Done false
13926 4499 Acked Flush 0 Done Sent Done false
13927 4500 Acked Write 7 Done Sent Done false
13928 4501 Acked Read 1 Done Sent Done false
13929 4502 Acked Read 4 Done Sent Done false
13930 4503 Acked Write 3 Done Sent Done false
13931 4504 Acked Write 4 Done Sent Done false
13932 4505 Acked Write 4 Done Sent Done false
13933 4506 Acked Write 2 Done Sent Done false
13934 4507 Acked Read 2 Done Sent Done false
13935 4508 Acked Read 1 Done Sent Done false
13936 4509 Acked Read 5 Done Sent Done false
13937 4510 Acked Read 10 Done Sent Done false
13938 4511 Acked Write 3 Done Sent Done false
13939 4512 Acked Read 6 Done Sent Done false
13940 4513 Acked Write 10 Done Sent Done false
13941 4514 Acked Write 8 Done Sent Done false
13942 4515 Acked Write 6 Done Sent Done false
13943 4516 Acked Write 5 Done Sent Done false
13944 4517 Acked Read 10 Done Sent Done false
13945 4518 Acked Flush 0 Done Sent Done false
13946 4519 Acked Read 6 Done Sent Done false
13947 4520 Acked Read 4 Done Sent Done false
13948 4521 Acked Read 8 Done Sent Done false
13949 4522 Acked Flush 0 Done Sent Done false
13950 4523 Acked Read 6 Done Sent Done false
13951 4524 Acked Write 8 Done Sent Done false
13952 4525 Acked Read 4 Done Sent Done false
13953 4526 Acked Write 8 Done Sent Done false
13954 4527 Acked Flush 0 Done Sent Done false
13955 4528 Acked Read 9 Done Sent Done false
13956 4529 Acked Read 7 Done Sent Done false
13957 4530 Acked Read 9 Done Sent Done false
13958 4531 Acked Write 4 Done Sent Done false
13959 4532 Acked Read 1 Done Sent Done false
13960 4533 Acked Read 5 Done Sent Done false
13961 4534 Acked Read 5 Done Sent Done false
13962 4535 Acked Read 9 Done Sent Done false
13963 4536 Acked Read 4 Done Sent Done false
13964 4537 Acked Read 10 Done Sent Done false
13965 4538 Acked Read 2 Done Sent Done false
13966 4539 Acked Read 4 Done Sent Done false
13967 4540 Acked Read 5 Done Sent Done false
13968 4541 Acked Read 8 Done Sent Done false
13969 4542 Acked Read 9 Done Sent Done false
13970 4543 Acked Flush 0 Done Sent Done false
13971 4544 Acked Read 9 Done Sent Done false
13972 4545 Acked Write 3 Done Sent Done false
13973 4546 Acked Read 9 Done Sent Done false
13974 4547 Acked Write 8 Done Sent Done false
13975 4548 Acked Read 9 Done Sent Done false
13976 4549 Acked Write 2 Done Sent Done false
13977 4550 Acked Write 10 Done Sent Done false
13978 4551 Acked Read 2 Done Sent Done false
13979 4552 Acked Write 1 Done Sent Done false
13980 4553 Acked Write 9 Done Sent Done false
13981 4554 Acked Read 7 Done Sent Done false
13982 4555 Acked Read 3 Done Sent Done false
13983 4556 Acked Read 3 Done Sent Done false
13984 4557 Acked Write 5 Done Sent Done false
13985 4558 Acked Read 8 Done Sent Done false
13986 4559 Acked Write 5 Done Sent Done false
13987 4560 Acked Write 6 Done Sent Done false
13988 4561 Acked Read 4 Done Sent Done false
13989 4562 Acked Read 8 Done Sent Done false
13990 4563 Acked Read 10 Done Sent Done false
13991 4564 Acked Read 8 Done Sent Done false
13992 4565 Acked Read 9 Done Sent Done false
13993 4566 Acked Write 6 Done Sent Done false
13994 4567 Acked Read 10 Done Sent Done false
13995 4568 Acked Read 8 Done Sent Done false
13996 4569 Acked Read 9 Done Sent Done false
13997 4570 Acked Read 3 Done Sent Done false
13998 4571 Acked Write 3 Done Sent Done false
13999 4572 Acked Write 2 Done Sent Done false
14000 4573 Acked Read 7 Done Sent Done false
14001 4574 Acked Read 3 Done Sent Done false
14002 4575 Acked Read 3 Done Sent Done false
14003 4576 Acked Write 9 Done Sent Done false
14004 4577 Acked Write 9 Done Sent Done false
14005 4578 Acked Read 6 Done Sent Done false
14006 4579 Acked Write 8 Done Sent Done false
14007 4580 Acked Read 10 Done Sent Done false
14008 4581 Acked Write 2 Done Sent Done false
14009 4582 Acked Write 3 Done Sent Done false
14010 4583 Acked Flush 0 Done Sent Done false
14011 4584 Acked Write 1 Done Sent Done false
14012 4585 Acked Write 3 Done Sent Done false
14013 4586 Acked Write 4 Done Sent Done false
14014 4587 Acked Read 6 Done Sent Done false
14015 4588 Acked Read 7 Done Sent Done false
14016 4589 Acked Read 4 Done Sent Done false
14017 4590 Acked Read 6 Done Sent Done false
14018 4591 Acked Read 8 Done Sent Done false
14019 4592 Acked Write 6 Done Sent Done false
14020 4593 Acked Write 7 Done Sent Done false
14021 4594 Acked Read 3 Done Sent Done false
14022 4595 Acked Write 7 Done Sent Done false
14023 4596 Acked Read 7 Done Sent Done false
14024 4597 Acked Flush 0 Done Sent Done false
14025 4598 Acked Write 4 Done Sent Done false
14026 4599 Acked Read 9 Done Sent Done false
14027 4600 Acked Read 9 Done Sent Done false
14028 4601 Acked Write 2 Done Sent Done false
14029 4602 Acked Write 2 Done Sent Done false
14030 4603 Acked Flush 0 Done Sent Done false
14031 4604 Acked Read 7 Done Sent Done false
14032 4605 Acked Read 9 Done Sent Done false
14033 4606 Acked Read 8 Done Sent Done false
14034 4607 Acked Read 5 Done Sent Done false
14035 4608 Acked Write 4 Done Sent Done false
14036 4609 Acked Write 3 Done Sent Done false
14037 4610 Acked Flush 0 Done Sent Done false
14038 4611 Acked Write 8 Done Sent Done false
14039 4612 Acked Read 7 Done Sent Done false
14040 4613 Acked Write 4 Done Sent Done false
14041 4614 Acked Write 9 Done Sent Done false
14042 4615 Acked Flush 0 Done Sent Done false
14043 4616 Acked Flush 0 Done Sent Done false
14044 4617 Acked Read 4 Done Sent Done false
14045 4618 Acked Write 4 Done Sent Done false
14046 4619 Acked Write 5 Done Sent Done false
14047 4620 Acked Write 10 Done Sent Done false
14048 4621 Acked Read 4 Done Sent Done false
14049 4622 Acked Flush 0 Done Sent Done false
14050 STATES DS:0 DS:1 DS:2 TOTAL
14051 Sent 0 302 0 302
14052 Done 302 0 302 604
14053 Skipped 0 0 0 0
14054 Error 0 0 0 0
14055 Last Flush: 4622 4320 4622
14056 Downstairs last five completed: 4320 4319 4318 4317 4316
14057 Upstairs last five completed: 4622 4621 4620 4619 4618
14058 CLIENT: Up:0 ds:302 act:2
14059 ----------------------------------------------------------------
14060 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14061 States: Active Offline Active
14062 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14063 4321 Acked Write 1 Done Sent Done false
14064 4322 Acked Read 5 Done Sent Done false
14065 4323 Acked Write 8 Done Sent Done false
14066 4324 Acked Read 1 Done Sent Done false
14067 4325 Acked Write 6 Done Sent Done false
14068 4326 Acked Read 8 Done Sent Done false
14069 4327 Acked Read 2 Done Sent Done false
14070 4328 Acked Read 7 Done Sent Done false
14071 4329 Acked Write 4 Done Sent Done false
14072 4330 Acked Flush 0 Done Sent Done false
14073 4331 Acked Write 2 Done Sent Done false
14074 4332 Acked Write 2 Done Sent Done false
14075 4333 Acked Write 5 Done Sent Done false
14076 4334 Acked Read 7 Done Sent Done false
14077 4335 Acked Write 2 Done Sent Done false
14078 4336 Acked Write 1 Done Sent Done false
14079 4337 Acked Read 8 Done Sent Done false
14080 4338 Acked Write 2 Done Sent Done false
14081 4339 Acked Write 2 Done Sent Done false
14082 4340 Acked Read 3 Done Sent Done false
14083 4341 Acked Read 9 Done Sent Done false
14084 4342 Acked Read 5 Done Sent Done false
14085 4343 Acked Read 6 Done Sent Done false
14086 4344 Acked Write 2 Done Sent Done false
14087 4345 Acked Read 1 Done Sent Done false
14088 4346 Acked Read 9 Done Sent Done false
14089 4347 Acked Read 4 Done Sent Done false
14090 4348 Acked Read 5 Done Sent Done false
14091 4349 Acked Write 1 Done Sent Done false
14092 4350 Acked Read 3 Done Sent Done false
14093 4351 Acked Write 6 Done Sent Done false
14094 4352 Acked Write 8 Done Sent Done false
14095 4353 Acked Write 2 Done Sent Done false
14096 4354 Acked Read 5 Done Sent Done false
14097 4355 Acked Write 1 Done Sent Done false
14098 4356 Acked Read 3 Done Sent Done false
14099 4357 Acked Read 2 Done Sent Done false
14100 4358 Acked Write 2 Done Sent Done false
14101 4359 Acked Flush 0 Done Sent Done false
14102 4360 Acked Write 8 Done Sent Done false
14103 4361 Acked Write 8 Done Sent Done false
14104 4362 Acked Read 7 Done Sent Done false
14105 4363 Acked Read 4 Done Sent Done false
14106 4364 Acked Read 4 Done Sent Done false
14107 4365 Acked Write 10 Done Sent Done false
14108 4366 Acked Read 6 Done Sent Done false
14109 4367 Acked Write 8 Done Sent Done false
14110 4368 Acked Read 6 Done Sent Done false
14111 4369 Acked Read 5 Done Sent Done false
14112 4370 Acked Flush 0 Done Sent Done false
14113 4371 Acked Write 4 Done Sent Done false
14114 4372 Acked Write 5 Done Sent Done false
14115 4373 Acked Read 8 Done Sent Done false
14116 4374 Acked Write 4 Done Sent Done false
14117 4375 Acked Write 6 Done Sent Done false
14118 4376 Acked Write 3 Done Sent Done false
14119 4377 Acked Write 5 Done Sent Done false
14120 4378 Acked Read 5 Done Sent Done false
14121 4379 Acked Write 4 Done Sent Done false
14122 4380 Acked Read 7 Done Sent Done false
14123 4381 Acked Read 3 Done Sent Done false
14124 4382 Acked Write 10 Done Sent Done false
14125 4383 Acked Read 10 Done Sent Done false
14126 4384 Acked Read 3 Done Sent Done false
14127 4385 Acked Write 1 Done Sent Done false
14128 4386 Acked Read 3 Done Sent Done false
14129 4387 Acked Write 1 Done Sent Done false
14130 4388 Acked Read 2 Done Sent Done false
14131 4389 Acked Read 8 Done Sent Done false
14132 4390 Acked Read 5 Done Sent Done false
14133 4391 Acked Write 7 Done Sent Done false
14134 4392 Acked Read 4 Done Sent Done false
14135 4393 Acked Read 1 Done Sent Done false
14136 4394 Acked Read 7 Done Sent Done false
14137 4395 Acked Read 2 Done Sent Done false
14138 4396 Acked Write 1 Done Sent Done false
14139 4397 Acked Read 10 Done Sent Done false
14140 4398 Acked Flush 0 Done Sent Done false
14141 4399 Acked Write 7 Done Sent Done false
14142 4400 Acked Read 5 Done Sent Done false
14143 4401 Acked Write 5 Done Sent Done false
14144 4402 Acked Write 7 Done Sent Done false
14145 4403 Acked Read 2 Done Sent Done false
14146 4404 Acked Read 10 Done Sent Done false
14147 4405 Acked Read 5 Done Sent Done false
14148 4406 Acked Read 1 Done Sent Done false
14149 4407 Acked Write 9 Done Sent Done false
14150 4408 Acked Read 4 Done Sent Done false
14151 4409 Acked Write 9 Done Sent Done false
14152 4410 Acked Read 2 Done Sent Done false
14153 4411 Acked Write 1 Done Sent Done false
14154 4412 Acked Read 1 Done Sent Done false
14155 4413 Acked Flush 0 Done Sent Done false
14156 4414 Acked Read 8 Done Sent Done false
14157 4415 Acked Write 7 Done Sent Done false
14158 4416 Acked Read 10 Done Sent Done false
14159 4417 Acked Read 10 Done Sent Done false
14160 4418 Acked Read 10 Done Sent Done false
14161 4419 Acked Write 4 Done Sent Done false
14162 4420 Acked Read 3 Done Sent Done false
14163 4421 Acked Read 10 Done Sent Done false
14164 4422 Acked Read 5 Done Sent Done false
14165 4423 Acked Read 3 Done Sent Done false
14166 4424 Acked Read 9 Done Sent Done false
14167 4425 Acked Read 10 Done Sent Done false
14168 4426 Acked Read 9 Done Sent Done false
14169 4427 Acked Read 4 Done Sent Done false
14170 4428 Acked Write 7 Done Sent Done false
14171 4429 Acked Write 8 Done Sent Done false
14172 4430 Acked Read 6 Done Sent Done false
14173 4431 Acked Write 6 Done Sent Done false
14174 4432 Acked Read 1 Done Sent Done false
14175 4433 Acked Write 3 Done Sent Done false
14176 4434 Acked Read 1 Done Sent Done false
14177 4435 Acked Read 7 Done Sent Done false
14178 4436 Acked Read 3 Done Sent Done false
14179 4437 Acked Read 3 Done Sent Done false
14180 4438 Acked Write 4 Done Sent Done false
14181 4439 Acked Flush 0 Done Sent Done false
14182 4440 Acked Read 7 Done Sent Done false
14183 4441 Acked Read 1 Done Sent Done false
14184 4442 Acked Write 9 Done Sent Done false
14185 4443 Acked Read 9 Done Sent Done false
14186 4444 Acked Write 10 Done Sent Done false
14187 4445 Acked Write 10 Done Sent Done false
14188 4446 Acked Flush 0 Done Sent Done false
14189 4447 Acked Read 2 Done Sent Done false
14190 4448 Acked Write 5 Done Sent Done false
14191 4449 Acked Flush 0 Done Sent Done false
14192 4450 Acked Read 3 Done Sent Done false
14193 4451 Acked Flush 0 Done Sent Done false
14194 4452 Acked Write 9 Done Sent Done false
14195 4453 Acked Flush 0 Done Sent Done false
14196 4454 Acked Write 2 Done Sent Done false
14197 4455 Acked Write 6 Done Sent Done false
14198 4456 Acked Write 3 Done Sent Done false
14199 4457 Acked Write 7 Done Sent Done false
14200 4458 Acked Write 4 Done Sent Done false
14201 4459 Acked Write 4 Done Sent Done false
14202 4460 Acked Read 6 Done Sent Done false
14203 4461 Acked Read 4 Done Sent Done false
14204 4462 Acked Write 10 Done Sent Done false
14205 4463 Acked Read 8 Done Sent Done false
14206 4464 Acked Write 9 Done Sent Done false
14207 4465 Acked Write 7 Done Sent Done false
14208 4466 Acked Write 4 Done Sent Done false
14209 4467 Acked Write 4 Done Sent Done false
14210 4468 Acked Write 1 Done Sent Done false
14211 4469 Acked Read 9 Done Sent Done false
14212 4470 Acked Flush 0 Done Sent Done false
14213 4471 Acked Flush 0 Done Sent Done false
14214 4472 Acked Read 1 Done Sent Done false
14215 4473 Acked Write 6 Done Sent Done false
14216 4474 Acked Flush 0 Done Sent Done false
14217 4475 Acked Write 3 Done Sent Done false
14218 4476 Acked Read 6 Done Sent Done false
14219 4477 Acked Write 1 Done Sent Done false
14220 4478 Acked Read 8 Done Sent Done false
14221 4479 Acked Read 5 Done Sent Done false
14222 4480 Acked Read 4 Done Sent Done false
14223 4481 Acked Write 10 Done Sent Done false
14224 4482 Acked Read 8 Done Sent Done false
14225 4483 Acked Read 2 Done Sent Done false
14226 4484 Acked Read 3 Done Sent Done false
14227 4485 Acked Read 2 Done Sent Done false
14228 4486 Acked Read 9 Done Sent Done false
14229 4487 Acked Write 4 Done Sent Done false
14230 4488 Acked Write 8 Done Sent Done false
14231 4489 Acked Read 9 Done Sent Done false
14232 4490 Acked Read 6 Done Sent Done false
14233 4491 Acked Flush 0 Done Sent Done false
14234 4492 Acked Read 10 Done Sent Done false
14235 4493 Acked Read 2 Done Sent Done false
14236 4494 Acked Write 4 Done Sent Done false
14237 4495 Acked Read 5 Done Sent Done false
14238 4496 Acked Read 5 Done Sent Done false
14239 4497 Acked Flush 0 Done Sent Done false
14240 4498 Acked Read 1 Done Sent Done false
14241 4499 Acked Flush 0 Done Sent Done false
14242 4500 Acked Write 7 Done Sent Done false
14243 4501 Acked Read 1 Done Sent Done false
14244 4502 Acked Read 4 Done Sent Done false
14245 4503 Acked Write 3 Done Sent Done false
14246 4504 Acked Write 4 Done Sent Done false
14247 4505 Acked Write 4 Done Sent Done false
14248 4506 Acked Write 2 Done Sent Done false
14249 4507 Acked Read 2 Done Sent Done false
14250 4508 Acked Read 1 Done Sent Done false
14251 4509 Acked Read 5 Done Sent Done false
14252 4510 Acked Read 10 Done Sent Done false
14253 4511 Acked Write 3 Done Sent Done false
14254 4512 Acked Read 6 Done Sent Done false
14255 4513 Acked Write 10 Done Sent Done false
14256 4514 Acked Write 8 Done Sent Done false
14257 4515 Acked Write 6 Done Sent Done false
14258 4516 Acked Write 5 Done Sent Done false
14259 4517 Acked Read 10 Done Sent Done false
14260 4518 Acked Flush 0 Done Sent Done false
14261 4519 Acked Read 6 Done Sent Done false
14262 4520 Acked Read 4 Done Sent Done false
14263 4521 Acked Read 8 Done Sent Done false
14264 4522 Acked Flush 0 Done Sent Done false
14265 4523 Acked Read 6 Done Sent Done false
14266 4524 Acked Write 8 Done Sent Done false
14267 4525 Acked Read 4 Done Sent Done false
14268 4526 Acked Write 8 Done Sent Done false
14269 4527 Acked Flush 0 Done Sent Done false
14270 4528 Acked Read 9 Done Sent Done false
14271 4529 Acked Read 7 Done Sent Done false
14272 4530 Acked Read 9 Done Sent Done false
14273 4531 Acked Write 4 Done Sent Done false
14274 4532 Acked Read 1 Done Sent Done false
14275 4533 Acked Read 5 Done Sent Done false
14276 4534 Acked Read 5 Done Sent Done false
14277 4535 Acked Read 9 Done Sent Done false
14278 4536 Acked Read 4 Done Sent Done false
14279 4537 Acked Read 10 Done Sent Done false
14280 4538 Acked Read 2 Done Sent Done false
14281 4539 Acked Read 4 Done Sent Done false
14282 4540 Acked Read 5 Done Sent Done false
14283 4541 Acked Read 8 Done Sent Done false
14284 4542 Acked Read 9 Done Sent Done false
14285 4543 Acked Flush 0 Done Sent Done false
14286 4544 Acked Read 9 Done Sent Done false
14287 4545 Acked Write 3 Done Sent Done false
14288 4546 Acked Read 9 Done Sent Done false
14289 4547 Acked Write 8 Done Sent Done false
14290 4548 Acked Read 9 Done Sent Done false
14291 4549 Acked Write 2 Done Sent Done false
14292 4550 Acked Write 10 Done Sent Done false
14293 4551 Acked Read 2 Done Sent Done false
14294 4552 Acked Write 1 Done Sent Done false
14295 4553 Acked Write 9 Done Sent Done false
14296 4554 Acked Read 7 Done Sent Done false
14297 4555 Acked Read 3 Done Sent Done false
14298 4556 Acked Read 3 Done Sent Done false
14299 4557 Acked Write 5 Done Sent Done false
14300 4558 Acked Read 8 Done Sent Done false
14301 4559 Acked Write 5 Done Sent Done false
14302 4560 Acked Write 6 Done Sent Done false
14303 4561 Acked Read 4 Done Sent Done false
14304 4562 Acked Read 8 Done Sent Done false
14305 4563 Acked Read 10 Done Sent Done false
14306 4564 Acked Read 8 Done Sent Done false
14307 4565 Acked Read 9 Done Sent Done false
14308 4566 Acked Write 6 Done Sent Done false
14309 4567 Acked Read 10 Done Sent Done false
14310 4568 Acked Read 8 Done Sent Done false
14311 4569 Acked Read 9 Done Sent Done false
14312 4570 Acked Read 3 Done Sent Done false
14313 4571 Acked Write 3 Done Sent Done false
14314 4572 Acked Write 2 Done Sent Done false
14315 4573 Acked Read 7 Done Sent Done false
14316 4574 Acked Read 3 Done Sent Done false
14317 4575 Acked Read 3 Done Sent Done false
14318 4576 Acked Write 9 Done Sent Done false
14319 4577 Acked Write 9 Done Sent Done false
14320 4578 Acked Read 6 Done Sent Done false
14321 4579 Acked Write 8 Done Sent Done false
14322 4580 Acked Read 10 Done Sent Done false
14323 4581 Acked Write 2 Done Sent Done false
14324 4582 Acked Write 3 Done Sent Done false
14325 4583 Acked Flush 0 Done Sent Done false
14326 4584 Acked Write 1 Done Sent Done false
14327 4585 Acked Write 3 Done Sent Done false
14328 4586 Acked Write 4 Done Sent Done false
14329 4587 Acked Read 6 Done Sent Done false
14330 4588 Acked Read 7 Done Sent Done false
14331 4589 Acked Read 4 Done Sent Done false
14332 4590 Acked Read 6 Done Sent Done false
14333 4591 Acked Read 8 Done Sent Done false
14334 4592 Acked Write 6 Done Sent Done false
14335 4593 Acked Write 7 Done Sent Done false
14336 4594 Acked Read 3 Done Sent Done false
14337 4595 Acked Write 7 Done Sent Done false
14338 4596 Acked Read 7 Done Sent Done false
14339 4597 Acked Flush 0 Done Sent Done false
14340 4598 Acked Write 4 Done Sent Done false
14341 4599 Acked Read 9 Done Sent Done false
14342 4600 Acked Read 9 Done Sent Done false
14343 4601 Acked Write 2 Done Sent Done false
14344 4602 Acked Write 2 Done Sent Done false
14345 4603 Acked Flush 0 Done Sent Done false
14346 4604 Acked Read 7 Done Sent Done false
14347 4605 Acked Read 9 Done Sent Done false
14348 4606 Acked Read 8 Done Sent Done false
14349 4607 Acked Read 5 Done Sent Done false
14350 4608 Acked Write 4 Done Sent Done false
14351 4609 Acked Write 3 Done Sent Done false
14352 4610 Acked Flush 0 Done Sent Done false
14353 4611 Acked Write 8 Done Sent Done false
14354 4612 Acked Read 7 Done Sent Done false
14355 4613 Acked Write 4 Done Sent Done false
14356 4614 Acked Write 9 Done Sent Done false
14357 4615 Acked Flush 0 Done Sent Done false
14358 4616 Acked Flush 0 Done Sent Done false
14359 4617 Acked Read 4 Done Sent Done false
14360 4618 Acked Write 4 Done Sent Done false
14361 4619 Acked Write 5 Done Sent Done false
14362 4620 Acked Write 10 Done Sent Done false
14363 4621 Acked Read 4 Done Sent Done false
14364 4622 Acked Flush 0 Done Sent Done false
14365 STATES DS:0 DS:1 DS:2 TOTAL
14366 Sent 0 302 0 302
14367 Done 302 0 302 604
14368 Skipped 0 0 0 0
14369 Error 0 0 0 0
14370 Last Flush: 4622 4320 4622
14371 Downstairs last five completed: 4320 4319 4318 4317 4316
14372 Upstairs last five completed: 4622 4621 4620 4619 4618
14373 CLIENT: Up:0 ds:302 act:2
143742025-01-13T15:54:49.819ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
143752025-01-13T15:54:49.819ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62379) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
143762025-01-13T15:54:49.821ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
143772025-01-13T15:54:49.821ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
143782025-01-13T15:54:49.821ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
143792025-01-13T15:54:49.821ZINFOcrucible: send last flush ID to this DS: 4320 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
143802025-01-13T15:54:49.821ZINFOcrucible: Replied this last flush ID: 4320 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
143812025-01-13T15:54:49.821ZINFOcrucible: [1] client re-new 302 jobs since flush 4320 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
143822025-01-13T15:54:49.822ZINFOcrucible: [1] Replayed 302 jobs since flush: 4320 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
14383 ----------------------------------------------------------------
14384 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
14385 Downstairs last five completed: 4622 4621 4620 4619 4618
14386 Upstairs last five completed: 4622 4621 4620 4619 4618
14387 CLIENT: Up:0 ds:0 act:3
14388 Replay: All jobs finished
14389 Main task has work to do, go find it
14390 got dsc Stop(2)
14391 stop 2
14392 [8830] Got stop action so:false kr:false
14393 Jan 13 15:54:50.074 INFO request completed, latency_us: 366, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 399ac431-cd03-43af-8daf-ad3e36df5ae8, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
143942025-01-13T15:54:50.074ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
143952025-01-13T15:54:50.074ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
143962025-01-13T15:54:50.074ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
143972025-01-13T15:54:50.075ZINFOcrucible: client task is exiting = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
14398 Jan 13 15:54:50.075 INFO request completed, latency_us: 139, response_code: 200, uri: /state/cid/2, method: GET, req_id: f3b2e934-9dbf-4417-a30d-bc1ae870bc79, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
14399 [8830] Exited with: ExitStatus(unix_wait_status(9))
14400 [8830][2] reports Exit
14401 Jan 13 15:54:54.077 INFO request completed, latency_us: 298, response_code: 200, uri: /state/cid/2, method: GET, req_id: 1675435e-a2e3-4a90-b347-bd99be8db204, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
14402 001/300 Write block 3869 len 28672 data: 3 3 4 4 5 4 4
14403 002/300 Read block 248 len 36864
14404 003/300 Flush
14405 004/300 Write block 3701 len 32768 data: 2 4 3 2 2 2 2 3
14406 005/300 Write block 4261 len 16384 data: 2 2 2 2
14407 006/300 Write block 705 len 28672 data: 4 4 4 3 2 2 2
14408 007/300 Write block 2823 len 28672 data: 5 5 5 5 5 2 2
14409 008/300 Read block 3257 len 32768
14410 009/300 Read block 3831 len 12288
14411 010/300 Read block 3078 len 32768
14412 011/300 Write block 3371 len 28672 data: 3 3 3 3 3 3 3
14413 012/300 Flush
14414 013/300 Write block 1948 len 4096 data: 2
14415 014/300 Write block 1147 len 20480 data: 5 5 4 4 4
14416 015/300 Write block 1263 len 12288 data: 4 4 4
14417 016/300 Read block 692 len 28672
14418 017/300 Read block 2407 len 12288
14419 018/300 Read block 1031 len 20480
14420 019/300 Write block 4882 len 12288 data: 2 2 3
14421 020/300 Read block 1255 len 40960
14422 021/300 Write block 2039 len 4096 data: 2
14423 022/300 Read block 2268 len 40960
14424 023/300 Read block 1828 len 28672
14425 024/300 Write block 1842 len 16384 data: 3 3 3 3
14426 025/300 Read block 2173 len 8192
14427 026/300 Write block 2333 len 4096 data: 4
14428 027/300 Write block 2474 len 36864 data: 3 4 3 4 4 5 5 6 6
14429 028/300 Read block 2225 len 20480
14430 029/300 Write block 1767 len 40960 data: 3 3 3 2 2 3 3 2 3 4
14431 030/300 Read block 727 len 20480
14432 031/300 Read block 2688 len 36864
14433 032/300 Read block 2175 len 20480
14434 033/300 Read block 2775 len 28672
14435 034/300 Flush
14436 035/300 Read block 480 len 28672
14437 036/300 Flush
14438 037/300 Flush
14439 038/300 Read block 1978 len 16384
14440 039/300 Read block 1859 len 36864
14441 040/300 Write block 2328 len 28672 data: 2 3 3 3 4 5 4
14442 041/300 Read block 2125 len 24576
14443 042/300 Read block 2896 len 24576
14444 043/300 Read block 2158 len 24576
14445 044/300 Flush
14446 045/300 Flush
14447 046/300 Write block 582 len 4096 data: 4
14448 047/300 Read block 1475 len 28672
14449 048/300 Read block 780 len 40960
14450 049/300 Flush
14451 050/300 Write block 3004 len 32768 data: 5 5 5 5 4 5 5 4
14452 051/300 Read block 2336 len 32768
14453 052/300 Read block 2128 len 28672
14454 053/300 Flush
14455 054/300 Write block 1292 len 12288 data: 3 5 5
14456 055/300 Write block 1850 len 16384 data: 2 2 2 2
14457 056/300 Read block 1484 len 40960
14458 057/300 Read block 270 len 28672
14459 058/300 Flush
14460 059/300 Flush
14461 060/300 Write block 4933 len 36864 data: 8 8 6 6 6 4 3 4 5
14462 061/300 Flush
14463 062/300 Write block 3987 len 12288 data: 5 5 4
14464 063/300 Write block 705 len 36864 data: 5 5 5 4 3 3 3 3 3
14465 064/300 Read block 1828 len 24576
14466 065/300 Read block 1338 len 28672
14467 066/300 Write block 2533 len 40960 data: 3 3 3 3 3 3 3 2 2 3
14468 067/300 Write block 687 len 24576 data: 4 4 3 2 2 2
14469 068/300 Write block 946 len 8192 data: 7 6
14470 069/300 Flush
14471 070/300 Write block 147 len 36864 data: 6 6 6 5 4 4 2 3 3
14472 071/300 Read block 1044 len 28672
14473 072/300 Write block 3109 len 24576 data: 3 3 3 3 3 2
14474 073/300 Read block 3369 len 24576
14475 074/300 Read block 2353 len 16384
14476 075/300 Write block 318 len 4096 data: 4
14477 076/300 Write block 523 len 20480 data: 5 5 5 3 4
14478 077/300 Write block 1549 len 36864 data: 3 2 2 2 2 2 3 3 3
14479 078/300 Read block 1522 len 8192
14480 079/300 Flush
14481 080/300 Read block 71 len 12288
14482 081/300 Write block 4753 len 20480 data: 4 4 4 4 3
14483 082/300 Read block 2494 len 32768
14484 083/300 Write block 3062 len 36864 data: 2 3 3 4 3 4 4 4 5
14485 084/300 Flush
14486 085/300 Read block 2336 len 36864
14487 086/300 Write block 1723 len 16384 data: 2 2 2 3
14488 087/300 Write block 3818 len 12288 data: 4 4 3
14489 088/300 Read block 4527 len 4096
14490 089/300 Write block 2749 len 36864 data: 5 5 6 5 4 3 3 3 3
14491 090/300 Write block 118 len 20480 data: 5 6 6 5 5
14492 091/300 Read block 2987 len 28672
14493 092/300 Write block 4375 len 16384 data: 3 3 3 4
14494 093/300 Write block 1894 len 8192 data: 4 4
14495 094/300 Write block 738 len 36864 data: 3 3 3 3 3 4 3 3 3
14496 095/300 Read block 2230 len 20480
14497 096/300 Write block 68 len 12288 data: 4 5 5
14498 097/300 Read block 4376 len 40960
14499 098/300 Read block 2349 len 16384
14500 099/300 Read block 3337 len 24576
14501 100/300 Read block 4710 len 4096
14502 101/300 Read block 2872 len 40960
14503 102/300 Read block 935 len 12288
14504 103/300 Flush
14505 104/300 Write block 3547 len 8192 data: 2 2
14506 105/300 Read block 582 len 4096
14507 106/300 Write block 484 len 4096 data: 3
14508 107/300 Write block 3991 len 20480 data: 4 4 3 4 4
14509 108/300 Write block 4582 len 20480 data: 2 2 2 3 3
14510 109/300 Read block 3733 len 40960
14511 110/300 Flush
14512 111/300 Write block 1555 len 36864 data: 4 4 4 3 3 4 3 3 3
14513 112/300 Write block 1078 len 4096 data: 2
14514 113/300 Write block 3020 len 36864 data: 6 6 6 5 7 5 4 5 5
14515 114/300 Write block 1289 len 16384 data: 2 2 2 4
14516 115/300 Write block 355 len 20480 data: 2 2 2 2 2
14517 116/300 Read block 2940 len 40960
14518 117/300 Read block 4625 len 8192
14519 118/300 Read block 3047 len 36864
14520 119/300 Write block 1235 len 36864 data: 3 3 4 4 4 4 4 3 3
14521 120/300 Write block 2634 len 32768 data: 2 3 4 5 5 5 4 4
14522 121/300 Write block 3226 len 32768 data: 8 8 6 6 6 6 6 5
14523 122/300 Write block 2256 len 20480 data: 3 2 2 2 2
14524 123/300 Write block 1219 len 8192 data: 3 3
14525 124/300 Read block 417 len 28672
14526 125/300 Read block 788 len 4096
14527 126/300 Read block 1197 len 16384
14528 127/300 Write block 1505 len 8192 data: 3 3
14529 128/300 Read block 4811 len 32768
14530 129/300 Read block 972 len 20480
14531 130/300 Write block 669 len 32768 data: 6 5 3 2 2 2 2 2
14532 131/300 Write block 4514 len 16384 data: 5 5 5 5
14533 132/300 Read block 4788 len 4096
14534 133/300 Write block 520 len 28672 data: 5 4 5 6 6 6 4
14535 134/300 Flush
14536 135/300 Write block 3108 len 20480 data: 3 4 4 4 4
14537 136/300 Read block 1048 len 12288
14538 137/300 Read block 4167 len 36864
14539 138/300 Read block 3649 len 16384
14540 139/300 Write block 3174 len 36864 data: 3 3 3 3 3 3 2 3 3
14541 140/300 Write block 4837 len 4096 data: 5
14542 141/300 Write block 3199 len 40960 data: 4 5 5 6 4 4 3 3 3 3
14543 142/300 Write block 4179 len 40960 data: 3 3 4 3 3 3 4 3 4 4
14544 143/300 Read block 4888 len 32768
14545 144/300 Read block 2026 len 28672
14546 145/300 Read block 1575 len 12288
14547 146/300 Read block 3018 len 8192
14548 147/300 Write block 120 len 12288 data: 7 6 6
14549 148/300 Read block 3683 len 32768
14550 149/300 Write block 507 len 4096 data: 2
14551 150/300 Read block 2828 len 36864
14552 151/300 Write block 1792 len 40960 data: 3 3 3 3 3 5 6 5 4 4
14553 152/300 Read block 1441 len 40960
14554 153/300 Read block 842 len 24576
14555 154/300 Write block 303 len 16384 data: 4 4 4 3
14556 155/300 Flush
14557 156/300 Flush
14558 157/300 Write block 1251 len 28672 data: 2 2 2 2 2 2 2
14559 158/300 Write block 4869 len 4096 data: 5
14560 159/300 Write block 1378 len 36864 data: 5 7 7 7 8 9 7 8 8
14561 160/300 Read block 3336 len 32768
14562 161/300 Read block 1743 len 20480
14563 162/300 Read block 1000 len 12288
14564 163/300 Write block 4300 len 8192 data: 7 8
14565 164/300 Read block 2902 len 32768
14566 165/300 Read block 1267 len 16384
14567 166/300 Write block 2334 len 28672 data: 5 4 3 3 3 5 5
14568 167/300 Read block 1563 len 28672
14569 168/300 Read block 2655 len 4096
14570 169/300 Write block 2830 len 16384 data: 2 2 2 2
14571 170/300 Read block 382 len 32768
14572 171/300 Write block 1980 len 36864 data: 4 4 4 4 5 4 4 3 3
14573 172/300 Read block 2694 len 40960
14574 173/300 Read block 2323 len 40960
14575 174/300 Write block 3516 len 32768 data: 4 5 4 5 3 3 3 4
14576 175/300 Read block 4353 len 28672
14577 176/300 Flush
14578 177/300 Read block 3079 len 4096
14579 178/300 Read block 967 len 32768
14580 179/300 Read block 4920 len 32768
14581 180/300 Read block 4285 len 12288
14582 181/300 Write block 2921 len 8192 data: 2 2
14583 182/300 Read block 3442 len 28672
14584 183/300 Read block 2696 len 32768
14585 184/300 Read block 1611 len 4096
14586 185/300 Write block 2770 len 4096 data: 4
14587 186/300 Flush
14588 187/300 Flush
14589 188/300 Read block 165 len 36864
14590 189/300 Read block 3888 len 16384
14591 190/300 Write block 793 len 20480 data: 3 3 2 2 3
14592 191/300 Read block 4582 len 28672
14593 192/300 Read block 3555 len 20480
14594 193/300 Read block 4766 len 16384
14595 194/300 Read block 2770 len 16384
14596 195/300 Flush
14597 196/300 Flush
14598 197/300 Read block 4773 len 12288
14599 198/300 Flush
14600 199/300 Read block 1224 len 8192
14601 200/300 Flush
14602 201/300 Read block 1450 len 36864
14603 202/300 Read block 2607 len 20480
14604 203/300 Flush
14605 204/300 Read block 4236 len 24576
14606 205/300 Read block 1301 len 4096
14607 206/300 Write block 1303 len 8192 data: 3 3
14608 207/300 Read block 4376 len 4096
14609 208/300 Read block 682 len 28672
14610 209/300 Read block 4304 len 40960
14611 210/300 Read block 4811 len 32768
14612 211/300 Read block 436 len 32768
14613 212/300 Read block 1388 len 8192
14614 213/300 Read block 765 len 36864
14615 214/300 Read block 1950 len 40960
14616 215/300 Write block 4309 len 4096 data: 2
14617 216/300 Read block 3943 len 16384
14618 217/300 Read block 2110 len 36864
14619 218/300 Write block 1693 len 40960 data: 2 3 4 4 3 3 5 4 3 2
14620 219/300 Write block 423 len 16384 data: 3 3 4 4
14621 220/300 Read block 2633 len 40960
14622 221/300 Read block 4117 len 24576
14623 222/300 Write block 3747 len 40960 data: 3 3 4 3 3 2 2 2 2 2
14624 223/300 Read block 2919 len 4096
14625 224/300 Read block 146 len 28672
14626 225/300 Flush
14627 226/300 Read block 4718 len 16384
14628 227/300 Read block 971 len 12288
14629 228/300 Write block 2877 len 28672 data: 3 2 3 4 4 4 5
14630 229/300 Read block 999 len 8192
14631 230/300 Write block 2596 len 24576 data: 10 8 6 6 3 3
14632 231/300 Write block 4359 len 4096 data: 3
14633 232/300 Write block 2950 len 32768 data: 4 4 5 6 5 5 3 3
14634 233/300 Read block 321 len 4096
14635 234/300 Write block 1982 len 4096 data: 5
14636 235/300 Flush
14637 236/300 Flush
14638 237/300 Read block 2392 len 8192
14639 238/300 Read block 2931 len 36864
14640 239/300 Write block 3974 len 36864 data: 3 3 3 4 5 5 5 5 4
14641 240/300 Write block 114 len 16384 data: 4 5 4 4
14642 241/300 Read block 4150 len 20480
14643 242/300 Read block 3954 len 32768
14644 243/300 Read block 3374 len 32768
14645 244/300 Write block 3469 len 16384 data: 3 3 3 3
14646 245/300 Flush
14647 246/300 Read block 2240 len 24576
14648 247/300 Write block 2844 len 36864 data: 3 3 3 5 5 3 3 3 3
14649 248/300 Read block 565 len 8192
14650 249/300 Write block 1156 len 32768 data: 2 2 2 2 2 3 3 3
14651 250/300 Write block 1851 len 28672 data: 3 3 3 2 3 4 4
14652 251/300 Write block 4484 len 24576 data: 3 2 2 2 2 2
14653 252/300 Read block 2808 len 36864
14654 253/300 Write block 782 len 24576 data: 3 4 4 3 3 2
14655 254/300 Flush
14656 255/300 Read block 63 len 8192
14657 256/300 Read block 2552 len 20480
14658 257/300 Write block 313 len 8192 data: 2 2
14659 258/300 Read block 1110 len 28672
14660 259/300 Flush
14661 260/300 Flush
14662 261/300 Read block 764 len 4096
14663 262/300 Write block 832 len 28672 data: 5 4 5 4 4 4 4
14664 263/300 Read block 2421 len 32768
14665 264/300 Write block 4884 len 40960 data: 4 3 5 5 6 6 4 4 4 4
14666 265/300 Write block 3778 len 12288 data: 5 6 5
14667 266/300 Read block 1184 len 40960
14668 267/300 Read block 3256 len 4096
14669 268/300 Read block 2981 len 20480
14670 269/300 Flush
14671 270/300 Read block 3425 len 32768
14672 271/300 Read block 1828 len 20480
14673 272/300 Write block 3778 len 32768 data: 6 7 6 4 3 3 4 3
14674 273/300 Read block 1824 len 16384
14675 274/300 Write block 4027 len 36864 data: 3 3 4 3 3 3 2 2 3
14676 275/300 Flush
14677 276/300 Read block 3546 len 20480
14678 277/300 Write block 1167 len 8192 data: 4 4
14679 278/300 Read block 1262 len 40960
14680 279/300 Write block 942 len 24576 data: 6 6 7 7 8 7
14681 280/300 Read block 3946 len 8192
14682 281/300 Read block 1090 len 4096
14683 282/300 Read block 3773 len 8192
14684 283/300 Read block 4178 len 40960
14685 284/300 Flush
14686 285/300 Write block 1827 len 12288 data: 5 4 3
14687 286/300 Write block 1705 len 4096 data: 3
14688 287/300 Write block 3763 len 40960 data: 4 4 4 4 5 3 3 4 5 6
14689 288/300 Write block 4351 len 36864 data: 3 3 3 3 3 3 3 3 4
14690 289/300 Write block 2195 len 28672 data: 5 5 6 6 5 5 5
14691 290/300 Flush
14692 291/300 Read block 4250 len 12288
14693 292/300 Read block 2112 len 16384
14694 293/300 Read block 4343 len 36864
14695 294/300 Read block 3057 len 12288
14696 295/300 Write block 4671 len 28672 data: 3 3 3 4 3 2 2
14697 296/300 Read block 3821 len 40960
14698 297/300 Read block 1974 len 4096
14699 298/300 Read block 1616 len 16384
14700 299/300 Read block 2479 len 28672
14701 300/300 Read block 186 len 8192
14702 301/300 Write block 373 len 40960 data: 5 5 8 8 8 8 6 6 5 3
14703 Main task has work to do, go find it
14704 got dsc Start(2)
14705 start 2
14706 Jan 13 15:54:54.319 INFO request completed, latency_us: 171, response_code: 204, uri: /start/cid/2, method: POST, req_id: 650110b2-c694-4866-be9b-e82677f117c5, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
14707 Replay: started 2, returned:Ok(())
14708 ----------------------------------------------------------------
14709 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
14710 States: Active Active Offline
14711 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14712 4623 Acked Write 7 Done Done Sent false
14713 4624 Acked Read 9 Done Done Sent false
14714 4625 Acked Flush 0 Done Done Sent false
14715 4626 Acked Write 8 Done Done Sent false
14716 4627 Acked Write 4 Done Done Sent false
14717 4628 Acked Write 7 Done Done Sent false
14718 4629 Acked Write 7 Done Done Sent false
14719 4630 Acked Read 8 Done Done Sent false
14720 4631 Acked Read 3 Done Done Sent false
14721 4632 Acked Read 8 Done Done Sent false
14722 4633 Acked Write 7 Done Done Sent false
14723 4634 Acked Flush 0 Done Done Sent false
14724 4635 Acked Write 1 Done Done Sent false
14725 4636 Acked Write 5 Done Done Sent false
14726 4637 Acked Write 3 Done Done Sent false
14727 4638 Acked Read 7 Done Done Sent false
14728 4639 Acked Read 3 Done Done Sent false
14729 4640 Acked Read 5 Done Done Sent false
14730 4641 Acked Write 3 Done Done Sent false
14731 4642 Acked Read 10 Done Done Sent false
14732 4643 Acked Write 1 Done Done Sent false
14733 4644 Acked Read 10 Done Done Sent false
14734 4645 Acked Read 7 Done Done Sent false
14735 4646 Acked Write 4 Done Done Sent false
14736 4647 Acked Read 2 Done Done Sent false
14737 4648 Acked Write 1 Done Done Sent false
14738 4649 Acked Write 9 Done Done Sent false
14739 4650 Acked Read 5 Done Done Sent false
14740 4651 Acked Write 10 Done Done Sent false
14741 4652 Acked Read 5 Done Done Sent false
14742 4653 Acked Read 9 Done Done Sent false
14743 4654 Acked Read 5 Done Done Sent false
14744 4655 Acked Read 7 Done Done Sent false
14745 4656 Acked Flush 0 Done Done Sent false
14746 4657 Acked Read 7 Done Done Sent false
14747 4658 Acked Flush 0 Done Done Sent false
14748 4659 Acked Flush 0 Done Done Sent false
14749 4660 Acked Read 4 Done Done Sent false
14750 4661 Acked Read 9 Done Done Sent false
14751 4662 Acked Write 7 Done Done Sent false
14752 4663 Acked Read 6 Done Done Sent false
14753 4664 Acked Read 6 Done Done Sent false
14754 4665 Acked Read 6 Done Done Sent false
14755 4666 Acked Flush 0 Done Done Sent false
14756 4667 Acked Flush 0 Done Done Sent false
14757 4668 Acked Write 1 Done Done Sent false
14758 4669 Acked Read 7 Done Done Sent false
14759 4670 Acked Read 10 Done Done Sent false
14760 4671 Acked Flush 0 Done Done Sent false
14761 4672 Acked Write 8 Done Done Sent false
14762 4673 Acked Read 8 Done Done Sent false
14763 4674 Acked Read 7 Done Done Sent false
14764 4675 Acked Flush 0 Done Done Sent false
14765 4676 Acked Write 3 Done Done Sent false
14766 4677 Acked Write 4 Done Done Sent false
14767 4678 Acked Read 10 Done Done Sent false
14768 4679 Acked Read 7 Done Done Sent false
14769 4680 Acked Flush 0 Done Done Sent false
14770 4681 Acked Flush 0 Done Done Sent false
14771 4682 Acked Write 9 Done Done Sent false
14772 4683 Acked Flush 0 Done Done Sent false
14773 4684 Acked Write 3 Done Done Sent false
14774 4685 Acked Write 9 Done Done Sent false
14775 4686 Acked Read 6 Done Done Sent false
14776 4687 Acked Read 7 Done Done Sent false
14777 4688 Acked Write 10 Done Done Sent false
14778 4689 Acked Write 6 Done Done Sent false
14779 4690 Acked Write 2 Done Done Sent false
14780 4691 Acked Flush 0 Done Done Sent false
14781 4692 Acked Write 9 Done Done Sent false
14782 4693 Acked Read 7 Done Done Sent false
14783 4694 Acked Write 6 Done Done Sent false
14784 4695 Acked Read 6 Done Done Sent false
14785 4696 Acked Read 4 Done Done Sent false
14786 4697 Acked Write 1 Done Done Sent false
14787 4698 Acked Write 5 Done Done Sent false
14788 4699 Acked Write 9 Done Done Sent false
14789 4700 Acked Read 2 Done Done Sent false
14790 4701 Acked Flush 0 Done Done Sent false
14791 4702 Acked Read 3 Done Done Sent false
14792 4703 Acked Write 5 Done Done Sent false
14793 4704 Acked Read 8 Done Done Sent false
14794 4705 Acked Write 9 Done Done Sent false
14795 4706 Acked Flush 0 Done Done Sent false
14796 4707 Acked Read 9 Done Done Sent false
14797 4708 Acked Write 4 Done Done Sent false
14798 4709 Acked Write 3 Done Done Sent false
14799 4710 Acked Read 1 Done Done Sent false
14800 4711 Acked Write 9 Done Done Sent false
14801 4712 Acked Write 5 Done Done Sent false
14802 4713 Acked Read 7 Done Done Sent false
14803 4714 Acked Write 4 Done Done Sent false
14804 4715 Acked Write 2 Done Done Sent false
14805 4716 Acked Write 9 Done Done Sent false
14806 4717 Acked Read 5 Done Done Sent false
14807 4718 Acked Write 3 Done Done Sent false
14808 4719 Acked Read 10 Done Done Sent false
14809 4720 Acked Read 4 Done Done Sent false
14810 4721 Acked Read 6 Done Done Sent false
14811 4722 Acked Read 1 Done Done Sent false
14812 4723 Acked Read 10 Done Done Sent false
14813 4724 Acked Read 3 Done Done Sent false
14814 4725 Acked Flush 0 Done Done Sent false
14815 4726 Acked Write 2 Done Done Sent false
14816 4727 Acked Read 1 Done Done Sent false
14817 4728 Acked Write 1 Done Done Sent false
14818 4729 Acked Write 5 Done Done Sent false
14819 4730 Acked Write 5 Done Done Sent false
14820 4731 Acked Read 10 Done Done Sent false
14821 4732 Acked Flush 0 Done Done Sent false
14822 4733 Acked Write 9 Done Done Sent false
14823 4734 Acked Write 1 Done Done Sent false
14824 4735 Acked Write 9 Done Done Sent false
14825 4736 Acked Write 4 Done Done Sent false
14826 4737 Acked Write 5 Done Done Sent false
14827 4738 Acked Read 10 Done Done Sent false
14828 4739 Acked Read 2 Done Done Sent false
14829 4740 Acked Read 9 Done Done Sent false
14830 4741 Acked Write 9 Done Done Sent false
14831 4742 Acked Write 8 Done Done Sent false
14832 4743 Acked Write 8 Done Done Sent false
14833 4744 Acked Write 5 Done Done Sent false
14834 4745 Acked Write 2 Done Done Sent false
14835 4746 Acked Read 7 Done Done Sent false
14836 4747 Acked Read 1 Done Done Sent false
14837 4748 Acked Read 4 Done Done Sent false
14838 4749 Acked Write 2 Done Done Sent false
14839 4750 Acked Read 8 Done Done Sent false
14840 4751 Acked Read 5 Done Done Sent false
14841 4752 Acked Write 8 Done Done Sent false
14842 4753 Acked Write 4 Done Done Sent false
14843 4754 Acked Read 1 Done Done Sent false
14844 4755 Acked Write 7 Done Done Sent false
14845 4756 Acked Flush 0 Done Done Sent false
14846 4757 Acked Write 5 Done Done Sent false
14847 4758 Acked Read 3 Done Done Sent false
14848 4759 Acked Read 9 Done Done Sent false
14849 4760 Acked Read 4 Done Done Sent false
14850 4761 Acked Write 9 Done Done Sent false
14851 4762 Acked Write 1 Done Done Sent false
14852 4763 Acked Write 10 Done Done Sent false
14853 4764 Acked Write 10 Done Done Sent false
14854 4765 Acked Read 8 Done Done Sent false
14855 4766 Acked Read 7 Done Done Sent false
14856 4767 Acked Read 3 Done Done Sent false
14857 4768 Acked Read 2 Done Done Sent false
14858 4769 Acked Write 3 Done Done Sent false
14859 4770 Acked Read 8 Done Done Sent false
14860 4771 Acked Write 1 Done Done Sent false
14861 4772 Acked Read 9 Done Done Sent false
14862 4773 Acked Write 10 Done Done Sent false
14863 4774 Acked Read 10 Done Done Sent false
14864 4775 Acked Read 6 Done Done Sent false
14865 4776 Acked Write 4 Done Done Sent false
14866 4777 Acked Flush 0 Done Done Sent false
14867 4778 Acked Flush 0 Done Done Sent false
14868 4779 Acked Write 7 Done Done Sent false
14869 4780 Acked Write 1 Done Done Sent false
14870 4781 Acked Write 9 Done Done Sent false
14871 4782 Acked Read 8 Done Done Sent false
14872 4783 Acked Read 5 Done Done Sent false
14873 4784 Acked Read 3 Done Done Sent false
14874 4785 Acked Write 2 Done Done Sent false
14875 4786 Acked Read 8 Done Done Sent false
14876 4787 Acked Read 4 Done Done Sent false
14877 4788 Acked Write 7 Done Done Sent false
14878 4789 Acked Read 7 Done Done Sent false
14879 4790 Acked Read 1 Done Done Sent false
14880 4791 Acked Write 4 Done Done Sent false
14881 4792 Acked Read 8 Done Done Sent false
14882 4793 Acked Write 9 Done Done Sent false
14883 4794 Acked Read 10 Done Done Sent false
14884 4795 Acked Read 10 Done Done Sent false
14885 4796 Acked Write 8 Done Done Sent false
14886 4797 Acked Read 7 Done Done Sent false
14887 4798 Acked Flush 0 Done Done Sent false
14888 4799 Acked Read 1 Done Done Sent false
14889 4800 Acked Read 8 Done Done Sent false
14890 4801 Acked Read 8 Done Done Sent false
14891 4802 Acked Read 3 Done Done Sent false
14892 4803 Acked Write 2 Done Done Sent false
14893 4804 Acked Read 7 Done Done Sent false
14894 4805 Acked Read 8 Done Done Sent false
14895 4806 Acked Read 1 Done Done Sent false
14896 4807 Acked Write 1 Done Done Sent false
14897 4808 Acked Flush 0 Done Done Sent false
14898 4809 Acked Flush 0 Done Done Sent false
14899 4810 Acked Read 9 Done Done Sent false
14900 4811 Acked Read 4 Done Done Sent false
14901 4812 Acked Write 5 Done Done Sent false
14902 4813 Acked Read 7 Done Done Sent false
14903 4814 Acked Read 5 Done Done Sent false
14904 4815 Acked Read 4 Done Done Sent false
14905 4816 Acked Read 4 Done Done Sent false
14906 4817 Acked Flush 0 Done Done Sent false
14907 4818 Acked Flush 0 Done Done Sent false
14908 4819 Acked Read 3 Done Done Sent false
14909 4820 Acked Flush 0 Done Done Sent false
14910 4821 Acked Read 2 Done Done Sent false
14911 4822 Acked Flush 0 Done Done Sent false
14912 4823 Acked Read 9 Done Done Sent false
14913 4824 Acked Read 5 Done Done Sent false
14914 4825 Acked Flush 0 Done Done Sent false
14915 4826 Acked Read 6 Done Done Sent false
14916 4827 Acked Read 1 Done Done Sent false
14917 4828 Acked Write 2 Done Done Sent false
14918 4829 Acked Read 1 Done Done Sent false
14919 4830 Acked Read 7 Done Done Sent false
14920 4831 Acked Read 10 Done Done Sent false
14921 4832 Acked Read 8 Done Done Sent false
14922 4833 Acked Read 8 Done Done Sent false
14923 4834 Acked Read 2 Done Done Sent false
14924 4835 Acked Read 9 Done Done Sent false
14925 4836 Acked Read 10 Done Done Sent false
14926 4837 Acked Write 1 Done Done Sent false
14927 4838 Acked Read 4 Done Done Sent false
14928 4839 Acked Read 9 Done Done Sent false
14929 4840 Acked Write 10 Done Done Sent false
14930 4841 Acked Write 4 Done Done Sent false
14931 4842 Acked Read 10 Done Done Sent false
14932 4843 Acked Read 6 Done Done Sent false
14933 4844 Acked Write 10 Done Done Sent false
14934 4845 Acked Read 1 Done Done Sent false
14935 4846 Acked Read 7 Done Done Sent false
14936 4847 Acked Flush 0 Done Done Sent false
14937 4848 Acked Read 4 Done Done Sent false
14938 4849 Acked Read 3 Done Done Sent false
14939 4850 Acked Write 7 Done Done Sent false
14940 4851 Acked Read 2 Done Done Sent false
14941 4852 Acked Write 6 Done Done Sent false
14942 4853 Acked Write 1 Done Done Sent false
14943 4854 Acked Write 8 Done Done Sent false
14944 4855 Acked Read 1 Done Done Sent false
14945 4856 Acked Write 1 Done Done Sent false
14946 4857 Acked Flush 0 Done Done Sent false
14947 4858 Acked Flush 0 Done Done Sent false
14948 4859 Acked Read 2 Done Done Sent false
14949 4860 Acked Read 9 Done Done Sent false
14950 4861 Acked Write 9 Done Done Sent false
14951 4862 Acked Write 4 Done Done Sent false
14952 4863 Acked Read 5 Done Done Sent false
14953 4864 Acked Read 8 Done Done Sent false
14954 4865 Acked Read 8 Done Done Sent false
14955 4866 Acked Write 4 Done Done Sent false
14956 4867 Acked Flush 0 Done Done Sent false
14957 4868 Acked Read 6 Done Done Sent false
14958 4869 Acked Write 9 Done Done Sent false
14959 4870 Acked Read 2 Done Done Sent false
14960 4871 Acked Write 8 Done Done Sent false
14961 4872 Acked Write 7 Done Done Sent false
14962 4873 Acked Write 6 Done Done Sent false
14963 4874 Acked Read 9 Done Done Sent false
14964 4875 Acked Write 6 Done Done Sent false
14965 4876 Acked Flush 0 Done Done Sent false
14966 4877 Acked Read 2 Done Done Sent false
14967 4878 Acked Read 5 Done Done Sent false
14968 4879 Acked Write 2 Done Done Sent false
14969 4880 Acked Read 7 Done Done Sent false
14970 4881 Acked Flush 0 Done Done Sent false
14971 4882 Acked Flush 0 Done Done Sent false
14972 4883 Acked Read 1 Done Done Sent false
14973 4884 Acked Write 7 Done Done Sent false
14974 4885 Acked Read 8 Done Done Sent false
14975 4886 Acked Write 10 Done Done Sent false
14976 4887 Acked Write 3 Done Done Sent false
14977 4888 Acked Read 10 Done Done Sent false
14978 4889 Acked Read 1 Done Done Sent false
14979 4890 Acked Read 5 Done Done Sent false
14980 4891 Acked Flush 0 Done Done Sent false
14981 4892 Acked Read 8 Done Done Sent false
14982 4893 Acked Read 5 Done Done Sent false
14983 4894 Acked Write 8 Done Done Sent false
14984 4895 Acked Read 4 Done Done Sent false
14985 4896 Acked Write 9 Done Done Sent false
14986 4897 Acked Flush 0 Done Done Sent false
14987 4898 Acked Read 5 Done Done Sent false
14988 4899 Acked Write 2 Done Done Sent false
14989 4900 Acked Read 10 Done Done Sent false
14990 4901 Acked Write 6 Done Done Sent false
14991 4902 Acked Read 2 Done Done Sent false
14992 4903 Acked Read 1 Done Done Sent false
14993 4904 Acked Read 2 Done Done Sent false
14994 4905 Acked Read 10 Done Done Sent false
14995 4906 Acked Flush 0 Done Done Sent false
14996 4907 Acked Write 3 Done Done Sent false
14997 4908 Acked Write 1 Done Done Sent false
14998 4909 Acked Write 10 Done Done Sent false
14999 4910 Acked Write 9 Done Done Sent false
15000 4911 Acked Write 7 Done Done Sent false
15001 4912 Acked Flush 0 Done Done Sent false
15002 4913 Acked Read 3 Done Done Sent false
15003 4914 Acked Read 4 Done Done Sent false
15004 4915 Acked Read 9 Done Done Sent false
15005 4916 Acked Read 3 Done Done Sent false
15006 4917 Acked Write 7 Done Done Sent false
15007 4918 Acked Read 10 Done Done Sent false
15008 4919 Acked Read 1 Done Done Sent false
15009 4920 Acked Read 4 Done Done Sent false
15010 4921 Acked Read 7 Done Done Sent false
15011 4922 Acked Read 2 Done Done Sent false
15012 4923 Acked Write 10 Sent Sent Sent false
15013 STATES DS:0 DS:1 DS:2 TOTAL
15014 Sent 1 1 301 303
15015 Done 300 300 0 600
15016 Skipped 0 0 0 0
15017 Error 0 0 0 0
15018 Last Flush: 4912 4912 4622
15019 Downstairs last five completed: 4622 4621 4620 4619 4618
15020 Upstairs last five completed: 4923 4922 4921 4920 4919
15021 CLIENT: Up:0 ds:301 act:2
15022 [8830] got start action
15023 [8830] I am going to restart
15024 Starting downstairs at port 8830
15025 Make output file at "/tmp/dsc/downstairs-8830.txt"
15026 [8830][2] reports Starting
15027 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1215)
15028 [8830][2] reports Running
15029 ----------------------------------------------------------------
15030 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15031 States: Active Active Offline
15032 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15033 4623 Acked Write 7 Done Done Sent false
15034 4624 Acked Read 9 Done Done Sent false
15035 4625 Acked Flush 0 Done Done Sent false
15036 4626 Acked Write 8 Done Done Sent false
15037 4627 Acked Write 4 Done Done Sent false
15038 4628 Acked Write 7 Done Done Sent false
15039 4629 Acked Write 7 Done Done Sent false
15040 4630 Acked Read 8 Done Done Sent false
15041 4631 Acked Read 3 Done Done Sent false
15042 4632 Acked Read 8 Done Done Sent false
15043 4633 Acked Write 7 Done Done Sent false
15044 4634 Acked Flush 0 Done Done Sent false
15045 4635 Acked Write 1 Done Done Sent false
15046 4636 Acked Write 5 Done Done Sent false
15047 4637 Acked Write 3 Done Done Sent false
15048 4638 Acked Read 7 Done Done Sent false
15049 4639 Acked Read 3 Done Done Sent false
15050 4640 Acked Read 5 Done Done Sent false
15051 4641 Acked Write 3 Done Done Sent false
15052 4642 Acked Read 10 Done Done Sent false
15053 4643 Acked Write 1 Done Done Sent false
15054 4644 Acked Read 10 Done Done Sent false
15055 4645 Acked Read 7 Done Done Sent false
15056 4646 Acked Write 4 Done Done Sent false
15057 4647 Acked Read 2 Done Done Sent false
15058 4648 Acked Write 1 Done Done Sent false
15059 4649 Acked Write 9 Done Done Sent false
15060 4650 Acked Read 5 Done Done Sent false
15061 4651 Acked Write 10 Done Done Sent false
15062 4652 Acked Read 5 Done Done Sent false
15063 4653 Acked Read 9 Done Done Sent false
15064 4654 Acked Read 5 Done Done Sent false
15065 4655 Acked Read 7 Done Done Sent false
15066 4656 Acked Flush 0 Done Done Sent false
15067 4657 Acked Read 7 Done Done Sent false
15068 4658 Acked Flush 0 Done Done Sent false
15069 4659 Acked Flush 0 Done Done Sent false
15070 4660 Acked Read 4 Done Done Sent false
15071 4661 Acked Read 9 Done Done Sent false
15072 4662 Acked Write 7 Done Done Sent false
15073 4663 Acked Read 6 Done Done Sent false
15074 4664 Acked Read 6 Done Done Sent false
15075 4665 Acked Read 6 Done Done Sent false
15076 4666 Acked Flush 0 Done Done Sent false
15077 4667 Acked Flush 0 Done Done Sent false
15078 4668 Acked Write 1 Done Done Sent false
15079 4669 Acked Read 7 Done Done Sent false
15080 4670 Acked Read 10 Done Done Sent false
15081 4671 Acked Flush 0 Done Done Sent false
15082 4672 Acked Write 8 Done Done Sent false
15083 4673 Acked Read 8 Done Done Sent false
15084 4674 Acked Read 7 Done Done Sent false
15085 4675 Acked Flush 0 Done Done Sent false
15086 4676 Acked Write 3 Done Done Sent false
15087 4677 Acked Write 4 Done Done Sent false
15088 4678 Acked Read 10 Done Done Sent false
15089 4679 Acked Read 7 Done Done Sent false
15090 4680 Acked Flush 0 Done Done Sent false
15091 4681 Acked Flush 0 Done Done Sent false
15092 4682 Acked Write 9 Done Done Sent false
15093 4683 Acked Flush 0 Done Done Sent false
15094 4684 Acked Write 3 Done Done Sent false
15095 4685 Acked Write 9 Done Done Sent false
15096 4686 Acked Read 6 Done Done Sent false
15097 4687 Acked Read 7 Done Done Sent false
15098 4688 Acked Write 10 Done Done Sent false
15099 4689 Acked Write 6 Done Done Sent false
15100 4690 Acked Write 2 Done Done Sent false
15101 4691 Acked Flush 0 Done Done Sent false
15102 4692 Acked Write 9 Done Done Sent false
15103 4693 Acked Read 7 Done Done Sent false
15104 4694 Acked Write 6 Done Done Sent false
15105 4695 Acked Read 6 Done Done Sent false
15106 4696 Acked Read 4 Done Done Sent false
15107 4697 Acked Write 1 Done Done Sent false
15108 4698 Acked Write 5 Done Done Sent false
15109 4699 Acked Write 9 Done Done Sent false
15110 4700 Acked Read 2 Done Done Sent false
15111 4701 Acked Flush 0 Done Done Sent false
15112 4702 Acked Read 3 Done Done Sent false
15113 4703 Acked Write 5 Done Done Sent false
15114 4704 Acked Read 8 Done Done Sent false
15115 4705 Acked Write 9 Done Done Sent false
15116 4706 Acked Flush 0 Done Done Sent false
15117 4707 Acked Read 9 Done Done Sent false
15118 4708 Acked Write 4 Done Done Sent false
15119 4709 Acked Write 3 Done Done Sent false
15120 4710 Acked Read 1 Done Done Sent false
15121 4711 Acked Write 9 Done Done Sent false
15122 4712 Acked Write 5 Done Done Sent false
15123 4713 Acked Read 7 Done Done Sent false
15124 4714 Acked Write 4 Done Done Sent false
15125 4715 Acked Write 2 Done Done Sent false
15126 4716 Acked Write 9 Done Done Sent false
15127 4717 Acked Read 5 Done Done Sent false
15128 4718 Acked Write 3 Done Done Sent false
15129 4719 Acked Read 10 Done Done Sent false
15130 4720 Acked Read 4 Done Done Sent false
15131 4721 Acked Read 6 Done Done Sent false
15132 4722 Acked Read 1 Done Done Sent false
15133 4723 Acked Read 10 Done Done Sent false
15134 4724 Acked Read 3 Done Done Sent false
15135 4725 Acked Flush 0 Done Done Sent false
15136 4726 Acked Write 2 Done Done Sent false
15137 4727 Acked Read 1 Done Done Sent false
15138 4728 Acked Write 1 Done Done Sent false
15139 4729 Acked Write 5 Done Done Sent false
15140 4730 Acked Write 5 Done Done Sent false
15141 4731 Acked Read 10 Done Done Sent false
15142 4732 Acked Flush 0 Done Done Sent false
15143 4733 Acked Write 9 Done Done Sent false
15144 4734 Acked Write 1 Done Done Sent false
15145 4735 Acked Write 9 Done Done Sent false
15146 4736 Acked Write 4 Done Done Sent false
15147 4737 Acked Write 5 Done Done Sent false
15148 4738 Acked Read 10 Done Done Sent false
15149 4739 Acked Read 2 Done Done Sent false
15150 4740 Acked Read 9 Done Done Sent false
15151 4741 Acked Write 9 Done Done Sent false
15152 4742 Acked Write 8 Done Done Sent false
15153 4743 Acked Write 8 Done Done Sent false
15154 4744 Acked Write 5 Done Done Sent false
15155 4745 Acked Write 2 Done Done Sent false
15156 4746 Acked Read 7 Done Done Sent false
15157 4747 Acked Read 1 Done Done Sent false
15158 4748 Acked Read 4 Done Done Sent false
15159 4749 Acked Write 2 Done Done Sent false
15160 4750 Acked Read 8 Done Done Sent false
15161 4751 Acked Read 5 Done Done Sent false
15162 4752 Acked Write 8 Done Done Sent false
15163 4753 Acked Write 4 Done Done Sent false
15164 4754 Acked Read 1 Done Done Sent false
15165 4755 Acked Write 7 Done Done Sent false
15166 4756 Acked Flush 0 Done Done Sent false
15167 4757 Acked Write 5 Done Done Sent false
15168 4758 Acked Read 3 Done Done Sent false
15169 4759 Acked Read 9 Done Done Sent false
15170 4760 Acked Read 4 Done Done Sent false
15171 4761 Acked Write 9 Done Done Sent false
15172 4762 Acked Write 1 Done Done Sent false
15173 4763 Acked Write 10 Done Done Sent false
15174 4764 Acked Write 10 Done Done Sent false
15175 4765 Acked Read 8 Done Done Sent false
15176 4766 Acked Read 7 Done Done Sent false
15177 4767 Acked Read 3 Done Done Sent false
15178 4768 Acked Read 2 Done Done Sent false
15179 4769 Acked Write 3 Done Done Sent false
15180 4770 Acked Read 8 Done Done Sent false
15181 4771 Acked Write 1 Done Done Sent false
15182 4772 Acked Read 9 Done Done Sent false
15183 4773 Acked Write 10 Done Done Sent false
15184 4774 Acked Read 10 Done Done Sent false
15185 4775 Acked Read 6 Done Done Sent false
15186 4776 Acked Write 4 Done Done Sent false
15187 4777 Acked Flush 0 Done Done Sent false
15188 4778 Acked Flush 0 Done Done Sent false
15189 4779 Acked Write 7 Done Done Sent false
15190 4780 Acked Write 1 Done Done Sent false
15191 4781 Acked Write 9 Done Done Sent false
15192 4782 Acked Read 8 Done Done Sent false
15193 4783 Acked Read 5 Done Done Sent false
15194 4784 Acked Read 3 Done Done Sent false
15195 4785 Acked Write 2 Done Done Sent false
15196 4786 Acked Read 8 Done Done Sent false
15197 4787 Acked Read 4 Done Done Sent false
15198 4788 Acked Write 7 Done Done Sent false
15199 4789 Acked Read 7 Done Done Sent false
15200 4790 Acked Read 1 Done Done Sent false
15201 4791 Acked Write 4 Done Done Sent false
15202 4792 Acked Read 8 Done Done Sent false
15203 4793 Acked Write 9 Done Done Sent false
15204 4794 Acked Read 10 Done Done Sent false
15205 4795 Acked Read 10 Done Done Sent false
15206 4796 Acked Write 8 Done Done Sent false
15207 4797 Acked Read 7 Done Done Sent false
15208 4798 Acked Flush 0 Done Done Sent false
15209 4799 Acked Read 1 Done Done Sent false
15210 4800 Acked Read 8 Done Done Sent false
15211 4801 Acked Read 8 Done Done Sent false
15212 4802 Acked Read 3 Done Done Sent false
15213 4803 Acked Write 2 Done Done Sent false
15214 4804 Acked Read 7 Done Done Sent false
15215 4805 Acked Read 8 Done Done Sent false
15216 4806 Acked Read 1 Done Done Sent false
15217 4807 Acked Write 1 Done Done Sent false
15218 4808 Acked Flush 0 Done Done Sent false
15219 4809 Acked Flush 0 Done Done Sent false
15220 4810 Acked Read 9 Done Done Sent false
15221 4811 Acked Read 4 Done Done Sent false
15222 4812 Acked Write 5 Done Done Sent false
15223 4813 Acked Read 7 Done Done Sent false
15224 4814 Acked Read 5 Done Done Sent false
15225 4815 Acked Read 4 Done Done Sent false
15226 4816 Acked Read 4 Done Done Sent false
15227 4817 Acked Flush 0 Done Done Sent false
15228 4818 Acked Flush 0 Done Done Sent false
15229 4819 Acked Read 3 Done Done Sent false
15230 4820 Acked Flush 0 Done Done Sent false
15231 4821 Acked Read 2 Done Done Sent false
15232 4822 Acked Flush 0 Done Done Sent false
15233 4823 Acked Read 9 Done Done Sent false
15234 4824 Acked Read 5 Done Done Sent false
15235 4825 Acked Flush 0 Done Done Sent false
15236 4826 Acked Read 6 Done Done Sent false
15237 4827 Acked Read 1 Done Done Sent false
15238 4828 Acked Write 2 Done Done Sent false
15239 4829 Acked Read 1 Done Done Sent false
15240 4830 Acked Read 7 Done Done Sent false
15241 4831 Acked Read 10 Done Done Sent false
15242 4832 Acked Read 8 Done Done Sent false
15243 4833 Acked Read 8 Done Done Sent false
15244 4834 Acked Read 2 Done Done Sent false
15245 4835 Acked Read 9 Done Done Sent false
15246 4836 Acked Read 10 Done Done Sent false
15247 4837 Acked Write 1 Done Done Sent false
15248 4838 Acked Read 4 Done Done Sent false
15249 4839 Acked Read 9 Done Done Sent false
15250 4840 Acked Write 10 Done Done Sent false
15251 4841 Acked Write 4 Done Done Sent false
15252 4842 Acked Read 10 Done Done Sent false
15253 4843 Acked Read 6 Done Done Sent false
15254 4844 Acked Write 10 Done Done Sent false
15255 4845 Acked Read 1 Done Done Sent false
15256 4846 Acked Read 7 Done Done Sent false
15257 4847 Acked Flush 0 Done Done Sent false
15258 4848 Acked Read 4 Done Done Sent false
15259 4849 Acked Read 3 Done Done Sent false
15260 4850 Acked Write 7 Done Done Sent false
15261 4851 Acked Read 2 Done Done Sent false
15262 4852 Acked Write 6 Done Done Sent false
15263 4853 Acked Write 1 Done Done Sent false
15264 4854 Acked Write 8 Done Done Sent false
15265 4855 Acked Read 1 Done Done Sent false
15266 4856 Acked Write 1 Done Done Sent false
15267 4857 Acked Flush 0 Done Done Sent false
15268 4858 Acked Flush 0 Done Done Sent false
15269 4859 Acked Read 2 Done Done Sent false
15270 4860 Acked Read 9 Done Done Sent false
15271 4861 Acked Write 9 Done Done Sent false
15272 4862 Acked Write 4 Done Done Sent false
15273 4863 Acked Read 5 Done Done Sent false
15274 4864 Acked Read 8 Done Done Sent false
15275 4865 Acked Read 8 Done Done Sent false
15276 4866 Acked Write 4 Done Done Sent false
15277 4867 Acked Flush 0 Done Done Sent false
15278 4868 Acked Read 6 Done Done Sent false
15279 4869 Acked Write 9 Done Done Sent false
15280 4870 Acked Read 2 Done Done Sent false
15281 4871 Acked Write 8 Done Done Sent false
15282 4872 Acked Write 7 Done Done Sent false
15283 4873 Acked Write 6 Done Done Sent false
15284 4874 Acked Read 9 Done Done Sent false
15285 4875 Acked Write 6 Done Done Sent false
15286 4876 Acked Flush 0 Done Done Sent false
15287 4877 Acked Read 2 Done Done Sent false
15288 4878 Acked Read 5 Done Done Sent false
15289 4879 Acked Write 2 Done Done Sent false
15290 4880 Acked Read 7 Done Done Sent false
15291 4881 Acked Flush 0 Done Done Sent false
15292 4882 Acked Flush 0 Done Done Sent false
15293 4883 Acked Read 1 Done Done Sent false
15294 4884 Acked Write 7 Done Done Sent false
15295 4885 Acked Read 8 Done Done Sent false
15296 4886 Acked Write 10 Done Done Sent false
15297 4887 Acked Write 3 Done Done Sent false
15298 4888 Acked Read 10 Done Done Sent false
15299 4889 Acked Read 1 Done Done Sent false
15300 4890 Acked Read 5 Done Done Sent false
15301 4891 Acked Flush 0 Done Done Sent false
15302 4892 Acked Read 8 Done Done Sent false
15303 4893 Acked Read 5 Done Done Sent false
15304 4894 Acked Write 8 Done Done Sent false
15305 4895 Acked Read 4 Done Done Sent false
15306 4896 Acked Write 9 Done Done Sent false
15307 4897 Acked Flush 0 Done Done Sent false
15308 4898 Acked Read 5 Done Done Sent false
15309 4899 Acked Write 2 Done Done Sent false
15310 4900 Acked Read 10 Done Done Sent false
15311 4901 Acked Write 6 Done Done Sent false
15312 4902 Acked Read 2 Done Done Sent false
15313 4903 Acked Read 1 Done Done Sent false
15314 4904 Acked Read 2 Done Done Sent false
15315 4905 Acked Read 10 Done Done Sent false
15316 4906 Acked Flush 0 Done Done Sent false
15317 4907 Acked Write 3 Done Done Sent false
15318 4908 Acked Write 1 Done Done Sent false
15319 4909 Acked Write 10 Done Done Sent false
15320 4910 Acked Write 9 Done Done Sent false
15321 4911 Acked Write 7 Done Done Sent false
15322 4912 Acked Flush 0 Done Done Sent false
15323 4913 Acked Read 3 Done Done Sent false
15324 4914 Acked Read 4 Done Done Sent false
15325 4915 Acked Read 9 Done Done Sent false
15326 4916 Acked Read 3 Done Done Sent false
15327 4917 Acked Write 7 Done Done Sent false
15328 4918 Acked Read 10 Done Done Sent false
15329 4919 Acked Read 1 Done Done Sent false
15330 4920 Acked Read 4 Done Done Sent false
15331 4921 Acked Read 7 Done Done Sent false
15332 4922 Acked Read 2 Done Done Sent false
15333 4923 Acked Write 10 Done Done Sent false
15334 4924 Acked Flush 0 Done Done Sent false
15335 STATES DS:0 DS:1 DS:2 TOTAL
15336 Sent 0 0 302 302
15337 Done 302 302 0 604
15338 Skipped 0 0 0 0
15339 Error 0 0 0 0
15340 Last Flush: 4924 4924 4622
15341 Downstairs last five completed: 4622 4621 4620 4619 4618
15342 Upstairs last five completed: 4924 4923 4922 4921 4920
15343 CLIENT: Up:0 ds:302 act:2
153442025-01-13T15:55:00.076ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
153452025-01-13T15:55:00.076ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36977) = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
153462025-01-13T15:55:00.077ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
153472025-01-13T15:55:00.078ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
153482025-01-13T15:55:00.078ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
153492025-01-13T15:55:00.078ZINFOcrucible: send last flush ID to this DS: 4622 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
153502025-01-13T15:55:00.078ZINFOcrucible: Replied this last flush ID: 4622 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
153512025-01-13T15:55:00.078ZINFOcrucible: [2] client re-new 302 jobs since flush 4622 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
153522025-01-13T15:55:00.078ZINFOcrucible: [2] Replayed 302 jobs since flush: 4622 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
15353 ----------------------------------------------------------------
15354 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
15355 Downstairs last five completed: 4924 4923 4922 4921 4920
15356 Upstairs last five completed: 4924 4923 4922 4921 4920
15357 CLIENT: Up:0 ds:0 act:3
15358 Replay: All jobs finished
15359 Main task has work to do, go find it
15360 got dsc Stop(1)
15361 stop 1
15362 [8820] Got stop action so:false kr:false
15363 Jan 13 15:55:02.324 INFO request completed, latency_us: 426, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 18ce0eda-64c2-40f3-9c3b-aa070cf61d27, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
153642025-01-13T15:55:02.324ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
153652025-01-13T15:55:02.325ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
15366 Jan 13 15:55:02.325 INFO request completed, latency_us: 136, response_code: 200, uri: /state/cid/1, method:{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:55:02.325125049Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
15367 GET, req_id: 44a67b39-0878-4707-9d8c-dc09c413f20b, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
153682025-01-13T15:55:02.325ZINFOcrucible: client task is exiting = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
15369 [8820] Exited with: ExitStatus(unix_wait_status(9))
15370 [8820][1] reports Exit
15371 Jan 13 15:55:06.327 INFO request completed, latency_us: 206, response_code: 200, uri: /state/cid/1, method: GET, req_id: 524f6766-062f-48e8-b743-488092ede1ad, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
15372 001/300 Read block 4869 len 4096
15373 002/300 Read block 406 len 4096
15374 003/300 Write block 4666 len 12288 data: 4 3 3
15375 004/300 Read block 1773 len 40960
15376 005/300 Read block 3501 len 4096
15377 006/300 Read block 1142 len 40960
15378 007/300 Read block 2846 len 36864
15379 008/300 Flush
15380 009/300 Read block 3747 len 16384
15381 010/300 Write block 1803 len 32768 data: 2 2 2 2 2 2 2 2
15382 011/300 Flush
15383 012/300 Read block 2877 len 20480
15384 013/300 Write block 4404 len 40960 data: 2 2 2 2 2 2 2 2 2 2
15385 014/300 Read block 640 len 12288
15386 015/300 Write block 1105 len 24576 data: 2 2 3 3 3 3
15387 016/300 Write block 106 len 12288 data: 4 3 4
15388 017/300 Read block 4014 len 28672
15389 018/300 Write block 3638 len 28672 data: 3 2 3 3 3 3 4
15390 019/300 Write block 478 len 36864 data: 5 5 3 3 2 3 4 4 4
15391 020/300 Read block 1937 len 28672
15392 021/300 Write block 2268 len 8192 data: 6 6
15393 022/300 Read block 4105 len 36864
15394 023/300 Write block 1174 len 4096 data: 3
15395 024/300 Read block 3281 len 36864
15396 025/300 Read block 342 len 4096
15397 026/300 Flush
15398 027/300 Write block 4502 len 8192 data: 5 5
15399 028/300 Read block 3510 len 20480
15400 029/300 Read block 1609 len 20480
15401 030/300 Write block 111 len 28672 data: 3 3 3 5 6 5 5
15402 031/300 Read block 4827 len 8192
15403 032/300 Write block 4308 len 8192 data: 3 3
15404 033/300 Write block 3473 len 16384 data: 3 4 4 3
15405 034/300 Read block 2974 len 24576
15406 035/300 Write block 937 len 32768 data: 3 2 3 3 4 7 7 8
15407 036/300 Read block 3834 len 36864
15408 037/300 Read block 3302 len 36864
15409 038/300 Read block 2413 len 40960
15410 039/300 Write block 990 len 24576 data: 4 4 3 2 2 2
15411 040/300 Write block 121 len 20480 data: 7 7 6 5 5
15412 041/300 Flush
15413 042/300 Flush
15414 043/300 Read block 1538 len 32768
15415 044/300 Write block 1759 len 4096 data: 4
15416 045/300 Read block 4950 len 36864
15417 046/300 Flush
15418 047/300 Read block 1605 len 8192
15419 048/300 Write block 922 len 16384 data: 3 3 3 3
15420 049/300 Read block 428 len 32768
15421 050/300 Read block 2239 len 40960
15422 051/300 Write block 1628 len 24576 data: 3 2 2 2 3 3
15423 052/300 Write block 2600 len 40960 data: 4 4 4 4 4 4 4 4 4 3
15424 053/300 Read block 3995 len 16384
15425 054/300 Read block 1937 len 24576
15426 055/300 Read block 2564 len 36864
15427 056/300 Read block 3772 len 36864
15428 057/300 Read block 4907 len 40960
15429 058/300 Flush
15430 059/300 Read block 387 len 28672
15431 060/300 Write block 4329 len 20480 data: 3 4 3 3 2
15432 061/300 Write block 353 len 16384 data: 2 2 3 3
15433 062/300 Write block 4310 len 12288 data: 2 2 2
15434 063/300 Write block 4281 len 8192 data: 3 3
15435 064/300 Flush
15436 065/300 Read block 822 len 20480
15437 066/300 Write block 2752 len 16384 data: 6 5 4 4
15438 067/300 Read block 1208 len 8192
15439 068/300 Write block 780 len 8192 data: 2 3
15440 069/300 Read block 701 len 32768
15441 070/300 Write block 2641 len 16384 data: 5 4 5 5
15442 071/300 Write block 1115 len 12288 data: 4 4 4
15443 072/300 Read block 15 len 16384
15444 073/300 Write block 4866 len 12288 data: 7 6 5
15445 074/300 Read block 4072 len 24576
15446 075/300 Read block 1014 len 40960
15447 076/300 Read block 3866 len 20480
15448 077/300 Write block 974 len 20480 data: 3 3 3 4 4
15449 078/300 Read block 987 len 40960
15450 079/300 Read block 2279 len 32768
15451 080/300 Read block 1268 len 20480
15452 081/300 Read block 4492 len 40960
15453 082/300 Flush
15454 083/300 Write block 534 len 24576 data: 4 4 4 3 4 4
15455 084/300 Write block 3066 len 8192 data: 4 5
15456 085/300 Read block 3728 len 16384
15457 086/300 Write block 4849 len 28672 data: 4 2 2 2 2 2 3
15458 087/300 Write block 706 len 20480 data: 6 6 5 4 4
15459 088/300 Read block 3055 len 40960
15460 089/300 Write block 1163 len 28672 data: 4 3 4 4 5 5 5
15461 090/300 Write block 1514 len 4096 data: 3
15462 091/300 Read block 2281 len 28672
15463 092/300 Write block 1952 len 40960 data: 3 3 3 3 3 3 2 2 3 3
15464 093/300 Read block 1194 len 8192
15465 094/300 Read block 3047 len 32768
15466 095/300 Read block 1284 len 20480
15467 096/300 Read block 1531 len 28672
15468 097/300 Write block 3564 len 8192 data: 4 4
15469 098/300 Write block 78 len 12288 data: 4 5 5
15470 099/300 Write block 762 len 36864 data: 4 4 4 4 5 5 6 6 6
15471 100/300 Read block 4379 len 20480
15472 101/300 Read block 3687 len 28672
15473 102/300 Write block 1139 len 28672 data: 2 2 3 4 5 5 5
15474 103/300 Write block 4530 len 12288 data: 3 4 5
15475 104/300 Read block 1829 len 16384
15476 105/300 Read block 3201 len 28672
15477 106/300 Read block 717 len 28672
15478 107/300 Read block 4612 len 12288
15479 108/300 Read block 3740 len 16384
15480 109/300 Flush
15481 110/300 Flush
15482 111/300 Write block 3696 len 4096 data: 2
15483 112/300 Read block 2129 len 12288
15484 113/300 Read block 4623 len 36864
15485 114/300 Read block 2778 len 32768
15486 115/300 Flush
15487 116/300 Write block 563 len 12288 data: 3 3 2
15488 117/300 Read block 3001 len 4096
15489 118/300 Write block 1098 len 20480 data: 2 2 2 2 2
15490 119/300 Read block 1812 len 20480
15491 120/300 Write block 4975 len 36864 data: 5 4 3 2 2 3 3 3 3
15492 121/300 Write block 3283 len 40960 data: 4 3 3 3 3 3 3 4 3 3
15493 122/300 Read block 2448 len 32768
15494 123/300 Read block 3109 len 8192
15495 124/300 Write block 4085 len 16384 data: 3 4 3 3
15496 125/300 Write block 3162 len 40960 data: 3 2 2 3 3 4 4 4 3 3
15497 126/300 Flush
15498 127/300 Flush
15499 128/300 Read block 2738 len 8192
15500 129/300 Read block 3920 len 40960
15501 130/300 Flush
15502 131/300 Write block 1853 len 20480 data: 4 3 4 5 5
15503 132/300 Write block 3482 len 12288 data: 5 5 6
15504 133/300 Read block 1161 len 4096
15505 134/300 Flush
15506 135/300 Read block 1172 len 16384
15507 136/300 Write block 3923 len 4096 data: 4
15508 137/300 Read block 3995 len 24576
15509 138/300 Read block 4853 len 24576
15510 139/300 Read block 3978 len 28672
15511 140/300 Read block 3441 len 8192
15512 141/300 Read block 4859 len 8192
15513 142/300 Write block 4208 len 32768 data: 4 4 4 6 4 5 4 6
15514 143/300 Read block 4892 len 32768
15515 144/300 Read block 1209 len 16384
15516 145/300 Read block 3275 len 16384
15517 146/300 Write block 2521 len 36864 data: 3 3 3 4 4 4 4 4 2
15518 147/300 Read block 782 len 8192
15519 148/300 Write block 3589 len 32768 data: 4 4 4 5 3 3 3 3
15520 149/300 Flush
15521 150/300 Write block 2269 len 8192 data: 7 5
15522 151/300 Read block 4504 len 40960
15523 152/300 Read block 2038 len 16384
15524 153/300 Read block 3507 len 12288
15525 154/300 Write block 597 len 28672 data: 4 3 3 3 3 3 2
15526 155/300 Read block 904 len 28672
15527 156/300 Read block 1212 len 12288
15528 157/300 Read block 4415 len 32768
15529 158/300 Flush
15530 159/300 Flush
15531 160/300 Write block 1811 len 24576 data: 3 3 4 4 4 4
15532 161/300 Read block 1443 len 24576
15533 162/300 Read block 2148 len 24576
15534 163/300 Read block 1393 len 32768
15535 164/300 Write block 2485 len 12288 data: 6 7 4
15536 165/300 Flush
15537 166/300 Write block 2129 len 40960 data: 4 3 2 2 2 3 3 2 2 3
15538 167/300 Read block 3674 len 24576
15539 168/300 Read block 3369 len 4096
15540 169/300 Read block 3202 len 12288
15541 170/300 Write block 625 len 16384 data: 3 3 3 3
15542 171/300 Write block 4556 len 36864 data: 2 2 2 2 3 3 3 3 3
15543 172/300 Read block 1384 len 4096
15544 173/300 Write block 1705 len 12288 data: 4 3 3
15545 174/300 Write block 1139 len 24576 data: 3 3 4 5 6 6
15546 175/300 Read block 2156 len 40960
15547 176/300 Write block 2355 len 24576 data: 2 2 2 3 2 2
15548 177/300 Read block 3712 len 36864
15549 178/300 Write block 4195 len 24576 data: 3 2 2 3 3 4
15550 179/300 Read block 4790 len 24576
15551 180/300 Write block 4242 len 12288 data: 3 4 4
15552 181/300 Write block 2203 len 24576 data: 5 5 5 4 3 4
15553 182/300 Write block 2963 len 4096 data: 3
15554 183/300 Read block 2713 len 12288
15555 184/300 Write block 3262 len 36864 data: 5 5 6 6 4 5 5 5 5
15556 185/300 Write block 4771 len 36864 data: 3 3 3 4 3 3 5 5 4
15557 186/300 Write block 4384 len 24576 data: 2 2 2 2 2 2
15558 187/300 Read block 4469 len 16384
15559 188/300 Flush
15560 189/300 Write block 4886 len 4096 data: 6
15561 190/300 Read block 1525 len 16384
15562 191/300 Read block 3822 len 28672
15563 192/300 Read block 3214 len 8192
15564 193/300 Read block 4472 len 32768
15565 194/300 Write block 1501 len 24576 data: 4 3 3 4 4 4
15566 195/300 Write block 4850 len 12288 data: 3 3 3
15567 196/300 Write block 2196 len 20480 data: 6 7 7 6 6
15568 197/300 Write block 3229 len 8192 data: 7 7
15569 198/300 Read block 2663 len 36864
15570 199/300 Read block 4217 len 12288
15571 200/300 Write block 2867 len 16384 data: 4 4 4 4
15572 201/300 Flush
15573 202/300 Write block 2563 len 20480 data: 4 3 3 2 2
15574 203/300 Write block 60 len 16384 data: 5 6 5 6
15575 204/300 Read block 1031 len 36864
15576 205/300 Write block 3763 len 28672 data: 5 5 5 5 6 4 4
15577 206/300 Read block 3993 len 40960
15578 207/300 Read block 348 len 28672
15579 208/300 Read block 2866 len 36864
15580 209/300 Write block 3453 len 24576 data: 4 3 3 3 3 2
15581 210/300 Read block 4322 len 24576
15582 211/300 Flush
15583 212/300 Write block 1508 len 28672 data: 3 3 3 3 3 3 4
15584 213/300 Read block 794 len 12288
15585 214/300 Flush
15586 215/300 Read block 176 len 32768
15587 216/300 Read block 3333 len 16384
15588 217/300 Write block 2099 len 16384 data: 3 3 4 3
15589 218/300 Write block 4796 len 8192 data: 2 3
15590 219/300 Write block 2865 len 36864 data: 5 4 5 5 5 5 5 4 3
15591 220/300 Read block 745 len 4096
15592 221/300 Write block 3465 len 28672 data: 3 3 3 2 4 4 4
15593 222/300 Read block 2808 len 4096
15594 223/300 Read block 4695 len 16384
15595 224/300 Read block 795 len 40960
15596 225/300 Write block 3476 len 8192 data: 4 3
15597 226/300 Write block 1039 len 24576 data: 4 3 3 5 5 6
15598 227/300 Flush
15599 228/300 Read block 4000 len 16384
15600 229/300 Write block 2580 len 8192 data: 3 3
15601 230/300 Write block 1995 len 16384 data: 2 2 2 2
15602 231/300 Write block 237 len 20480 data: 2 3 3 4 5
15603 232/300 Write block 2569 len 32768 data: 2 3 3 3 2 2 2 2
15604 233/300 Read block 75 len 4096
15605 234/300 Write block 2147 len 40960 data: 3 4 6 6 5 4 3 3 4 2
15606 235/300 Flush
15607 236/300 Write block 3816 len 32768 data: 3 4 5 5 4 3 3 3
15608 237/300 Read block 4539 len 8192
15609 238/300 Write block 1166 len 40960 data: 5 6 6 6 4 5 4 5 4 3
15610 239/300 Read block 3804 len 16384
15611 240/300 Write block 3845 len 32768 data: 4 4 2 2 2 2 2 2
15612 241/300 Write block 988 len 8192 data: 5 5
15613 242/300 Read block 4683 len 4096
15614 243/300 Read block 892 len 8192
15615 244/300 Read block 2009 len 24576
15616 245/300 Flush
15617 246/300 Write block 4566 len 16384 data: 4 4 4 4
15618 247/300 Read block 555 len 32768
15619 248/300 Flush
15620 249/300 Write block 106 len 28672 data: 5 4 5 4 4 4 4
15621 250/300 Write block 48 len 32768 data: 2 3 3 3 2 2 2 3
15622 251/300 Read block 4141 len 8192
15623 252/300 Read block 1627 len 28672
15624 253/300 Write block 3754 len 20480 data: 3 3 3 2 3
15625 254/300 Write block 2255 len 4096 data: 3
15626 255/300 Write block 1529 len 28672 data: 5 5 4 2 2 2 2
15627 256/300 Read block 1848 len 40960
15628 257/300 Read block 398 len 24576
15629 258/300 Flush
15630 259/300 Write block 2375 len 20480 data: 3 3 3 3 2
15631 260/300 Write block 2037 len 28672 data: 2 2 3 2 2 2 3
15632 261/300 Flush
15633 262/300 Read block 3252 len 16384
15634 263/300 Read block 4787 len 28672
15635 264/300 Read block 3860 len 40960
15636 265/300 Write block 4029 len 32768 data: 5 4 4 4 3 3 4 3
15637 266/300 Read block 2979 len 16384
15638 267/300 Read block 1157 len 20480
15639 268/300 Write block 4507 len 32768 data: 3 4 4 5 5 5 4 6
15640 269/300 Read block 157 len 24576
15641 270/300 Write block 2078 len 12288 data: 5 5 4
15642 271/300 Read block 1536 len 4096
15643 272/300 Write block 3635 len 8192 data: 4 3
15644 273/300 Flush
15645 274/300 Read block 2407 len 4096
15646 275/300 Flush
15647 276/300 Read block 1199 len 4096
15648 277/300 Write block 2726 len 40960 data: 2 2 3 3 5 5 5 4 5 4
15649 278/300 Read block 4040 len 28672
15650 279/300 Write block 1959 len 8192 data: 3 4
15651 280/300 Flush
15652 281/300 Flush
15653 282/300 Read block 771 len 4096
15654 283/300 Write block 2810 len 24576 data: 4 2 3 3 4 4
15655 284/300 Read block 3705 len 28672
15656 285/300 Read block 3299 len 20480
15657 286/300 Read block 3715 len 4096
15658 287/300 Read block 4190 len 4096
15659 288/300 Write block 2287 len 40960 data: 2 2 2 2 2 2 3 3 3 3
15660 289/300 Write block 3146 len 40960 data: 3 3 3 2 2 3 3 3 4 4
15661 290/300 Read block 3109 len 32768
15662 291/300 Read block 4187 len 12288
15663 292/300 Read block 227 len 8192
15664 293/300 Read block 3392 len 16384
15665 294/300 Flush
15666 295/300 Write block 3190 len 32768 data: 8 8 7 5 4 5 5 4
15667 296/300 Read block 2130 len 8192
15668 297/300 Read block 1154 len 32768
15669 298/300 Read block 1906 len 4096
15670 299/300 Read block 4744 len 8192
15671 300/300 Read block 107 len 32768
15672 301/300 Write block 3001 len 8192 data: 2 4
15673 Main task has work to do, go find it
15674 got dsc Start(1)
15675 start 1
15676 Jan 13 15:55:06.564 INFO request completed, latency_us: 203, response_code: 204, uri: /start/cid/1, method: POST, req_id: 9498c7a9-564c-48ce-89fd-e930db61dbbf, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
15677 Replay: started 1, returned:Ok(())
15678 ----------------------------------------------------------------
15679 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
15680 States: Active Offline Active
15681 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15682 4925 Acked Read 1 Done Sent Done false
15683 4926 Acked Read 1 Done Sent Done false
15684 4927 Acked Write 3 Done Sent Done false
15685 4928 Acked Read 10 Done Sent Done false
15686 4929 Acked Read 1 Done Sent Done false
15687 4930 Acked Read 10 Done Sent Done false
15688 4931 Acked Read 9 Done Sent Done false
15689 4932 Acked Flush 0 Done Sent Done false
15690 4933 Acked Read 4 Done Sent Done false
15691 4934 Acked Write 8 Done Sent Done false
15692 4935 Acked Flush 0 Done Sent Done false
15693 4936 Acked Read 5 Done Sent Done false
15694 4937 Acked Write 10 Done Sent Done false
15695 4938 Acked Read 3 Done Sent Done false
15696 4939 Acked Write 6 Done Sent Done false
15697 4940 Acked Write 3 Done Sent Done false
15698 4941 Acked Read 7 Done Sent Done false
15699 4942 Acked Write 7 Done Sent Done false
15700 4943 Acked Write 9 Done Sent Done false
15701 4944 Acked Read 7 Done Sent Done false
15702 4945 Acked Write 2 Done Sent Done false
15703 4946 Acked Read 9 Done Sent Done false
15704 4947 Acked Write 1 Done Sent Done false
15705 4948 Acked Read 9 Done Sent Done false
15706 4949 Acked Read 1 Done Sent Done false
15707 4950 Acked Flush 0 Done Sent Done false
15708 4951 Acked Write 2 Done Sent Done false
15709 4952 Acked Read 5 Done Sent Done false
15710 4953 Acked Read 5 Done Sent Done false
15711 4954 Acked Write 7 Done Sent Done false
15712 4955 Acked Read 2 Done Sent Done false
15713 4956 Acked Write 2 Done Sent Done false
15714 4957 Acked Write 4 Done Sent Done false
15715 4958 Acked Read 6 Done Sent Done false
15716 4959 Acked Write 8 Done Sent Done false
15717 4960 Acked Read 9 Done Sent Done false
15718 4961 Acked Read 9 Done Sent Done false
15719 4962 Acked Read 10 Done Sent Done false
15720 4963 Acked Write 6 Done Sent Done false
15721 4964 Acked Write 5 Done Sent Done false
15722 4965 Acked Flush 0 Done Sent Done false
15723 4966 Acked Flush 0 Done Sent Done false
15724 4967 Acked Read 8 Done Sent Done false
15725 4968 Acked Write 1 Done Sent Done false
15726 4969 Acked Read 9 Done Sent Done false
15727 4970 Acked Flush 0 Done Sent Done false
15728 4971 Acked Read 2 Done Sent Done false
15729 4972 Acked Write 4 Done Sent Done false
15730 4973 Acked Read 8 Done Sent Done false
15731 4974 Acked Read 10 Done Sent Done false
15732 4975 Acked Write 6 Done Sent Done false
15733 4976 Acked Write 10 Done Sent Done false
15734 4977 Acked Read 4 Done Sent Done false
15735 4978 Acked Read 6 Done Sent Done false
15736 4979 Acked Read 9 Done Sent Done false
15737 4980 Acked Read 9 Done Sent Done false
15738 4981 Acked Read 10 Done Sent Done false
15739 4982 Acked Flush 0 Done Sent Done false
15740 4983 Acked Read 7 Done Sent Done false
15741 4984 Acked Write 5 Done Sent Done false
15742 4985 Acked Write 4 Done Sent Done false
15743 4986 Acked Write 3 Done Sent Done false
15744 4987 Acked Write 2 Done Sent Done false
15745 4988 Acked Flush 0 Done Sent Done false
15746 4989 Acked Read 5 Done Sent Done false
15747 4990 Acked Write 4 Done Sent Done false
15748 4991 Acked Read 2 Done Sent Done false
15749 4992 Acked Write 2 Done Sent Done false
15750 4993 Acked Read 8 Done Sent Done false
15751 4994 Acked Write 4 Done Sent Done false
15752 4995 Acked Write 3 Done Sent Done false
15753 4996 Acked Read 4 Done Sent Done false
15754 4997 Acked Write 3 Done Sent Done false
15755 4998 Acked Read 6 Done Sent Done false
15756 4999 Acked Read 10 Done Sent Done false
15757 5000 Acked Read 5 Done Sent Done false
15758 5001 Acked Write 5 Done Sent Done false
15759 5002 Acked Read 10 Done Sent Done false
15760 5003 Acked Read 8 Done Sent Done false
15761 5004 Acked Read 5 Done Sent Done false
15762 5005 Acked Read 10 Done Sent Done false
15763 5006 Acked Flush 0 Done Sent Done false
15764 5007 Acked Write 6 Done Sent Done false
15765 5008 Acked Write 2 Done Sent Done false
15766 5009 Acked Read 4 Done Sent Done false
15767 5010 Acked Write 7 Done Sent Done false
15768 5011 Acked Write 5 Done Sent Done false
15769 5012 Acked Read 10 Done Sent Done false
15770 5013 Acked Write 7 Done Sent Done false
15771 5014 Acked Write 1 Done Sent Done false
15772 5015 Acked Read 7 Done Sent Done false
15773 5016 Acked Write 10 Done Sent Done false
15774 5017 Acked Read 2 Done Sent Done false
15775 5018 Acked Read 8 Done Sent Done false
15776 5019 Acked Read 5 Done Sent Done false
15777 5020 Acked Read 7 Done Sent Done false
15778 5021 Acked Write 2 Done Sent Done false
15779 5022 Acked Write 3 Done Sent Done false
15780 5023 Acked Write 9 Done Sent Done false
15781 5024 Acked Read 5 Done Sent Done false
15782 5025 Acked Read 7 Done Sent Done false
15783 5026 Acked Write 7 Done Sent Done false
15784 5027 Acked Write 3 Done Sent Done false
15785 5028 Acked Read 4 Done Sent Done false
15786 5029 Acked Read 7 Done Sent Done false
15787 5030 Acked Read 7 Done Sent Done false
15788 5031 Acked Read 3 Done Sent Done false
15789 5032 Acked Read 4 Done Sent Done false
15790 5033 Acked Flush 0 Done Sent Done false
15791 5034 Acked Flush 0 Done Sent Done false
15792 5035 Acked Write 1 Done Sent Done false
15793 5036 Acked Read 3 Done Sent Done false
15794 5037 Acked Read 9 Done Sent Done false
15795 5038 Acked Read 8 Done Sent Done false
15796 5039 Acked Flush 0 Done Sent Done false
15797 5040 Acked Write 3 Done Sent Done false
15798 5041 Acked Read 1 Done Sent Done false
15799 5042 Acked Write 5 Done Sent Done false
15800 5043 Acked Read 5 Done Sent Done false
15801 5044 Acked Write 9 Done Sent Done false
15802 5045 Acked Write 10 Done Sent Done false
15803 5046 Acked Read 8 Done Sent Done false
15804 5047 Acked Read 2 Done Sent Done false
15805 5048 Acked Write 4 Done Sent Done false
15806 5049 Acked Write 10 Done Sent Done false
15807 5050 Acked Flush 0 Done Sent Done false
15808 5051 Acked Flush 0 Done Sent Done false
15809 5052 Acked Read 2 Done Sent Done false
15810 5053 Acked Read 10 Done Sent Done false
15811 5054 Acked Flush 0 Done Sent Done false
15812 5055 Acked Write 5 Done Sent Done false
15813 5056 Acked Write 3 Done Sent Done false
15814 5057 Acked Read 1 Done Sent Done false
15815 5058 Acked Flush 0 Done Sent Done false
15816 5059 Acked Read 4 Done Sent Done false
15817 5060 Acked Write 1 Done Sent Done false
15818 5061 Acked Read 6 Done Sent Done false
15819 5062 Acked Read 6 Done Sent Done false
15820 5063 Acked Read 7 Done Sent Done false
15821 5064 Acked Read 2 Done Sent Done false
15822 5065 Acked Read 2 Done Sent Done false
15823 5066 Acked Write 8 Done Sent Done false
15824 5067 Acked Read 8 Done Sent Done false
15825 5068 Acked Read 4 Done Sent Done false
15826 5069 Acked Read 4 Done Sent Done false
15827 5070 Acked Write 9 Done Sent Done false
15828 5071 Acked Read 2 Done Sent Done false
15829 5072 Acked Write 8 Done Sent Done false
15830 5073 Acked Flush 0 Done Sent Done false
15831 5074 Acked Write 2 Done Sent Done false
15832 5075 Acked Read 10 Done Sent Done false
15833 5076 Acked Read 4 Done Sent Done false
15834 5077 Acked Read 3 Done Sent Done false
15835 5078 Acked Write 7 Done Sent Done false
15836 5079 Acked Read 7 Done Sent Done false
15837 5080 Acked Read 3 Done Sent Done false
15838 5081 Acked Read 8 Done Sent Done false
15839 5082 Acked Flush 0 Done Sent Done false
15840 5083 Acked Flush 0 Done Sent Done false
15841 5084 Acked Write 6 Done Sent Done false
15842 5085 Acked Read 6 Done Sent Done false
15843 5086 Acked Read 6 Done Sent Done false
15844 5087 Acked Read 8 Done Sent Done false
15845 5088 Acked Write 3 Done Sent Done false
15846 5089 Acked Flush 0 Done Sent Done false
15847 5090 Acked Write 10 Done Sent Done false
15848 5091 Acked Read 6 Done Sent Done false
15849 5092 Acked Read 1 Done Sent Done false
15850 5093 Acked Read 3 Done Sent Done false
15851 5094 Acked Write 4 Done Sent Done false
15852 5095 Acked Write 9 Done Sent Done false
15853 5096 Acked Read 1 Done Sent Done false
15854 5097 Acked Write 3 Done Sent Done false
15855 5098 Acked Write 6 Done Sent Done false
15856 5099 Acked Read 10 Done Sent Done false
15857 5100 Acked Write 6 Done Sent Done false
15858 5101 Acked Read 9 Done Sent Done false
15859 5102 Acked Write 6 Done Sent Done false
15860 5103 Acked Read 6 Done Sent Done false
15861 5104 Acked Write 3 Done Sent Done false
15862 5105 Acked Write 6 Done Sent Done false
15863 5106 Acked Write 1 Done Sent Done false
15864 5107 Acked Read 3 Done Sent Done false
15865 5108 Acked Write 9 Done Sent Done false
15866 5109 Acked Write 9 Done Sent Done false
15867 5110 Acked Write 6 Done Sent Done false
15868 5111 Acked Read 4 Done Sent Done false
15869 5112 Acked Flush 0 Done Sent Done false
15870 5113 Acked Write 1 Done Sent Done false
15871 5114 Acked Read 4 Done Sent Done false
15872 5115 Acked Read 7 Done Sent Done false
15873 5116 Acked Read 2 Done Sent Done false
15874 5117 Acked Read 8 Done Sent Done false
15875 5118 Acked Write 6 Done Sent Done false
15876 5119 Acked Write 3 Done Sent Done false
15877 5120 Acked Write 5 Done Sent Done false
15878 5121 Acked Write 2 Done Sent Done false
15879 5122 Acked Read 9 Done Sent Done false
15880 5123 Acked Read 3 Done Sent Done false
15881 5124 Acked Write 4 Done Sent Done false
15882 5125 Acked Flush 0 Done Sent Done false
15883 5126 Acked Write 5 Done Sent Done false
15884 5127 Acked Write 4 Done Sent Done false
15885 5128 Acked Read 9 Done Sent Done false
15886 5129 Acked Write 7 Done Sent Done false
15887 5130 Acked Read 10 Done Sent Done false
15888 5131 Acked Read 7 Done Sent Done false
15889 5132 Acked Read 9 Done Sent Done false
15890 5133 Acked Write 6 Done Sent Done false
15891 5134 Acked Read 6 Done Sent Done false
15892 5135 Acked Flush 0 Done Sent Done false
15893 5136 Acked Write 7 Done Sent Done false
15894 5137 Acked Read 3 Done Sent Done false
15895 5138 Acked Flush 0 Done Sent Done false
15896 5139 Acked Read 8 Done Sent Done false
15897 5140 Acked Read 4 Done Sent Done false
15898 5141 Acked Write 4 Done Sent Done false
15899 5142 Acked Write 2 Done Sent Done false
15900 5143 Acked Write 9 Done Sent Done false
15901 5144 Acked Read 1 Done Sent Done false
15902 5145 Acked Write 7 Done Sent Done false
15903 5146 Acked Read 1 Done Sent Done false
15904 5147 Acked Read 4 Done Sent Done false
15905 5148 Acked Read 10 Done Sent Done false
15906 5149 Acked Write 2 Done Sent Done false
15907 5150 Acked Write 6 Done Sent Done false
15908 5151 Acked Flush 0 Done Sent Done false
15909 5152 Acked Read 4 Done Sent Done false
15910 5153 Acked Write 2 Done Sent Done false
15911 5154 Acked Write 4 Done Sent Done false
15912 5155 Acked Write 5 Done Sent Done false
15913 5156 Acked Write 8 Done Sent Done false
15914 5157 Acked Read 1 Done Sent Done false
15915 5158 Acked Write 10 Done Sent Done false
15916 5159 Acked Flush 0 Done Sent Done false
15917 5160 Acked Write 8 Done Sent Done false
15918 5161 Acked Read 2 Done Sent Done false
15919 5162 Acked Write 10 Done Sent Done false
15920 5163 Acked Read 4 Done Sent Done false
15921 5164 Acked Write 8 Done Sent Done false
15922 5165 Acked Write 2 Done Sent Done false
15923 5166 Acked Read 1 Done Sent Done false
15924 5167 Acked Read 2 Done Sent Done false
15925 5168 Acked Read 6 Done Sent Done false
15926 5169 Acked Flush 0 Done Sent Done false
15927 5170 Acked Write 4 Done Sent Done false
15928 5171 Acked Read 8 Done Sent Done false
15929 5172 Acked Flush 0 Done Sent Done false
15930 5173 Acked Write 7 Done Sent Done false
15931 5174 Acked Write 8 Done Sent Done false
15932 5175 Acked Read 2 Done Sent Done false
15933 5176 Acked Read 7 Done Sent Done false
15934 5177 Acked Write 5 Done Sent Done false
15935 5178 Acked Write 1 Done Sent Done false
15936 5179 Acked Write 7 Done Sent Done false
15937 5180 Acked Read 10 Done Sent Done false
15938 5181 Acked Read 6 Done Sent Done false
15939 5182 Acked Flush 0 Done Sent Done false
15940 5183 Acked Write 5 Done Sent Done false
15941 5184 Acked Write 7 Done Sent Done false
15942 5185 Acked Flush 0 Done Sent Done false
15943 5186 Acked Read 4 Done Sent Done false
15944 5187 Acked Read 7 Done Sent Done false
15945 5188 Acked Read 10 Done Sent Done false
15946 5189 Acked Write 8 Done Sent Done false
15947 5190 Acked Read 4 Done Sent Done false
15948 5191 Acked Read 5 Done Sent Done false
15949 5192 Acked Write 8 Done Sent Done false
15950 5193 Acked Read 6 Done Sent Done false
15951 5194 Acked Write 3 Done Sent Done false
15952 5195 Acked Read 1 Done Sent Done false
15953 5196 Acked Write 2 Done Sent Done false
15954 5197 Acked Flush 0 Done Sent Done false
15955 5198 Acked Read 1 Done Sent Done false
15956 5199 Acked Flush 0 Done Sent Done false
15957 5200 Acked Read 1 Done Sent Done false
15958 5201 Acked Write 10 Done Sent Done false
15959 5202 Acked Read 7 Done Sent Done false
15960 5203 Acked Write 2 Done Sent Done false
15961 5204 Acked Flush 0 Done Sent Done false
15962 5205 Acked Flush 0 Done Sent Done false
15963 5206 Acked Read 1 Done Sent Done false
15964 5207 Acked Write 6 Done Sent Done false
15965 5208 Acked Read 7 Done Sent Done false
15966 5209 Acked Read 5 Done Sent Done false
15967 5210 Acked Read 1 Done Sent Done false
15968 5211 Acked Read 1 Done Sent Done false
15969 5212 Acked Write 10 Done Sent Done false
15970 5213 Acked Write 10 Done Sent Done false
15971 5214 Acked Read 8 Done Sent Done false
15972 5215 Acked Read 3 Done Sent Done false
15973 5216 Acked Read 2 Done Sent Done false
15974 5217 Acked Read 4 Done Sent Done false
15975 5218 Acked Flush 0 Done Sent Done false
15976 5219 Acked Write 8 Done Sent Done false
15977 5220 Acked Read 2 Done Sent Done false
15978 5221 Acked Read 8 Done Sent Done false
15979 5222 Acked Read 1 Done Sent Done false
15980 5223 Acked Read 2 Done Sent Done false
15981 5224 Acked Read 8 Done Sent Done false
15982 5225 Acked Write 2 Done Sent Done false
15983 STATES DS:0 DS:1 DS:2 TOTAL
15984 Sent 0 301 0 301
15985 Done 301 0 301 602
15986 Skipped 0 0 0 0
15987 Error 0 0 0 0
15988 Last Flush: 5218 4924 5218
15989 Downstairs last five completed: 4924 4923 4922 4921 4920
15990 Upstairs last five completed: 5225 5224 5223 5222 5221
15991 CLIENT: Up:0 ds:301 act:2
15992 [8820] got start action
15993 [8820] I am going to restart
15994 Starting downstairs at port 8820
15995 Make output file at "/tmp/dsc/downstairs-8820.txt"
15996 [8820][1] reports Starting
15997 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1216)
15998 [8820][1] reports Running
15999 ----------------------------------------------------------------
16000 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
16001 States: Active Offline Active
16002 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16003 4925 Acked Read 1 Done Sent Done false
16004 4926 Acked Read 1 Done Sent Done false
16005 4927 Acked Write 3 Done Sent Done false
16006 4928 Acked Read 10 Done Sent Done false
16007 4929 Acked Read 1 Done Sent Done false
16008 4930 Acked Read 10 Done Sent Done false
16009 4931 Acked Read 9 Done Sent Done false
16010 4932 Acked Flush 0 Done Sent Done false
16011 4933 Acked Read 4 Done Sent Done false
16012 4934 Acked Write 8 Done Sent Done false
16013 4935 Acked Flush 0 Done Sent Done false
16014 4936 Acked Read 5 Done Sent Done false
16015 4937 Acked Write 10 Done Sent Done false
16016 4938 Acked Read 3 Done Sent Done false
16017 4939 Acked Write 6 Done Sent Done false
16018 4940 Acked Write 3 Done Sent Done false
16019 4941 Acked Read 7 Done Sent Done false
16020 4942 Acked Write 7 Done Sent Done false
16021 4943 Acked Write 9 Done Sent Done false
16022 4944 Acked Read 7 Done Sent Done false
16023 4945 Acked Write 2 Done Sent Done false
16024 4946 Acked Read 9 Done Sent Done false
16025 4947 Acked Write 1 Done Sent Done false
16026 4948 Acked Read 9 Done Sent Done false
16027 4949 Acked Read 1 Done Sent Done false
16028 4950 Acked Flush 0 Done Sent Done false
16029 4951 Acked Write 2 Done Sent Done false
16030 4952 Acked Read 5 Done Sent Done false
16031 4953 Acked Read 5 Done Sent Done false
16032 4954 Acked Write 7 Done Sent Done false
16033 4955 Acked Read 2 Done Sent Done false
16034 4956 Acked Write 2 Done Sent Done false
16035 4957 Acked Write 4 Done Sent Done false
16036 4958 Acked Read 6 Done Sent Done false
16037 4959 Acked Write 8 Done Sent Done false
16038 4960 Acked Read 9 Done Sent Done false
16039 4961 Acked Read 9 Done Sent Done false
16040 4962 Acked Read 10 Done Sent Done false
16041 4963 Acked Write 6 Done Sent Done false
16042 4964 Acked Write 5 Done Sent Done false
16043 4965 Acked Flush 0 Done Sent Done false
16044 4966 Acked Flush 0 Done Sent Done false
16045 4967 Acked Read 8 Done Sent Done false
16046 4968 Acked Write 1 Done Sent Done false
16047 4969 Acked Read 9 Done Sent Done false
16048 4970 Acked Flush 0 Done Sent Done false
16049 4971 Acked Read 2 Done Sent Done false
16050 4972 Acked Write 4 Done Sent Done false
16051 4973 Acked Read 8 Done Sent Done false
16052 4974 Acked Read 10 Done Sent Done false
16053 4975 Acked Write 6 Done Sent Done false
16054 4976 Acked Write 10 Done Sent Done false
16055 4977 Acked Read 4 Done Sent Done false
16056 4978 Acked Read 6 Done Sent Done false
16057 4979 Acked Read 9 Done Sent Done false
16058 4980 Acked Read 9 Done Sent Done false
16059 4981 Acked Read 10 Done Sent Done false
16060 4982 Acked Flush 0 Done Sent Done false
16061 4983 Acked Read 7 Done Sent Done false
16062 4984 Acked Write 5 Done Sent Done false
16063 4985 Acked Write 4 Done Sent Done false
16064 4986 Acked Write 3 Done Sent Done false
16065 4987 Acked Write 2 Done Sent Done false
16066 4988 Acked Flush 0 Done Sent Done false
16067 4989 Acked Read 5 Done Sent Done false
16068 4990 Acked Write 4 Done Sent Done false
16069 4991 Acked Read 2 Done Sent Done false
16070 4992 Acked Write 2 Done Sent Done false
16071 4993 Acked Read 8 Done Sent Done false
16072 4994 Acked Write 4 Done Sent Done false
16073 4995 Acked Write 3 Done Sent Done false
16074 4996 Acked Read 4 Done Sent Done false
16075 4997 Acked Write 3 Done Sent Done false
16076 4998 Acked Read 6 Done Sent Done false
16077 4999 Acked Read 10 Done Sent Done false
16078 5000 Acked Read 5 Done Sent Done false
16079 5001 Acked Write 5 Done Sent Done false
16080 5002 Acked Read 10 Done Sent Done false
16081 5003 Acked Read 8 Done Sent Done false
16082 5004 Acked Read 5 Done Sent Done false
16083 5005 Acked Read 10 Done Sent Done false
16084 5006 Acked Flush 0 Done Sent Done false
16085 5007 Acked Write 6 Done Sent Done false
16086 5008 Acked Write 2 Done Sent Done false
16087 5009 Acked Read 4 Done Sent Done false
16088 5010 Acked Write 7 Done Sent Done false
16089 5011 Acked Write 5 Done Sent Done false
16090 5012 Acked Read 10 Done Sent Done false
16091 5013 Acked Write 7 Done Sent Done false
16092 5014 Acked Write 1 Done Sent Done false
16093 5015 Acked Read 7 Done Sent Done false
16094 5016 Acked Write 10 Done Sent Done false
16095 5017 Acked Read 2 Done Sent Done false
16096 5018 Acked Read 8 Done Sent Done false
16097 5019 Acked Read 5 Done Sent Done false
16098 5020 Acked Read 7 Done Sent Done false
16099 5021 Acked Write 2 Done Sent Done false
16100 5022 Acked Write 3 Done Sent Done false
16101 5023 Acked Write 9 Done Sent Done false
16102 5024 Acked Read 5 Done Sent Done false
16103 5025 Acked Read 7 Done Sent Done false
16104 5026 Acked Write 7 Done Sent Done false
16105 5027 Acked Write 3 Done Sent Done false
16106 5028 Acked Read 4 Done Sent Done false
16107 5029 Acked Read 7 Done Sent Done false
16108 5030 Acked Read 7 Done Sent Done false
16109 5031 Acked Read 3 Done Sent Done false
16110 5032 Acked Read 4 Done Sent Done false
16111 5033 Acked Flush 0 Done Sent Done false
16112 5034 Acked Flush 0 Done Sent Done false
16113 5035 Acked Write 1 Done Sent Done false
16114 5036 Acked Read 3 Done Sent Done false
16115 5037 Acked Read 9 Done Sent Done false
16116 5038 Acked Read 8 Done Sent Done false
16117 5039 Acked Flush 0 Done Sent Done false
16118 5040 Acked Write 3 Done Sent Done false
16119 5041 Acked Read 1 Done Sent Done false
16120 5042 Acked Write 5 Done Sent Done false
16121 5043 Acked Read 5 Done Sent Done false
16122 5044 Acked Write 9 Done Sent Done false
16123 5045 Acked Write 10 Done Sent Done false
16124 5046 Acked Read 8 Done Sent Done false
16125 5047 Acked Read 2 Done Sent Done false
16126 5048 Acked Write 4 Done Sent Done false
16127 5049 Acked Write 10 Done Sent Done false
16128 5050 Acked Flush 0 Done Sent Done false
16129 5051 Acked Flush 0 Done Sent Done false
16130 5052 Acked Read 2 Done Sent Done false
16131 5053 Acked Read 10 Done Sent Done false
16132 5054 Acked Flush 0 Done Sent Done false
16133 5055 Acked Write 5 Done Sent Done false
16134 5056 Acked Write 3 Done Sent Done false
16135 5057 Acked Read 1 Done Sent Done false
16136 5058 Acked Flush 0 Done Sent Done false
16137 5059 Acked Read 4 Done Sent Done false
16138 5060 Acked Write 1 Done Sent Done false
16139 5061 Acked Read 6 Done Sent Done false
16140 5062 Acked Read 6 Done Sent Done false
16141 5063 Acked Read 7 Done Sent Done false
16142 5064 Acked Read 2 Done Sent Done false
16143 5065 Acked Read 2 Done Sent Done false
16144 5066 Acked Write 8 Done Sent Done false
16145 5067 Acked Read 8 Done Sent Done false
16146 5068 Acked Read 4 Done Sent Done false
16147 5069 Acked Read 4 Done Sent Done false
16148 5070 Acked Write 9 Done Sent Done false
16149 5071 Acked Read 2 Done Sent Done false
16150 5072 Acked Write 8 Done Sent Done false
16151 5073 Acked Flush 0 Done Sent Done false
16152 5074 Acked Write 2 Done Sent Done false
16153 5075 Acked Read 10 Done Sent Done false
16154 5076 Acked Read 4 Done Sent Done false
16155 5077 Acked Read 3 Done Sent Done false
16156 5078 Acked Write 7 Done Sent Done false
16157 5079 Acked Read 7 Done Sent Done false
16158 5080 Acked Read 3 Done Sent Done false
16159 5081 Acked Read 8 Done Sent Done false
16160 5082 Acked Flush 0 Done Sent Done false
16161 5083 Acked Flush 0 Done Sent Done false
16162 5084 Acked Write 6 Done Sent Done false
16163 5085 Acked Read 6 Done Sent Done false
16164 5086 Acked Read 6 Done Sent Done false
16165 5087 Acked Read 8 Done Sent Done false
16166 5088 Acked Write 3 Done Sent Done false
16167 5089 Acked Flush 0 Done Sent Done false
16168 5090 Acked Write 10 Done Sent Done false
16169 5091 Acked Read 6 Done Sent Done false
16170 5092 Acked Read 1 Done Sent Done false
16171 5093 Acked Read 3 Done Sent Done false
16172 5094 Acked Write 4 Done Sent Done false
16173 5095 Acked Write 9 Done Sent Done false
16174 5096 Acked Read 1 Done Sent Done false
16175 5097 Acked Write 3 Done Sent Done false
16176 5098 Acked Write 6 Done Sent Done false
16177 5099 Acked Read 10 Done Sent Done false
16178 5100 Acked Write 6 Done Sent Done false
16179 5101 Acked Read 9 Done Sent Done false
16180 5102 Acked Write 6 Done Sent Done false
16181 5103 Acked Read 6 Done Sent Done false
16182 5104 Acked Write 3 Done Sent Done false
16183 5105 Acked Write 6 Done Sent Done false
16184 5106 Acked Write 1 Done Sent Done false
16185 5107 Acked Read 3 Done Sent Done false
16186 5108 Acked Write 9 Done Sent Done false
16187 5109 Acked Write 9 Done Sent Done false
16188 5110 Acked Write 6 Done Sent Done false
16189 5111 Acked Read 4 Done Sent Done false
16190 5112 Acked Flush 0 Done Sent Done false
16191 5113 Acked Write 1 Done Sent Done false
16192 5114 Acked Read 4 Done Sent Done false
16193 5115 Acked Read 7 Done Sent Done false
16194 5116 Acked Read 2 Done Sent Done false
16195 5117 Acked Read 8 Done Sent Done false
16196 5118 Acked Write 6 Done Sent Done false
16197 5119 Acked Write 3 Done Sent Done false
16198 5120 Acked Write 5 Done Sent Done false
16199 5121 Acked Write 2 Done Sent Done false
16200 5122 Acked Read 9 Done Sent Done false
16201 5123 Acked Read 3 Done Sent Done false
16202 5124 Acked Write 4 Done Sent Done false
16203 5125 Acked Flush 0 Done Sent Done false
16204 5126 Acked Write 5 Done Sent Done false
16205 5127 Acked Write 4 Done Sent Done false
16206 5128 Acked Read 9 Done Sent Done false
16207 5129 Acked Write 7 Done Sent Done false
16208 5130 Acked Read 10 Done Sent Done false
16209 5131 Acked Read 7 Done Sent Done false
16210 5132 Acked Read 9 Done Sent Done false
16211 5133 Acked Write 6 Done Sent Done false
16212 5134 Acked Read 6 Done Sent Done false
16213 5135 Acked Flush 0 Done Sent Done false
16214 5136 Acked Write 7 Done Sent Done false
16215 5137 Acked Read 3 Done Sent Done false
16216 5138 Acked Flush 0 Done Sent Done false
16217 5139 Acked Read 8 Done Sent Done false
16218 5140 Acked Read 4 Done Sent Done false
16219 5141 Acked Write 4 Done Sent Done false
16220 5142 Acked Write 2 Done Sent Done false
16221 5143 Acked Write 9 Done Sent Done false
16222 5144 Acked Read 1 Done Sent Done false
16223 5145 Acked Write 7 Done Sent Done false
16224 5146 Acked Read 1 Done Sent Done false
16225 5147 Acked Read 4 Done Sent Done false
16226 5148 Acked Read 10 Done Sent Done false
16227 5149 Acked Write 2 Done Sent Done false
16228 5150 Acked Write 6 Done Sent Done false
16229 5151 Acked Flush 0 Done Sent Done false
16230 5152 Acked Read 4 Done Sent Done false
16231 5153 Acked Write 2 Done Sent Done false
16232 5154 Acked Write 4 Done Sent Done false
16233 5155 Acked Write 5 Done Sent Done false
16234 5156 Acked Write 8 Done Sent Done false
16235 5157 Acked Read 1 Done Sent Done false
16236 5158 Acked Write 10 Done Sent Done false
16237 5159 Acked Flush 0 Done Sent Done false
16238 5160 Acked Write 8 Done Sent Done false
16239 5161 Acked Read 2 Done Sent Done false
16240 5162 Acked Write 10 Done Sent Done false
16241 5163 Acked Read 4 Done Sent Done false
16242 5164 Acked Write 8 Done Sent Done false
16243 5165 Acked Write 2 Done Sent Done false
16244 5166 Acked Read 1 Done Sent Done false
16245 5167 Acked Read 2 Done Sent Done false
16246 5168 Acked Read 6 Done Sent Done false
16247 5169 Acked Flush 0 Done Sent Done false
16248 5170 Acked Write 4 Done Sent Done false
16249 5171 Acked Read 8 Done Sent Done false
16250 5172 Acked Flush 0 Done Sent Done false
16251 5173 Acked Write 7 Done Sent Done false
16252 5174 Acked Write 8 Done Sent Done false
16253 5175 Acked Read 2 Done Sent Done false
16254 5176 Acked Read 7 Done Sent Done false
16255 5177 Acked Write 5 Done Sent Done false
16256 5178 Acked Write 1 Done Sent Done false
16257 5179 Acked Write 7 Done Sent Done false
16258 5180 Acked Read 10 Done Sent Done false
16259 5181 Acked Read 6 Done Sent Done false
16260 5182 Acked Flush 0 Done Sent Done false
16261 5183 Acked Write 5 Done Sent Done false
16262 5184 Acked Write 7 Done Sent Done false
16263 5185 Acked Flush 0 Done Sent Done false
16264 5186 Acked Read 4 Done Sent Done false
16265 5187 Acked Read 7 Done Sent Done false
16266 5188 Acked Read 10 Done Sent Done false
16267 5189 Acked Write 8 Done Sent Done false
16268 5190 Acked Read 4 Done Sent Done false
16269 5191 Acked Read 5 Done Sent Done false
16270 5192 Acked Write 8 Done Sent Done false
16271 5193 Acked Read 6 Done Sent Done false
16272 5194 Acked Write 3 Done Sent Done false
16273 5195 Acked Read 1 Done Sent Done false
16274 5196 Acked Write 2 Done Sent Done false
16275 5197 Acked Flush 0 Done Sent Done false
16276 5198 Acked Read 1 Done Sent Done false
16277 5199 Acked Flush 0 Done Sent Done false
16278 5200 Acked Read 1 Done Sent Done false
16279 5201 Acked Write 10 Done Sent Done false
16280 5202 Acked Read 7 Done Sent Done false
16281 5203 Acked Write 2 Done Sent Done false
16282 5204 Acked Flush 0 Done Sent Done false
16283 5205 Acked Flush 0 Done Sent Done false
16284 5206 Acked Read 1 Done Sent Done false
16285 5207 Acked Write 6 Done Sent Done false
16286 5208 Acked Read 7 Done Sent Done false
16287 5209 Acked Read 5 Done Sent Done false
16288 5210 Acked Read 1 Done Sent Done false
16289 5211 Acked Read 1 Done Sent Done false
16290 5212 Acked Write 10 Done Sent Done false
16291 5213 Acked Write 10 Done Sent Done false
16292 5214 Acked Read 8 Done Sent Done false
16293 5215 Acked Read 3 Done Sent Done false
16294 5216 Acked Read 2 Done Sent Done false
16295 5217 Acked Read 4 Done Sent Done false
16296 5218 Acked Flush 0 Done Sent Done false
16297 5219 Acked Write 8 Done Sent Done false
16298 5220 Acked Read 2 Done Sent Done false
16299 5221 Acked Read 8 Done Sent Done false
16300 5222 Acked Read 1 Done Sent Done false
16301 5223 Acked Read 2 Done Sent Done false
16302 5224 Acked Read 8 Done Sent Done false
16303 5225 Acked Write 2 Done Sent Done false
16304 5226 Acked Flush 0 Done Sent Done false
16305 STATES DS:0 DS:1 DS:2 TOTAL
16306 Sent 0 302 0 302
16307 Done 302 0 302 604
16308 Skipped 0 0 0 0
16309 Error 0 0 0 0
16310 Last Flush: 5226 4924 5226
16311 Downstairs last five completed: 4924 4923 4922 4921 4920
16312 Upstairs last five completed: 5226 5225 5224 5223 5222
16313 CLIENT: Up:0 ds:302 act:2
163142025-01-13T15:55:12.325ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
163152025-01-13T15:55:12.326ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36928) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
163162025-01-13T15:55:12.327ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
163172025-01-13T15:55:12.327ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
163182025-01-13T15:55:12.328ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
163192025-01-13T15:55:12.328ZINFOcrucible: send last flush ID to this DS: 4924 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
163202025-01-13T15:55:12.328ZINFOcrucible: Replied this last flush ID: 4924 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
163212025-01-13T15:55:12.328ZINFOcrucible: [1] client re-new 302 jobs since flush 4924 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
163222025-01-13T15:55:12.328ZINFOcrucible: [1] Replayed 302 jobs since flush: 4924 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
16323 ----------------------------------------------------------------
16324 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
16325 Downstairs last five completed: 5226 5225 5224 5223 5222
16326 Upstairs last five completed: 5226 5225 5224 5223 5222
16327 CLIENT: Up:0 ds:0 act:3
16328 Replay: All jobs finished
16329 Main task has work to do, go find it
16330 got dsc Stop(1)
16331 stop 1
16332 Jan 13 15:55:14.570 INFO request completed, latency_us: [8820] Got stop action so:false kr:false
16333 357, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 9298d294-3eef-4853-b117-baac4108ca12, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
163342025-01-13T15:55:14.571ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
163352025-01-13T15:55:14.571ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
163362025-01-13T15:55:14.571ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
16337 Jan 13 15:55:14.571 INFO request completed, latency_us: 116, response_code: 200, uri: /state/cid/1, method: GET, req_id: c9be1ffe-b1ce-4bcb-9cb5-b56af549ad9b, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
163382025-01-13T15:55:14.571ZINFOcrucible: client task is exiting = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
16339 [8820] Exited with: ExitStatus(unix_wait_status(9))
16340 [8820][1] reports Exit
16341 Jan 13 15:55:18.574 INFO request completed, latency_us: 349, response_code: 200, uri: /state/cid/1, method: GET, req_id: b64dbf60-a357-4248-8c40-397ca54d7b74, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
16342 001/300 Write block 487 len 4096 data: 3
16343 002/300 Read block 2817 len 24576
16344 003/300 Read block 2830 len 12288
16345 004/300 Write block 872 len 40960 data: 3 2 4 3 5 4 4 5 6 5
16346 005/300 Write block 4829 len 32768 data: 3 3 2 3 3 4 5 5
16347 006/300 Write block 4257 len 8192 data: 4 4
16348 007/300 Write block 4317 len 16384 data: 5 5 4 5
16349 008/300 Read block 3656 len 40960
16350 009/300 Write block 3186 len 32768 data: 5 5 6 9 9 9 8 6
16351 010/300 Read block 504 len 4096
16352 011/300 Read block 2673 len 32768
16353 012/300 Read block 2740 len 36864
16354 013/300 Write block 696 len 28672 data: 5 4 4 4 4 4 4
16355 014/300 Read block 1922 len 16384
16356 015/300 Flush
16357 016/300 Read block 2811 len 20480
16358 017/300 Read block 641 len 40960
16359 018/300 Read block 1578 len 16384
16360 019/300 Write block 662 len 32768 data: 6 6 8 7 7 6 7 7
16361 020/300 Read block 4834 len 28672
16362 021/300 Write block 4378 len 8192 data: 5 4
16363 022/300 Read block 2692 len 24576
16364 023/300 Write block 4825 len 24576 data: 2 2 2 2 4 4
16365 024/300 Read block 223 len 24576
16366 025/300 Read block 4706 len 4096
16367 026/300 Read block 481 len 28672
16368 027/300 Write block 2511 len 40960 data: 2 2 2 2 2 3 3 3 3 3
16369 028/300 Write block 639 len 12288 data: 2 3 3
16370 029/300 Read block 4447 len 8192
16371 030/300 Read block 3388 len 36864
16372 031/300 Read block 4519 len 28672
16373 032/300 Read block 3191 len 36864
16374 033/300 Read block 1021 len 24576
16375 034/300 Read block 2422 len 4096
16376 035/300 Read block 3471 len 8192
16377 036/300 Read block 1637 len 8192
16378 037/300 Read block 4512 len 24576
16379 038/300 Read block 4780 len 8192
16380 039/300 Write block 3454 len 12288 data: 4 4 4
16381 040/300 Flush
16382 041/300 Write block 3637 len 32768 data: 3 4 3 4 4 4 4 5
16383 042/300 Read block 3674 len 40960
16384 043/300 Read block 1936 len 32768
16385 044/300 Read block 885 len 36864
16386 045/300 Read block 974 len 28672
16387 046/300 Write block 2072 len 12288 data: 2 2 2
16388 047/300 Write block 2120 len 16384 data: 5 5 6 5
16389 048/300 Read block 2510 len 36864
16390 049/300 Read block 706 len 28672
16391 050/300 Write block 2224 len 20480 data: 5 5 6 6 6
16392 051/300 Write block 125 len 24576 data: 6 5 5 5 5 6
16393 052/300 Flush
16394 053/300 Read block 3370 len 20480
16395 054/300 Write block 3780 len 16384 data: 7 5 4 4
16396 055/300 Write block 4393 len 32768 data: 3 3 3 4 5 5 5 4
16397 056/300 Read block 4091 len 28672
16398 057/300 Read block 4014 len 32768
16399 058/300 Write block 56 len 4096 data: 4
16400 059/300 Write block 664 len 28672 data: 9 8 8 7 8 8 6
16401 060/300 Read block 1686 len 4096
16402 061/300 Read block 1150 len 16384
16403 062/300 Read block 1791 len 32768
16404 063/300 Write block 281 len 32768 data: 3 4 4 5 7 7 7 6
16405 064/300 Write block 3348 len 40960 data: 4 5 4 5 4 3 3 4 5 5
16406 065/300 Read block 2951 len 12288
16407 066/300 Read block 4240 len 16384
16408 067/300 Read block 1532 len 32768
16409 068/300 Flush
16410 069/300 Read block 3416 len 16384
16411 070/300 Write block 3513 len 12288 data: 7 8 7
16412 071/300 Read block 4980 len 40960
16413 072/300 Read block 556 len 40960
16414 073/300 Flush
16415 074/300 Write block 960 len 4096 data: 5
16416 075/300 Read block 4305 len 16384
16417 076/300 Write block 4803 len 8192 data: 4 4
16418 077/300 Write block 4639 len 4096 data: 3
16419 078/300 Read block 3873 len 20480
16420 079/300 Read block 425 len 36864
16421 080/300 Read block 2440 len 20480
16422 081/300 Read block 3400 len 16384
16423 082/300 Write block 221 len 4096 data: 4
16424 083/300 Write block 4851 len 4096 data: 4
16425 084/300 Write block 350 len 32768 data: 2 3 2 3 3 4 4 3
16426 085/300 Read block 438 len 40960
16427 086/300 Read block 2 len 32768
16428 087/300 Read block 1749 len 16384
16429 088/300 Read block 230 len 20480
16430 089/300 Read block 4075 len 8192
16431 090/300 Flush
16432 091/300 Read block 3849 len 40960
16433 092/300 Read block 1165 len 40960
16434 093/300 Read block 4360 len 40960
16435 094/300 Write block 1657 len 28672 data: 2 3 3 3 3 3 3
16436 095/300 Flush
16437 096/300 Read block 2117 len 28672
16438 097/300 Read block 4367 len 12288
16439 098/300 Read block 3073 len 36864
16440 099/300 Read block 1102 len 4096
16441 100/300 Write block 2916 len 20480 data: 3 3 3 2 2
16442 101/300 Read block 100 len 32768
16443 102/300 Read block 3211 len 4096
16444 103/300 Read block 632 len 28672
16445 104/300 Read block 825 len 40960
16446 105/300 Write block 1936 len 16384 data: 2 2 2 3
16447 106/300 Read block 1670 len 24576
16448 107/300 Flush
16449 108/300 Read block 2241 len 20480
16450 109/300 Read block 2057 len 8192
16451 110/300 Write block 312 len 12288 data: 3 3 3
16452 111/300 Write block 1299 len 36864 data: 4 4 3 3 4 4 2 2 2
16453 112/300 Write block 4246 len 28672 data: 4 3 2 3 5 5 5
16454 113/300 Write block 2902 len 16384 data: 5 3 2 2
16455 114/300 Read block 4471 len 20480
16456 115/300 Write block 425 len 32768 data: 5 5 4 5 3 3 3 3
16457 116/300 Read block 1643 len 4096
16458 117/300 Read block 1249 len 4096
16459 118/300 Write block 781 len 12288 data: 4 4 5
16460 119/300 Read block 1594 len 16384
16461 120/300 Write block 3288 len 16384 data: 4 4 5 4
16462 121/300 Read block 3086 len 4096
16463 122/300 Read block 2077 len 12288
16464 123/300 Write block 803 len 24576 data: 4 4 4 4 3 3
16465 124/300 Read block 630 len 20480
16466 125/300 Flush
16467 126/300 Write block 4912 len 36864 data: 2 2 2 2 2 3 3 3 3
16468 127/300 Write block 2834 len 12288 data: 3 3 3
16469 128/300 Read block 4665 len 12288
16470 129/300 Write block 3137 len 36864 data: 2 3 3 3 3 4 4 4 4
16471 130/300 Write block 65 len 8192 data: 4 4
16472 131/300 Write block 1319 len 40960 data: 4 4 4 3 2 2 3 4 4 5
16473 132/300 Write block 1099 len 24576 data: 3 3 3 3 2 2
16474 133/300 Read block 3111 len 4096
16475 134/300 Write block 22 len 4096 data: 3
16476 135/300 Read block 2705 len 4096
16477 136/300 Write block 1785 len 4096 data: 2
16478 137/300 Write block 1613 len 16384 data: 4 3 3 6
16479 138/300 Write block 1126 len 28672 data: 2 2 2 2 2 2 3
16480 139/300 Flush
16481 140/300 Read block 605 len 36864
16482 141/300 Write block 2658 len 12288 data: 2 2 2
16483 142/300 Write block 2768 len 24576 data: 5 5 5 4 2 2
16484 143/300 Write block 722 len 4096 data: 3
16485 144/300 Read block 329 len 32768
16486 145/300 Read block 3324 len 24576
16487 146/300 Read block 1070 len 4096
16488 147/300 Write block 1076 len 20480 data: 2 2 3 3 4
16489 148/300 Read block 2676 len 36864
16490 149/300 Read block 2109 len 40960
16491 150/300 Read block 1492 len 36864
16492 151/300 Read block 3228 len 20480
16493 152/300 Read block 922 len 32768
16494 153/300 Write block 4070 len 20480 data: 4 4 3 4 4
16495 154/300 Read block 4945 len 36864
16496 155/300 Read block 3121 len 20480
16497 156/300 Read block 681 len 28672
16498 157/300 Read block 3478 len 12288
16499 158/300 Read block 2390 len 4096
16500 159/300 Flush
16501 160/300 Read block 2609 len 40960
16502 161/300 Read block 914 len 8192
16503 162/300 Flush
16504 163/300 Read block 2261 len 20480
16505 164/300 Write block 712 len 32768 data: 4 4 4 4 4 4 4 4
16506 165/300 Read block 3632 len 36864
16507 166/300 Read block 125 len 12288
16508 167/300 Write block 348 len 12288 data: 4 3 3
16509 168/300 Read block 4325 len 36864
16510 169/300 Read block 2621 len 20480
16511 170/300 Flush
16512 171/300 Write block 2593 len 16384 data: 9 9 10 11
16513 172/300 Read block 2885 len 36864
16514 173/300 Write block 1237 len 36864 data: 5 5 5 5 5 4 4 3 3
16515 174/300 Write block 1493 len 12288 data: 5 5 4
16516 175/300 Write block 4732 len 36864 data: 6 5 6 8 7 7 6 6 6
16517 176/300 Read block 4180 len 20480
16518 177/300 Write block 1498 len 20480 data: 6 7 6 5 4
16519 178/300 Read block 4187 len 40960
16520 179/300 Read block 2552 len 32768
16521 180/300 Read block 2880 len 28672
16522 181/300 Flush
16523 182/300 Flush
16524 183/300 Read block 4431 len 12288
16525 184/300 Read block 1990 len 40960
16526 185/300 Read block 3040 len 28672
16527 186/300 Write block 4501 len 12288 data: 5 6 6
16528 187/300 Read block 4008 len 28672
16529 188/300 Read block 3781 len 8192
16530 189/300 Read block 146 len 40960
16531 190/300 Write block 4869 len 40960 data: 6 4 3 3 3 3 3 2 2 2
16532 191/300 Write block 1407 len 36864 data: 2 3 4 4 4 5 4 4 5
16533 192/300 Read block 4974 len 36864
16534 193/300 Write block 4448 len 32768 data: 8 7 6 5 4 3 4 4
16535 194/300 Flush
16536 195/300 Write block 1649 len 8192 data: 4 5
16537 196/300 Write block 3931 len 32768 data: 3 3 3 3 3 3 4 4
16538 197/300 Write block 988 len 8192 data: 6 6
16539 198/300 Read block 3546 len 40960
16540 199/300 Write block 4052 len 40960 data: 4 4 4 4 4 4 5 3 3 2
16541 200/300 Read block 4554 len 16384
16542 201/300 Read block 885 len 12288
16543 202/300 Write block 2379 len 12288 data: 3 3 4
16544 203/300 Flush
16545 204/300 Read block 4034 len 28672
16546 205/300 Read block 319 len 36864
16547 206/300 Read block 2550 len 12288
16548 207/300 Read block 4583 len 20480
16549 208/300 Write block 4841 len 4096 data: 4
16550 209/300 Write block 2640 len 20480 data: 5 6 5 6 6
16551 210/300 Read block 2522 len 8192
16552 211/300 Read block 3481 len 12288
16553 212/300 Flush
16554 213/300 Read block 2257 len 16384
16555 214/300 Write block 2080 len 36864 data: 5 4 5 7 5 4 4 4 3
16556 215/300 Read block 4258 len 28672
16557 216/300 Write block 3588 len 8192 data: 3 5
16558 217/300 Write block 3318 len 16384 data: 2 2 3 4
16559 218/300 Read block 4620 len 16384
16560 219/300 Write block 1695 len 4096 data: 5
16561 220/300 Read block 3789 len 24576
16562 221/300 Flush
16563 222/300 Write block 691 len 28672 data: 3 3 3 4 4 6 5
16564 223/300 Flush
16565 224/300 Write block 407 len 8192 data: 3 3
16566 225/300 Write block 726 len 28672 data: 4 5 5 5 6 6 5
16567 226/300 Write block 2643 len 8192 data: 7 7
16568 227/300 Flush
16569 228/300 Flush
16570 229/300 Write block 4020 len 8192 data: 4 3
16571 230/300 Flush
16572 231/300 Read block 3076 len 24576
16573 232/300 Read block 2197 len 32768
16574 233/300 Read block 55 len 12288
16575 234/300 Read block 2534 len 8192
16576 235/300 Write block 1834 len 40960 data: 3 3 3 4 5 4 3 3 4 4
16577 236/300 Write block 67 len 4096 data: 4
16578 237/300 Write block 777 len 40960 data: 4 4 3 3 5 5 6 5 4 4
16579 238/300 Write block 3272 len 20480 data: 5 7 6 7 6
16580 239/300 Read block 2474 len 4096
16581 240/300 Read block 4696 len 28672
16582 241/300 Read block 3405 len 12288
16583 242/300 Flush
16584 243/300 Write block 1568 len 24576 data: 3 3 5 4 3 3
16585 244/300 Read block 4551 len 4096
16586 245/300 Read block 125 len 28672
16587 246/300 Write block 2996 len 36864 data: 2 2 2 2 2 3 5 5 6
16588 247/300 Write block 1266 len 40960 data: 3 3 3 3 3 5 4 4 4 4
16589 248/300 Read block 4477 len 36864
16590 249/300 Read block 655 len 24576
16591 250/300 Write block 3008 len 36864 data: 5 6 6 5 4 3 3 3 3
16592 251/300 Read block 2957 len 24576
16593 252/300 Write block 3979 len 32768 data: 6 6 6 5 5 6 5 5
16594 253/300 Read block 3736 len 4096
16595 254/300 Write block 4483 len 12288 data: 3 4 3
16596 255/300 Write block 1520 len 16384 data: 5 4 5 5
16597 256/300 Write block 1493 len 24576 data: 6 6 5 4 4 7
16598 257/300 Read block 4908 len 24576
16599 258/300 Read block 4618 len 32768
16600 259/300 Read block 2446 len 12288
16601 260/300 Write block 1978 len 4096 data: 3
16602 261/300 Read block 2208 len 28672
16603 262/300 Write block 4395 len 40960 data: 4 5 6 6 6 5 3 3 2 3
16604 263/300 Read block 2666 len 36864
16605 264/300 Read block 4293 len 28672
16606 265/300 Read block 2205 len 36864
16607 266/300 Write block 4744 len 24576 data: 5 2 2 2 2 5
16608 267/300 Read block 1725 len 20480
16609 268/300 Flush
16610 269/300 Write block 2226 len 20480 data: 7 7 7 6 6
16611 270/300 Read block 3076 len 16384
16612 271/300 Write block 3706 len 28672 data: 3 3 4 3 3 3 2
16613 272/300 Read block 4969 len 16384
16614 273/300 Read block 3753 len 40960
16615 274/300 Flush
16616 275/300 Read block 4109 len 24576
16617 276/300 Write block 760 len 40960 data: 5 5 5 5 5 5 6 6 7 7
16618 277/300 Read block 1578 len 8192
16619 278/300 Write block 3270 len 20480 data: 6 5 6 8 7
16620 279/300 Write block 4 len 40960 data: 5 5 4 4 3 3 2 2 2 2
16621 280/300 Flush
16622 281/300 Write block 4395 len 20480 data: 5 6 7 7 7
16623 282/300 Write block 4350 len 40960 data: 4 4 4 4 4 4 4 4 4 5
16624 283/300 Read block 1860 len 8192
16625 284/300 Read block 3725 len 12288
16626 285/300 Write block 1193 len 32768 data: 2 2 2 2 2 2 2 2
16627 286/300 Read block 1880 len 40960
16628 287/300 Write block 1290 len 28672 data: 3 3 5 6 6 5 6
16629 288/300 Write block 2812 len 4096 data: 4
16630 289/300 Write block 1762 len 36864 data: 4 4 5 5 4 4 4 4 3
16631 290/300 Read block 157 len 28672
16632 291/300 Flush
16633 292/300 Read block 2545 len 8192
16634 293/300 Read block 2738 len 32768
16635 294/300 Flush
16636 295/300 Write block 3556 len 20480 data: 2 2 3 4 4
16637 296/300 Read block 3326 len 20480
16638 297/300 Read block 3912 len 20480
16639 298/300 Read block 3925 len 28672
16640 299/300 Write block 960 len 16384 data: 6 6 6 5
16641 300/300 Read block 916 len 32768
16642 301/300 Flush
16643 Main task has work to do, go find it
16644 got dsc Start(1)
16645 start 1
16646 Jan 13 15:55:18.821 INFO request completed, latency_us: 183, response_code: 204, uri: /start/cid/1, method: POST, req_id: acd55d7d-a9d2-498a-ad9e-9781427d62fe, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
16647 Replay: started 1, returned:Ok(())
16648 ----------------------------------------------------------------
16649 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16650 States: Active Offline Active
16651 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16652 5227 Acked Write 1 Done Sent Done false
16653 5228 Acked Read 6 Done Sent Done false
16654 5229 Acked Read 3 Done Sent Done false
16655 5230 Acked Write 10 Done Sent Done false
16656 5231 Acked Write 8 Done Sent Done false
16657 5232 Acked Write 2 Done Sent Done false
16658 5233 Acked Write 4 Done Sent Done false
16659 5234 Acked Read 10 Done Sent Done false
16660 5235 Acked Write 8 Done Sent Done false
16661 5236 Acked Read 1 Done Sent Done false
16662 5237 Acked Read 8 Done Sent Done false
16663 5238 Acked Read 9 Done Sent Done false
16664 5239 Acked Write 7 Done Sent Done false
16665 5240 Acked Read 4 Done Sent Done false
16666 5241 Acked Flush 0 Done Sent Done false
16667 5242 Acked Read 5 Done Sent Done false
16668 5243 Acked Read 10 Done Sent Done false
16669 5244 Acked Read 4 Done Sent Done false
16670 5245 Acked Write 8 Done Sent Done false
16671 5246 Acked Read 7 Done Sent Done false
16672 5247 Acked Write 2 Done Sent Done false
16673 5248 Acked Read 6 Done Sent Done false
16674 5249 Acked Write 6 Done Sent Done false
16675 5250 Acked Read 6 Done Sent Done false
16676 5251 Acked Read 1 Done Sent Done false
16677 5252 Acked Read 7 Done Sent Done false
16678 5253 Acked Write 10 Done Sent Done false
16679 5254 Acked Write 3 Done Sent Done false
16680 5255 Acked Read 2 Done Sent Done false
16681 5256 Acked Read 9 Done Sent Done false
16682 5257 Acked Read 7 Done Sent Done false
16683 5258 Acked Read 9 Done Sent Done false
16684 5259 Acked Read 6 Done Sent Done false
16685 5260 Acked Read 1 Done Sent Done false
16686 5261 Acked Read 2 Done Sent Done false
16687 5262 Acked Read 2 Done Sent Done false
16688 5263 Acked Read 6 Done Sent Done false
16689 5264 Acked Read 2 Done Sent Done false
16690 5265 Acked Write 3 Done Sent Done false
16691 5266 Acked Flush 0 Done Sent Done false
16692 5267 Acked Write 8 Done Sent Done false
16693 5268 Acked Read 10 Done Sent Done false
16694 5269 Acked Read 8 Done Sent Done false
16695 5270 Acked Read 9 Done Sent Done false
16696 5271 Acked Read 7 Done Sent Done false
16697 5272 Acked Write 3 Done Sent Done false
16698 5273 Acked Write 4 Done Sent Done false
16699 5274 Acked Read 9 Done Sent Done false
16700 5275 Acked Read 7 Done Sent Done false
16701 5276 Acked Write 5 Done Sent Done false
16702 5277 Acked Write 6 Done Sent Done false
16703 5278 Acked Flush 0 Done Sent Done false
16704 5279 Acked Read 5 Done Sent Done false
16705 5280 Acked Write 4 Done Sent Done false
16706 5281 Acked Write 8 Done Sent Done false
16707 5282 Acked Read 7 Done Sent Done false
16708 5283 Acked Read 8 Done Sent Done false
16709 5284 Acked Write 1 Done Sent Done false
16710 5285 Acked Write 7 Done Sent Done false
16711 5286 Acked Read 1 Done Sent Done false
16712 5287 Acked Read 4 Done Sent Done false
16713 5288 Acked Read 8 Done Sent Done false
16714 5289 Acked Write 8 Done Sent Done false
16715 5290 Acked Write 10 Done Sent Done false
16716 5291 Acked Read 3 Done Sent Done false
16717 5292 Acked Read 4 Done Sent Done false
16718 5293 Acked Read 8 Done Sent Done false
16719 5294 Acked Flush 0 Done Sent Done false
16720 5295 Acked Read 4 Done Sent Done false
16721 5296 Acked Write 3 Done Sent Done false
16722 5297 Acked Read 10 Done Sent Done false
16723 5298 Acked Read 10 Done Sent Done false
16724 5299 Acked Flush 0 Done Sent Done false
16725 5300 Acked Write 1 Done Sent Done false
16726 5301 Acked Read 4 Done Sent Done false
16727 5302 Acked Write 2 Done Sent Done false
16728 5303 Acked Write 1 Done Sent Done false
16729 5304 Acked Read 5 Done Sent Done false
16730 5305 Acked Read 9 Done Sent Done false
16731 5306 Acked Read 5 Done Sent Done false
16732 5307 Acked Read 4 Done Sent Done false
16733 5308 Acked Write 1 Done Sent Done false
16734 5309 Acked Write 1 Done Sent Done false
16735 5310 Acked Write 8 Done Sent Done false
16736 5311 Acked Read 10 Done Sent Done false
16737 5312 Acked Read 8 Done Sent Done false
16738 5313 Acked Read 4 Done Sent Done false
16739 5314 Acked Read 5 Done Sent Done false
16740 5315 Acked Read 2 Done Sent Done false
16741 5316 Acked Flush 0 Done Sent Done false
16742 5317 Acked Read 10 Done Sent Done false
16743 5318 Acked Read 10 Done Sent Done false
16744 5319 Acked Read 10 Done Sent Done false
16745 5320 Acked Write 7 Done Sent Done false
16746 5321 Acked Flush 0 Done Sent Done false
16747 5322 Acked Read 7 Done Sent Done false
16748 5323 Acked Read 3 Done Sent Done false
16749 5324 Acked Read 9 Done Sent Done false
16750 5325 Acked Read 1 Done Sent Done false
16751 5326 Acked Write 5 Done Sent Done false
16752 5327 Acked Read 8 Done Sent Done false
16753 5328 Acked Read 1 Done Sent Done false
16754 5329 Acked Read 7 Done Sent Done false
16755 5330 Acked Read 10 Done Sent Done false
16756 5331 Acked Write 4 Done Sent Done false
16757 5332 Acked Read 6 Done Sent Done false
16758 5333 Acked Flush 0 Done Sent Done false
16759 5334 Acked Read 5 Done Sent Done false
16760 5335 Acked Read 2 Done Sent Done false
16761 5336 Acked Write 3 Done Sent Done false
16762 5337 Acked Write 9 Done Sent Done false
16763 5338 Acked Write 7 Done Sent Done false
16764 5339 Acked Write 4 Done Sent Done false
16765 5340 Acked Read 5 Done Sent Done false
16766 5341 Acked Write 8 Done Sent Done false
16767 5342 Acked Read 1 Done Sent Done false
16768 5343 Acked Read 1 Done Sent Done false
16769 5344 Acked Write 3 Done Sent Done false
16770 5345 Acked Read 4 Done Sent Done false
16771 5346 Acked Write 4 Done Sent Done false
16772 5347 Acked Read 1 Done Sent Done false
16773 5348 Acked Read 3 Done Sent Done false
16774 5349 Acked Write 6 Done Sent Done false
16775 5350 Acked Read 5 Done Sent Done false
16776 5351 Acked Flush 0 Done Sent Done false
16777 5352 Acked Write 9 Done Sent Done false
16778 5353 Acked Write 3 Done Sent Done false
16779 5354 Acked Read 3 Done Sent Done false
16780 5355 Acked Write 9 Done Sent Done false
16781 5356 Acked Write 2 Done Sent Done false
16782 5357 Acked Write 10 Done Sent Done false
16783 5358 Acked Write 6 Done Sent Done false
16784 5359 Acked Read 1 Done Sent Done false
16785 5360 Acked Write 1 Done Sent Done false
16786 5361 Acked Read 1 Done Sent Done false
16787 5362 Acked Write 1 Done Sent Done false
16788 5363 Acked Write 4 Done Sent Done false
16789 5364 Acked Write 7 Done Sent Done false
16790 5365 Acked Flush 0 Done Sent Done false
16791 5366 Acked Read 9 Done Sent Done false
16792 5367 Acked Write 3 Done Sent Done false
16793 5368 Acked Write 6 Done Sent Done false
16794 5369 Acked Write 1 Done Sent Done false
16795 5370 Acked Read 8 Done Sent Done false
16796 5371 Acked Read 6 Done Sent Done false
16797 5372 Acked Read 1 Done Sent Done false
16798 5373 Acked Write 5 Done Sent Done false
16799 5374 Acked Read 9 Done Sent Done false
16800 5375 Acked Read 10 Done Sent Done false
16801 5376 Acked Read 9 Done Sent Done false
16802 5377 Acked Read 5 Done Sent Done false
16803 5378 Acked Read 8 Done Sent Done false
16804 5379 Acked Write 5 Done Sent Done false
16805 5380 Acked Read 9 Done Sent Done false
16806 5381 Acked Read 5 Done Sent Done false
16807 5382 Acked Read 7 Done Sent Done false
16808 5383 Acked Read 3 Done Sent Done false
16809 5384 Acked Read 1 Done Sent Done false
16810 5385 Acked Flush 0 Done Sent Done false
16811 5386 Acked Read 10 Done Sent Done false
16812 5387 Acked Read 2 Done Sent Done false
16813 5388 Acked Flush 0 Done Sent Done false
16814 5389 Acked Read 5 Done Sent Done false
16815 5390 Acked Write 8 Done Sent Done false
16816 5391 Acked Read 9 Done Sent Done false
16817 5392 Acked Read 3 Done Sent Done false
16818 5393 Acked Write 3 Done Sent Done false
16819 5394 Acked Read 9 Done Sent Done false
16820 5395 Acked Read 5 Done Sent Done false
16821 5396 Acked Flush 0 Done Sent Done false
16822 5397 Acked Write 4 Done Sent Done false
16823 5398 Acked Read 9 Done Sent Done false
16824 5399 Acked Write 9 Done Sent Done false
16825 5400 Acked Write 3 Done Sent Done false
16826 5401 Acked Write 9 Done Sent Done false
16827 5402 Acked Read 5 Done Sent Done false
16828 5403 Acked Write 5 Done Sent Done false
16829 5404 Acked Read 10 Done Sent Done false
16830 5405 Acked Read 8 Done Sent Done false
16831 5406 Acked Read 7 Done Sent Done false
16832 5407 Acked Flush 0 Done Sent Done false
16833 5408 Acked Flush 0 Done Sent Done false
16834 5409 Acked Read 3 Done Sent Done false
16835 5410 Acked Read 10 Done Sent Done false
16836 5411 Acked Read 7 Done Sent Done false
16837 5412 Acked Write 3 Done Sent Done false
16838 5413 Acked Read 7 Done Sent Done false
16839 5414 Acked Read 2 Done Sent Done false
16840 5415 Acked Read 10 Done Sent Done false
16841 5416 Acked Write 10 Done Sent Done false
16842 5417 Acked Write 9 Done Sent Done false
16843 5418 Acked Read 9 Done Sent Done false
16844 5419 Acked Write 8 Done Sent Done false
16845 5420 Acked Flush 0 Done Sent Done false
16846 5421 Acked Write 2 Done Sent Done false
16847 5422 Acked Write 8 Done Sent Done false
16848 5423 Acked Write 2 Done Sent Done false
16849 5424 Acked Read 10 Done Sent Done false
16850 5425 Acked Write 10 Done Sent Done false
16851 5426 Acked Read 4 Done Sent Done false
16852 5427 Acked Read 3 Done Sent Done false
16853 5428 Acked Write 3 Done Sent Done false
16854 5429 Acked Flush 0 Done Sent Done false
16855 5430 Acked Read 7 Done Sent Done false
16856 5431 Acked Read 9 Done Sent Done false
16857 5432 Acked Read 3 Done Sent Done false
16858 5433 Acked Read 5 Done Sent Done false
16859 5434 Acked Write 1 Done Sent Done false
16860 5435 Acked Write 5 Done Sent Done false
16861 5436 Acked Read 2 Done Sent Done false
16862 5437 Acked Read 3 Done Sent Done false
16863 5438 Acked Flush 0 Done Sent Done false
16864 5439 Acked Read 4 Done Sent Done false
16865 5440 Acked Write 9 Done Sent Done false
16866 5441 Acked Read 7 Done Sent Done false
16867 5442 Acked Write 2 Done Sent Done false
16868 5443 Acked Write 4 Done Sent Done false
16869 5444 Acked Read 4 Done Sent Done false
16870 5445 Acked Write 1 Done Sent Done false
16871 5446 Acked Read 6 Done Sent Done false
16872 5447 Acked Flush 0 Done Sent Done false
16873 5448 Acked Write 7 Done Sent Done false
16874 5449 Acked Flush 0 Done Sent Done false
16875 5450 Acked Write 2 Done Sent Done false
16876 5451 Acked Write 7 Done Sent Done false
16877 5452 Acked Write 2 Done Sent Done false
16878 5453 Acked Flush 0 Done Sent Done false
16879 5454 Acked Flush 0 Done Sent Done false
16880 5455 Acked Write 2 Done Sent Done false
16881 5456 Acked Flush 0 Done Sent Done false
16882 5457 Acked Read 6 Done Sent Done false
16883 5458 Acked Read 8 Done Sent Done false
16884 5459 Acked Read 3 Done Sent Done false
16885 5460 Acked Read 2 Done Sent Done false
16886 5461 Acked Write 10 Done Sent Done false
16887 5462 Acked Write 1 Done Sent Done false
16888 5463 Acked Write 10 Done Sent Done false
16889 5464 Acked Write 5 Done Sent Done false
16890 5465 Acked Read 1 Done Sent Done false
16891 5466 Acked Read 7 Done Sent Done false
16892 5467 Acked Read 3 Done Sent Done false
16893 5468 Acked Flush 0 Done Sent Done false
16894 5469 Acked Write 6 Done Sent Done false
16895 5470 Acked Read 1 Done Sent Done false
16896 5471 Acked Read 7 Done Sent Done false
16897 5472 Acked Write 9 Done Sent Done false
16898 5473 Acked Write 10 Done Sent Done false
16899 5474 Acked Read 9 Done Sent Done false
16900 5475 Acked Read 6 Done Sent Done false
16901 5476 Acked Write 9 Done Sent Done false
16902 5477 Acked Read 6 Done Sent Done false
16903 5478 Acked Write 8 Done Sent Done false
16904 5479 Acked Read 1 Done Sent Done false
16905 5480 Acked Write 3 Done Sent Done false
16906 5481 Acked Write 4 Done Sent Done false
16907 5482 Acked Write 6 Done Sent Done false
16908 5483 Acked Read 6 Done Sent Done false
16909 5484 Acked Read 8 Done Sent Done false
16910 5485 Acked Read 3 Done Sent Done false
16911 5486 Acked Write 1 Done Sent Done false
16912 5487 Acked Read 7 Done Sent Done false
16913 5488 Acked Write 10 Done Sent Done false
16914 5489 Acked Read 9 Done Sent Done false
16915 5490 Acked Read 7 Done Sent Done false
16916 5491 Acked Read 9 Done Sent Done false
16917 5492 Acked Write 6 Done Sent Done false
16918 5493 Acked Read 5 Done Sent Done false
16919 5494 Acked Flush 0 Done Sent Done false
16920 5495 Acked Write 5 Done Sent Done false
16921 5496 Acked Read 4 Done Sent Done false
16922 5497 Acked Write 7 Done Sent Done false
16923 5498 Acked Read 4 Done Sent Done false
16924 5499 Acked Read 10 Done Sent Done false
16925 5500 Acked Flush 0 Done Sent Done false
16926 5501 Acked Read 6 Done Sent Done false
16927 5502 Acked Write 10 Done Sent Done false
16928 5503 Acked Read 2 Done Sent Done false
16929 5504 Acked Write 5 Done Sent Done false
16930 5505 Acked Write 10 Done Sent Done false
16931 5506 Acked Flush 0 Done Sent Done false
16932 5507 Acked Write 5 Done Sent Done false
16933 5508 Acked Write 10 Done Sent Done false
16934 5509 Acked Read 2 Done Sent Done false
16935 5510 Acked Read 3 Done Sent Done false
16936 5511 Acked Write 8 Done Sent Done false
16937 5512 Acked Read 10 Done Sent Done false
16938 5513 Acked Write 7 Done Sent Done false
16939 5514 Acked Write 1 Done Sent Done false
16940 5515 Acked Write 9 Done Sent Done false
16941 5516 Acked Read 7 Done Sent Done false
16942 5517 Acked Flush 0 Done Sent Done false
16943 5518 Acked Read 2 Done Sent Done false
16944 5519 Acked Read 8 Done Sent Done false
16945 5520 Acked Flush 0 Done Sent Done false
16946 5521 Acked Write 5 Done Sent Done false
16947 5522 Acked Read 5 Done Sent Done false
16948 5523 Acked Read 5 Done Sent Done false
16949 5524 Acked Read 7 Done Sent Done false
16950 5525 Acked Write 4 Done Sent Done false
16951 5526 Acked Read 8 Done Sent Done false
16952 5527 Acked Flush 0 Done Sent Done false
16953 STATES DS:0 DS:1 DS:2 TOTAL
16954 Sent 0 301 0 301
16955 Done 301 0 301 602
16956 Skipped 0 0 0 0
16957 Error 0 0 0 0
16958 Last Flush: 5527 5226 5527
16959 Downstairs last five completed: 5226 5225 5224 5223 5222
16960 Upstairs last five completed: 5527 5526 5525 5524 5523
16961 CLIENT: Up:0 ds:301 act:2
16962 [8820] got start action
16963 [8820] I am going to restart
16964 Starting downstairs at port 8820
16965 Make output file at "/tmp/dsc/downstairs-8820.txt"
16966 [8820][1] reports Starting
16967 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1217)
16968 [8820][1] reports Running
16969 ----------------------------------------------------------------
16970 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16971 States: Active Offline Active
16972 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16973 5227 Acked Write 1 Done Sent Done false
16974 5228 Acked Read 6 Done Sent Done false
16975 5229 Acked Read 3 Done Sent Done false
16976 5230 Acked Write 10 Done Sent Done false
16977 5231 Acked Write 8 Done Sent Done false
16978 5232 Acked Write 2 Done Sent Done false
16979 5233 Acked Write 4 Done Sent Done false
16980 5234 Acked Read 10 Done Sent Done false
16981 5235 Acked Write 8 Done Sent Done false
16982 5236 Acked Read 1 Done Sent Done false
16983 5237 Acked Read 8 Done Sent Done false
16984 5238 Acked Read 9 Done Sent Done false
16985 5239 Acked Write 7 Done Sent Done false
16986 5240 Acked Read 4 Done Sent Done false
16987 5241 Acked Flush 0 Done Sent Done false
16988 5242 Acked Read 5 Done Sent Done false
16989 5243 Acked Read 10 Done Sent Done false
16990 5244 Acked Read 4 Done Sent Done false
16991 5245 Acked Write 8 Done Sent Done false
16992 5246 Acked Read 7 Done Sent Done false
16993 5247 Acked Write 2 Done Sent Done false
16994 5248 Acked Read 6 Done Sent Done false
16995 5249 Acked Write 6 Done Sent Done false
16996 5250 Acked Read 6 Done Sent Done false
16997 5251 Acked Read 1 Done Sent Done false
16998 5252 Acked Read 7 Done Sent Done false
16999 5253 Acked Write 10 Done Sent Done false
17000 5254 Acked Write 3 Done Sent Done false
17001 5255 Acked Read 2 Done Sent Done false
17002 5256 Acked Read 9 Done Sent Done false
17003 5257 Acked Read 7 Done Sent Done false
17004 5258 Acked Read 9 Done Sent Done false
17005 5259 Acked Read 6 Done Sent Done false
17006 5260 Acked Read 1 Done Sent Done false
17007 5261 Acked Read 2 Done Sent Done false
17008 5262 Acked Read 2 Done Sent Done false
17009 5263 Acked Read 6 Done Sent Done false
17010 5264 Acked Read 2 Done Sent Done false
17011 5265 Acked Write 3 Done Sent Done false
17012 5266 Acked Flush 0 Done Sent Done false
17013 5267 Acked Write 8 Done Sent Done false
17014 5268 Acked Read 10 Done Sent Done false
17015 5269 Acked Read 8 Done Sent Done false
17016 5270 Acked Read 9 Done Sent Done false
17017 5271 Acked Read 7 Done Sent Done false
17018 5272 Acked Write 3 Done Sent Done false
17019 5273 Acked Write 4 Done Sent Done false
17020 5274 Acked Read 9 Done Sent Done false
17021 5275 Acked Read 7 Done Sent Done false
17022 5276 Acked Write 5 Done Sent Done false
17023 5277 Acked Write 6 Done Sent Done false
17024 5278 Acked Flush 0 Done Sent Done false
17025 5279 Acked Read 5 Done Sent Done false
17026 5280 Acked Write 4 Done Sent Done false
17027 5281 Acked Write 8 Done Sent Done false
17028 5282 Acked Read 7 Done Sent Done false
17029 5283 Acked Read 8 Done Sent Done false
17030 5284 Acked Write 1 Done Sent Done false
17031 5285 Acked Write 7 Done Sent Done false
17032 5286 Acked Read 1 Done Sent Done false
17033 5287 Acked Read 4 Done Sent Done false
17034 5288 Acked Read 8 Done Sent Done false
17035 5289 Acked Write 8 Done Sent Done false
17036 5290 Acked Write 10 Done Sent Done false
17037 5291 Acked Read 3 Done Sent Done false
17038 5292 Acked Read 4 Done Sent Done false
17039 5293 Acked Read 8 Done Sent Done false
17040 5294 Acked Flush 0 Done Sent Done false
17041 5295 Acked Read 4 Done Sent Done false
17042 5296 Acked Write 3 Done Sent Done false
17043 5297 Acked Read 10 Done Sent Done false
17044 5298 Acked Read 10 Done Sent Done false
17045 5299 Acked Flush 0 Done Sent Done false
17046 5300 Acked Write 1 Done Sent Done false
17047 5301 Acked Read 4 Done Sent Done false
17048 5302 Acked Write 2 Done Sent Done false
17049 5303 Acked Write 1 Done Sent Done false
17050 5304 Acked Read 5 Done Sent Done false
17051 5305 Acked Read 9 Done Sent Done false
17052 5306 Acked Read 5 Done Sent Done false
17053 5307 Acked Read 4 Done Sent Done false
17054 5308 Acked Write 1 Done Sent Done false
17055 5309 Acked Write 1 Done Sent Done false
17056 5310 Acked Write 8 Done Sent Done false
17057 5311 Acked Read 10 Done Sent Done false
17058 5312 Acked Read 8 Done Sent Done false
17059 5313 Acked Read 4 Done Sent Done false
17060 5314 Acked Read 5 Done Sent Done false
17061 5315 Acked Read 2 Done Sent Done false
17062 5316 Acked Flush 0 Done Sent Done false
17063 5317 Acked Read 10 Done Sent Done false
17064 5318 Acked Read 10 Done Sent Done false
17065 5319 Acked Read 10 Done Sent Done false
17066 5320 Acked Write 7 Done Sent Done false
17067 5321 Acked Flush 0 Done Sent Done false
17068 5322 Acked Read 7 Done Sent Done false
17069 5323 Acked Read 3 Done Sent Done false
17070 5324 Acked Read 9 Done Sent Done false
17071 5325 Acked Read 1 Done Sent Done false
17072 5326 Acked Write 5 Done Sent Done false
17073 5327 Acked Read 8 Done Sent Done false
17074 5328 Acked Read 1 Done Sent Done false
17075 5329 Acked Read 7 Done Sent Done false
17076 5330 Acked Read 10 Done Sent Done false
17077 5331 Acked Write 4 Done Sent Done false
17078 5332 Acked Read 6 Done Sent Done false
17079 5333 Acked Flush 0 Done Sent Done false
17080 5334 Acked Read 5 Done Sent Done false
17081 5335 Acked Read 2 Done Sent Done false
17082 5336 Acked Write 3 Done Sent Done false
17083 5337 Acked Write 9 Done Sent Done false
17084 5338 Acked Write 7 Done Sent Done false
17085 5339 Acked Write 4 Done Sent Done false
17086 5340 Acked Read 5 Done Sent Done false
17087 5341 Acked Write 8 Done Sent Done false
17088 5342 Acked Read 1 Done Sent Done false
17089 5343 Acked Read 1 Done Sent Done false
17090 5344 Acked Write 3 Done Sent Done false
17091 5345 Acked Read 4 Done Sent Done false
17092 5346 Acked Write 4 Done Sent Done false
17093 5347 Acked Read 1 Done Sent Done false
17094 5348 Acked Read 3 Done Sent Done false
17095 5349 Acked Write 6 Done Sent Done false
17096 5350 Acked Read 5 Done Sent Done false
17097 5351 Acked Flush 0 Done Sent Done false
17098 5352 Acked Write 9 Done Sent Done false
17099 5353 Acked Write 3 Done Sent Done false
17100 5354 Acked Read 3 Done Sent Done false
17101 5355 Acked Write 9 Done Sent Done false
17102 5356 Acked Write 2 Done Sent Done false
17103 5357 Acked Write 10 Done Sent Done false
17104 5358 Acked Write 6 Done Sent Done false
17105 5359 Acked Read 1 Done Sent Done false
17106 5360 Acked Write 1 Done Sent Done false
17107 5361 Acked Read 1 Done Sent Done false
17108 5362 Acked Write 1 Done Sent Done false
17109 5363 Acked Write 4 Done Sent Done false
17110 5364 Acked Write 7 Done Sent Done false
17111 5365 Acked Flush 0 Done Sent Done false
17112 5366 Acked Read 9 Done Sent Done false
17113 5367 Acked Write 3 Done Sent Done false
17114 5368 Acked Write 6 Done Sent Done false
17115 5369 Acked Write 1 Done Sent Done false
17116 5370 Acked Read 8 Done Sent Done false
17117 5371 Acked Read 6 Done Sent Done false
17118 5372 Acked Read 1 Done Sent Done false
17119 5373 Acked Write 5 Done Sent Done false
17120 5374 Acked Read 9 Done Sent Done false
17121 5375 Acked Read 10 Done Sent Done false
17122 5376 Acked Read 9 Done Sent Done false
17123 5377 Acked Read 5 Done Sent Done false
17124 5378 Acked Read 8 Done Sent Done false
17125 5379 Acked Write 5 Done Sent Done false
17126 5380 Acked Read 9 Done Sent Done false
17127 5381 Acked Read 5 Done Sent Done false
17128 5382 Acked Read 7 Done Sent Done false
17129 5383 Acked Read 3 Done Sent Done false
17130 5384 Acked Read 1 Done Sent Done false
17131 5385 Acked Flush 0 Done Sent Done false
17132 5386 Acked Read 10 Done Sent Done false
17133 5387 Acked Read 2 Done Sent Done false
17134 5388 Acked Flush 0 Done Sent Done false
17135 5389 Acked Read 5 Done Sent Done false
17136 5390 Acked Write 8 Done Sent Done false
17137 5391 Acked Read 9 Done Sent Done false
17138 5392 Acked Read 3 Done Sent Done false
17139 5393 Acked Write 3 Done Sent Done false
17140 5394 Acked Read 9 Done Sent Done false
17141 5395 Acked Read 5 Done Sent Done false
17142 5396 Acked Flush 0 Done Sent Done false
17143 5397 Acked Write 4 Done Sent Done false
17144 5398 Acked Read 9 Done Sent Done false
17145 5399 Acked Write 9 Done Sent Done false
17146 5400 Acked Write 3 Done Sent Done false
17147 5401 Acked Write 9 Done Sent Done false
17148 5402 Acked Read 5 Done Sent Done false
17149 5403 Acked Write 5 Done Sent Done false
17150 5404 Acked Read 10 Done Sent Done false
17151 5405 Acked Read 8 Done Sent Done false
17152 5406 Acked Read 7 Done Sent Done false
17153 5407 Acked Flush 0 Done Sent Done false
17154 5408 Acked Flush 0 Done Sent Done false
17155 5409 Acked Read 3 Done Sent Done false
17156 5410 Acked Read 10 Done Sent Done false
17157 5411 Acked Read 7 Done Sent Done false
17158 5412 Acked Write 3 Done Sent Done false
17159 5413 Acked Read 7 Done Sent Done false
17160 5414 Acked Read 2 Done Sent Done false
17161 5415 Acked Read 10 Done Sent Done false
17162 5416 Acked Write 10 Done Sent Done false
17163 5417 Acked Write 9 Done Sent Done false
17164 5418 Acked Read 9 Done Sent Done false
17165 5419 Acked Write 8 Done Sent Done false
17166 5420 Acked Flush 0 Done Sent Done false
17167 5421 Acked Write 2 Done Sent Done false
17168 5422 Acked Write 8 Done Sent Done false
17169 5423 Acked Write 2 Done Sent Done false
17170 5424 Acked Read 10 Done Sent Done false
17171 5425 Acked Write 10 Done Sent Done false
17172 5426 Acked Read 4 Done Sent Done false
17173 5427 Acked Read 3 Done Sent Done false
17174 5428 Acked Write 3 Done Sent Done false
17175 5429 Acked Flush 0 Done Sent Done false
17176 5430 Acked Read 7 Done Sent Done false
17177 5431 Acked Read 9 Done Sent Done false
17178 5432 Acked Read 3 Done Sent Done false
17179 5433 Acked Read 5 Done Sent Done false
17180 5434 Acked Write 1 Done Sent Done false
17181 5435 Acked Write 5 Done Sent Done false
17182 5436 Acked Read 2 Done Sent Done false
17183 5437 Acked Read 3 Done Sent Done false
17184 5438 Acked Flush 0 Done Sent Done false
17185 5439 Acked Read 4 Done Sent Done false
17186 5440 Acked Write 9 Done Sent Done false
17187 5441 Acked Read 7 Done Sent Done false
17188 5442 Acked Write 2 Done Sent Done false
17189 5443 Acked Write 4 Done Sent Done false
17190 5444 Acked Read 4 Done Sent Done false
17191 5445 Acked Write 1 Done Sent Done false
17192 5446 Acked Read 6 Done Sent Done false
17193 5447 Acked Flush 0 Done Sent Done false
17194 5448 Acked Write 7 Done Sent Done false
17195 5449 Acked Flush 0 Done Sent Done false
17196 5450 Acked Write 2 Done Sent Done false
17197 5451 Acked Write 7 Done Sent Done false
17198 5452 Acked Write 2 Done Sent Done false
17199 5453 Acked Flush 0 Done Sent Done false
17200 5454 Acked Flush 0 Done Sent Done false
17201 5455 Acked Write 2 Done Sent Done false
17202 5456 Acked Flush 0 Done Sent Done false
17203 5457 Acked Read 6 Done Sent Done false
17204 5458 Acked Read 8 Done Sent Done false
17205 5459 Acked Read 3 Done Sent Done false
17206 5460 Acked Read 2 Done Sent Done false
17207 5461 Acked Write 10 Done Sent Done false
17208 5462 Acked Write 1 Done Sent Done false
17209 5463 Acked Write 10 Done Sent Done false
17210 5464 Acked Write 5 Done Sent Done false
17211 5465 Acked Read 1 Done Sent Done false
17212 5466 Acked Read 7 Done Sent Done false
17213 5467 Acked Read 3 Done Sent Done false
17214 5468 Acked Flush 0 Done Sent Done false
17215 5469 Acked Write 6 Done Sent Done false
17216 5470 Acked Read 1 Done Sent Done false
17217 5471 Acked Read 7 Done Sent Done false
17218 5472 Acked Write 9 Done Sent Done false
17219 5473 Acked Write 10 Done Sent Done false
17220 5474 Acked Read 9 Done Sent Done false
17221 5475 Acked Read 6 Done Sent Done false
17222 5476 Acked Write 9 Done Sent Done false
17223 5477 Acked Read 6 Done Sent Done false
17224 5478 Acked Write 8 Done Sent Done false
17225 5479 Acked Read 1 Done Sent Done false
17226 5480 Acked Write 3 Done Sent Done false
17227 5481 Acked Write 4 Done Sent Done false
17228 5482 Acked Write 6 Done Sent Done false
17229 5483 Acked Read 6 Done Sent Done false
17230 5484 Acked Read 8 Done Sent Done false
17231 5485 Acked Read 3 Done Sent Done false
17232 5486 Acked Write 1 Done Sent Done false
17233 5487 Acked Read 7 Done Sent Done false
17234 5488 Acked Write 10 Done Sent Done false
17235 5489 Acked Read 9 Done Sent Done false
17236 5490 Acked Read 7 Done Sent Done false
17237 5491 Acked Read 9 Done Sent Done false
17238 5492 Acked Write 6 Done Sent Done false
17239 5493 Acked Read 5 Done Sent Done false
17240 5494 Acked Flush 0 Done Sent Done false
17241 5495 Acked Write 5 Done Sent Done false
17242 5496 Acked Read 4 Done Sent Done false
17243 5497 Acked Write 7 Done Sent Done false
17244 5498 Acked Read 4 Done Sent Done false
17245 5499 Acked Read 10 Done Sent Done false
17246 5500 Acked Flush 0 Done Sent Done false
17247 5501 Acked Read 6 Done Sent Done false
17248 5502 Acked Write 10 Done Sent Done false
17249 5503 Acked Read 2 Done Sent Done false
17250 5504 Acked Write 5 Done Sent Done false
17251 5505 Acked Write 10 Done Sent Done false
17252 5506 Acked Flush 0 Done Sent Done false
17253 5507 Acked Write 5 Done Sent Done false
17254 5508 Acked Write 10 Done Sent Done false
17255 5509 Acked Read 2 Done Sent Done false
17256 5510 Acked Read 3 Done Sent Done false
17257 5511 Acked Write 8 Done Sent Done false
17258 5512 Acked Read 10 Done Sent Done false
17259 5513 Acked Write 7 Done Sent Done false
17260 5514 Acked Write 1 Done Sent Done false
17261 5515 Acked Write 9 Done Sent Done false
17262 5516 Acked Read 7 Done Sent Done false
17263 5517 Acked Flush 0 Done Sent Done false
17264 5518 Acked Read 2 Done Sent Done false
17265 5519 Acked Read 8 Done Sent Done false
17266 5520 Acked Flush 0 Done Sent Done false
17267 5521 Acked Write 5 Done Sent Done false
17268 5522 Acked Read 5 Done Sent Done false
17269 5523 Acked Read 5 Done Sent Done false
17270 5524 Acked Read 7 Done Sent Done false
17271 5525 Acked Write 4 Done Sent Done false
17272 5526 Acked Read 8 Done Sent Done false
17273 5527 Acked Flush 0 Done Sent Done false
17274 STATES DS:0 DS:1 DS:2 TOTAL
17275 Sent 0 301 0 301
17276 Done 301 0 301 602
17277 Skipped 0 0 0 0
17278 Error 0 0 0 0
17279 Last Flush: 5527 5226 5527
17280 Downstairs last five completed: 5226 5225 5224 5223 5222
17281 Upstairs last five completed: 5527 5526 5525 5524 5523
17282 CLIENT: Up:0 ds:301 act:2
172832025-01-13T15:55:24.573ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
172842025-01-13T15:55:24.573ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59500) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
172852025-01-13T15:55:24.574ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
172862025-01-13T15:55:24.574ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
172872025-01-13T15:55:24.574ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
172882025-01-13T15:55:24.574ZINFOcrucible: send last flush ID to this DS: 5226 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
172892025-01-13T15:55:24.574ZINFOcrucible: Replied this last flush ID: 5226 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
172902025-01-13T15:55:24.574ZINFOcrucible: [1] client re-new 301 jobs since flush 5226 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
172912025-01-13T15:55:24.575ZINFOcrucible: [1] Replayed 301 jobs since flush: 5226 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
17292 ----------------------------------------------------------------
17293 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
17294 Downstairs last five completed: 5527 5526 5525 5524 5523
17295 Upstairs last five completed: 5527 5526 5525 5524 5523
17296 CLIENT: Up:0 ds:0 act:3
17297 Replay: All jobs finished
17298 Main task has work to do, go find it
17299 got dsc Stop(1)
17300 stop 1
17301 Jan 13 15:55:26.827 INFO request completed, latency_us:[8820] Got stop action so:false kr:false
17302 352, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 9dce1361-13f1-4202-b07e-f7f0def4b2fc, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
173032025-01-13T15:55:26.827ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
173042025-01-13T15:55:26.827ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
173052025-01-13T15:55:26.827ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
17306 Jan 13 15:55:26.828 INFO request completed, latency_us:{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-13T15:55:26.828046068Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
17307 146, response_code: 200, uri: /state/cid/1, method: GET, req_id: dc1dc7f4-3258-47c3-9a6f-35c70229a451, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
17308 [8820] Exited with: ExitStatus(unix_wait_status(9))
17309 [8820][1] reports Exit
17310 Jan 13 15:55:30.831 INFO request completed, latency_us: 177, response_code: 200, uri: /state/cid/1, method: GET, req_id: bf304aed-af27-4835-955d-52395048c370, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
17311 001/300 Write block 2027 len 36864 data: 9 8 7 6 5 4 4 4 3
17312 002/300 Write block 3853 len 20480 data: 2 4 3 3 4
17313 003/300 Read block 4112 len 12288
17314 004/300 Write block 631 len 36864 data: 2 2 3 2 2 2 2 2 3
17315 005/300 Read block 4834 len 24576
17316 006/300 Read block 4197 len 32768
17317 007/300 Read block 873 len 24576
17318 008/300 Write block 2468 len 8192 data: 4 4
17319 009/300 Flush
17320 010/300 Write block 3850 len 40960 data: 3 3 3 3 5 4 4 5 6 6
17321 011/300 Write block 1444 len 20480 data: 3 2 2 2 3
17322 012/300 Read block 362 len 8192
17323 013/300 Read block 859 len 36864
17324 014/300 Flush
17325 015/300 Write block 3675 len 8192 data: 2 2
17326 016/300 Flush
17327 017/300 Read block 222 len 24576
17328 018/300 Read block 4318 len 24576
17329 019/300 Read block 2293 len 24576
17330 020/300 Write block 485 len 12288 data: 5 5 4
17331 021/300 Read block 3672 len 20480
17332 022/300 Flush
17333 023/300 Write block 1621 len 8192 data: 6 5
17334 024/300 Read block 2026 len 12288
17335 025/300 Write block 1518 len 20480 data: 6 5 6 5 6
17336 026/300 Write block 2993 len 20480 data: 3 3 2 3 3
17337 027/300 Write block 2887 len 16384 data: 4 4 4 4
17338 028/300 Read block 4404 len 8192
17339 029/300 Read block 2496 len 4096
17340 030/300 Flush
17341 031/300 Read block 2682 len 40960
17342 032/300 Flush
17343 033/300 Read block 2024 len 28672
17344 034/300 Write block 463 len 28672 data: 3 3 3 3 3 3 4
17345 035/300 Write block 1781 len 32768 data: 3 3 3 3 3 2 2 2
17346 036/300 Flush
17347 037/300 Flush
17348 038/300 Read block 2477 len 36864
17349 039/300 Write block 3899 len 40960 data: 6 5 5 4 3 3 2 2 2 2
17350 040/300 Write block 2336 len 16384 data: 4 4 4 6
17351 041/300 Write block 2777 len 16384 data: 4 3 3 4
17352 042/300 Read block 4454 len 24576
17353 043/300 Read block 1172 len 40960
17354 044/300 Flush
17355 045/300 Write block 4270 len 32768 data: 5 5 5 4 4 5 4 3
17356 046/300 Read block 4071 len 12288
17357 047/300 Read block 3005 len 28672
17358 048/300 Write block 4974 len 12288 data: 4 6 5
17359 049/300 Read block 1502 len 32768
17360 050/300 Read block 1696 len 36864
17361 051/300 Flush
17362 052/300 Read block 914 len 40960
17363 053/300 Read block 817 len 20480
17364 054/300 Read block 1444 len 28672
17365 055/300 Write block 1260 len 32768 data: 2 2 2 5 5 5 4 4
17366 056/300 Write block 1022 len 4096 data: 6
17367 057/300 Write block 998 len 24576 data: 2 2 2 2 2 3
17368 058/300 Write block 4762 len 16384 data: 2 2 2 3
17369 059/300 Write block 2409 len 8192 data: 2 2
17370 060/300 Read block 3362 len 32768
17371 061/300 Read block 2972 len 24576
17372 062/300 Flush
17373 063/300 Read block 1710 len 24576
17374 064/300 Flush
17375 065/300 Read block 4644 len 12288
17376 066/300 Read block 3357 len 24576
17377 067/300 Read block 4785 len 4096
17378 068/300 Flush
17379 069/300 Write block 33 len 24576 data: 3 3 3 3 3 3
17380 070/300 Write block 2198 len 8192 data: 8 7
17381 071/300 Write block 873 len 4096 data: 3
17382 072/300 Write block 3214 len 12288 data: 3 2 3
17383 073/300 Write block 4052 len 28672 data: 5 5 5 5 5 5 6
17384 074/300 Read block 636 len 16384
17385 075/300 Write block 2170 len 36864 data: 3 7 7 7 6 6 6 6 5
17386 076/300 Read block 4513 len 28672
17387 077/300 Flush
17388 078/300 Read block 3362 len 40960
17389 079/300 Write block 2971 len 32768 data: 6 6 5 4 5 5 5 3
17390 080/300 Read block 1567 len 4096
17391 081/300 Read block 4006 len 16384
17392 082/300 Read block 2688 len 32768
17393 083/300 Write block 1920 len 32768 data: 3 3 3 3 2 2 2 2
17394 084/300 Read block 3162 len 12288
17395 085/300 Read block 4172 len 28672
17396 086/300 Read block 4714 len 20480
17397 087/300 Write block 2925 len 24576 data: 2 4 4 4 4 4
17398 088/300 Read block 985 len 32768
17399 089/300 Read block 3181 len 28672
17400 090/300 Read block 1393 len 32768
17401 091/300 Read block 2644 len 8192
17402 092/300 Read block 199 len 32768
17403 093/300 Read block 3175 len 40960
17404 094/300 Write block 1256 len 8192 data: 3 3
17405 095/300 Write block 4185 len 20480 data: 5 4 5 5 5
17406 096/300 Read block 3421 len 32768
17407 097/300 Write block 3027 len 32768 data: 6 6 4 3 3 3 4 4
17408 098/300 Read block 4405 len 20480
17409 099/300 Read block 2307 len 16384
17410 100/300 Read block 1434 len 4096
17411 101/300 Read block 3602 len 24576
17412 102/300 Write block 4910 len 12288 data: 2 2 3
17413 103/300 Read block 2659 len 16384
17414 104/300 Write block 4424 len 24576 data: 2 2 4 4 4 4
17415 105/300 Read block 2953 len 16384
17416 106/300 Read block 2499 len 36864
17417 107/300 Write block 1569 len 36864 data: 4 6 5 4 4 4 4 4 5
17418 108/300 Read block 3625 len 12288
17419 109/300 Read block 4744 len 8192
17420 110/300 Read block 2580 len 28672
17421 111/300 Read block 3665 len 16384
17422 112/300 Write block 4673 len 24576 data: 4 5 4 3 3 2
17423 113/300 Write block 2245 len 28672 data: 3 4 4 3 3 3 3
17424 114/300 Read block 1828 len 16384
17425 115/300 Write block 4302 len 24576 data: 7 8 7 7 5 4
17426 116/300 Read block 3958 len 32768
17427 117/300 Read block 1028 len 32768
17428 118/300 Read block 4888 len 4096
17429 119/300 Write block 842 len 40960 data: 5 4 3 3 3 3 3 2 3 3
17430 120/300 Read block 3313 len 20480
17431 121/300 Flush
17432 122/300 Flush
17433 123/300 Write block 2698 len 32768 data: 5 5 5 6 6 6 5 5
17434 124/300 Read block 3849 len 8192
17435 125/300 Read block 3483 len 16384
17436 126/300 Write block 2171 len 8192 data: 8 8
17437 127/300 Write block 714 len 24576 data: 5 5 5 5 5 5
17438 128/300 Write block 4808 len 4096 data: 4
17439 129/300 Write block 2487 len 24576 data: 5 4 4 4 4 3
17440 130/300 Flush
17441 131/300 Write block 658 len 36864 data: 4 4 5 6 7 7 10 9 9
17442 132/300 Flush
17443 133/300 Read block 328 len 28672
17444 134/300 Write block 4811 len 8192 data: 3 3
17445 135/300 Read block 4228 len 16384
17446 136/300 Write block 1016 len 36864 data: 2 2 3 3 5 5 7 4 5
17447 137/300 Read block 2440 len 28672
17448 138/300 Read block 4022 len 20480
17449 139/300 Read block 4726 len 32768
17450 140/300 Read block 4014 len 40960
17451 141/300 Write block 2038 len 20480 data: 3 4 3 3 3
17452 142/300 Flush
17453 143/300 Write block 3748 len 8192 data: 4 5
17454 144/300 Flush
17455 145/300 Read block 1045 len 32768
17456 146/300 Write block 3824 len 32768 data: 4 4 5 6 5 6 6 6
17457 147/300 Flush
17458 148/300 Read block 2491 len 16384
17459 149/300 Write block 1046 len 16384 data: 4 5 6 7
17460 150/300 Read block 1811 len 8192
17461 151/300 Read block 3624 len 28672
17462 152/300 Read block 2237 len 4096
17463 153/300 Write block 4391 len 8192 data: 2 3
17464 154/300 Flush
17465 155/300 Write block 3480 len 36864 data: 3 4 6 6 7 7 7 8 8
17466 156/300 Read block 3315 len 16384
17467 157/300 Write block 3764 len 40960 data: 6 6 6 7 5 5 5 6 7 5
17468 158/300 Read block 3096 len 16384
17469 159/300 Read block 4803 len 12288
17470 160/300 Read block 543 len 4096
17471 161/300 Read block 4176 len 4096
17472 162/300 Read block 3515 len 40960
17473 163/300 Read block 4639 len 36864
17474 164/300 Read block 2253 len 20480
17475 165/300 Write block 107 len 32768 data: 5 6 5 5 5 5 4 6
17476 166/300 Write block 23 len 12288 data: 5 6 6
17477 167/300 Write block 3664 len 28672 data: 3 3 5 6 5 6 5
17478 168/300 Read block 4117 len 16384
17479 169/300 Read block 146 len 8192
17480 170/300 Write block 4503 len 20480 data: 7 5 2 3 4
17481 171/300 Flush
17482 172/300 Write block 92 len 8192 data: 2 2
17483 173/300 Write block 73 len 32768 data: 2 2 2 2 3 5 6 6
17484 174/300 Read block 703 len 8192
17485 175/300 Write block 4565 len 8192 data: 3 5
17486 176/300 Write block 2652 len 32768 data: 3 3 3 3 3 2 3 3
17487 177/300 Read block 544 len 20480
17488 178/300 Write block 2525 len 28672 data: 5 5 5 5 3 2 3
17489 179/300 Read block 1085 len 40960
17490 180/300 Write block 869 len 12288 data: 6 5 3
17491 181/300 Write block 4768 len 24576 data: 3 3 3 4 4 4
17492 182/300 Read block 2043 len 28672
17493 183/300 Read block 1265 len 40960
17494 184/300 Write block 1252 len 12288 data: 3 3 3
17495 185/300 Flush
17496 186/300 Write block 749 len 36864 data: 4 4 3 4 4 5 6 6 5
17497 187/300 Read block 1647 len 8192
17498 188/300 Read block 623 len 28672
17499 189/300 Write block 4976 len 4096 data: 6
17500 190/300 Read block 2073 len 20480
17501 191/300 Write block 1865 len 12288 data: 3 3 3
17502 192/300 Write block 2335 len 4096 data: 5
17503 193/300 Read block 426 len 40960
17504 194/300 Write block 4266 len 20480 data: 4 4 5 5 6
17505 195/300 Read block 2645 len 12288
17506 196/300 Read block 2412 len 12288
17507 197/300 Write block 4049 len 40960 data: 2 3 4 6 6 6 6 6 6 7
17508 198/300 Read block 4690 len 4096
17509 199/300 Write block 886 len 8192 data: 3 3
17510 200/300 Read block 1052 len 32768
17511 201/300 Read block 1414 len 32768
17512 202/300 Write block 464 len 12288 data: 4 4 4
17513 203/300 Flush
17514 204/300 Flush
17515 205/300 Read block 3822 len 12288
17516 206/300 Read block 2738 len 20480
17517 207/300 Read block 1749 len 36864
17518 208/300 Write block 3260 len 12288 data: 2 3 6
17519 209/300 Write block 4858 len 12288 data: 3 4 5
17520 210/300 Write block 1541 len 8192 data: 3 3
17521 211/300 Write block 4908 len 24576 data: 3 2 3 3 4 3
17522 212/300 Write block 2025 len 8192 data: 7 9
17523 213/300 Read block 3488 len 28672
17524 214/300 Write block 3145 len 4096 data: 5
17525 215/300 Read block 771 len 4096
17526 216/300 Read block 1840 len 16384
17527 217/300 Read block 2597 len 32768
17528 218/300 Write block 3163 len 36864 data: 3 3 4 4 5 5 5 4 4
17529 219/300 Flush
17530 220/300 Flush
17531 221/300 Write block 1856 len 12288 data: 6 6 4
17532 222/300 Read block 4743 len 24576
17533 223/300 Flush
17534 224/300 Read block 181 len 20480
17535 225/300 Read block 684 len 12288
17536 226/300 Read block 383 len 32768
17537 227/300 Read block 907 len 40960
17538 228/300 Write block 838 len 8192 data: 5 4
17539 229/300 Write block 211 len 40960 data: 6 6 6 6 7 6 6 6 4 4
17540 230/300 Read block 1496 len 40960
17541 231/300 Read block 2793 len 12288
17542 232/300 Flush
17543 233/300 Read block 2796 len 32768
17544 234/300 Write block 2272 len 4096 data: 6
17545 235/300 Write block 1062 len 20480 data: 4 4 3 2 2
17546 236/300 Write block 3366 len 36864 data: 2 2 2 2 3 4 4 4 4
17547 237/300 Write block 2377 len 4096 data: 4
17548 238/300 Write block 2865 len 12288 data: 6 5 6
17549 239/300 Flush
17550 240/300 Write block 4762 len 16384 data: 3 3 3 4
17551 241/300 Read block 3488 len 40960
17552 242/300 Read block 4728 len 36864
17553 243/300 Read block 1275 len 28672
17554 244/300 Read block 2885 len 12288
17555 245/300 Flush
17556 246/300 Write block 880 len 4096 data: 7
17557 247/300 Flush
17558 248/300 Read block 4900 len 4096
17559 249/300 Read block 2147 len 24576
17560 250/300 Read block 4674 len 28672
17561 251/300 Write block 1792 len 16384 data: 4 4 4 4
17562 252/300 Flush
17563 253/300 Read block 3742 len 8192
17564 254/300 Flush
17565 255/300 Read block 3834 len 12288
17566 256/300 Read block 2313 len 32768
17567 257/300 Flush
17568 258/300 Write block 3621 len 28672 data: 3 2 2 2 3 3 3
17569 259/300 Flush
17570 260/300 Flush
17571 261/300 Write block 1311 len 20480 data: 2 3 3 3 3
17572 262/300 Read block 405 len 32768
17573 263/300 Write block 2196 len 12288 data: 7 8 9
17574 264/300 Read block 1950 len 4096
17575 265/300 Write block 1434 len 28672 data: 3 4 4 5 3 3 5
17576 266/300 Write block 1771 len 12288 data: 3 4 4
17577 267/300 Read block 3899 len 28672
17578 268/300 Read block 288 len 32768
17579 269/300 Read block 4423 len 40960
17580 270/300 Read block 2095 len 16384
17581 271/300 Flush
17582 272/300 Read block 2376 len 32768
17583 273/300 Read block 4620 len 4096
17584 274/300 Write block 2992 len 20480 data: 3 4 4 3 4
17585 275/300 Write block 2847 len 36864 data: 6 6 4 4 4 4 3 2 2
17586 276/300 Write block 352 len 28672 data: 3 4 4 5 5 4 3
17587 277/300 Read block 2179 len 4096
17588 278/300 Read block 469 len 32768
17589 279/300 Read block 1106 len 16384
17590 280/300 Read block 1942 len 28672
17591 281/300 Write block 4406 len 28672 data: 3 3 3 3 3 3 3
17592 282/300 Write block 2449 len 36864 data: 2 2 2 2 2 2 2 2 3
17593 283/300 Write block 2803 len 36864 data: 4 3 3 3 3 3 3 5 3
17594 284/300 Write block 1502 len 40960 data: 5 4 5 5 5 3 4 4 4 4
17595 285/300 Read block 129 len 28672
17596 286/300 Write block 2961 len 40960 data: 2 2 4 5 5 5 5 5 5 5
17597 287/300 Write block 2647 len 36864 data: 5 5 4 5 4 4 4 4 4
17598 288/300 Read block 4727 len 32768
17599 289/300 Flush
17600 290/300 Read block 3158 len 20480
17601 291/300 Read block 3530 len 24576
17602 292/300 Write block 4208 len 40960 data: 5 5 5 7 5 6 5 7 7 7
17603 293/300 Read block 1284 len 16384
17604 294/300 Read block 4193 len 12288
17605 295/300 Write block 4991 len 8192 data: 3 2
17606 296/300 Write block 4175 len 16384 data: 4 4 4 3
17607 297/300 Read block 4826 len 28672
17608 298/300 Read block 341 len 28672
17609 299/300 Read block 2665 len 24576
17610 300/300 Write block 1247 len 8192 data: 4 4
17611 301/300 Read block 198 len 12288
17612 Main task has work to do, go find it
17613 got dsc Start(1)
17614 start 1
17615 Jan 13 15:55:31.069 INFO request completed, latency_us: 247, response_code: 204, uri: /start/cid/1, method: POST, req_id: 3f3441b5-30f3-47e5-8c6a-0e5e4faf288b, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
17616 Replay: started 1, returned:Ok(())
17617 ----------------------------------------------------------------
17618 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
17619 States: Active Offline Active
17620 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17621 5528 Acked Write 9 Done Sent Done false
17622 5529 Acked Write 5 Done Sent Done false
17623 5530 Acked Read 3 Done Sent Done false
17624 5531 Acked Write 9 Done Sent Done false
17625 5532 Acked Read 6 Done Sent Done false
17626 5533 Acked Read 8 Done Sent Done false
17627 5534 Acked Read 6 Done Sent Done false
17628 5535 Acked Write 2 Done Sent Done false
17629 5536 Acked Flush 0 Done Sent Done false
17630 5537 Acked Write 10 Done Sent Done false
17631 5538 Acked Write 5 Done Sent Done false
17632 5539 Acked Read 2 Done Sent Done false
17633 5540 Acked Read 9 Done Sent Done false
17634 5541 Acked Flush 0 Done Sent Done false
17635 5542 Acked Write 2 Done Sent Done false
17636 5543 Acked Flush 0 Done Sent Done false
17637 5544 Acked Read 6 Done Sent Done false
17638 5545 Acked Read 6 Done Sent Done false
17639 5546 Acked Read 6 Done Sent Done false
17640 5547 Acked Write 3 Done Sent Done false
17641 5548 Acked Read 5 Done Sent Done false
17642 5549 Acked Flush 0 Done Sent Done false
17643 5550 Acked Write 2 Done Sent Done false
17644 5551 Acked Read 3 Done Sent Done false
17645 5552 Acked Write 5 Done Sent Done false
17646 5553 Acked Write 5 Done Sent Done false
17647 5554 Acked Write 4 Done Sent Done false
17648 5555 Acked Read 2 Done Sent Done false
17649 5556 Acked Read 1 Done Sent Done false
17650 5557 Acked Flush 0 Done Sent Done false
17651 5558 Acked Read 10 Done Sent Done false
17652 5559 Acked Flush 0 Done Sent Done false
17653 5560 Acked Read 7 Done Sent Done false
17654 5561 Acked Write 7 Done Sent Done false
17655 5562 Acked Write 8 Done Sent Done false
17656 5563 Acked Flush 0 Done Sent Done false
17657 5564 Acked Flush 0 Done Sent Done false
17658 5565 Acked Read 9 Done Sent Done false
17659 5566 Acked Write 10 Done Sent Done false
17660 5567 Acked Write 4 Done Sent Done false
17661 5568 Acked Write 4 Done Sent Done false
17662 5569 Acked Read 6 Done Sent Done false
17663 5570 Acked Read 10 Done Sent Done false
17664 5571 Acked Flush 0 Done Sent Done false
17665 5572 Acked Write 8 Done Sent Done false
17666 5573 Acked Read 3 Done Sent Done false
17667 5574 Acked Read 7 Done Sent Done false
17668 5575 Acked Write 3 Done Sent Done false
17669 5576 Acked Read 8 Done Sent Done false
17670 5577 Acked Read 9 Done Sent Done false
17671 5578 Acked Flush 0 Done Sent Done false
17672 5579 Acked Read 10 Done Sent Done false
17673 5580 Acked Read 5 Done Sent Done false
17674 5581 Acked Read 7 Done Sent Done false
17675 5582 Acked Write 8 Done Sent Done false
17676 5583 Acked Write 1 Done Sent Done false
17677 5584 Acked Write 6 Done Sent Done false
17678 5585 Acked Write 4 Done Sent Done false
17679 5586 Acked Write 2 Done Sent Done false
17680 5587 Acked Read 8 Done Sent Done false
17681 5588 Acked Read 6 Done Sent Done false
17682 5589 Acked Flush 0 Done Sent Done false
17683 5590 Acked Read 6 Done Sent Done false
17684 5591 Acked Flush 0 Done Sent Done false
17685 5592 Acked Read 3 Done Sent Done false
17686 5593 Acked Read 6 Done Sent Done false
17687 5594 Acked Read 1 Done Sent Done false
17688 5595 Acked Flush 0 Done Sent Done false
17689 5596 Acked Write 6 Done Sent Done false
17690 5597 Acked Write 2 Done Sent Done false
17691 5598 Acked Write 1 Done Sent Done false
17692 5599 Acked Write 3 Done Sent Done false
17693 5600 Acked Write 7 Done Sent Done false
17694 5601 Acked Read 4 Done Sent Done false
17695 5602 Acked Write 9 Done Sent Done false
17696 5603 Acked Read 7 Done Sent Done false
17697 5604 Acked Flush 0 Done Sent Done false
17698 5605 Acked Read 10 Done Sent Done false
17699 5606 Acked Write 8 Done Sent Done false
17700 5607 Acked Read 1 Done Sent Done false
17701 5608 Acked Read 4 Done Sent Done false
17702 5609 Acked Read 8 Done Sent Done false
17703 5610 Acked Write 8 Done Sent Done false
17704 5611 Acked Read 3 Done Sent Done false
17705 5612 Acked Read 7 Done Sent Done false
17706 5613 Acked Read 5 Done Sent Done false
17707 5614 Acked Write 6 Done Sent Done false
17708 5615 Acked Read 8 Done Sent Done false
17709 5616 Acked Read 7 Done Sent Done false
17710 5617 Acked Read 8 Done Sent Done false
17711 5618 Acked Read 2 Done Sent Done false
17712 5619 Acked Read 8 Done Sent Done false
17713 5620 Acked Read 10 Done Sent Done false
17714 5621 Acked Write 2 Done Sent Done false
17715 5622 Acked Write 5 Done Sent Done false
17716 5623 Acked Read 8 Done Sent Done false
17717 5624 Acked Write 8 Done Sent Done false
17718 5625 Acked Read 5 Done Sent Done false
17719 5626 Acked Read 4 Done Sent Done false
17720 5627 Acked Read 1 Done Sent Done false
17721 5628 Acked Read 6 Done Sent Done false
17722 5629 Acked Write 3 Done Sent Done false
17723 5630 Acked Read 4 Done Sent Done false
17724 5631 Acked Write 6 Done Sent Done false
17725 5632 Acked Read 4 Done Sent Done false
17726 5633 Acked Read 9 Done Sent Done false
17727 5634 Acked Write 9 Done Sent Done false
17728 5635 Acked Read 3 Done Sent Done false
17729 5636 Acked Read 2 Done Sent Done false
17730 5637 Acked Read 7 Done Sent Done false
17731 5638 Acked Read 4 Done Sent Done false
17732 5639 Acked Write 6 Done Sent Done false
17733 5640 Acked Write 7 Done Sent Done false
17734 5641 Acked Read 4 Done Sent Done false
17735 5642 Acked Write 6 Done Sent Done false
17736 5643 Acked Read 8 Done Sent Done false
17737 5644 Acked Read 8 Done Sent Done false
17738 5645 Acked Read 1 Done Sent Done false
17739 5646 Acked Write 10 Done Sent Done false
17740 5647 Acked Read 5 Done Sent Done false
17741 5648 Acked Flush 0 Done Sent Done false
17742 5649 Acked Flush 0 Done Sent Done false
17743 5650 Acked Write 8 Done Sent Done false
17744 5651 Acked Read 2 Done Sent Done false
17745 5652 Acked Read 4 Done Sent Done false
17746 5653 Acked Write 2 Done Sent Done false
17747 5654 Acked Write 6 Done Sent Done false
17748 5655 Acked Write 1 Done Sent Done false
17749 5656 Acked Write 6 Done Sent Done false
17750 5657 Acked Flush 0 Done Sent Done false
17751 5658 Acked Write 9 Done Sent Done false
17752 5659 Acked Flush 0 Done Sent Done false
17753 5660 Acked Read 7 Done Sent Done false
17754 5661 Acked Write 2 Done Sent Done false
17755 5662 Acked Read 4 Done Sent Done false
17756 5663 Acked Write 9 Done Sent Done false
17757 5664 Acked Read 7 Done Sent Done false
17758 5665 Acked Read 5 Done Sent Done false
17759 5666 Acked Read 8 Done Sent Done false
17760 5667 Acked Read 10 Done Sent Done false
17761 5668 Acked Write 5 Done Sent Done false
17762 5669 Acked Flush 0 Done Sent Done false
17763 5670 Acked Write 2 Done Sent Done false
17764 5671 Acked Flush 0 Done Sent Done false
17765 5672 Acked Read 8 Done Sent Done false
17766 5673 Acked Write 8 Done Sent Done false
17767 5674 Acked Flush 0 Done Sent Done false
17768 5675 Acked Read 4 Done Sent Done false
17769 5676 Acked Write 4 Done Sent Done false
17770 5677 Acked Read 2 Done Sent Done false
17771 5678 Acked Read 7 Done Sent Done false
17772 5679 Acked Read 1 Done Sent Done false
17773 5680 Acked Write 2 Done Sent Done false
17774 5681 Acked Flush 0 Done Sent Done false
17775 5682 Acked Write 9 Done Sent Done false
17776 5683 Acked Read 4 Done Sent Done false
17777 5684 Acked Write 10 Done Sent Done false
17778 5685 Acked Read 4 Done Sent Done false
17779 5686 Acked Read 3 Done Sent Done false
17780 5687 Acked Read 1 Done Sent Done false
17781 5688 Acked Read 1 Done Sent Done false
17782 5689 Acked Read 10 Done Sent Done false
17783 5690 Acked Read 9 Done Sent Done false
17784 5691 Acked Read 5 Done Sent Done false
17785 5692 Acked Write 8 Done Sent Done false
17786 5693 Acked Write 3 Done Sent Done false
17787 5694 Acked Write 7 Done Sent Done false
17788 5695 Acked Read 4 Done Sent Done false
17789 5696 Acked Read 2 Done Sent Done false
17790 5697 Acked Write 5 Done Sent Done false
17791 5698 Acked Flush 0 Done Sent Done false
17792 5699 Acked Write 2 Done Sent Done false
17793 5700 Acked Write 8 Done Sent Done false
17794 5701 Acked Read 2 Done Sent Done false
17795 5702 Acked Write 2 Done Sent Done false
17796 5703 Acked Write 8 Done Sent Done false
17797 5704 Acked Read 5 Done Sent Done false
17798 5705 Acked Write 7 Done Sent Done false
17799 5706 Acked Read 10 Done Sent Done false
17800 5707 Acked Write 3 Done Sent Done false
17801 5708 Acked Write 6 Done Sent Done false
17802 5709 Acked Read 7 Done Sent Done false
17803 5710 Acked Read 10 Done Sent Done false
17804 5711 Acked Write 3 Done Sent Done false
17805 5712 Acked Flush 0 Done Sent Done false
17806 5713 Acked Write 9 Done Sent Done false
17807 5714 Acked Read 2 Done Sent Done false
17808 5715 Acked Read 7 Done Sent Done false
17809 5716 Acked Write 1 Done Sent Done false
17810 5717 Acked Read 5 Done Sent Done false
17811 5718 Acked Write 3 Done Sent Done false
17812 5719 Acked Write 1 Done Sent Done false
17813 5720 Acked Read 10 Done Sent Done false
17814 5721 Acked Write 5 Done Sent Done false
17815 5722 Acked Read 3 Done Sent Done false
17816 5723 Acked Read 3 Done Sent Done false
17817 5724 Acked Write 10 Done Sent Done false
17818 5725 Acked Read 1 Done Sent Done false
17819 5726 Acked Write 2 Done Sent Done false
17820 5727 Acked Read 8 Done Sent Done false
17821 5728 Acked Read 8 Done Sent Done false
17822 5729 Acked Write 3 Done Sent Done false
17823 5730 Acked Flush 0 Done Sent Done false
17824 5731 Acked Flush 0 Done Sent Done false
17825 5732 Acked Read 3 Done Sent Done false
17826 5733 Acked Read 5 Done Sent Done false
17827 5734 Acked Read 9 Done Sent Done false
17828 5735 Acked Write 3 Done Sent Done false
17829 5736 Acked Write 3 Done Sent Done false
17830 5737 Acked Write 2 Done Sent Done false
17831 5738 Acked Write 6 Done Sent Done false
17832 5739 Acked Write 2 Done Sent Done false
17833 5740 Acked Read 7 Done Sent Done false
17834 5741 Acked Write 1 Done Sent Done false
17835 5742 Acked Read 1 Done Sent Done false
17836 5743 Acked Read 4 Done Sent Done false
17837 5744 Acked Read 8 Done Sent Done false
17838 5745 Acked Write 9 Done Sent Done false
17839 5746 Acked Flush 0 Done Sent Done false
17840 5747 Acked Flush 0 Done Sent Done false
17841 5748 Acked Write 3 Done Sent Done false
17842 5749 Acked Read 6 Done Sent Done false
17843 5750 Acked Flush 0 Done Sent Done false
17844 5751 Acked Read 5 Done Sent Done false
17845 5752 Acked Read 3 Done Sent Done false
17846 5753 Acked Read 8 Done Sent Done false
17847 5754 Acked Read 10 Done Sent Done false
17848 5755 Acked Write 2 Done Sent Done false
17849 5756 Acked Write 10 Done Sent Done false
17850 5757 Acked Read 10 Done Sent Done false
17851 5758 Acked Read 3 Done Sent Done false
17852 5759 Acked Flush 0 Done Sent Done false
17853 5760 Acked Read 8 Done Sent Done false
17854 5761 Acked Write 1 Done Sent Done false
17855 5762 Acked Write 5 Done Sent Done false
17856 5763 Acked Write 9 Done Sent Done false
17857 5764 Acked Write 1 Done Sent Done false
17858 5765 Acked Write 3 Done Sent Done false
17859 5766 Acked Flush 0 Done Sent Done false
17860 5767 Acked Write 4 Done Sent Done false
17861 5768 Acked Read 10 Done Sent Done false
17862 5769 Acked Read 9 Done Sent Done false
17863 5770 Acked Read 7 Done Sent Done false
17864 5771 Acked Read 3 Done Sent Done false
17865 5772 Acked Flush 0 Done Sent Done false
17866 5773 Acked Write 1 Done Sent Done false
17867 5774 Acked Flush 0 Done Sent Done false
17868 5775 Acked Read 1 Done Sent Done false
17869 5776 Acked Read 6 Done Sent Done false
17870 5777 Acked Read 7 Done Sent Done false
17871 5778 Acked Write 4 Done Sent Done false
17872 5779 Acked Flush 0 Done Sent Done false
17873 5780 Acked Read 2 Done Sent Done false
17874 5781 Acked Flush 0 Done Sent Done false
17875 5782 Acked Read 3 Done Sent Done false
17876 5783 Acked Read 8 Done Sent Done false
17877 5784 Acked Flush 0 Done Sent Done false
17878 5785 Acked Write 7 Done Sent Done false
17879 5786 Acked Flush 0 Done Sent Done false
17880 5787 Acked Flush 0 Done Sent Done false
17881 5788 Acked Write 5 Done Sent Done false
17882 5789 Acked Read 8 Done Sent Done false
17883 5790 Acked Write 3 Done Sent Done false
17884 5791 Acked Read 1 Done Sent Done false
17885 5792 Acked Write 7 Done Sent Done false
17886 5793 Acked Write 3 Done Sent Done false
17887 5794 Acked Read 7 Done Sent Done false
17888 5795 Acked Read 8 Done Sent Done false
17889 5796 Acked Read 10 Done Sent Done false
17890 5797 Acked Read 4 Done Sent Done false
17891 5798 Acked Flush 0 Done Sent Done false
17892 5799 Acked Read 8 Done Sent Done false
17893 5800 Acked Read 1 Done Sent Done false
17894 5801 Acked Write 5 Done Sent Done false
17895 5802 Acked Write 9 Done Sent Done false
17896 5803 Acked Write 7 Done Sent Done false
17897 5804 Acked Read 1 Done Sent Done false
17898 5805 Acked Read 8 Done Sent Done false
17899 5806 Acked Read 4 Done Sent Done false
17900 5807 Acked Read 7 Done Sent Done false
17901 5808 Acked Write 7 Done Sent Done false
17902 5809 Acked Write 9 Done Sent Done false
17903 5810 Acked Write 9 Done Sent Done false
17904 5811 Acked Write 10 Done Sent Done false
17905 5812 Acked Read 7 Done Sent Done false
17906 5813 Acked Write 10 Done Sent Done false
17907 5814 Acked Write 9 Done Sent Done false
17908 5815 Acked Read 8 Done Sent Done false
17909 5816 Acked Flush 0 Done Sent Done false
17910 5817 Acked Read 5 Done Sent Done false
17911 5818 Acked Read 6 Done Sent Done false
17912 5819 Acked Write 10 Done Sent Done false
17913 5820 Acked Read 4 Done Sent Done false
17914 5821 Acked Read 3 Done Sent Done false
17915 5822 Acked Write 2 Done Sent Done false
17916 5823 Acked Write 4 Done Sent Done false
17917 5824 Acked Read 7 Done Sent Done false
17918 5825 Acked Read 7 Done Sent Done false
17919 5826 Acked Read 6 Done Sent Done false
17920 5827 Acked Write 2 Done Sent Done false
17921 5828 Acked Read 3 Done Sent Done false
17922 STATES DS:0 DS:1 DS:2 TOTAL
17923 Sent 0 301 0 301
17924 Done 301 0 301 602
17925 Skipped 0 0 0 0
17926 Error 0 0 0 0
17927 Last Flush: 5816 5527 5816
17928 Downstairs last five completed: 5527 5526 5525 5524 5523
17929 Upstairs last five completed: 5828 5827 5826 5825 5824
17930 CLIENT: Up:0 ds:301 act:2
17931 [8820] got start action
17932 [8820] I am going to restart
17933 Starting downstairs at port 8820
17934 Make output file at "/tmp/dsc/downstairs-8820.txt"
17935 [8820][1] reports Starting
17936 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1218)
17937 [8820][1] reports Running
17938 ----------------------------------------------------------------
17939 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17940 States: Active Offline Active
17941 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17942 5528 Acked Write 9 Done Sent Done false
17943 5529 Acked Write 5 Done Sent Done false
17944 5530 Acked Read 3 Done Sent Done false
17945 5531 Acked Write 9 Done Sent Done false
17946 5532 Acked Read 6 Done Sent Done false
17947 5533 Acked Read 8 Done Sent Done false
17948 5534 Acked Read 6 Done Sent Done false
17949 5535 Acked Write 2 Done Sent Done false
17950 5536 Acked Flush 0 Done Sent Done false
17951 5537 Acked Write 10 Done Sent Done false
17952 5538 Acked Write 5 Done Sent Done false
17953 5539 Acked Read 2 Done Sent Done false
17954 5540 Acked Read 9 Done Sent Done false
17955 5541 Acked Flush 0 Done Sent Done false
17956 5542 Acked Write 2 Done Sent Done false
17957 5543 Acked Flush 0 Done Sent Done false
17958 5544 Acked Read 6 Done Sent Done false
17959 5545 Acked Read 6 Done Sent Done false
17960 5546 Acked Read 6 Done Sent Done false
17961 5547 Acked Write 3 Done Sent Done false
17962 5548 Acked Read 5 Done Sent Done false
17963 5549 Acked Flush 0 Done Sent Done false
17964 5550 Acked Write 2 Done Sent Done false
17965 5551 Acked Read 3 Done Sent Done false
17966 5552 Acked Write 5 Done Sent Done false
17967 5553 Acked Write 5 Done Sent Done false
17968 5554 Acked Write 4 Done Sent Done false
17969 5555 Acked Read 2 Done Sent Done false
17970 5556 Acked Read 1 Done Sent Done false
17971 5557 Acked Flush 0 Done Sent Done false
17972 5558 Acked Read 10 Done Sent Done false
17973 5559 Acked Flush 0 Done Sent Done false
17974 5560 Acked Read 7 Done Sent Done false
17975 5561 Acked Write 7 Done Sent Done false
17976 5562 Acked Write 8 Done Sent Done false
17977 5563 Acked Flush 0 Done Sent Done false
17978 5564 Acked Flush 0 Done Sent Done false
17979 5565 Acked Read 9 Done Sent Done false
17980 5566 Acked Write 10 Done Sent Done false
17981 5567 Acked Write 4 Done Sent Done false
17982 5568 Acked Write 4 Done Sent Done false
17983 5569 Acked Read 6 Done Sent Done false
17984 5570 Acked Read 10 Done Sent Done false
17985 5571 Acked Flush 0 Done Sent Done false
17986 5572 Acked Write 8 Done Sent Done false
17987 5573 Acked Read 3 Done Sent Done false
17988 5574 Acked Read 7 Done Sent Done false
17989 5575 Acked Write 3 Done Sent Done false
17990 5576 Acked Read 8 Done Sent Done false
17991 5577 Acked Read 9 Done Sent Done false
17992 5578 Acked Flush 0 Done Sent Done false
17993 5579 Acked Read 10 Done Sent Done false
17994 5580 Acked Read 5 Done Sent Done false
17995 5581 Acked Read 7 Done Sent Done false
17996 5582 Acked Write 8 Done Sent Done false
17997 5583 Acked Write 1 Done Sent Done false
17998 5584 Acked Write 6 Done Sent Done false
17999 5585 Acked Write 4 Done Sent Done false
18000 5586 Acked Write 2 Done Sent Done false
18001 5587 Acked Read 8 Done Sent Done false
18002 5588 Acked Read 6 Done Sent Done false
18003 5589 Acked Flush 0 Done Sent Done false
18004 5590 Acked Read 6 Done Sent Done false
18005 5591 Acked Flush 0 Done Sent Done false
18006 5592 Acked Read 3 Done Sent Done false
18007 5593 Acked Read 6 Done Sent Done false
18008 5594 Acked Read 1 Done Sent Done false
18009 5595 Acked Flush 0 Done Sent Done false
18010 5596 Acked Write 6 Done Sent Done false
18011 5597 Acked Write 2 Done Sent Done false
18012 5598 Acked Write 1 Done Sent Done false
18013 5599 Acked Write 3 Done Sent Done false
18014 5600 Acked Write 7 Done Sent Done false
18015 5601 Acked Read 4 Done Sent Done false
18016 5602 Acked Write 9 Done Sent Done false
18017 5603 Acked Read 7 Done Sent Done false
18018 5604 Acked Flush 0 Done Sent Done false
18019 5605 Acked Read 10 Done Sent Done false
18020 5606 Acked Write 8 Done Sent Done false
18021 5607 Acked Read 1 Done Sent Done false
18022 5608 Acked Read 4 Done Sent Done false
18023 5609 Acked Read 8 Done Sent Done false
18024 5610 Acked Write 8 Done Sent Done false
18025 5611 Acked Read 3 Done Sent Done false
18026 5612 Acked Read 7 Done Sent Done false
18027 5613 Acked Read 5 Done Sent Done false
18028 5614 Acked Write 6 Done Sent Done false
18029 5615 Acked Read 8 Done Sent Done false
18030 5616 Acked Read 7 Done Sent Done false
18031 5617 Acked Read 8 Done Sent Done false
18032 5618 Acked Read 2 Done Sent Done false
18033 5619 Acked Read 8 Done Sent Done false
18034 5620 Acked Read 10 Done Sent Done false
18035 5621 Acked Write 2 Done Sent Done false
18036 5622 Acked Write 5 Done Sent Done false
18037 5623 Acked Read 8 Done Sent Done false
18038 5624 Acked Write 8 Done Sent Done false
18039 5625 Acked Read 5 Done Sent Done false
18040 5626 Acked Read 4 Done Sent Done false
18041 5627 Acked Read 1 Done Sent Done false
18042 5628 Acked Read 6 Done Sent Done false
18043 5629 Acked Write 3 Done Sent Done false
18044 5630 Acked Read 4 Done Sent Done false
18045 5631 Acked Write 6 Done Sent Done false
18046 5632 Acked Read 4 Done Sent Done false
18047 5633 Acked Read 9 Done Sent Done false
18048 5634 Acked Write 9 Done Sent Done false
18049 5635 Acked Read 3 Done Sent Done false
18050 5636 Acked Read 2 Done Sent Done false
18051 5637 Acked Read 7 Done Sent Done false
18052 5638 Acked Read 4 Done Sent Done false
18053 5639 Acked Write 6 Done Sent Done false
18054 5640 Acked Write 7 Done Sent Done false
18055 5641 Acked Read 4 Done Sent Done false
18056 5642 Acked Write 6 Done Sent Done false
18057 5643 Acked Read 8 Done Sent Done false
18058 5644 Acked Read 8 Done Sent Done false
18059 5645 Acked Read 1 Done Sent Done false
18060 5646 Acked Write 10 Done Sent Done false
18061 5647 Acked Read 5 Done Sent Done false
18062 5648 Acked Flush 0 Done Sent Done false
18063 5649 Acked Flush 0 Done Sent Done false
18064 5650 Acked Write 8 Done Sent Done false
18065 5651 Acked Read 2 Done Sent Done false
18066 5652 Acked Read 4 Done Sent Done false
18067 5653 Acked Write 2 Done Sent Done false
18068 5654 Acked Write 6 Done Sent Done false
18069 5655 Acked Write 1 Done Sent Done false
18070 5656 Acked Write 6 Done Sent Done false
18071 5657 Acked Flush 0 Done Sent Done false
18072 5658 Acked Write 9 Done Sent Done false
18073 5659 Acked Flush 0 Done Sent Done false
18074 5660 Acked Read 7 Done Sent Done false
18075 5661 Acked Write 2 Done Sent Done false
18076 5662 Acked Read 4 Done Sent Done false
18077 5663 Acked Write 9 Done Sent Done false
18078 5664 Acked Read 7 Done Sent Done false
18079 5665 Acked Read 5 Done Sent Done false
18080 5666 Acked Read 8 Done Sent Done false
18081 5667 Acked Read 10 Done Sent Done false
18082 5668 Acked Write 5 Done Sent Done false
18083 5669 Acked Flush 0 Done Sent Done false
18084 5670 Acked Write 2 Done Sent Done false
18085 5671 Acked Flush 0 Done Sent Done false
18086 5672 Acked Read 8 Done Sent Done false
18087 5673 Acked Write 8 Done Sent Done false
18088 5674 Acked Flush 0 Done Sent Done false
18089 5675 Acked Read 4 Done Sent Done false
18090 5676 Acked Write 4 Done Sent Done false
18091 5677 Acked Read 2 Done Sent Done false
18092 5678 Acked Read 7 Done Sent Done false
18093 5679 Acked Read 1 Done Sent Done false
18094 5680 Acked Write 2 Done Sent Done false
18095 5681 Acked Flush 0 Done Sent Done false
18096 5682 Acked Write 9 Done Sent Done false
18097 5683 Acked Read 4 Done Sent Done false
18098 5684 Acked Write 10 Done Sent Done false
18099 5685 Acked Read 4 Done Sent Done false
18100 5686 Acked Read 3 Done Sent Done false
18101 5687 Acked Read 1 Done Sent Done false
18102 5688 Acked Read 1 Done Sent Done false
18103 5689 Acked Read 10 Done Sent Done false
18104 5690 Acked Read 9 Done Sent Done false
18105 5691 Acked Read 5 Done Sent Done false
18106 5692 Acked Write 8 Done Sent Done false
18107 5693 Acked Write 3 Done Sent Done false
18108 5694 Acked Write 7 Done Sent Done false
18109 5695 Acked Read 4 Done Sent Done false
18110 5696 Acked Read 2 Done Sent Done false
18111 5697 Acked Write 5 Done Sent Done false
18112 5698 Acked Flush 0 Done Sent Done false
18113 5699 Acked Write 2 Done Sent Done false
18114 5700 Acked Write 8 Done Sent Done false
18115 5701 Acked Read 2 Done Sent Done false
18116 5702 Acked Write 2 Done Sent Done false
18117 5703 Acked Write 8 Done Sent Done false
18118 5704 Acked Read 5 Done Sent Done false
18119 5705 Acked Write 7 Done Sent Done false
18120 5706 Acked Read 10 Done Sent Done false
18121 5707 Acked Write 3 Done Sent Done false
18122 5708 Acked Write 6 Done Sent Done false
18123 5709 Acked Read 7 Done Sent Done false
18124 5710 Acked Read 10 Done Sent Done false
18125 5711 Acked Write 3 Done Sent Done false
18126 5712 Acked Flush 0 Done Sent Done false
18127 5713 Acked Write 9 Done Sent Done false
18128 5714 Acked Read 2 Done Sent Done false
18129 5715 Acked Read 7 Done Sent Done false
18130 5716 Acked Write 1 Done Sent Done false
18131 5717 Acked Read 5 Done Sent Done false
18132 5718 Acked Write 3 Done Sent Done false
18133 5719 Acked Write 1 Done Sent Done false
18134 5720 Acked Read 10 Done Sent Done false
18135 5721 Acked Write 5 Done Sent Done false
18136 5722 Acked Read 3 Done Sent Done false
18137 5723 Acked Read 3 Done Sent Done false
18138 5724 Acked Write 10 Done Sent Done false
18139 5725 Acked Read 1 Done Sent Done false
18140 5726 Acked Write 2 Done Sent Done false
18141 5727 Acked Read 8 Done Sent Done false
18142 5728 Acked Read 8 Done Sent Done false
18143 5729 Acked Write 3 Done Sent Done false
18144 5730 Acked Flush 0 Done Sent Done false
18145 5731 Acked Flush 0 Done Sent Done false
18146 5732 Acked Read 3 Done Sent Done false
18147 5733 Acked Read 5 Done Sent Done false
18148 5734 Acked Read 9 Done Sent Done false
18149 5735 Acked Write 3 Done Sent Done false
18150 5736 Acked Write 3 Done Sent Done false
18151 5737 Acked Write 2 Done Sent Done false
18152 5738 Acked Write 6 Done Sent Done false
18153 5739 Acked Write 2 Done Sent Done false
18154 5740 Acked Read 7 Done Sent Done false
18155 5741 Acked Write 1 Done Sent Done false
18156 5742 Acked Read 1 Done Sent Done false
18157 5743 Acked Read 4 Done Sent Done false
18158 5744 Acked Read 8 Done Sent Done false
18159 5745 Acked Write 9 Done Sent Done false
18160 5746 Acked Flush 0 Done Sent Done false
18161 5747 Acked Flush 0 Done Sent Done false
18162 5748 Acked Write 3 Done Sent Done false
18163 5749 Acked Read 6 Done Sent Done false
18164 5750 Acked Flush 0 Done Sent Done false
18165 5751 Acked Read 5 Done Sent Done false
18166 5752 Acked Read 3 Done Sent Done false
18167 5753 Acked Read 8 Done Sent Done false
18168 5754 Acked Read 10 Done Sent Done false
18169 5755 Acked Write 2 Done Sent Done false
18170 5756 Acked Write 10 Done Sent Done false
18171 5757 Acked Read 10 Done Sent Done false
18172 5758 Acked Read 3 Done Sent Done false
18173 5759 Acked Flush 0 Done Sent Done false
18174 5760 Acked Read 8 Done Sent Done false
18175 5761 Acked Write 1 Done Sent Done false
18176 5762 Acked Write 5 Done Sent Done false
18177 5763 Acked Write 9 Done Sent Done false
18178 5764 Acked Write 1 Done Sent Done false
18179 5765 Acked Write 3 Done Sent Done false
18180 5766 Acked Flush 0 Done Sent Done false
18181 5767 Acked Write 4 Done Sent Done false
18182 5768 Acked Read 10 Done Sent Done false
18183 5769 Acked Read 9 Done Sent Done false
18184 5770 Acked Read 7 Done Sent Done false
18185 5771 Acked Read 3 Done Sent Done false
18186 5772 Acked Flush 0 Done Sent Done false
18187 5773 Acked Write 1 Done Sent Done false
18188 5774 Acked Flush 0 Done Sent Done false
18189 5775 Acked Read 1 Done Sent Done false
18190 5776 Acked Read 6 Done Sent Done false
18191 5777 Acked Read 7 Done Sent Done false
18192 5778 Acked Write 4 Done Sent Done false
18193 5779 Acked Flush 0 Done Sent Done false
18194 5780 Acked Read 2 Done Sent Done false
18195 5781 Acked Flush 0 Done Sent Done false
18196 5782 Acked Read 3 Done Sent Done false
18197 5783 Acked Read 8 Done Sent Done false
18198 5784 Acked Flush 0 Done Sent Done false
18199 5785 Acked Write 7 Done Sent Done false
18200 5786 Acked Flush 0 Done Sent Done false
18201 5787 Acked Flush 0 Done Sent Done false
18202 5788 Acked Write 5 Done Sent Done false
18203 5789 Acked Read 8 Done Sent Done false
18204 5790 Acked Write 3 Done Sent Done false
18205 5791 Acked Read 1 Done Sent Done false
18206 5792 Acked Write 7 Done Sent Done false
18207 5793 Acked Write 3 Done Sent Done false
18208 5794 Acked Read 7 Done Sent Done false
18209 5795 Acked Read 8 Done Sent Done false
18210 5796 Acked Read 10 Done Sent Done false
18211 5797 Acked Read 4 Done Sent Done false
18212 5798 Acked Flush 0 Done Sent Done false
18213 5799 Acked Read 8 Done Sent Done false
18214 5800 Acked Read 1 Done Sent Done false
18215 5801 Acked Write 5 Done Sent Done false
18216 5802 Acked Write 9 Done Sent Done false
18217 5803 Acked Write 7 Done Sent Done false
18218 5804 Acked Read 1 Done Sent Done false
18219 5805 Acked Read 8 Done Sent Done false
18220 5806 Acked Read 4 Done Sent Done false
18221 5807 Acked Read 7 Done Sent Done false
18222 5808 Acked Write 7 Done Sent Done false
18223 5809 Acked Write 9 Done Sent Done false
18224 5810 Acked Write 9 Done Sent Done false
18225 5811 Acked Write 10 Done Sent Done false
18226 5812 Acked Read 7 Done Sent Done false
18227 5813 Acked Write 10 Done Sent Done false
18228 5814 Acked Write 9 Done Sent Done false
18229 5815 Acked Read 8 Done Sent Done false
18230 5816 Acked Flush 0 Done Sent Done false
18231 5817 Acked Read 5 Done Sent Done false
18232 5818 Acked Read 6 Done Sent Done false
18233 5819 Acked Write 10 Done Sent Done false
18234 5820 Acked Read 4 Done Sent Done false
18235 5821 Acked Read 3 Done Sent Done false
18236 5822 Acked Write 2 Done Sent Done false
18237 5823 Acked Write 4 Done Sent Done false
18238 5824 Acked Read 7 Done Sent Done false
18239 5825 Acked Read 7 Done Sent Done false
18240 5826 Acked Read 6 Done Sent Done false
18241 5827 Acked Write 2 Done Sent Done false
18242 5828 Acked Read 3 Done Sent Done false
18243 5829 Acked Flush 0 Done Sent Done false
18244 STATES DS:0 DS:1 DS:2 TOTAL
18245 Sent 0 302 0 302
18246 Done 302 0 302 604
18247 Skipped 0 0 0 0
18248 Error 0 0 0 0
18249 Last Flush: 5829 5527 5829
18250 Downstairs last five completed: 5527 5526 5525 5524 5523
18251 Upstairs last five completed: 5829 5828 5827 5826 5825
18252 CLIENT: Up:0 ds:302 act:2
182532025-01-13T15:55:36.829ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
182542025-01-13T15:55:36.829ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:46735) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
182552025-01-13T15:55:36.831ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
182562025-01-13T15:55:36.831ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
182572025-01-13T15:55:36.831ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
182582025-01-13T15:55:36.831ZINFOcrucible: send last flush ID to this DS: 5527 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
182592025-01-13T15:55:36.832ZINFOcrucible: Replied this last flush ID: 5527 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
182602025-01-13T15:55:36.832ZINFOcrucible: [1] client re-new 302 jobs since flush 5527 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
182612025-01-13T15:55:36.832ZINFOcrucible: [1] Replayed 302 jobs since flush: 5527 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
18262 ----------------------------------------------------------------
18263 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
18264 Downstairs last five completed: 5829 5828 5827 5826 5825
18265 Upstairs last five completed: 5829 5828 5827 5826 5825
18266 CLIENT: Up:0 ds:0 act:3
18267 Replay: All jobs finished
18268 Main task has work to do, go find it
18269 got dsc Stop(2)
18270 stop 2
18271 Jan 13 15:55:39.076 INFO request completed, latency_us: 619, response_code: [8830] Got stop action so:false kr:false
18272 204, uri: /stop/cid/2, method: POST, req_id: 629fbd60-8782-4d99-a465-63f8b559061a, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
182732025-01-13T15:55:39.077ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
18274 Jan 13 15:55:39.077 INFO request completed, latency_us: 117, response_code: 200, uri: /state/cid/2, method: {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:55:39.077277836Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
18275 GET, req_id: e199d456-3719-42eb-9779-1f030c3b24ec, remote_addr:{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:55:39.077334997Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
18276 127.0.0.1:52422{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-13T15:55:39.07737983Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
18277 , local_addr: 127.0.0.1:9998
18278 [8830] Exited with: ExitStatus(unix_wait_status(9))
18279 [8830][2] reports Exit
18280 Jan 13 15:55:43.080 INFO request completed, latency_us: 190, response_code: 200, uri: /state/cid/2, method: GET, req_id: 10c63cfa-7da8-4dec-9884-87c4b075e13a, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
18281 001/300 Flush
18282 002/300 Write block 1100 len 20480 data: 4 4 4 3 3
18283 003/300 Read block 3075 len 8192
18284 004/300 Read block 429 len 32768
18285 005/300 Write block 1745 len 4096 data: 8
18286 006/300 Write block 4305 len 16384 data: 8 6 5 4
18287 007/300 Read block 2786 len 32768
18288 008/300 Read block 2345 len 32768
18289 009/300 Write block 867 len 8192 data: 6 5
18290 010/300 Read block 4350 len 16384
18291 011/300 Flush
18292 012/300 Write block 3050 len 20480 data: 4 4 3 2 2
18293 013/300 Write block 2136 len 16384 data: 3 3 4 3
18294 014/300 Read block 4188 len 8192
18295 015/300 Read block 4304 len 4096
18296 016/300 Read block 1486 len 24576
18297 017/300 Read block 2170 len 20480
18298 018/300 Write block 3952 len 8192 data: 2 3
18299 019/300 Read block 186 len 20480
18300 020/300 Read block 4911 len 32768
18301 021/300 Read block 500 len 28672
18302 022/300 Read block 2631 len 36864
18303 023/300 Write block 2480 len 4096 data: 6
18304 024/300 Write block 1038 len 36864 data: 3 5 4 4 6 6 7 6 5
18305 025/300 Read block 3326 len 40960
18306 026/300 Read block 4043 len 4096
18307 027/300 Write block 2161 len 28672 data: 2 2 2 2 2 3 3
18308 028/300 Write block 9 len 8192 data: 4 3
18309 029/300 Read block 585 len 28672
18310 030/300 Write block 368 len 28672 data: 3 2 2 3 4 6 6
18311 031/300 Write block 1741 len 28672 data: 5 6 7 7 9 6 4
18312 032/300 Read block 2621 len 40960
18313 033/300 Read block 4698 len 40960
18314 034/300 Write block 2692 len 32768 data: 2 2 2 2 2 4 6 6
18315 035/300 Flush
18316 036/300 Write block 2372 len 32768 data: 2 2 2 4 4 5 4 4
18317 037/300 Write block 2439 len 40960 data: 4 4 4 4 5 5 3 3 3 3
18318 038/300 Read block 4493 len 4096
18319 039/300 Write block 1831 len 16384 data: 3 3 3 4
18320 040/300 Read block 83 len 24576
18321 041/300 Write block 3386 len 32768 data: 3 3 3 5 7 5 5 6
18322 042/300 Write block 3183 len 40960 data: 3 3 4 6 6 7 10 10 10 9
18323 043/300 Read block 1252 len 40960
18324 044/300 Read block 41 len 32768
18325 045/300 Write block 1168 len 24576 data: 7 7 5 6 5 6
18326 046/300 Write block 3722 len 8192 data: 6 6
18327 047/300 Write block 168 len 20480 data: 4 4 4 4 5
18328 048/300 Read block 4575 len 8192
18329 049/300 Write block 4284 len 8192 data: 4 4
18330 050/300 Flush
18331 051/300 Read block 433 len 32768
18332 052/300 Read block 2557 len 8192
18333 053/300 Read block 1787 len 16384
18334 054/300 Flush
18335 055/300 Write block 155 len 16384 data: 4 3 3 3
18336 056/300 Flush
18337 057/300 Read block 3641 len 28672
18338 058/300 Write block 274 len 20480 data: 4 3 2 2 2
18339 059/300 Read block 4026 len 16384
18340 060/300 Read block 2837 len 24576
18341 061/300 Write block 173 len 8192 data: 6 6
18342 062/300 Read block 201 len 16384
18343 063/300 Read block 983 len 16384
18344 064/300 Read block 293 len 40960
18345 065/300 Write block 4972 len 16384 data: 3 3 5 7
18346 066/300 Read block 535 len 8192
18347 067/300 Write block 3804 len 28672 data: 6 4 3 3 3 4 5
18348 068/300 Read block 1736 len 8192
18349 069/300 Flush
18350 070/300 Flush
18351 071/300 Read block 2159 len 32768
18352 072/300 Read block 1331 len 20480
18353 073/300 Write block 103 len 32768 data: 5 5 5 6 6 7 6 6
18354 074/300 Read block 587 len 4096
18355 075/300 Flush
18356 076/300 Read block 610 len 40960
18357 077/300 Read block 372 len 24576
18358 078/300 Flush
18359 079/300 Write block 4865 len 12288 data: 6 8 7
18360 080/300 Read block 75 len 28672
18361 081/300 Write block 3400 len 12288 data: 5 6 4
18362 082/300 Flush
18363 083/300 Read block 2476 len 40960
18364 084/300 Flush
18365 085/300 Read block 2116 len 24576
18366 086/300 Read block 897 len 8192
18367 087/300 Write block 4120 len 36864 data: 3 3 2 2 2 2 2 2 2
18368 088/300 Flush
18369 089/300 Write block 4943 len 28672 data: 5 6 4 3 4 3 3
18370 090/300 Write block 2930 len 16384 data: 5 4 4 2
18371 091/300 Read block 3921 len 12288
18372 092/300 Flush
18373 093/300 Read block 2352 len 4096
18374 094/300 Read block 606 len 36864
18375 095/300 Write block 1970 len 32768 data: 4 5 5 4 4 4 3 3
18376 096/300 Read block 3228 len 24576
18377 097/300 Read block 3486 len 28672
18378 098/300 Write block 3714 len 12288 data: 2 3 4
18379 099/300 Write block 4927 len 4096 data: 3
18380 100/300 Write block 3344 len 32768 data: 2 2 2 3 5 6 5 6
18381 101/300 Read block 2962 len 36864
18382 102/300 Read block 4943 len 32768
18383 103/300 Write block 3277 len 36864 data: 6 5 5 4 4 3 5 4 4
18384 104/300 Write block 1741 len 4096 data: 6
18385 105/300 Flush
18386 106/300 Read block 1349 len 28672
18387 107/300 Read block 231 len 28672
18388 108/300 Read block 124 len 4096
18389 109/300 Read block 2332 len 36864
18390 110/300 Read block 1208 len 32768
18391 111/300 Read block 4364 len 24576
18392 112/300 Read block 311 len 16384
18393 113/300 Read block 253 len 8192
18394 114/300 Read block 1888 len 32768
18395 115/300 Read block 1801 len 40960
18396 116/300 Write block 4286 len 20480 data: 4 4 3 2 2
18397 117/300 Write block 908 len 20480 data: 3 3 3 3 2
18398 118/300 Read block 2421 len 36864
18399 119/300 Write block 861 len 16384 data: 4 5 5 5
18400 120/300 Read block 859 len 12288
18401 121/300 Read block 3022 len 20480
18402 122/300 Flush
18403 123/300 Read block 821 len 20480
18404 124/300 Read block 4614 len 12288
18405 125/300 Write block 2095 len 12288 data: 3 4 4
18406 126/300 Read block 1580 len 36864
18407 127/300 Write block 2197 len 12288 data: 9 10 8
18408 128/300 Flush
18409 129/300 Flush
18410 130/300 Write block 364 len 32768 data: 2 2 2 3 4 3 3 4
18411 131/300 Write block 4449 len 28672 data: 8 7 6 5 4 5 5
18412 132/300 Read block 4888 len 16384
18413 133/300 Read block 2126 len 16384
18414 134/300 Read block 3136 len 36864
18415 135/300 Read block 4891 len 40960
18416 136/300 Read block 209 len 32768
18417 137/300 Read block 4473 len 4096
18418 138/300 Read block 3923 len 40960
18419 139/300 Read block 1973 len 20480
18420 140/300 Write block 3886 len 16384 data: 4 5 5 4
18421 141/300 Read block 2941 len 24576
18422 142/300 Write block 3317 len 28672 data: 2 3 3 4 5 4 3
18423 143/300 Flush
18424 144/300 Write block 3377 len 20480 data: 4 3 3 3 3
18425 145/300 Write block 3324 len 24576 data: 4 4 5 5 4 4
18426 146/300 Write block 4471 len 4096 data: 3
18427 147/300 Write block 4913 len 16384 data: 4 3 3 3
18428 148/300 Write block 4912 len 12288 data: 5 5 4
18429 149/300 Read block 3055 len 20480
18430 150/300 Write block 4882 len 40960 data: 3 3 5 4 7 6 7 7 5 5
18431 151/300 Read block 2873 len 12288
18432 152/300 Flush
18433 153/300 Read block 855 len 24576
18434 154/300 Write block 2724 len 16384 data: 3 2 3 3
18435 155/300 Read block 3778 len 4096
18436 156/300 Write block 1418 len 4096 data: 4
18437 157/300 Write block 3137 len 8192 data: 3 4
18438 158/300 Write block 1944 len 20480 data: 3 2 2 2 3
18439 159/300 Write block 1442 len 4096 data: 6
18440 160/300 Write block 750 len 32768 data: 5 4 5 5 6 7 7 6
18441 161/300 Write block 2308 len 12288 data: 3 4 4
18442 162/300 Read block 1607 len 24576
18443 163/300 Flush
18444 164/300 Read block 3487 len 20480
18445 165/300 Write block 992 len 20480 data: 4 3 3 3 2
18446 166/300 Read block 4039 len 12288
18447 167/300 Read block 132 len 4096
18448 168/300 Read block 1361 len 20480
18449 169/300 Write block 4841 len 12288 data: 5 4 4
18450 170/300 Read block 864 len 32768
18451 171/300 Read block 643 len 4096
18452 172/300 Read block 638 len 8192
18453 173/300 Read block 4071 len 8192
18454 174/300 Write block 509 len 40960 data: 2 3 4 5 5 5 5 5 4 4
18455 175/300 Write block 133 len 20480 data: 5 5 5 3 3
18456 176/300 Read block 313 len 40960
18457 177/300 Read block 3631 len 36864
18458 178/300 Write block 246 len 8192 data: 3 2
18459 179/300 Write block 1800 len 12288 data: 5 5 3
18460 180/300 Read block 3690 len 36864
18461 181/300 Read block 3610 len 8192
18462 182/300 Write block 4092 len 8192 data: 2 2
18463 183/300 Read block 3925 len 20480
18464 184/300 Write block 1436 len 24576 data: 5 6 4 4 6 5
18465 185/300 Write block 757 len 36864 data: 7 5 5 6 6 6 6 6 6
18466 186/300 Write block 319 len 32768 data: 3 3 3 3 3 3 3 2
18467 187/300 Read block 1903 len 32768
18468 188/300 Read block 4662 len 8192
18469 189/300 Write block 2679 len 28672 data: 5 4 3 3 4 4 3
18470 190/300 Write block 2987 len 40960 data: 2 2 2 2 2 4 5 5 4 5
18471 191/300 Write block 1415 len 32768 data: 6 4 4 5 5 5 5 5
18472 192/300 Read block 1892 len 8192
18473 193/300 Read block 4715 len 8192
18474 194/300 Write block 685 len 24576 data: 4 4 5 5 4 3
18475 195/300 Write block 4271 len 32768 data: 6 6 5 5 6 5 4 3
18476 196/300 Read block 2719 len 8192
18477 197/300 Read block 4486 len 8192
18478 198/300 Flush
18479 199/300 Read block 1349 len 36864
18480 200/300 Write block 3764 len 36864 data: 7 7 7 8 6 6 6 7 8
18481 201/300 Write block 1196 len 32768 data: 3 3 3 3 3 3 3 5
18482 202/300 Flush
18483 203/300 Write block 3353 len 40960 data: 4 4 5 6 6 4 4 4 4 3
18484 204/300 Read block 1752 len 24576
18485 205/300 Read block 3263 len 12288
18486 206/300 Write block 299 len 8192 data: 2 2
18487 207/300 Read block 3439 len 12288
18488 208/300 Read block 2062 len 4096
18489 209/300 Read block 1110 len 32768
18490 210/300 Read block 4407 len 28672
18491 211/300 Read block 3780 len 20480
18492 212/300 Write block 2659 len 32768 data: 4 3 3 4 4 4 4 4
18493 213/300 Read block 3007 len 40960
18494 214/300 Write block 395 len 28672 data: 3 2 4 7 6 5 4
18495 215/300 Read block 2416 len 12288
18496 216/300 Read block 3001 len 24576
18497 217/300 Write block 3706 len 24576 data: 4 4 5 4 4 4
18498 218/300 Read block 2863 len 16384
18499 219/300 Write block 2950 len 4096 data: 5
18500 220/300 Flush
18501 221/300 Write block 4461 len 20480 data: 4 4 3 3 4
18502 222/300 Read block 2420 len 20480
18503 223/300 Read block 2550 len 4096
18504 224/300 Write block 3807 len 16384 data: 4 4 5 6
18505 225/300 Write block 326 len 4096 data: 3
18506 226/300 Write block 3663 len 24576 data: 3 4 4 6 7 6
18507 227/300 Read block 4854 len 16384
18508 228/300 Read block 2937 len 12288
18509 229/300 Write block 3362 len 16384 data: 4 3 2 2
18510 230/300 Read block 4400 len 8192
18511 231/300 Write block 4382 len 40960 data: 2 2 3 3 3 3 3 3 2 3
18512 232/300 Read block 727 len 4096
18513 233/300 Read block 2463 len 24576
18514 234/300 Write block 3326 len 8192 data: 6 6
18515 235/300 Write block 2436 len 16384 data: 4 4 4 5
18516 236/300 Write block 3078 len 28672 data: 5 5 5 4 3 3 5
18517 237/300 Write block 4038 len 4096 data: 2
18518 238/300 Write block 2518 len 32768 data: 4 4 4 4 4 4 5 6
18519 239/300 Read block 1 len 40960
18520 240/300 Write block 4682 len 20480 data: 4 4 6 5 4
18521 241/300 Read block 2469 len 36864
18522 242/300 Read block 542 len 32768
18523 243/300 Read block 2497 len 32768
18524 244/300 Read block 28 len 12288
18525 245/300 Read block 3034 len 28672
18526 246/300 Read block 1669 len 24576
18527 247/300 Read block 4813 len 36864
18528 248/300 Flush
18529 249/300 Write block 3010 len 8192 data: 7 6
18530 250/300 Read block 4548 len 20480
18531 251/300 Write block 1991 len 8192 data: 2 2
18532 252/300 Write block 2396 len 32768 data: 6 4 4 3 3 3 3 3
18533 253/300 Read block 4795 len 12288
18534 254/300 Flush
18535 255/300 Write block 3099 len 32768 data: 4 4 4 3 2 2 2 2
18536 256/300 Read block 1799 len 8192
18537 257/300 Read block 1992 len 32768
18538 258/300 Read block 4365 len 40960
18539 259/300 Write block 1060 len 20480 data: 5 5 5 5 4
18540 260/300 Read block 4421 len 12288
18541 261/300 Write block 1279 len 24576 data: 3 3 4 3 3 3
18542 262/300 Read block 4975 len 24576
18543 263/300 Read block 1027 len 32768
18544 264/300 Read block 990 len 12288
18545 265/300 Write block 4605 len 24576 data: 5 4 4 2 3 2
18546 266/300 Read block 2670 len 40960
18547 267/300 Flush
18548 268/300 Read block 3853 len 32768
18549 269/300 Read block 1796 len 36864
18550 270/300 Write block 981 len 24576 data: 4 4 3 4 4 4
18551 271/300 Write block 3405 len 20480 data: 3 4 4 4 4
18552 272/300 Read block 1071 len 12288
18553 273/300 Read block 818 len 32768
18554 274/300 Flush
18555 275/300 Read block 862 len 28672
18556 276/300 Write block 4777 len 28672 data: 6 6 5 4 3 5 5
18557 277/300 Read block 1856 len 20480
18558 278/300 Write block 3734 len 12288 data: 3 3 3
18559 279/300 Read block 2486 len 8192
18560 280/300 Write block 4543 len 20480 data: 3 3 3 3 3
18561 281/300 Read block 1478 len 16384
18562 282/300 Write block 1621 len 20480 data: 7 6 5 5 4
18563 283/300 Write block 538 len 24576 data: 5 5 3 3 4 4
18564 284/300 Write block 1966 len 32768 data: 5 5 6 6 5 6 6 5
18565 285/300 Write block 608 len 20480 data: 2 2 2 2 2
18566 286/300 Read block 2391 len 20480
18567 287/300 Write block 3113 len 16384 data: 4 3 3 3
18568 288/300 Write block 3772 len 8192 data: 9 6
18569 289/300 Write block 4548 len 4096 data: 3
18570 290/300 Flush
18571 291/300 Read block 1217 len 16384
18572 292/300 Read block 1566 len 32768
18573 293/300 Write block 2900 len 40960 data: 5 6 6 4 3 3 2 3 5 5
18574 294/300 Read block 3609 len 12288
18575 295/300 Write block 43 len 40960 data: 2 2 2 2 2 3 4 4 4 3
18576 296/300 Write block 4435 len 12288 data: 2 3 3
18577 297/300 Read block 3374 len 12288
18578 298/300 Write block 2819 len 20480 data: 5 6 6 5 6
18579 299/300 Write block 4546 len 36864 data: 4 4 4 3 2 2 2 2 2
18580 300/300 Read block 2135 len 8192
18581 301/300 Write block 2171 len 40960 data: 9 9 8 7 7 7 7 6 5 5
18582 Main task has work to do, go find it
18583 got dsc Start(2)
18584 start 2
18585 Jan 13 15:55:43.318 INFO request completed, latency_us: 180, response_code: 204, uri: /start/cid/2, method: POST, req_id: 53de4e18-56c6-48d9-afb1-28899f3440d5, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
18586 Replay: started 2, returned:Ok(())
18587 ----------------------------------------------------------------
18588 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
18589 States: Active Active Offline
18590 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18591 5830 Acked Flush 0 Done Done Sent false
18592 5831 Acked Write 5 Done Done Sent false
18593 5832 Acked Read 2 Done Done Sent false
18594 5833 Acked Read 8 Done Done Sent false
18595 5834 Acked Write 1 Done Done Sent false
18596 5835 Acked Write 4 Done Done Sent false
18597 5836 Acked Read 8 Done Done Sent false
18598 5837 Acked Read 8 Done Done Sent false
18599 5838 Acked Write 2 Done Done Sent false
18600 5839 Acked Read 4 Done Done Sent false
18601 5840 Acked Flush 0 Done Done Sent false
18602 5841 Acked Write 5 Done Done Sent false
18603 5842 Acked Write 4 Done Done Sent false
18604 5843 Acked Read 2 Done Done Sent false
18605 5844 Acked Read 1 Done Done Sent false
18606 5845 Acked Read 6 Done Done Sent false
18607 5846 Acked Read 5 Done Done Sent false
18608 5847 Acked Write 2 Done Done Sent false
18609 5848 Acked Read 5 Done Done Sent false
18610 5849 Acked Read 8 Done Done Sent false
18611 5850 Acked Read 7 Done Done Sent false
18612 5851 Acked Read 9 Done Done Sent false
18613 5852 Acked Write 1 Done Done Sent false
18614 5853 Acked Write 9 Done Done Sent false
18615 5854 Acked Read 10 Done Done Sent false
18616 5855 Acked Read 1 Done Done Sent false
18617 5856 Acked Write 7 Done Done Sent false
18618 5857 Acked Write 2 Done Done Sent false
18619 5858 Acked Read 7 Done Done Sent false
18620 5859 Acked Write 7 Done Done Sent false
18621 5860 Acked Write 7 Done Done Sent false
18622 5861 Acked Read 10 Done Done Sent false
18623 5862 Acked Read 10 Done Done Sent false
18624 5863 Acked Write 8 Done Done Sent false
18625 5864 Acked Flush 0 Done Done Sent false
18626 5865 Acked Write 8 Done Done Sent false
18627 5866 Acked Write 10 Done Done Sent false
18628 5867 Acked Read 1 Done Done Sent false
18629 5868 Acked Write 4 Done Done Sent false
18630 5869 Acked Read 6 Done Done Sent false
18631 5870 Acked Write 8 Done Done Sent false
18632 5871 Acked Write 10 Done Done Sent false
18633 5872 Acked Read 10 Done Done Sent false
18634 5873 Acked Read 8 Done Done Sent false
18635 5874 Acked Write 6 Done Done Sent false
18636 5875 Acked Write 2 Done Done Sent false
18637 5876 Acked Write 5 Done Done Sent false
18638 5877 Acked Read 2 Done Done Sent false
18639 5878 Acked Write 2 Done Done Sent false
18640 5879 Acked Flush 0 Done Done Sent false
18641 5880 Acked Read 8 Done Done Sent false
18642 5881 Acked Read 2 Done Done Sent false
18643 5882 Acked Read 4 Done Done Sent false
18644 5883 Acked Flush 0 Done Done Sent false
18645 5884 Acked Write 4 Done Done Sent false
18646 5885 Acked Flush 0 Done Done Sent false
18647 5886 Acked Read 7 Done Done Sent false
18648 5887 Acked Write 5 Done Done Sent false
18649 5888 Acked Read 4 Done Done Sent false
18650 5889 Acked Read 6 Done Done Sent false
18651 5890 Acked Write 2 Done Done Sent false
18652 5891 Acked Read 4 Done Done Sent false
18653 5892 Acked Read 4 Done Done Sent false
18654 5893 Acked Read 10 Done Done Sent false
18655 5894 Acked Write 4 Done Done Sent false
18656 5895 Acked Read 2 Done Done Sent false
18657 5896 Acked Write 7 Done Done Sent false
18658 5897 Acked Read 2 Done Done Sent false
18659 5898 Acked Flush 0 Done Done Sent false
18660 5899 Acked Flush 0 Done Done Sent false
18661 5900 Acked Read 8 Done Done Sent false
18662 5901 Acked Read 5 Done Done Sent false
18663 5902 Acked Write 8 Done Done Sent false
18664 5903 Acked Read 1 Done Done Sent false
18665 5904 Acked Flush 0 Done Done Sent false
18666 5905 Acked Read 10 Done Done Sent false
18667 5906 Acked Read 6 Done Done Sent false
18668 5907 Acked Flush 0 Done Done Sent false
18669 5908 Acked Write 3 Done Done Sent false
18670 5909 Acked Read 7 Done Done Sent false
18671 5910 Acked Write 3 Done Done Sent false
18672 5911 Acked Flush 0 Done Done Sent false
18673 5912 Acked Read 10 Done Done Sent false
18674 5913 Acked Flush 0 Done Done Sent false
18675 5914 Acked Read 6 Done Done Sent false
18676 5915 Acked Read 2 Done Done Sent false
18677 5916 Acked Write 9 Done Done Sent false
18678 5917 Acked Flush 0 Done Done Sent false
18679 5918 Acked Write 7 Done Done Sent false
18680 5919 Acked Write 4 Done Done Sent false
18681 5920 Acked Read 3 Done Done Sent false
18682 5921 Acked Flush 0 Done Done Sent false
18683 5922 Acked Read 1 Done Done Sent false
18684 5923 Acked Read 9 Done Done Sent false
18685 5924 Acked Write 8 Done Done Sent false
18686 5925 Acked Read 6 Done Done Sent false
18687 5926 Acked Read 7 Done Done Sent false
18688 5927 Acked Write 3 Done Done Sent false
18689 5928 Acked Write 1 Done Done Sent false
18690 5929 Acked Write 8 Done Done Sent false
18691 5930 Acked Read 9 Done Done Sent false
18692 5931 Acked Read 8 Done Done Sent false
18693 5932 Acked Write 9 Done Done Sent false
18694 5933 Acked Write 1 Done Done Sent false
18695 5934 Acked Flush 0 Done Done Sent false
18696 5935 Acked Read 7 Done Done Sent false
18697 5936 Acked Read 7 Done Done Sent false
18698 5937 Acked Read 1 Done Done Sent false
18699 5938 Acked Read 9 Done Done Sent false
18700 5939 Acked Read 8 Done Done Sent false
18701 5940 Acked Read 6 Done Done Sent false
18702 5941 Acked Read 4 Done Done Sent false
18703 5942 Acked Read 2 Done Done Sent false
18704 5943 Acked Read 8 Done Done Sent false
18705 5944 Acked Read 10 Done Done Sent false
18706 5945 Acked Write 5 Done Done Sent false
18707 5946 Acked Write 5 Done Done Sent false
18708 5947 Acked Read 9 Done Done Sent false
18709 5948 Acked Write 4 Done Done Sent false
18710 5949 Acked Read 3 Done Done Sent false
18711 5950 Acked Read 5 Done Done Sent false
18712 5951 Acked Flush 0 Done Done Sent false
18713 5952 Acked Read 5 Done Done Sent false
18714 5953 Acked Read 3 Done Done Sent false
18715 5954 Acked Write 3 Done Done Sent false
18716 5955 Acked Read 9 Done Done Sent false
18717 5956 Acked Write 3 Done Done Sent false
18718 5957 Acked Flush 0 Done Done Sent false
18719 5958 Acked Flush 0 Done Done Sent false
18720 5959 Acked Write 8 Done Done Sent false
18721 5960 Acked Write 7 Done Done Sent false
18722 5961 Acked Read 4 Done Done Sent false
18723 5962 Acked Read 4 Done Done Sent false
18724 5963 Acked Read 9 Done Done Sent false
18725 5964 Acked Read 10 Done Done Sent false
18726 5965 Acked Read 8 Done Done Sent false
18727 5966 Acked Read 1 Done Done Sent false
18728 5967 Acked Read 10 Done Done Sent false
18729 5968 Acked Read 5 Done Done Sent false
18730 5969 Acked Write 4 Done Done Sent false
18731 5970 Acked Read 6 Done Done Sent false
18732 5971 Acked Write 7 Done Done Sent false
18733 5972 Acked Flush 0 Done Done Sent false
18734 5973 Acked Write 5 Done Done Sent false
18735 5974 Acked Write 6 Done Done Sent false
18736 5975 Acked Write 1 Done Done Sent false
18737 5976 Acked Write 4 Done Done Sent false
18738 5977 Acked Write 3 Done Done Sent false
18739 5978 Acked Read 5 Done Done Sent false
18740 5979 Acked Write 10 Done Done Sent false
18741 5980 Acked Read 3 Done Done Sent false
18742 5981 Acked Flush 0 Done Done Sent false
18743 5982 Acked Read 6 Done Done Sent false
18744 5983 Acked Write 4 Done Done Sent false
18745 5984 Acked Read 1 Done Done Sent false
18746 5985 Acked Write 1 Done Done Sent false
18747 5986 Acked Write 2 Done Done Sent false
18748 5987 Acked Write 5 Done Done Sent false
18749 5988 Acked Write 1 Done Done Sent false
18750 5989 Acked Write 8 Done Done Sent false
18751 5990 Acked Write 3 Done Done Sent false
18752 5991 Acked Read 6 Done Done Sent false
18753 5992 Acked Flush 0 Done Done Sent false
18754 5993 Acked Read 5 Done Done Sent false
18755 5994 Acked Write 5 Done Done Sent false
18756 5995 Acked Read 3 Done Done Sent false
18757 5996 Acked Read 1 Done Done Sent false
18758 5997 Acked Read 5 Done Done Sent false
18759 5998 Acked Write 3 Done Done Sent false
18760 5999 Acked Read 8 Done Done Sent false
18761 6000 Acked Read 1 Done Done Sent false
18762 6001 Acked Read 2 Done Done Sent false
18763 6002 Acked Read 2 Done Done Sent false
18764 6003 Acked Write 10 Done Done Sent false
18765 6004 Acked Write 5 Done Done Sent false
18766 6005 Acked Read 10 Done Done Sent false
18767 6006 Acked Read 9 Done Done Sent false
18768 6007 Acked Write 2 Done Done Sent false
18769 6008 Acked Write 3 Done Done Sent false
18770 6009 Acked Read 9 Done Done Sent false
18771 6010 Acked Read 2 Done Done Sent false
18772 6011 Acked Write 2 Done Done Sent false
18773 6012 Acked Read 5 Done Done Sent false
18774 6013 Acked Write 6 Done Done Sent false
18775 6014 Acked Write 9 Done Done Sent false
18776 6015 Acked Write 8 Done Done Sent false
18777 6016 Acked Read 8 Done Done Sent false
18778 6017 Acked Read 2 Done Done Sent false
18779 6018 Acked Write 7 Done Done Sent false
18780 6019 Acked Write 10 Done Done Sent false
18781 6020 Acked Write 8 Done Done Sent false
18782 6021 Acked Read 2 Done Done Sent false
18783 6022 Acked Read 2 Done Done Sent false
18784 6023 Acked Write 6 Done Done Sent false
18785 6024 Acked Write 8 Done Done Sent false
18786 6025 Acked Read 2 Done Done Sent false
18787 6026 Acked Read 2 Done Done Sent false
18788 6027 Acked Flush 0 Done Done Sent false
18789 6028 Acked Read 9 Done Done Sent false
18790 6029 Acked Write 9 Done Done Sent false
18791 6030 Acked Write 8 Done Done Sent false
18792 6031 Acked Flush 0 Done Done Sent false
18793 6032 Acked Write 10 Done Done Sent false
18794 6033 Acked Read 6 Done Done Sent false
18795 6034 Acked Read 3 Done Done Sent false
18796 6035 Acked Write 2 Done Done Sent false
18797 6036 Acked Read 3 Done Done Sent false
18798 6037 Acked Read 1 Done Done Sent false
18799 6038 Acked Read 8 Done Done Sent false
18800 6039 Acked Read 7 Done Done Sent false
18801 6040 Acked Read 5 Done Done Sent false
18802 6041 Acked Write 8 Done Done Sent false
18803 6042 Acked Read 10 Done Done Sent false
18804 6043 Acked Write 7 Done Done Sent false
18805 6044 Acked Read 3 Done Done Sent false
18806 6045 Acked Read 6 Done Done Sent false
18807 6046 Acked Write 6 Done Done Sent false
18808 6047 Acked Read 4 Done Done Sent false
18809 6048 Acked Write 1 Done Done Sent false
18810 6049 Acked Flush 0 Done Done Sent false
18811 6050 Acked Write 5 Done Done Sent false
18812 6051 Acked Read 5 Done Done Sent false
18813 6052 Acked Read 1 Done Done Sent false
18814 6053 Acked Write 4 Done Done Sent false
18815 6054 Acked Write 1 Done Done Sent false
18816 6055 Acked Write 6 Done Done Sent false
18817 6056 Acked Read 4 Done Done Sent false
18818 6057 Acked Read 3 Done Done Sent false
18819 6058 Acked Write 4 Done Done Sent false
18820 6059 Acked Read 2 Done Done Sent false
18821 6060 Acked Write 10 Done Done Sent false
18822 6061 Acked Read 1 Done Done Sent false
18823 6062 Acked Read 6 Done Done Sent false
18824 6063 Acked Write 2 Done Done Sent false
18825 6064 Acked Write 4 Done Done Sent false
18826 6065 Acked Write 7 Done Done Sent false
18827 6066 Acked Write 1 Done Done Sent false
18828 6067 Acked Write 8 Done Done Sent false
18829 6068 Acked Read 10 Done Done Sent false
18830 6069 Acked Write 5 Done Done Sent false
18831 6070 Acked Read 9 Done Done Sent false
18832 6071 Acked Read 8 Done Done Sent false
18833 6072 Acked Read 8 Done Done Sent false
18834 6073 Acked Read 3 Done Done Sent false
18835 6074 Acked Read 7 Done Done Sent false
18836 6075 Acked Read 6 Done Done Sent false
18837 6076 Acked Read 9 Done Done Sent false
18838 6077 Acked Flush 0 Done Done Sent false
18839 6078 Acked Write 2 Done Done Sent false
18840 6079 Acked Read 5 Done Done Sent false
18841 6080 Acked Write 2 Done Done Sent false
18842 6081 Acked Write 8 Done Done Sent false
18843 6082 Acked Read 3 Done Done Sent false
18844 6083 Acked Flush 0 Done Done Sent false
18845 6084 Acked Write 8 Done Done Sent false
18846 6085 Acked Read 2 Done Done Sent false
18847 6086 Acked Read 8 Done Done Sent false
18848 6087 Acked Read 10 Done Done Sent false
18849 6088 Acked Write 5 Done Done Sent false
18850 6089 Acked Read 3 Done Done Sent false
18851 6090 Acked Write 6 Done Done Sent false
18852 6091 Acked Read 6 Done Done Sent false
18853 6092 Acked Read 8 Done Done Sent false
18854 6093 Acked Read 3 Done Done Sent false
18855 6094 Acked Write 6 Done Done Sent false
18856 6095 Acked Read 10 Done Done Sent false
18857 6096 Acked Flush 0 Done Done Sent false
18858 6097 Acked Read 8 Done Done Sent false
18859 6098 Acked Read 9 Done Done Sent false
18860 6099 Acked Write 6 Done Done Sent false
18861 6100 Acked Write 5 Done Done Sent false
18862 6101 Acked Read 3 Done Done Sent false
18863 6102 Acked Read 8 Done Done Sent false
18864 6103 Acked Flush 0 Done Done Sent false
18865 6104 Acked Read 7 Done Done Sent false
18866 6105 Acked Write 7 Done Done Sent false
18867 6106 Acked Read 5 Done Done Sent false
18868 6107 Acked Write 3 Done Done Sent false
18869 6108 Acked Read 2 Done Done Sent false
18870 6109 Acked Write 5 Done Done Sent false
18871 6110 Acked Read 4 Done Done Sent false
18872 6111 Acked Write 5 Done Done Sent false
18873 6112 Acked Write 6 Done Done Sent false
18874 6113 Acked Write 8 Done Done Sent false
18875 6114 Acked Write 5 Done Done Sent false
18876 6115 Acked Read 5 Done Done Sent false
18877 6116 Acked Write 4 Done Done Sent false
18878 6117 Acked Write 2 Done Done Sent false
18879 6118 Acked Write 1 Done Done Sent false
18880 6119 Acked Flush 0 Done Done Sent false
18881 6120 Acked Read 4 Done Done Sent false
18882 6121 Acked Read 8 Done Done Sent false
18883 6122 Acked Write 10 Done Done Sent false
18884 6123 Acked Read 3 Done Done Sent false
18885 6124 Acked Write 10 Done Done Sent false
18886 6125 Acked Write 3 Done Done Sent false
18887 6126 Acked Read 3 Done Done Sent false
18888 6127 Acked Write 5 Done Done Sent false
18889 6128 Acked Write 9 Done Done Sent false
18890 6129 Acked Read 2 Done Done Sent false
18891 6130 Acked Write 10 Sent Sent Sent false
18892 STATES DS:0 DS:1 DS:2 TOTAL
18893 Sent 1 1 301 303
18894 Done 300 300 0 600
18895 Skipped 0 0 0 0
18896 Error 0 0 0 0
18897 Last Flush: 6119 6119 5829
18898 Downstairs last five completed: 5829 5828 5827 5826 5825
18899 Upstairs last five completed: 6130 6129 6128 6127 6126
18900 CLIENT: Up:0 ds:301 act:2
18901 [8830] got start action
18902 [8830] I am going to restart
18903 Starting downstairs at port 8830
18904 Make output file at "/tmp/dsc/downstairs-8830.txt"
18905 [8830][2] reports Starting
18906 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1219)
18907 [8830][2] reports Running
18908 ----------------------------------------------------------------
18909 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18910 States: Active Active Offline
18911 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18912 5830 Acked Flush 0 Done Done Sent false
18913 5831 Acked Write 5 Done Done Sent false
18914 5832 Acked Read 2 Done Done Sent false
18915 5833 Acked Read 8 Done Done Sent false
18916 5834 Acked Write 1 Done Done Sent false
18917 5835 Acked Write 4 Done Done Sent false
18918 5836 Acked Read 8 Done Done Sent false
18919 5837 Acked Read 8 Done Done Sent false
18920 5838 Acked Write 2 Done Done Sent false
18921 5839 Acked Read 4 Done Done Sent false
18922 5840 Acked Flush 0 Done Done Sent false
18923 5841 Acked Write 5 Done Done Sent false
18924 5842 Acked Write 4 Done Done Sent false
18925 5843 Acked Read 2 Done Done Sent false
18926 5844 Acked Read 1 Done Done Sent false
18927 5845 Acked Read 6 Done Done Sent false
18928 5846 Acked Read 5 Done Done Sent false
18929 5847 Acked Write 2 Done Done Sent false
18930 5848 Acked Read 5 Done Done Sent false
18931 5849 Acked Read 8 Done Done Sent false
18932 5850 Acked Read 7 Done Done Sent false
18933 5851 Acked Read 9 Done Done Sent false
18934 5852 Acked Write 1 Done Done Sent false
18935 5853 Acked Write 9 Done Done Sent false
18936 5854 Acked Read 10 Done Done Sent false
18937 5855 Acked Read 1 Done Done Sent false
18938 5856 Acked Write 7 Done Done Sent false
18939 5857 Acked Write 2 Done Done Sent false
18940 5858 Acked Read 7 Done Done Sent false
18941 5859 Acked Write 7 Done Done Sent false
18942 5860 Acked Write 7 Done Done Sent false
18943 5861 Acked Read 10 Done Done Sent false
18944 5862 Acked Read 10 Done Done Sent false
18945 5863 Acked Write 8 Done Done Sent false
18946 5864 Acked Flush 0 Done Done Sent false
18947 5865 Acked Write 8 Done Done Sent false
18948 5866 Acked Write 10 Done Done Sent false
18949 5867 Acked Read 1 Done Done Sent false
18950 5868 Acked Write 4 Done Done Sent false
18951 5869 Acked Read 6 Done Done Sent false
18952 5870 Acked Write 8 Done Done Sent false
18953 5871 Acked Write 10 Done Done Sent false
18954 5872 Acked Read 10 Done Done Sent false
18955 5873 Acked Read 8 Done Done Sent false
18956 5874 Acked Write 6 Done Done Sent false
18957 5875 Acked Write 2 Done Done Sent false
18958 5876 Acked Write 5 Done Done Sent false
18959 5877 Acked Read 2 Done Done Sent false
18960 5878 Acked Write 2 Done Done Sent false
18961 5879 Acked Flush 0 Done Done Sent false
18962 5880 Acked Read 8 Done Done Sent false
18963 5881 Acked Read 2 Done Done Sent false
18964 5882 Acked Read 4 Done Done Sent false
18965 5883 Acked Flush 0 Done Done Sent false
18966 5884 Acked Write 4 Done Done Sent false
18967 5885 Acked Flush 0 Done Done Sent false
18968 5886 Acked Read 7 Done Done Sent false
18969 5887 Acked Write 5 Done Done Sent false
18970 5888 Acked Read 4 Done Done Sent false
18971 5889 Acked Read 6 Done Done Sent false
18972 5890 Acked Write 2 Done Done Sent false
18973 5891 Acked Read 4 Done Done Sent false
18974 5892 Acked Read 4 Done Done Sent false
18975 5893 Acked Read 10 Done Done Sent false
18976 5894 Acked Write 4 Done Done Sent false
18977 5895 Acked Read 2 Done Done Sent false
18978 5896 Acked Write 7 Done Done Sent false
18979 5897 Acked Read 2 Done Done Sent false
18980 5898 Acked Flush 0 Done Done Sent false
18981 5899 Acked Flush 0 Done Done Sent false
18982 5900 Acked Read 8 Done Done Sent false
18983 5901 Acked Read 5 Done Done Sent false
18984 5902 Acked Write 8 Done Done Sent false
18985 5903 Acked Read 1 Done Done Sent false
18986 5904 Acked Flush 0 Done Done Sent false
18987 5905 Acked Read 10 Done Done Sent false
18988 5906 Acked Read 6 Done Done Sent false
18989 5907 Acked Flush 0 Done Done Sent false
18990 5908 Acked Write 3 Done Done Sent false
18991 5909 Acked Read 7 Done Done Sent false
18992 5910 Acked Write 3 Done Done Sent false
18993 5911 Acked Flush 0 Done Done Sent false
18994 5912 Acked Read 10 Done Done Sent false
18995 5913 Acked Flush 0 Done Done Sent false
18996 5914 Acked Read 6 Done Done Sent false
18997 5915 Acked Read 2 Done Done Sent false
18998 5916 Acked Write 9 Done Done Sent false
18999 5917 Acked Flush 0 Done Done Sent false
19000 5918 Acked Write 7 Done Done Sent false
19001 5919 Acked Write 4 Done Done Sent false
19002 5920 Acked Read 3 Done Done Sent false
19003 5921 Acked Flush 0 Done Done Sent false
19004 5922 Acked Read 1 Done Done Sent false
19005 5923 Acked Read 9 Done Done Sent false
19006 5924 Acked Write 8 Done Done Sent false
19007 5925 Acked Read 6 Done Done Sent false
19008 5926 Acked Read 7 Done Done Sent false
19009 5927 Acked Write 3 Done Done Sent false
19010 5928 Acked Write 1 Done Done Sent false
19011 5929 Acked Write 8 Done Done Sent false
19012 5930 Acked Read 9 Done Done Sent false
19013 5931 Acked Read 8 Done Done Sent false
19014 5932 Acked Write 9 Done Done Sent false
19015 5933 Acked Write 1 Done Done Sent false
19016 5934 Acked Flush 0 Done Done Sent false
19017 5935 Acked Read 7 Done Done Sent false
19018 5936 Acked Read 7 Done Done Sent false
19019 5937 Acked Read 1 Done Done Sent false
19020 5938 Acked Read 9 Done Done Sent false
19021 5939 Acked Read 8 Done Done Sent false
19022 5940 Acked Read 6 Done Done Sent false
19023 5941 Acked Read 4 Done Done Sent false
19024 5942 Acked Read 2 Done Done Sent false
19025 5943 Acked Read 8 Done Done Sent false
19026 5944 Acked Read 10 Done Done Sent false
19027 5945 Acked Write 5 Done Done Sent false
19028 5946 Acked Write 5 Done Done Sent false
19029 5947 Acked Read 9 Done Done Sent false
19030 5948 Acked Write 4 Done Done Sent false
19031 5949 Acked Read 3 Done Done Sent false
19032 5950 Acked Read 5 Done Done Sent false
19033 5951 Acked Flush 0 Done Done Sent false
19034 5952 Acked Read 5 Done Done Sent false
19035 5953 Acked Read 3 Done Done Sent false
19036 5954 Acked Write 3 Done Done Sent false
19037 5955 Acked Read 9 Done Done Sent false
19038 5956 Acked Write 3 Done Done Sent false
19039 5957 Acked Flush 0 Done Done Sent false
19040 5958 Acked Flush 0 Done Done Sent false
19041 5959 Acked Write 8 Done Done Sent false
19042 5960 Acked Write 7 Done Done Sent false
19043 5961 Acked Read 4 Done Done Sent false
19044 5962 Acked Read 4 Done Done Sent false
19045 5963 Acked Read 9 Done Done Sent false
19046 5964 Acked Read 10 Done Done Sent false
19047 5965 Acked Read 8 Done Done Sent false
19048 5966 Acked Read 1 Done Done Sent false
19049 5967 Acked Read 10 Done Done Sent false
19050 5968 Acked Read 5 Done Done Sent false
19051 5969 Acked Write 4 Done Done Sent false
19052 5970 Acked Read 6 Done Done Sent false
19053 5971 Acked Write 7 Done Done Sent false
19054 5972 Acked Flush 0 Done Done Sent false
19055 5973 Acked Write 5 Done Done Sent false
19056 5974 Acked Write 6 Done Done Sent false
19057 5975 Acked Write 1 Done Done Sent false
19058 5976 Acked Write 4 Done Done Sent false
19059 5977 Acked Write 3 Done Done Sent false
19060 5978 Acked Read 5 Done Done Sent false
19061 5979 Acked Write 10 Done Done Sent false
19062 5980 Acked Read 3 Done Done Sent false
19063 5981 Acked Flush 0 Done Done Sent false
19064 5982 Acked Read 6 Done Done Sent false
19065 5983 Acked Write 4 Done Done Sent false
19066 5984 Acked Read 1 Done Done Sent false
19067 5985 Acked Write 1 Done Done Sent false
19068 5986 Acked Write 2 Done Done Sent false
19069 5987 Acked Write 5 Done Done Sent false
19070 5988 Acked Write 1 Done Done Sent false
19071 5989 Acked Write 8 Done Done Sent false
19072 5990 Acked Write 3 Done Done Sent false
19073 5991 Acked Read 6 Done Done Sent false
19074 5992 Acked Flush 0 Done Done Sent false
19075 5993 Acked Read 5 Done Done Sent false
19076 5994 Acked Write 5 Done Done Sent false
19077 5995 Acked Read 3 Done Done Sent false
19078 5996 Acked Read 1 Done Done Sent false
19079 5997 Acked Read 5 Done Done Sent false
19080 5998 Acked Write 3 Done Done Sent false
19081 5999 Acked Read 8 Done Done Sent false
19082 6000 Acked Read 1 Done Done Sent false
19083 6001 Acked Read 2 Done Done Sent false
19084 6002 Acked Read 2 Done Done Sent false
19085 6003 Acked Write 10 Done Done Sent false
19086 6004 Acked Write 5 Done Done Sent false
19087 6005 Acked Read 10 Done Done Sent false
19088 6006 Acked Read 9 Done Done Sent false
19089 6007 Acked Write 2 Done Done Sent false
19090 6008 Acked Write 3 Done Done Sent false
19091 6009 Acked Read 9 Done Done Sent false
19092 6010 Acked Read 2 Done Done Sent false
19093 6011 Acked Write 2 Done Done Sent false
19094 6012 Acked Read 5 Done Done Sent false
19095 6013 Acked Write 6 Done Done Sent false
19096 6014 Acked Write 9 Done Done Sent false
19097 6015 Acked Write 8 Done Done Sent false
19098 6016 Acked Read 8 Done Done Sent false
19099 6017 Acked Read 2 Done Done Sent false
19100 6018 Acked Write 7 Done Done Sent false
19101 6019 Acked Write 10 Done Done Sent false
19102 6020 Acked Write 8 Done Done Sent false
19103 6021 Acked Read 2 Done Done Sent false
19104 6022 Acked Read 2 Done Done Sent false
19105 6023 Acked Write 6 Done Done Sent false
19106 6024 Acked Write 8 Done Done Sent false
19107 6025 Acked Read 2 Done Done Sent false
19108 6026 Acked Read 2 Done Done Sent false
19109 6027 Acked Flush 0 Done Done Sent false
19110 6028 Acked Read 9 Done Done Sent false
19111 6029 Acked Write 9 Done Done Sent false
19112 6030 Acked Write 8 Done Done Sent false
19113 6031 Acked Flush 0 Done Done Sent false
19114 6032 Acked Write 10 Done Done Sent false
19115 6033 Acked Read 6 Done Done Sent false
19116 6034 Acked Read 3 Done Done Sent false
19117 6035 Acked Write 2 Done Done Sent false
19118 6036 Acked Read 3 Done Done Sent false
19119 6037 Acked Read 1 Done Done Sent false
19120 6038 Acked Read 8 Done Done Sent false
19121 6039 Acked Read 7 Done Done Sent false
19122 6040 Acked Read 5 Done Done Sent false
19123 6041 Acked Write 8 Done Done Sent false
19124 6042 Acked Read 10 Done Done Sent false
19125 6043 Acked Write 7 Done Done Sent false
19126 6044 Acked Read 3 Done Done Sent false
19127 6045 Acked Read 6 Done Done Sent false
19128 6046 Acked Write 6 Done Done Sent false
19129 6047 Acked Read 4 Done Done Sent false
19130 6048 Acked Write 1 Done Done Sent false
19131 6049 Acked Flush 0 Done Done Sent false
19132 6050 Acked Write 5 Done Done Sent false
19133 6051 Acked Read 5 Done Done Sent false
19134 6052 Acked Read 1 Done Done Sent false
19135 6053 Acked Write 4 Done Done Sent false
19136 6054 Acked Write 1 Done Done Sent false
19137 6055 Acked Write 6 Done Done Sent false
19138 6056 Acked Read 4 Done Done Sent false
19139 6057 Acked Read 3 Done Done Sent false
19140 6058 Acked Write 4 Done Done Sent false
19141 6059 Acked Read 2 Done Done Sent false
19142 6060 Acked Write 10 Done Done Sent false
19143 6061 Acked Read 1 Done Done Sent false
19144 6062 Acked Read 6 Done Done Sent false
19145 6063 Acked Write 2 Done Done Sent false
19146 6064 Acked Write 4 Done Done Sent false
19147 6065 Acked Write 7 Done Done Sent false
19148 6066 Acked Write 1 Done Done Sent false
19149 6067 Acked Write 8 Done Done Sent false
19150 6068 Acked Read 10 Done Done Sent false
19151 6069 Acked Write 5 Done Done Sent false
19152 6070 Acked Read 9 Done Done Sent false
19153 6071 Acked Read 8 Done Done Sent false
19154 6072 Acked Read 8 Done Done Sent false
19155 6073 Acked Read 3 Done Done Sent false
19156 6074 Acked Read 7 Done Done Sent false
19157 6075 Acked Read 6 Done Done Sent false
19158 6076 Acked Read 9 Done Done Sent false
19159 6077 Acked Flush 0 Done Done Sent false
19160 6078 Acked Write 2 Done Done Sent false
19161 6079 Acked Read 5 Done Done Sent false
19162 6080 Acked Write 2 Done Done Sent false
19163 6081 Acked Write 8 Done Done Sent false
19164 6082 Acked Read 3 Done Done Sent false
19165 6083 Acked Flush 0 Done Done Sent false
19166 6084 Acked Write 8 Done Done Sent false
19167 6085 Acked Read 2 Done Done Sent false
19168 6086 Acked Read 8 Done Done Sent false
19169 6087 Acked Read 10 Done Done Sent false
19170 6088 Acked Write 5 Done Done Sent false
19171 6089 Acked Read 3 Done Done Sent false
19172 6090 Acked Write 6 Done Done Sent false
19173 6091 Acked Read 6 Done Done Sent false
19174 6092 Acked Read 8 Done Done Sent false
19175 6093 Acked Read 3 Done Done Sent false
19176 6094 Acked Write 6 Done Done Sent false
19177 6095 Acked Read 10 Done Done Sent false
19178 6096 Acked Flush 0 Done Done Sent false
19179 6097 Acked Read 8 Done Done Sent false
19180 6098 Acked Read 9 Done Done Sent false
19181 6099 Acked Write 6 Done Done Sent false
19182 6100 Acked Write 5 Done Done Sent false
19183 6101 Acked Read 3 Done Done Sent false
19184 6102 Acked Read 8 Done Done Sent false
19185 6103 Acked Flush 0 Done Done Sent false
19186 6104 Acked Read 7 Done Done Sent false
19187 6105 Acked Write 7 Done Done Sent false
19188 6106 Acked Read 5 Done Done Sent false
19189 6107 Acked Write 3 Done Done Sent false
19190 6108 Acked Read 2 Done Done Sent false
19191 6109 Acked Write 5 Done Done Sent false
19192 6110 Acked Read 4 Done Done Sent false
19193 6111 Acked Write 5 Done Done Sent false
19194 6112 Acked Write 6 Done Done Sent false
19195 6113 Acked Write 8 Done Done Sent false
19196 6114 Acked Write 5 Done Done Sent false
19197 6115 Acked Read 5 Done Done Sent false
19198 6116 Acked Write 4 Done Done Sent false
19199 6117 Acked Write 2 Done Done Sent false
19200 6118 Acked Write 1 Done Done Sent false
19201 6119 Acked Flush 0 Done Done Sent false
19202 6120 Acked Read 4 Done Done Sent false
19203 6121 Acked Read 8 Done Done Sent false
19204 6122 Acked Write 10 Done Done Sent false
19205 6123 Acked Read 3 Done Done Sent false
19206 6124 Acked Write 10 Done Done Sent false
19207 6125 Acked Write 3 Done Done Sent false
19208 6126 Acked Read 3 Done Done Sent false
19209 6127 Acked Write 5 Done Done Sent false
19210 6128 Acked Write 9 Done Done Sent false
19211 6129 Acked Read 2 Done Done Sent false
19212 6130 Acked Write 10 Done Done Sent false
19213 6131 Acked Flush 0 Done Done Sent false
19214 STATES DS:0 DS:1 DS:2 TOTAL
19215 Sent 0 0 302 302
19216 Done 302 302 0 604
19217 Skipped 0 0 0 0
19218 Error 0 0 0 0
19219 Last Flush: 6131 6131 5829
19220 Downstairs last five completed: 5829 5828 5827 5826 5825
19221 Upstairs last five completed: 6131 6130 6129 6128 6127
19222 CLIENT: Up:0 ds:302 act:2
192232025-01-13T15:55:49.078ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
192242025-01-13T15:55:49.078ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41370) = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
192252025-01-13T15:55:49.080ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
192262025-01-13T15:55:49.080ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
192272025-01-13T15:55:49.081ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
192282025-01-13T15:55:49.081ZINFOcrucible: send last flush ID to this DS: 5829 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
192292025-01-13T15:55:49.081ZINFOcrucible: Replied this last flush ID: 5829 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
192302025-01-13T15:55:49.081ZINFOcrucible: [2] client re-new 302 jobs since flush 5829 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
192312025-01-13T15:55:49.081ZINFOcrucible: [2] Replayed 302 jobs since flush: 5829 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
19232 ----------------------------------------------------------------
19233 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
19234 Downstairs last five completed: 6131 6130 6129 6128 6127
19235 Upstairs last five completed: 6131 6130 6129 6128 6127
19236 CLIENT: Up:0 ds:0 act:3
19237 Replay: All jobs finished
19238 Main task has work to do, go find it
19239 got dsc Stop(2)
19240 stop 2
19241 Jan 13 15:55:51.323 INFO request completed, [8830] Got stop action so:false kr:false
19242 latency_us: 361, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 228bf768-6164-4a9c-a124-1b391e07e06f, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
192432025-01-13T15:55:51.323ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
192442025-01-13T15:55:51.323ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
192452025-01-13T15:55:51.323ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
19246 Jan 13 15:55:51.323 INFO request completed, latency_us: 113, response_code: 200, uri: /state/cid/2, method: GET, req_id: 9f52198b-8cb5-4ef8-bcc6-b466bc1a07f5, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
192472025-01-13T15:55:51.323ZINFOcrucible: client task is exiting = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
19248 [8830] Exited with: ExitStatus(unix_wait_status(9))
19249 [8830][2] reports Exit
19250 Jan 13 15:55:55.325 INFO request completed, latency_us: 175, response_code: 200, uri: /state/cid/2, method: GET, req_id: a3c5df12-7215-474e-95cf-861926ec4fd4, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
19251 001/300 Flush
19252 002/300 Read block 4223 len 16384
19253 003/300 Flush
19254 004/300 Flush
19255 005/300 Read block 3582 len 16384
19256 006/300 Read block 4686 len 32768
19257 007/300 Read block 2073 len 28672
19258 008/300 Flush
19259 009/300 Write block 4263 len 36864 data: 3 3 2 5 5 6 6 7 7
19260 010/300 Write block 172 len 8192 data: 6 7
19261 011/300 Flush
19262 012/300 Write block 599 len 40960 data: 4 4 4 4 3 2 2 2 2 3
19263 013/300 Write block 4331 len 12288 data: 4 4 3
19264 014/300 Read block 4604 len 12288
19265 015/300 Write block 4960 len 8192 data: 5 3
19266 016/300 Flush
19267 017/300 Write block 2596 len 8192 data: 12 9
19268 018/300 Flush
19269 019/300 Read block 2984 len 20480
19270 020/300 Write block 1332 len 4096 data: 3
19271 021/300 Read block 2038 len 16384
19272 022/300 Write block 697 len 28672 data: 6 5 5 5 5 5 4
19273 023/300 Read block 1142 len 4096
19274 024/300 Write block 1784 len 16384 data: 4 4 3 3
19275 025/300 Write block 1549 len 20480 data: 4 3 3 3 3
19276 026/300 Read block 2398 len 16384
19277 027/300 Read block 1864 len 16384
19278 028/300 Read block 4532 len 16384
19279 029/300 Read block 3223 len 4096
19280 030/300 Write block 575 len 12288 data: 5 6 6
19281 031/300 Read block 2474 len 36864
19282 032/300 Read block 2843 len 28672
19283 033/300 Read block 519 len 36864
19284 034/300 Read block 2546 len 20480
19285 035/300 Write block 3049 len 20480 data: 4 5 5 4 3
19286 036/300 Read block 4048 len 20480
19287 037/300 Flush
19288 038/300 Read block 4365 len 40960
19289 039/300 Read block 1537 len 16384
19290 040/300 Read block 443 len 36864
19291 041/300 Read block 2719 len 8192
19292 042/300 Write block 3607 len 4096 data: 2
19293 043/300 Flush
19294 044/300 Read block 2828 len 24576
19295 045/300 Write block 839 len 4096 data: 5
19296 046/300 Read block 3694 len 28672
19297 047/300 Read block 4214 len 16384
19298 048/300 Read block 2800 len 40960
19299 049/300 Read block 4110 len 4096
19300 050/300 Read block 2243 len 20480
19301 051/300 Flush
19302 052/300 Read block 4992 len 32768
19303 053/300 Write block 3116 len 40960 data: 4 3 4 4 2 3 4 3 3 3
19304 054/300 Read block 4856 len 12288
19305 055/300 Write block 4596 len 12288 data: 7 7 6
19306 056/300 Write block 3191 len 36864 data: 11 10 7 5 6 6 5 4 5
19307 057/300 Write block 4452 len 12288 data: 6 5 6
19308 058/300 Write block 4164 len 36864 data: 5 4 3 3 3 3 3 3 5
19309 059/300 Flush
19310 060/300 Write block 4609 len 20480 data: 4 3 2 2 3
19311 061/300 Read block 4924 len 24576
19312 062/300 Write block 4434 len 36864 data: 2 3 4 4 3 4 4 5 5
19313 063/300 Read block 3744 len 32768
19314 064/300 Write block 1850 len 24576 data: 3 4 4 5 4 5
19315 065/300 Write block 2588 len 28672 data: 2 2 5 7 8 10 10
19316 066/300 Write block 1812 len 28672 data: 4 5 5 5 5 5 6
19317 067/300 Write block 1893 len 4096 data: 4
19318 068/300 Write block 2501 len 36864 data: 3 3 3 2 2 2 2 2 2
19319 069/300 Write block 1219 len 40960 data: 4 4 3 4 4 4 3 4 3 3
19320 070/300 Write block 949 len 40960 data: 5 4 4 3 2 3 3 3 2 3
19321 071/300 Read block 1619 len 8192
19322 072/300 Read block 2137 len 4096
19323 073/300 Read block 3976 len 24576
19324 074/300 Write block 2051 len 8192 data: 7 6
19325 075/300 Write block 506 len 32768 data: 2 3 2 3 4 5 6 6
19326 076/300 Read block 3743 len 28672
19327 077/300 Read block 4801 len 12288
19328 078/300 Write block 3509 len 20480 data: 5 5 6 6 8
19329 079/300 Write block 3704 len 16384 data: 3 3 5 5
19330 080/300 Read block 4102 len 40960
19331 081/300 Read block 1800 len 32768
19332 082/300 Write block 4391 len 40960 data: 4 4 4 4 6 7 8 8 8 6
19333 083/300 Read block 3505 len 4096
19334 084/300 Write block 1961 len 4096 data: 4
19335 085/300 Read block 2760 len 20480
19336 086/300 Write block 1008 len 4096 data: 3
19337 087/300 Read block 1681 len 24576
19338 088/300 Write block 4561 len 40960 data: 4 4 4 4 4 6 5 5 5 3
19339 089/300 Write block 282 len 40960 data: 5 5 6 8 8 8 7 5 6 6
19340 090/300 Read block 3795 len 36864
19341 091/300 Write block 3744 len 12288 data: 3 3 3
19342 092/300 Write block 2257 len 40960 data: 3 3 3 3 2 4 6 7 6 6
19343 093/300 Read block 47 len 32768
19344 094/300 Read block 2730 len 12288
19345 095/300 Write block 239 len 40960 data: 4 5 6 4 3 3 3 4 3 2
19346 096/300 Write block 4703 len 4096 data: 2
19347 097/300 Read block 2939 len 8192
19348 098/300 Write block 3024 len 24576 data: 8 6 5 7 7 5
19349 099/300 Write block 2839 len 28672 data: 3 2 2 2 3 4 4
19350 100/300 Read block 4614 len 24576
19351 101/300 Write block 3822 len 8192 data: 4 4
19352 102/300 Write block 34 len 12288 data: 4 4 4
19353 103/300 Read block 1621 len 24576
19354 104/300 Read block 4538 len 4096
19355 105/300 Flush
19356 106/300 Flush
19357 107/300 Flush
19358 108/300 Read block 1255 len 36864
19359 109/300 Write block 2136 len 40960 data: 4 4 5 4 3 4 4 4 2 2
19360 110/300 Read block 4128 len 16384
19361 111/300 Read block 1944 len 8192
19362 112/300 Write block 1832 len 36864 data: 4 4 5 4 4 5 6 5 4
19363 113/300 Write block 4875 len 12288 data: 4 3 3
19364 114/300 Write block 2650 len 20480 data: 6 5 5 5 5
19365 115/300 Read block 3537 len 16384
19366 116/300 Read block 2706 len 36864
19367 117/300 Read block 4870 len 16384
19368 118/300 Read block 4570 len 28672
19369 119/300 Write block 2220 len 24576 data: 5 5 5 6 6 6
19370 120/300 Read block 4083 len 4096
19371 121/300 Read block 2191 len 8192
19372 122/300 Read block 3682 len 8192
19373 123/300 Write block 3041 len 8192 data: 5 4
19374 124/300 Write block 1632 len 24576 data: 4 4 3 3 3 4
19375 125/300 Write block 2697 len 36864 data: 5 7 7 6 7 7 7 6 6
19376 126/300 Read block 2627 len 24576
19377 127/300 Write block 137 len 16384 data: 4 3 2 3
19378 128/300 Read block 699 len 40960
19379 129/300 Write block 900 len 16384 data: 3 2 2 2
19380 130/300 Write block 1252 len 16384 data: 4 4 4 3
19381 131/300 Write block 4667 len 36864 data: 4 4 3 3 4 4 5 6 5
19382 132/300 Flush
19383 133/300 Write block 3748 len 8192 data: 5 6
19384 134/300 Write block 2107 len 16384 data: 3 4 5 5
19385 135/300 Write block 2472 len 20480 data: 3 3 4 5 4
19386 136/300 Write block 1861 len 24576 data: 3 4 3 3 4 4
19387 137/300 Write block 4109 len 16384 data: 7 7 7 7
19388 138/300 Read block 100 len 20480
19389 139/300 Read block 1465 len 24576
19390 140/300 Flush
19391 141/300 Write block 3093 len 32768 data: 3 3 4 4 4 4 5 5
19392 142/300 Write block 1338 len 4096 data: 8
19393 143/300 Read block 1910 len 40960
19394 144/300 Read block 1726 len 20480
19395 145/300 Write block 1516 len 32768 data: 5 6 7 6 7 6 7 6
19396 146/300 Read block 2242 len 28672
19397 147/300 Read block 4879 len 20480
19398 148/300 Read block 4766 len 20480
19399 149/300 Write block 1933 len 24576 data: 3 2 2 3 3 3
19400 150/300 Write block 832 len 28672 data: 6 5 6 5 5 5 6
19401 151/300 Read block 2853 len 20480
19402 152/300 Write block 4297 len 8192 data: 5 7
19403 153/300 Read block 255 len 8192
19404 154/300 Read block 2982 len 4096
19405 155/300 Read block 1476 len 36864
19406 156/300 Read block 2814 len 36864
19407 157/300 Write block 9 len 4096 data: 5
19408 158/300 Read block 3691 len 4096
19409 159/300 Write block 4525 len 36864 data: 4 4 3 2 3 4 5 6 4
19410 160/300 Read block 1647 len 16384
19411 161/300 Write block 1373 len 32768 data: 4 4 3 4 6 6 8 8
19412 162/300 Write block 3868 len 28672 data: 3 4 4 5 5 6 5
19413 163/300 Read block 420 len 40960
19414 164/300 Read block 3571 len 4096
19415 165/300 Read block 4652 len 12288
19416 166/300 Read block 1749 len 8192
19417 167/300 Write block 4725 len 20480 data: 2 2 2 3 4
19418 168/300 Read block 2093 len 12288
19419 169/300 Read block 1750 len 28672
19420 170/300 Read block 1699 len 12288
19421 171/300 Read block 614 len 32768
19422 172/300 Read block 4360 len 40960
19423 173/300 Write block 1936 len 8192 data: 4 4
19424 174/300 Read block 435 len 4096
19425 175/300 Write block 150 len 28672 data: 6 5 5 3 4 5 4
19426 176/300 Read block 2290 len 12288
19427 177/300 Read block 591 len 36864
19428 178/300 Read block 1136 len 8192
19429 179/300 Read block 199 len 20480
19430 180/300 Read block 2328 len 36864
19431 181/300 Write block 2940 len 12288 data: 6 5 5
19432 182/300 Read block 2146 len 40960
19433 183/300 Read block 4113 len 20480
19434 184/300 Read block 4109 len 36864
19435 185/300 Read block 3552 len 4096
19436 186/300 Read block 1943 len 32768
19437 187/300 Write block 2603 len 8192 data: 5 5
19438 188/300 Read block 4641 len 8192
19439 189/300 Read block 2210 len 4096
19440 190/300 Flush
19441 191/300 Flush
19442 192/300 Write block 1559 len 8192 data: 4 5
19443 193/300 Write block 1626 len 32768 data: 4 4 4 3 3 3 5 5
19444 194/300 Read block 4332 len 36864
19445 195/300 Write block 3386 len 16384 data: 4 4 4 6
19446 196/300 Write block 1571 len 8192 data: 6 5
19447 197/300 Read block 298 len 32768
19448 198/300 Read block 2875 len 28672
19449 199/300 Read block 2566 len 24576
19450 200/300 Flush
19451 201/300 Read block 3431 len 32768
19452 202/300 Write block 4589 len 24576 data: 2 2 3 3 3 5
19453 203/300 Read block 4140 len 16384
19454 204/300 Read block 2100 len 12288
19455 205/300 Write block 1078 len 32768 data: 4 4 5 4 5 5 6 6
19456 206/300 Read block 2272 len 16384
19457 207/300 Read block 1767 len 36864
19458 208/300 Write block 903 len 8192 data: 3 2
19459 209/300 Flush
19460 210/300 Read block 2902 len 40960
19461 211/300 Read block 4389 len 20480
19462 212/300 Flush
19463 213/300 Read block 3246 len 28672
19464 214/300 Write block 1616 len 36864 data: 7 6 5 5 5 8 7 6 6
19465 215/300 Read block 534 len 8192
19466 216/300 Write block 2605 len 32768 data: 5 5 5 5 4 4 4 3
19467 217/300 Flush
19468 218/300 Read block 2309 len 32768
19469 219/300 Read block 853 len 20480
19470 220/300 Write block 3755 len 8192 data: 4 4
19471 221/300 Read block 4426 len 40960
19472 222/300 Write block 4815 len 24576 data: 3 3 4 3 4 4
19473 223/300 Read block 1649 len 32768
19474 224/300 Write block 4101 len 24576 data: 2 2 2 2 3 4
19475 225/300 Write block 2471 len 24576 data: 3 4 4 5 6 5
19476 226/300 Write block 1950 len 16384 data: 2 2 4 4
19477 227/300 Write block 747 len 36864 data: 3 3 5 6 5 6 6 7 8
19478 228/300 Read block 1225 len 4096
19479 229/300 Read block 65 len 16384
19480 230/300 Read block 4647 len 24576
19481 231/300 Write block 2747 len 36864 data: 5 5 6 6 7 7 6 5 5
19482 232/300 Write block 1782 len 12288 data: 4 4 5
19483 233/300 Write block 2373 len 36864 data: 3 3 5 5 6 5 5 4 5
19484 234/300 Write block 2603 len 20480 data: 6 6 6 6 6
19485 235/300 Read block 3482 len 32768
19486 236/300 Write block 2930 len 40960 data: 6 5 5 3 2 2 3 4 6 6
19487 237/300 Write block 4979 len 16384 data: 3 4 4 4
19488 238/300 Write block 3706 len 16384 data: 6 6 6 5
19489 239/300 Write block 1542 len 40960 data: 4 3 3 2 2 2 2 5 4 4
19490 240/300 Write block 2392 len 4096 data: 5
19491 241/300 Read block 2723 len 32768
19492 242/300 Read block 591 len 16384
19493 243/300 Write block 2101 len 36864 data: 5 4 3 3 3 3 4 5 6
19494 244/300 Write block 1868 len 8192 data: 3 3
19495 245/300 Flush
19496 246/300 Read block 4875 len 28672
19497 247/300 Read block 946 len 40960
19498 248/300 Write block 1328 len 36864 data: 6 4 3 2 4 4 5 7 9
19499 249/300 Write block 4039 len 8192 data: 2 2
19500 250/300 Write block 1920 len 28672 data: 4 4 4 4 3 3 3
19501 251/300 Read block 569 len 16384
19502 252/300 Write block 3206 len 32768 data: 4 4 4 4 4 4 3 3
19503 253/300 Read block 251 len 32768
19504 254/300 Write block 2544 len 24576 data: 4 5 6 6 6 6
19505 255/300 Flush
19506 256/300 Read block 2837 len 8192
19507 257/300 Flush
19508 258/300 Write block 2614 len 4096 data: 3
19509 259/300 Read block 3551 len 28672
19510 260/300 Write block 3983 len 16384 data: 6 7 6 6
19511 261/300 Read block 4349 len 32768
19512 262/300 Read block 3919 len 28672
19513 263/300 Write block 2314 len 28672 data: 2 2 3 3 3 3 3
19514 264/300 Flush
19515 265/300 Write block 3895 len 36864 data: 5 5 5 5 7 6 6 5 4
19516 266/300 Read block 1601 len 24576
19517 267/300 Write block 1263 len 32768 data: 6 6 6 5 5 4 4 4
19518 268/300 Read block 3335 len 24576
19519 269/300 Read block 175 len 8192
19520 270/300 Write block 3924 len 8192 data: 4 3
19521 271/300 Read block 2061 len 4096
19522 272/300 Read block 3808 len 36864
19523 273/300 Write block 164 len 36864 data: 2 2 2 3 5 5 5 5 7
19524 274/300 Read block 3156 len 20480
19525 275/300 Write block 4497 len 32768 data: 6 5 5 5 6 7 8 6
19526 276/300 Read block 57 len 20480
19527 277/300 Read block 4507 len 36864
19528 278/300 Read block 4418 len 40960
19529 279/300 Flush
19530 280/300 Read block 4976 len 20480
19531 281/300 Write block 2126 len 32768 data: 5 5 4 5 4 3 3 3
19532 282/300 Write block 1407 len 32768 data: 3 4 5 5 5 6 5 5
19533 283/300 Write block 2308 len 40960 data: 4 5 5 4 2 2 3 3 4 4
19534 284/300 Flush
19535 285/300 Flush
19536 286/300 Read block 4798 len 16384
19537 287/300 Read block 4843 len 36864
19538 288/300 Read block 1248 len 32768
19539 289/300 Write block 833 len 8192 data: 6 7
19540 290/300 Write block 4012 len 8192 data: 4 4
19541 291/300 Read block 3390 len 12288
19542 292/300 Read block 2736 len 20480
19543 293/300 Read block 4169 len 8192
19544 294/300 Read block 2271 len 36864
19545 295/300 Write block 1577 len 28672 data: 6 4 5 4 4 5 4
19546 296/300 Read block 4185 len 20480
19547 297/300 Flush
19548 298/300 Read block 1092 len 12288
19549 299/300 Read block 4501 len 24576
19550 300/300 Write block 2418 len 8192 data: 4 3
19551 301/300 Read block 1739 len 16384
19552 Main task has work to do, go find it
19553 got dsc Start(2)
19554 start 2
19555 Jan 13 15:55:55.565 INFO request completed, latency_us: 172, response_code: 204, uri: /start/cid/2, method: POST, req_id: 60df396a-8d9e-45f6-aa52-297c9272304e, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
19556 Replay: started 2, returned:Ok(())
19557 ----------------------------------------------------------------
19558 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
19559 States: Active Active Offline
19560 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19561 6132 Acked Flush 0 Done Done Sent false
19562 6133 Acked Read 4 Done Done Sent false
19563 6134 Acked Flush 0 Done Done Sent false
19564 6135 Acked Flush 0 Done Done Sent false
19565 6136 Acked Read 4 Done Done Sent false
19566 6137 Acked Read 8 Done Done Sent false
19567 6138 Acked Read 7 Done Done Sent false
19568 6139 Acked Flush 0 Done Done Sent false
19569 6140 Acked Write 9 Done Done Sent false
19570 6141 Acked Write 2 Done Done Sent false
19571 6142 Acked Flush 0 Done Done Sent false
19572 6143 Acked Write 10 Done Done Sent false
19573 6144 Acked Write 3 Done Done Sent false
19574 6145 Acked Read 3 Done Done Sent false
19575 6146 Acked Write 2 Done Done Sent false
19576 6147 Acked Flush 0 Done Done Sent false
19577 6148 Acked Write 2 Done Done Sent false
19578 6149 Acked Flush 0 Done Done Sent false
19579 6150 Acked Read 5 Done Done Sent false
19580 6151 Acked Write 1 Done Done Sent false
19581 6152 Acked Read 4 Done Done Sent false
19582 6153 Acked Write 7 Done Done Sent false
19583 6154 Acked Read 1 Done Done Sent false
19584 6155 Acked Write 4 Done Done Sent false
19585 6156 Acked Write 5 Done Done Sent false
19586 6157 Acked Read 4 Done Done Sent false
19587 6158 Acked Read 4 Done Done Sent false
19588 6159 Acked Read 4 Done Done Sent false
19589 6160 Acked Read 1 Done Done Sent false
19590 6161 Acked Write 3 Done Done Sent false
19591 6162 Acked Read 9 Done Done Sent false
19592 6163 Acked Read 7 Done Done Sent false
19593 6164 Acked Read 9 Done Done Sent false
19594 6165 Acked Read 5 Done Done Sent false
19595 6166 Acked Write 5 Done Done Sent false
19596 6167 Acked Read 5 Done Done Sent false
19597 6168 Acked Flush 0 Done Done Sent false
19598 6169 Acked Read 10 Done Done Sent false
19599 6170 Acked Read 4 Done Done Sent false
19600 6171 Acked Read 9 Done Done Sent false
19601 6172 Acked Read 2 Done Done Sent false
19602 6173 Acked Write 1 Done Done Sent false
19603 6174 Acked Flush 0 Done Done Sent false
19604 6175 Acked Read 6 Done Done Sent false
19605 6176 Acked Write 1 Done Done Sent false
19606 6177 Acked Read 7 Done Done Sent false
19607 6178 Acked Read 4 Done Done Sent false
19608 6179 Acked Read 10 Done Done Sent false
19609 6180 Acked Read 1 Done Done Sent false
19610 6181 Acked Read 5 Done Done Sent false
19611 6182 Acked Flush 0 Done Done Sent false
19612 6183 Acked Read 8 Done Done Sent false
19613 6184 Acked Write 10 Done Done Sent false
19614 6185 Acked Read 3 Done Done Sent false
19615 6186 Acked Write 3 Done Done Sent false
19616 6187 Acked Write 9 Done Done Sent false
19617 6188 Acked Write 3 Done Done Sent false
19618 6189 Acked Write 9 Done Done Sent false
19619 6190 Acked Flush 0 Done Done Sent false
19620 6191 Acked Write 5 Done Done Sent false
19621 6192 Acked Read 6 Done Done Sent false
19622 6193 Acked Write 9 Done Done Sent false
19623 6194 Acked Read 8 Done Done Sent false
19624 6195 Acked Write 6 Done Done Sent false
19625 6196 Acked Write 7 Done Done Sent false
19626 6197 Acked Write 7 Done Done Sent false
19627 6198 Acked Write 1 Done Done Sent false
19628 6199 Acked Write 9 Done Done Sent false
19629 6200 Acked Write 10 Done Done Sent false
19630 6201 Acked Write 10 Done Done Sent false
19631 6202 Acked Read 2 Done Done Sent false
19632 6203 Acked Read 1 Done Done Sent false
19633 6204 Acked Read 6 Done Done Sent false
19634 6205 Acked Write 2 Done Done Sent false
19635 6206 Acked Write 8 Done Done Sent false
19636 6207 Acked Read 7 Done Done Sent false
19637 6208 Acked Read 3 Done Done Sent false
19638 6209 Acked Write 5 Done Done Sent false
19639 6210 Acked Write 4 Done Done Sent false
19640 6211 Acked Read 10 Done Done Sent false
19641 6212 Acked Read 8 Done Done Sent false
19642 6213 Acked Write 10 Done Done Sent false
19643 6214 Acked Read 1 Done Done Sent false
19644 6215 Acked Write 1 Done Done Sent false
19645 6216 Acked Read 5 Done Done Sent false
19646 6217 Acked Write 1 Done Done Sent false
19647 6218 Acked Read 6 Done Done Sent false
19648 6219 Acked Write 10 Done Done Sent false
19649 6220 Acked Write 10 Done Done Sent false
19650 6221 Acked Read 9 Done Done Sent false
19651 6222 Acked Write 3 Done Done Sent false
19652 6223 Acked Write 10 Done Done Sent false
19653 6224 Acked Read 8 Done Done Sent false
19654 6225 Acked Read 3 Done Done Sent false
19655 6226 Acked Write 10 Done Done Sent false
19656 6227 Acked Write 1 Done Done Sent false
19657 6228 Acked Read 2 Done Done Sent false
19658 6229 Acked Write 6 Done Done Sent false
19659 6230 Acked Write 7 Done Done Sent false
19660 6231 Acked Read 6 Done Done Sent false
19661 6232 Acked Write 2 Done Done Sent false
19662 6233 Acked Write 3 Done Done Sent false
19663 6234 Acked Read 6 Done Done Sent false
19664 6235 Acked Read 1 Done Done Sent false
19665 6236 Acked Flush 0 Done Done Sent false
19666 6237 Acked Flush 0 Done Done Sent false
19667 6238 Acked Flush 0 Done Done Sent false
19668 6239 Acked Read 9 Done Done Sent false
19669 6240 Acked Write 10 Done Done Sent false
19670 6241 Acked Read 4 Done Done Sent false
19671 6242 Acked Read 2 Done Done Sent false
19672 6243 Acked Write 9 Done Done Sent false
19673 6244 Acked Write 3 Done Done Sent false
19674 6245 Acked Write 5 Done Done Sent false
19675 6246 Acked Read 4 Done Done Sent false
19676 6247 Acked Read 9 Done Done Sent false
19677 6248 Acked Read 4 Done Done Sent false
19678 6249 Acked Read 7 Done Done Sent false
19679 6250 Acked Write 6 Done Done Sent false
19680 6251 Acked Read 1 Done Done Sent false
19681 6252 Acked Read 2 Done Done Sent false
19682 6253 Acked Read 2 Done Done Sent false
19683 6254 Acked Write 2 Done Done Sent false
19684 6255 Acked Write 6 Done Done Sent false
19685 6256 Acked Write 9 Done Done Sent false
19686 6257 Acked Read 6 Done Done Sent false
19687 6258 Acked Write 4 Done Done Sent false
19688 6259 Acked Read 10 Done Done Sent false
19689 6260 Acked Write 4 Done Done Sent false
19690 6261 Acked Write 4 Done Done Sent false
19691 6262 Acked Write 9 Done Done Sent false
19692 6263 Acked Flush 0 Done Done Sent false
19693 6264 Acked Write 2 Done Done Sent false
19694 6265 Acked Write 4 Done Done Sent false
19695 6266 Acked Write 5 Done Done Sent false
19696 6267 Acked Write 6 Done Done Sent false
19697 6268 Acked Write 4 Done Done Sent false
19698 6269 Acked Read 5 Done Done Sent false
19699 6270 Acked Read 6 Done Done Sent false
19700 6271 Acked Flush 0 Done Done Sent false
19701 6272 Acked Write 8 Done Done Sent false
19702 6273 Acked Write 1 Done Done Sent false
19703 6274 Acked Read 10 Done Done Sent false
19704 6275 Acked Read 5 Done Done Sent false
19705 6276 Acked Write 8 Done Done Sent false
19706 6277 Acked Read 7 Done Done Sent false
19707 6278 Acked Read 5 Done Done Sent false
19708 6279 Acked Read 5 Done Done Sent false
19709 6280 Acked Write 6 Done Done Sent false
19710 6281 Acked Write 7 Done Done Sent false
19711 6282 Acked Read 5 Done Done Sent false
19712 6283 Acked Write 2 Done Done Sent false
19713 6284 Acked Read 2 Done Done Sent false
19714 6285 Acked Read 1 Done Done Sent false
19715 6286 Acked Read 9 Done Done Sent false
19716 6287 Acked Read 9 Done Done Sent false
19717 6288 Acked Write 1 Done Done Sent false
19718 6289 Acked Read 1 Done Done Sent false
19719 6290 Acked Write 9 Done Done Sent false
19720 6291 Acked Read 4 Done Done Sent false
19721 6292 Acked Write 8 Done Done Sent false
19722 6293 Acked Write 7 Done Done Sent false
19723 6294 Acked Read 10 Done Done Sent false
19724 6295 Acked Read 1 Done Done Sent false
19725 6296 Acked Read 3 Done Done Sent false
19726 6297 Acked Read 2 Done Done Sent false
19727 6298 Acked Write 5 Done Done Sent false
19728 6299 Acked Read 3 Done Done Sent false
19729 6300 Acked Read 7 Done Done Sent false
19730 6301 Acked Read 3 Done Done Sent false
19731 6302 Acked Read 8 Done Done Sent false
19732 6303 Acked Read 10 Done Done Sent false
19733 6304 Acked Write 2 Done Done Sent false
19734 6305 Acked Read 1 Done Done Sent false
19735 6306 Acked Write 7 Done Done Sent false
19736 6307 Acked Read 3 Done Done Sent false
19737 6308 Acked Read 9 Done Done Sent false
19738 6309 Acked Read 2 Done Done Sent false
19739 6310 Acked Read 5 Done Done Sent false
19740 6311 Acked Read 9 Done Done Sent false
19741 6312 Acked Write 3 Done Done Sent false
19742 6313 Acked Read 10 Done Done Sent false
19743 6314 Acked Read 5 Done Done Sent false
19744 6315 Acked Read 9 Done Done Sent false
19745 6316 Acked Read 1 Done Done Sent false
19746 6317 Acked Read 8 Done Done Sent false
19747 6318 Acked Write 2 Done Done Sent false
19748 6319 Acked Read 2 Done Done Sent false
19749 6320 Acked Read 1 Done Done Sent false
19750 6321 Acked Flush 0 Done Done Sent false
19751 6322 Acked Flush 0 Done Done Sent false
19752 6323 Acked Write 2 Done Done Sent false
19753 6324 Acked Write 8 Done Done Sent false
19754 6325 Acked Read 9 Done Done Sent false
19755 6326 Acked Write 4 Done Done Sent false
19756 6327 Acked Write 2 Done Done Sent false
19757 6328 Acked Read 8 Done Done Sent false
19758 6329 Acked Read 7 Done Done Sent false
19759 6330 Acked Read 6 Done Done Sent false
19760 6331 Acked Flush 0 Done Done Sent false
19761 6332 Acked Read 8 Done Done Sent false
19762 6333 Acked Write 6 Done Done Sent false
19763 6334 Acked Read 4 Done Done Sent false
19764 6335 Acked Read 3 Done Done Sent false
19765 6336 Acked Write 8 Done Done Sent false
19766 6337 Acked Read 4 Done Done Sent false
19767 6338 Acked Read 9 Done Done Sent false
19768 6339 Acked Write 2 Done Done Sent false
19769 6340 Acked Flush 0 Done Done Sent false
19770 6341 Acked Read 10 Done Done Sent false
19771 6342 Acked Read 5 Done Done Sent false
19772 6343 Acked Flush 0 Done Done Sent false
19773 6344 Acked Read 7 Done Done Sent false
19774 6345 Acked Write 9 Done Done Sent false
19775 6346 Acked Read 2 Done Done Sent false
19776 6347 Acked Write 8 Done Done Sent false
19777 6348 Acked Flush 0 Done Done Sent false
19778 6349 Acked Read 8 Done Done Sent false
19779 6350 Acked Read 5 Done Done Sent false
19780 6351 Acked Write 2 Done Done Sent false
19781 6352 Acked Read 10 Done Done Sent false
19782 6353 Acked Write 6 Done Done Sent false
19783 6354 Acked Read 8 Done Done Sent false
19784 6355 Acked Write 6 Done Done Sent false
19785 6356 Acked Write 6 Done Done Sent false
19786 6357 Acked Write 4 Done Done Sent false
19787 6358 Acked Write 9 Done Done Sent false
19788 6359 Acked Read 1 Done Done Sent false
19789 6360 Acked Read 4 Done Done Sent false
19790 6361 Acked Read 6 Done Done Sent false
19791 6362 Acked Write 9 Done Done Sent false
19792 6363 Acked Write 3 Done Done Sent false
19793 6364 Acked Write 9 Done Done Sent false
19794 6365 Acked Write 5 Done Done Sent false
19795 6366 Acked Read 8 Done Done Sent false
19796 6367 Acked Write 10 Done Done Sent false
19797 6368 Acked Write 4 Done Done Sent false
19798 6369 Acked Write 4 Done Done Sent false
19799 6370 Acked Write 10 Done Done Sent false
19800 6371 Acked Write 1 Done Done Sent false
19801 6372 Acked Read 8 Done Done Sent false
19802 6373 Acked Read 4 Done Done Sent false
19803 6374 Acked Write 9 Done Done Sent false
19804 6375 Acked Write 2 Done Done Sent false
19805 6376 Acked Flush 0 Done Done Sent false
19806 6377 Acked Read 7 Done Done Sent false
19807 6378 Acked Read 10 Done Done Sent false
19808 6379 Acked Write 9 Done Done Sent false
19809 6380 Acked Write 2 Done Done Sent false
19810 6381 Acked Write 7 Done Done Sent false
19811 6382 Acked Read 4 Done Done Sent false
19812 6383 Acked Write 8 Done Done Sent false
19813 6384 Acked Read 8 Done Done Sent false
19814 6385 Acked Write 6 Done Done Sent false
19815 6386 Acked Flush 0 Done Done Sent false
19816 6387 Acked Read 2 Done Done Sent false
19817 6388 Acked Flush 0 Done Done Sent false
19818 6389 Acked Write 1 Done Done Sent false
19819 6390 Acked Read 7 Done Done Sent false
19820 6391 Acked Write 4 Done Done Sent false
19821 6392 Acked Read 8 Done Done Sent false
19822 6393 Acked Read 7 Done Done Sent false
19823 6394 Acked Write 7 Done Done Sent false
19824 6395 Acked Flush 0 Done Done Sent false
19825 6396 Acked Write 9 Done Done Sent false
19826 6397 Acked Read 6 Done Done Sent false
19827 6398 Acked Write 8 Done Done Sent false
19828 6399 Acked Read 6 Done Done Sent false
19829 6400 Acked Read 2 Done Done Sent false
19830 6401 Acked Write 2 Done Done Sent false
19831 6402 Acked Read 1 Done Done Sent false
19832 6403 Acked Read 9 Done Done Sent false
19833 6404 Acked Write 9 Done Done Sent false
19834 6405 Acked Read 5 Done Done Sent false
19835 6406 Acked Write 8 Done Done Sent false
19836 6407 Acked Read 5 Done Done Sent false
19837 6408 Acked Read 9 Done Done Sent false
19838 6409 Acked Read 10 Done Done Sent false
19839 6410 Acked Flush 0 Done Done Sent false
19840 6411 Acked Read 5 Done Done Sent false
19841 6412 Acked Write 8 Done Done Sent false
19842 6413 Acked Write 8 Done Done Sent false
19843 6414 Acked Write 10 Done Done Sent false
19844 6415 Acked Flush 0 Done Done Sent false
19845 6416 Acked Flush 0 Done Done Sent false
19846 6417 Acked Read 4 Done Done Sent false
19847 6418 Acked Read 9 Done Done Sent false
19848 6419 Acked Read 8 Done Done Sent false
19849 6420 Acked Write 2 Done Done Sent false
19850 6421 Acked Write 2 Done Done Sent false
19851 6422 Acked Read 3 Done Done Sent false
19852 6423 Acked Read 5 Done Done Sent false
19853 6424 Acked Read 2 Done Done Sent false
19854 6425 Acked Read 9 Done Done Sent false
19855 6426 Acked Write 7 Done Done Sent false
19856 6427 Acked Read 5 Done Done Sent false
19857 6428 Acked Flush 0 Done Done Sent false
19858 6429 Acked Read 3 Done Done Sent false
19859 6430 Acked Read 6 Done Done Sent false
19860 6431 Acked Write 2 Done Done Sent false
19861 6432 Acked Read 4 Done Done Sent false
19862 STATES DS:0 DS:1 DS:2 TOTAL
19863 Sent 0 0 301 301
19864 Done 301 301 0 602
19865 Skipped 0 0 0 0
19866 Error 0 0 0 0
19867 Last Flush: 6428 6428 6131
19868 Downstairs last five completed: 6131 6130 6129 6128 6127
19869 Upstairs last five completed: 6432 6431 6430 6429 6428
19870 CLIENT: Up:0 ds:301 act:2
19871 [8830] got start action
19872 [8830] I am going to restart
19873 Starting downstairs at port 8830
19874 Make output file at "/tmp/dsc/downstairs-8830.txt"
19875 [8830][2] reports Starting
19876 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1220)
19877 [8830][2] reports Running
19878 ----------------------------------------------------------------
19879 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19880 States: Active Active Offline
19881 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19882 6132 Acked Flush 0 Done Done Sent false
19883 6133 Acked Read 4 Done Done Sent false
19884 6134 Acked Flush 0 Done Done Sent false
19885 6135 Acked Flush 0 Done Done Sent false
19886 6136 Acked Read 4 Done Done Sent false
19887 6137 Acked Read 8 Done Done Sent false
19888 6138 Acked Read 7 Done Done Sent false
19889 6139 Acked Flush 0 Done Done Sent false
19890 6140 Acked Write 9 Done Done Sent false
19891 6141 Acked Write 2 Done Done Sent false
19892 6142 Acked Flush 0 Done Done Sent false
19893 6143 Acked Write 10 Done Done Sent false
19894 6144 Acked Write 3 Done Done Sent false
19895 6145 Acked Read 3 Done Done Sent false
19896 6146 Acked Write 2 Done Done Sent false
19897 6147 Acked Flush 0 Done Done Sent false
19898 6148 Acked Write 2 Done Done Sent false
19899 6149 Acked Flush 0 Done Done Sent false
19900 6150 Acked Read 5 Done Done Sent false
19901 6151 Acked Write 1 Done Done Sent false
19902 6152 Acked Read 4 Done Done Sent false
19903 6153 Acked Write 7 Done Done Sent false
19904 6154 Acked Read 1 Done Done Sent false
19905 6155 Acked Write 4 Done Done Sent false
19906 6156 Acked Write 5 Done Done Sent false
19907 6157 Acked Read 4 Done Done Sent false
19908 6158 Acked Read 4 Done Done Sent false
19909 6159 Acked Read 4 Done Done Sent false
19910 6160 Acked Read 1 Done Done Sent false
19911 6161 Acked Write 3 Done Done Sent false
19912 6162 Acked Read 9 Done Done Sent false
19913 6163 Acked Read 7 Done Done Sent false
19914 6164 Acked Read 9 Done Done Sent false
19915 6165 Acked Read 5 Done Done Sent false
19916 6166 Acked Write 5 Done Done Sent false
19917 6167 Acked Read 5 Done Done Sent false
19918 6168 Acked Flush 0 Done Done Sent false
19919 6169 Acked Read 10 Done Done Sent false
19920 6170 Acked Read 4 Done Done Sent false
19921 6171 Acked Read 9 Done Done Sent false
19922 6172 Acked Read 2 Done Done Sent false
19923 6173 Acked Write 1 Done Done Sent false
19924 6174 Acked Flush 0 Done Done Sent false
19925 6175 Acked Read 6 Done Done Sent false
19926 6176 Acked Write 1 Done Done Sent false
19927 6177 Acked Read 7 Done Done Sent false
19928 6178 Acked Read 4 Done Done Sent false
19929 6179 Acked Read 10 Done Done Sent false
19930 6180 Acked Read 1 Done Done Sent false
19931 6181 Acked Read 5 Done Done Sent false
19932 6182 Acked Flush 0 Done Done Sent false
19933 6183 Acked Read 8 Done Done Sent false
19934 6184 Acked Write 10 Done Done Sent false
19935 6185 Acked Read 3 Done Done Sent false
19936 6186 Acked Write 3 Done Done Sent false
19937 6187 Acked Write 9 Done Done Sent false
19938 6188 Acked Write 3 Done Done Sent false
19939 6189 Acked Write 9 Done Done Sent false
19940 6190 Acked Flush 0 Done Done Sent false
19941 6191 Acked Write 5 Done Done Sent false
19942 6192 Acked Read 6 Done Done Sent false
19943 6193 Acked Write 9 Done Done Sent false
19944 6194 Acked Read 8 Done Done Sent false
19945 6195 Acked Write 6 Done Done Sent false
19946 6196 Acked Write 7 Done Done Sent false
19947 6197 Acked Write 7 Done Done Sent false
19948 6198 Acked Write 1 Done Done Sent false
19949 6199 Acked Write 9 Done Done Sent false
19950 6200 Acked Write 10 Done Done Sent false
19951 6201 Acked Write 10 Done Done Sent false
19952 6202 Acked Read 2 Done Done Sent false
19953 6203 Acked Read 1 Done Done Sent false
19954 6204 Acked Read 6 Done Done Sent false
19955 6205 Acked Write 2 Done Done Sent false
19956 6206 Acked Write 8 Done Done Sent false
19957 6207 Acked Read 7 Done Done Sent false
19958 6208 Acked Read 3 Done Done Sent false
19959 6209 Acked Write 5 Done Done Sent false
19960 6210 Acked Write 4 Done Done Sent false
19961 6211 Acked Read 10 Done Done Sent false
19962 6212 Acked Read 8 Done Done Sent false
19963 6213 Acked Write 10 Done Done Sent false
19964 6214 Acked Read 1 Done Done Sent false
19965 6215 Acked Write 1 Done Done Sent false
19966 6216 Acked Read 5 Done Done Sent false
19967 6217 Acked Write 1 Done Done Sent false
19968 6218 Acked Read 6 Done Done Sent false
19969 6219 Acked Write 10 Done Done Sent false
19970 6220 Acked Write 10 Done Done Sent false
19971 6221 Acked Read 9 Done Done Sent false
19972 6222 Acked Write 3 Done Done Sent false
19973 6223 Acked Write 10 Done Done Sent false
19974 6224 Acked Read 8 Done Done Sent false
19975 6225 Acked Read 3 Done Done Sent false
19976 6226 Acked Write 10 Done Done Sent false
19977 6227 Acked Write 1 Done Done Sent false
19978 6228 Acked Read 2 Done Done Sent false
19979 6229 Acked Write 6 Done Done Sent false
19980 6230 Acked Write 7 Done Done Sent false
19981 6231 Acked Read 6 Done Done Sent false
19982 6232 Acked Write 2 Done Done Sent false
19983 6233 Acked Write 3 Done Done Sent false
19984 6234 Acked Read 6 Done Done Sent false
19985 6235 Acked Read 1 Done Done Sent false
19986 6236 Acked Flush 0 Done Done Sent false
19987 6237 Acked Flush 0 Done Done Sent false
19988 6238 Acked Flush 0 Done Done Sent false
19989 6239 Acked Read 9 Done Done Sent false
19990 6240 Acked Write 10 Done Done Sent false
19991 6241 Acked Read 4 Done Done Sent false
19992 6242 Acked Read 2 Done Done Sent false
19993 6243 Acked Write 9 Done Done Sent false
19994 6244 Acked Write 3 Done Done Sent false
19995 6245 Acked Write 5 Done Done Sent false
19996 6246 Acked Read 4 Done Done Sent false
19997 6247 Acked Read 9 Done Done Sent false
19998 6248 Acked Read 4 Done Done Sent false
19999 6249 Acked Read 7 Done Done Sent false
20000 6250 Acked Write 6 Done Done Sent false
20001 6251 Acked Read 1 Done Done Sent false
20002 6252 Acked Read 2 Done Done Sent false
20003 6253 Acked Read 2 Done Done Sent false
20004 6254 Acked Write 2 Done Done Sent false
20005 6255 Acked Write 6 Done Done Sent false
20006 6256 Acked Write 9 Done Done Sent false
20007 6257 Acked Read 6 Done Done Sent false
20008 6258 Acked Write 4 Done Done Sent false
20009 6259 Acked Read 10 Done Done Sent false
20010 6260 Acked Write 4 Done Done Sent false
20011 6261 Acked Write 4 Done Done Sent false
20012 6262 Acked Write 9 Done Done Sent false
20013 6263 Acked Flush 0 Done Done Sent false
20014 6264 Acked Write 2 Done Done Sent false
20015 6265 Acked Write 4 Done Done Sent false
20016 6266 Acked Write 5 Done Done Sent false
20017 6267 Acked Write 6 Done Done Sent false
20018 6268 Acked Write 4 Done Done Sent false
20019 6269 Acked Read 5 Done Done Sent false
20020 6270 Acked Read 6 Done Done Sent false
20021 6271 Acked Flush 0 Done Done Sent false
20022 6272 Acked Write 8 Done Done Sent false
20023 6273 Acked Write 1 Done Done Sent false
20024 6274 Acked Read 10 Done Done Sent false
20025 6275 Acked Read 5 Done Done Sent false
20026 6276 Acked Write 8 Done Done Sent false
20027 6277 Acked Read 7 Done Done Sent false
20028 6278 Acked Read 5 Done Done Sent false
20029 6279 Acked Read 5 Done Done Sent false
20030 6280 Acked Write 6 Done Done Sent false
20031 6281 Acked Write 7 Done Done Sent false
20032 6282 Acked Read 5 Done Done Sent false
20033 6283 Acked Write 2 Done Done Sent false
20034 6284 Acked Read 2 Done Done Sent false
20035 6285 Acked Read 1 Done Done Sent false
20036 6286 Acked Read 9 Done Done Sent false
20037 6287 Acked Read 9 Done Done Sent false
20038 6288 Acked Write 1 Done Done Sent false
20039 6289 Acked Read 1 Done Done Sent false
20040 6290 Acked Write 9 Done Done Sent false
20041 6291 Acked Read 4 Done Done Sent false
20042 6292 Acked Write 8 Done Done Sent false
20043 6293 Acked Write 7 Done Done Sent false
20044 6294 Acked Read 10 Done Done Sent false
20045 6295 Acked Read 1 Done Done Sent false
20046 6296 Acked Read 3 Done Done Sent false
20047 6297 Acked Read 2 Done Done Sent false
20048 6298 Acked Write 5 Done Done Sent false
20049 6299 Acked Read 3 Done Done Sent false
20050 6300 Acked Read 7 Done Done Sent false
20051 6301 Acked Read 3 Done Done Sent false
20052 6302 Acked Read 8 Done Done Sent false
20053 6303 Acked Read 10 Done Done Sent false
20054 6304 Acked Write 2 Done Done Sent false
20055 6305 Acked Read 1 Done Done Sent false
20056 6306 Acked Write 7 Done Done Sent false
20057 6307 Acked Read 3 Done Done Sent false
20058 6308 Acked Read 9 Done Done Sent false
20059 6309 Acked Read 2 Done Done Sent false
20060 6310 Acked Read 5 Done Done Sent false
20061 6311 Acked Read 9 Done Done Sent false
20062 6312 Acked Write 3 Done Done Sent false
20063 6313 Acked Read 10 Done Done Sent false
20064 6314 Acked Read 5 Done Done Sent false
20065 6315 Acked Read 9 Done Done Sent false
20066 6316 Acked Read 1 Done Done Sent false
20067 6317 Acked Read 8 Done Done Sent false
20068 6318 Acked Write 2 Done Done Sent false
20069 6319 Acked Read 2 Done Done Sent false
20070 6320 Acked Read 1 Done Done Sent false
20071 6321 Acked Flush 0 Done Done Sent false
20072 6322 Acked Flush 0 Done Done Sent false
20073 6323 Acked Write 2 Done Done Sent false
20074 6324 Acked Write 8 Done Done Sent false
20075 6325 Acked Read 9 Done Done Sent false
20076 6326 Acked Write 4 Done Done Sent false
20077 6327 Acked Write 2 Done Done Sent false
20078 6328 Acked Read 8 Done Done Sent false
20079 6329 Acked Read 7 Done Done Sent false
20080 6330 Acked Read 6 Done Done Sent false
20081 6331 Acked Flush 0 Done Done Sent false
20082 6332 Acked Read 8 Done Done Sent false
20083 6333 Acked Write 6 Done Done Sent false
20084 6334 Acked Read 4 Done Done Sent false
20085 6335 Acked Read 3 Done Done Sent false
20086 6336 Acked Write 8 Done Done Sent false
20087 6337 Acked Read 4 Done Done Sent false
20088 6338 Acked Read 9 Done Done Sent false
20089 6339 Acked Write 2 Done Done Sent false
20090 6340 Acked Flush 0 Done Done Sent false
20091 6341 Acked Read 10 Done Done Sent false
20092 6342 Acked Read 5 Done Done Sent false
20093 6343 Acked Flush 0 Done Done Sent false
20094 6344 Acked Read 7 Done Done Sent false
20095 6345 Acked Write 9 Done Done Sent false
20096 6346 Acked Read 2 Done Done Sent false
20097 6347 Acked Write 8 Done Done Sent false
20098 6348 Acked Flush 0 Done Done Sent false
20099 6349 Acked Read 8 Done Done Sent false
20100 6350 Acked Read 5 Done Done Sent false
20101 6351 Acked Write 2 Done Done Sent false
20102 6352 Acked Read 10 Done Done Sent false
20103 6353 Acked Write 6 Done Done Sent false
20104 6354 Acked Read 8 Done Done Sent false
20105 6355 Acked Write 6 Done Done Sent false
20106 6356 Acked Write 6 Done Done Sent false
20107 6357 Acked Write 4 Done Done Sent false
20108 6358 Acked Write 9 Done Done Sent false
20109 6359 Acked Read 1 Done Done Sent false
20110 6360 Acked Read 4 Done Done Sent false
20111 6361 Acked Read 6 Done Done Sent false
20112 6362 Acked Write 9 Done Done Sent false
20113 6363 Acked Write 3 Done Done Sent false
20114 6364 Acked Write 9 Done Done Sent false
20115 6365 Acked Write 5 Done Done Sent false
20116 6366 Acked Read 8 Done Done Sent false
20117 6367 Acked Write 10 Done Done Sent false
20118 6368 Acked Write 4 Done Done Sent false
20119 6369 Acked Write 4 Done Done Sent false
20120 6370 Acked Write 10 Done Done Sent false
20121 6371 Acked Write 1 Done Done Sent false
20122 6372 Acked Read 8 Done Done Sent false
20123 6373 Acked Read 4 Done Done Sent false
20124 6374 Acked Write 9 Done Done Sent false
20125 6375 Acked Write 2 Done Done Sent false
20126 6376 Acked Flush 0 Done Done Sent false
20127 6377 Acked Read 7 Done Done Sent false
20128 6378 Acked Read 10 Done Done Sent false
20129 6379 Acked Write 9 Done Done Sent false
20130 6380 Acked Write 2 Done Done Sent false
20131 6381 Acked Write 7 Done Done Sent false
20132 6382 Acked Read 4 Done Done Sent false
20133 6383 Acked Write 8 Done Done Sent false
20134 6384 Acked Read 8 Done Done Sent false
20135 6385 Acked Write 6 Done Done Sent false
20136 6386 Acked Flush 0 Done Done Sent false
20137 6387 Acked Read 2 Done Done Sent false
20138 6388 Acked Flush 0 Done Done Sent false
20139 6389 Acked Write 1 Done Done Sent false
20140 6390 Acked Read 7 Done Done Sent false
20141 6391 Acked Write 4 Done Done Sent false
20142 6392 Acked Read 8 Done Done Sent false
20143 6393 Acked Read 7 Done Done Sent false
20144 6394 Acked Write 7 Done Done Sent false
20145 6395 Acked Flush 0 Done Done Sent false
20146 6396 Acked Write 9 Done Done Sent false
20147 6397 Acked Read 6 Done Done Sent false
20148 6398 Acked Write 8 Done Done Sent false
20149 6399 Acked Read 6 Done Done Sent false
20150 6400 Acked Read 2 Done Done Sent false
20151 6401 Acked Write 2 Done Done Sent false
20152 6402 Acked Read 1 Done Done Sent false
20153 6403 Acked Read 9 Done Done Sent false
20154 6404 Acked Write 9 Done Done Sent false
20155 6405 Acked Read 5 Done Done Sent false
20156 6406 Acked Write 8 Done Done Sent false
20157 6407 Acked Read 5 Done Done Sent false
20158 6408 Acked Read 9 Done Done Sent false
20159 6409 Acked Read 10 Done Done Sent false
20160 6410 Acked Flush 0 Done Done Sent false
20161 6411 Acked Read 5 Done Done Sent false
20162 6412 Acked Write 8 Done Done Sent false
20163 6413 Acked Write 8 Done Done Sent false
20164 6414 Acked Write 10 Done Done Sent false
20165 6415 Acked Flush 0 Done Done Sent false
20166 6416 Acked Flush 0 Done Done Sent false
20167 6417 Acked Read 4 Done Done Sent false
20168 6418 Acked Read 9 Done Done Sent false
20169 6419 Acked Read 8 Done Done Sent false
20170 6420 Acked Write 2 Done Done Sent false
20171 6421 Acked Write 2 Done Done Sent false
20172 6422 Acked Read 3 Done Done Sent false
20173 6423 Acked Read 5 Done Done Sent false
20174 6424 Acked Read 2 Done Done Sent false
20175 6425 Acked Read 9 Done Done Sent false
20176 6426 Acked Write 7 Done Done Sent false
20177 6427 Acked Read 5 Done Done Sent false
20178 6428 Acked Flush 0 Done Done Sent false
20179 6429 Acked Read 3 Done Done Sent false
20180 6430 Acked Read 6 Done Done Sent false
20181 6431 Acked Write 2 Done Done Sent false
20182 6432 Acked Read 4 Done Done Sent false
20183 6433 Acked Flush 0 Done Done Sent false
20184 STATES DS:0 DS:1 DS:2 TOTAL
20185 Sent 0 0 302 302
20186 Done 302 302 0 604
20187 Skipped 0 0 0 0
20188 Error 0 0 0 0
20189 Last Flush: 6433 6433 6131
20190 Downstairs last five completed: 6131 6130 6129 6128 6127
20191 Upstairs last five completed: 6433 6432 6431 6430 6429
20192 CLIENT: Up:0 ds:302 act:2
201932025-01-13T15:56:01.325ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
201942025-01-13T15:56:01.325ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54625) = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
201952025-01-13T15:56:01.326ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
201962025-01-13T15:56:01.327ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
201972025-01-13T15:56:01.327ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
201982025-01-13T15:56:01.327ZINFOcrucible: send last flush ID to this DS: 6131 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
201992025-01-13T15:56:01.327ZINFOcrucible: Replied this last flush ID: 6131 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
202002025-01-13T15:56:01.327ZINFOcrucible: [2] client re-new 302 jobs since flush 6131 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
202012025-01-13T15:56:01.328ZINFOcrucible: [2] Replayed 302 jobs since flush: 6131 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
20202 ----------------------------------------------------------------
20203 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
20204 Downstairs last five completed: 6433 6432 6431 6430 6429
20205 Upstairs last five completed: 6433 6432 6431 6430 6429
20206 CLIENT: Up:0 ds:0 act:3
20207 Replay: All jobs finished
20208 Main task has work to do, go find it
20209 got dsc Stop(1)
20210 stop 1
20211 [8820] Got stop action so:false kr:false
20212 Jan 13 15:56:03.570 INFO request completed, latency_us: 376, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 63a65b14-7dd7-4e34-a2c3-ac9a164aa758, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
202132025-01-13T15:56:03.571ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
202142025-01-13T15:56:03.571ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
202152025-01-13T15:56:03.571ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
20216 Jan 13 15:56:03.571 INFO request completed, latency_us: 138, response_code: 200, uri: /state/cid/1, method: GET, req_id: 69ebad0c-7784-4109-b25b-9f7bbce81157, remote_addr{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-13T15:56:03.571652738Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
20217 : 127.0.0.1:52422, local_addr: 127.0.0.1:9998
20218 [8820] Exited with: ExitStatus(unix_wait_status(9))
20219 [8820][1] reports Exit
20220 Jan 13 15:56:07.573 INFO request completed, latency_us: 329, response_code: 200, uri: /state/cid/1, method: GET, req_id: a2aa39af-948c-4112-97cc-785a909cc11c, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
20221 001/300 Write block 2687 len 8192 data: 4 4
20222 002/300 Write block 2589 len 12288 data: 3 6 8
20223 003/300 Read block 3250 len 8192
20224 004/300 Read block 1016 len 8192
20225 005/300 Flush
20226 006/300 Write block 2231 len 12288 data: 6 4 4
20227 007/300 Flush
20228 008/300 Read block 1904 len 32768
20229 009/300 Read block 3559 len 12288
20230 010/300 Read block 495 len 28672
20231 011/300 Write block 3701 len 28672 data: 3 5 4 4 4 7 7
20232 012/300 Read block 2270 len 24576
20233 013/300 Read block 4858 len 40960
20234 014/300 Write block 2559 len 8192 data: 4 4
20235 015/300 Write block 474 len 36864 data: 5 5 5 6 6 6 4 4 3
20236 016/300 Read block 1703 len 32768
20237 017/300 Read block 1382 len 16384
20238 018/300 Read block 3494 len 28672
20239 019/300 Write block 1464 len 12288 data: 2 2 2
20240 020/300 Write block 1805 len 20480 data: 3 3 3 3 3
20241 021/300 Read block 2657 len 12288
20242 022/300 Write block 3596 len 28672 data: 4 3 4 4 3 4 4
20243 023/300 Read block 3347 len 8192
20244 024/300 Read block 3713 len 32768
20245 025/300 Read block 1419 len 24576
20246 026/300 Write block 4306 len 24576 data: 7 6 5 4 3 3
20247 027/300 Write block 2548 len 40960 data: 7 7 6 6 4 6 6 6 5 5
20248 028/300 Read block 3826 len 28672
20249 029/300 Flush
20250 030/300 Write block 3471 len 24576 data: 5 4 4 5 5 5
20251 031/300 Write block 4784 len 8192 data: 7 6
20252 032/300 Write block 1942 len 40960 data: 3 3 4 3 3 3 4 2 3 3
20253 033/300 Read block 4618 len 4096
20254 034/300 Read block 1726 len 20480
20255 035/300 Read block 1191 len 4096
20256 036/300 Read block 3551 len 40960
20257 037/300 Flush
20258 038/300 Flush
20259 039/300 Flush
20260 040/300 Write block 3088 len 36864 data: 5 5 3 3 3 4 4 5 5
20261 041/300 Write block 2548 len 24576 data: 8 8 7 7 5 7
20262 042/300 Write block 2078 len 32768 data: 6 6 6 5 6 8 6 5
20263 043/300 Read block 440 len 32768
20264 044/300 Read block 2857 len 40960
20265 045/300 Read block 1774 len 20480
20266 046/300 Read block 4141 len 8192
20267 047/300 Read block 1054 len 8192
20268 048/300 Write block 3210 len 32768 data: 5 5 4 4 4 3 4 4
20269 049/300 Write block 1903 len 36864 data: 2 2 2 3 3 3 3 2 2
20270 050/300 Write block 2916 len 20480 data: 4 4 4 3 3
20271 051/300 Flush
20272 052/300 Read block 1810 len 24576
20273 053/300 Write block 1472 len 40960 data: 2 3 3 3 3 2 2 2 2 2
20274 054/300 Write block 1190 len 12288 data: 4 4 3
20275 055/300 Flush
20276 056/300 Flush
20277 057/300 Write block 2551 len 40960 data: 8 6 8 7 7 6 6 5 5 5
20278 058/300 Write block 1018 len 36864 data: 4 4 6 6 8 5 6 5 6
20279 059/300 Write block 664 len 8192 data: 11 10
20280 060/300 Write block 3729 len 12288 data: 8 4 3
20281 061/300 Read block 595 len 12288
20282 062/300 Read block 4115 len 12288
20283 063/300 Read block 2997 len 32768
20284 064/300 Write block 4421 len 16384 data: 2 2 2 3
20285 065/300 Flush
20286 066/300 Read block 4109 len 16384
20287 067/300 Read block 2348 len 12288
20288 068/300 Flush
20289 069/300 Write block 3990 len 12288 data: 4 5 5
20290 070/300 Write block 3676 len 20480 data: 3 2 2 2 2
20291 071/300 Write block 4782 len 4096 data: 6
20292 072/300 Read block 3587 len 28672
20293 073/300 Read block 420 len 24576
20294 074/300 Read block 1771 len 20480
20295 075/300 Write block 1660 len 28672 data: 4 4 4 4 3 4 4
20296 076/300 Read block 1560 len 20480
20297 077/300 Read block 190 len 4096
20298 078/300 Read block 4793 len 40960
20299 079/300 Read block 1382 len 4096
20300 080/300 Flush
20301 081/300 Read block 1652 len 16384
20302 082/300 Read block 3573 len 40960
20303 083/300 Write block 1305 len 20480 data: 3 3 3 2 2
20304 084/300 Write block 4073 len 8192 data: 5 5
20305 085/300 Flush
20306 086/300 Write block 882 len 12288 data: 5 4 4
20307 087/300 Write block 1139 len 12288 data: 4 4 5
20308 088/300 Write block 4694 len 8192 data: 5 5
20309 089/300 Flush
20310 090/300 Write block 3318 len 28672 data: 4 4 5 6 5 4 5
20311 091/300 Read block 1235 len 8192
20312 092/300 Read block 2927 len 28672
20313 093/300 Write block 1989 len 40960 data: 2 2 3 3 4 4 3 3 3 3
20314 094/300 Read block 2844 len 16384
20315 095/300 Read block 1734 len 28672
20316 096/300 Write block 4857 len 8192 data: 3 4
20317 097/300 Flush
20318 098/300 Write block 3225 len 40960 data: 6 9 9 7 8 8 7 7 6 5
20319 099/300 Write block 91 len 20480 data: 2 3 3 2 2
20320 100/300 Read block 567 len 16384
20321 101/300 Write block 1811 len 20480 data: 4 5 6 6 6
20322 102/300 Write block 4567 len 36864 data: 6 6 6 4 3 4 4 4 4
20323 103/300 Flush
20324 104/300 Read block 3173 len 12288
20325 105/300 Write block 3613 len 16384 data: 3 4 4 4
20326 106/300 Write block 606 len 20480 data: 3 3 4 3 3
20327 107/300 Write block 375 len 20480 data: 9 9 9 9 7
20328 108/300 Read block 206 len 12288
20329 109/300 Read block 4615 len 4096
20330 110/300 Read block 4604 len 40960
20331 111/300 Read block 4086 len 36864
20332 112/300 Write block 4847 len 20480 data: 4 2 5 4 5
20333 113/300 Read block 3182 len 20480
20334 114/300 Read block 1772 len 4096
20335 115/300 Write block 4748 len 20480 data: 3 6 4 5 5
20336 116/300 Read block 523 len 32768
20337 117/300 Read block 3834 len 20480
20338 118/300 Read block 635 len 20480
20339 119/300 Write block 4181 len 36864 data: 5 4 4 4 6 5 6 6 6
20340 120/300 Read block 4304 len 8192
20341 121/300 Read block 2453 len 32768
20342 122/300 Write block 3142 len 8192 data: 5 5
20343 123/300 Read block 4746 len 16384
20344 124/300 Flush
20345 125/300 Write block 649 len 20480 data: 4 4 4 4 4
20346 126/300 Flush
20347 127/300 Read block 910 len 20480
20348 128/300 Read block 2640 len 36864
20349 129/300 Read block 335 len 12288
20350 130/300 Read block 1308 len 36864
20351 131/300 Read block 659 len 36864
20352 132/300 Write block 4379 len 16384 data: 5 3 3 3
20353 133/300 Read block 4423 len 32768
20354 134/300 Write block 3177 len 8192 data: 4 4
20355 135/300 Write block 1105 len 28672 data: 3 3 4 4 4 4 4
20356 136/300 Flush
20357 137/300 Read block 2857 len 20480
20358 138/300 Read block 29 len 28672
20359 139/300 Write block 3543 len 36864 data: 2 2 2 2 3 3 2 2 2
20360 140/300 Read block 3547 len 4096
20361 141/300 Read block 1872 len 32768
20362 142/300 Read block 1299 len 24576
20363 143/300 Write block 4725 len 40960 data: 3 3 3 4 5 4 5 7 6 7
20364 144/300 Write block 2621 len 4096 data: 3
20365 145/300 Write block 2043 len 24576 data: 4 3 4 5 5 6
20366 146/300 Flush
20367 147/300 Read block 2892 len 32768
20368 148/300 Write block 2019 len 40960 data: 6 5 5 7 8 9 8 10 10 9
20369 149/300 Read block 1185 len 12288
20370 150/300 Write block 1826 len 32768 data: 5 6 5 4 3 4 5 5
20371 151/300 Flush
20372 152/300 Read block 1704 len 8192
20373 153/300 Write block 961 len 36864 data: 7 7 6 4 4 3 4 4 2
20374 154/300 Write block 667 len 24576 data: 8 9 9 7 4 3
20375 155/300 Read block 1615 len 12288
20376 156/300 Read block 2626 len 36864
20377 157/300 Write block 4279 len 40960 data: 3 3 4 4 3 5 5 5 5 4
20378 158/300 Read block 3128 len 20480
20379 159/300 Read block 2071 len 36864
20380 160/300 Read block 3011 len 4096
20381 161/300 Read block 2008 len 16384
20382 162/300 Write block 1016 len 4096 data: 3
20383 163/300 Write block 4341 len 24576 data: 2 2 4 4 5 4
20384 164/300 Write block 3963 len 4096 data: 5
20385 165/300 Flush
20386 166/300 Write block 2609 len 20480 data: 5 5 5 4 3
20387 167/300 Read block 633 len 24576
20388 168/300 Write block 1090 len 16384 data: 5 4 3 3
20389 169/300 Read block 4257 len 32768
20390 170/300 Write block 1075 len 4096 data: 2
20391 171/300 Write block 1142 len 32768 data: 6 7 7 6 5 6 6 5
20392 172/300 Flush
20393 173/300 Write block 823 len 8192 data: 4 5
20394 174/300 Write block 946 len 20480 data: 9 8 5 6 5
20395 175/300 Read block 446 len 40960
20396 176/300 Read block 4088 len 32768
20397 177/300 Write block 4501 len 16384 data: 7 8 9 7
20398 178/300 Read block 2507 len 36864
20399 179/300 Read block 1012 len 20480
20400 180/300 Write block 1494 len 28672 data: 7 6 5 5 8 8 7
20401 181/300 Write block 4333 len 20480 data: 4 4 4 4 4
20402 182/300 Write block 776 len 8192 data: 3 5
20403 183/300 Flush
20404 184/300 Read block 3154 len 16384
20405 185/300 Write block 3263 len 36864 data: 6 7 7 5 6 6 6 7 6
20406 186/300 Read block 993 len 20480
20407 187/300 Write block 4566 len 12288 data: 7 7 7
20408 188/300 Write block 3005 len 12288 data: 6 6 6
20409 189/300 Read block 1283 len 40960
20410 190/300 Write block 1949 len 8192 data: 3 4
20411 191/300 Write block 808 len 20480 data: 4 4 4 4 2
20412 192/300 Read block 1254 len 4096
20413 193/300 Write block 3011 len 40960 data: 7 5 4 4 4 4 2 2 4 7
20414 194/300 Read block 881 len 40960
20415 195/300 Read block 1086 len 8192
20416 196/300 Read block 4493 len 36864
20417 197/300 Read block 3904 len 36864
20418 198/300 Read block 2582 len 32768
20419 199/300 Write block 1078 len 24576 data: 5 5 6 5 6 6
20420 200/300 Write block 3035 len 24576 data: 5 4 4 3 3 4
20421 201/300 Write block 921 len 8192 data: 3 4
20422 202/300 Write block 356 len 12288 data: 6 5 4
20423 203/300 Write block 4503 len 8192 data: 10 8
20424 204/300 Flush
20425 205/300 Write block 1928 len 8192 data: 2 2
20426 206/300 Read block 348 len 32768
20427 207/300 Read block 1087 len 16384
20428 208/300 Write block 4960 len 36864 data: 6 4 4 3 3 3 4 5 6
20429 209/300 Read block 3643 len 24576
20430 210/300 Read block 3549 len 20480
20431 211/300 Read block 4756 len 40960
20432 212/300 Read block 3280 len 12288
20433 213/300 Write block 1039 len 24576 data: 6 5 5 7 7 8
20434 214/300 Write block 974 len 4096 data: 4
20435 215/300 Read block 4735 len 20480
20436 216/300 Read block 1818 len 28672
20437 217/300 Write block 4216 len 32768 data: 8 8 6 6 6 6 4 6
20438 218/300 Write block 2934 len 4096 data: 3
20439 219/300 Read block 1112 len 32768
20440 220/300 Flush
20441 221/300 Read block 4179 len 8192
20442 222/300 Read block 3082 len 36864
20443 223/300 Flush
20444 224/300 Write block 1026 len 32768 data: 7 6 5 4 5 3 3 3
20445 225/300 Read block 4689 len 36864
20446 226/300 Read block 2243 len 20480
20447 227/300 Read block 686 len 12288
20448 228/300 Read block 792 len 40960
20449 229/300 Read block 1216 len 24576
20450 230/300 Flush
20451 231/300 Read block 2205 len 8192
20452 232/300 Read block 893 len 36864
20453 233/300 Read block 3021 len 24576
20454 234/300 Read block 1674 len 20480
20455 235/300 Flush
20456 236/300 Read block 1783 len 28672
20457 237/300 Write block 327 len 16384 data: 3 4 4 4
20458 238/300 Read block 859 len 24576
20459 239/300 Write block 2344 len 12288 data: 7 7 6
20460 240/300 Read block 1953 len 32768
20461 241/300 Read block 1911 len 36864
20462 242/300 Flush
20463 243/300 Write block 1344 len 8192 data: 2 2
20464 244/300 Write block 1854 len 20480 data: 5 6 7 7 5
20465 245/300 Write block 2439 len 16384 data: 6 5 5 5
20466 246/300 Write block 4531 len 12288 data: 6 7 5
20467 247/300 Read block 2879 len 16384
20468 248/300 Write block 4384 len 20480 data: 4 4 4 4 4
20469 249/300 Flush
20470 250/300 Read block 2669 len 28672
20471 251/300 Write block 1330 len 8192 data: 4 3
20472 252/300 Write block 2397 len 16384 data: 5 5 4 4
20473 253/300 Read block 4389 len 12288
20474 254/300 Read block 2969 len 36864
20475 255/300 Read block 4084 len 32768
20476 256/300 Write block 3950 len 40960 data: 3 2 3 4 4 3 4 4 4 4
20477 257/300 Write block 3851 len 20480 data: 4 4 4 6 5
20478 258/300 Write block 2150 len 4096 data: 7
20479 259/300 Write block 1965 len 28672 data: 4 6 6 7 7 6 7
20480 260/300 Write block 3241 len 40960 data: 3 3 3 3 2 2 2 2 2 2
20481 261/300 Write block 3639 len 36864 data: 4 5 5 5 5 6 4 5 4
20482 262/300 Write block 962 len 36864 data: 8 7 5 5 4 5 5 3 2
20483 263/300 Write block 2802 len 36864 data: 4 5 4 4 4 4 4 4 6
20484 264/300 Write block 4404 len 20480 data: 4 3 4 4 4
20485 265/300 Read block 4062 len 12288
20486 266/300 Read block 4387 len 40960
20487 267/300 Read block 2602 len 8192
20488 268/300 Read block 1954 len 4096
20489 269/300 Flush
20490 270/300 Read block 4369 len 8192
20491 271/300 Write block 3269 len 16384 data: 7 8 7 7
20492 272/300 Write block 41 len 20480 data: 2 2 3 3 3
20493 273/300 Read block 4815 len 32768
20494 274/300 Read block 1645 len 32768
20495 275/300 Write block 4860 len 8192 data: 6 5
20496 276/300 Write block 2791 len 8192 data: 5 5
20497 277/300 Read block 4748 len 4096
20498 278/300 Write block 148 len 36864 data: 7 7 7 6 6 4 5 6 5
20499 279/300 Flush
20500 280/300 Read block 3090 len 16384
20501 281/300 Write block 1722 len 24576 data: 2 3 3 3 4 4
20502 282/300 Write block 4121 len 36864 data: 4 3 3 3 3 3 3 3 3
20503 283/300 Read block 3635 len 4096
20504 284/300 Write block 1966 len 24576 data: 7 7 8 8 7 8
20505 285/300 Write block 3308 len 36864 data: 3 3 3 3 3 3 2 2 2
20506 286/300 Read block 1961 len 4096
20507 287/300 Read block 3875 len 16384
20508 288/300 Read block 3816 len 24576
20509 289/300 Write block 2988 len 20480 data: 3 3 3 3 5
20510 290/300 Flush
20511 291/300 Read block 4078 len 36864
20512 292/300 Write block 2405 len 20480 data: 3 3 3 2 3
20513 293/300 Read block 4638 len 32768
20514 294/300 Read block 2592 len 32768
20515 295/300 Write block 383 len 32768 data: 4 4 4 5 5 4 4 4
20516 296/300 Write block 2652 len 12288 data: 6 6 6
20517 297/300 Read block 877 len 8192
20518 298/300 Read block 4738 len 20480
20519 299/300 Read block 3421 len 28672
20520 300/300 Read block 2205 len 40960
20521 301/300 Read block 4959 len 16384
20522 Main task has work to do, go find it
20523 got dsc Start(1)
20524 start 1
20525 Jan 13 15:56:07.809 INFO request completed, latency_us: 210, response_code: 204, uri: /start/cid/1, method: POST, req_id: 78733431-ec21-422d-89da-cf0ce690af7a, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
20526 Replay: started 1, returned:Ok(())
20527 ----------------------------------------------------------------
20528 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
20529 States: Active Offline Active
20530 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20531 6434 Acked Write 2 Done Sent Done false
20532 6435 Acked Write 3 Done Sent Done false
20533 6436 Acked Read 2 Done Sent Done false
20534 6437 Acked Read 2 Done Sent Done false
20535 6438 Acked Flush 0 Done Sent Done false
20536 6439 Acked Write 3 Done Sent Done false
20537 6440 Acked Flush 0 Done Sent Done false
20538 6441 Acked Read 8 Done Sent Done false
20539 6442 Acked Read 3 Done Sent Done false
20540 6443 Acked Read 7 Done Sent Done false
20541 6444 Acked Write 7 Done Sent Done false
20542 6445 Acked Read 6 Done Sent Done false
20543 6446 Acked Read 10 Done Sent Done false
20544 6447 Acked Write 2 Done Sent Done false
20545 6448 Acked Write 9 Done Sent Done false
20546 6449 Acked Read 8 Done Sent Done false
20547 6450 Acked Read 4 Done Sent Done false
20548 6451 Acked Read 7 Done Sent Done false
20549 6452 Acked Write 3 Done Sent Done false
20550 6453 Acked Write 5 Done Sent Done false
20551 6454 Acked Read 3 Done Sent Done false
20552 6455 Acked Write 7 Done Sent Done false
20553 6456 Acked Read 2 Done Sent Done false
20554 6457 Acked Read 8 Done Sent Done false
20555 6458 Acked Read 6 Done Sent Done false
20556 6459 Acked Write 6 Done Sent Done false
20557 6460 Acked Write 10 Done Sent Done false
20558 6461 Acked Read 7 Done Sent Done false
20559 6462 Acked Flush 0 Done Sent Done false
20560 6463 Acked Write 6 Done Sent Done false
20561 6464 Acked Write 2 Done Sent Done false
20562 6465 Acked Write 10 Done Sent Done false
20563 6466 Acked Read 1 Done Sent Done false
20564 6467 Acked Read 5 Done Sent Done false
20565 6468 Acked Read 1 Done Sent Done false
20566 6469 Acked Read 10 Done Sent Done false
20567 6470 Acked Flush 0 Done Sent Done false
20568 6471 Acked Flush 0 Done Sent Done false
20569 6472 Acked Flush 0 Done Sent Done false
20570 6473 Acked Write 9 Done Sent Done false
20571 6474 Acked Write 6 Done Sent Done false
20572 6475 Acked Write 8 Done Sent Done false
20573 6476 Acked Read 8 Done Sent Done false
20574 6477 Acked Read 10 Done Sent Done false
20575 6478 Acked Read 5 Done Sent Done false
20576 6479 Acked Read 2 Done Sent Done false
20577 6480 Acked Read 2 Done Sent Done false
20578 6481 Acked Write 8 Done Sent Done false
20579 6482 Acked Write 9 Done Sent Done false
20580 6483 Acked Write 5 Done Sent Done false
20581 6484 Acked Flush 0 Done Sent Done false
20582 6485 Acked Read 6 Done Sent Done false
20583 6486 Acked Write 10 Done Sent Done false
20584 6487 Acked Write 3 Done Sent Done false
20585 6488 Acked Flush 0 Done Sent Done false
20586 6489 Acked Flush 0 Done Sent Done false
20587 6490 Acked Write 10 Done Sent Done false
20588 6491 Acked Write 9 Done Sent Done false
20589 6492 Acked Write 2 Done Sent Done false
20590 6493 Acked Write 3 Done Sent Done false
20591 6494 Acked Read 3 Done Sent Done false
20592 6495 Acked Read 3 Done Sent Done false
20593 6496 Acked Read 8 Done Sent Done false
20594 6497 Acked Write 4 Done Sent Done false
20595 6498 Acked Flush 0 Done Sent Done false
20596 6499 Acked Read 4 Done Sent Done false
20597 6500 Acked Read 3 Done Sent Done false
20598 6501 Acked Flush 0 Done Sent Done false
20599 6502 Acked Write 3 Done Sent Done false
20600 6503 Acked Write 5 Done Sent Done false
20601 6504 Acked Write 1 Done Sent Done false
20602 6505 Acked Read 7 Done Sent Done false
20603 6506 Acked Read 6 Done Sent Done false
20604 6507 Acked Read 5 Done Sent Done false
20605 6508 Acked Write 7 Done Sent Done false
20606 6509 Acked Read 5 Done Sent Done false
20607 6510 Acked Read 1 Done Sent Done false
20608 6511 Acked Read 10 Done Sent Done false
20609 6512 Acked Read 1 Done Sent Done false
20610 6513 Acked Flush 0 Done Sent Done false
20611 6514 Acked Read 4 Done Sent Done false
20612 6515 Acked Read 10 Done Sent Done false
20613 6516 Acked Write 5 Done Sent Done false
20614 6517 Acked Write 2 Done Sent Done false
20615 6518 Acked Flush 0 Done Sent Done false
20616 6519 Acked Write 3 Done Sent Done false
20617 6520 Acked Write 3 Done Sent Done false
20618 6521 Acked Write 2 Done Sent Done false
20619 6522 Acked Flush 0 Done Sent Done false
20620 6523 Acked Write 7 Done Sent Done false
20621 6524 Acked Read 2 Done Sent Done false
20622 6525 Acked Read 7 Done Sent Done false
20623 6526 Acked Write 10 Done Sent Done false
20624 6527 Acked Read 4 Done Sent Done false
20625 6528 Acked Read 7 Done Sent Done false
20626 6529 Acked Write 2 Done Sent Done false
20627 6530 Acked Flush 0 Done Sent Done false
20628 6531 Acked Write 10 Done Sent Done false
20629 6532 Acked Write 5 Done Sent Done false
20630 6533 Acked Read 4 Done Sent Done false
20631 6534 Acked Write 5 Done Sent Done false
20632 6535 Acked Write 9 Done Sent Done false
20633 6536 Acked Flush 0 Done Sent Done false
20634 6537 Acked Read 3 Done Sent Done false
20635 6538 Acked Write 4 Done Sent Done false
20636 6539 Acked Write 5 Done Sent Done false
20637 6540 Acked Write 5 Done Sent Done false
20638 6541 Acked Read 3 Done Sent Done false
20639 6542 Acked Read 1 Done Sent Done false
20640 6543 Acked Read 10 Done Sent Done false
20641 6544 Acked Read 9 Done Sent Done false
20642 6545 Acked Write 5 Done Sent Done false
20643 6546 Acked Read 5 Done Sent Done false
20644 6547 Acked Read 1 Done Sent Done false
20645 6548 Acked Write 5 Done Sent Done false
20646 6549 Acked Read 8 Done Sent Done false
20647 6550 Acked Read 5 Done Sent Done false
20648 6551 Acked Read 5 Done Sent Done false
20649 6552 Acked Write 9 Done Sent Done false
20650 6553 Acked Read 2 Done Sent Done false
20651 6554 Acked Read 8 Done Sent Done false
20652 6555 Acked Write 2 Done Sent Done false
20653 6556 Acked Read 4 Done Sent Done false
20654 6557 Acked Flush 0 Done Sent Done false
20655 6558 Acked Write 5 Done Sent Done false
20656 6559 Acked Flush 0 Done Sent Done false
20657 6560 Acked Read 5 Done Sent Done false
20658 6561 Acked Read 9 Done Sent Done false
20659 6562 Acked Read 3 Done Sent Done false
20660 6563 Acked Read 9 Done Sent Done false
20661 6564 Acked Read 9 Done Sent Done false
20662 6565 Acked Write 4 Done Sent Done false
20663 6566 Acked Read 8 Done Sent Done false
20664 6567 Acked Write 2 Done Sent Done false
20665 6568 Acked Write 7 Done Sent Done false
20666 6569 Acked Flush 0 Done Sent Done false
20667 6570 Acked Read 5 Done Sent Done false
20668 6571 Acked Read 7 Done Sent Done false
20669 6572 Acked Write 9 Done Sent Done false
20670 6573 Acked Read 1 Done Sent Done false
20671 6574 Acked Read 8 Done Sent Done false
20672 6575 Acked Read 6 Done Sent Done false
20673 6576 Acked Write 10 Done Sent Done false
20674 6577 Acked Write 1 Done Sent Done false
20675 6578 Acked Write 6 Done Sent Done false
20676 6579 Acked Flush 0 Done Sent Done false
20677 6580 Acked Read 8 Done Sent Done false
20678 6581 Acked Write 10 Done Sent Done false
20679 6582 Acked Read 3 Done Sent Done false
20680 6583 Acked Write 8 Done Sent Done false
20681 6584 Acked Flush 0 Done Sent Done false
20682 6585 Acked Read 2 Done Sent Done false
20683 6586 Acked Write 9 Done Sent Done false
20684 6587 Acked Write 6 Done Sent Done false
20685 6588 Acked Read 3 Done Sent Done false
20686 6589 Acked Read 9 Done Sent Done false
20687 6590 Acked Write 10 Done Sent Done false
20688 6591 Acked Read 5 Done Sent Done false
20689 6592 Acked Read 9 Done Sent Done false
20690 6593 Acked Read 1 Done Sent Done false
20691 6594 Acked Read 4 Done Sent Done false
20692 6595 Acked Write 1 Done Sent Done false
20693 6596 Acked Write 6 Done Sent Done false
20694 6597 Acked Write 1 Done Sent Done false
20695 6598 Acked Flush 0 Done Sent Done false
20696 6599 Acked Write 5 Done Sent Done false
20697 6600 Acked Read 6 Done Sent Done false
20698 6601 Acked Write 4 Done Sent Done false
20699 6602 Acked Read 8 Done Sent Done false
20700 6603 Acked Write 1 Done Sent Done false
20701 6604 Acked Write 8 Done Sent Done false
20702 6605 Acked Flush 0 Done Sent Done false
20703 6606 Acked Write 2 Done Sent Done false
20704 6607 Acked Write 5 Done Sent Done false
20705 6608 Acked Read 10 Done Sent Done false
20706 6609 Acked Read 8 Done Sent Done false
20707 6610 Acked Write 4 Done Sent Done false
20708 6611 Acked Read 9 Done Sent Done false
20709 6612 Acked Read 5 Done Sent Done false
20710 6613 Acked Write 7 Done Sent Done false
20711 6614 Acked Write 5 Done Sent Done false
20712 6615 Acked Write 2 Done Sent Done false
20713 6616 Acked Flush 0 Done Sent Done false
20714 6617 Acked Read 4 Done Sent Done false
20715 6618 Acked Write 9 Done Sent Done false
20716 6619 Acked Read 5 Done Sent Done false
20717 6620 Acked Write 3 Done Sent Done false
20718 6621 Acked Write 3 Done Sent Done false
20719 6622 Acked Read 10 Done Sent Done false
20720 6623 Acked Write 2 Done Sent Done false
20721 6624 Acked Write 5 Done Sent Done false
20722 6625 Acked Read 1 Done Sent Done false
20723 6626 Acked Write 10 Done Sent Done false
20724 6627 Acked Read 10 Done Sent Done false
20725 6628 Acked Read 2 Done Sent Done false
20726 6629 Acked Read 9 Done Sent Done false
20727 6630 Acked Read 9 Done Sent Done false
20728 6631 Acked Read 8 Done Sent Done false
20729 6632 Acked Write 6 Done Sent Done false
20730 6633 Acked Write 6 Done Sent Done false
20731 6634 Acked Write 2 Done Sent Done false
20732 6635 Acked Write 3 Done Sent Done false
20733 6636 Acked Write 2 Done Sent Done false
20734 6637 Acked Flush 0 Done Sent Done false
20735 6638 Acked Write 2 Done Sent Done false
20736 6639 Acked Read 8 Done Sent Done false
20737 6640 Acked Read 4 Done Sent Done false
20738 6641 Acked Write 9 Done Sent Done false
20739 6642 Acked Read 6 Done Sent Done false
20740 6643 Acked Read 5 Done Sent Done false
20741 6644 Acked Read 10 Done Sent Done false
20742 6645 Acked Read 3 Done Sent Done false
20743 6646 Acked Write 6 Done Sent Done false
20744 6647 Acked Write 1 Done Sent Done false
20745 6648 Acked Read 5 Done Sent Done false
20746 6649 Acked Read 7 Done Sent Done false
20747 6650 Acked Write 8 Done Sent Done false
20748 6651 Acked Write 1 Done Sent Done false
20749 6652 Acked Read 8 Done Sent Done false
20750 6653 Acked Flush 0 Done Sent Done false
20751 6654 Acked Read 2 Done Sent Done false
20752 6655 Acked Read 9 Done Sent Done false
20753 6656 Acked Flush 0 Done Sent Done false
20754 6657 Acked Write 8 Done Sent Done false
20755 6658 Acked Read 9 Done Sent Done false
20756 6659 Acked Read 5 Done Sent Done false
20757 6660 Acked Read 3 Done Sent Done false
20758 6661 Acked Read 10 Done Sent Done false
20759 6662 Acked Read 6 Done Sent Done false
20760 6663 Acked Flush 0 Done Sent Done false
20761 6664 Acked Read 2 Done Sent Done false
20762 6665 Acked Read 9 Done Sent Done false
20763 6666 Acked Read 6 Done Sent Done false
20764 6667 Acked Read 5 Done Sent Done false
20765 6668 Acked Flush 0 Done Sent Done false
20766 6669 Acked Read 7 Done Sent Done false
20767 6670 Acked Write 4 Done Sent Done false
20768 6671 Acked Read 6 Done Sent Done false
20769 6672 Acked Write 3 Done Sent Done false
20770 6673 Acked Read 8 Done Sent Done false
20771 6674 Acked Read 9 Done Sent Done false
20772 6675 Acked Flush 0 Done Sent Done false
20773 6676 Acked Write 2 Done Sent Done false
20774 6677 Acked Write 5 Done Sent Done false
20775 6678 Acked Write 4 Done Sent Done false
20776 6679 Acked Write 3 Done Sent Done false
20777 6680 Acked Read 4 Done Sent Done false
20778 6681 Acked Write 5 Done Sent Done false
20779 6682 Acked Flush 0 Done Sent Done false
20780 6683 Acked Read 7 Done Sent Done false
20781 6684 Acked Write 2 Done Sent Done false
20782 6685 Acked Write 4 Done Sent Done false
20783 6686 Acked Read 3 Done Sent Done false
20784 6687 Acked Read 9 Done Sent Done false
20785 6688 Acked Read 8 Done Sent Done false
20786 6689 Acked Write 10 Done Sent Done false
20787 6690 Acked Write 5 Done Sent Done false
20788 6691 Acked Write 1 Done Sent Done false
20789 6692 Acked Write 7 Done Sent Done false
20790 6693 Acked Write 10 Done Sent Done false
20791 6694 Acked Write 9 Done Sent Done false
20792 6695 Acked Write 9 Done Sent Done false
20793 6696 Acked Write 9 Done Sent Done false
20794 6697 Acked Write 5 Done Sent Done false
20795 6698 Acked Read 3 Done Sent Done false
20796 6699 Acked Read 10 Done Sent Done false
20797 6700 Acked Read 2 Done Sent Done false
20798 6701 Acked Read 1 Done Sent Done false
20799 6702 Acked Flush 0 Done Sent Done false
20800 6703 Acked Read 2 Done Sent Done false
20801 6704 Acked Write 4 Done Sent Done false
20802 6705 Acked Write 5 Done Sent Done false
20803 6706 Acked Read 8 Done Sent Done false
20804 6707 Acked Read 8 Done Sent Done false
20805 6708 Acked Write 2 Done Sent Done false
20806 6709 Acked Write 2 Done Sent Done false
20807 6710 Acked Read 1 Done Sent Done false
20808 6711 Acked Write 9 Done Sent Done false
20809 6712 Acked Flush 0 Done Sent Done false
20810 6713 Acked Read 4 Done Sent Done false
20811 6714 Acked Write 6 Done Sent Done false
20812 6715 Acked Write 9 Done Sent Done false
20813 6716 Acked Read 1 Done Sent Done false
20814 6717 Acked Write 6 Done Sent Done false
20815 6718 Acked Write 9 Done Sent Done false
20816 6719 Acked Read 1 Done Sent Done false
20817 6720 Acked Read 4 Done Sent Done false
20818 6721 Acked Read 6 Done Sent Done false
20819 6722 Acked Write 5 Done Sent Done false
20820 6723 Acked Flush 0 Done Sent Done false
20821 6724 Acked Read 9 Done Sent Done false
20822 6725 Acked Write 5 Done Sent Done false
20823 6726 Acked Read 8 Done Sent Done false
20824 6727 Acked Read 8 Done Sent Done false
20825 6728 Acked Write 8 Done Sent Done false
20826 6729 Acked Write 3 Done Sent Done false
20827 6730 Acked Read 2 Done Sent Done false
20828 6731 Acked Read 5 Done Sent Done false
20829 6732 Acked Read 7 Done Sent Done false
20830 6733 Acked Read 10 Done Sent Done false
20831 6734 Acked Read 4 Done Sent Done false
20832 STATES DS:0 DS:1 DS:2 TOTAL
20833 Sent 0 301 0 301
20834 Done 301 0 301 602
20835 Skipped 0 0 0 0
20836 Error 0 0 0 0
20837 Last Flush: 6723 6433 6723
20838 Downstairs last five completed: 6433 6432 6431 6430 6429
20839 Upstairs last five completed: 6734 6733 6732 6731 6730
20840 CLIENT: Up:0 ds:301 act:2
20841 ----------------------------------------------------------------
20842 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20843 States: Active Offline Active
20844 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20845 6434 Acked Write 2 Done Sent Done false
20846 6435 Acked Write 3 Done Sent Done false
20847 6436 Acked Read 2 Done Sent Done false
20848 6437 Acked Read 2 Done Sent Done false
20849 6438 Acked Flush 0 Done Sent Done false
20850 6439 Acked Write 3 Done Sent Done false
20851 6440 Acked Flush 0 Done Sent Done false
20852 6441 Acked Read 8 Done Sent Done false
20853 6442 Acked Read 3 Done Sent Done false
20854 6443 Acked Read 7 Done Sent Done false
20855 6444 Acked Write 7 Done Sent Done false
20856 6445 Acked Read 6 Done Sent Done false
20857 6446 Acked Read 10 Done Sent Done false
20858 6447 Acked Write 2 Done Sent Done false
20859 6448 Acked Write 9 Done Sent Done false
20860 6449 Acked Read 8 Done Sent Done false
20861 6450 Acked Read 4 Done Sent Done false
20862 6451 Acked Read 7 Done Sent Done false
20863 6452 Acked Write 3 Done Sent Done false
20864 6453 Acked Write 5 Done Sent Done false
20865 6454 Acked Read 3 Done Sent Done false
20866 6455 Acked Write 7 Done Sent Done false
20867 6456 Acked Read 2 Done Sent Done false
20868 6457 Acked Read 8 Done Sent Done false
20869 6458 Acked Read 6 Done Sent Done false
20870 6459 Acked Write 6 Done Sent Done false
20871 6460 Acked Write 10 Done Sent Done false
20872 6461 Acked Read 7 Done Sent Done false
20873 6462 Acked Flush 0 Done Sent Done false
20874 6463 Acked Write 6 Done Sent Done false
20875 6464 Acked Write 2 Done Sent Done false
20876 6465 Acked Write 10 Done Sent Done false
20877 6466 Acked Read 1 Done Sent Done false
20878 6467 Acked Read 5 Done Sent Done false
20879 6468 Acked Read 1 Done Sent Done false
20880 6469 Acked Read 10 Done Sent Done false
20881 6470 Acked Flush 0 Done Sent Done false
20882 6471 Acked Flush 0 Done Sent Done false
20883 6472 Acked Flush 0 Done Sent Done false
20884 6473 Acked Write 9 Done Sent Done false
20885 6474 Acked Write 6 Done Sent Done false
20886 6475 Acked Write 8 Done Sent Done false
20887 6476 Acked Read 8 Done Sent Done false
20888 6477 Acked Read 10 Done Sent Done false
20889 6478 Acked Read 5 Done Sent Done false
20890 6479 Acked Read 2 Done Sent Done false
20891 6480 Acked Read 2 Done Sent Done false
20892 6481 Acked Write 8 Done Sent Done false
20893 6482 Acked Write 9 Done Sent Done false
20894 6483 Acked Write 5 Done Sent Done false
20895 6484 Acked Flush 0 Done Sent Done false
20896 6485 Acked Read 6 Done Sent Done false
20897 6486 Acked Write 10 Done Sent Done false
20898 6487 Acked Write 3 Done Sent Done false
20899 6488 Acked Flush 0 Done Sent Done false
20900 6489 Acked Flush 0 Done Sent Done false
20901 6490 Acked Write 10 Done Sent Done false
20902 6491 Acked Write 9 Done Sent Done false
20903 6492 Acked Write 2 Done Sent Done false
20904 6493 Acked Write 3 Done Sent Done false
20905 6494 Acked Read 3 Done Sent Done false
20906 6495 Acked Read 3 Done Sent Done false
20907 6496 Acked Read 8 Done Sent Done false
20908 6497 Acked Write 4 Done Sent Done false
20909 6498 Acked Flush 0 Done Sent Done false
20910 6499 Acked Read 4 Done Sent Done false
20911 6500 Acked Read 3 Done Sent Done false
20912 6501 Acked Flush 0 Done Sent Done false
20913 6502 Acked Write 3 Done Sent Done false
20914 6503 Acked Write 5 Done Sent Done false
20915 6504 Acked Write 1 Done Sent Done false
20916 6505 Acked Read 7 Done Sent Done false
20917 6506 Acked Read 6 Done Sent Done false
20918 6507 Acked Read 5 Done Sent Done false
20919 6508 Acked Write 7 Done Sent Done false
20920 6509 Acked Read 5 Done Sent Done false
20921 6510 Acked Read 1 Done Sent Done false
20922 6511 Acked Read 10 Done Sent Done false
20923 6512 Acked Read 1 Done Sent Done false
20924 6513 Acked Flush 0 Done Sent Done false
20925 6514 Acked Read 4 Done Sent Done false
20926 6515 Acked Read 10 Done Sent Done false
20927 6516 Acked Write 5 Done Sent Done false
20928 6517 Acked Write 2 Done Sent Done false
20929 6518 Acked Flush 0 Done Sent Done false
20930 6519 Acked Write 3 Done Sent Done false
20931 6520 Acked Write 3 Done Sent Done false
20932 6521 Acked Write 2 Done Sent Done false
20933 6522 Acked Flush 0 Done Sent Done false
20934 6523 Acked Write 7 Done Sent Done false
20935 6524 Acked Read 2 Done Sent Done false
20936 6525 Acked Read 7 Done Sent Done false
20937 6526 Acked Write 10 Done Sent Done false
20938 6527 Acked Read 4 Done Sent Done false
20939 6528 Acked Read 7 Done Sent Done false
20940 6529 Acked Write 2 Done Sent Done false
20941 6530 Acked Flush 0 Done Sent Done false
20942 6531 Acked Write 10 Done Sent Done false
20943 6532 Acked Write 5 Done Sent Done false
20944 6533 Acked Read 4 Done Sent Done false
20945 6534 Acked Write 5 Done Sent Done false
20946 6535 Acked Write 9 Done Sent Done false
20947 6536 Acked Flush 0 Done Sent Done false
20948 6537 Acked Read 3 Done Sent Done false
20949 6538 Acked Write 4 Done Sent Done false
20950 6539 Acked Write 5 Done Sent Done false
20951 6540 Acked Write 5 Done Sent Done false
20952 6541 Acked Read 3 Done Sent Done false
20953 6542 Acked Read 1 Done Sent Done false
20954 6543 Acked Read 10 Done Sent Done false
20955 6544 Acked Read 9 Done Sent Done false
20956 6545 Acked Write 5 Done Sent Done false
20957 6546 Acked Read 5 Done Sent Done false
20958 6547 Acked Read 1 Done Sent Done false
20959 6548 Acked Write 5 Done Sent Done false
20960 6549 Acked Read 8 Done Sent Done false
20961 6550 Acked Read 5 Done Sent Done false
20962 6551 Acked Read 5 Done Sent Done false
20963 6552 Acked Write 9 Done Sent Done false
20964 6553 Acked Read 2 Done Sent Done false
20965 6554 Acked Read 8 Done Sent Done false
20966 6555 Acked Write 2 Done Sent Done false
20967 6556 Acked Read 4 Done Sent Done false
20968 6557 Acked Flush 0 Done Sent Done false
20969 6558 Acked Write 5 Done Sent Done false
20970 6559 Acked Flush 0 Done Sent Done false
20971 6560 Acked Read 5 Done Sent Done false
20972 6561 Acked Read 9 Done Sent Done false
20973 6562 Acked Read 3 Done Sent Done false
20974 6563 Acked Read 9 Done Sent Done false
20975 6564 Acked Read 9 Done Sent Done false
20976 6565 Acked Write 4 Done Sent Done false
20977 6566 Acked Read 8 Done Sent Done false
20978 6567 Acked Write 2 Done Sent Done false
20979 6568 Acked Write 7 Done Sent Done false
20980 6569 Acked Flush 0 Done Sent Done false
20981 6570 Acked Read 5 Done Sent Done false
20982 6571 Acked Read 7 Done Sent Done false
20983 6572 Acked Write 9 Done Sent Done false
20984 6573 Acked Read 1 Done Sent Done false
20985 6574 Acked Read 8 Done Sent Done false
20986 6575 Acked Read 6 Done Sent Done false
20987 6576 Acked Write 10 Done Sent Done false
20988 6577 Acked Write 1 Done Sent Done false
20989 6578 Acked Write 6 Done Sent Done false
20990 6579 Acked Flush 0 Done Sent Done false
20991 6580 Acked Read 8 Done Sent Done false
20992 6581 Acked Write 10 Done Sent Done false
20993 6582 Acked Read 3 Done Sent Done false
20994 6583 Acked Write 8 Done Sent Done false
20995 6584 Acked Flush 0 Done Sent Done false
20996 6585 Acked Read 2 Done Sent Done false
20997 6586 Acked Write 9 Done Sent Done false
20998 6587 Acked Write 6 Done Sent Done false
20999 6588 Acked Read 3 Done Sent Done false
21000 6589 Acked Read 9 Done Sent Done false
21001 6590 Acked Write 10 Done Sent Done false
21002 6591 Acked Read 5 Done Sent Done false
21003 6592 Acked Read 9 Done Sent Done false
21004 6593 Acked Read 1 Done Sent Done false
21005 6594 Acked Read 4 Done Sent Done false
21006 6595 Acked Write 1 Done Sent Done false
21007 6596 Acked Write 6 Done Sent Done false
21008 6597 Acked Write 1 Done Sent Done false
21009 6598 Acked Flush 0 Done Sent Done false
21010 6599 Acked Write 5 Done Sent Done false
21011 6600 Acked Read 6 Done Sent Done false
21012 6601 Acked Write 4 Done Sent Done false
21013 6602 Acked Read 8 Done Sent Done false
21014 6603 Acked Write 1 Done Sent Done false
21015 6604 Acked Write 8 Done Sent Done false
21016 6605 Acked Flush 0 Done Sent Done false
21017 6606 Acked Write 2 Done Sent Done false
21018 6607 Acked Write 5 Done Sent Done false
21019 6608 Acked Read 10 Done Sent Done false
21020 6609 Acked Read 8 Done Sent Done false
21021 6610 Acked Write 4 Done Sent Done false
21022 6611 Acked Read 9 Done Sent Done false
21023 6612 Acked Read 5 Done Sent Done false
21024 6613 Acked Write 7 Done Sent Done false
21025 6614 Acked Write 5 Done Sent Done false
21026 6615 Acked Write 2 Done Sent Done false
21027 6616 Acked Flush 0 Done Sent Done false
21028 6617 Acked Read 4 Done Sent Done false
21029 6618 Acked Write 9 Done Sent Done false
21030 6619 Acked Read 5 Done Sent Done false
21031 6620 Acked Write 3 Done Sent Done false
21032 6621 Acked Write 3 Done Sent Done false
21033 6622 Acked Read 10 Done Sent Done false
21034 6623 Acked Write 2 Done Sent Done false
21035 6624 Acked Write 5 Done Sent Done false
21036 6625 Acked Read 1 Done Sent Done false
21037 6626 Acked Write 10 Done Sent Done false
21038 6627 Acked Read 10 Done Sent Done false
21039 6628 Acked Read 2 Done Sent Done false
21040 6629 Acked Read 9 Done Sent Done false
21041 6630 Acked Read 9 Done Sent Done false
21042 6631 Acked Read 8 Done Sent Done false
21043 6632 Acked Write 6 Done Sent Done false
21044 6633 Acked Write 6 Done Sent Done false
21045 6634 Acked Write 2 Done Sent Done false
21046 6635 Acked Write 3 Done Sent Done false
21047 6636 Acked Write 2 Done Sent Done false
21048 6637 Acked Flush 0 Done Sent Done false
21049 6638 Acked Write 2 Done Sent Done false
21050 6639 Acked Read 8 Done Sent Done false
21051 6640 Acked Read 4 Done Sent Done false
21052 6641 Acked Write 9 Done Sent Done false
21053 6642 Acked Read 6 Done Sent Done false
21054 6643 Acked Read 5 Done Sent Done false
21055 6644 Acked Read 10 Done Sent Done false
21056 6645 Acked Read 3 Done Sent Done false
21057 6646 Acked Write 6 Done Sent Done false
21058 6647 Acked Write 1 Done Sent Done false
21059 6648 Acked Read 5 Done Sent Done false
21060 6649 Acked Read 7 Done Sent Done false
21061 6650 Acked Write 8 Done Sent Done false
21062 6651 Acked Write 1 Done Sent Done false
21063 6652 Acked Read 8 Done Sent Done false
21064 6653 Acked Flush 0 Done Sent Done false
21065 6654 Acked Read 2 Done Sent Done false
21066 6655 Acked Read 9 Done Sent Done false
21067 6656 Acked Flush 0 Done Sent Done false
21068 6657 Acked Write 8 Done Sent Done false
21069 6658 Acked Read 9 Done Sent Done false
21070 6659 Acked Read 5 Done Sent Done false
21071 6660 Acked Read 3 Done Sent Done false
21072 6661 Acked Read 10 Done Sent Done false
21073 6662 Acked Read 6 Done Sent Done false
21074 6663 Acked Flush 0 Done Sent Done false
21075 6664 Acked Read 2 Done Sent Done false
21076 6665 Acked Read 9 Done Sent Done false
21077 6666 Acked Read 6 Done Sent Done false
21078 6667 Acked Read 5 Done Sent Done false
21079 6668 Acked Flush 0 Done Sent Done false
21080 6669 Acked Read 7 Done Sent Done false
21081 6670 Acked Write 4 Done Sent Done false
21082 6671 Acked Read 6 Done Sent Done false
21083 6672 Acked Write 3 Done Sent Done false
21084 6673 Acked Read 8 Done Sent Done false
21085 6674 Acked Read 9 Done Sent Done false
21086 6675 Acked Flush 0 Done Sent Done false
21087 6676 Acked Write 2 Done Sent Done false
21088 6677 Acked Write 5 Done Sent Done false
21089 6678 Acked Write 4 Done Sent Done false
21090 6679 Acked Write 3 Done Sent Done false
21091 6680 Acked Read 4 Done Sent Done false
21092 6681 Acked Write 5 Done Sent Done false
21093 6682 Acked Flush 0 Done Sent Done false
21094 6683 Acked Read 7 Done Sent Done false
21095 6684 Acked Write 2 Done Sent Done false
21096 6685 Acked Write 4 Done Sent Done false
21097 6686 Acked Read 3 Done Sent Done false
21098 6687 Acked Read 9 Done Sent Done false
21099 6688 Acked Read 8 Done Sent Done false
21100 6689 Acked Write 10 Done Sent Done false
21101 6690 Acked Write 5 Done Sent Done false
21102 6691 Acked Write 1 Done Sent Done false
21103 6692 Acked Write 7 Done Sent Done false
21104 6693 Acked Write 10 Done Sent Done false
21105 6694 Acked Write 9 Done Sent Done false
21106 6695 Acked Write 9 Done Sent Done false
21107 6696 Acked Write 9 Done Sent Done false
21108 6697 Acked Write 5 Done Sent Done false
21109 6698 Acked Read 3 Done Sent Done false
21110 6699 Acked Read 10 Done Sent Done false
21111 6700 Acked Read 2 Done Sent Done false
21112 6701 Acked Read 1 Done Sent Done false
21113 6702 Acked Flush 0 Done Sent Done false
21114 6703 Acked Read 2 Done Sent Done false
21115 6704 Acked Write 4 Done Sent Done false
21116 6705 Acked Write 5 Done Sent Done false
21117 6706 Acked Read 8 Done Sent Done false
21118 6707 Acked Read 8 Done Sent Done false
21119 6708 Acked Write 2 Done Sent Done false
21120 6709 Acked Write 2 Done Sent Done false
21121 6710 Acked Read 1 Done Sent Done false
21122 6711 Acked Write 9 Done Sent Done false
21123 6712 Acked Flush 0 Done Sent Done false
21124 6713 Acked Read 4 Done Sent Done false
21125 6714 Acked Write 6 Done Sent Done false
21126 6715 Acked Write 9 Done Sent Done false
21127 6716 Acked Read 1 Done Sent Done false
21128 6717 Acked Write 6 Done Sent Done false
21129 6718 Acked Write 9 Done Sent Done false
21130 6719 Acked Read 1 Done Sent Done false
21131 6720 Acked Read 4 Done Sent Done false
21132 6721 Acked Read 6 Done Sent Done false
21133 6722 Acked Write 5 Done Sent Done false
21134 6723 Acked Flush 0 Done Sent Done false
21135 6724 Acked Read 9 Done Sent Done false
21136 6725 Acked Write 5 Done Sent Done false
21137 6726 Acked Read 8 Done Sent Done false
21138 6727 Acked Read 8 Done Sent Done false
21139 6728 Acked Write 8 Done Sent Done false
21140 6729 Acked Write 3 Done Sent Done false
21141 6730 Acked Read 2 Done Sent Done false
21142 6731 Acked Read 5 Done Sent Done false
21143 6732 Acked Read 7 Done Sent Done false
21144 6733 Acked Read 10 Done Sent Done false
21145 6734 Acked Read 4 Done Sent Done false
21146 6735 Acked Flush 0 Done Sent Done false
21147 STATES DS:0 DS:1 DS:2 TOTAL
21148 Sent 0 302 0 302
21149 Done 302 0 302 604
21150 Skipped 0 0 0 0
21151 Error 0 0 0 0
21152 Last Flush: 6735 6433 6735
21153 Downstairs last five completed: 6433 6432 6431 6430 6429
21154 Upstairs last five completed: 6735 6734 6733 6732 6731
21155 CLIENT: Up:0 ds:302 act:2
211562025-01-13T15:56:13.572ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
211572025-01-13T15:56:13.573ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
211582025-01-13T15:56:13.573ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
211592025-01-13T15:56:13.573ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
211602025-01-13T15:56:13.573ZINFOcrucible: client task is exiting = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
21161 [8820] got start action
21162 [8820] I am going to restart
21163 Starting downstairs at port 8820
21164 Make output file at "/tmp/dsc/downstairs-8820.txt"
21165 [8820][1] reports Starting
21166 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1221)
21167 [8820][1] reports Running
21168 ----------------------------------------------------------------
21169 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21170 States: Active Offline Active
21171 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21172 6434 Acked Write 2 Done Sent Done false
21173 6435 Acked Write 3 Done Sent Done false
21174 6436 Acked Read 2 Done Sent Done false
21175 6437 Acked Read 2 Done Sent Done false
21176 6438 Acked Flush 0 Done Sent Done false
21177 6439 Acked Write 3 Done Sent Done false
21178 6440 Acked Flush 0 Done Sent Done false
21179 6441 Acked Read 8 Done Sent Done false
21180 6442 Acked Read 3 Done Sent Done false
21181 6443 Acked Read 7 Done Sent Done false
21182 6444 Acked Write 7 Done Sent Done false
21183 6445 Acked Read 6 Done Sent Done false
21184 6446 Acked Read 10 Done Sent Done false
21185 6447 Acked Write 2 Done Sent Done false
21186 6448 Acked Write 9 Done Sent Done false
21187 6449 Acked Read 8 Done Sent Done false
21188 6450 Acked Read 4 Done Sent Done false
21189 6451 Acked Read 7 Done Sent Done false
21190 6452 Acked Write 3 Done Sent Done false
21191 6453 Acked Write 5 Done Sent Done false
21192 6454 Acked Read 3 Done Sent Done false
21193 6455 Acked Write 7 Done Sent Done false
21194 6456 Acked Read 2 Done Sent Done false
21195 6457 Acked Read 8 Done Sent Done false
21196 6458 Acked Read 6 Done Sent Done false
21197 6459 Acked Write 6 Done Sent Done false
21198 6460 Acked Write 10 Done Sent Done false
21199 6461 Acked Read 7 Done Sent Done false
21200 6462 Acked Flush 0 Done Sent Done false
21201 6463 Acked Write 6 Done Sent Done false
21202 6464 Acked Write 2 Done Sent Done false
21203 6465 Acked Write 10 Done Sent Done false
21204 6466 Acked Read 1 Done Sent Done false
21205 6467 Acked Read 5 Done Sent Done false
21206 6468 Acked Read 1 Done Sent Done false
21207 6469 Acked Read 10 Done Sent Done false
21208 6470 Acked Flush 0 Done Sent Done false
21209 6471 Acked Flush 0 Done Sent Done false
21210 6472 Acked Flush 0 Done Sent Done false
21211 6473 Acked Write 9 Done Sent Done false
21212 6474 Acked Write 6 Done Sent Done false
21213 6475 Acked Write 8 Done Sent Done false
21214 6476 Acked Read 8 Done Sent Done false
21215 6477 Acked Read 10 Done Sent Done false
21216 6478 Acked Read 5 Done Sent Done false
21217 6479 Acked Read 2 Done Sent Done false
21218 6480 Acked Read 2 Done Sent Done false
21219 6481 Acked Write 8 Done Sent Done false
21220 6482 Acked Write 9 Done Sent Done false
21221 6483 Acked Write 5 Done Sent Done false
21222 6484 Acked Flush 0 Done Sent Done false
21223 6485 Acked Read 6 Done Sent Done false
21224 6486 Acked Write 10 Done Sent Done false
21225 6487 Acked Write 3 Done Sent Done false
21226 6488 Acked Flush 0 Done Sent Done false
21227 6489 Acked Flush 0 Done Sent Done false
21228 6490 Acked Write 10 Done Sent Done false
21229 6491 Acked Write 9 Done Sent Done false
21230 6492 Acked Write 2 Done Sent Done false
21231 6493 Acked Write 3 Done Sent Done false
21232 6494 Acked Read 3 Done Sent Done false
21233 6495 Acked Read 3 Done Sent Done false
21234 6496 Acked Read 8 Done Sent Done false
21235 6497 Acked Write 4 Done Sent Done false
21236 6498 Acked Flush 0 Done Sent Done false
21237 6499 Acked Read 4 Done Sent Done false
21238 6500 Acked Read 3 Done Sent Done false
21239 6501 Acked Flush 0 Done Sent Done false
21240 6502 Acked Write 3 Done Sent Done false
21241 6503 Acked Write 5 Done Sent Done false
21242 6504 Acked Write 1 Done Sent Done false
21243 6505 Acked Read 7 Done Sent Done false
21244 6506 Acked Read 6 Done Sent Done false
21245 6507 Acked Read 5 Done Sent Done false
21246 6508 Acked Write 7 Done Sent Done false
21247 6509 Acked Read 5 Done Sent Done false
21248 6510 Acked Read 1 Done Sent Done false
21249 6511 Acked Read 10 Done Sent Done false
21250 6512 Acked Read 1 Done Sent Done false
21251 6513 Acked Flush 0 Done Sent Done false
21252 6514 Acked Read 4 Done Sent Done false
21253 6515 Acked Read 10 Done Sent Done false
21254 6516 Acked Write 5 Done Sent Done false
21255 6517 Acked Write 2 Done Sent Done false
21256 6518 Acked Flush 0 Done Sent Done false
21257 6519 Acked Write 3 Done Sent Done false
21258 6520 Acked Write 3 Done Sent Done false
21259 6521 Acked Write 2 Done Sent Done false
21260 6522 Acked Flush 0 Done Sent Done false
21261 6523 Acked Write 7 Done Sent Done false
21262 6524 Acked Read 2 Done Sent Done false
21263 6525 Acked Read 7 Done Sent Done false
21264 6526 Acked Write 10 Done Sent Done false
21265 6527 Acked Read 4 Done Sent Done false
21266 6528 Acked Read 7 Done Sent Done false
21267 6529 Acked Write 2 Done Sent Done false
21268 6530 Acked Flush 0 Done Sent Done false
21269 6531 Acked Write 10 Done Sent Done false
21270 6532 Acked Write 5 Done Sent Done false
21271 6533 Acked Read 4 Done Sent Done false
21272 6534 Acked Write 5 Done Sent Done false
21273 6535 Acked Write 9 Done Sent Done false
21274 6536 Acked Flush 0 Done Sent Done false
21275 6537 Acked Read 3 Done Sent Done false
21276 6538 Acked Write 4 Done Sent Done false
21277 6539 Acked Write 5 Done Sent Done false
21278 6540 Acked Write 5 Done Sent Done false
21279 6541 Acked Read 3 Done Sent Done false
21280 6542 Acked Read 1 Done Sent Done false
21281 6543 Acked Read 10 Done Sent Done false
21282 6544 Acked Read 9 Done Sent Done false
21283 6545 Acked Write 5 Done Sent Done false
21284 6546 Acked Read 5 Done Sent Done false
21285 6547 Acked Read 1 Done Sent Done false
21286 6548 Acked Write 5 Done Sent Done false
21287 6549 Acked Read 8 Done Sent Done false
21288 6550 Acked Read 5 Done Sent Done false
21289 6551 Acked Read 5 Done Sent Done false
21290 6552 Acked Write 9 Done Sent Done false
21291 6553 Acked Read 2 Done Sent Done false
21292 6554 Acked Read 8 Done Sent Done false
21293 6555 Acked Write 2 Done Sent Done false
21294 6556 Acked Read 4 Done Sent Done false
21295 6557 Acked Flush 0 Done Sent Done false
21296 6558 Acked Write 5 Done Sent Done false
21297 6559 Acked Flush 0 Done Sent Done false
21298 6560 Acked Read 5 Done Sent Done false
21299 6561 Acked Read 9 Done Sent Done false
21300 6562 Acked Read 3 Done Sent Done false
21301 6563 Acked Read 9 Done Sent Done false
21302 6564 Acked Read 9 Done Sent Done false
21303 6565 Acked Write 4 Done Sent Done false
21304 6566 Acked Read 8 Done Sent Done false
21305 6567 Acked Write 2 Done Sent Done false
21306 6568 Acked Write 7 Done Sent Done false
21307 6569 Acked Flush 0 Done Sent Done false
21308 6570 Acked Read 5 Done Sent Done false
21309 6571 Acked Read 7 Done Sent Done false
21310 6572 Acked Write 9 Done Sent Done false
21311 6573 Acked Read 1 Done Sent Done false
21312 6574 Acked Read 8 Done Sent Done false
21313 6575 Acked Read 6 Done Sent Done false
21314 6576 Acked Write 10 Done Sent Done false
21315 6577 Acked Write 1 Done Sent Done false
21316 6578 Acked Write 6 Done Sent Done false
21317 6579 Acked Flush 0 Done Sent Done false
21318 6580 Acked Read 8 Done Sent Done false
21319 6581 Acked Write 10 Done Sent Done false
21320 6582 Acked Read 3 Done Sent Done false
21321 6583 Acked Write 8 Done Sent Done false
21322 6584 Acked Flush 0 Done Sent Done false
21323 6585 Acked Read 2 Done Sent Done false
21324 6586 Acked Write 9 Done Sent Done false
21325 6587 Acked Write 6 Done Sent Done false
21326 6588 Acked Read 3 Done Sent Done false
21327 6589 Acked Read 9 Done Sent Done false
21328 6590 Acked Write 10 Done Sent Done false
21329 6591 Acked Read 5 Done Sent Done false
21330 6592 Acked Read 9 Done Sent Done false
21331 6593 Acked Read 1 Done Sent Done false
21332 6594 Acked Read 4 Done Sent Done false
21333 6595 Acked Write 1 Done Sent Done false
21334 6596 Acked Write 6 Done Sent Done false
21335 6597 Acked Write 1 Done Sent Done false
21336 6598 Acked Flush 0 Done Sent Done false
21337 6599 Acked Write 5 Done Sent Done false
21338 6600 Acked Read 6 Done Sent Done false
21339 6601 Acked Write 4 Done Sent Done false
21340 6602 Acked Read 8 Done Sent Done false
21341 6603 Acked Write 1 Done Sent Done false
21342 6604 Acked Write 8 Done Sent Done false
21343 6605 Acked Flush 0 Done Sent Done false
21344 6606 Acked Write 2 Done Sent Done false
21345 6607 Acked Write 5 Done Sent Done false
21346 6608 Acked Read 10 Done Sent Done false
21347 6609 Acked Read 8 Done Sent Done false
21348 6610 Acked Write 4 Done Sent Done false
21349 6611 Acked Read 9 Done Sent Done false
21350 6612 Acked Read 5 Done Sent Done false
21351 6613 Acked Write 7 Done Sent Done false
21352 6614 Acked Write 5 Done Sent Done false
21353 6615 Acked Write 2 Done Sent Done false
21354 6616 Acked Flush 0 Done Sent Done false
21355 6617 Acked Read 4 Done Sent Done false
21356 6618 Acked Write 9 Done Sent Done false
21357 6619 Acked Read 5 Done Sent Done false
21358 6620 Acked Write 3 Done Sent Done false
21359 6621 Acked Write 3 Done Sent Done false
21360 6622 Acked Read 10 Done Sent Done false
21361 6623 Acked Write 2 Done Sent Done false
21362 6624 Acked Write 5 Done Sent Done false
21363 6625 Acked Read 1 Done Sent Done false
21364 6626 Acked Write 10 Done Sent Done false
21365 6627 Acked Read 10 Done Sent Done false
21366 6628 Acked Read 2 Done Sent Done false
21367 6629 Acked Read 9 Done Sent Done false
21368 6630 Acked Read 9 Done Sent Done false
21369 6631 Acked Read 8 Done Sent Done false
21370 6632 Acked Write 6 Done Sent Done false
21371 6633 Acked Write 6 Done Sent Done false
21372 6634 Acked Write 2 Done Sent Done false
21373 6635 Acked Write 3 Done Sent Done false
21374 6636 Acked Write 2 Done Sent Done false
21375 6637 Acked Flush 0 Done Sent Done false
21376 6638 Acked Write 2 Done Sent Done false
21377 6639 Acked Read 8 Done Sent Done false
21378 6640 Acked Read 4 Done Sent Done false
21379 6641 Acked Write 9 Done Sent Done false
21380 6642 Acked Read 6 Done Sent Done false
21381 6643 Acked Read 5 Done Sent Done false
21382 6644 Acked Read 10 Done Sent Done false
21383 6645 Acked Read 3 Done Sent Done false
21384 6646 Acked Write 6 Done Sent Done false
21385 6647 Acked Write 1 Done Sent Done false
21386 6648 Acked Read 5 Done Sent Done false
21387 6649 Acked Read 7 Done Sent Done false
21388 6650 Acked Write 8 Done Sent Done false
21389 6651 Acked Write 1 Done Sent Done false
21390 6652 Acked Read 8 Done Sent Done false
21391 6653 Acked Flush 0 Done Sent Done false
21392 6654 Acked Read 2 Done Sent Done false
21393 6655 Acked Read 9 Done Sent Done false
21394 6656 Acked Flush 0 Done Sent Done false
21395 6657 Acked Write 8 Done Sent Done false
21396 6658 Acked Read 9 Done Sent Done false
21397 6659 Acked Read 5 Done Sent Done false
21398 6660 Acked Read 3 Done Sent Done false
21399 6661 Acked Read 10 Done Sent Done false
21400 6662 Acked Read 6 Done Sent Done false
21401 6663 Acked Flush 0 Done Sent Done false
21402 6664 Acked Read 2 Done Sent Done false
21403 6665 Acked Read 9 Done Sent Done false
21404 6666 Acked Read 6 Done Sent Done false
21405 6667 Acked Read 5 Done Sent Done false
21406 6668 Acked Flush 0 Done Sent Done false
21407 6669 Acked Read 7 Done Sent Done false
21408 6670 Acked Write 4 Done Sent Done false
21409 6671 Acked Read 6 Done Sent Done false
21410 6672 Acked Write 3 Done Sent Done false
21411 6673 Acked Read 8 Done Sent Done false
21412 6674 Acked Read 9 Done Sent Done false
21413 6675 Acked Flush 0 Done Sent Done false
21414 6676 Acked Write 2 Done Sent Done false
21415 6677 Acked Write 5 Done Sent Done false
21416 6678 Acked Write 4 Done Sent Done false
21417 6679 Acked Write 3 Done Sent Done false
21418 6680 Acked Read 4 Done Sent Done false
21419 6681 Acked Write 5 Done Sent Done false
21420 6682 Acked Flush 0 Done Sent Done false
21421 6683 Acked Read 7 Done Sent Done false
21422 6684 Acked Write 2 Done Sent Done false
21423 6685 Acked Write 4 Done Sent Done false
21424 6686 Acked Read 3 Done Sent Done false
21425 6687 Acked Read 9 Done Sent Done false
21426 6688 Acked Read 8 Done Sent Done false
21427 6689 Acked Write 10 Done Sent Done false
21428 6690 Acked Write 5 Done Sent Done false
21429 6691 Acked Write 1 Done Sent Done false
21430 6692 Acked Write 7 Done Sent Done false
21431 6693 Acked Write 10 Done Sent Done false
21432 6694 Acked Write 9 Done Sent Done false
21433 6695 Acked Write 9 Done Sent Done false
21434 6696 Acked Write 9 Done Sent Done false
21435 6697 Acked Write 5 Done Sent Done false
21436 6698 Acked Read 3 Done Sent Done false
21437 6699 Acked Read 10 Done Sent Done false
21438 6700 Acked Read 2 Done Sent Done false
21439 6701 Acked Read 1 Done Sent Done false
21440 6702 Acked Flush 0 Done Sent Done false
21441 6703 Acked Read 2 Done Sent Done false
21442 6704 Acked Write 4 Done Sent Done false
21443 6705 Acked Write 5 Done Sent Done false
21444 6706 Acked Read 8 Done Sent Done false
21445 6707 Acked Read 8 Done Sent Done false
21446 6708 Acked Write 2 Done Sent Done false
21447 6709 Acked Write 2 Done Sent Done false
21448 6710 Acked Read 1 Done Sent Done false
21449 6711 Acked Write 9 Done Sent Done false
21450 6712 Acked Flush 0 Done Sent Done false
21451 6713 Acked Read 4 Done Sent Done false
21452 6714 Acked Write 6 Done Sent Done false
21453 6715 Acked Write 9 Done Sent Done false
21454 6716 Acked Read 1 Done Sent Done false
21455 6717 Acked Write 6 Done Sent Done false
21456 6718 Acked Write 9 Done Sent Done false
21457 6719 Acked Read 1 Done Sent Done false
21458 6720 Acked Read 4 Done Sent Done false
21459 6721 Acked Read 6 Done Sent Done false
21460 6722 Acked Write 5 Done Sent Done false
21461 6723 Acked Flush 0 Done Sent Done false
21462 6724 Acked Read 9 Done Sent Done false
21463 6725 Acked Write 5 Done Sent Done false
21464 6726 Acked Read 8 Done Sent Done false
21465 6727 Acked Read 8 Done Sent Done false
21466 6728 Acked Write 8 Done Sent Done false
21467 6729 Acked Write 3 Done Sent Done false
21468 6730 Acked Read 2 Done Sent Done false
21469 6731 Acked Read 5 Done Sent Done false
21470 6732 Acked Read 7 Done Sent Done false
21471 6733 Acked Read 10 Done Sent Done false
21472 6734 Acked Read 4 Done Sent Done false
21473 6735 Acked Flush 0 Done Sent Done false
21474 STATES DS:0 DS:1 DS:2 TOTAL
21475 Sent 0 302 0 302
21476 Done 302 0 302 604
21477 Skipped 0 0 0 0
21478 Error 0 0 0 0
21479 Last Flush: 6735 6433 6735
21480 Downstairs last five completed: 6433 6432 6431 6430 6429
21481 Upstairs last five completed: 6735 6734 6733 6732 6731
21482 CLIENT: Up:0 ds:302 act:2
21483 ----------------------------------------------------------------
21484 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21485 States: Active Offline Active
21486 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21487 6434 Acked Write 2 Done Sent Done false
21488 6435 Acked Write 3 Done Sent Done false
21489 6436 Acked Read 2 Done Sent Done false
21490 6437 Acked Read 2 Done Sent Done false
21491 6438 Acked Flush 0 Done Sent Done false
21492 6439 Acked Write 3 Done Sent Done false
21493 6440 Acked Flush 0 Done Sent Done false
21494 6441 Acked Read 8 Done Sent Done false
21495 6442 Acked Read 3 Done Sent Done false
21496 6443 Acked Read 7 Done Sent Done false
21497 6444 Acked Write 7 Done Sent Done false
21498 6445 Acked Read 6 Done Sent Done false
21499 6446 Acked Read 10 Done Sent Done false
21500 6447 Acked Write 2 Done Sent Done false
21501 6448 Acked Write 9 Done Sent Done false
21502 6449 Acked Read 8 Done Sent Done false
21503 6450 Acked Read 4 Done Sent Done false
21504 6451 Acked Read 7 Done Sent Done false
21505 6452 Acked Write 3 Done Sent Done false
21506 6453 Acked Write 5 Done Sent Done false
21507 6454 Acked Read 3 Done Sent Done false
21508 6455 Acked Write 7 Done Sent Done false
21509 6456 Acked Read 2 Done Sent Done false
21510 6457 Acked Read 8 Done Sent Done false
21511 6458 Acked Read 6 Done Sent Done false
21512 6459 Acked Write 6 Done Sent Done false
21513 6460 Acked Write 10 Done Sent Done false
21514 6461 Acked Read 7 Done Sent Done false
21515 6462 Acked Flush 0 Done Sent Done false
21516 6463 Acked Write 6 Done Sent Done false
21517 6464 Acked Write 2 Done Sent Done false
21518 6465 Acked Write 10 Done Sent Done false
21519 6466 Acked Read 1 Done Sent Done false
21520 6467 Acked Read 5 Done Sent Done false
21521 6468 Acked Read 1 Done Sent Done false
21522 6469 Acked Read 10 Done Sent Done false
21523 6470 Acked Flush 0 Done Sent Done false
21524 6471 Acked Flush 0 Done Sent Done false
21525 6472 Acked Flush 0 Done Sent Done false
21526 6473 Acked Write 9 Done Sent Done false
21527 6474 Acked Write 6 Done Sent Done false
21528 6475 Acked Write 8 Done Sent Done false
21529 6476 Acked Read 8 Done Sent Done false
21530 6477 Acked Read 10 Done Sent Done false
21531 6478 Acked Read 5 Done Sent Done false
21532 6479 Acked Read 2 Done Sent Done false
21533 6480 Acked Read 2 Done Sent Done false
21534 6481 Acked Write 8 Done Sent Done false
21535 6482 Acked Write 9 Done Sent Done false
21536 6483 Acked Write 5 Done Sent Done false
21537 6484 Acked Flush 0 Done Sent Done false
21538 6485 Acked Read 6 Done Sent Done false
21539 6486 Acked Write 10 Done Sent Done false
21540 6487 Acked Write 3 Done Sent Done false
21541 6488 Acked Flush 0 Done Sent Done false
21542 6489 Acked Flush 0 Done Sent Done false
21543 6490 Acked Write 10 Done Sent Done false
21544 6491 Acked Write 9 Done Sent Done false
21545 6492 Acked Write 2 Done Sent Done false
21546 6493 Acked Write 3 Done Sent Done false
21547 6494 Acked Read 3 Done Sent Done false
21548 6495 Acked Read 3 Done Sent Done false
21549 6496 Acked Read 8 Done Sent Done false
21550 6497 Acked Write 4 Done Sent Done false
21551 6498 Acked Flush 0 Done Sent Done false
21552 6499 Acked Read 4 Done Sent Done false
21553 6500 Acked Read 3 Done Sent Done false
21554 6501 Acked Flush 0 Done Sent Done false
21555 6502 Acked Write 3 Done Sent Done false
21556 6503 Acked Write 5 Done Sent Done false
21557 6504 Acked Write 1 Done Sent Done false
21558 6505 Acked Read 7 Done Sent Done false
21559 6506 Acked Read 6 Done Sent Done false
21560 6507 Acked Read 5 Done Sent Done false
21561 6508 Acked Write 7 Done Sent Done false
21562 6509 Acked Read 5 Done Sent Done false
21563 6510 Acked Read 1 Done Sent Done false
21564 6511 Acked Read 10 Done Sent Done false
21565 6512 Acked Read 1 Done Sent Done false
21566 6513 Acked Flush 0 Done Sent Done false
21567 6514 Acked Read 4 Done Sent Done false
21568 6515 Acked Read 10 Done Sent Done false
21569 6516 Acked Write 5 Done Sent Done false
21570 6517 Acked Write 2 Done Sent Done false
21571 6518 Acked Flush 0 Done Sent Done false
21572 6519 Acked Write 3 Done Sent Done false
21573 6520 Acked Write 3 Done Sent Done false
21574 6521 Acked Write 2 Done Sent Done false
21575 6522 Acked Flush 0 Done Sent Done false
21576 6523 Acked Write 7 Done Sent Done false
21577 6524 Acked Read 2 Done Sent Done false
21578 6525 Acked Read 7 Done Sent Done false
21579 6526 Acked Write 10 Done Sent Done false
21580 6527 Acked Read 4 Done Sent Done false
21581 6528 Acked Read 7 Done Sent Done false
21582 6529 Acked Write 2 Done Sent Done false
21583 6530 Acked Flush 0 Done Sent Done false
21584 6531 Acked Write 10 Done Sent Done false
21585 6532 Acked Write 5 Done Sent Done false
21586 6533 Acked Read 4 Done Sent Done false
21587 6534 Acked Write 5 Done Sent Done false
21588 6535 Acked Write 9 Done Sent Done false
21589 6536 Acked Flush 0 Done Sent Done false
21590 6537 Acked Read 3 Done Sent Done false
21591 6538 Acked Write 4 Done Sent Done false
21592 6539 Acked Write 5 Done Sent Done false
21593 6540 Acked Write 5 Done Sent Done false
21594 6541 Acked Read 3 Done Sent Done false
21595 6542 Acked Read 1 Done Sent Done false
21596 6543 Acked Read 10 Done Sent Done false
21597 6544 Acked Read 9 Done Sent Done false
21598 6545 Acked Write 5 Done Sent Done false
21599 6546 Acked Read 5 Done Sent Done false
21600 6547 Acked Read 1 Done Sent Done false
21601 6548 Acked Write 5 Done Sent Done false
21602 6549 Acked Read 8 Done Sent Done false
21603 6550 Acked Read 5 Done Sent Done false
21604 6551 Acked Read 5 Done Sent Done false
21605 6552 Acked Write 9 Done Sent Done false
21606 6553 Acked Read 2 Done Sent Done false
21607 6554 Acked Read 8 Done Sent Done false
21608 6555 Acked Write 2 Done Sent Done false
21609 6556 Acked Read 4 Done Sent Done false
21610 6557 Acked Flush 0 Done Sent Done false
21611 6558 Acked Write 5 Done Sent Done false
21612 6559 Acked Flush 0 Done Sent Done false
21613 6560 Acked Read 5 Done Sent Done false
21614 6561 Acked Read 9 Done Sent Done false
21615 6562 Acked Read 3 Done Sent Done false
21616 6563 Acked Read 9 Done Sent Done false
21617 6564 Acked Read 9 Done Sent Done false
21618 6565 Acked Write 4 Done Sent Done false
21619 6566 Acked Read 8 Done Sent Done false
21620 6567 Acked Write 2 Done Sent Done false
21621 6568 Acked Write 7 Done Sent Done false
21622 6569 Acked Flush 0 Done Sent Done false
21623 6570 Acked Read 5 Done Sent Done false
21624 6571 Acked Read 7 Done Sent Done false
21625 6572 Acked Write 9 Done Sent Done false
21626 6573 Acked Read 1 Done Sent Done false
21627 6574 Acked Read 8 Done Sent Done false
21628 6575 Acked Read 6 Done Sent Done false
21629 6576 Acked Write 10 Done Sent Done false
21630 6577 Acked Write 1 Done Sent Done false
21631 6578 Acked Write 6 Done Sent Done false
21632 6579 Acked Flush 0 Done Sent Done false
21633 6580 Acked Read 8 Done Sent Done false
21634 6581 Acked Write 10 Done Sent Done false
21635 6582 Acked Read 3 Done Sent Done false
21636 6583 Acked Write 8 Done Sent Done false
21637 6584 Acked Flush 0 Done Sent Done false
21638 6585 Acked Read 2 Done Sent Done false
21639 6586 Acked Write 9 Done Sent Done false
21640 6587 Acked Write 6 Done Sent Done false
21641 6588 Acked Read 3 Done Sent Done false
21642 6589 Acked Read 9 Done Sent Done false
21643 6590 Acked Write 10 Done Sent Done false
21644 6591 Acked Read 5 Done Sent Done false
21645 6592 Acked Read 9 Done Sent Done false
21646 6593 Acked Read 1 Done Sent Done false
21647 6594 Acked Read 4 Done Sent Done false
21648 6595 Acked Write 1 Done Sent Done false
21649 6596 Acked Write 6 Done Sent Done false
21650 6597 Acked Write 1 Done Sent Done false
21651 6598 Acked Flush 0 Done Sent Done false
21652 6599 Acked Write 5 Done Sent Done false
21653 6600 Acked Read 6 Done Sent Done false
21654 6601 Acked Write 4 Done Sent Done false
21655 6602 Acked Read 8 Done Sent Done false
21656 6603 Acked Write 1 Done Sent Done false
21657 6604 Acked Write 8 Done Sent Done false
21658 6605 Acked Flush 0 Done Sent Done false
21659 6606 Acked Write 2 Done Sent Done false
21660 6607 Acked Write 5 Done Sent Done false
21661 6608 Acked Read 10 Done Sent Done false
21662 6609 Acked Read 8 Done Sent Done false
21663 6610 Acked Write 4 Done Sent Done false
21664 6611 Acked Read 9 Done Sent Done false
21665 6612 Acked Read 5 Done Sent Done false
21666 6613 Acked Write 7 Done Sent Done false
21667 6614 Acked Write 5 Done Sent Done false
21668 6615 Acked Write 2 Done Sent Done false
21669 6616 Acked Flush 0 Done Sent Done false
21670 6617 Acked Read 4 Done Sent Done false
21671 6618 Acked Write 9 Done Sent Done false
21672 6619 Acked Read 5 Done Sent Done false
21673 6620 Acked Write 3 Done Sent Done false
21674 6621 Acked Write 3 Done Sent Done false
21675 6622 Acked Read 10 Done Sent Done false
21676 6623 Acked Write 2 Done Sent Done false
21677 6624 Acked Write 5 Done Sent Done false
21678 6625 Acked Read 1 Done Sent Done false
21679 6626 Acked Write 10 Done Sent Done false
21680 6627 Acked Read 10 Done Sent Done false
21681 6628 Acked Read 2 Done Sent Done false
21682 6629 Acked Read 9 Done Sent Done false
21683 6630 Acked Read 9 Done Sent Done false
21684 6631 Acked Read 8 Done Sent Done false
21685 6632 Acked Write 6 Done Sent Done false
21686 6633 Acked Write 6 Done Sent Done false
21687 6634 Acked Write 2 Done Sent Done false
21688 6635 Acked Write 3 Done Sent Done false
21689 6636 Acked Write 2 Done Sent Done false
21690 6637 Acked Flush 0 Done Sent Done false
21691 6638 Acked Write 2 Done Sent Done false
21692 6639 Acked Read 8 Done Sent Done false
21693 6640 Acked Read 4 Done Sent Done false
21694 6641 Acked Write 9 Done Sent Done false
21695 6642 Acked Read 6 Done Sent Done false
21696 6643 Acked Read 5 Done Sent Done false
21697 6644 Acked Read 10 Done Sent Done false
21698 6645 Acked Read 3 Done Sent Done false
21699 6646 Acked Write 6 Done Sent Done false
21700 6647 Acked Write 1 Done Sent Done false
21701 6648 Acked Read 5 Done Sent Done false
21702 6649 Acked Read 7 Done Sent Done false
21703 6650 Acked Write 8 Done Sent Done false
21704 6651 Acked Write 1 Done Sent Done false
21705 6652 Acked Read 8 Done Sent Done false
21706 6653 Acked Flush 0 Done Sent Done false
21707 6654 Acked Read 2 Done Sent Done false
21708 6655 Acked Read 9 Done Sent Done false
21709 6656 Acked Flush 0 Done Sent Done false
21710 6657 Acked Write 8 Done Sent Done false
21711 6658 Acked Read 9 Done Sent Done false
21712 6659 Acked Read 5 Done Sent Done false
21713 6660 Acked Read 3 Done Sent Done false
21714 6661 Acked Read 10 Done Sent Done false
21715 6662 Acked Read 6 Done Sent Done false
21716 6663 Acked Flush 0 Done Sent Done false
21717 6664 Acked Read 2 Done Sent Done false
21718 6665 Acked Read 9 Done Sent Done false
21719 6666 Acked Read 6 Done Sent Done false
21720 6667 Acked Read 5 Done Sent Done false
21721 6668 Acked Flush 0 Done Sent Done false
21722 6669 Acked Read 7 Done Sent Done false
21723 6670 Acked Write 4 Done Sent Done false
21724 6671 Acked Read 6 Done Sent Done false
21725 6672 Acked Write 3 Done Sent Done false
21726 6673 Acked Read 8 Done Sent Done false
21727 6674 Acked Read 9 Done Sent Done false
21728 6675 Acked Flush 0 Done Sent Done false
21729 6676 Acked Write 2 Done Sent Done false
21730 6677 Acked Write 5 Done Sent Done false
21731 6678 Acked Write 4 Done Sent Done false
21732 6679 Acked Write 3 Done Sent Done false
21733 6680 Acked Read 4 Done Sent Done false
21734 6681 Acked Write 5 Done Sent Done false
21735 6682 Acked Flush 0 Done Sent Done false
21736 6683 Acked Read 7 Done Sent Done false
21737 6684 Acked Write 2 Done Sent Done false
21738 6685 Acked Write 4 Done Sent Done false
21739 6686 Acked Read 3 Done Sent Done false
21740 6687 Acked Read 9 Done Sent Done false
21741 6688 Acked Read 8 Done Sent Done false
21742 6689 Acked Write 10 Done Sent Done false
21743 6690 Acked Write 5 Done Sent Done false
21744 6691 Acked Write 1 Done Sent Done false
21745 6692 Acked Write 7 Done Sent Done false
21746 6693 Acked Write 10 Done Sent Done false
21747 6694 Acked Write 9 Done Sent Done false
21748 6695 Acked Write 9 Done Sent Done false
21749 6696 Acked Write 9 Done Sent Done false
21750 6697 Acked Write 5 Done Sent Done false
21751 6698 Acked Read 3 Done Sent Done false
21752 6699 Acked Read 10 Done Sent Done false
21753 6700 Acked Read 2 Done Sent Done false
21754 6701 Acked Read 1 Done Sent Done false
21755 6702 Acked Flush 0 Done Sent Done false
21756 6703 Acked Read 2 Done Sent Done false
21757 6704 Acked Write 4 Done Sent Done false
21758 6705 Acked Write 5 Done Sent Done false
21759 6706 Acked Read 8 Done Sent Done false
21760 6707 Acked Read 8 Done Sent Done false
21761 6708 Acked Write 2 Done Sent Done false
21762 6709 Acked Write 2 Done Sent Done false
21763 6710 Acked Read 1 Done Sent Done false
21764 6711 Acked Write 9 Done Sent Done false
21765 6712 Acked Flush 0 Done Sent Done false
21766 6713 Acked Read 4 Done Sent Done false
21767 6714 Acked Write 6 Done Sent Done false
21768 6715 Acked Write 9 Done Sent Done false
21769 6716 Acked Read 1 Done Sent Done false
21770 6717 Acked Write 6 Done Sent Done false
21771 6718 Acked Write 9 Done Sent Done false
21772 6719 Acked Read 1 Done Sent Done false
21773 6720 Acked Read 4 Done Sent Done false
21774 6721 Acked Read 6 Done Sent Done false
21775 6722 Acked Write 5 Done Sent Done false
21776 6723 Acked Flush 0 Done Sent Done false
21777 6724 Acked Read 9 Done Sent Done false
21778 6725 Acked Write 5 Done Sent Done false
21779 6726 Acked Read 8 Done Sent Done false
21780 6727 Acked Read 8 Done Sent Done false
21781 6728 Acked Write 8 Done Sent Done false
21782 6729 Acked Write 3 Done Sent Done false
21783 6730 Acked Read 2 Done Sent Done false
21784 6731 Acked Read 5 Done Sent Done false
21785 6732 Acked Read 7 Done Sent Done false
21786 6733 Acked Read 10 Done Sent Done false
21787 6734 Acked Read 4 Done Sent Done false
21788 6735 Acked Flush 0 Done Sent Done false
21789 STATES DS:0 DS:1 DS:2 TOTAL
21790 Sent 0 302 0 302
21791 Done 302 0 302 604
21792 Skipped 0 0 0 0
21793 Error 0 0 0 0
21794 Last Flush: 6735 6433 6735
21795 Downstairs last five completed: 6433 6432 6431 6430 6429
21796 Upstairs last five completed: 6735 6734 6733 6732 6731
21797 CLIENT: Up:0 ds:302 act:2
217982025-01-13T15:56:23.574ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
217992025-01-13T15:56:23.575ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41676) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
218002025-01-13T15:56:23.576ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
218012025-01-13T15:56:23.576ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
218022025-01-13T15:56:23.577ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
218032025-01-13T15:56:23.577ZINFOcrucible: send last flush ID to this DS: 6433 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
218042025-01-13T15:56:23.577ZINFOcrucible: Replied this last flush ID: 6433 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
218052025-01-13T15:56:23.577ZINFOcrucible: [1] client re-new 302 jobs since flush 6433 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
218062025-01-13T15:56:23.577ZINFOcrucible: [1] Replayed 302 jobs since flush: 6433 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
21807 ----------------------------------------------------------------
21808 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
21809 Downstairs last five completed: 6735 6734 6733 6732 6731
21810 Upstairs last five completed: 6735 6734 6733 6732 6731
21811 CLIENT: Up:0 ds:0 act:3
21812 Replay: All jobs finished
21813 Main task has work to do, go find it
21814 got dsc Stop(0)
21815 stop 0
21816 Jan 13 15:56:23.820 [8810] Got stop action so:false kr:false
21817 INFO request completed, latency_us: 556, response_code: 204, uri: /stop/cid/0, method: POST, req_id: f40dbd4b-29ca-4256-bccb-456ca0a36842, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
218182025-01-13T15:56:23.821ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
218192025-01-13T15:56:23.821ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
218202025-01-13T15:56:23.821ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
21821 Jan 13 15:56:23.821 INFO request completed, latency_us: 150, response_code: 200, uri: /state/cid/0, method:{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-13T15:56:23.821330374Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
21822 GET, req_id: f0fe64bc-9bb5-4717-90ea-2fc73ec4a656, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
21823 [8810] Exited with: ExitStatus(unix_wait_status(9))
21824 [8810][0] reports Exit
21825 Jan 13 15:56:27.824 INFO request completed, latency_us: 349, response_code: 200, uri: /state/cid/0, method: GET, req_id: aa0d8272-efa6-42f6-8de0-f26b98c9414d, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
21826 001/300 Flush
21827 002/300 Write block 3244 len 4096 data: 4
21828 003/300 Write block 1946 len 8192 data: 4 4
21829 004/300 Write block 889 len 12288 data: 3 3 3
21830 005/300 Read block 3228 len 12288
21831 006/300 Flush
21832 007/300 Write block 2600 len 16384 data: 5 5 5 7
21833 008/300 Read block 4550 len 8192
21834 009/300 Read block 2492 len 12288
21835 010/300 Write block 2723 len 8192 data: 2 4
21836 011/300 Read block 1683 len 8192
21837 012/300 Write block 2879 len 28672 data: 4 5 5 5 6 5 5
21838 013/300 Read block 4350 len 16384
21839 014/300 Read block 2407 len 12288
21840 015/300 Read block 2080 len 24576
21841 016/300 Write block 3311 len 36864 data: 4 4 4 3 3 3 3 5 5
21842 017/300 Read block 449 len 16384
21843 018/300 Read block 3442 len 8192
21844 019/300 Flush
21845 020/300 Flush
21846 021/300 Read block 4299 len 16384
21847 022/300 Write block 3677 len 32768 data: 3 3 3 3 2 3 3 3
21848 023/300 Read block 4330 len 16384
21849 024/300 Write block 4101 len 32768 data: 3 3 3 3 4 5 6 6
21850 025/300 Flush
21851 026/300 Write block 2741 len 40960 data: 3 3 3 2 3 3 6 6 7 7
21852 027/300 Read block 4516 len 8192
21853 028/300 Read block 4774 len 16384
21854 029/300 Write block 1160 len 20480 data: 3 4 4 5 4
21855 030/300 Flush
21856 031/300 Write block 4336 len 4096 data: 5
21857 032/300 Read block 655 len 12288
21858 033/300 Write block 2722 len 28672 data: 2 3 5 3 4 4 4
21859 034/300 Read block 2527 len 24576
21860 035/300 Write block 3121 len 36864 data: 4 5 4 4 4 5 4 4 3
21861 036/300 Read block 1660 len 28672
21862 037/300 Write block 2563 len 16384 data: 5 4 4 3
21863 038/300 Flush
21864 039/300 Read block 865 len 16384
21865 040/300 Flush
21866 041/300 Read block 3057 len 32768
21867 042/300 Read block 2872 len 36864
21868 043/300 Write block 2483 len 36864 data: 6 7 7 8 6 5 5 5 5
21869 044/300 Write block 2893 len 8192 data: 5 6
21870 045/300 Read block 3994 len 12288
21871 046/300 Read block 2110 len 4096
21872 047/300 Read block 2593 len 16384
21873 048/300 Read block 1142 len 16384
21874 049/300 Read block 2233 len 16384
21875 050/300 Flush
21876 051/300 Read block 3157 len 4096
21877 052/300 Read block 411 len 28672
21878 053/300 Flush
21879 054/300 Write block 2733 len 20480 data: 5 6 5 4 4
21880 055/300 Read block 40 len 16384
21881 056/300 Flush
21882 057/300 Write block 4029 len 24576 data: 6 5 5 5 4 4
21883 058/300 Read block 4002 len 24576
21884 059/300 Write block 1955 len 32768 data: 4 4 4 3 4 5 5 3
21885 060/300 Read block 3870 len 8192
21886 061/300 Write block 27 len 12288 data: 7 6 6
21887 062/300 Write block 4756 len 8192 data: 5 4
21888 063/300 Read block 1689 len 12288
21889 064/300 Read block 1185 len 20480
21890 065/300 Flush
21891 066/300 Read block 3959 len 20480
21892 067/300 Write block 4643 len 8192 data: 4 3
21893 068/300 Read block 3572 len 20480
21894 069/300 Write block 77 len 40960 data: 4 6 7 7 5 5 4 4 2 3
21895 070/300 Read block 1358 len 28672
21896 071/300 Write block 4216 len 4096 data: 9
21897 072/300 Write block 1018 len 16384 data: 5 5 7 7
21898 073/300 Write block 732 len 4096 data: 6
21899 074/300 Write block 4656 len 12288 data: 3 4 4
21900 075/300 Write block 3915 len 32768 data: 5 5 4 4 4 3 6 4
21901 076/300 Read block 2127 len 20480
21902 077/300 Write block 1565 len 16384 data: 3 4 3 4
21903 078/300 Read block 1898 len 4096
21904 079/300 Write block 1214 len 4096 data: 3
21905 080/300 Read block 1886 len 40960
21906 081/300 Read block 10 len 36864
21907 082/300 Write block 3369 len 28672 data: 3 4 5 5 5 5 4
21908 083/300 Write block 3389 len 16384 data: 7 8 6 6
21909 084/300 Write block 3107 len 36864 data: 3 4 5 5 5 5 5 4 4
21910 085/300 Write block 164 len 8192 data: 3 3
21911 086/300 Write block 2937 len 8192 data: 5 7
21912 087/300 Read block 3935 len 20480
21913 088/300 Write block 3879 len 12288 data: 3 3 3
21914 089/300 Read block 3816 len 12288
21915 090/300 Write block 1100 len 20480 data: 5 5 5 4 4
21916 091/300 Write block 1038 len 20480 data: 4 7 6 6 8
21917 092/300 Write block 775 len 24576 data: 5 4 6 5 4 4
21918 093/300 Read block 2786 len 12288
21919 094/300 Write block 334 len 36864 data: 2 2 2 2 2 3 3 4 2
21920 095/300 Flush
21921 096/300 Write block 4341 len 12288 data: 3 3 5
21922 097/300 Write block 878 len 12288 data: 5 6 8
21923 098/300 Read block 1175 len 20480
21924 099/300 Read block 3350 len 24576
21925 100/300 Read block 4458 len 4096
21926 101/300 Write block 2973 len 4096 data: 6
21927 102/300 Flush
21928 103/300 Write block 2684 len 36864 data: 5 4 3 5 5 4 3 2 3
21929 104/300 Read block 1270 len 40960
21930 105/300 Write block 3282 len 32768 data: 4 6 5 5 4 4 5 5
21931 106/300 Write block 1357 len 20480 data: 4 3 3 4 3
21932 107/300 Read block 4456 len 28672
21933 108/300 Read block 1280 len 24576
21934 109/300 Read block 3335 len 28672
21935 110/300 Write block 1546 len 24576 data: 3 3 3 6 5 5
21936 111/300 Write block 4948 len 20480 data: 4 4 4 4 4
21937 112/300 Write block 2563 len 24576 data: 6 5 5 4 3 2
21938 113/300 Write block 1301 len 8192 data: 4 4
21939 114/300 Write block 1084 len 20480 data: 7 7 7 6 6
21940 115/300 Flush
21941 116/300 Write block 2535 len 40960 data: 4 4 4 4 4 3 3 4 4 5
21942 117/300 Flush
21943 118/300 Write block 1506 len 8192 data: 6 4
21944 119/300 Read block 3747 len 32768
21945 120/300 Read block 756 len 36864
21946 121/300 Read block 4139 len 8192
21947 122/300 Read block 1156 len 8192
21948 123/300 Read block 383 len 32768
21949 124/300 Read block 4712 len 24576
21950 125/300 Write block 943 len 28672 data: 8 9 8 10 9 6 7
21951 126/300 Write block 4092 len 16384 data: 3 3 2 2
21952 127/300 Read block 76 len 24576
21953 128/300 Read block 1783 len 8192
21954 129/300 Read block 744 len 20480
21955 130/300 Write block 1481 len 8192 data: 3 2
21956 131/300 Read block 2738 len 28672
21957 132/300 Read block 2349 len 16384
21958 133/300 Write block 2259 len 8192 data: 4 4
21959 134/300 Read block 1955 len 32768
21960 135/300 Write block 3853 len 24576 data: 5 7 6 5 6 7
21961 136/300 Read block 4524 len 4096
21962 137/300 Read block 587 len 12288
21963 138/300 Flush
21964 139/300 Read block 811 len 8192
21965 140/300 Write block 2521 len 16384 data: 5 5 5 6
21966 141/300 Read block 352 len 40960
21967 142/300 Write block 1932 len 28672 data: 2 4 3 3 5 5 4
21968 143/300 Write block 3976 len 28672 data: 4 5 6 7 7 7 6
21969 144/300 Flush
21970 145/300 Flush
21971 146/300 Write block 1677 len 12288 data: 3 3 2
21972 147/300 Write block 3508 len 24576 data: 4 6 6 7 7 9
21973 148/300 Read block 4348 len 40960
21974 149/300 Read block 4656 len 8192
21975 150/300 Write block 1840 len 12288 data: 5 4 5
21976 151/300 Write block 3727 len 8192 data: 8 8
21977 152/300 Write block 150 len 8192 data: 8 7
21978 153/300 Flush
21979 154/300 Write block 3341 len 32768 data: 2 2 2 3 3 3 4 6
21980 155/300 Read block 2685 len 16384
21981 156/300 Read block 3713 len 16384
21982 157/300 Read block 3057 len 24576
21983 158/300 Read block 2456 len 28672
21984 159/300 Read block 3618 len 28672
21985 160/300 Write block 350 len 20480 data: 4 4 4 5 5
21986 161/300 Write block 2878 len 28672 data: 3 5 6 6 6 7 6
21987 162/300 Write block 1649 len 8192 data: 5 6
21988 163/300 Flush
21989 164/300 Read block 707 len 28672
21990 165/300 Read block 1545 len 40960
21991 166/300 Read block 463 len 36864
21992 167/300 Read block 4275 len 32768
21993 168/300 Write block 4402 len 28672 data: 4 3 5 4 5 5 5
21994 169/300 Write block 2825 len 8192 data: 6 6
21995 170/300 Read block 290 len 28672
21996 171/300 Write block 1498 len 20480 data: 9 9 8 6 6
21997 172/300 Write block 2043 len 24576 data: 5 4 5 6 6 7
21998 173/300 Read block 946 len 36864
21999 174/300 Write block 3281 len 12288 data: 5 5 7
22000 175/300 Write block 4734 len 20480 data: 8 9 8 8 7
22001 176/300 Read block 3528 len 32768
22002 177/300 Read block 4915 len 32768
22003 178/300 Write block 4636 len 40960 data: 3 3 3 4 3 3 4 5 4 3
22004 179/300 Read block 3647 len 24576
22005 180/300 Write block 1814 len 16384 data: 7 7 6 6
22006 181/300 Read block 4475 len 28672
22007 182/300 Read block 2926 len 32768
22008 183/300 Write block 3776 len 20480 data: 5 5 7 8 8
22009 184/300 Read block 4408 len 28672
22010 185/300 Flush
22011 186/300 Read block 1428 len 4096
22012 187/300 Read block 1114 len 40960
22013 188/300 Write block 2205 len 20480 data: 6 5 4 5 6
22014 189/300 Read block 270 len 12288
22015 190/300 Write block 1449 len 8192 data: 3 2
22016 191/300 Read block 4370 len 8192
22017 192/300 Write block 538 len 16384 data: 6 6 4 4
22018 193/300 Flush
22019 194/300 Read block 2834 len 24576
22020 195/300 Read block 2223 len 12288
22021 196/300 Read block 3379 len 16384
22022 197/300 Write block 1068 len 28672 data: 2 3 3 3 2 2 2
22023 198/300 Read block 1335 len 4096
22024 199/300 Write block 1050 len 32768 data: 5 5 6 6 6 6 7 6
22025 200/300 Write block 3209 len 16384 data: 5 6 6 5
22026 201/300 Flush
22027 202/300 Read block 3221 len 24576
22028 203/300 Read block 3099 len 24576
22029 204/300 Read block 530 len 4096
22030 205/300 Read block 1340 len 32768
22031 206/300 Flush
22032 207/300 Write block 3020 len 36864 data: 8 7 7 6 9 7 6 8 8
22033 208/300 Read block 2669 len 24576
22034 209/300 Write block 4598 len 12288 data: 7 6 8
22035 210/300 Read block 1570 len 28672
22036 211/300 Write block 2612 len 28672 data: 5 4 4 3 3 2 3
22037 212/300 Write block 1973 len 28672 data: 6 5 5 4 4 4 4
22038 213/300 Read block 307 len 28672
22039 214/300 Read block 662 len 12288
22040 215/300 Flush
22041 216/300 Write block 145 len 24576 data: 6 6 7 8 8 9
22042 217/300 Read block 2828 len 32768
22043 218/300 Read block 2146 len 20480
22044 219/300 Read block 3217 len 28672
22045 220/300 Read block 1302 len 20480
22046 221/300 Write block 4279 len 32768 data: 4 4 5 5 4 6 6 6
22047 222/300 Write block 884 len 8192 data: 5 3
22048 223/300 Read block 2181 len 24576
22049 224/300 Write block 4911 len 8192 data: 4 6
22050 225/300 Read block 4931 len 24576
22051 226/300 Write block 2457 len 36864 data: 4 3 3 3 3 3 3 2 3
22052 227/300 Flush
22053 228/300 Write block 2928 len 20480 data: 5 5 7 6 6
22054 229/300 Write block 3407 len 20480 data: 5 5 5 4 4
22055 230/300 Read block 1593 len 4096
22056 231/300 Read block 3235 len 40960
22057 232/300 Write block 451 len 16384 data: 3 3 3 4
22058 233/300 Write block 2 len 20480 data: 3 4 6 6 5
22059 234/300 Read block 2517 len 24576
22060 235/300 Write block 360 len 36864 data: 2 2 2 2 3 3 3 4 5
22061 236/300 Write block 4455 len 4096 data: 6
22062 237/300 Flush
22063 238/300 Flush
22064 239/300 Read block 2354 len 28672
22065 240/300 Read block 3147 len 8192
22066 241/300 Read block 1672 len 16384
22067 242/300 Read block 1373 len 4096
22068 243/300 Read block 1639 len 36864
22069 244/300 Write block 394 len 40960 data: 3 4 3 5 8 7 6 5 5 5
22070 245/300 Read block 3132 len 24576
22071 246/300 Write block 1929 len 36864 data: 3 3 3 3 5 4 4 6 6
22072 247/300 Read block 4841 len 40960
22073 248/300 Read block 4627 len 28672
22074 249/300 Write block 2256 len 16384 data: 4 4 4 5
22075 250/300 Write block 1768 len 24576 data: 5 5 4 4 5 5
22076 251/300 Read block 4497 len 32768
22077 252/300 Read block 2344 len 24576
22078 253/300 Read block 1186 len 28672
22079 254/300 Write block 3709 len 40960 data: 6 5 5 3 2 3 4 5 4 4
22080 255/300 Read block 4338 len 28672
22081 256/300 Read block 498 len 40960
22082 257/300 Read block 3769 len 36864
22083 258/300 Read block 919 len 20480
22084 259/300 Flush
22085 260/300 Write block 359 len 32768 data: 3 3 3 3 3 4 4 4
22086 261/300 Read block 2441 len 12288
22087 262/300 Write block 4589 len 28672 data: 3 3 4 4 4 6 5
22088 263/300 Read block 569 len 8192
22089 264/300 Read block 4968 len 8192
22090 265/300 Read block 263 len 32768
22091 266/300 Read block 2868 len 8192
22092 267/300 Read block 4480 len 12288
22093 268/300 Read block 361 len 4096
22094 269/300 Read block 1547 len 20480
22095 270/300 Read block 483 len 20480
22096 271/300 Write block 3411 len 40960 data: 5 3 3 4 4 3 3 3 3 3
22097 272/300 Read block 1959 len 36864
22098 273/300 Write block 3471 len 16384 data: 6 5 5 6
22099 274/300 Read block 3059 len 4096
22100 275/300 Write block 349 len 8192 data: 4 5
22101 276/300 Read block 3414 len 16384
22102 277/300 Flush
22103 278/300 Read block 4364 len 40960
22104 279/300 Read block 2009 len 28672
22105 280/300 Write block 2230 len 16384 data: 7 7 5 5
22106 281/300 Read block 4297 len 32768
22107 282/300 Write block 3423 len 4096 data: 3
22108 283/300 Write block 2179 len 4096 data: 6
22109 284/300 Read block 4142 len 16384
22110 285/300 Write block 4013 len 4096 data: 5
22111 286/300 Read block 1755 len 8192
22112 287/300 Read block 4681 len 20480
22113 288/300 Read block 2564 len 28672
22114 289/300 Read block 1016 len 12288
22115 290/300 Write block 993 len 36864 data: 4 4 4 3 2 3 3 3 3
22116 291/300 Read block 1863 len 16384
22117 292/300 Read block 4179 len 24576
22118 293/300 Read block 2018 len 24576
22119 294/300 Write block 2116 len 28672 data: 2 4 3 3 6 6 7
22120 295/300 Read block 2812 len 20480
22121 296/300 Write block 4547 len 32768 data: 5 5 4 3 3 3 3 3
22122 297/300 Write block 1578 len 8192 data: 5 6
22123 298/300 Read block 3174 len 12288
22124 299/300 Read block 1653 len 36864
22125 300/300 Flush
22126 301/300 Write block 3284 len 20480 data: 6 6 5 5 6
22127 Main task has work to do, go find it
22128 got dsc Start(0)
22129 start 0
22130 Jan 13 15:56:28.060 INFO request completed, latency_us: 206, response_code: 204, uri: /start/cid/0, method: POST, req_id: 83c7581f-1560-4db3-9a3b-d8717cd2ec2d, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
22131 Replay: started 0, returned:Ok(())
22132 ----------------------------------------------------------------
22133 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
22134 States: Offline Active Active
22135 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22136 6736 Acked Flush 0 Sent Done Done false
22137 6737 Acked Write 1 Sent Done Done false
22138 6738 Acked Write 2 Sent Done Done false
22139 6739 Acked Write 3 Sent Done Done false
22140 6740 Acked Read 3 Sent Done Done false
22141 6741 Acked Flush 0 Sent Done Done false
22142 6742 Acked Write 4 Sent Done Done false
22143 6743 Acked Read 2 Sent Done Done false
22144 6744 Acked Read 3 Sent Done Done false
22145 6745 Acked Write 2 Sent Done Done false
22146 6746 Acked Read 2 Sent Done Done false
22147 6747 Acked Write 7 Sent Done Done false
22148 6748 Acked Read 4 Sent Done Done false
22149 6749 Acked Read 3 Sent Done Done false
22150 6750 Acked Read 6 Sent Done Done false
22151 6751 Acked Write 9 Sent Done Done false
22152 6752 Acked Read 4 Sent Done Done false
22153 6753 Acked Read 2 Sent Done Done false
22154 6754 Acked Flush 0 Sent Done Done false
22155 6755 Acked Flush 0 Sent Done Done false
22156 6756 Acked Read 4 Sent Done Done false
22157 6757 Acked Write 8 Sent Done Done false
22158 6758 Acked Read 4 Sent Done Done false
22159 6759 Acked Write 8 Sent Done Done false
22160 6760 Acked Flush 0 Sent Done Done false
22161 6761 Acked Write 10 Sent Done Done false
22162 6762 Acked Read 2 Sent Done Done false
22163 6763 Acked Read 4 Sent Done Done false
22164 6764 Acked Write 5 Sent Done Done false
22165 6765 Acked Flush 0 Sent Done Done false
22166 6766 Acked Write 1 Sent Done Done false
22167 6767 Acked Read 3 Sent Done Done false
22168 6768 Acked Write 7 Sent Done Done false
22169 6769 Acked Read 6 Sent Done Done false
22170 6770 Acked Write 9 Sent Done Done false
22171 6771 Acked Read 7 Sent Done Done false
22172 6772 Acked Write 4 Sent Done Done false
22173 6773 Acked Flush 0 Sent Done Done false
22174 6774 Acked Read 4 Sent Done Done false
22175 6775 Acked Flush 0 Sent Done Done false
22176 6776 Acked Read 8 Sent Done Done false
22177 6777 Acked Read 9 Sent Done Done false
22178 6778 Acked Write 9 Sent Done Done false
22179 6779 Acked Write 2 Sent Done Done false
22180 6780 Acked Read 3 Sent Done Done false
22181 6781 Acked Read 1 Sent Done Done false
22182 6782 Acked Read 4 Sent Done Done false
22183 6783 Acked Read 4 Sent Done Done false
22184 6784 Acked Read 4 Sent Done Done false
22185 6785 Acked Flush 0 Sent Done Done false
22186 6786 Acked Read 1 Sent Done Done false
22187 6787 Acked Read 7 Sent Done Done false
22188 6788 Acked Flush 0 Sent Done Done false
22189 6789 Acked Write 5 Sent Done Done false
22190 6790 Acked Read 4 Sent Done Done false
22191 6791 Acked Flush 0 Sent Done Done false
22192 6792 Acked Write 6 Sent Done Done false
22193 6793 Acked Read 6 Sent Done Done false
22194 6794 Acked Write 8 Sent Done Done false
22195 6795 Acked Read 2 Sent Done Done false
22196 6796 Acked Write 3 Sent Done Done false
22197 6797 Acked Write 2 Sent Done Done false
22198 6798 Acked Read 3 Sent Done Done false
22199 6799 Acked Read 5 Sent Done Done false
22200 6800 Acked Flush 0 Sent Done Done false
22201 6801 Acked Read 5 Sent Done Done false
22202 6802 Acked Write 2 Sent Done Done false
22203 6803 Acked Read 5 Sent Done Done false
22204 6804 Acked Write 10 Sent Done Done false
22205 6805 Acked Read 7 Sent Done Done false
22206 6806 Acked Write 1 Sent Done Done false
22207 6807 Acked Write 4 Sent Done Done false
22208 6808 Acked Write 1 Sent Done Done false
22209 6809 Acked Write 3 Sent Done Done false
22210 6810 Acked Write 8 Sent Done Done false
22211 6811 Acked Read 5 Sent Done Done false
22212 6812 Acked Write 4 Sent Done Done false
22213 6813 Acked Read 1 Sent Done Done false
22214 6814 Acked Write 1 Sent Done Done false
22215 6815 Acked Read 10 Sent Done Done false
22216 6816 Acked Read 9 Sent Done Done false
22217 6817 Acked Write 7 Sent Done Done false
22218 6818 Acked Write 4 Sent Done Done false
22219 6819 Acked Write 9 Sent Done Done false
22220 6820 Acked Write 2 Sent Done Done false
22221 6821 Acked Write 2 Sent Done Done false
22222 6822 Acked Read 5 Sent Done Done false
22223 6823 Acked Write 3 Sent Done Done false
22224 6824 Acked Read 3 Sent Done Done false
22225 6825 Acked Write 5 Sent Done Done false
22226 6826 Acked Write 5 Sent Done Done false
22227 6827 Acked Write 6 Sent Done Done false
22228 6828 Acked Read 3 Sent Done Done false
22229 6829 Acked Write 9 Sent Done Done false
22230 6830 Acked Flush 0 Sent Done Done false
22231 6831 Acked Write 3 Sent Done Done false
22232 6832 Acked Write 3 Sent Done Done false
22233 6833 Acked Read 5 Sent Done Done false
22234 6834 Acked Read 6 Sent Done Done false
22235 6835 Acked Read 1 Sent Done Done false
22236 6836 Acked Write 1 Sent Done Done false
22237 6837 Acked Flush 0 Sent Done Done false
22238 6838 Acked Write 9 Sent Done Done false
22239 6839 Acked Read 10 Sent Done Done false
22240 6840 Acked Write 8 Sent Done Done false
22241 6841 Acked Write 5 Sent Done Done false
22242 6842 Acked Read 7 Sent Done Done false
22243 6843 Acked Read 6 Sent Done Done false
22244 6844 Acked Read 7 Sent Done Done false
22245 6845 Acked Write 6 Sent Done Done false
22246 6846 Acked Write 5 Sent Done Done false
22247 6847 Acked Write 6 Sent Done Done false
22248 6848 Acked Write 2 Sent Done Done false
22249 6849 Acked Write 5 Sent Done Done false
22250 6850 Acked Flush 0 Sent Done Done false
22251 6851 Acked Write 10 Sent Done Done false
22252 6852 Acked Flush 0 Sent Done Done false
22253 6853 Acked Write 2 Sent Done Done false
22254 6854 Acked Read 8 Sent Done Done false
22255 6855 Acked Read 9 Sent Done Done false
22256 6856 Acked Read 2 Sent Done Done false
22257 6857 Acked Read 2 Sent Done Done false
22258 6858 Acked Read 8 Sent Done Done false
22259 6859 Acked Read 6 Sent Done Done false
22260 6860 Acked Write 7 Sent Done Done false
22261 6861 Acked Write 4 Sent Done Done false
22262 6862 Acked Read 6 Sent Done Done false
22263 6863 Acked Read 2 Sent Done Done false
22264 6864 Acked Read 5 Sent Done Done false
22265 6865 Acked Write 2 Sent Done Done false
22266 6866 Acked Read 7 Sent Done Done false
22267 6867 Acked Read 4 Sent Done Done false
22268 6868 Acked Write 2 Sent Done Done false
22269 6869 Acked Read 8 Sent Done Done false
22270 6870 Acked Write 6 Sent Done Done false
22271 6871 Acked Read 1 Sent Done Done false
22272 6872 Acked Read 3 Sent Done Done false
22273 6873 Acked Flush 0 Sent Done Done false
22274 6874 Acked Read 2 Sent Done Done false
22275 6875 Acked Write 4 Sent Done Done false
22276 6876 Acked Read 10 Sent Done Done false
22277 6877 Acked Write 7 Sent Done Done false
22278 6878 Acked Write 7 Sent Done Done false
22279 6879 Acked Flush 0 Sent Done Done false
22280 6880 Acked Flush 0 Sent Done Done false
22281 6881 Acked Write 3 Sent Done Done false
22282 6882 Acked Write 6 Sent Done Done false
22283 6883 Acked Read 10 Sent Done Done false
22284 6884 Acked Read 2 Sent Done Done false
22285 6885 Acked Write 3 Sent Done Done false
22286 6886 Acked Write 2 Sent Done Done false
22287 6887 Acked Write 2 Sent Done Done false
22288 6888 Acked Flush 0 Sent Done Done false
22289 6889 Acked Write 8 Sent Done Done false
22290 6890 Acked Read 4 Sent Done Done false
22291 6891 Acked Read 4 Sent Done Done false
22292 6892 Acked Read 6 Sent Done Done false
22293 6893 Acked Read 7 Sent Done Done false
22294 6894 Acked Read 7 Sent Done Done false
22295 6895 Acked Write 5 Sent Done Done false
22296 6896 Acked Write 7 Sent Done Done false
22297 6897 Acked Write 2 Sent Done Done false
22298 6898 Acked Flush 0 Sent Done Done false
22299 6899 Acked Read 7 Sent Done Done false
22300 6900 Acked Read 10 Sent Done Done false
22301 6901 Acked Read 9 Sent Done Done false
22302 6902 Acked Read 8 Sent Done Done false
22303 6903 Acked Write 7 Sent Done Done false
22304 6904 Acked Write 2 Sent Done Done false
22305 6905 Acked Read 7 Sent Done Done false
22306 6906 Acked Write 5 Sent Done Done false
22307 6907 Acked Write 6 Sent Done Done false
22308 6908 Acked Read 9 Sent Done Done false
22309 6909 Acked Write 3 Sent Done Done false
22310 6910 Acked Write 5 Sent Done Done false
22311 6911 Acked Read 8 Sent Done Done false
22312 6912 Acked Read 8 Sent Done Done false
22313 6913 Acked Write 10 Sent Done Done false
22314 6914 Acked Read 6 Sent Done Done false
22315 6915 Acked Write 4 Sent Done Done false
22316 6916 Acked Read 7 Sent Done Done false
22317 6917 Acked Read 8 Sent Done Done false
22318 6918 Acked Write 5 Sent Done Done false
22319 6919 Acked Read 7 Sent Done Done false
22320 6920 Acked Flush 0 Sent Done Done false
22321 6921 Acked Read 1 Sent Done Done false
22322 6922 Acked Read 10 Sent Done Done false
22323 6923 Acked Write 5 Sent Done Done false
22324 6924 Acked Read 3 Sent Done Done false
22325 6925 Acked Write 2 Sent Done Done false
22326 6926 Acked Read 2 Sent Done Done false
22327 6927 Acked Write 4 Sent Done Done false
22328 6928 Acked Flush 0 Sent Done Done false
22329 6929 Acked Read 6 Sent Done Done false
22330 6930 Acked Read 3 Sent Done Done false
22331 6931 Acked Read 4 Sent Done Done false
22332 6932 Acked Write 7 Sent Done Done false
22333 6933 Acked Read 1 Sent Done Done false
22334 6934 Acked Write 8 Sent Done Done false
22335 6935 Acked Write 4 Sent Done Done false
22336 6936 Acked Flush 0 Sent Done Done false
22337 6937 Acked Read 6 Sent Done Done false
22338 6938 Acked Read 6 Sent Done Done false
22339 6939 Acked Read 1 Sent Done Done false
22340 6940 Acked Read 8 Sent Done Done false
22341 6941 Acked Flush 0 Sent Done Done false
22342 6942 Acked Write 9 Sent Done Done false
22343 6943 Acked Read 6 Sent Done Done false
22344 6944 Acked Write 3 Sent Done Done false
22345 6945 Acked Read 7 Sent Done Done false
22346 6946 Acked Write 7 Sent Done Done false
22347 6947 Acked Write 7 Sent Done Done false
22348 6948 Acked Read 7 Sent Done Done false
22349 6949 Acked Read 3 Sent Done Done false
22350 6950 Acked Flush 0 Sent Done Done false
22351 6951 Acked Write 6 Sent Done Done false
22352 6952 Acked Read 8 Sent Done Done false
22353 6953 Acked Read 5 Sent Done Done false
22354 6954 Acked Read 7 Sent Done Done false
22355 6955 Acked Read 5 Sent Done Done false
22356 6956 Acked Write 8 Sent Done Done false
22357 6957 Acked Write 2 Sent Done Done false
22358 6958 Acked Read 6 Sent Done Done false
22359 6959 Acked Write 2 Sent Done Done false
22360 6960 Acked Read 6 Sent Done Done false
22361 6961 Acked Write 9 Sent Done Done false
22362 6962 Acked Flush 0 Sent Done Done false
22363 6963 Acked Write 5 Sent Done Done false
22364 6964 Acked Write 5 Sent Done Done false
22365 6965 Acked Read 1 Sent Done Done false
22366 6966 Acked Read 10 Sent Done Done false
22367 6967 Acked Write 4 Sent Done Done false
22368 6968 Acked Write 5 Sent Done Done false
22369 6969 Acked Read 6 Sent Done Done false
22370 6970 Acked Write 9 Sent Done Done false
22371 6971 Acked Write 1 Sent Done Done false
22372 6972 Acked Flush 0 Sent Done Done false
22373 6973 Acked Flush 0 Sent Done Done false
22374 6974 Acked Read 7 Sent Done Done false
22375 6975 Acked Read 2 Sent Done Done false
22376 6976 Acked Read 4 Sent Done Done false
22377 6977 Acked Read 1 Sent Done Done false
22378 6978 Acked Read 9 Sent Done Done false
22379 6979 Acked Write 10 Sent Done Done false
22380 6980 Acked Read 6 Sent Done Done false
22381 6981 Acked Write 9 Sent Done Done false
22382 6982 Acked Read 10 Sent Done Done false
22383 6983 Acked Read 7 Sent Done Done false
22384 6984 Acked Write 4 Sent Done Done false
22385 6985 Acked Write 6 Sent Done Done false
22386 6986 Acked Read 8 Sent Done Done false
22387 6987 Acked Read 6 Sent Done Done false
22388 6988 Acked Read 7 Sent Done Done false
22389 6989 Acked Write 10 Sent Done Done false
22390 6990 Acked Read 7 Sent Done Done false
22391 6991 Acked Read 10 Sent Done Done false
22392 6992 Acked Read 9 Sent Done Done false
22393 6993 Acked Read 5 Sent Done Done false
22394 6994 Acked Flush 0 Sent Done Done false
22395 6995 Acked Write 8 Sent Done Done false
22396 6996 Acked Read 3 Sent Done Done false
22397 6997 Acked Write 7 Sent Done Done false
22398 6998 Acked Read 2 Sent Done Done false
22399 6999 Acked Read 2 Sent Done Done false
22400 7000 Acked Read 8 Sent Done Done false
22401 7001 Acked Read 2 Sent Done Done false
22402 7002 Acked Read 3 Sent Done Done false
22403 7003 Acked Read 1 Sent Done Done false
22404 7004 Acked Read 5 Sent Done Done false
22405 7005 Acked Read 5 Sent Done Done false
22406 7006 Acked Write 10 Sent Done Done false
22407 7007 Acked Read 9 Sent Done Done false
22408 7008 Acked Write 4 Sent Done Done false
22409 7009 Acked Read 1 Sent Done Done false
22410 7010 Acked Write 2 Sent Done Done false
22411 7011 Acked Read 4 Sent Done Done false
22412 7012 Acked Flush 0 Sent Done Done false
22413 7013 Acked Read 10 Sent Done Done false
22414 7014 Acked Read 7 Sent Done Done false
22415 7015 Acked Write 4 Sent Done Done false
22416 7016 Acked Read 8 Sent Done Done false
22417 7017 Acked Write 1 Sent Done Done false
22418 7018 Acked Write 1 Sent Done Done false
22419 7019 Acked Read 4 Sent Done Done false
22420 7020 Acked Write 1 Sent Done Done false
22421 7021 Acked Read 2 Sent Done Done false
22422 7022 Acked Read 5 Sent Done Done false
22423 7023 Acked Read 7 Sent Done Done false
22424 7024 Acked Read 3 Sent Done Done false
22425 7025 Acked Write 9 Sent Done Done false
22426 7026 Acked Read 4 Sent Done Done false
22427 7027 Acked Read 6 Sent Done Done false
22428 7028 Acked Read 6 Sent Done Done false
22429 7029 Acked Write 7 Sent Done Done false
22430 7030 Acked Read 5 Sent Done Done false
22431 7031 Acked Write 8 Sent Done Done false
22432 7032 Acked Write 2 Sent Done Done false
22433 7033 Acked Read 3 Sent Done Done false
22434 7034 Acked Read 9 Sent Done Done false
22435 7035 Acked Flush 0 Sent Done Done false
22436 7036 Acked Write 5 Sent Done Done false
22437 STATES DS:0 DS:1 DS:2 TOTAL
22438 Sent 301 0 0 301
22439 Done 0 301 301 602
22440 Skipped 0 0 0 0
22441 Error 0 0 0 0
22442 Last Flush: 6735 7035 7035
22443 Downstairs last five completed: 6735 6734 6733 6732 6731
22444 Upstairs last five completed: 7036 7035 7034 7033 7032
22445 CLIENT: Up:0 ds:301 act:2
22446 [8810] got start action
22447 [8810] I am going to restart
22448 Starting downstairs at port 8810
22449 Make output file at "/tmp/dsc/downstairs-8810.txt"
22450 [8810][0] reports Starting
22451 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1222)
22452 [8810][0] reports Running
22453 ----------------------------------------------------------------
22454 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22455 States: Offline Active Active
22456 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22457 6736 Acked Flush 0 Sent Done Done false
22458 6737 Acked Write 1 Sent Done Done false
22459 6738 Acked Write 2 Sent Done Done false
22460 6739 Acked Write 3 Sent Done Done false
22461 6740 Acked Read 3 Sent Done Done false
22462 6741 Acked Flush 0 Sent Done Done false
22463 6742 Acked Write 4 Sent Done Done false
22464 6743 Acked Read 2 Sent Done Done false
22465 6744 Acked Read 3 Sent Done Done false
22466 6745 Acked Write 2 Sent Done Done false
22467 6746 Acked Read 2 Sent Done Done false
22468 6747 Acked Write 7 Sent Done Done false
22469 6748 Acked Read 4 Sent Done Done false
22470 6749 Acked Read 3 Sent Done Done false
22471 6750 Acked Read 6 Sent Done Done false
22472 6751 Acked Write 9 Sent Done Done false
22473 6752 Acked Read 4 Sent Done Done false
22474 6753 Acked Read 2 Sent Done Done false
22475 6754 Acked Flush 0 Sent Done Done false
22476 6755 Acked Flush 0 Sent Done Done false
22477 6756 Acked Read 4 Sent Done Done false
22478 6757 Acked Write 8 Sent Done Done false
22479 6758 Acked Read 4 Sent Done Done false
22480 6759 Acked Write 8 Sent Done Done false
22481 6760 Acked Flush 0 Sent Done Done false
22482 6761 Acked Write 10 Sent Done Done false
22483 6762 Acked Read 2 Sent Done Done false
22484 6763 Acked Read 4 Sent Done Done false
22485 6764 Acked Write 5 Sent Done Done false
22486 6765 Acked Flush 0 Sent Done Done false
22487 6766 Acked Write 1 Sent Done Done false
22488 6767 Acked Read 3 Sent Done Done false
22489 6768 Acked Write 7 Sent Done Done false
22490 6769 Acked Read 6 Sent Done Done false
22491 6770 Acked Write 9 Sent Done Done false
22492 6771 Acked Read 7 Sent Done Done false
22493 6772 Acked Write 4 Sent Done Done false
22494 6773 Acked Flush 0 Sent Done Done false
22495 6774 Acked Read 4 Sent Done Done false
22496 6775 Acked Flush 0 Sent Done Done false
22497 6776 Acked Read 8 Sent Done Done false
22498 6777 Acked Read 9 Sent Done Done false
22499 6778 Acked Write 9 Sent Done Done false
22500 6779 Acked Write 2 Sent Done Done false
22501 6780 Acked Read 3 Sent Done Done false
22502 6781 Acked Read 1 Sent Done Done false
22503 6782 Acked Read 4 Sent Done Done false
22504 6783 Acked Read 4 Sent Done Done false
22505 6784 Acked Read 4 Sent Done Done false
22506 6785 Acked Flush 0 Sent Done Done false
22507 6786 Acked Read 1 Sent Done Done false
22508 6787 Acked Read 7 Sent Done Done false
22509 6788 Acked Flush 0 Sent Done Done false
22510 6789 Acked Write 5 Sent Done Done false
22511 6790 Acked Read 4 Sent Done Done false
22512 6791 Acked Flush 0 Sent Done Done false
22513 6792 Acked Write 6 Sent Done Done false
22514 6793 Acked Read 6 Sent Done Done false
22515 6794 Acked Write 8 Sent Done Done false
22516 6795 Acked Read 2 Sent Done Done false
22517 6796 Acked Write 3 Sent Done Done false
22518 6797 Acked Write 2 Sent Done Done false
22519 6798 Acked Read 3 Sent Done Done false
22520 6799 Acked Read 5 Sent Done Done false
22521 6800 Acked Flush 0 Sent Done Done false
22522 6801 Acked Read 5 Sent Done Done false
22523 6802 Acked Write 2 Sent Done Done false
22524 6803 Acked Read 5 Sent Done Done false
22525 6804 Acked Write 10 Sent Done Done false
22526 6805 Acked Read 7 Sent Done Done false
22527 6806 Acked Write 1 Sent Done Done false
22528 6807 Acked Write 4 Sent Done Done false
22529 6808 Acked Write 1 Sent Done Done false
22530 6809 Acked Write 3 Sent Done Done false
22531 6810 Acked Write 8 Sent Done Done false
22532 6811 Acked Read 5 Sent Done Done false
22533 6812 Acked Write 4 Sent Done Done false
22534 6813 Acked Read 1 Sent Done Done false
22535 6814 Acked Write 1 Sent Done Done false
22536 6815 Acked Read 10 Sent Done Done false
22537 6816 Acked Read 9 Sent Done Done false
22538 6817 Acked Write 7 Sent Done Done false
22539 6818 Acked Write 4 Sent Done Done false
22540 6819 Acked Write 9 Sent Done Done false
22541 6820 Acked Write 2 Sent Done Done false
22542 6821 Acked Write 2 Sent Done Done false
22543 6822 Acked Read 5 Sent Done Done false
22544 6823 Acked Write 3 Sent Done Done false
22545 6824 Acked Read 3 Sent Done Done false
22546 6825 Acked Write 5 Sent Done Done false
22547 6826 Acked Write 5 Sent Done Done false
22548 6827 Acked Write 6 Sent Done Done false
22549 6828 Acked Read 3 Sent Done Done false
22550 6829 Acked Write 9 Sent Done Done false
22551 6830 Acked Flush 0 Sent Done Done false
22552 6831 Acked Write 3 Sent Done Done false
22553 6832 Acked Write 3 Sent Done Done false
22554 6833 Acked Read 5 Sent Done Done false
22555 6834 Acked Read 6 Sent Done Done false
22556 6835 Acked Read 1 Sent Done Done false
22557 6836 Acked Write 1 Sent Done Done false
22558 6837 Acked Flush 0 Sent Done Done false
22559 6838 Acked Write 9 Sent Done Done false
22560 6839 Acked Read 10 Sent Done Done false
22561 6840 Acked Write 8 Sent Done Done false
22562 6841 Acked Write 5 Sent Done Done false
22563 6842 Acked Read 7 Sent Done Done false
22564 6843 Acked Read 6 Sent Done Done false
22565 6844 Acked Read 7 Sent Done Done false
22566 6845 Acked Write 6 Sent Done Done false
22567 6846 Acked Write 5 Sent Done Done false
22568 6847 Acked Write 6 Sent Done Done false
22569 6848 Acked Write 2 Sent Done Done false
22570 6849 Acked Write 5 Sent Done Done false
22571 6850 Acked Flush 0 Sent Done Done false
22572 6851 Acked Write 10 Sent Done Done false
22573 6852 Acked Flush 0 Sent Done Done false
22574 6853 Acked Write 2 Sent Done Done false
22575 6854 Acked Read 8 Sent Done Done false
22576 6855 Acked Read 9 Sent Done Done false
22577 6856 Acked Read 2 Sent Done Done false
22578 6857 Acked Read 2 Sent Done Done false
22579 6858 Acked Read 8 Sent Done Done false
22580 6859 Acked Read 6 Sent Done Done false
22581 6860 Acked Write 7 Sent Done Done false
22582 6861 Acked Write 4 Sent Done Done false
22583 6862 Acked Read 6 Sent Done Done false
22584 6863 Acked Read 2 Sent Done Done false
22585 6864 Acked Read 5 Sent Done Done false
22586 6865 Acked Write 2 Sent Done Done false
22587 6866 Acked Read 7 Sent Done Done false
22588 6867 Acked Read 4 Sent Done Done false
22589 6868 Acked Write 2 Sent Done Done false
22590 6869 Acked Read 8 Sent Done Done false
22591 6870 Acked Write 6 Sent Done Done false
22592 6871 Acked Read 1 Sent Done Done false
22593 6872 Acked Read 3 Sent Done Done false
22594 6873 Acked Flush 0 Sent Done Done false
22595 6874 Acked Read 2 Sent Done Done false
22596 6875 Acked Write 4 Sent Done Done false
22597 6876 Acked Read 10 Sent Done Done false
22598 6877 Acked Write 7 Sent Done Done false
22599 6878 Acked Write 7 Sent Done Done false
22600 6879 Acked Flush 0 Sent Done Done false
22601 6880 Acked Flush 0 Sent Done Done false
22602 6881 Acked Write 3 Sent Done Done false
22603 6882 Acked Write 6 Sent Done Done false
22604 6883 Acked Read 10 Sent Done Done false
22605 6884 Acked Read 2 Sent Done Done false
22606 6885 Acked Write 3 Sent Done Done false
22607 6886 Acked Write 2 Sent Done Done false
22608 6887 Acked Write 2 Sent Done Done false
22609 6888 Acked Flush 0 Sent Done Done false
22610 6889 Acked Write 8 Sent Done Done false
22611 6890 Acked Read 4 Sent Done Done false
22612 6891 Acked Read 4 Sent Done Done false
22613 6892 Acked Read 6 Sent Done Done false
22614 6893 Acked Read 7 Sent Done Done false
22615 6894 Acked Read 7 Sent Done Done false
22616 6895 Acked Write 5 Sent Done Done false
22617 6896 Acked Write 7 Sent Done Done false
22618 6897 Acked Write 2 Sent Done Done false
22619 6898 Acked Flush 0 Sent Done Done false
22620 6899 Acked Read 7 Sent Done Done false
22621 6900 Acked Read 10 Sent Done Done false
22622 6901 Acked Read 9 Sent Done Done false
22623 6902 Acked Read 8 Sent Done Done false
22624 6903 Acked Write 7 Sent Done Done false
22625 6904 Acked Write 2 Sent Done Done false
22626 6905 Acked Read 7 Sent Done Done false
22627 6906 Acked Write 5 Sent Done Done false
22628 6907 Acked Write 6 Sent Done Done false
22629 6908 Acked Read 9 Sent Done Done false
22630 6909 Acked Write 3 Sent Done Done false
22631 6910 Acked Write 5 Sent Done Done false
22632 6911 Acked Read 8 Sent Done Done false
22633 6912 Acked Read 8 Sent Done Done false
22634 6913 Acked Write 10 Sent Done Done false
22635 6914 Acked Read 6 Sent Done Done false
22636 6915 Acked Write 4 Sent Done Done false
22637 6916 Acked Read 7 Sent Done Done false
22638 6917 Acked Read 8 Sent Done Done false
22639 6918 Acked Write 5 Sent Done Done false
22640 6919 Acked Read 7 Sent Done Done false
22641 6920 Acked Flush 0 Sent Done Done false
22642 6921 Acked Read 1 Sent Done Done false
22643 6922 Acked Read 10 Sent Done Done false
22644 6923 Acked Write 5 Sent Done Done false
22645 6924 Acked Read 3 Sent Done Done false
22646 6925 Acked Write 2 Sent Done Done false
22647 6926 Acked Read 2 Sent Done Done false
22648 6927 Acked Write 4 Sent Done Done false
22649 6928 Acked Flush 0 Sent Done Done false
22650 6929 Acked Read 6 Sent Done Done false
22651 6930 Acked Read 3 Sent Done Done false
22652 6931 Acked Read 4 Sent Done Done false
22653 6932 Acked Write 7 Sent Done Done false
22654 6933 Acked Read 1 Sent Done Done false
22655 6934 Acked Write 8 Sent Done Done false
22656 6935 Acked Write 4 Sent Done Done false
22657 6936 Acked Flush 0 Sent Done Done false
22658 6937 Acked Read 6 Sent Done Done false
22659 6938 Acked Read 6 Sent Done Done false
22660 6939 Acked Read 1 Sent Done Done false
22661 6940 Acked Read 8 Sent Done Done false
22662 6941 Acked Flush 0 Sent Done Done false
22663 6942 Acked Write 9 Sent Done Done false
22664 6943 Acked Read 6 Sent Done Done false
22665 6944 Acked Write 3 Sent Done Done false
22666 6945 Acked Read 7 Sent Done Done false
22667 6946 Acked Write 7 Sent Done Done false
22668 6947 Acked Write 7 Sent Done Done false
22669 6948 Acked Read 7 Sent Done Done false
22670 6949 Acked Read 3 Sent Done Done false
22671 6950 Acked Flush 0 Sent Done Done false
22672 6951 Acked Write 6 Sent Done Done false
22673 6952 Acked Read 8 Sent Done Done false
22674 6953 Acked Read 5 Sent Done Done false
22675 6954 Acked Read 7 Sent Done Done false
22676 6955 Acked Read 5 Sent Done Done false
22677 6956 Acked Write 8 Sent Done Done false
22678 6957 Acked Write 2 Sent Done Done false
22679 6958 Acked Read 6 Sent Done Done false
22680 6959 Acked Write 2 Sent Done Done false
22681 6960 Acked Read 6 Sent Done Done false
22682 6961 Acked Write 9 Sent Done Done false
22683 6962 Acked Flush 0 Sent Done Done false
22684 6963 Acked Write 5 Sent Done Done false
22685 6964 Acked Write 5 Sent Done Done false
22686 6965 Acked Read 1 Sent Done Done false
22687 6966 Acked Read 10 Sent Done Done false
22688 6967 Acked Write 4 Sent Done Done false
22689 6968 Acked Write 5 Sent Done Done false
22690 6969 Acked Read 6 Sent Done Done false
22691 6970 Acked Write 9 Sent Done Done false
22692 6971 Acked Write 1 Sent Done Done false
22693 6972 Acked Flush 0 Sent Done Done false
22694 6973 Acked Flush 0 Sent Done Done false
22695 6974 Acked Read 7 Sent Done Done false
22696 6975 Acked Read 2 Sent Done Done false
22697 6976 Acked Read 4 Sent Done Done false
22698 6977 Acked Read 1 Sent Done Done false
22699 6978 Acked Read 9 Sent Done Done false
22700 6979 Acked Write 10 Sent Done Done false
22701 6980 Acked Read 6 Sent Done Done false
22702 6981 Acked Write 9 Sent Done Done false
22703 6982 Acked Read 10 Sent Done Done false
22704 6983 Acked Read 7 Sent Done Done false
22705 6984 Acked Write 4 Sent Done Done false
22706 6985 Acked Write 6 Sent Done Done false
22707 6986 Acked Read 8 Sent Done Done false
22708 6987 Acked Read 6 Sent Done Done false
22709 6988 Acked Read 7 Sent Done Done false
22710 6989 Acked Write 10 Sent Done Done false
22711 6990 Acked Read 7 Sent Done Done false
22712 6991 Acked Read 10 Sent Done Done false
22713 6992 Acked Read 9 Sent Done Done false
22714 6993 Acked Read 5 Sent Done Done false
22715 6994 Acked Flush 0 Sent Done Done false
22716 6995 Acked Write 8 Sent Done Done false
22717 6996 Acked Read 3 Sent Done Done false
22718 6997 Acked Write 7 Sent Done Done false
22719 6998 Acked Read 2 Sent Done Done false
22720 6999 Acked Read 2 Sent Done Done false
22721 7000 Acked Read 8 Sent Done Done false
22722 7001 Acked Read 2 Sent Done Done false
22723 7002 Acked Read 3 Sent Done Done false
22724 7003 Acked Read 1 Sent Done Done false
22725 7004 Acked Read 5 Sent Done Done false
22726 7005 Acked Read 5 Sent Done Done false
22727 7006 Acked Write 10 Sent Done Done false
22728 7007 Acked Read 9 Sent Done Done false
22729 7008 Acked Write 4 Sent Done Done false
22730 7009 Acked Read 1 Sent Done Done false
22731 7010 Acked Write 2 Sent Done Done false
22732 7011 Acked Read 4 Sent Done Done false
22733 7012 Acked Flush 0 Sent Done Done false
22734 7013 Acked Read 10 Sent Done Done false
22735 7014 Acked Read 7 Sent Done Done false
22736 7015 Acked Write 4 Sent Done Done false
22737 7016 Acked Read 8 Sent Done Done false
22738 7017 Acked Write 1 Sent Done Done false
22739 7018 Acked Write 1 Sent Done Done false
22740 7019 Acked Read 4 Sent Done Done false
22741 7020 Acked Write 1 Sent Done Done false
22742 7021 Acked Read 2 Sent Done Done false
22743 7022 Acked Read 5 Sent Done Done false
22744 7023 Acked Read 7 Sent Done Done false
22745 7024 Acked Read 3 Sent Done Done false
22746 7025 Acked Write 9 Sent Done Done false
22747 7026 Acked Read 4 Sent Done Done false
22748 7027 Acked Read 6 Sent Done Done false
22749 7028 Acked Read 6 Sent Done Done false
22750 7029 Acked Write 7 Sent Done Done false
22751 7030 Acked Read 5 Sent Done Done false
22752 7031 Acked Write 8 Sent Done Done false
22753 7032 Acked Write 2 Sent Done Done false
22754 7033 Acked Read 3 Sent Done Done false
22755 7034 Acked Read 9 Sent Done Done false
22756 7035 Acked Flush 0 Sent Done Done false
22757 7036 Acked Write 5 Sent Done Done false
22758 7037 Acked Flush 0 Sent Done Done false
22759 STATES DS:0 DS:1 DS:2 TOTAL
22760 Sent 302 0 0 302
22761 Done 0 302 302 604
22762 Skipped 0 0 0 0
22763 Error 0 0 0 0
22764 Last Flush: 6735 7037 7037
22765 Downstairs last five completed: 6735 6734 6733 6732 6731
22766 Upstairs last five completed: 7037 7036 7035 7034 7033
22767 CLIENT: Up:0 ds:302 act:2
227682025-01-13T15:56:33.823ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
227692025-01-13T15:56:33.823ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41457) = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
227702025-01-13T15:56:33.824ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
227712025-01-13T15:56:33.825ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
227722025-01-13T15:56:33.825ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
227732025-01-13T15:56:33.825ZINFOcrucible: send last flush ID to this DS: 6735 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
227742025-01-13T15:56:33.825ZINFOcrucible: Replied this last flush ID: 6735 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
227752025-01-13T15:56:33.825ZINFOcrucible: [0] client re-new 302 jobs since flush 6735 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
227762025-01-13T15:56:33.825ZINFOcrucible: [0] Replayed 302 jobs since flush: 6735 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
22777 ----------------------------------------------------------------
22778 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
22779 Downstairs last five completed: 7037 7036 7035 7034 7033
22780 Upstairs last five completed: 7037 7036 7035 7034 7033
22781 CLIENT: Up:0 ds:0 act:3
22782 Replay: All jobs finished
22783 Main task has work to do, go find it
22784 got dsc Stop(0)
22785 stop 0
22786 Jan 13 15:56:36.066[8810] Got stop action so:false kr:false
22787 INFO request completed, latency_us: 325, response_code: 204, uri: /stop/cid/0, method: POST, req_id: e29e64c9-63d2-4e91-abb8-ddfd48e8a781, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
227882025-01-13T15:56:36.067ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
22789 Jan 13 15:56:36.067 INFO request completed, latency_us: 143, response_code: 200, uri: /state/cid/0, method: GET, req_id: 5e19c839-a388-4fc9-a2af-922648c7e9e6, remote_addr{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:56:36.067838776Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
22790 : 127.0.0.1:52422, {"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:56:36.06787595Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
22791 local_addr: 127.0.0.1:9998
227922025-01-13T15:56:36.067ZINFOcrucible: client task is exiting = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
22793 [8810] Exited with: ExitStatus(unix_wait_status(9))
22794 [8810][0] reports Exit
22795 Jan 13 15:56:40.070 INFO request completed, latency_us: 186, response_code: 200, uri: /state/cid/0, method: GET, req_id: 218e48b3-6c73-427e-a700-edf8d152c758, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
22796 001/300 Write block 868 len 28672 data: 6 7 6 4 4 4 5
22797 002/300 Write block 2065 len 4096 data: 3
22798 003/300 Flush
22799 004/300 Read block 3543 len 20480
22800 005/300 Read block 3697 len 32768
22801 006/300 Write block 3408 len 32768 data: 6 6 5 6 4 4 5 5
22802 007/300 Flush
22803 008/300 Read block 3065 len 24576
22804 009/300 Read block 4099 len 28672
22805 010/300 Read block 3112 len 24576
22806 011/300 Read block 2337 len 8192
22807 012/300 Read block 49 len 16384
22808 013/300 Read block 1658 len 36864
22809 014/300 Read block 4927 len 8192
22810 015/300 Read block 718 len 8192
22811 016/300 Write block 3764 len 32768 data: 8 8 8 9 7 7 7 8
22812 017/300 Read block 288 len 24576
22813 018/300 Write block 1027 len 36864 data: 7 6 5 6 4 4 4 4 5
22814 019/300 Write block 4591 len 8192 data: 5 5
22815 020/300 Read block 4646 len 28672
22816 021/300 Read block 1344 len 24576
22817 022/300 Read block 4983 len 36864
22818 023/300 Read block 2354 len 24576
22819 024/300 Write block 4511 len 8192 data: 6 6
22820 025/300 Read block 2842 len 40960
22821 026/300 Read block 3813 len 12288
22822 027/300 Read block 4283 len 8192
22823 028/300 Flush
22824 029/300 Flush
22825 030/300 Write block 135 len 16384 data: 6 4 5 4
22826 031/300 Write block 2150 len 36864 data: 8 6 5 4 4 5 3 2 2
22827 032/300 Read block 2382 len 28672
22828 033/300 Write block 2422 len 40960 data: 3 3 3 3 3 3 3 3 2 2
22829 034/300 Write block 4834 len 36864 data: 5 6 6 6 4 4 3 6 5
22830 035/300 Write block 157 len 8192 data: 4 4
22831 036/300 Write block 465 len 16384 data: 5 5 4 4
22832 037/300 Read block 4414 len 36864
22833 038/300 Write block 230 len 36864 data: 4 3 2 2 2 2 2 3 4
22834 039/300 Write block 3905 len 28672 data: 3 3 3 3 2 2 3
22835 040/300 Write block 2081 len 16384 data: 6 7 9 7
22836 041/300 Write block 4838 len 32768 data: 5 5 4 7 6 5 5 5
22837 042/300 Read block 4958 len 8192
22838 043/300 Read block 2723 len 8192
22839 044/300 Read block 3292 len 28672
22840 045/300 Write block 240 len 4096 data: 6
22841 046/300 Write block 283 len 40960 data: 6 7 9 9 9 8 6 7 7 5
22842 047/300 Write block 360 len 28672 data: 4 4 4 4 5 5 5
22843 048/300 Write block 3244 len 24576 data: 5 3 3 3 3 3
22844 049/300 Read block 1135 len 36864
22845 050/300 Read block 4290 len 4096
22846 051/300 Read block 4453 len 32768
22847 052/300 Read block 4980 len 20480
22848 053/300 Flush
22849 054/300 Write block 2899 len 40960 data: 4 6 7 7 5 4 4 3 4 6
22850 055/300 Write block 1911 len 40960 data: 3 2 2 2 3 3 3 3 3 5
22851 056/300 Read block 1066 len 12288
22852 057/300 Read block 1953 len 20480
22853 058/300 Read block 4668 len 20480
22854 059/300 Write block 475 len 28672 data: 6 6 7 7 7 5 5
22855 060/300 Write block 2604 len 40960 data: 7 7 7 7 6 6 6 6 6 5
22856 061/300 Read block 1401 len 24576
22857 062/300 Flush
22858 063/300 Write block 3839 len 12288 data: 5 5 5
22859 064/300 Read block 4359 len 40960
22860 065/300 Write block 1188 len 16384 data: 4 4 5 5
22861 066/300 Write block 1098 len 40960 data: 3 4 6 6 6 5 5 4 4 5
22862 067/300 Read block 283 len 20480
22863 068/300 Write block 2123 len 8192 data: 6 5
22864 069/300 Write block 791 len 28672 data: 2 2 4 4 3 3 4
22865 070/300 Read block 4353 len 28672
22866 071/300 Write block 803 len 8192 data: 5 5
22867 072/300 Write block 2563 len 24576 data: 7 6 6 5 4 3
22868 073/300 Write block 3329 len 32768 data: 5 4 3 3 3 2 2 3
22869 074/300 Write block 3466 len 40960 data: 4 4 3 5 5 7 6 6 7 6
22870 075/300 Write block 2230 len 20480 data: 8 8 6 6 3
22871 076/300 Read block 4098 len 8192
22872 077/300 Write block 4179 len 28672 data: 4 4 6 5 5 5 7
22873 078/300 Read block 2250 len 16384
22874 079/300 Read block 2906 len 24576
22875 080/300 Write block 492 len 4096 data: 2
22876 081/300 Write block 1423 len 36864 data: 4 5 5 6 6 7 7 6 5
22877 082/300 Flush
22878 083/300 Read block 2583 len 24576
22879 084/300 Write block 2244 len 12288 data: 3 4 5
22880 085/300 Write block 2824 len 16384 data: 6 7 7 6
22881 086/300 Flush
22882 087/300 Flush
22883 088/300 Read block 707 len 24576
22884 089/300 Read block 3400 len 8192
22885 090/300 Read block 856 len 28672
22886 091/300 Write block 4674 len 16384 data: 7 6 4 4
22887 092/300 Read block 562 len 20480
22888 093/300 Read block 3895 len 28672
22889 094/300 Read block 3838 len 28672
22890 095/300 Write block 1147 len 28672 data: 7 7 6 5 5 3 2
22891 096/300 Read block 4055 len 4096
22892 097/300 Read block 2963 len 8192
22893 098/300 Write block 1012 len 24576 data: 3 2 3 2 4 3
22894 099/300 Read block 2391 len 4096
22895 100/300 Read block 4821 len 16384
22896 101/300 Read block 4109 len 40960
22897 102/300 Write block 1483 len 4096 data: 2
22898 103/300 Read block 3272 len 8192
22899 104/300 Flush
22900 105/300 Flush
22901 106/300 Flush
22902 107/300 Read block 4955 len 36864
22903 108/300 Write block 2658 len 28672 data: 4 5 4 4 5 5 5
22904 109/300 Read block 341 len 12288
22905 110/300 Read block 2032 len 40960
22906 111/300 Write block 39 len 36864 data: 2 2 3 3 4 4 4 3 3
22907 112/300 Write block 4661 len 40960 data: 5 4 5 5 5 5 5 5 4 4
22908 113/300 Read block 825 len 8192
22909 114/300 Read block 1746 len 24576
22910 115/300 Read block 59 len 20480
22911 116/300 Write block 3984 len 32768 data: 8 7 7 6 6 5 5 6
22912 117/300 Flush
22913 118/300 Read block 2981 len 12288
22914 119/300 Read block 4162 len 24576
22915 120/300 Write block 835 len 16384 data: 6 6 6 7
22916 121/300 Write block 1603 len 16384 data: 2 4 5 5
22917 122/300 Flush
22918 123/300 Write block 783 len 32768 data: 7 6 5 5 3 2 2 2
22919 124/300 Flush
22920 125/300 Write block 211 len 16384 data: 7 7 7 7
22921 126/300 Flush
22922 127/300 Read block 4968 len 32768
22923 128/300 Write block 1808 len 4096 data: 4
22924 129/300 Write block 2310 len 24576 data: 6 5 3 3 4 4
22925 130/300 Read block 4857 len 20480
22926 131/300 Read block 147 len 32768
22927 132/300 Read block 2921 len 36864
22928 133/300 Write block 3961 len 28672 data: 5 5 6 5 5 4 4
22929 134/300 Write block 79 len 36864 data: 8 8 6 6 5 5 3 4 3
22930 135/300 Write block 770 len 20480 data: 7 5 4 5 5
22931 136/300 Read block 858 len 16384
22932 137/300 Flush
22933 138/300 Read block 448 len 40960
22934 139/300 Read block 2323 len 20480
22935 140/300 Write block 3045 len 12288 data: 5 4 4
22936 141/300 Flush
22937 142/300 Read block 1020 len 4096
22938 143/300 Read block 1923 len 28672
22939 144/300 Flush
22940 145/300 Read block 1301 len 8192
22941 146/300 Flush
22942 147/300 Write block 2085 len 20480 data: 6 5 5 4 4
22943 148/300 Write block 2448 len 24576 data: 4 3 3 3 3 3
22944 149/300 Read block 2481 len 36864
22945 150/300 Read block 4056 len 36864
22946 151/300 Write block 4924 len 40960 data: 2 2 3 4 5 6 7 9 9 9
22947 152/300 Write block 4094 len 36864 data: 3 3 2 2 2 2 2 4 4
22948 153/300 Write block 1620 len 36864 data: 6 9 8 7 7 5 5 5 5
22949 154/300 Write block 1930 len 40960 data: 4 4 4 6 5 5 7 7 5 4
22950 155/300 Write block 1451 len 20480 data: 2 2 3 3 2
22951 156/300 Write block 4113 len 8192 data: 6 5
22952 157/300 Read block 3280 len 32768
22953 158/300 Write block 703 len 32768 data: 5 4 6 7 7 6 5 5
22954 159/300 Read block 535 len 24576
22955 160/300 Read block 1984 len 24576
22956 161/300 Write block 1302 len 28672 data: 5 5 5 4 4 4 3
22957 162/300 Write block 2052 len 36864 data: 7 5 5 3 3 3 3 3 3
22958 163/300 Flush
22959 164/300 Read block 2402 len 36864
22960 165/300 Write block 3225 len 16384 data: 7 10 10 8
22961 166/300 Write block 4887 len 8192 data: 7 8
22962 167/300 Flush
22963 168/300 Write block 2633 len 40960 data: 2 3 4 5 6 6 6 6 7 6
22964 169/300 Write block 3743 len 16384 data: 3 4 4 4
22965 170/300 Read block 733 len 12288
22966 171/300 Read block 4435 len 36864
22967 172/300 Write block 898 len 8192 data: 4 3
22968 173/300 Read block 525 len 20480
22969 174/300 Read block 2477 len 20480
22970 175/300 Write block 49 len 36864 data: 5 5 5 4 3 3 4 5 4
22971 176/300 Read block 3387 len 28672
22972 177/300 Read block 3443 len 4096
22973 178/300 Read block 1690 len 28672
22974 179/300 Read block 3353 len 28672
22975 180/300 Write block 3161 len 16384 data: 3 4 4 4
22976 181/300 Write block 2837 len 32768 data: 2 2 4 3 3 3 4 5
22977 182/300 Read block 4119 len 12288
22978 183/300 Read block 2784 len 28672
22979 184/300 Read block 1019 len 40960
22980 185/300 Read block 941 len 16384
22981 186/300 Flush
22982 187/300 Write block 3293 len 16384 data: 4 4 2 2
22983 188/300 Read block 731 len 8192
22984 189/300 Flush
22985 190/300 Write block 1912 len 36864 data: 3 3 3 4 4 4 4 4 6
22986 191/300 Write block 4135 len 4096 data: 2
22987 192/300 Write block 3269 len 28672 data: 8 9 8 8 9 8 8
22988 193/300 Read block 3962 len 12288
22989 194/300 Read block 2424 len 40960
22990 195/300 Write block 1398 len 36864 data: 2 2 2 2 2 2 2 2 2
22991 196/300 Read block 4991 len 32768
22992 197/300 Write block 4940 len 4096 data: 5
22993 198/300 Read block 1190 len 12288
22994 199/300 Write block 3842 len 40960 data: 5 5 5 5 5 3 3 3 4 5
22995 200/300 Write block 1104 len 16384 data: 6 5 5 6
22996 201/300 Write block 1786 len 40960 data: 4 4 3 3 3 3 5 5 5 5
22997 202/300 Write block 3295 len 16384 data: 3 3 3 3
22998 203/300 Read block 869 len 20480
22999 204/300 Write block 4387 len 16384 data: 5 5 4 3
23000 205/300 Read block 3877 len 12288
23001 206/300 Read block 1353 len 4096
23002 207/300 Read block 322 len 4096
23003 208/300 Write block 939 len 36864 data: 4 4 5 8 9 10 9 11 10
23004 209/300 Write block 3745 len 8192 data: 5 5
23005 210/300 Write block 225 len 40960 data: 5 4 3 3 4 5 4 3 3 3
23006 211/300 Read block 142 len 24576
23007 212/300 Read block 3466 len 16384
23008 213/300 Read block 887 len 24576
23009 214/300 Write block 4581 len 40960 data: 3 3 3 3 4 4 2 2 4 4
23010 215/300 Read block 4658 len 32768
23011 216/300 Write block 3782 len 12288 data: 5 5 5
23012 217/300 Read block 1495 len 8192
23013 218/300 Read block 2664 len 8192
23014 219/300 Write block 2948 len 28672 data: 4 5 6 5 6 7 6
23015 220/300 Read block 1060 len 12288
23016 221/300 Read block 1832 len 16384
23017 222/300 Read block 2996 len 4096
23018 223/300 Write block 3816 len 12288 data: 4 5 6
23019 224/300 Write block 2235 len 16384 data: 2 2 2 2
23020 225/300 Flush
23021 226/300 Read block 913 len 20480
23022 227/300 Write block 1847 len 16384 data: 2 2 2 4
23023 228/300 Read block 2260 len 40960
23024 229/300 Read block 867 len 36864
23025 230/300 Read block 4111 len 32768
23026 231/300 Read block 1581 len 24576
23027 232/300 Read block 371 len 4096
23028 233/300 Write block 3545 len 36864 data: 3 3 4 4 3 3 3 2 3
23029 234/300 Write block 4330 len 36864 data: 5 5 5 5 5 5 6 5 3
23030 235/300 Read block 4254 len 32768
23031 236/300 Read block 4867 len 16384
23032 237/300 Read block 1373 len 36864
23033 238/300 Write block 2189 len 32768 data: 6 6 4 4 6 5 6 8
23034 239/300 Write block 2350 len 20480 data: 2 2 2 2 2
23035 240/300 Write block 2415 len 40960 data: 4 4 4 5 4 3 2 4 4 4
23036 241/300 Flush
23037 242/300 Read block 4944 len 20480
23038 243/300 Read block 3627 len 12288
23039 244/300 Read block 551 len 40960
23040 245/300 Write block 2655 len 20480 data: 5 4 3 5 6
23041 246/300 Write block 2932 len 40960 data: 7 4 4 3 4 6 8 7 7 6
23042 247/300 Read block 979 len 32768
23043 248/300 Read block 3930 len 40960
23044 249/300 Flush
23045 250/300 Read block 4228 len 16384
23046 251/300 Read block 23 len 32768
23047 252/300 Write block 4631 len 8192 data: 2 3
23048 253/300 Read block 3442 len 40960
23049 254/300 Flush
23050 255/300 Write block 4886 len 12288 data: 8 8 9
23051 256/300 Write block 1278 len 4096 data: 5
23052 257/300 Write block 1097 len 24576 data: 3 4 5 7 7 7
23053 258/300 Flush
23054 259/300 Read block 4049 len 24576
23055 260/300 Write block 1701 len 24576 data: 4 3 2 2 5 4
23056 261/300 Read block 4957 len 40960
23057 262/300 Read block 3069 len 4096
23058 263/300 Read block 3960 len 40960
23059 264/300 Read block 545 len 32768
23060 265/300 Flush
23061 266/300 Read block 3348 len 20480
23062 267/300 Write block 327 len 32768 data: 4 5 5 5 3 2 2 3
23063 268/300 Read block 4782 len 28672
23064 269/300 Write block 1804 len 12288 data: 3 4 4
23065 270/300 Write block 1037 len 8192 data: 3 5
23066 271/300 Write block 2221 len 40960 data: 6 6 7 7 7 8 8 8 7 9
23067 272/300 Read block 2540 len 16384
23068 273/300 Read block 4124 len 24576
23069 274/300 Read block 1532 len 20480
23070 275/300 Flush
23071 276/300 Read block 767 len 28672
23072 277/300 Write block 560 len 4096 data: 3
23073 278/300 Read block 2177 len 20480
23074 279/300 Read block 3240 len 24576
23075 280/300 Flush
23076 281/300 Read block 1312 len 32768
23077 282/300 Write block 969 len 8192 data: 4 3
23078 283/300 Flush
23079 284/300 Read block 3111 len 40960
23080 285/300 Read block 4680 len 4096
23081 286/300 Write block 446 len 40960 data: 4 4 4 4 3 4 4 4 5 3
23082 287/300 Write block 3247 len 32768 data: 4 4 4 3 2 2 2 2
23083 288/300 Write block 1319 len 36864 data: 5 5 5 4 3 3 4 5 5
23084 289/300 Write block 4181 len 28672 data: 7 6 6 6 8 6 7
23085 290/300 Write block 4761 len 20480 data: 2 4 4 4 5
23086 291/300 Read block 2894 len 28672
23087 292/300 Write block 247 len 4096 data: 4
23088 293/300 Read block 2783 len 24576
23089 294/300 Flush
23090 295/300 Read block 2287 len 20480
23091 296/300 Read block 4817 len 40960
23092 297/300 Read block 4083 len 12288
23093 298/300 Write block 2797 len 16384 data: 4 5 4 5
23094 299/300 Read block 2419 len 12288
23095 300/300 Write block 3349 len 32768 data: 7 6 7 5 5 5 6 7
23096 301/300 Write block 1360 len 28672 data: 5 4 3 3 3 3 3
23097 Main task has work to do, go find it
23098 got dsc Start(0)
23099 start 0
23100 Jan 13 15:56:40.309 INFO request completed, latency_us: 245, response_code: 204, uri: /start/cid/0, method: POST, req_id: 0511ed14-db44-4b8d-bd43-21e48db60adb, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
23101 Replay: started 0, returned:Ok(())
23102 ----------------------------------------------------------------
23103 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
23104 States: Offline Active Active
23105 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23106 7038 Acked Write 7 Sent Done Done false
23107 7039 Acked Write 1 Sent Done Done false
23108 7040 Acked Flush 0 Sent Done Done false
23109 7041 Acked Read 5 Sent Done Done false
23110 7042 Acked Read 8 Sent Done Done false
23111 7043 Acked Write 8 Sent Done Done false
23112 7044 Acked Flush 0 Sent Done Done false
23113 7045 Acked Read 6 Sent Done Done false
23114 7046 Acked Read 7 Sent Done Done false
23115 7047 Acked Read 6 Sent Done Done false
23116 7048 Acked Read 2 Sent Done Done false
23117 7049 Acked Read 4 Sent Done Done false
23118 7050 Acked Read 9 Sent Done Done false
23119 7051 Acked Read 2 Sent Done Done false
23120 7052 Acked Read 2 Sent Done Done false
23121 7053 Acked Write 8 Sent Done Done false
23122 7054 Acked Read 6 Sent Done Done false
23123 7055 Acked Write 9 Sent Done Done false
23124 7056 Acked Write 2 Sent Done Done false
23125 7057 Acked Read 7 Sent Done Done false
23126 7058 Acked Read 6 Sent Done Done false
23127 7059 Acked Read 9 Sent Done Done false
23128 7060 Acked Read 6 Sent Done Done false
23129 7061 Acked Write 2 Sent Done Done false
23130 7062 Acked Read 10 Sent Done Done false
23131 7063 Acked Read 3 Sent Done Done false
23132 7064 Acked Read 2 Sent Done Done false
23133 7065 Acked Flush 0 Sent Done Done false
23134 7066 Acked Flush 0 Sent Done Done false
23135 7067 Acked Write 4 Sent Done Done false
23136 7068 Acked Write 9 Sent Done Done false
23137 7069 Acked Read 7 Sent Done Done false
23138 7070 Acked Write 10 Sent Done Done false
23139 7071 Acked Write 9 Sent Done Done false
23140 7072 Acked Write 2 Sent Done Done false
23141 7073 Acked Write 4 Sent Done Done false
23142 7074 Acked Read 9 Sent Done Done false
23143 7075 Acked Write 9 Sent Done Done false
23144 7076 Acked Write 7 Sent Done Done false
23145 7077 Acked Write 4 Sent Done Done false
23146 7078 Acked Write 8 Sent Done Done false
23147 7079 Acked Read 2 Sent Done Done false
23148 7080 Acked Read 2 Sent Done Done false
23149 7081 Acked Read 7 Sent Done Done false
23150 7082 Acked Write 1 Sent Done Done false
23151 7083 Acked Write 10 Sent Done Done false
23152 7084 Acked Write 7 Sent Done Done false
23153 7085 Acked Write 6 Sent Done Done false
23154 7086 Acked Read 9 Sent Done Done false
23155 7087 Acked Read 1 Sent Done Done false
23156 7088 Acked Read 8 Sent Done Done false
23157 7089 Acked Read 5 Sent Done Done false
23158 7090 Acked Flush 0 Sent Done Done false
23159 7091 Acked Write 10 Sent Done Done false
23160 7092 Acked Write 10 Sent Done Done false
23161 7093 Acked Read 3 Sent Done Done false
23162 7094 Acked Read 5 Sent Done Done false
23163 7095 Acked Read 5 Sent Done Done false
23164 7096 Acked Write 7 Sent Done Done false
23165 7097 Acked Write 10 Sent Done Done false
23166 7098 Acked Read 6 Sent Done Done false
23167 7099 Acked Flush 0 Sent Done Done false
23168 7100 Acked Write 3 Sent Done Done false
23169 7101 Acked Read 10 Sent Done Done false
23170 7102 Acked Write 4 Sent Done Done false
23171 7103 Acked Write 10 Sent Done Done false
23172 7104 Acked Read 5 Sent Done Done false
23173 7105 Acked Write 2 Sent Done Done false
23174 7106 Acked Write 7 Sent Done Done false
23175 7107 Acked Read 7 Sent Done Done false
23176 7108 Acked Write 2 Sent Done Done false
23177 7109 Acked Write 6 Sent Done Done false
23178 7110 Acked Write 8 Sent Done Done false
23179 7111 Acked Write 10 Sent Done Done false
23180 7112 Acked Write 5 Sent Done Done false
23181 7113 Acked Read 2 Sent Done Done false
23182 7114 Acked Write 7 Sent Done Done false
23183 7115 Acked Read 4 Sent Done Done false
23184 7116 Acked Read 6 Sent Done Done false
23185 7117 Acked Write 1 Sent Done Done false
23186 7118 Acked Write 9 Sent Done Done false
23187 7119 Acked Flush 0 Sent Done Done false
23188 7120 Acked Read 6 Sent Done Done false
23189 7121 Acked Write 3 Sent Done Done false
23190 7122 Acked Write 4 Sent Done Done false
23191 7123 Acked Flush 0 Sent Done Done false
23192 7124 Acked Flush 0 Sent Done Done false
23193 7125 Acked Read 6 Sent Done Done false
23194 7126 Acked Read 2 Sent Done Done false
23195 7127 Acked Read 7 Sent Done Done false
23196 7128 Acked Write 4 Sent Done Done false
23197 7129 Acked Read 5 Sent Done Done false
23198 7130 Acked Read 7 Sent Done Done false
23199 7131 Acked Read 7 Sent Done Done false
23200 7132 Acked Write 7 Sent Done Done false
23201 7133 Acked Read 1 Sent Done Done false
23202 7134 Acked Read 2 Sent Done Done false
23203 7135 Acked Write 6 Sent Done Done false
23204 7136 Acked Read 1 Sent Done Done false
23205 7137 Acked Read 4 Sent Done Done false
23206 7138 Acked Read 10 Sent Done Done false
23207 7139 Acked Write 1 Sent Done Done false
23208 7140 Acked Read 2 Sent Done Done false
23209 7141 Acked Flush 0 Sent Done Done false
23210 7142 Acked Flush 0 Sent Done Done false
23211 7143 Acked Flush 0 Sent Done Done false
23212 7144 Acked Read 9 Sent Done Done false
23213 7145 Acked Write 7 Sent Done Done false
23214 7146 Acked Read 3 Sent Done Done false
23215 7147 Acked Read 10 Sent Done Done false
23216 7148 Acked Write 9 Sent Done Done false
23217 7149 Acked Write 10 Sent Done Done false
23218 7150 Acked Read 2 Sent Done Done false
23219 7151 Acked Read 6 Sent Done Done false
23220 7152 Acked Read 5 Sent Done Done false
23221 7153 Acked Write 8 Sent Done Done false
23222 7154 Acked Flush 0 Sent Done Done false
23223 7155 Acked Read 3 Sent Done Done false
23224 7156 Acked Read 6 Sent Done Done false
23225 7157 Acked Write 4 Sent Done Done false
23226 7158 Acked Write 4 Sent Done Done false
23227 7159 Acked Flush 0 Sent Done Done false
23228 7160 Acked Write 8 Sent Done Done false
23229 7161 Acked Flush 0 Sent Done Done false
23230 7162 Acked Write 4 Sent Done Done false
23231 7163 Acked Flush 0 Sent Done Done false
23232 7164 Acked Read 8 Sent Done Done false
23233 7165 Acked Write 1 Sent Done Done false
23234 7166 Acked Write 6 Sent Done Done false
23235 7167 Acked Read 5 Sent Done Done false
23236 7168 Acked Read 8 Sent Done Done false
23237 7169 Acked Read 9 Sent Done Done false
23238 7170 Acked Write 7 Sent Done Done false
23239 7171 Acked Write 9 Sent Done Done false
23240 7172 Acked Write 5 Sent Done Done false
23241 7173 Acked Read 4 Sent Done Done false
23242 7174 Acked Flush 0 Sent Done Done false
23243 7175 Acked Read 10 Sent Done Done false
23244 7176 Acked Read 5 Sent Done Done false
23245 7177 Acked Write 3 Sent Done Done false
23246 7178 Acked Flush 0 Sent Done Done false
23247 7179 Acked Read 1 Sent Done Done false
23248 7180 Acked Read 7 Sent Done Done false
23249 7181 Acked Flush 0 Sent Done Done false
23250 7182 Acked Read 2 Sent Done Done false
23251 7183 Acked Flush 0 Sent Done Done false
23252 7184 Acked Write 5 Sent Done Done false
23253 7185 Acked Write 6 Sent Done Done false
23254 7186 Acked Read 9 Sent Done Done false
23255 7187 Acked Read 9 Sent Done Done false
23256 7188 Acked Write 10 Sent Done Done false
23257 7189 Acked Write 9 Sent Done Done false
23258 7190 Acked Write 9 Sent Done Done false
23259 7191 Acked Write 10 Sent Done Done false
23260 7192 Acked Write 5 Sent Done Done false
23261 7193 Acked Write 2 Sent Done Done false
23262 7194 Acked Read 8 Sent Done Done false
23263 7195 Acked Write 8 Sent Done Done false
23264 7196 Acked Read 6 Sent Done Done false
23265 7197 Acked Read 6 Sent Done Done false
23266 7198 Acked Write 7 Sent Done Done false
23267 7199 Acked Write 9 Sent Done Done false
23268 7200 Acked Flush 0 Sent Done Done false
23269 7201 Acked Read 9 Sent Done Done false
23270 7202 Acked Write 4 Sent Done Done false
23271 7203 Acked Write 2 Sent Done Done false
23272 7204 Acked Flush 0 Sent Done Done false
23273 7205 Acked Write 10 Sent Done Done false
23274 7206 Acked Write 4 Sent Done Done false
23275 7207 Acked Read 3 Sent Done Done false
23276 7208 Acked Read 9 Sent Done Done false
23277 7209 Acked Write 2 Sent Done Done false
23278 7210 Acked Read 5 Sent Done Done false
23279 7211 Acked Read 5 Sent Done Done false
23280 7212 Acked Write 9 Sent Done Done false
23281 7213 Acked Read 7 Sent Done Done false
23282 7214 Acked Read 1 Sent Done Done false
23283 7215 Acked Read 7 Sent Done Done false
23284 7216 Acked Read 7 Sent Done Done false
23285 7217 Acked Write 4 Sent Done Done false
23286 7218 Acked Write 8 Sent Done Done false
23287 7219 Acked Read 3 Sent Done Done false
23288 7220 Acked Read 7 Sent Done Done false
23289 7221 Acked Read 10 Sent Done Done false
23290 7222 Acked Read 4 Sent Done Done false
23291 7223 Acked Flush 0 Sent Done Done false
23292 7224 Acked Write 4 Sent Done Done false
23293 7225 Acked Read 2 Sent Done Done false
23294 7226 Acked Flush 0 Sent Done Done false
23295 7227 Acked Write 9 Sent Done Done false
23296 7228 Acked Write 1 Sent Done Done false
23297 7229 Acked Write 7 Sent Done Done false
23298 7230 Acked Read 3 Sent Done Done false
23299 7231 Acked Read 10 Sent Done Done false
23300 7232 Acked Write 9 Sent Done Done false
23301 7233 Acked Read 8 Sent Done Done false
23302 7234 Acked Write 1 Sent Done Done false
23303 7235 Acked Read 3 Sent Done Done false
23304 7236 Acked Write 10 Sent Done Done false
23305 7237 Acked Write 4 Sent Done Done false
23306 7238 Acked Write 10 Sent Done Done false
23307 7239 Acked Write 4 Sent Done Done false
23308 7240 Acked Read 5 Sent Done Done false
23309 7241 Acked Write 4 Sent Done Done false
23310 7242 Acked Read 3 Sent Done Done false
23311 7243 Acked Read 1 Sent Done Done false
23312 7244 Acked Read 1 Sent Done Done false
23313 7245 Acked Write 9 Sent Done Done false
23314 7246 Acked Write 2 Sent Done Done false
23315 7247 Acked Write 10 Sent Done Done false
23316 7248 Acked Read 6 Sent Done Done false
23317 7249 Acked Read 4 Sent Done Done false
23318 7250 Acked Read 6 Sent Done Done false
23319 7251 Acked Write 10 Sent Done Done false
23320 7252 Acked Read 8 Sent Done Done false
23321 7253 Acked Write 3 Sent Done Done false
23322 7254 Acked Read 2 Sent Done Done false
23323 7255 Acked Read 2 Sent Done Done false
23324 7256 Acked Write 7 Sent Done Done false
23325 7257 Acked Read 3 Sent Done Done false
23326 7258 Acked Read 4 Sent Done Done false
23327 7259 Acked Read 1 Sent Done Done false
23328 7260 Acked Write 3 Sent Done Done false
23329 7261 Acked Write 4 Sent Done Done false
23330 7262 Acked Flush 0 Sent Done Done false
23331 7263 Acked Read 5 Sent Done Done false
23332 7264 Acked Write 4 Sent Done Done false
23333 7265 Acked Read 10 Sent Done Done false
23334 7266 Acked Read 9 Sent Done Done false
23335 7267 Acked Read 8 Sent Done Done false
23336 7268 Acked Read 6 Sent Done Done false
23337 7269 Acked Read 1 Sent Done Done false
23338 7270 Acked Write 9 Sent Done Done false
23339 7271 Acked Write 9 Sent Done Done false
23340 7272 Acked Read 8 Sent Done Done false
23341 7273 Acked Read 4 Sent Done Done false
23342 7274 Acked Read 9 Sent Done Done false
23343 7275 Acked Write 8 Sent Done Done false
23344 7276 Acked Write 5 Sent Done Done false
23345 7277 Acked Write 10 Sent Done Done false
23346 7278 Acked Flush 0 Sent Done Done false
23347 7279 Acked Read 5 Sent Done Done false
23348 7280 Acked Read 3 Sent Done Done false
23349 7281 Acked Read 10 Sent Done Done false
23350 7282 Acked Write 5 Sent Done Done false
23351 7283 Acked Write 10 Sent Done Done false
23352 7284 Acked Read 8 Sent Done Done false
23353 7285 Acked Read 10 Sent Done Done false
23354 7286 Acked Flush 0 Sent Done Done false
23355 7287 Acked Read 4 Sent Done Done false
23356 7288 Acked Read 8 Sent Done Done false
23357 7289 Acked Write 2 Sent Done Done false
23358 7290 Acked Read 10 Sent Done Done false
23359 7291 Acked Flush 0 Sent Done Done false
23360 7292 Acked Write 3 Sent Done Done false
23361 7293 Acked Write 1 Sent Done Done false
23362 7294 Acked Write 6 Sent Done Done false
23363 7295 Acked Flush 0 Sent Done Done false
23364 7296 Acked Read 6 Sent Done Done false
23365 7297 Acked Write 6 Sent Done Done false
23366 7298 Acked Read 10 Sent Done Done false
23367 7299 Acked Read 1 Sent Done Done false
23368 7300 Acked Read 10 Sent Done Done false
23369 7301 Acked Read 8 Sent Done Done false
23370 7302 Acked Flush 0 Sent Done Done false
23371 7303 Acked Read 5 Sent Done Done false
23372 7304 Acked Write 8 Sent Done Done false
23373 7305 Acked Read 7 Sent Done Done false
23374 7306 Acked Write 3 Sent Done Done false
23375 7307 Acked Write 2 Sent Done Done false
23376 7308 Acked Write 10 Sent Done Done false
23377 7309 Acked Read 4 Sent Done Done false
23378 7310 Acked Read 6 Sent Done Done false
23379 7311 Acked Read 5 Sent Done Done false
23380 7312 Acked Flush 0 Sent Done Done false
23381 7313 Acked Read 7 Sent Done Done false
23382 7314 Acked Write 1 Sent Done Done false
23383 7315 Acked Read 5 Sent Done Done false
23384 7316 Acked Read 6 Sent Done Done false
23385 7317 Acked Flush 0 Sent Done Done false
23386 7318 Acked Read 8 Sent Done Done false
23387 7319 Acked Write 2 Sent Done Done false
23388 7320 Acked Flush 0 Sent Done Done false
23389 7321 Acked Read 10 Sent Done Done false
23390 7322 Acked Read 1 Sent Done Done false
23391 7323 Acked Write 10 Sent Done Done false
23392 7324 Acked Write 8 Sent Done Done false
23393 7325 Acked Write 9 Sent Done Done false
23394 7326 Acked Write 7 Sent Done Done false
23395 7327 Acked Write 5 Sent Done Done false
23396 7328 Acked Read 7 Sent Done Done false
23397 7329 Acked Write 1 Sent Done Done false
23398 7330 Acked Read 6 Sent Done Done false
23399 7331 Acked Flush 0 Sent Done Done false
23400 7332 Acked Read 5 Sent Done Done false
23401 7333 Acked Read 10 Sent Done Done false
23402 7334 Acked Read 3 Sent Done Done false
23403 7335 Acked Write 4 Sent Done Done false
23404 7336 Acked Read 3 Sent Done Done false
23405 7337 Acked Write 8 Sent Done Done false
23406 7338 Acked Write 7 Sent Done Done false
23407 STATES DS:0 DS:1 DS:2 TOTAL
23408 Sent 301 0 0 301
23409 Done 0 301 301 602
23410 Skipped 0 0 0 0
23411 Error 0 0 0 0
23412 Last Flush: 7037 7331 7331
23413 Downstairs last five completed: 7037 7036 7035 7034 7033
23414 Upstairs last five completed: 7338 7337 7336 7335 7334
23415 CLIENT: Up:0 ds:301 act:2
23416 [8810] got start action
23417 [8810] I am going to restart
23418 Starting downstairs at port 8810
23419 Make output file at "/tmp/dsc/downstairs-8810.txt"
23420 [8810][0] reports Starting
23421 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1223)
23422 [8810][0] reports Running
23423 ----------------------------------------------------------------
23424 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23425 States: Offline Active Active
23426 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23427 7038 Acked Write 7 Sent Done Done false
23428 7039 Acked Write 1 Sent Done Done false
23429 7040 Acked Flush 0 Sent Done Done false
23430 7041 Acked Read 5 Sent Done Done false
23431 7042 Acked Read 8 Sent Done Done false
23432 7043 Acked Write 8 Sent Done Done false
23433 7044 Acked Flush 0 Sent Done Done false
23434 7045 Acked Read 6 Sent Done Done false
23435 7046 Acked Read 7 Sent Done Done false
23436 7047 Acked Read 6 Sent Done Done false
23437 7048 Acked Read 2 Sent Done Done false
23438 7049 Acked Read 4 Sent Done Done false
23439 7050 Acked Read 9 Sent Done Done false
23440 7051 Acked Read 2 Sent Done Done false
23441 7052 Acked Read 2 Sent Done Done false
23442 7053 Acked Write 8 Sent Done Done false
23443 7054 Acked Read 6 Sent Done Done false
23444 7055 Acked Write 9 Sent Done Done false
23445 7056 Acked Write 2 Sent Done Done false
23446 7057 Acked Read 7 Sent Done Done false
23447 7058 Acked Read 6 Sent Done Done false
23448 7059 Acked Read 9 Sent Done Done false
23449 7060 Acked Read 6 Sent Done Done false
23450 7061 Acked Write 2 Sent Done Done false
23451 7062 Acked Read 10 Sent Done Done false
23452 7063 Acked Read 3 Sent Done Done false
23453 7064 Acked Read 2 Sent Done Done false
23454 7065 Acked Flush 0 Sent Done Done false
23455 7066 Acked Flush 0 Sent Done Done false
23456 7067 Acked Write 4 Sent Done Done false
23457 7068 Acked Write 9 Sent Done Done false
23458 7069 Acked Read 7 Sent Done Done false
23459 7070 Acked Write 10 Sent Done Done false
23460 7071 Acked Write 9 Sent Done Done false
23461 7072 Acked Write 2 Sent Done Done false
23462 7073 Acked Write 4 Sent Done Done false
23463 7074 Acked Read 9 Sent Done Done false
23464 7075 Acked Write 9 Sent Done Done false
23465 7076 Acked Write 7 Sent Done Done false
23466 7077 Acked Write 4 Sent Done Done false
23467 7078 Acked Write 8 Sent Done Done false
23468 7079 Acked Read 2 Sent Done Done false
23469 7080 Acked Read 2 Sent Done Done false
23470 7081 Acked Read 7 Sent Done Done false
23471 7082 Acked Write 1 Sent Done Done false
23472 7083 Acked Write 10 Sent Done Done false
23473 7084 Acked Write 7 Sent Done Done false
23474 7085 Acked Write 6 Sent Done Done false
23475 7086 Acked Read 9 Sent Done Done false
23476 7087 Acked Read 1 Sent Done Done false
23477 7088 Acked Read 8 Sent Done Done false
23478 7089 Acked Read 5 Sent Done Done false
23479 7090 Acked Flush 0 Sent Done Done false
23480 7091 Acked Write 10 Sent Done Done false
23481 7092 Acked Write 10 Sent Done Done false
23482 7093 Acked Read 3 Sent Done Done false
23483 7094 Acked Read 5 Sent Done Done false
23484 7095 Acked Read 5 Sent Done Done false
23485 7096 Acked Write 7 Sent Done Done false
23486 7097 Acked Write 10 Sent Done Done false
23487 7098 Acked Read 6 Sent Done Done false
23488 7099 Acked Flush 0 Sent Done Done false
23489 7100 Acked Write 3 Sent Done Done false
23490 7101 Acked Read 10 Sent Done Done false
23491 7102 Acked Write 4 Sent Done Done false
23492 7103 Acked Write 10 Sent Done Done false
23493 7104 Acked Read 5 Sent Done Done false
23494 7105 Acked Write 2 Sent Done Done false
23495 7106 Acked Write 7 Sent Done Done false
23496 7107 Acked Read 7 Sent Done Done false
23497 7108 Acked Write 2 Sent Done Done false
23498 7109 Acked Write 6 Sent Done Done false
23499 7110 Acked Write 8 Sent Done Done false
23500 7111 Acked Write 10 Sent Done Done false
23501 7112 Acked Write 5 Sent Done Done false
23502 7113 Acked Read 2 Sent Done Done false
23503 7114 Acked Write 7 Sent Done Done false
23504 7115 Acked Read 4 Sent Done Done false
23505 7116 Acked Read 6 Sent Done Done false
23506 7117 Acked Write 1 Sent Done Done false
23507 7118 Acked Write 9 Sent Done Done false
23508 7119 Acked Flush 0 Sent Done Done false
23509 7120 Acked Read 6 Sent Done Done false
23510 7121 Acked Write 3 Sent Done Done false
23511 7122 Acked Write 4 Sent Done Done false
23512 7123 Acked Flush 0 Sent Done Done false
23513 7124 Acked Flush 0 Sent Done Done false
23514 7125 Acked Read 6 Sent Done Done false
23515 7126 Acked Read 2 Sent Done Done false
23516 7127 Acked Read 7 Sent Done Done false
23517 7128 Acked Write 4 Sent Done Done false
23518 7129 Acked Read 5 Sent Done Done false
23519 7130 Acked Read 7 Sent Done Done false
23520 7131 Acked Read 7 Sent Done Done false
23521 7132 Acked Write 7 Sent Done Done false
23522 7133 Acked Read 1 Sent Done Done false
23523 7134 Acked Read 2 Sent Done Done false
23524 7135 Acked Write 6 Sent Done Done false
23525 7136 Acked Read 1 Sent Done Done false
23526 7137 Acked Read 4 Sent Done Done false
23527 7138 Acked Read 10 Sent Done Done false
23528 7139 Acked Write 1 Sent Done Done false
23529 7140 Acked Read 2 Sent Done Done false
23530 7141 Acked Flush 0 Sent Done Done false
23531 7142 Acked Flush 0 Sent Done Done false
23532 7143 Acked Flush 0 Sent Done Done false
23533 7144 Acked Read 9 Sent Done Done false
23534 7145 Acked Write 7 Sent Done Done false
23535 7146 Acked Read 3 Sent Done Done false
23536 7147 Acked Read 10 Sent Done Done false
23537 7148 Acked Write 9 Sent Done Done false
23538 7149 Acked Write 10 Sent Done Done false
23539 7150 Acked Read 2 Sent Done Done false
23540 7151 Acked Read 6 Sent Done Done false
23541 7152 Acked Read 5 Sent Done Done false
23542 7153 Acked Write 8 Sent Done Done false
23543 7154 Acked Flush 0 Sent Done Done false
23544 7155 Acked Read 3 Sent Done Done false
23545 7156 Acked Read 6 Sent Done Done false
23546 7157 Acked Write 4 Sent Done Done false
23547 7158 Acked Write 4 Sent Done Done false
23548 7159 Acked Flush 0 Sent Done Done false
23549 7160 Acked Write 8 Sent Done Done false
23550 7161 Acked Flush 0 Sent Done Done false
23551 7162 Acked Write 4 Sent Done Done false
23552 7163 Acked Flush 0 Sent Done Done false
23553 7164 Acked Read 8 Sent Done Done false
23554 7165 Acked Write 1 Sent Done Done false
23555 7166 Acked Write 6 Sent Done Done false
23556 7167 Acked Read 5 Sent Done Done false
23557 7168 Acked Read 8 Sent Done Done false
23558 7169 Acked Read 9 Sent Done Done false
23559 7170 Acked Write 7 Sent Done Done false
23560 7171 Acked Write 9 Sent Done Done false
23561 7172 Acked Write 5 Sent Done Done false
23562 7173 Acked Read 4 Sent Done Done false
23563 7174 Acked Flush 0 Sent Done Done false
23564 7175 Acked Read 10 Sent Done Done false
23565 7176 Acked Read 5 Sent Done Done false
23566 7177 Acked Write 3 Sent Done Done false
23567 7178 Acked Flush 0 Sent Done Done false
23568 7179 Acked Read 1 Sent Done Done false
23569 7180 Acked Read 7 Sent Done Done false
23570 7181 Acked Flush 0 Sent Done Done false
23571 7182 Acked Read 2 Sent Done Done false
23572 7183 Acked Flush 0 Sent Done Done false
23573 7184 Acked Write 5 Sent Done Done false
23574 7185 Acked Write 6 Sent Done Done false
23575 7186 Acked Read 9 Sent Done Done false
23576 7187 Acked Read 9 Sent Done Done false
23577 7188 Acked Write 10 Sent Done Done false
23578 7189 Acked Write 9 Sent Done Done false
23579 7190 Acked Write 9 Sent Done Done false
23580 7191 Acked Write 10 Sent Done Done false
23581 7192 Acked Write 5 Sent Done Done false
23582 7193 Acked Write 2 Sent Done Done false
23583 7194 Acked Read 8 Sent Done Done false
23584 7195 Acked Write 8 Sent Done Done false
23585 7196 Acked Read 6 Sent Done Done false
23586 7197 Acked Read 6 Sent Done Done false
23587 7198 Acked Write 7 Sent Done Done false
23588 7199 Acked Write 9 Sent Done Done false
23589 7200 Acked Flush 0 Sent Done Done false
23590 7201 Acked Read 9 Sent Done Done false
23591 7202 Acked Write 4 Sent Done Done false
23592 7203 Acked Write 2 Sent Done Done false
23593 7204 Acked Flush 0 Sent Done Done false
23594 7205 Acked Write 10 Sent Done Done false
23595 7206 Acked Write 4 Sent Done Done false
23596 7207 Acked Read 3 Sent Done Done false
23597 7208 Acked Read 9 Sent Done Done false
23598 7209 Acked Write 2 Sent Done Done false
23599 7210 Acked Read 5 Sent Done Done false
23600 7211 Acked Read 5 Sent Done Done false
23601 7212 Acked Write 9 Sent Done Done false
23602 7213 Acked Read 7 Sent Done Done false
23603 7214 Acked Read 1 Sent Done Done false
23604 7215 Acked Read 7 Sent Done Done false
23605 7216 Acked Read 7 Sent Done Done false
23606 7217 Acked Write 4 Sent Done Done false
23607 7218 Acked Write 8 Sent Done Done false
23608 7219 Acked Read 3 Sent Done Done false
23609 7220 Acked Read 7 Sent Done Done false
23610 7221 Acked Read 10 Sent Done Done false
23611 7222 Acked Read 4 Sent Done Done false
23612 7223 Acked Flush 0 Sent Done Done false
23613 7224 Acked Write 4 Sent Done Done false
23614 7225 Acked Read 2 Sent Done Done false
23615 7226 Acked Flush 0 Sent Done Done false
23616 7227 Acked Write 9 Sent Done Done false
23617 7228 Acked Write 1 Sent Done Done false
23618 7229 Acked Write 7 Sent Done Done false
23619 7230 Acked Read 3 Sent Done Done false
23620 7231 Acked Read 10 Sent Done Done false
23621 7232 Acked Write 9 Sent Done Done false
23622 7233 Acked Read 8 Sent Done Done false
23623 7234 Acked Write 1 Sent Done Done false
23624 7235 Acked Read 3 Sent Done Done false
23625 7236 Acked Write 10 Sent Done Done false
23626 7237 Acked Write 4 Sent Done Done false
23627 7238 Acked Write 10 Sent Done Done false
23628 7239 Acked Write 4 Sent Done Done false
23629 7240 Acked Read 5 Sent Done Done false
23630 7241 Acked Write 4 Sent Done Done false
23631 7242 Acked Read 3 Sent Done Done false
23632 7243 Acked Read 1 Sent Done Done false
23633 7244 Acked Read 1 Sent Done Done false
23634 7245 Acked Write 9 Sent Done Done false
23635 7246 Acked Write 2 Sent Done Done false
23636 7247 Acked Write 10 Sent Done Done false
23637 7248 Acked Read 6 Sent Done Done false
23638 7249 Acked Read 4 Sent Done Done false
23639 7250 Acked Read 6 Sent Done Done false
23640 7251 Acked Write 10 Sent Done Done false
23641 7252 Acked Read 8 Sent Done Done false
23642 7253 Acked Write 3 Sent Done Done false
23643 7254 Acked Read 2 Sent Done Done false
23644 7255 Acked Read 2 Sent Done Done false
23645 7256 Acked Write 7 Sent Done Done false
23646 7257 Acked Read 3 Sent Done Done false
23647 7258 Acked Read 4 Sent Done Done false
23648 7259 Acked Read 1 Sent Done Done false
23649 7260 Acked Write 3 Sent Done Done false
23650 7261 Acked Write 4 Sent Done Done false
23651 7262 Acked Flush 0 Sent Done Done false
23652 7263 Acked Read 5 Sent Done Done false
23653 7264 Acked Write 4 Sent Done Done false
23654 7265 Acked Read 10 Sent Done Done false
23655 7266 Acked Read 9 Sent Done Done false
23656 7267 Acked Read 8 Sent Done Done false
23657 7268 Acked Read 6 Sent Done Done false
23658 7269 Acked Read 1 Sent Done Done false
23659 7270 Acked Write 9 Sent Done Done false
23660 7271 Acked Write 9 Sent Done Done false
23661 7272 Acked Read 8 Sent Done Done false
23662 7273 Acked Read 4 Sent Done Done false
23663 7274 Acked Read 9 Sent Done Done false
23664 7275 Acked Write 8 Sent Done Done false
23665 7276 Acked Write 5 Sent Done Done false
23666 7277 Acked Write 10 Sent Done Done false
23667 7278 Acked Flush 0 Sent Done Done false
23668 7279 Acked Read 5 Sent Done Done false
23669 7280 Acked Read 3 Sent Done Done false
23670 7281 Acked Read 10 Sent Done Done false
23671 7282 Acked Write 5 Sent Done Done false
23672 7283 Acked Write 10 Sent Done Done false
23673 7284 Acked Read 8 Sent Done Done false
23674 7285 Acked Read 10 Sent Done Done false
23675 7286 Acked Flush 0 Sent Done Done false
23676 7287 Acked Read 4 Sent Done Done false
23677 7288 Acked Read 8 Sent Done Done false
23678 7289 Acked Write 2 Sent Done Done false
23679 7290 Acked Read 10 Sent Done Done false
23680 7291 Acked Flush 0 Sent Done Done false
23681 7292 Acked Write 3 Sent Done Done false
23682 7293 Acked Write 1 Sent Done Done false
23683 7294 Acked Write 6 Sent Done Done false
23684 7295 Acked Flush 0 Sent Done Done false
23685 7296 Acked Read 6 Sent Done Done false
23686 7297 Acked Write 6 Sent Done Done false
23687 7298 Acked Read 10 Sent Done Done false
23688 7299 Acked Read 1 Sent Done Done false
23689 7300 Acked Read 10 Sent Done Done false
23690 7301 Acked Read 8 Sent Done Done false
23691 7302 Acked Flush 0 Sent Done Done false
23692 7303 Acked Read 5 Sent Done Done false
23693 7304 Acked Write 8 Sent Done Done false
23694 7305 Acked Read 7 Sent Done Done false
23695 7306 Acked Write 3 Sent Done Done false
23696 7307 Acked Write 2 Sent Done Done false
23697 7308 Acked Write 10 Sent Done Done false
23698 7309 Acked Read 4 Sent Done Done false
23699 7310 Acked Read 6 Sent Done Done false
23700 7311 Acked Read 5 Sent Done Done false
23701 7312 Acked Flush 0 Sent Done Done false
23702 7313 Acked Read 7 Sent Done Done false
23703 7314 Acked Write 1 Sent Done Done false
23704 7315 Acked Read 5 Sent Done Done false
23705 7316 Acked Read 6 Sent Done Done false
23706 7317 Acked Flush 0 Sent Done Done false
23707 7318 Acked Read 8 Sent Done Done false
23708 7319 Acked Write 2 Sent Done Done false
23709 7320 Acked Flush 0 Sent Done Done false
23710 7321 Acked Read 10 Sent Done Done false
23711 7322 Acked Read 1 Sent Done Done false
23712 7323 Acked Write 10 Sent Done Done false
23713 7324 Acked Write 8 Sent Done Done false
23714 7325 Acked Write 9 Sent Done Done false
23715 7326 Acked Write 7 Sent Done Done false
23716 7327 Acked Write 5 Sent Done Done false
23717 7328 Acked Read 7 Sent Done Done false
23718 7329 Acked Write 1 Sent Done Done false
23719 7330 Acked Read 6 Sent Done Done false
23720 7331 Acked Flush 0 Sent Done Done false
23721 7332 Acked Read 5 Sent Done Done false
23722 7333 Acked Read 10 Sent Done Done false
23723 7334 Acked Read 3 Sent Done Done false
23724 7335 Acked Write 4 Sent Done Done false
23725 7336 Acked Read 3 Sent Done Done false
23726 7337 Acked Write 8 Sent Done Done false
23727 7338 Acked Write 7 Sent Done Done false
23728 7339 Acked Flush 0 Sent Done Done false
23729 STATES DS:0 DS:1 DS:2 TOTAL
23730 Sent 302 0 0 302
23731 Done 0 302 302 604
23732 Skipped 0 0 0 0
23733 Error 0 0 0 0
23734 Last Flush: 7037 7339 7339
23735 Downstairs last five completed: 7037 7036 7035 7034 7033
23736 Upstairs last five completed: 7339 7338 7337 7336 7335
23737 CLIENT: Up:0 ds:302 act:2
237382025-01-13T15:56:46.068ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
237392025-01-13T15:56:46.068ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51092) = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
237402025-01-13T15:56:46.070ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
237412025-01-13T15:56:46.070ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
237422025-01-13T15:56:46.070ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
237432025-01-13T15:56:46.070ZINFOcrucible: send last flush ID to this DS: 7037 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
237442025-01-13T15:56:46.071ZINFOcrucible: Replied this last flush ID: 7037 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
237452025-01-13T15:56:46.071ZINFOcrucible: [0] client re-new 302 jobs since flush 7037 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
237462025-01-13T15:56:46.071ZINFOcrucible: [0] Replayed 302 jobs since flush: 7037 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
23747 ----------------------------------------------------------------
23748 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
23749 Downstairs last five completed: 7339 7338 7337 7336 7335
23750 Upstairs last five completed: 7339 7338 7337 7336 7335
23751 CLIENT: Up:0 ds:0 act:3
23752 Replay: All jobs finished
23753 Main task has work to do, go find it
23754 got dsc Stop(1)
23755 stop 1
23756 [8820] Got stop action so:false kr:false
23757 Jan 13 15:56:48.315 INFO request completed, latency_us: 348, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 5bcc800c-8713-47c4-940b-85c0638a0c25, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
237582025-01-13T15:56:48.316ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
23759 Jan 13 15:56:48.316 INFO request completed, latency_us: 158, response_code: 200, uri: {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:56:48.316221955Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
23760 /state/cid/1, method: GET, req_id: 4bc6cb32-2966-4d95-8933-af23054e8a1a, remote_addr: 127.{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:56:48.316272167Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
23761 0.0.1:52422, local_addr: 127.0.0.1:9998
237622025-01-13T15:56:48.316ZINFOcrucible: client task is exiting = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
23763 [8820] Exited with: ExitStatus(unix_wait_status(9))
23764 [8820][1] reports Exit
23765 Jan 13 15:56:52.319 INFO request completed, latency_us: 210, response_code: 200, uri: /state/cid/1, method: GET, req_id: 298e42ee-eca2-41dd-a623-b57b0d96993e, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
23766 001/300 Write block 2379 len 16384 data: 6 5 6 4
23767 002/300 Write block 1583 len 40960 data: 5 5 5 6 5 5 5 6 7 6
23768 003/300 Write block 626 len 8192 data: 4 4
23769 004/300 Write block 2404 len 28672 data: 3 4 4 4 3 4 3
23770 005/300 Read block 3535 len 4096
23771 006/300 Read block 4882 len 36864
23772 007/300 Read block 1185 len 40960
23773 008/300 Read block 3129 len 8192
23774 009/300 Write block 3951 len 36864 data: 3 4 5 5 4 5 5 5 5
23775 010/300 Read block 3718 len 4096
23776 011/300 Write block 1231 len 4096 data: 3
23777 012/300 Read block 1623 len 40960
23778 013/300 Read block 1629 len 8192
23779 014/300 Write block 1794 len 4096 data: 6
23780 015/300 Read block 2600 len 40960
23781 016/300 Read block 1180 len 32768
23782 017/300 Read block 2310 len 32768
23783 018/300 Write block 2891 len 20480 data: 4 5 6 7 4
23784 019/300 Write block 2108 len 28672 data: 6 7 6 5 4 2 2
23785 020/300 Write block 3975 len 28672 data: 4 5 6 7 8 8 8
23786 021/300 Read block 3983 len 32768
23787 022/300 Read block 2857 len 40960
23788 023/300 Flush
23789 024/300 Write block 3327 len 24576 data: 7 5 6 5 4 4
23790 025/300 Write block 1085 len 28672 data: 8 8 7 7 5 6 5
23791 026/300 Read block 1670 len 12288
23792 027/300 Read block 2970 len 24576
23793 028/300 Write block 4769 len 8192 data: 4 4
23794 029/300 Write block 3272 len 36864 data: 9 10 9 9 7 7 6 6 5
23795 030/300 Read block 2340 len 32768
23796 031/300 Read block 313 len 8192
23797 032/300 Read block 159 len 8192
23798 033/300 Flush
23799 034/300 Read block 1723 len 32768
23800 035/300 Write block 3576 len 32768 data: 3 2 2 2 2 2 4 4
23801 036/300 Write block 3203 len 8192 data: 5 5
23802 037/300 Write block 1467 len 4096 data: 2
23803 038/300 Write block 646 len 4096 data: 5
23804 039/300 Flush
23805 040/300 Read block 559 len 40960
23806 041/300 Flush
23807 042/300 Read block 4042 len 28672
23808 043/300 Write block 3073 len 24576 data: 6 5 5 6 5 6
23809 044/300 Read block 580 len 36864
23810 045/300 Write block 4406 len 32768 data: 6 6 6 4 4 4 4 3
23811 046/300 Write block 3546 len 8192 data: 4 5
23812 047/300 Flush
23813 048/300 Read block 1546 len 32768
23814 049/300 Write block 28 len 32768 data: 7 7 7 5 4 4 5 5
23815 050/300 Write block 3057 len 8192 data: 2 2
23816 051/300 Flush
23817 052/300 Read block 3747 len 32768
23818 053/300 Write block 4731 len 4096 data: 6
23819 054/300 Write block 1695 len 32768 data: 6 5 4 4 6 5 5 4
23820 055/300 Write block 3183 len 16384 data: 4 4 5 7
23821 056/300 Read block 3285 len 16384
23822 057/300 Read block 403 len 40960
23823 058/300 Read block 788 len 40960
23824 059/300 Write block 3474 len 24576 data: 8 7 6 4 2 2
23825 060/300 Read block 4126 len 28672
23826 061/300 Write block 2673 len 4096 data: 3
23827 062/300 Flush
23828 063/300 Flush
23829 064/300 Write block 2252 len 12288 data: 3 3 3
23830 065/300 Write block 2822 len 36864 data: 6 7 7 8 8 7 3 3 3
23831 066/300 Write block 1712 len 40960 data: 2 2 2 2 2 2 2 2 2 2
23832 067/300 Write block 4731 len 8192 data: 7 8
23833 068/300 Read block 3345 len 20480
23834 069/300 Read block 1885 len 12288
23835 070/300 Flush
23836 071/300 Flush
23837 072/300 Read block 4121 len 8192
23838 073/300 Flush
23839 074/300 Read block 1839 len 36864
23840 075/300 Write block 1141 len 40960 data: 6 7 8 8 7 6 8 8 7 6
23841 076/300 Read block 4087 len 32768
23842 077/300 Read block 381 len 36864
23843 078/300 Read block 2062 len 40960
23844 079/300 Write block 1844 len 4096 data: 4
23845 080/300 Read block 508 len 28672
23846 081/300 Write block 4625 len 36864 data: 5 5 5 4 3 3 3 4 3
23847 082/300 Read block 1366 len 32768
23848 083/300 Read block 1445 len 24576
23849 084/300 Write block 3300 len 16384 data: 2 3 3 2
23850 085/300 Write block 1246 len 40960 data: 3 5 5 4 2 3 5 5 5 4
23851 086/300 Flush
23852 087/300 Read block 3322 len 24576
23853 088/300 Read block 1904 len 16384
23854 089/300 Read block 4578 len 20480
23855 090/300 Read block 3900 len 32768
23856 091/300 Write block 4310 len 4096 data: 4
23857 092/300 Write block 329 len 20480 data: 6 6 4 3 3
23858 093/300 Read block 146 len 36864
23859 094/300 Flush
23860 095/300 Write block 2805 len 28672 data: 5 5 5 5 5 7 4
23861 096/300 Write block 2332 len 32768 data: 5 6 6 6 5 5 5 7
23862 097/300 Read block 140 len 36864
23863 098/300 Write block 2576 len 20480 data: 3 2 2 2 4
23864 099/300 Write block 4052 len 4096 data: 7
23865 100/300 Write block 2422 len 32768 data: 5 5 5 4 4 4 4 4
23866 101/300 Read block 2646 len 12288
23867 102/300 Write block 3299 len 12288 data: 2 3 4
23868 103/300 Read block 575 len 4096
23869 104/300 Flush
23870 105/300 Read block 4008 len 28672
23871 106/300 Read block 3809 len 24576
23872 107/300 Read block 2687 len 4096
23873 108/300 Read block 1975 len 36864
23874 109/300 Write block 2001 len 20480 data: 2 4 4 4 4
23875 110/300 Write block 404 len 16384 data: 4 3 2 4
23876 111/300 Write block 3300 len 36864 data: 4 5 4 3 2 3 3 3 4
23877 112/300 Read block 1647 len 8192
23878 113/300 Read block 133 len 4096
23879 114/300 Write block 480 len 20480 data: 6 6 4 4 5
23880 115/300 Read block 4061 len 36864
23881 116/300 Read block 4634 len 24576
23882 117/300 Read block 4552 len 24576
23883 118/300 Read block 3705 len 24576
23884 119/300 Read block 2430 len 28672
23885 120/300 Read block 2780 len 40960
23886 121/300 Read block 2467 len 32768
23887 122/300 Read block 3696 len 32768
23888 123/300 Read block 3491 len 4096
23889 124/300 Flush
23890 125/300 Write block 3192 len 20480 data: 11 8 6 7 7
23891 126/300 Read block 605 len 8192
23892 127/300 Read block 4382 len 36864
23893 128/300 Flush
23894 129/300 Write block 896 len 16384 data: 3 4 5 4
23895 130/300 Write block 2873 len 12288 data: 4 3 4
23896 131/300 Read block 4252 len 4096
23897 132/300 Read block 3657 len 12288
23898 133/300 Read block 48 len 8192
23899 134/300 Read block 1918 len 16384
23900 135/300 Write block 2052 len 24576 data: 8 6 6 4 4 4
23901 136/300 Write block 2573 len 28672 data: 3 3 3 4 3 3 3
23902 137/300 Read block 2412 len 32768
23903 138/300 Write block 278 len 24576 data: 3 2 2 4 6 7
23904 139/300 Write block 628 len 32768 data: 4 2 2 3 3 4 3 3
23905 140/300 Flush
23906 141/300 Read block 4384 len 40960
23907 142/300 Read block 1571 len 16384
23908 143/300 Read block 541 len 4096
23909 144/300 Write block 960 len 8192 data: 7 8
23910 145/300 Write block 1540 len 24576 data: 4 4 5 4 4 3
23911 146/300 Flush
23912 147/300 Write block 462 len 40960 data: 3 4 5 6 6 5 5 5 3 3
23913 148/300 Write block 4601 len 36864 data: 5 5 5 5 6 5 5 3 5
23914 149/300 Read block 863 len 4096
23915 150/300 Read block 4675 len 32768
23916 151/300 Read block 803 len 12288
23917 152/300 Write block 3604 len 4096 data: 3
23918 153/300 Write block 2826 len 8192 data: 9 8
23919 154/300 Write block 3213 len 24576 data: 5 5 4 5 5 5
23920 155/300 Read block 4874 len 24576
23921 156/300 Write block 1023 len 16384 data: 6 7 6 8
23922 157/300 Write block 1521 len 40960 data: 7 8 7 5 4 4 5 5 6 6
23923 158/300 Read block 2181 len 36864
23924 159/300 Read block 2670 len 36864
23925 160/300 Write block 4748 len 4096 data: 4
23926 161/300 Read block 328 len 28672
23927 162/300 Read block 2683 len 40960
23928 163/300 Read block 4346 len 4096
23929 164/300 Write block 1839 len 20480 data: 6 6 5 6 5
23930 165/300 Write block 3373 len 16384 data: 6 6 5 4
23931 166/300 Write block 4857 len 4096 data: 4
23932 167/300 Write block 978 len 32768 data: 5 3 3 5 5 4 5 5
23933 168/300 Write block 3132 len 24576 data: 2 2 3 3 5 4
23934 169/300 Read block 1542 len 20480
23935 170/300 Write block 773 len 4096 data: 6
23936 171/300 Read block 4029 len 8192
23937 172/300 Read block 4566 len 32768
23938 173/300 Read block 4786 len 32768
23939 174/300 Read block 2050 len 20480
23940 175/300 Read block 2086 len 16384
23941 176/300 Read block 2770 len 32768
23942 177/300 Read block 2968 len 12288
23943 178/300 Read block 443 len 28672
23944 179/300 Write block 1343 len 24576 data: 4 3 3 2 2 2
23945 180/300 Write block 2684 len 8192 data: 6 5
23946 181/300 Read block 4739 len 28672
23947 182/300 Write block 425 len 36864 data: 6 6 5 6 4 4 4 4 4
23948 183/300 Flush
23949 184/300 Read block 1109 len 8192
23950 185/300 Write block 1731 len 32768 data: 5 5 5 3 3 3 2 2
23951 186/300 Write block 2580 len 8192 data: 5 4
23952 187/300 Read block 1378 len 32768
23953 188/300 Write block 4898 len 20480 data: 2 2 2 2 3
23954 189/300 Write block 2595 len 20480 data: 11 13 10 7 7
23955 190/300 Write block 2385 len 16384 data: 4 4 3 3
23956 191/300 Write block 1694 len 12288 data: 4 7 6
23957 192/300 Read block 1434 len 4096
23958 193/300 Read block 4685 len 12288
23959 194/300 Read block 4307 len 40960
23960 195/300 Write block 2137 len 28672 data: 5 6 5 4 5 5 5
23961 196/300 Write block 4861 len 4096 data: 6
23962 197/300 Read block 4607 len 32768
23963 198/300 Write block 3706 len 32768 data: 8 8 7 7 6 6 4 3
23964 199/300 Write block 1547 len 4096 data: 4
23965 200/300 Read block 3780 len 8192
23966 201/300 Read block 2588 len 4096
23967 202/300 Write block 4541 len 40960 data: 3 3 4 4 4 5 6 6 5 4
23968 203/300 Read block 1204 len 40960
23969 204/300 Read block 3648 len 28672
23970 205/300 Read block 4000 len 32768
23971 206/300 Flush
23972 207/300 Write block 3848 len 36864 data: 4 4 5 6 5 6 8 7 6
23973 208/300 Read block 4643 len 4096
23974 209/300 Read block 1818 len 16384
23975 210/300 Read block 3305 len 8192
23976 211/300 Write block 1312 len 40960 data: 4 4 4 4 4 3 3 6 6 6
23977 212/300 Read block 3186 len 4096
23978 213/300 Read block 4102 len 36864
23979 214/300 Write block 3671 len 4096 data: 5
23980 215/300 Write block 2520 len 36864 data: 5 6 6 6 7 7 6 6 6
23981 216/300 Read block 2528 len 4096
23982 217/300 Write block 2354 len 28672 data: 3 3 3 3 4 3 3
23983 218/300 Read block 2469 len 36864
23984 219/300 Read block 3402 len 32768
23985 220/300 Read block 899 len 4096
23986 221/300 Read block 364 len 8192
23987 222/300 Write block 780 len 20480 data: 5 6 6 8 7
23988 223/300 Read block 934 len 20480
23989 224/300 Write block 284 len 40960 data: 8 10 10 10 9 7 8 8 6 3
23990 225/300 Read block 453 len 16384
23991 226/300 Read block 1924 len 32768
23992 227/300 Write block 2233 len 20480 data: 7 4 3 3 3
23993 228/300 Write block 2209 len 28672 data: 7 7 6 6 6 8 8
23994 229/300 Write block 2391 len 32768 data: 4 6 5 5 5 7 6 6
23995 230/300 Read block 13 len 40960
23996 231/300 Read block 3545 len 16384
23997 232/300 Read block 2713 len 32768
23998 233/300 Read block 3435 len 12288
23999 234/300 Write block 833 len 28672 data: 7 8 7 7 7 8 6
24000 235/300 Flush
24001 236/300 Read block 2934 len 16384
24002 237/300 Write block 2824 len 24576 data: 8 9 10 9 4 4
24003 238/300 Write block 3186 len 24576 data: 8 7 8 11 11 12
24004 239/300 Write block 3055 len 40960 data: 2 2 3 3 2 2 2 3 4 4
24005 240/300 Write block 2263 len 40960 data: 7 8 7 7 6 7 8 6 7 7
24006 241/300 Read block 1291 len 16384
24007 242/300 Write block 4423 len 12288 data: 3 4 3
24008 243/300 Read block 3448 len 12288
24009 244/300 Read block 2100 len 12288
24010 245/300 Read block 751 len 12288
24011 246/300 Read block 3101 len 8192
24012 247/300 Write block 928 len 32768 data: 3 2 2 2 2 3 3 3
24013 248/300 Read block 2420 len 4096
24014 249/300 Read block 2548 len 4096
24015 250/300 Read block 4381 len 24576
24016 251/300 Read block 4795 len 40960
24017 252/300 Read block 1055 len 36864
24018 253/300 Read block 3279 len 36864
24019 254/300 Flush
24020 255/300 Read block 2963 len 40960
24021 256/300 Write block 571 len 32768 data: 4 4 4 6 6 7 7 6
24022 257/300 Read block 1478 len 40960
24023 258/300 Read block 1147 len 16384
24024 259/300 Write block 4379 len 32768 data: 6 4 4 4 3 5 5 5
24025 260/300 Read block 2397 len 32768
24026 261/300 Flush
24027 262/300 Read block 1842 len 12288
24028 263/300 Write block 496 len 32768 data: 2 3 3 2 3 4 5 4
24029 264/300 Read block 2088 len 40960
24030 265/300 Write block 1509 len 8192 data: 5 5
24031 266/300 Read block 722 len 28672
24032 267/300 Write block 3243 len 36864 data: 4 6 4 4 5 5 5 4 3
24033 268/300 Read block 2411 len 36864
24034 269/300 Read block 3506 len 32768
24035 270/300 Read block 1249 len 40960
24036 271/300 Flush
24037 272/300 Write block 2413 len 28672 data: 5 4 5 5 5 6 5
24038 273/300 Read block 4072 len 40960
24039 274/300 Read block 4755 len 12288
24040 275/300 Write block 4656 len 4096 data: 4
24041 276/300 Write block 4428 len 4096 data: 5
24042 277/300 Read block 1529 len 16384
24043 278/300 Read block 1546 len 40960
24044 279/300 Write block 4974 len 4096 data: 6
24045 280/300 Read block 4880 len 16384
24046 281/300 Flush
24047 282/300 Read block 4332 len 8192
24048 283/300 Read block 4324 len 12288
24049 284/300 Write block 1135 len 12288 data: 3 3 2
24050 285/300 Write block 4672 len 32768 data: 5 6 8 7 5 5 3 3
24051 286/300 Read block 24 len 32768
24052 287/300 Read block 1856 len 40960
24053 288/300 Read block 4976 len 24576
24054 289/300 Read block 4424 len 24576
24055 290/300 Read block 4318 len 24576
24056 291/300 Read block 3533 len 32768
24057 292/300 Read block 1532 len 28672
24058 293/300 Read block 33 len 36864
24059 294/300 Read block 631 len 4096
24060 295/300 Write block 4007 len 12288 data: 4 5 4
24061 296/300 Write block 557 len 8192 data: 3 3
24062 297/300 Read block 568 len 32768
24063 298/300 Read block 4023 len 4096
24064 299/300 Flush
24065 300/300 Read block 660 len 20480
24066 301/300 Write block 4290 len 36864 data: 3 2 2 2 2 2 3 6 8
24067 Main task has work to do, go find it
24068 got dsc Start(1)
24069 start 1
24070 Jan 13 15:56:52.565 INFO request completed, latency_us: 169, response_code: 204, uri: /start/cid/1, method: POST, req_id: a12fcd7f-6818-4116-b864-d16b2857ab9e, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
24071 Replay: started 1, returned:Ok(())
24072 ----------------------------------------------------------------
24073 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
24074 States: Active Offline Active
24075 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24076 7340 Acked Write 4 Done Sent Done false
24077 7341 Acked Write 10 Done Sent Done false
24078 7342 Acked Write 2 Done Sent Done false
24079 7343 Acked Write 7 Done Sent Done false
24080 7344 Acked Read 1 Done Sent Done false
24081 7345 Acked Read 9 Done Sent Done false
24082 7346 Acked Read 10 Done Sent Done false
24083 7347 Acked Read 2 Done Sent Done false
24084 7348 Acked Write 9 Done Sent Done false
24085 7349 Acked Read 1 Done Sent Done false
24086 7350 Acked Write 1 Done Sent Done false
24087 7351 Acked Read 10 Done Sent Done false
24088 7352 Acked Read 2 Done Sent Done false
24089 7353 Acked Write 1 Done Sent Done false
24090 7354 Acked Read 10 Done Sent Done false
24091 7355 Acked Read 8 Done Sent Done false
24092 7356 Acked Read 8 Done Sent Done false
24093 7357 Acked Write 5 Done Sent Done false
24094 7358 Acked Write 7 Done Sent Done false
24095 7359 Acked Write 7 Done Sent Done false
24096 7360 Acked Read 8 Done Sent Done false
24097 7361 Acked Read 10 Done Sent Done false
24098 7362 Acked Flush 0 Done Sent Done false
24099 7363 Acked Write 6 Done Sent Done false
24100 7364 Acked Write 7 Done Sent Done false
24101 7365 Acked Read 3 Done Sent Done false
24102 7366 Acked Read 6 Done Sent Done false
24103 7367 Acked Write 2 Done Sent Done false
24104 7368 Acked Write 9 Done Sent Done false
24105 7369 Acked Read 8 Done Sent Done false
24106 7370 Acked Read 2 Done Sent Done false
24107 7371 Acked Read 2 Done Sent Done false
24108 7372 Acked Flush 0 Done Sent Done false
24109 7373 Acked Read 8 Done Sent Done false
24110 7374 Acked Write 8 Done Sent Done false
24111 7375 Acked Write 2 Done Sent Done false
24112 7376 Acked Write 1 Done Sent Done false
24113 7377 Acked Write 1 Done Sent Done false
24114 7378 Acked Flush 0 Done Sent Done false
24115 7379 Acked Read 10 Done Sent Done false
24116 7380 Acked Flush 0 Done Sent Done false
24117 7381 Acked Read 7 Done Sent Done false
24118 7382 Acked Write 6 Done Sent Done false
24119 7383 Acked Read 9 Done Sent Done false
24120 7384 Acked Write 8 Done Sent Done false
24121 7385 Acked Write 2 Done Sent Done false
24122 7386 Acked Flush 0 Done Sent Done false
24123 7387 Acked Read 8 Done Sent Done false
24124 7388 Acked Write 8 Done Sent Done false
24125 7389 Acked Write 2 Done Sent Done false
24126 7390 Acked Flush 0 Done Sent Done false
24127 7391 Acked Read 8 Done Sent Done false
24128 7392 Acked Write 1 Done Sent Done false
24129 7393 Acked Write 8 Done Sent Done false
24130 7394 Acked Write 4 Done Sent Done false
24131 7395 Acked Read 4 Done Sent Done false
24132 7396 Acked Read 10 Done Sent Done false
24133 7397 Acked Read 10 Done Sent Done false
24134 7398 Acked Write 6 Done Sent Done false
24135 7399 Acked Read 7 Done Sent Done false
24136 7400 Acked Write 1 Done Sent Done false
24137 7401 Acked Flush 0 Done Sent Done false
24138 7402 Acked Flush 0 Done Sent Done false
24139 7403 Acked Write 3 Done Sent Done false
24140 7404 Acked Write 9 Done Sent Done false
24141 7405 Acked Write 10 Done Sent Done false
24142 7406 Acked Write 2 Done Sent Done false
24143 7407 Acked Read 5 Done Sent Done false
24144 7408 Acked Read 3 Done Sent Done false
24145 7409 Acked Flush 0 Done Sent Done false
24146 7410 Acked Flush 0 Done Sent Done false
24147 7411 Acked Read 2 Done Sent Done false
24148 7412 Acked Flush 0 Done Sent Done false
24149 7413 Acked Read 9 Done Sent Done false
24150 7414 Acked Write 10 Done Sent Done false
24151 7415 Acked Read 8 Done Sent Done false
24152 7416 Acked Read 9 Done Sent Done false
24153 7417 Acked Read 10 Done Sent Done false
24154 7418 Acked Write 1 Done Sent Done false
24155 7419 Acked Read 7 Done Sent Done false
24156 7420 Acked Write 9 Done Sent Done false
24157 7421 Acked Read 8 Done Sent Done false
24158 7422 Acked Read 6 Done Sent Done false
24159 7423 Acked Write 4 Done Sent Done false
24160 7424 Acked Write 10 Done Sent Done false
24161 7425 Acked Flush 0 Done Sent Done false
24162 7426 Acked Read 6 Done Sent Done false
24163 7427 Acked Read 4 Done Sent Done false
24164 7428 Acked Read 5 Done Sent Done false
24165 7429 Acked Read 8 Done Sent Done false
24166 7430 Acked Write 1 Done Sent Done false
24167 7431 Acked Write 5 Done Sent Done false
24168 7432 Acked Read 9 Done Sent Done false
24169 7433 Acked Flush 0 Done Sent Done false
24170 7434 Acked Write 7 Done Sent Done false
24171 7435 Acked Write 8 Done Sent Done false
24172 7436 Acked Read 9 Done Sent Done false
24173 7437 Acked Write 5 Done Sent Done false
24174 7438 Acked Write 1 Done Sent Done false
24175 7439 Acked Write 8 Done Sent Done false
24176 7440 Acked Read 3 Done Sent Done false
24177 7441 Acked Write 3 Done Sent Done false
24178 7442 Acked Read 1 Done Sent Done false
24179 7443 Acked Flush 0 Done Sent Done false
24180 7444 Acked Read 7 Done Sent Done false
24181 7445 Acked Read 6 Done Sent Done false
24182 7446 Acked Read 1 Done Sent Done false
24183 7447 Acked Read 9 Done Sent Done false
24184 7448 Acked Write 5 Done Sent Done false
24185 7449 Acked Write 4 Done Sent Done false
24186 7450 Acked Write 9 Done Sent Done false
24187 7451 Acked Read 2 Done Sent Done false
24188 7452 Acked Read 1 Done Sent Done false
24189 7453 Acked Write 5 Done Sent Done false
24190 7454 Acked Read 9 Done Sent Done false
24191 7455 Acked Read 6 Done Sent Done false
24192 7456 Acked Read 6 Done Sent Done false
24193 7457 Acked Read 6 Done Sent Done false
24194 7458 Acked Read 7 Done Sent Done false
24195 7459 Acked Read 10 Done Sent Done false
24196 7460 Acked Read 8 Done Sent Done false
24197 7461 Acked Read 8 Done Sent Done false
24198 7462 Acked Read 1 Done Sent Done false
24199 7463 Acked Flush 0 Done Sent Done false
24200 7464 Acked Write 5 Done Sent Done false
24201 7465 Acked Read 2 Done Sent Done false
24202 7466 Acked Read 9 Done Sent Done false
24203 7467 Acked Flush 0 Done Sent Done false
24204 7468 Acked Write 4 Done Sent Done false
24205 7469 Acked Write 3 Done Sent Done false
24206 7470 Acked Read 1 Done Sent Done false
24207 7471 Acked Read 3 Done Sent Done false
24208 7472 Acked Read 2 Done Sent Done false
24209 7473 Acked Read 4 Done Sent Done false
24210 7474 Acked Write 6 Done Sent Done false
24211 7475 Acked Write 7 Done Sent Done false
24212 7476 Acked Read 8 Done Sent Done false
24213 7477 Acked Write 6 Done Sent Done false
24214 7478 Acked Write 8 Done Sent Done false
24215 7479 Acked Flush 0 Done Sent Done false
24216 7480 Acked Read 10 Done Sent Done false
24217 7481 Acked Read 4 Done Sent Done false
24218 7482 Acked Read 1 Done Sent Done false
24219 7483 Acked Write 2 Done Sent Done false
24220 7484 Acked Write 6 Done Sent Done false
24221 7485 Acked Flush 0 Done Sent Done false
24222 7486 Acked Write 10 Done Sent Done false
24223 7487 Acked Write 9 Done Sent Done false
24224 7488 Acked Read 1 Done Sent Done false
24225 7489 Acked Read 8 Done Sent Done false
24226 7490 Acked Read 3 Done Sent Done false
24227 7491 Acked Write 1 Done Sent Done false
24228 7492 Acked Write 2 Done Sent Done false
24229 7493 Acked Write 6 Done Sent Done false
24230 7494 Acked Read 6 Done Sent Done false
24231 7495 Acked Write 4 Done Sent Done false
24232 7496 Acked Write 10 Done Sent Done false
24233 7497 Acked Read 9 Done Sent Done false
24234 7498 Acked Read 9 Done Sent Done false
24235 7499 Acked Write 1 Done Sent Done false
24236 7500 Acked Read 7 Done Sent Done false
24237 7501 Acked Read 10 Done Sent Done false
24238 7502 Acked Read 1 Done Sent Done false
24239 7503 Acked Write 5 Done Sent Done false
24240 7504 Acked Write 4 Done Sent Done false
24241 7505 Acked Write 1 Done Sent Done false
24242 7506 Acked Write 8 Done Sent Done false
24243 7507 Acked Write 6 Done Sent Done false
24244 7508 Acked Read 5 Done Sent Done false
24245 7509 Acked Write 1 Done Sent Done false
24246 7510 Acked Read 2 Done Sent Done false
24247 7511 Acked Read 8 Done Sent Done false
24248 7512 Acked Read 8 Done Sent Done false
24249 7513 Acked Read 5 Done Sent Done false
24250 7514 Acked Read 4 Done Sent Done false
24251 7515 Acked Read 8 Done Sent Done false
24252 7516 Acked Read 3 Done Sent Done false
24253 7517 Acked Read 7 Done Sent Done false
24254 7518 Acked Write 6 Done Sent Done false
24255 7519 Acked Write 2 Done Sent Done false
24256 7520 Acked Read 7 Done Sent Done false
24257 7521 Acked Write 9 Done Sent Done false
24258 7522 Acked Flush 0 Done Sent Done false
24259 7523 Acked Read 2 Done Sent Done false
24260 7524 Acked Write 8 Done Sent Done false
24261 7525 Acked Write 2 Done Sent Done false
24262 7526 Acked Read 8 Done Sent Done false
24263 7527 Acked Write 5 Done Sent Done false
24264 7528 Acked Write 5 Done Sent Done false
24265 7529 Acked Write 4 Done Sent Done false
24266 7530 Acked Write 3 Done Sent Done false
24267 7531 Acked Read 1 Done Sent Done false
24268 7532 Acked Read 3 Done Sent Done false
24269 7533 Acked Read 10 Done Sent Done false
24270 7534 Acked Write 7 Done Sent Done false
24271 7535 Acked Write 1 Done Sent Done false
24272 7536 Acked Read 8 Done Sent Done false
24273 7537 Acked Write 8 Done Sent Done false
24274 7538 Acked Write 1 Done Sent Done false
24275 7539 Acked Read 2 Done Sent Done false
24276 7540 Acked Read 1 Done Sent Done false
24277 7541 Acked Write 10 Done Sent Done false
24278 7542 Acked Read 10 Done Sent Done false
24279 7543 Acked Read 7 Done Sent Done false
24280 7544 Acked Read 8 Done Sent Done false
24281 7545 Acked Flush 0 Done Sent Done false
24282 7546 Acked Write 9 Done Sent Done false
24283 7547 Acked Read 1 Done Sent Done false
24284 7548 Acked Read 4 Done Sent Done false
24285 7549 Acked Read 2 Done Sent Done false
24286 7550 Acked Write 10 Done Sent Done false
24287 7551 Acked Read 1 Done Sent Done false
24288 7552 Acked Read 9 Done Sent Done false
24289 7553 Acked Write 1 Done Sent Done false
24290 7554 Acked Write 9 Done Sent Done false
24291 7555 Acked Read 1 Done Sent Done false
24292 7556 Acked Write 7 Done Sent Done false
24293 7557 Acked Read 9 Done Sent Done false
24294 7558 Acked Read 8 Done Sent Done false
24295 7559 Acked Read 1 Done Sent Done false
24296 7560 Acked Read 2 Done Sent Done false
24297 7561 Acked Write 5 Done Sent Done false
24298 7562 Acked Read 5 Done Sent Done false
24299 7563 Acked Write 10 Done Sent Done false
24300 7564 Acked Read 4 Done Sent Done false
24301 7565 Acked Read 8 Done Sent Done false
24302 7566 Acked Write 5 Done Sent Done false
24303 7567 Acked Write 7 Done Sent Done false
24304 7568 Acked Write 8 Done Sent Done false
24305 7569 Acked Read 10 Done Sent Done false
24306 7570 Acked Read 4 Done Sent Done false
24307 7571 Acked Read 8 Done Sent Done false
24308 7572 Acked Read 3 Done Sent Done false
24309 7573 Acked Write 7 Done Sent Done false
24310 7574 Acked Flush 0 Done Sent Done false
24311 7575 Acked Read 4 Done Sent Done false
24312 7576 Acked Write 6 Done Sent Done false
24313 7577 Acked Write 6 Done Sent Done false
24314 7578 Acked Write 10 Done Sent Done false
24315 7579 Acked Write 10 Done Sent Done false
24316 7580 Acked Read 4 Done Sent Done false
24317 7581 Acked Write 3 Done Sent Done false
24318 7582 Acked Read 3 Done Sent Done false
24319 7583 Acked Read 3 Done Sent Done false
24320 7584 Acked Read 3 Done Sent Done false
24321 7585 Acked Read 2 Done Sent Done false
24322 7586 Acked Write 8 Done Sent Done false
24323 7587 Acked Read 1 Done Sent Done false
24324 7588 Acked Read 1 Done Sent Done false
24325 7589 Acked Read 6 Done Sent Done false
24326 7590 Acked Read 10 Done Sent Done false
24327 7591 Acked Read 9 Done Sent Done false
24328 7592 Acked Read 9 Done Sent Done false
24329 7593 Acked Flush 0 Done Sent Done false
24330 7594 Acked Read 10 Done Sent Done false
24331 7595 Acked Write 8 Done Sent Done false
24332 7596 Acked Read 10 Done Sent Done false
24333 7597 Acked Read 4 Done Sent Done false
24334 7598 Acked Write 8 Done Sent Done false
24335 7599 Acked Read 8 Done Sent Done false
24336 7600 Acked Flush 0 Done Sent Done false
24337 7601 Acked Read 3 Done Sent Done false
24338 7602 Acked Write 8 Done Sent Done false
24339 7603 Acked Read 10 Done Sent Done false
24340 7604 Acked Write 2 Done Sent Done false
24341 7605 Acked Read 7 Done Sent Done false
24342 7606 Acked Write 9 Done Sent Done false
24343 7607 Acked Read 9 Done Sent Done false
24344 7608 Acked Read 8 Done Sent Done false
24345 7609 Acked Read 10 Done Sent Done false
24346 7610 Acked Flush 0 Done Sent Done false
24347 7611 Acked Write 7 Done Sent Done false
24348 7612 Acked Read 10 Done Sent Done false
24349 7613 Acked Read 3 Done Sent Done false
24350 7614 Acked Write 1 Done Sent Done false
24351 7615 Acked Write 1 Done Sent Done false
24352 7616 Acked Read 4 Done Sent Done false
24353 7617 Acked Read 10 Done Sent Done false
24354 7618 Acked Write 1 Done Sent Done false
24355 7619 Acked Read 4 Done Sent Done false
24356 7620 Acked Flush 0 Done Sent Done false
24357 7621 Acked Read 2 Done Sent Done false
24358 7622 Acked Read 3 Done Sent Done false
24359 7623 Acked Write 3 Done Sent Done false
24360 7624 Acked Write 8 Done Sent Done false
24361 7625 Acked Read 8 Done Sent Done false
24362 7626 Acked Read 10 Done Sent Done false
24363 7627 Acked Read 6 Done Sent Done false
24364 7628 Acked Read 6 Done Sent Done false
24365 7629 Acked Read 6 Done Sent Done false
24366 7630 Acked Read 8 Done Sent Done false
24367 7631 Acked Read 7 Done Sent Done false
24368 7632 Acked Read 9 Done Sent Done false
24369 7633 Acked Read 1 Done Sent Done false
24370 7634 Acked Write 3 Done Sent Done false
24371 7635 Acked Write 2 Done Sent Done false
24372 7636 Acked Read 8 Done Sent Done false
24373 7637 Acked Read 1 Done Sent Done false
24374 7638 Acked Flush 0 Done Sent Done false
24375 7639 Acked Read 5 Done Sent Done false
24376 7640 Acked Write 9 Sent Sent Sent false
24377 STATES DS:0 DS:1 DS:2 TOTAL
24378 Sent 1 301 1 303
24379 Done 300 0 300 600
24380 Skipped 0 0 0 0
24381 Error 0 0 0 0
24382 Last Flush: 7638 7339 7638
24383 Downstairs last five completed: 7339 7338 7337 7336 7335
24384 Upstairs last five completed: 7640 7639 7638 7637 7636
24385 CLIENT: Up:0 ds:301 act:2
24386 [8820] got start action
24387 [8820] I am going to restart
24388 Starting downstairs at port 8820
24389 Make output file at "/tmp/dsc/downstairs-8820.txt"
24390 [8820][1] reports Starting
24391 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1224)
24392 [8820][1] reports Running
24393 ----------------------------------------------------------------
24394 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24395 States: Active Offline Active
24396 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24397 7340 Acked Write 4 Done Sent Done false
24398 7341 Acked Write 10 Done Sent Done false
24399 7342 Acked Write 2 Done Sent Done false
24400 7343 Acked Write 7 Done Sent Done false
24401 7344 Acked Read 1 Done Sent Done false
24402 7345 Acked Read 9 Done Sent Done false
24403 7346 Acked Read 10 Done Sent Done false
24404 7347 Acked Read 2 Done Sent Done false
24405 7348 Acked Write 9 Done Sent Done false
24406 7349 Acked Read 1 Done Sent Done false
24407 7350 Acked Write 1 Done Sent Done false
24408 7351 Acked Read 10 Done Sent Done false
24409 7352 Acked Read 2 Done Sent Done false
24410 7353 Acked Write 1 Done Sent Done false
24411 7354 Acked Read 10 Done Sent Done false
24412 7355 Acked Read 8 Done Sent Done false
24413 7356 Acked Read 8 Done Sent Done false
24414 7357 Acked Write 5 Done Sent Done false
24415 7358 Acked Write 7 Done Sent Done false
24416 7359 Acked Write 7 Done Sent Done false
24417 7360 Acked Read 8 Done Sent Done false
24418 7361 Acked Read 10 Done Sent Done false
24419 7362 Acked Flush 0 Done Sent Done false
24420 7363 Acked Write 6 Done Sent Done false
24421 7364 Acked Write 7 Done Sent Done false
24422 7365 Acked Read 3 Done Sent Done false
24423 7366 Acked Read 6 Done Sent Done false
24424 7367 Acked Write 2 Done Sent Done false
24425 7368 Acked Write 9 Done Sent Done false
24426 7369 Acked Read 8 Done Sent Done false
24427 7370 Acked Read 2 Done Sent Done false
24428 7371 Acked Read 2 Done Sent Done false
24429 7372 Acked Flush 0 Done Sent Done false
24430 7373 Acked Read 8 Done Sent Done false
24431 7374 Acked Write 8 Done Sent Done false
24432 7375 Acked Write 2 Done Sent Done false
24433 7376 Acked Write 1 Done Sent Done false
24434 7377 Acked Write 1 Done Sent Done false
24435 7378 Acked Flush 0 Done Sent Done false
24436 7379 Acked Read 10 Done Sent Done false
24437 7380 Acked Flush 0 Done Sent Done false
24438 7381 Acked Read 7 Done Sent Done false
24439 7382 Acked Write 6 Done Sent Done false
24440 7383 Acked Read 9 Done Sent Done false
24441 7384 Acked Write 8 Done Sent Done false
24442 7385 Acked Write 2 Done Sent Done false
24443 7386 Acked Flush 0 Done Sent Done false
24444 7387 Acked Read 8 Done Sent Done false
24445 7388 Acked Write 8 Done Sent Done false
24446 7389 Acked Write 2 Done Sent Done false
24447 7390 Acked Flush 0 Done Sent Done false
24448 7391 Acked Read 8 Done Sent Done false
24449 7392 Acked Write 1 Done Sent Done false
24450 7393 Acked Write 8 Done Sent Done false
24451 7394 Acked Write 4 Done Sent Done false
24452 7395 Acked Read 4 Done Sent Done false
24453 7396 Acked Read 10 Done Sent Done false
24454 7397 Acked Read 10 Done Sent Done false
24455 7398 Acked Write 6 Done Sent Done false
24456 7399 Acked Read 7 Done Sent Done false
24457 7400 Acked Write 1 Done Sent Done false
24458 7401 Acked Flush 0 Done Sent Done false
24459 7402 Acked Flush 0 Done Sent Done false
24460 7403 Acked Write 3 Done Sent Done false
24461 7404 Acked Write 9 Done Sent Done false
24462 7405 Acked Write 10 Done Sent Done false
24463 7406 Acked Write 2 Done Sent Done false
24464 7407 Acked Read 5 Done Sent Done false
24465 7408 Acked Read 3 Done Sent Done false
24466 7409 Acked Flush 0 Done Sent Done false
24467 7410 Acked Flush 0 Done Sent Done false
24468 7411 Acked Read 2 Done Sent Done false
24469 7412 Acked Flush 0 Done Sent Done false
24470 7413 Acked Read 9 Done Sent Done false
24471 7414 Acked Write 10 Done Sent Done false
24472 7415 Acked Read 8 Done Sent Done false
24473 7416 Acked Read 9 Done Sent Done false
24474 7417 Acked Read 10 Done Sent Done false
24475 7418 Acked Write 1 Done Sent Done false
24476 7419 Acked Read 7 Done Sent Done false
24477 7420 Acked Write 9 Done Sent Done false
24478 7421 Acked Read 8 Done Sent Done false
24479 7422 Acked Read 6 Done Sent Done false
24480 7423 Acked Write 4 Done Sent Done false
24481 7424 Acked Write 10 Done Sent Done false
24482 7425 Acked Flush 0 Done Sent Done false
24483 7426 Acked Read 6 Done Sent Done false
24484 7427 Acked Read 4 Done Sent Done false
24485 7428 Acked Read 5 Done Sent Done false
24486 7429 Acked Read 8 Done Sent Done false
24487 7430 Acked Write 1 Done Sent Done false
24488 7431 Acked Write 5 Done Sent Done false
24489 7432 Acked Read 9 Done Sent Done false
24490 7433 Acked Flush 0 Done Sent Done false
24491 7434 Acked Write 7 Done Sent Done false
24492 7435 Acked Write 8 Done Sent Done false
24493 7436 Acked Read 9 Done Sent Done false
24494 7437 Acked Write 5 Done Sent Done false
24495 7438 Acked Write 1 Done Sent Done false
24496 7439 Acked Write 8 Done Sent Done false
24497 7440 Acked Read 3 Done Sent Done false
24498 7441 Acked Write 3 Done Sent Done false
24499 7442 Acked Read 1 Done Sent Done false
24500 7443 Acked Flush 0 Done Sent Done false
24501 7444 Acked Read 7 Done Sent Done false
24502 7445 Acked Read 6 Done Sent Done false
24503 7446 Acked Read 1 Done Sent Done false
24504 7447 Acked Read 9 Done Sent Done false
24505 7448 Acked Write 5 Done Sent Done false
24506 7449 Acked Write 4 Done Sent Done false
24507 7450 Acked Write 9 Done Sent Done false
24508 7451 Acked Read 2 Done Sent Done false
24509 7452 Acked Read 1 Done Sent Done false
24510 7453 Acked Write 5 Done Sent Done false
24511 7454 Acked Read 9 Done Sent Done false
24512 7455 Acked Read 6 Done Sent Done false
24513 7456 Acked Read 6 Done Sent Done false
24514 7457 Acked Read 6 Done Sent Done false
24515 7458 Acked Read 7 Done Sent Done false
24516 7459 Acked Read 10 Done Sent Done false
24517 7460 Acked Read 8 Done Sent Done false
24518 7461 Acked Read 8 Done Sent Done false
24519 7462 Acked Read 1 Done Sent Done false
24520 7463 Acked Flush 0 Done Sent Done false
24521 7464 Acked Write 5 Done Sent Done false
24522 7465 Acked Read 2 Done Sent Done false
24523 7466 Acked Read 9 Done Sent Done false
24524 7467 Acked Flush 0 Done Sent Done false
24525 7468 Acked Write 4 Done Sent Done false
24526 7469 Acked Write 3 Done Sent Done false
24527 7470 Acked Read 1 Done Sent Done false
24528 7471 Acked Read 3 Done Sent Done false
24529 7472 Acked Read 2 Done Sent Done false
24530 7473 Acked Read 4 Done Sent Done false
24531 7474 Acked Write 6 Done Sent Done false
24532 7475 Acked Write 7 Done Sent Done false
24533 7476 Acked Read 8 Done Sent Done false
24534 7477 Acked Write 6 Done Sent Done false
24535 7478 Acked Write 8 Done Sent Done false
24536 7479 Acked Flush 0 Done Sent Done false
24537 7480 Acked Read 10 Done Sent Done false
24538 7481 Acked Read 4 Done Sent Done false
24539 7482 Acked Read 1 Done Sent Done false
24540 7483 Acked Write 2 Done Sent Done false
24541 7484 Acked Write 6 Done Sent Done false
24542 7485 Acked Flush 0 Done Sent Done false
24543 7486 Acked Write 10 Done Sent Done false
24544 7487 Acked Write 9 Done Sent Done false
24545 7488 Acked Read 1 Done Sent Done false
24546 7489 Acked Read 8 Done Sent Done false
24547 7490 Acked Read 3 Done Sent Done false
24548 7491 Acked Write 1 Done Sent Done false
24549 7492 Acked Write 2 Done Sent Done false
24550 7493 Acked Write 6 Done Sent Done false
24551 7494 Acked Read 6 Done Sent Done false
24552 7495 Acked Write 4 Done Sent Done false
24553 7496 Acked Write 10 Done Sent Done false
24554 7497 Acked Read 9 Done Sent Done false
24555 7498 Acked Read 9 Done Sent Done false
24556 7499 Acked Write 1 Done Sent Done false
24557 7500 Acked Read 7 Done Sent Done false
24558 7501 Acked Read 10 Done Sent Done false
24559 7502 Acked Read 1 Done Sent Done false
24560 7503 Acked Write 5 Done Sent Done false
24561 7504 Acked Write 4 Done Sent Done false
24562 7505 Acked Write 1 Done Sent Done false
24563 7506 Acked Write 8 Done Sent Done false
24564 7507 Acked Write 6 Done Sent Done false
24565 7508 Acked Read 5 Done Sent Done false
24566 7509 Acked Write 1 Done Sent Done false
24567 7510 Acked Read 2 Done Sent Done false
24568 7511 Acked Read 8 Done Sent Done false
24569 7512 Acked Read 8 Done Sent Done false
24570 7513 Acked Read 5 Done Sent Done false
24571 7514 Acked Read 4 Done Sent Done false
24572 7515 Acked Read 8 Done Sent Done false
24573 7516 Acked Read 3 Done Sent Done false
24574 7517 Acked Read 7 Done Sent Done false
24575 7518 Acked Write 6 Done Sent Done false
24576 7519 Acked Write 2 Done Sent Done false
24577 7520 Acked Read 7 Done Sent Done false
24578 7521 Acked Write 9 Done Sent Done false
24579 7522 Acked Flush 0 Done Sent Done false
24580 7523 Acked Read 2 Done Sent Done false
24581 7524 Acked Write 8 Done Sent Done false
24582 7525 Acked Write 2 Done Sent Done false
24583 7526 Acked Read 8 Done Sent Done false
24584 7527 Acked Write 5 Done Sent Done false
24585 7528 Acked Write 5 Done Sent Done false
24586 7529 Acked Write 4 Done Sent Done false
24587 7530 Acked Write 3 Done Sent Done false
24588 7531 Acked Read 1 Done Sent Done false
24589 7532 Acked Read 3 Done Sent Done false
24590 7533 Acked Read 10 Done Sent Done false
24591 7534 Acked Write 7 Done Sent Done false
24592 7535 Acked Write 1 Done Sent Done false
24593 7536 Acked Read 8 Done Sent Done false
24594 7537 Acked Write 8 Done Sent Done false
24595 7538 Acked Write 1 Done Sent Done false
24596 7539 Acked Read 2 Done Sent Done false
24597 7540 Acked Read 1 Done Sent Done false
24598 7541 Acked Write 10 Done Sent Done false
24599 7542 Acked Read 10 Done Sent Done false
24600 7543 Acked Read 7 Done Sent Done false
24601 7544 Acked Read 8 Done Sent Done false
24602 7545 Acked Flush 0 Done Sent Done false
24603 7546 Acked Write 9 Done Sent Done false
24604 7547 Acked Read 1 Done Sent Done false
24605 7548 Acked Read 4 Done Sent Done false
24606 7549 Acked Read 2 Done Sent Done false
24607 7550 Acked Write 10 Done Sent Done false
24608 7551 Acked Read 1 Done Sent Done false
24609 7552 Acked Read 9 Done Sent Done false
24610 7553 Acked Write 1 Done Sent Done false
24611 7554 Acked Write 9 Done Sent Done false
24612 7555 Acked Read 1 Done Sent Done false
24613 7556 Acked Write 7 Done Sent Done false
24614 7557 Acked Read 9 Done Sent Done false
24615 7558 Acked Read 8 Done Sent Done false
24616 7559 Acked Read 1 Done Sent Done false
24617 7560 Acked Read 2 Done Sent Done false
24618 7561 Acked Write 5 Done Sent Done false
24619 7562 Acked Read 5 Done Sent Done false
24620 7563 Acked Write 10 Done Sent Done false
24621 7564 Acked Read 4 Done Sent Done false
24622 7565 Acked Read 8 Done Sent Done false
24623 7566 Acked Write 5 Done Sent Done false
24624 7567 Acked Write 7 Done Sent Done false
24625 7568 Acked Write 8 Done Sent Done false
24626 7569 Acked Read 10 Done Sent Done false
24627 7570 Acked Read 4 Done Sent Done false
24628 7571 Acked Read 8 Done Sent Done false
24629 7572 Acked Read 3 Done Sent Done false
24630 7573 Acked Write 7 Done Sent Done false
24631 7574 Acked Flush 0 Done Sent Done false
24632 7575 Acked Read 4 Done Sent Done false
24633 7576 Acked Write 6 Done Sent Done false
24634 7577 Acked Write 6 Done Sent Done false
24635 7578 Acked Write 10 Done Sent Done false
24636 7579 Acked Write 10 Done Sent Done false
24637 7580 Acked Read 4 Done Sent Done false
24638 7581 Acked Write 3 Done Sent Done false
24639 7582 Acked Read 3 Done Sent Done false
24640 7583 Acked Read 3 Done Sent Done false
24641 7584 Acked Read 3 Done Sent Done false
24642 7585 Acked Read 2 Done Sent Done false
24643 7586 Acked Write 8 Done Sent Done false
24644 7587 Acked Read 1 Done Sent Done false
24645 7588 Acked Read 1 Done Sent Done false
24646 7589 Acked Read 6 Done Sent Done false
24647 7590 Acked Read 10 Done Sent Done false
24648 7591 Acked Read 9 Done Sent Done false
24649 7592 Acked Read 9 Done Sent Done false
24650 7593 Acked Flush 0 Done Sent Done false
24651 7594 Acked Read 10 Done Sent Done false
24652 7595 Acked Write 8 Done Sent Done false
24653 7596 Acked Read 10 Done Sent Done false
24654 7597 Acked Read 4 Done Sent Done false
24655 7598 Acked Write 8 Done Sent Done false
24656 7599 Acked Read 8 Done Sent Done false
24657 7600 Acked Flush 0 Done Sent Done false
24658 7601 Acked Read 3 Done Sent Done false
24659 7602 Acked Write 8 Done Sent Done false
24660 7603 Acked Read 10 Done Sent Done false
24661 7604 Acked Write 2 Done Sent Done false
24662 7605 Acked Read 7 Done Sent Done false
24663 7606 Acked Write 9 Done Sent Done false
24664 7607 Acked Read 9 Done Sent Done false
24665 7608 Acked Read 8 Done Sent Done false
24666 7609 Acked Read 10 Done Sent Done false
24667 7610 Acked Flush 0 Done Sent Done false
24668 7611 Acked Write 7 Done Sent Done false
24669 7612 Acked Read 10 Done Sent Done false
24670 7613 Acked Read 3 Done Sent Done false
24671 7614 Acked Write 1 Done Sent Done false
24672 7615 Acked Write 1 Done Sent Done false
24673 7616 Acked Read 4 Done Sent Done false
24674 7617 Acked Read 10 Done Sent Done false
24675 7618 Acked Write 1 Done Sent Done false
24676 7619 Acked Read 4 Done Sent Done false
24677 7620 Acked Flush 0 Done Sent Done false
24678 7621 Acked Read 2 Done Sent Done false
24679 7622 Acked Read 3 Done Sent Done false
24680 7623 Acked Write 3 Done Sent Done false
24681 7624 Acked Write 8 Done Sent Done false
24682 7625 Acked Read 8 Done Sent Done false
24683 7626 Acked Read 10 Done Sent Done false
24684 7627 Acked Read 6 Done Sent Done false
24685 7628 Acked Read 6 Done Sent Done false
24686 7629 Acked Read 6 Done Sent Done false
24687 7630 Acked Read 8 Done Sent Done false
24688 7631 Acked Read 7 Done Sent Done false
24689 7632 Acked Read 9 Done Sent Done false
24690 7633 Acked Read 1 Done Sent Done false
24691 7634 Acked Write 3 Done Sent Done false
24692 7635 Acked Write 2 Done Sent Done false
24693 7636 Acked Read 8 Done Sent Done false
24694 7637 Acked Read 1 Done Sent Done false
24695 7638 Acked Flush 0 Done Sent Done false
24696 7639 Acked Read 5 Done Sent Done false
24697 7640 Acked Write 9 Done Sent Done false
24698 7641 Acked Flush 0 Done Sent Done false
24699 STATES DS:0 DS:1 DS:2 TOTAL
24700 Sent 0 302 0 302
24701 Done 302 0 302 604
24702 Skipped 0 0 0 0
24703 Error 0 0 0 0
24704 Last Flush: 7641 7339 7641
24705 Downstairs last five completed: 7339 7338 7337 7336 7335
24706 Upstairs last five completed: 7641 7640 7639 7638 7637
24707 CLIENT: Up:0 ds:302 act:2
247082025-01-13T15:56:58.317ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
247092025-01-13T15:56:58.317ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61806) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
247102025-01-13T15:56:58.319ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
247112025-01-13T15:56:58.319ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
247122025-01-13T15:56:58.319ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
247132025-01-13T15:56:58.319ZINFOcrucible: send last flush ID to this DS: 7339 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
247142025-01-13T15:56:58.320ZINFOcrucible: Replied this last flush ID: 7339 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
247152025-01-13T15:56:58.320ZINFOcrucible: [1] client re-new 302 jobs since flush 7339 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
247162025-01-13T15:56:58.320ZINFOcrucible: [1] Replayed 302 jobs since flush: 7339 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
24717 ----------------------------------------------------------------
24718 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
24719 Downstairs last five completed: 7641 7640 7639 7638 7637
24720 Upstairs last five completed: 7641 7640 7639 7638 7637
24721 CLIENT: Up:0 ds:0 act:3
24722 Replay: All jobs finished
24723 Main task has work to do, go find it
24724 got dsc Stop(0)
24725 stop 0
24726 Jan 13 15:57:00.572[8810] Got stop action so:false kr:false
24727 INFO request completed, latency_us: 557, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 56014bcb-cb68-4fea-8991-d6888ebca23a, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
247282025-01-13T15:57:00.572ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
24729 Jan 13 15:57:00.572 INFO request completed, latency_us: 148, response_code: 200, uri: /state/cid/0, method: GET, req_id: 3a4e8e3e-5bf7-411b-81a0-4e1d640860af, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
247302025-01-13T15:57:00.572ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
247312025-01-13T15:57:00.573ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
247322025-01-13T15:57:00.573ZINFOcrucible: client task is exiting = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
24733 [8810] Exited with: ExitStatus(unix_wait_status(9))
24734 [8810][0] reports Exit
24735 Jan 13 15:57:04.575 INFO request completed, latency_us: 315, response_code: 200, uri: /state/cid/0, method: GET, req_id: e8da1a9b-913d-4c44-8f8a-a19e2bc18b6c, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
24736 001/300 Write block 4734 len 40960 data: 9 10 9 9 8 7 7 5 4 4
24737 002/300 Read block 4019 len 40960
24738 003/300 Write block 701 len 4096 data: 6
24739 004/300 Read block 2406 len 24576
24740 005/300 Read block 1204 len 12288
24741 006/300 Write block 3810 len 4096 data: 7
24742 007/300 Write block 386 len 20480 data: 6 6 5 5 5
24743 008/300 Write block 743 len 16384 data: 5 4 4 4
24744 009/300 Read block 1481 len 40960
24745 010/300 Flush
24746 011/300 Write block 3640 len 20480 data: 6 6 6 6 7
24747 012/300 Write block 1260 len 40960 data: 3 3 3 7 7 7 6 6 5 5
24748 013/300 Read block 1271 len 40960
24749 014/300 Read block 2544 len 40960
24750 015/300 Read block 3505 len 40960
24751 016/300 Write block 1224 len 16384 data: 5 4 5 4
24752 017/300 Read block 4325 len 32768
24753 018/300 Read block 2770 len 20480
24754 019/300 Write block 2527 len 20480 data: 7 7 4 3 4
24755 020/300 Read block 1565 len 16384
24756 021/300 Read block 3765 len 40960
24757 022/300 Flush
24758 023/300 Write block 1379 len 8192 data: 9 9
24759 024/300 Write block 1958 len 24576 data: 4 5 6 6 4 4
24760 025/300 Read block 933 len 20480
24761 026/300 Write block 2524 len 40960 data: 8 8 7 8 8 5 4 5 3 4
24762 027/300 Write block 3256 len 12288 data: 3 2 2
24763 028/300 Read block 2404 len 24576
24764 029/300 Write block 2605 len 24576 data: 8 8 8 7 7 7
24765 030/300 Write block 1083 len 4096 data: 7
24766 031/300 Read block 1767 len 4096
24767 032/300 Read block 2483 len 20480
24768 033/300 Read block 3022 len 40960
24769 034/300 Read block 1108 len 40960
24770 035/300 Read block 492 len 36864
24771 036/300 Write block 2884 len 12288 data: 7 6 6
24772 037/300 Read block 834 len 4096
24773 038/300 Read block 2293 len 16384
24774 039/300 Write block 1747 len 12288 data: 5 3 4
24775 040/300 Read block 3517 len 36864
24776 041/300 Write block 2908 len 24576 data: 7 6 4 3 3 3
24777 042/300 Read block 100 len 32768
24778 043/300 Write block 657 len 8192 data: 3 5
24779 044/300 Read block 2798 len 36864
24780 045/300 Write block 8 len 16384 data: 4 6 4 3
24781 046/300 Read block 2892 len 4096
24782 047/300 Write block 2522 len 4096 data: 7
24783 048/300 Flush
24784 049/300 Read block 2393 len 4096
24785 050/300 Write block 1849 len 28672 data: 3 5 5 5 6 6 7
24786 051/300 Read block 2503 len 36864
24787 052/300 Read block 1740 len 16384
24788 053/300 Read block 2859 len 28672
24789 054/300 Read block 834 len 24576
24790 055/300 Write block 2867 len 36864 data: 7 6 6 6 6 5 5 4 5
24791 056/300 Write block 3600 len 36864 data: 4 5 5 4 4 3 3 3 2
24792 057/300 Write block 387 len 32768 data: 7 6 6 6 4 4 4 4
24793 058/300 Write block 3777 len 32768 data: 6 8 9 9 6 6 6 6
24794 059/300 Write block 3785 len 32768 data: 4 3 3 4 4 4 6 7
24795 060/300 Read block 1565 len 12288
24796 061/300 Write block 179 len 16384 data: 4 3 4 4
24797 062/300 Read block 3378 len 28672
24798 063/300 Flush
24799 064/300 Write block 1384 len 12288 data: 8 9 9
24800 065/300 Flush
24801 066/300 Write block 4077 len 4096 data: 4
24802 067/300 Write block 1601 len 20480 data: 3 2 3 5 6
24803 068/300 Read block 1584 len 40960
24804 069/300 Write block 1663 len 40960 data: 5 4 5 5 5 7 6 4 4 5
24805 070/300 Write block 1606 len 4096 data: 6
24806 071/300 Read block 1227 len 20480
24807 072/300 Write block 4191 len 40960 data: 5 3 3 3 4 3 3 4 4 5
24808 073/300 Read block 1780 len 4096
24809 074/300 Flush
24810 075/300 Write block 1297 len 24576 data: 4 4 5 5 5 6
24811 076/300 Write block 2428 len 8192 data: 5 5
24812 077/300 Write block 2777 len 32768 data: 5 4 4 5 4 5 5 3
24813 078/300 Write block 1093 len 40960 data: 4 3 3 3 4 5 6 8 8 8
24814 079/300 Read block 593 len 36864
24815 080/300 Read block 2531 len 24576
24816 081/300 Read block 2719 len 32768
24817 082/300 Read block 322 len 12288
24818 083/300 Read block 394 len 32768
24819 084/300 Write block 4581 len 8192 data: 4 4
24820 085/300 Flush
24821 086/300 Read block 2048 len 36864
24822 087/300 Write block 147 len 24576 data: 8 9 9 10 8 7
24823 088/300 Read block 2172 len 32768
24824 089/300 Write block 2935 len 12288 data: 4 5 7
24825 090/300 Read block 4480 len 20480
24826 091/300 Write block 3409 len 40960 data: 7 6 7 5 5 6 6 4 4 4
24827 092/300 Write block 4815 len 24576 data: 4 4 5 4 5 5
24828 093/300 Read block 1244 len 8192
24829 094/300 Write block 652 len 32768 data: 5 5 3 3 3 4 6 5
24830 095/300 Read block 4214 len 8192
24831 096/300 Write block 613 len 4096 data: 3
24832 097/300 Write block 2909 len 24576 data: 7 5 4 4 4 3
24833 098/300 Write block 1288 len 4096 data: 3
24834 099/300 Read block 2421 len 8192
24835 100/300 Write block 2295 len 28672 data: 4 4 3 3 3 3 3
24836 101/300 Read block 3537 len 16384
24837 102/300 Read block 4186 len 36864
24838 103/300 Read block 3693 len 4096
24839 104/300 Read block 40 len 24576
24840 105/300 Write block 343 len 8192 data: 3 3
24841 106/300 Flush
24842 107/300 Write block 816 len 36864 data: 2 2 3 3 3 4 4 5 6
24843 108/300 Read block 3865 len 36864
24844 109/300 Read block 636 len 8192
24845 110/300 Read block 1672 len 8192
24846 111/300 Write block 2162 len 16384 data: 3 3 3 3
24847 112/300 Write block 3421 len 36864 data: 4 3 4 3 3 3 2 2 2
24848 113/300 Write block 2500 len 24576 data: 3 4 4 4 3 3
24849 114/300 Write block 527 len 36864 data: 5 3 4 4 5 4 3 5 5
24850 115/300 Read block 136 len 12288
24851 116/300 Read block 2139 len 40960
24852 117/300 Flush
24853 118/300 Read block 4568 len 4096
24854 119/300 Read block 4873 len 32768
24855 120/300 Write block 3736 len 16384 data: 4 2 2 2
24856 121/300 Write block 2690 len 20480 data: 4 3 4 3 3
24857 122/300 Write block 4888 len 24576 data: 10 8 6 6 5 5
24858 123/300 Write block 2364 len 40960 data: 3 3 3 3 3 3 3 2 3 4
24859 124/300 Write block 4747 len 28672 data: 3 5 7 5 6 6 5
24860 125/300 Read block 2303 len 28672
24861 126/300 Write block 3569 len 12288 data: 4 4 3
24862 127/300 Read block 2042 len 16384
24863 128/300 Read block 3753 len 28672
24864 129/300 Write block 684 len 4096 data: 3
24865 130/300 Write block 404 len 16384 data: 5 4 3 5
24866 131/300 Read block 987 len 4096
24867 132/300 Read block 672 len 40960
24868 133/300 Read block 109 len 28672
24869 134/300 Write block 4524 len 36864 data: 5 5 5 4 3 4 5 7 8
24870 135/300 Read block 2548 len 36864
24871 136/300 Write block 1599 len 4096 data: 3
24872 137/300 Write block 4716 len 8192 data: 2 2
24873 138/300 Read block 2 len 36864
24874 139/300 Read block 3072 len 24576
24875 140/300 Read block 1836 len 4096
24876 141/300 Read block 727 len 32768
24877 142/300 Read block 1785 len 40960
24878 143/300 Flush
24879 144/300 Flush
24880 145/300 Read block 1316 len 28672
24881 146/300 Write block 4146 len 20480 data: 3 3 3 3 3
24882 147/300 Write block 2872 len 40960 data: 6 6 5 6 5 4 4 6 7 7
24883 148/300 Flush
24884 149/300 Write block 668 len 4096 data: 10
24885 150/300 Write block 2695 len 4096 data: 3
24886 151/300 Flush
24887 152/300 Read block 145 len 40960
24888 153/300 Write block 4268 len 32768 data: 7 7 8 8 7 6 6 7
24889 154/300 Read block 2518 len 4096
24890 155/300 Write block 1270 len 20480 data: 5 6 5 5 5
24891 156/300 Write block 3098 len 4096 data: 5
24892 157/300 Write block 1974 len 24576 data: 6 6 5 5 5 5
24893 158/300 Flush
24894 159/300 Flush
24895 160/300 Read block 156 len 32768
24896 161/300 Read block 2273 len 24576
24897 162/300 Write block 152 len 12288 data: 8 5 6
24898 163/300 Read block 3737 len 32768
24899 164/300 Read block 1047 len 40960
24900 165/300 Write block 4288 len 16384 data: 5 3 4 3
24901 166/300 Write block 3557 len 40960 data: 3 4 5 5 4 4 4 5 5 3
24902 167/300 Read block 3991 len 12288
24903 168/300 Read block 4534 len 28672
24904 169/300 Read block 1995 len 28672
24905 170/300 Flush
24906 171/300 Read block 3038 len 36864
24907 172/300 Flush
24908 173/300 Flush
24909 174/300 Write block 670 len 8192 data: 8 5
24910 175/300 Read block 698 len 32768
24911 176/300 Write block 4062 len 20480 data: 2 2 2 2 2
24912 177/300 Write block 2763 len 28672 data: 4 4 3 4 5 6 6
24913 178/300 Flush
24914 179/300 Write block 1945 len 36864 data: 4 5 5 5 4 5 4 5 5
24915 180/300 Read block 1173 len 12288
24916 181/300 Flush
24917 182/300 Write block 4579 len 28672 data: 5 3 5 5 4 4 5
24918 183/300 Read block 3810 len 40960
24919 184/300 Write block 2170 len 40960 data: 4 10 10 9 8 8 8 8 7 7
24920 185/300 Read block 801 len 36864
24921 186/300 Read block 674 len 12288
24922 187/300 Write block 3473 len 8192 data: 7 9
24923 188/300 Read block 945 len 28672
24924 189/300 Read block 3455 len 16384
24925 190/300 Read block 1745 len 36864
24926 191/300 Read block 960 len 4096
24927 192/300 Write block 471 len 36864 data: 4 3 3 6 7 7 8 8 8
24928 193/300 Flush
24929 194/300 Flush
24930 195/300 Read block 3455 len 12288
24931 196/300 Read block 1788 len 24576
24932 197/300 Write block 441 len 16384 data: 2 2 2 3
24933 198/300 Read block 3357 len 36864
24934 199/300 Write block 415 len 36864 data: 2 3 4 4 4 4 3 2 4
24935 200/300 Flush
24936 201/300 Read block 1958 len 28672
24937 202/300 Write block 3582 len 28672 data: 5 5 4 4 4 3 4
24938 203/300 Write block 3627 len 4096 data: 4
24939 204/300 Read block 3911 len 28672
24940 205/300 Write block 4438 len 8192 data: 4 5
24941 206/300 Read block 4704 len 24576
24942 207/300 Read block 3731 len 28672
24943 208/300 Read block 4535 len 40960
24944 209/300 Read block 3860 len 40960
24945 210/300 Write block 2151 len 4096 data: 7
24946 211/300 Write block 4795 len 32768 data: 2 3 4 3 3 5 4 4
24947 212/300 Write block 2748 len 28672 data: 7 8 8 8 8 7 6
24948 213/300 Write block 505 len 24576 data: 2 3 4 3 4 5
24949 214/300 Write block 558 len 20480 data: 4 3 4 3 3
24950 215/300 Read block 2704 len 16384
24951 216/300 Read block 3030 len 24576
24952 217/300 Write block 2697 len 16384 data: 6 8 8 7
24953 218/300 Read block 3701 len 32768
24954 219/300 Read block 3342 len 28672
24955 220/300 Read block 1634 len 20480
24956 221/300 Write block 2736 len 16384 data: 5 5 4 3
24957 222/300 Write block 4029 len 40960 data: 7 6 6 6 5 5 5 4 3 3
24958 223/300 Write block 2721 len 28672 data: 2 3 4 6 4 5 5
24959 224/300 Read block 4576 len 16384
24960 225/300 Read block 2975 len 16384
24961 226/300 Read block 4185 len 8192
24962 227/300 Read block 2115 len 20480
24963 228/300 Read block 2537 len 12288
24964 229/300 Write block 3429 len 4096 data: 3
24965 230/300 Read block 4799 len 4096
24966 231/300 Read block 2464 len 32768
24967 232/300 Read block 4248 len 28672
24968 233/300 Read block 3932 len 28672
24969 234/300 Read block 4321 len 32768
24970 235/300 Write block 1338 len 32768 data: 9 7 6 5 5 5 4 4
24971 236/300 Read block 1487 len 36864
24972 237/300 Read block 3586 len 20480
24973 238/300 Write block 1650 len 4096 data: 7
24974 239/300 Read block 1501 len 24576
24975 240/300 Read block 278 len 12288
24976 241/300 Write block 1435 len 40960 data: 5 6 7 5 5 7 6 7 5 4
24977 242/300 Read block 2558 len 4096
24978 243/300 Read block 3918 len 4096
24979 244/300 Write block 186 len 4096 data: 4
24980 245/300 Write block 2954 len 4096 data: 7
24981 246/300 Flush
24982 247/300 Read block 1976 len 32768
24983 248/300 Read block 4785 len 28672
24984 249/300 Flush
24985 250/300 Flush
24986 251/300 Write block 2955 len 24576 data: 6 4 4 3 2 2
24987 252/300 Read block 4683 len 8192
24988 253/300 Read block 1339 len 8192
24989 254/300 Write block 1834 len 12288 data: 6 5 5
24990 255/300 Read block 3521 len 8192
24991 256/300 Read block 1153 len 40960
24992 257/300 Read block 1392 len 28672
24993 258/300 Read block 3204 len 36864
24994 259/300 Write block 732 len 28672 data: 7 5 4 3 3 3 4
24995 260/300 Read block 1980 len 12288
24996 261/300 Read block 1447 len 12288
24997 262/300 Write block 2105 len 32768 data: 4 4 5 7 8 7 6 5
24998 263/300 Read block 2351 len 36864
24999 264/300 Read block 857 len 16384
25000 265/300 Read block 2414 len 12288
25001 266/300 Read block 4799 len 16384
25002 267/300 Read block 4974 len 4096
25003 268/300 Flush
25004 269/300 Write block 2152 len 28672 data: 6 5 5 6 4 3 3
25005 270/300 Flush
25006 271/300 Flush
25007 272/300 Write block 1923 len 16384 data: 5 4 4 4
25008 273/300 Flush
25009 274/300 Read block 2358 len 12288
25010 275/300 Write block 2158 len 24576 data: 4 2 2 3 4 4
25011 276/300 Write block 4763 len 16384 data: 5 5 6 3
25012 277/300 Read block 4365 len 16384
25013 278/300 Flush
25014 279/300 Flush
25015 280/300 Read block 4768 len 28672
25016 281/300 Write block 4830 len 4096 data: 5
25017 282/300 Read block 2132 len 20480
25018 283/300 Flush
25019 284/300 Read block 2617 len 28672
25020 285/300 Write block 1924 len 32768 data: 5 5 5 3 3 4 5 5
25021 286/300 Write block 3175 len 36864 data: 4 4 5 5 4 3 4 4 5
25022 287/300 Write block 3747 len 32768 data: 4 6 7 4 4 3 3 4
25023 288/300 Write block 716 len 32768 data: 6 6 6 6 3 3 4 5
25024 289/300 Flush
25025 290/300 Read block 2406 len 32768
25026 291/300 Read block 3233 len 28672
25027 292/300 Write block 367 len 28672 data: 5 6 4 4 5 5 7
25028 293/300 Write block 4383 len 40960 data: 4 6 6 6 6 6 5 4 5 5
25029 294/300 Write block 430 len 12288 data: 5 5 5
25030 295/300 Read block 2051 len 4096
25031 296/300 Read block 4968 len 4096
25032 297/300 Write block 1448 len 36864 data: 4 4 3 3 3 4 4 3 2
25033 298/300 Read block 1752 len 20480
25034 299/300 Write block 1639 len 28672 data: 3 3 3 2 2 2 3
25035 300/300 Read block 4966 len 24576
25036 301/300 Read block 2218 len 4096
25037 Main task has work to do, go find it
25038 got dsc Start(0)
25039 start 0
25040 Jan 13 15:57:04.816 INFO request completed, latency_us: 206, response_code: 204, uri: /start/cid/0, method: POST, req_id: bb2de555-6c09-4285-aeb7-aa6d0b911e37, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
25041 Replay: started 0, returned:Ok(())
25042 ----------------------------------------------------------------
25043 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
25044 States: Offline Active Active
25045 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25046 7642 Acked Write 10 Sent Done Done false
25047 7643 Acked Read 10 Sent Done Done false
25048 7644 Acked Write 1 Sent Done Done false
25049 7645 Acked Read 6 Sent Done Done false
25050 7646 Acked Read 3 Sent Done Done false
25051 7647 Acked Write 1 Sent Done Done false
25052 7648 Acked Write 5 Sent Done Done false
25053 7649 Acked Write 4 Sent Done Done false
25054 7650 Acked Read 10 Sent Done Done false
25055 7651 Acked Flush 0 Sent Done Done false
25056 7652 Acked Write 5 Sent Done Done false
25057 7653 Acked Write 10 Sent Done Done false
25058 7654 Acked Read 10 Sent Done Done false
25059 7655 Acked Read 10 Sent Done Done false
25060 7656 Acked Read 10 Sent Done Done false
25061 7657 Acked Write 4 Sent Done Done false
25062 7658 Acked Read 8 Sent Done Done false
25063 7659 Acked Read 5 Sent Done Done false
25064 7660 Acked Write 5 Sent Done Done false
25065 7661 Acked Read 4 Sent Done Done false
25066 7662 Acked Read 10 Sent Done Done false
25067 7663 Acked Flush 0 Sent Done Done false
25068 7664 Acked Write 2 Sent Done Done false
25069 7665 Acked Write 6 Sent Done Done false
25070 7666 Acked Read 5 Sent Done Done false
25071 7667 Acked Write 10 Sent Done Done false
25072 7668 Acked Write 3 Sent Done Done false
25073 7669 Acked Read 6 Sent Done Done false
25074 7670 Acked Write 6 Sent Done Done false
25075 7671 Acked Write 1 Sent Done Done false
25076 7672 Acked Read 1 Sent Done Done false
25077 7673 Acked Read 5 Sent Done Done false
25078 7674 Acked Read 10 Sent Done Done false
25079 7675 Acked Read 10 Sent Done Done false
25080 7676 Acked Read 9 Sent Done Done false
25081 7677 Acked Write 3 Sent Done Done false
25082 7678 Acked Read 1 Sent Done Done false
25083 7679 Acked Read 4 Sent Done Done false
25084 7680 Acked Write 3 Sent Done Done false
25085 7681 Acked Read 9 Sent Done Done false
25086 7682 Acked Write 6 Sent Done Done false
25087 7683 Acked Read 8 Sent Done Done false
25088 7684 Acked Write 2 Sent Done Done false
25089 7685 Acked Read 9 Sent Done Done false
25090 7686 Acked Write 4 Sent Done Done false
25091 7687 Acked Read 1 Sent Done Done false
25092 7688 Acked Write 1 Sent Done Done false
25093 7689 Acked Flush 0 Sent Done Done false
25094 7690 Acked Read 1 Sent Done Done false
25095 7691 Acked Write 7 Sent Done Done false
25096 7692 Acked Read 9 Sent Done Done false
25097 7693 Acked Read 4 Sent Done Done false
25098 7694 Acked Read 7 Sent Done Done false
25099 7695 Acked Read 6 Sent Done Done false
25100 7696 Acked Write 9 Sent Done Done false
25101 7697 Acked Write 9 Sent Done Done false
25102 7698 Acked Write 8 Sent Done Done false
25103 7699 Acked Write 8 Sent Done Done false
25104 7700 Acked Write 8 Sent Done Done false
25105 7701 Acked Read 3 Sent Done Done false
25106 7702 Acked Write 4 Sent Done Done false
25107 7703 Acked Read 7 Sent Done Done false
25108 7704 Acked Flush 0 Sent Done Done false
25109 7705 Acked Write 3 Sent Done Done false
25110 7706 Acked Flush 0 Sent Done Done false
25111 7707 Acked Write 1 Sent Done Done false
25112 7708 Acked Write 5 Sent Done Done false
25113 7709 Acked Read 10 Sent Done Done false
25114 7710 Acked Write 10 Sent Done Done false
25115 7711 Acked Write 1 Sent Done Done false
25116 7712 Acked Read 5 Sent Done Done false
25117 7713 Acked Write 10 Sent Done Done false
25118 7714 Acked Read 1 Sent Done Done false
25119 7715 Acked Flush 0 Sent Done Done false
25120 7716 Acked Write 6 Sent Done Done false
25121 7717 Acked Write 2 Sent Done Done false
25122 7718 Acked Write 8 Sent Done Done false
25123 7719 Acked Write 10 Sent Done Done false
25124 7720 Acked Read 9 Sent Done Done false
25125 7721 Acked Read 6 Sent Done Done false
25126 7722 Acked Read 8 Sent Done Done false
25127 7723 Acked Read 3 Sent Done Done false
25128 7724 Acked Read 8 Sent Done Done false
25129 7725 Acked Write 2 Sent Done Done false
25130 7726 Acked Flush 0 Sent Done Done false
25131 7727 Acked Read 9 Sent Done Done false
25132 7728 Acked Write 6 Sent Done Done false
25133 7729 Acked Read 8 Sent Done Done false
25134 7730 Acked Write 3 Sent Done Done false
25135 7731 Acked Read 5 Sent Done Done false
25136 7732 Acked Write 10 Sent Done Done false
25137 7733 Acked Write 6 Sent Done Done false
25138 7734 Acked Read 2 Sent Done Done false
25139 7735 Acked Write 8 Sent Done Done false
25140 7736 Acked Read 2 Sent Done Done false
25141 7737 Acked Write 1 Sent Done Done false
25142 7738 Acked Write 6 Sent Done Done false
25143 7739 Acked Write 1 Sent Done Done false
25144 7740 Acked Read 2 Sent Done Done false
25145 7741 Acked Write 7 Sent Done Done false
25146 7742 Acked Read 4 Sent Done Done false
25147 7743 Acked Read 9 Sent Done Done false
25148 7744 Acked Read 1 Sent Done Done false
25149 7745 Acked Read 6 Sent Done Done false
25150 7746 Acked Write 2 Sent Done Done false
25151 7747 Acked Flush 0 Sent Done Done false
25152 7748 Acked Write 9 Sent Done Done false
25153 7749 Acked Read 9 Sent Done Done false
25154 7750 Acked Read 2 Sent Done Done false
25155 7751 Acked Read 2 Sent Done Done false
25156 7752 Acked Write 4 Sent Done Done false
25157 7753 Acked Write 9 Sent Done Done false
25158 7754 Acked Write 6 Sent Done Done false
25159 7755 Acked Write 9 Sent Done Done false
25160 7756 Acked Read 3 Sent Done Done false
25161 7757 Acked Read 10 Sent Done Done false
25162 7758 Acked Flush 0 Sent Done Done false
25163 7759 Acked Read 1 Sent Done Done false
25164 7760 Acked Read 8 Sent Done Done false
25165 7761 Acked Write 4 Sent Done Done false
25166 7762 Acked Write 5 Sent Done Done false
25167 7763 Acked Write 6 Sent Done Done false
25168 7764 Acked Write 10 Sent Done Done false
25169 7765 Acked Write 7 Sent Done Done false
25170 7766 Acked Read 7 Sent Done Done false
25171 7767 Acked Write 3 Sent Done Done false
25172 7768 Acked Read 4 Sent Done Done false
25173 7769 Acked Read 7 Sent Done Done false
25174 7770 Acked Write 1 Sent Done Done false
25175 7771 Acked Write 4 Sent Done Done false
25176 7772 Acked Read 1 Sent Done Done false
25177 7773 Acked Read 10 Sent Done Done false
25178 7774 Acked Read 7 Sent Done Done false
25179 7775 Acked Write 9 Sent Done Done false
25180 7776 Acked Read 9 Sent Done Done false
25181 7777 Acked Write 1 Sent Done Done false
25182 7778 Acked Write 2 Sent Done Done false
25183 7779 Acked Read 9 Sent Done Done false
25184 7780 Acked Read 6 Sent Done Done false
25185 7781 Acked Read 1 Sent Done Done false
25186 7782 Acked Read 8 Sent Done Done false
25187 7783 Acked Read 10 Sent Done Done false
25188 7784 Acked Flush 0 Sent Done Done false
25189 7785 Acked Flush 0 Sent Done Done false
25190 7786 Acked Read 7 Sent Done Done false
25191 7787 Acked Write 5 Sent Done Done false
25192 7788 Acked Write 10 Sent Done Done false
25193 7789 Acked Flush 0 Sent Done Done false
25194 7790 Acked Write 1 Sent Done Done false
25195 7791 Acked Write 1 Sent Done Done false
25196 7792 Acked Flush 0 Sent Done Done false
25197 7793 Acked Read 10 Sent Done Done false
25198 7794 Acked Write 8 Sent Done Done false
25199 7795 Acked Read 1 Sent Done Done false
25200 7796 Acked Write 5 Sent Done Done false
25201 7797 Acked Write 1 Sent Done Done false
25202 7798 Acked Write 6 Sent Done Done false
25203 7799 Acked Flush 0 Sent Done Done false
25204 7800 Acked Flush 0 Sent Done Done false
25205 7801 Acked Read 8 Sent Done Done false
25206 7802 Acked Read 6 Sent Done Done false
25207 7803 Acked Write 3 Sent Done Done false
25208 7804 Acked Read 8 Sent Done Done false
25209 7805 Acked Read 10 Sent Done Done false
25210 7806 Acked Write 4 Sent Done Done false
25211 7807 Acked Write 10 Sent Done Done false
25212 7808 Acked Read 3 Sent Done Done false
25213 7809 Acked Read 7 Sent Done Done false
25214 7810 Acked Read 7 Sent Done Done false
25215 7811 Acked Flush 0 Sent Done Done false
25216 7812 Acked Read 9 Sent Done Done false
25217 7813 Acked Flush 0 Sent Done Done false
25218 7814 Acked Flush 0 Sent Done Done false
25219 7815 Acked Write 2 Sent Done Done false
25220 7816 Acked Read 8 Sent Done Done false
25221 7817 Acked Write 5 Sent Done Done false
25222 7818 Acked Write 7 Sent Done Done false
25223 7819 Acked Flush 0 Sent Done Done false
25224 7820 Acked Write 9 Sent Done Done false
25225 7821 Acked Read 3 Sent Done Done false
25226 7822 Acked Flush 0 Sent Done Done false
25227 7823 Acked Write 7 Sent Done Done false
25228 7824 Acked Read 10 Sent Done Done false
25229 7825 Acked Write 10 Sent Done Done false
25230 7826 Acked Read 9 Sent Done Done false
25231 7827 Acked Read 3 Sent Done Done false
25232 7828 Acked Write 2 Sent Done Done false
25233 7829 Acked Read 7 Sent Done Done false
25234 7830 Acked Read 4 Sent Done Done false
25235 7831 Acked Read 9 Sent Done Done false
25236 7832 Acked Read 1 Sent Done Done false
25237 7833 Acked Write 9 Sent Done Done false
25238 7834 Acked Flush 0 Sent Done Done false
25239 7835 Acked Flush 0 Sent Done Done false
25240 7836 Acked Read 3 Sent Done Done false
25241 7837 Acked Read 6 Sent Done Done false
25242 7838 Acked Write 4 Sent Done Done false
25243 7839 Acked Read 9 Sent Done Done false
25244 7840 Acked Write 9 Sent Done Done false
25245 7841 Acked Flush 0 Sent Done Done false
25246 7842 Acked Read 7 Sent Done Done false
25247 7843 Acked Write 7 Sent Done Done false
25248 7844 Acked Write 1 Sent Done Done false
25249 7845 Acked Read 7 Sent Done Done false
25250 7846 Acked Write 2 Sent Done Done false
25251 7847 Acked Read 6 Sent Done Done false
25252 7848 Acked Read 7 Sent Done Done false
25253 7849 Acked Read 10 Sent Done Done false
25254 7850 Acked Read 10 Sent Done Done false
25255 7851 Acked Write 1 Sent Done Done false
25256 7852 Acked Write 8 Sent Done Done false
25257 7853 Acked Write 7 Sent Done Done false
25258 7854 Acked Write 6 Sent Done Done false
25259 7855 Acked Write 5 Sent Done Done false
25260 7856 Acked Read 4 Sent Done Done false
25261 7857 Acked Read 6 Sent Done Done false
25262 7858 Acked Write 4 Sent Done Done false
25263 7859 Acked Read 8 Sent Done Done false
25264 7860 Acked Read 7 Sent Done Done false
25265 7861 Acked Read 5 Sent Done Done false
25266 7862 Acked Write 4 Sent Done Done false
25267 7863 Acked Write 10 Sent Done Done false
25268 7864 Acked Write 7 Sent Done Done false
25269 7865 Acked Read 4 Sent Done Done false
25270 7866 Acked Read 4 Sent Done Done false
25271 7867 Acked Read 2 Sent Done Done false
25272 7868 Acked Read 5 Sent Done Done false
25273 7869 Acked Read 3 Sent Done Done false
25274 7870 Acked Write 1 Sent Done Done false
25275 7871 Acked Read 1 Sent Done Done false
25276 7872 Acked Read 8 Sent Done Done false
25277 7873 Acked Read 7 Sent Done Done false
25278 7874 Acked Read 7 Sent Done Done false
25279 7875 Acked Read 8 Sent Done Done false
25280 7876 Acked Write 8 Sent Done Done false
25281 7877 Acked Read 9 Sent Done Done false
25282 7878 Acked Read 5 Sent Done Done false
25283 7879 Acked Write 1 Sent Done Done false
25284 7880 Acked Read 6 Sent Done Done false
25285 7881 Acked Read 3 Sent Done Done false
25286 7882 Acked Write 10 Sent Done Done false
25287 7883 Acked Read 1 Sent Done Done false
25288 7884 Acked Read 1 Sent Done Done false
25289 7885 Acked Write 1 Sent Done Done false
25290 7886 Acked Write 1 Sent Done Done false
25291 7887 Acked Flush 0 Sent Done Done false
25292 7888 Acked Read 8 Sent Done Done false
25293 7889 Acked Read 7 Sent Done Done false
25294 7890 Acked Flush 0 Sent Done Done false
25295 7891 Acked Flush 0 Sent Done Done false
25296 7892 Acked Write 6 Sent Done Done false
25297 7893 Acked Read 2 Sent Done Done false
25298 7894 Acked Read 2 Sent Done Done false
25299 7895 Acked Write 3 Sent Done Done false
25300 7896 Acked Read 2 Sent Done Done false
25301 7897 Acked Read 10 Sent Done Done false
25302 7898 Acked Read 7 Sent Done Done false
25303 7899 Acked Read 9 Sent Done Done false
25304 7900 Acked Write 7 Sent Done Done false
25305 7901 Acked Read 3 Sent Done Done false
25306 7902 Acked Read 3 Sent Done Done false
25307 7903 Acked Write 8 Sent Done Done false
25308 7904 Acked Read 9 Sent Done Done false
25309 7905 Acked Read 4 Sent Done Done false
25310 7906 Acked Read 3 Sent Done Done false
25311 7907 Acked Read 4 Sent Done Done false
25312 7908 Acked Read 1 Sent Done Done false
25313 7909 Acked Flush 0 Sent Done Done false
25314 7910 Acked Write 7 Sent Done Done false
25315 7911 Acked Flush 0 Sent Done Done false
25316 7912 Acked Flush 0 Sent Done Done false
25317 7913 Acked Write 4 Sent Done Done false
25318 7914 Acked Flush 0 Sent Done Done false
25319 7915 Acked Read 3 Sent Done Done false
25320 7916 Acked Write 6 Sent Done Done false
25321 7917 Acked Write 4 Sent Done Done false
25322 7918 Acked Read 4 Sent Done Done false
25323 7919 Acked Flush 0 Sent Done Done false
25324 7920 Acked Flush 0 Sent Done Done false
25325 7921 Acked Read 7 Sent Done Done false
25326 7922 Acked Write 1 Sent Done Done false
25327 7923 Acked Read 5 Sent Done Done false
25328 7924 Acked Flush 0 Sent Done Done false
25329 7925 Acked Read 7 Sent Done Done false
25330 7926 Acked Write 8 Sent Done Done false
25331 7927 Acked Write 9 Sent Done Done false
25332 7928 Acked Write 8 Sent Done Done false
25333 7929 Acked Write 8 Sent Done Done false
25334 7930 Acked Flush 0 Sent Done Done false
25335 7931 Acked Read 8 Sent Done Done false
25336 7932 Acked Read 7 Sent Done Done false
25337 7933 Acked Write 7 Sent Done Done false
25338 7934 Acked Write 10 Sent Done Done false
25339 7935 Acked Write 3 Sent Done Done false
25340 7936 Acked Read 1 Sent Done Done false
25341 7937 Acked Read 1 Sent Done Done false
25342 7938 Acked Write 9 Sent Done Done false
25343 7939 Acked Read 5 Sent Done Done false
25344 7940 Acked Write 7 Sent Done Done false
25345 7941 Acked Read 6 Sent Done Done false
25346 7942 Acked Read 1 Sent Done Done false
25347 STATES DS:0 DS:1 DS:2 TOTAL
25348 Sent 301 0 0 301
25349 Done 0 301 301 602
25350 Skipped 0 0 0 0
25351 Error 0 0 0 0
25352 Last Flush: 7641 7930 7930
25353 Downstairs last five completed: 7641 7640 7639 7638 7637
25354 Upstairs last five completed: 7942 7941 7940 7939 7938
25355 CLIENT: Up:0 ds:301 act:2
25356 [8810] got start action
25357 [8810] I am going to restart
25358 Starting downstairs at port 8810
25359 Make output file at "/tmp/dsc/downstairs-8810.txt"
25360 [8810][0] reports Starting
25361 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1225)
25362 [8810][0] reports Running
25363 ----------------------------------------------------------------
25364 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
25365 States: Offline Active Active
25366 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25367 7642 Acked Write 10 Sent Done Done false
25368 7643 Acked Read 10 Sent Done Done false
25369 7644 Acked Write 1 Sent Done Done false
25370 7645 Acked Read 6 Sent Done Done false
25371 7646 Acked Read 3 Sent Done Done false
25372 7647 Acked Write 1 Sent Done Done false
25373 7648 Acked Write 5 Sent Done Done false
25374 7649 Acked Write 4 Sent Done Done false
25375 7650 Acked Read 10 Sent Done Done false
25376 7651 Acked Flush 0 Sent Done Done false
25377 7652 Acked Write 5 Sent Done Done false
25378 7653 Acked Write 10 Sent Done Done false
25379 7654 Acked Read 10 Sent Done Done false
25380 7655 Acked Read 10 Sent Done Done false
25381 7656 Acked Read 10 Sent Done Done false
25382 7657 Acked Write 4 Sent Done Done false
25383 7658 Acked Read 8 Sent Done Done false
25384 7659 Acked Read 5 Sent Done Done false
25385 7660 Acked Write 5 Sent Done Done false
25386 7661 Acked Read 4 Sent Done Done false
25387 7662 Acked Read 10 Sent Done Done false
25388 7663 Acked Flush 0 Sent Done Done false
25389 7664 Acked Write 2 Sent Done Done false
25390 7665 Acked Write 6 Sent Done Done false
25391 7666 Acked Read 5 Sent Done Done false
25392 7667 Acked Write 10 Sent Done Done false
25393 7668 Acked Write 3 Sent Done Done false
25394 7669 Acked Read 6 Sent Done Done false
25395 7670 Acked Write 6 Sent Done Done false
25396 7671 Acked Write 1 Sent Done Done false
25397 7672 Acked Read 1 Sent Done Done false
25398 7673 Acked Read 5 Sent Done Done false
25399 7674 Acked Read 10 Sent Done Done false
25400 7675 Acked Read 10 Sent Done Done false
25401 7676 Acked Read 9 Sent Done Done false
25402 7677 Acked Write 3 Sent Done Done false
25403 7678 Acked Read 1 Sent Done Done false
25404 7679 Acked Read 4 Sent Done Done false
25405 7680 Acked Write 3 Sent Done Done false
25406 7681 Acked Read 9 Sent Done Done false
25407 7682 Acked Write 6 Sent Done Done false
25408 7683 Acked Read 8 Sent Done Done false
25409 7684 Acked Write 2 Sent Done Done false
25410 7685 Acked Read 9 Sent Done Done false
25411 7686 Acked Write 4 Sent Done Done false
25412 7687 Acked Read 1 Sent Done Done false
25413 7688 Acked Write 1 Sent Done Done false
25414 7689 Acked Flush 0 Sent Done Done false
25415 7690 Acked Read 1 Sent Done Done false
25416 7691 Acked Write 7 Sent Done Done false
25417 7692 Acked Read 9 Sent Done Done false
25418 7693 Acked Read 4 Sent Done Done false
25419 7694 Acked Read 7 Sent Done Done false
25420 7695 Acked Read 6 Sent Done Done false
25421 7696 Acked Write 9 Sent Done Done false
25422 7697 Acked Write 9 Sent Done Done false
25423 7698 Acked Write 8 Sent Done Done false
25424 7699 Acked Write 8 Sent Done Done false
25425 7700 Acked Write 8 Sent Done Done false
25426 7701 Acked Read 3 Sent Done Done false
25427 7702 Acked Write 4 Sent Done Done false
25428 7703 Acked Read 7 Sent Done Done false
25429 7704 Acked Flush 0 Sent Done Done false
25430 7705 Acked Write 3 Sent Done Done false
25431 7706 Acked Flush 0 Sent Done Done false
25432 7707 Acked Write 1 Sent Done Done false
25433 7708 Acked Write 5 Sent Done Done false
25434 7709 Acked Read 10 Sent Done Done false
25435 7710 Acked Write 10 Sent Done Done false
25436 7711 Acked Write 1 Sent Done Done false
25437 7712 Acked Read 5 Sent Done Done false
25438 7713 Acked Write 10 Sent Done Done false
25439 7714 Acked Read 1 Sent Done Done false
25440 7715 Acked Flush 0 Sent Done Done false
25441 7716 Acked Write 6 Sent Done Done false
25442 7717 Acked Write 2 Sent Done Done false
25443 7718 Acked Write 8 Sent Done Done false
25444 7719 Acked Write 10 Sent Done Done false
25445 7720 Acked Read 9 Sent Done Done false
25446 7721 Acked Read 6 Sent Done Done false
25447 7722 Acked Read 8 Sent Done Done false
25448 7723 Acked Read 3 Sent Done Done false
25449 7724 Acked Read 8 Sent Done Done false
25450 7725 Acked Write 2 Sent Done Done false
25451 7726 Acked Flush 0 Sent Done Done false
25452 7727 Acked Read 9 Sent Done Done false
25453 7728 Acked Write 6 Sent Done Done false
25454 7729 Acked Read 8 Sent Done Done false
25455 7730 Acked Write 3 Sent Done Done false
25456 7731 Acked Read 5 Sent Done Done false
25457 7732 Acked Write 10 Sent Done Done false
25458 7733 Acked Write 6 Sent Done Done false
25459 7734 Acked Read 2 Sent Done Done false
25460 7735 Acked Write 8 Sent Done Done false
25461 7736 Acked Read 2 Sent Done Done false
25462 7737 Acked Write 1 Sent Done Done false
25463 7738 Acked Write 6 Sent Done Done false
25464 7739 Acked Write 1 Sent Done Done false
25465 7740 Acked Read 2 Sent Done Done false
25466 7741 Acked Write 7 Sent Done Done false
25467 7742 Acked Read 4 Sent Done Done false
25468 7743 Acked Read 9 Sent Done Done false
25469 7744 Acked Read 1 Sent Done Done false
25470 7745 Acked Read 6 Sent Done Done false
25471 7746 Acked Write 2 Sent Done Done false
25472 7747 Acked Flush 0 Sent Done Done false
25473 7748 Acked Write 9 Sent Done Done false
25474 7749 Acked Read 9 Sent Done Done false
25475 7750 Acked Read 2 Sent Done Done false
25476 7751 Acked Read 2 Sent Done Done false
25477 7752 Acked Write 4 Sent Done Done false
25478 7753 Acked Write 9 Sent Done Done false
25479 7754 Acked Write 6 Sent Done Done false
25480 7755 Acked Write 9 Sent Done Done false
25481 7756 Acked Read 3 Sent Done Done false
25482 7757 Acked Read 10 Sent Done Done false
25483 7758 Acked Flush 0 Sent Done Done false
25484 7759 Acked Read 1 Sent Done Done false
25485 7760 Acked Read 8 Sent Done Done false
25486 7761 Acked Write 4 Sent Done Done false
25487 7762 Acked Write 5 Sent Done Done false
25488 7763 Acked Write 6 Sent Done Done false
25489 7764 Acked Write 10 Sent Done Done false
25490 7765 Acked Write 7 Sent Done Done false
25491 7766 Acked Read 7 Sent Done Done false
25492 7767 Acked Write 3 Sent Done Done false
25493 7768 Acked Read 4 Sent Done Done false
25494 7769 Acked Read 7 Sent Done Done false
25495 7770 Acked Write 1 Sent Done Done false
25496 7771 Acked Write 4 Sent Done Done false
25497 7772 Acked Read 1 Sent Done Done false
25498 7773 Acked Read 10 Sent Done Done false
25499 7774 Acked Read 7 Sent Done Done false
25500 7775 Acked Write 9 Sent Done Done false
25501 7776 Acked Read 9 Sent Done Done false
25502 7777 Acked Write 1 Sent Done Done false
25503 7778 Acked Write 2 Sent Done Done false
25504 7779 Acked Read 9 Sent Done Done false
25505 7780 Acked Read 6 Sent Done Done false
25506 7781 Acked Read 1 Sent Done Done false
25507 7782 Acked Read 8 Sent Done Done false
25508 7783 Acked Read 10 Sent Done Done false
25509 7784 Acked Flush 0 Sent Done Done false
25510 7785 Acked Flush 0 Sent Done Done false
25511 7786 Acked Read 7 Sent Done Done false
25512 7787 Acked Write 5 Sent Done Done false
25513 7788 Acked Write 10 Sent Done Done false
25514 7789 Acked Flush 0 Sent Done Done false
25515 7790 Acked Write 1 Sent Done Done false
25516 7791 Acked Write 1 Sent Done Done false
25517 7792 Acked Flush 0 Sent Done Done false
25518 7793 Acked Read 10 Sent Done Done false
25519 7794 Acked Write 8 Sent Done Done false
25520 7795 Acked Read 1 Sent Done Done false
25521 7796 Acked Write 5 Sent Done Done false
25522 7797 Acked Write 1 Sent Done Done false
25523 7798 Acked Write 6 Sent Done Done false
25524 7799 Acked Flush 0 Sent Done Done false
25525 7800 Acked Flush 0 Sent Done Done false
25526 7801 Acked Read 8 Sent Done Done false
25527 7802 Acked Read 6 Sent Done Done false
25528 7803 Acked Write 3 Sent Done Done false
25529 7804 Acked Read 8 Sent Done Done false
25530 7805 Acked Read 10 Sent Done Done false
25531 7806 Acked Write 4 Sent Done Done false
25532 7807 Acked Write 10 Sent Done Done false
25533 7808 Acked Read 3 Sent Done Done false
25534 7809 Acked Read 7 Sent Done Done false
25535 7810 Acked Read 7 Sent Done Done false
25536 7811 Acked Flush 0 Sent Done Done false
25537 7812 Acked Read 9 Sent Done Done false
25538 7813 Acked Flush 0 Sent Done Done false
25539 7814 Acked Flush 0 Sent Done Done false
25540 7815 Acked Write 2 Sent Done Done false
25541 7816 Acked Read 8 Sent Done Done false
25542 7817 Acked Write 5 Sent Done Done false
25543 7818 Acked Write 7 Sent Done Done false
25544 7819 Acked Flush 0 Sent Done Done false
25545 7820 Acked Write 9 Sent Done Done false
25546 7821 Acked Read 3 Sent Done Done false
25547 7822 Acked Flush 0 Sent Done Done false
25548 7823 Acked Write 7 Sent Done Done false
25549 7824 Acked Read 10 Sent Done Done false
25550 7825 Acked Write 10 Sent Done Done false
25551 7826 Acked Read 9 Sent Done Done false
25552 7827 Acked Read 3 Sent Done Done false
25553 7828 Acked Write 2 Sent Done Done false
25554 7829 Acked Read 7 Sent Done Done false
25555 7830 Acked Read 4 Sent Done Done false
25556 7831 Acked Read 9 Sent Done Done false
25557 7832 Acked Read 1 Sent Done Done false
25558 7833 Acked Write 9 Sent Done Done false
25559 7834 Acked Flush 0 Sent Done Done false
25560 7835 Acked Flush 0 Sent Done Done false
25561 7836 Acked Read 3 Sent Done Done false
25562 7837 Acked Read 6 Sent Done Done false
25563 7838 Acked Write 4 Sent Done Done false
25564 7839 Acked Read 9 Sent Done Done false
25565 7840 Acked Write 9 Sent Done Done false
25566 7841 Acked Flush 0 Sent Done Done false
25567 7842 Acked Read 7 Sent Done Done false
25568 7843 Acked Write 7 Sent Done Done false
25569 7844 Acked Write 1 Sent Done Done false
25570 7845 Acked Read 7 Sent Done Done false
25571 7846 Acked Write 2 Sent Done Done false
25572 7847 Acked Read 6 Sent Done Done false
25573 7848 Acked Read 7 Sent Done Done false
25574 7849 Acked Read 10 Sent Done Done false
25575 7850 Acked Read 10 Sent Done Done false
25576 7851 Acked Write 1 Sent Done Done false
25577 7852 Acked Write 8 Sent Done Done false
25578 7853 Acked Write 7 Sent Done Done false
25579 7854 Acked Write 6 Sent Done Done false
25580 7855 Acked Write 5 Sent Done Done false
25581 7856 Acked Read 4 Sent Done Done false
25582 7857 Acked Read 6 Sent Done Done false
25583 7858 Acked Write 4 Sent Done Done false
25584 7859 Acked Read 8 Sent Done Done false
25585 7860 Acked Read 7 Sent Done Done false
25586 7861 Acked Read 5 Sent Done Done false
25587 7862 Acked Write 4 Sent Done Done false
25588 7863 Acked Write 10 Sent Done Done false
25589 7864 Acked Write 7 Sent Done Done false
25590 7865 Acked Read 4 Sent Done Done false
25591 7866 Acked Read 4 Sent Done Done false
25592 7867 Acked Read 2 Sent Done Done false
25593 7868 Acked Read 5 Sent Done Done false
25594 7869 Acked Read 3 Sent Done Done false
25595 7870 Acked Write 1 Sent Done Done false
25596 7871 Acked Read 1 Sent Done Done false
25597 7872 Acked Read 8 Sent Done Done false
25598 7873 Acked Read 7 Sent Done Done false
25599 7874 Acked Read 7 Sent Done Done false
25600 7875 Acked Read 8 Sent Done Done false
25601 7876 Acked Write 8 Sent Done Done false
25602 7877 Acked Read 9 Sent Done Done false
25603 7878 Acked Read 5 Sent Done Done false
25604 7879 Acked Write 1 Sent Done Done false
25605 7880 Acked Read 6 Sent Done Done false
25606 7881 Acked Read 3 Sent Done Done false
25607 7882 Acked Write 10 Sent Done Done false
25608 7883 Acked Read 1 Sent Done Done false
25609 7884 Acked Read 1 Sent Done Done false
25610 7885 Acked Write 1 Sent Done Done false
25611 7886 Acked Write 1 Sent Done Done false
25612 7887 Acked Flush 0 Sent Done Done false
25613 7888 Acked Read 8 Sent Done Done false
25614 7889 Acked Read 7 Sent Done Done false
25615 7890 Acked Flush 0 Sent Done Done false
25616 7891 Acked Flush 0 Sent Done Done false
25617 7892 Acked Write 6 Sent Done Done false
25618 7893 Acked Read 2 Sent Done Done false
25619 7894 Acked Read 2 Sent Done Done false
25620 7895 Acked Write 3 Sent Done Done false
25621 7896 Acked Read 2 Sent Done Done false
25622 7897 Acked Read 10 Sent Done Done false
25623 7898 Acked Read 7 Sent Done Done false
25624 7899 Acked Read 9 Sent Done Done false
25625 7900 Acked Write 7 Sent Done Done false
25626 7901 Acked Read 3 Sent Done Done false
25627 7902 Acked Read 3 Sent Done Done false
25628 7903 Acked Write 8 Sent Done Done false
25629 7904 Acked Read 9 Sent Done Done false
25630 7905 Acked Read 4 Sent Done Done false
25631 7906 Acked Read 3 Sent Done Done false
25632 7907 Acked Read 4 Sent Done Done false
25633 7908 Acked Read 1 Sent Done Done false
25634 7909 Acked Flush 0 Sent Done Done false
25635 7910 Acked Write 7 Sent Done Done false
25636 7911 Acked Flush 0 Sent Done Done false
25637 7912 Acked Flush 0 Sent Done Done false
25638 7913 Acked Write 4 Sent Done Done false
25639 7914 Acked Flush 0 Sent Done Done false
25640 7915 Acked Read 3 Sent Done Done false
25641 7916 Acked Write 6 Sent Done Done false
25642 7917 Acked Write 4 Sent Done Done false
25643 7918 Acked Read 4 Sent Done Done false
25644 7919 Acked Flush 0 Sent Done Done false
25645 7920 Acked Flush 0 Sent Done Done false
25646 7921 Acked Read 7 Sent Done Done false
25647 7922 Acked Write 1 Sent Done Done false
25648 7923 Acked Read 5 Sent Done Done false
25649 7924 Acked Flush 0 Sent Done Done false
25650 7925 Acked Read 7 Sent Done Done false
25651 7926 Acked Write 8 Sent Done Done false
25652 7927 Acked Write 9 Sent Done Done false
25653 7928 Acked Write 8 Sent Done Done false
25654 7929 Acked Write 8 Sent Done Done false
25655 7930 Acked Flush 0 Sent Done Done false
25656 7931 Acked Read 8 Sent Done Done false
25657 7932 Acked Read 7 Sent Done Done false
25658 7933 Acked Write 7 Sent Done Done false
25659 7934 Acked Write 10 Sent Done Done false
25660 7935 Acked Write 3 Sent Done Done false
25661 7936 Acked Read 1 Sent Done Done false
25662 7937 Acked Read 1 Sent Done Done false
25663 7938 Acked Write 9 Sent Done Done false
25664 7939 Acked Read 5 Sent Done Done false
25665 7940 Acked Write 7 Sent Done Done false
25666 7941 Acked Read 6 Sent Done Done false
25667 7942 Acked Read 1 Sent Done Done false
25668 7943 Acked Flush 0 Sent Done Done false
25669 STATES DS:0 DS:1 DS:2 TOTAL
25670 Sent 302 0 0 302
25671 Done 0 302 302 604
25672 Skipped 0 0 0 0
25673 Error 0 0 0 0
25674 Last Flush: 7641 7943 7943
25675 Downstairs last five completed: 7641 7640 7639 7638 7637
25676 Upstairs last five completed: 7943 7942 7941 7940 7939
25677 CLIENT: Up:0 ds:302 act:2
256782025-01-13T15:57:10.574ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
256792025-01-13T15:57:10.574ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:44445) = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
256802025-01-13T15:57:10.575ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
256812025-01-13T15:57:10.576ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
256822025-01-13T15:57:10.576ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
256832025-01-13T15:57:10.576ZINFOcrucible: send last flush ID to this DS: 7641 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
256842025-01-13T15:57:10.576ZINFOcrucible: Replied this last flush ID: 7641 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
256852025-01-13T15:57:10.576ZINFOcrucible: [0] client re-new 302 jobs since flush 7641 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
256862025-01-13T15:57:10.576ZINFOcrucible: [0] Replayed 302 jobs since flush: 7641 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
25687 ----------------------------------------------------------------
25688 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
25689 Downstairs last five completed: 7943 7942 7941 7940 7939
25690 Upstairs last five completed: 7943 7942 7941 7940 7939
25691 CLIENT: Up:0 ds:0 act:3
25692 Replay: All jobs finished
25693 Main task has work to do, go find it
25694 got dsc Stop(2)
25695 stop 2
25696 Jan 13 15:57:12.823[8830] Got stop action so:false kr:false
25697 INFO request completed, latency_us: 378, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 0dd9f8a2-3a08-481a-909c-e259c822400f, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
256982025-01-13T15:57:12.824ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
256992025-01-13T15:57:12.824ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
257002025-01-13T15:57:12.824ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
257012025-01-13T15:57:12.824ZINFOcrucible: client task is exiting = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
25702 Jan 13 15:57:12.824 INFO request completed, latency_us: 189, response_code: 200, uri: /state/cid/2, method: GET, req_id: d7865cd9-2f60-4f46-8e07-7c86f22bde03, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
25703 [8830] Exited with: ExitStatus(unix_wait_status(9))
25704 [8830][2] reports Exit
25705 Jan 13 15:57:16.827 INFO request completed, latency_us: 367, response_code: 200, uri: /state/cid/2, method: GET, req_id: 494bc821-0384-45c7-979c-0418e65967a0, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
25706 001/300 Flush
25707 002/300 Flush
25708 003/300 Write block 25 len 28672 data: 7 6 8 8 8 8 6
25709 004/300 Write block 629 len 28672 data: 3 3 4 4 5 4 4
25710 005/300 Read block 3233 len 28672
25711 006/300 Read block 462 len 20480
25712 007/300 Write block 1456 len 40960 data: 3 2 2 2 2 2 2 2 3 3
25713 008/300 Read block 4525 len 8192
25714 009/300 Read block 2214 len 16384
25715 010/300 Write block 343 len 12288 data: 4 4 3
25716 011/300 Read block 4832 len 36864
25717 012/300 Write block 2679 len 32768 data: 6 5 4 4 5 7 6 4
25718 013/300 Read block 4585 len 36864
25719 014/300 Read block 3292 len 40960
25720 015/300 Write block 500 len 4096 data: 4
25721 016/300 Read block 690 len 40960
25722 017/300 Read block 1167 len 16384
25723 018/300 Write block 4681 len 40960 data: 5 5 5 7 6 5 4 5 3 3
25724 019/300 Read block 3868 len 24576
25725 020/300 Flush
25726 021/300 Write block 4064 len 40960 data: 3 3 3 2 2 3 5 5 4 6
25727 022/300 Write block 1378 len 36864 data: 7 10 10 8 9 10 9 10 10
25728 023/300 Write block 1729 len 8192 data: 5 5
25729 024/300 Read block 4360 len 28672
25730 025/300 Read block 231 len 8192
25731 026/300 Read block 1469 len 16384
25732 027/300 Flush
25733 028/300 Read block 3211 len 36864
25734 029/300 Write block 4528 len 28672 data: 4 5 6 8 9 6 4
25735 030/300 Read block 2018 len 40960
25736 031/300 Write block 3026 len 12288 data: 7 9 9
25737 032/300 Read block 1501 len 8192
25738 033/300 Read block 4895 len 40960
25739 034/300 Read block 3629 len 28672
25740 035/300 Flush
25741 036/300 Read block 2424 len 20480
25742 037/300 Write block 1240 len 20480 data: 6 6 5 5 4
25743 038/300 Read block 2512 len 12288
25744 039/300 Write block 358 len 32768 data: 5 4 5 5 5 5 6 6
25745 040/300 Read block 3251 len 20480
25746 041/300 Read block 3487 len 4096
25747 042/300 Write block 1407 len 40960 data: 4 5 6 6 6 7 6 6 7 5
25748 043/300 Write block 759 len 20480 data: 6 7 7 7 7
25749 044/300 Flush
25750 045/300 Write block 2677 len 16384 data: 4 4 7 6
25751 046/300 Write block 1452 len 4096 data: 4
25752 047/300 Read block 594 len 40960
25753 048/300 Read block 1273 len 20480
25754 049/300 Read block 2371 len 40960
25755 050/300 Write block 1749 len 8192 data: 5 5
25756 051/300 Read block 2 len 40960
25757 052/300 Read block 3824 len 20480
25758 053/300 Write block 1240 len 8192 data: 7 7
25759 054/300 Write block 2974 len 4096 data: 5
25760 055/300 Write block 1870 len 12288 data: 3 3 3
25761 056/300 Write block 253 len 4096 data: 2
25762 057/300 Read block 3170 len 36864
25763 058/300 Flush
25764 059/300 Read block 1957 len 16384
25765 060/300 Write block 4188 len 20480 data: 7 7 5 6 4
25766 061/300 Write block 2067 len 32768 data: 3 2 2 2 2 3 3 3
25767 062/300 Read block 3697 len 32768
25768 063/300 Read block 4106 len 24576
25769 064/300 Write block 2762 len 8192 data: 3 5
25770 065/300 Read block 524 len 40960
25771 066/300 Read block 3218 len 24576
25772 067/300 Read block 2917 len 8192
25773 068/300 Flush
25774 069/300 Read block 2030 len 28672
25775 070/300 Read block 116 len 12288
25776 071/300 Read block 4215 len 4096
25777 072/300 Read block 2283 len 8192
25778 073/300 Read block 2416 len 32768
25779 074/300 Read block 29 len 16384
25780 075/300 Write block 1653 len 8192 data: 3 2
25781 076/300 Read block 2014 len 20480
25782 077/300 Flush
25783 078/300 Write block 4514 len 20480 data: 7 6 6 6 6
25784 079/300 Read block 2515 len 36864
25785 080/300 Write block 1148 len 32768 data: 9 8 7 6 4 3 2 2
25786 081/300 Write block 237 len 4096 data: 4
25787 082/300 Flush
25788 083/300 Write block 1612 len 32768 data: 3 5 4 4 8 7 6 6
25789 084/300 Read block 4866 len 4096
25790 085/300 Read block 1705 len 40960
25791 086/300 Read block 3595 len 32768
25792 087/300 Read block 3950 len 32768
25793 088/300 Read block 949 len 8192
25794 089/300 Read block 3996 len 28672
25795 090/300 Write block 1052 len 4096 data: 7
25796 091/300 Read block 1209 len 12288
25797 092/300 Flush
25798 093/300 Read block 3654 len 32768
25799 094/300 Read block 2188 len 8192
25800 095/300 Read block 1072 len 32768
25801 096/300 Write block 2093 len 12288 data: 4 4 4
25802 097/300 Write block 3514 len 24576 data: 9 8 5 6 5 6
25803 098/300 Read block 2011 len 28672
25804 099/300 Read block 4264 len 28672
25805 100/300 Read block 4388 len 40960
25806 101/300 Read block 4377 len 28672
25807 102/300 Read block 3039 len 36864
25808 103/300 Read block 4664 len 16384
25809 104/300 Write block 211 len 16384 data: 8 8 8 8
25810 105/300 Read block 4305 len 28672
25811 106/300 Read block 1134 len 4096
25812 107/300 Write block 3300 len 24576 data: 5 6 5 4 3 4
25813 108/300 Flush
25814 109/300 Flush
25815 110/300 Write block 4298 len 24576 data: 9 7 8 9 8 9
25816 111/300 Read block 4599 len 36864
25817 112/300 Read block 2409 len 24576
25818 113/300 Read block 4567 len 12288
25819 114/300 Read block 4540 len 20480
25820 115/300 Write block 1169 len 40960 data: 8 6 7 6 7 5 4 4 6 5
25821 116/300 Flush
25822 117/300 Read block 1319 len 20480
25823 118/300 Flush
25824 119/300 Read block 4776 len 36864
25825 120/300 Read block 3073 len 12288
25826 121/300 Write block 252 len 8192 data: 2 3
25827 122/300 Read block 240 len 36864
25828 123/300 Read block 4586 len 28672
25829 124/300 Read block 4407 len 8192
25830 125/300 Write block 3551 len 4096 data: 4
25831 126/300 Read block 35 len 40960
25832 127/300 Flush
25833 128/300 Read block 2640 len 36864
25834 129/300 Read block 1143 len 20480
25835 130/300 Flush
25836 131/300 Read block 3944 len 20480
25837 132/300 Read block 1123 len 24576
25838 133/300 Read block 917 len 36864
25839 134/300 Flush
25840 135/300 Read block 2094 len 16384
25841 136/300 Flush
25842 137/300 Write block 3362 len 28672 data: 5 4 3 3 3 3 3
25843 138/300 Write block 2512 len 32768 data: 3 3 3 3 4 4 5 5
25844 139/300 Read block 3100 len 36864
25845 140/300 Write block 4172 len 40960 data: 6 5 4 5 5 5 4 5 5 8
25846 141/300 Read block 258 len 28672
25847 142/300 Read block 3115 len 24576
25848 143/300 Write block 3740 len 4096 data: 2
25849 144/300 Flush
25850 145/300 Read block 3675 len 28672
25851 146/300 Flush
25852 147/300 Flush
25853 148/300 Read block 2186 len 32768
25854 149/300 Flush
25855 150/300 Read block 4399 len 8192
25856 151/300 Read block 1163 len 24576
25857 152/300 Read block 1713 len 20480
25858 153/300 Read block 2386 len 8192
25859 154/300 Read block 2220 len 16384
25860 155/300 Read block 1799 len 8192
25861 156/300 Write block 2773 len 4096 data: 3
25862 157/300 Read block 614 len 40960
25863 158/300 Write block 796 len 20480 data: 4 5 3 2 2
25864 159/300 Write block 2001 len 28672 data: 3 5 5 5 5 5 5
25865 160/300 Write block 862 len 20480 data: 6 6 6 5 6
25866 161/300 Read block 4857 len 32768
25867 162/300 Write block 3423 len 16384 data: 5 4 4 4
25868 163/300 Read block 404 len 36864
25869 164/300 Read block 3001 len 4096
25870 165/300 Write block 4816 len 16384 data: 5 6 5 6
25871 166/300 Read block 4222 len 28672
25872 167/300 Write block 4061 len 4096 data: 3
25873 168/300 Read block 3237 len 40960
25874 169/300 Read block 4017 len 4096
25875 170/300 Write block 1525 len 4096 data: 5
25876 171/300 Read block 160 len 20480
25877 172/300 Write block 3319 len 40960 data: 6 6 7 6 5 6 5 7 8 6
25878 173/300 Read block 2499 len 16384
25879 174/300 Read block 1177 len 32768
25880 175/300 Read block 141 len 40960
25881 176/300 Write block 2865 len 32768 data: 7 6 8 7 7 7 7 7
25882 177/300 Read block 4900 len 28672
25883 178/300 Read block 2953 len 24576
25884 179/300 Write block 4585 len 28672 data: 6 5 3 3 5 5 6
25885 180/300 Read block 1442 len 32768
25886 181/300 Flush
25887 182/300 Flush
25888 183/300 Read block 3795 len 4096
25889 184/300 Read block 107 len 40960
25890 185/300 Write block 1078 len 12288 data: 6 6 7
25891 186/300 Read block 2921 len 16384
25892 187/300 Read block 1589 len 8192
25893 188/300 Write block 3579 len 24576 data: 3 3 3 6 6 5
25894 189/300 Write block 1311 len 36864 data: 3 5 5 5 5 5 4 4 7
25895 190/300 Flush
25896 191/300 Write block 2204 len 32768 data: 6 7 6 5 6 8 8 7
25897 192/300 Flush
25898 193/300 Write block 1587 len 20480 data: 6 6 6 7 8
25899 194/300 Read block 3135 len 8192
25900 195/300 Write block 2228 len 16384 data: 9 8 10 9
25901 196/300 Write block 1994 len 32768 data: 5 4 4 4 4 2 2 4
25902 197/300 Read block 3436 len 40960
25903 198/300 Read block 2285 len 20480
25904 199/300 Write block 760 len 32768 data: 8 8 8 8 7 7 7 7
25905 200/300 Write block 3276 len 24576 data: 8 8 7 7 6 6
25906 201/300 Read block 1798 len 36864
25907 202/300 Read block 4117 len 20480
25908 203/300 Write block 4693 len 20480 data: 5 6 6 4 4
25909 204/300 Write block 4338 len 40960 data: 4 3 2 4 4 6 5 6 5 5
25910 205/300 Flush
25911 206/300 Read block 1144 len 16384
25912 207/300 Write block 259 len 16384 data: 2 2 2 2
25913 208/300 Write block 1435 len 28672 data: 6 7 8 6 6 8 7
25914 209/300 Write block 1062 len 16384 data: 6 6 5 3
25915 210/300 Write block 3679 len 24576 data: 4 4 3 4 4 4
25916 211/300 Read block 1139 len 32768
25917 212/300 Write block 120 len 36864 data: 8 8 8 7 6 7 6 6 6
25918 213/300 Write block 497 len 12288 data: 4 4 3
25919 214/300 Write block 160 len 32768 data: 4 3 3 2 4 4 3 4
25920 215/300 Write block 1636 len 20480 data: 4 5 3 4 4
25921 216/300 Read block 334 len 20480
25922 217/300 Read block 4131 len 8192
25923 218/300 Write block 4710 len 4096 data: 4
25924 219/300 Write block 1110 len 16384 data: 5 5 3 4
25925 220/300 Flush
25926 221/300 Read block 2960 len 28672
25927 222/300 Read block 546 len 36864
25928 223/300 Read block 989 len 12288
25929 224/300 Write block 2518 len 12288 data: 6 6 6
25930 225/300 Flush
25931 226/300 Read block 3743 len 8192
25932 227/300 Read block 17 len 8192
25933 228/300 Read block 3159 len 28672
25934 229/300 Read block 1325 len 4096
25935 230/300 Write block 381 len 12288 data: 6 4 5
25936 231/300 Write block 3769 len 16384 data: 8 8 9 10
25937 232/300 Read block 234 len 8192
25938 233/300 Write block 992 len 32768 data: 5 5 5 5 4 3 4 4
25939 234/300 Write block 56 len 40960 data: 6 5 5 5 6 7 6 7 6 5
25940 235/300 Write block 500 len 32768 data: 5 5 6 5 2 3 4 5
25941 236/300 Write block 43 len 32768 data: 5 5 5 4 4 4 6 6
25942 237/300 Write block 2834 len 8192 data: 4 4
25943 238/300 Write block 810 len 40960 data: 5 5 3 2 2 2 3 3 4 4
25944 239/300 Write block 2167 len 36864 data: 4 3 3 5 11 11 10 9 9
25945 240/300 Write block 2381 len 28672 data: 7 5 5 4 5 5 4
25946 241/300 Write block 2558 len 4096 data: 6
25947 242/300 Flush
25948 243/300 Read block 1314 len 8192
25949 244/300 Write block 4561 len 16384 data: 5 5 5 5
25950 245/300 Write block 774 len 8192 data: 6 6
25951 246/300 Read block 4444 len 12288
25952 247/300 Read block 3676 len 36864
25953 248/300 Read block 4889 len 16384
25954 249/300 Flush
25955 250/300 Write block 1155 len 12288 data: 3 3 3
25956 251/300 Write block 3019 len 20480 data: 5 9 8 8 7
25957 252/300 Write block 2022 len 16384 data: 8 9 10 9
25958 253/300 Flush
25959 254/300 Write block 1899 len 4096 data: 5
25960 255/300 Read block 2363 len 20480
25961 256/300 Read block 3737 len 28672
25962 257/300 Write block 1924 len 32768 data: 6 6 6 4 4 5 6 6
25963 258/300 Write block 2714 len 4096 data: 4
25964 259/300 Read block 1349 len 4096
25965 260/300 Write block 1428 len 32768 data: 8 8 7 6 5 5 4 7
25966 261/300 Read block 2235 len 36864
25967 262/300 Read block 1371 len 4096
25968 263/300 Read block 482 len 20480
25969 264/300 Flush
25970 265/300 Read block 51 len 20480
25971 266/300 Flush
25972 267/300 Flush
25973 268/300 Write block 4720 len 40960 data: 2 3 3 3 2 4 4 4 5 6
25974 269/300 Write block 3550 len 36864 data: 4 5 3 4 3 2 3 4 5
25975 270/300 Flush
25976 271/300 Write block 772 len 24576 data: 5 7 7 7 5 7
25977 272/300 Read block 360 len 32768
25978 273/300 Read block 1402 len 20480
25979 274/300 Read block 485 len 8192
25980 275/300 Write block 1111 len 20480 data: 6 4 5 4 5
25981 276/300 Read block 3566 len 24576
25982 277/300 Read block 2095 len 28672
25983 278/300 Read block 14 len 20480
25984 279/300 Write block 2999 len 8192 data: 3 3
25985 280/300 Read block 100 len 36864
25986 281/300 Write block 3640 len 4096 data: 7
25987 282/300 Write block 3976 len 8192 data: 6 7
25988 283/300 Write block 703 len 28672 data: 6 5 7 8 8 7 6
25989 284/300 Read block 4941 len 28672
25990 285/300 Write block 1756 len 24576 data: 2 3 4 5 4 5
25991 286/300 Read block 2513 len 8192
25992 287/300 Flush
25993 288/300 Flush
25994 289/300 Read block 4043 len 40960
25995 290/300 Write block 535 len 24576 data: 6 5 4 7 7 5
25996 291/300 Write block 4423 len 16384 data: 4 5 4 5
25997 292/300 Read block 342 len 24576
25998 293/300 Write block 2317 len 32768 data: 5 4 4 4 3 3 3 3
25999 294/300 Read block 2332 len 20480
26000 295/300 Read block 760 len 8192
26001 296/300 Write block 1891 len 20480 data: 3 4 5 5 5
26002 297/300 Write block 116 len 24576 data: 6 6 6 7 9 9
26003 298/300 Write block 135 len 32768 data: 7 5 6 5 3 4 4 6
26004 299/300 Write block 1515 len 24576 data: 3 6 7 8 7 8
26005 300/300 Write block 4513 len 28672 data: 5 8 7 7 7 7 7
26006 301/300 Read block 1694 len 8192
26007 Main task has work to do, go find it
26008 got dsc Start(2)
26009 start 2
26010 Jan 13 15:57:17.065 INFO request completed, latency_us: 188, response_code: 204, uri: /start/cid/2, method: POST, req_id: dbb74acc-bf19-4114-8de3-7537fb7382f3, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
26011 Replay: started 2, returned:Ok(())
26012 ----------------------------------------------------------------
26013 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
26014 States: Active Active Offline
26015 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26016 7944 Acked Flush 0 Done Done Sent false
26017 7945 Acked Flush 0 Done Done Sent false
26018 7946 Acked Write 7 Done Done Sent false
26019 7947 Acked Write 7 Done Done Sent false
26020 7948 Acked Read 7 Done Done Sent false
26021 7949 Acked Read 5 Done Done Sent false
26022 7950 Acked Write 10 Done Done Sent false
26023 7951 Acked Read 2 Done Done Sent false
26024 7952 Acked Read 4 Done Done Sent false
26025 7953 Acked Write 3 Done Done Sent false
26026 7954 Acked Read 9 Done Done Sent false
26027 7955 Acked Write 8 Done Done Sent false
26028 7956 Acked Read 9 Done Done Sent false
26029 7957 Acked Read 10 Done Done Sent false
26030 7958 Acked Write 1 Done Done Sent false
26031 7959 Acked Read 10 Done Done Sent false
26032 7960 Acked Read 4 Done Done Sent false
26033 7961 Acked Write 10 Done Done Sent false
26034 7962 Acked Read 6 Done Done Sent false
26035 7963 Acked Flush 0 Done Done Sent false
26036 7964 Acked Write 10 Done Done Sent false
26037 7965 Acked Write 9 Done Done Sent false
26038 7966 Acked Write 2 Done Done Sent false
26039 7967 Acked Read 7 Done Done Sent false
26040 7968 Acked Read 2 Done Done Sent false
26041 7969 Acked Read 4 Done Done Sent false
26042 7970 Acked Flush 0 Done Done Sent false
26043 7971 Acked Read 9 Done Done Sent false
26044 7972 Acked Write 7 Done Done Sent false
26045 7973 Acked Read 10 Done Done Sent false
26046 7974 Acked Write 3 Done Done Sent false
26047 7975 Acked Read 2 Done Done Sent false
26048 7976 Acked Read 10 Done Done Sent false
26049 7977 Acked Read 7 Done Done Sent false
26050 7978 Acked Flush 0 Done Done Sent false
26051 7979 Acked Read 5 Done Done Sent false
26052 7980 Acked Write 5 Done Done Sent false
26053 7981 Acked Read 3 Done Done Sent false
26054 7982 Acked Write 8 Done Done Sent false
26055 7983 Acked Read 5 Done Done Sent false
26056 7984 Acked Read 1 Done Done Sent false
26057 7985 Acked Write 10 Done Done Sent false
26058 7986 Acked Write 5 Done Done Sent false
26059 7987 Acked Flush 0 Done Done Sent false
26060 7988 Acked Write 4 Done Done Sent false
26061 7989 Acked Write 1 Done Done Sent false
26062 7990 Acked Read 10 Done Done Sent false
26063 7991 Acked Read 5 Done Done Sent false
26064 7992 Acked Read 10 Done Done Sent false
26065 7993 Acked Write 2 Done Done Sent false
26066 7994 Acked Read 10 Done Done Sent false
26067 7995 Acked Read 5 Done Done Sent false
26068 7996 Acked Write 2 Done Done Sent false
26069 7997 Acked Write 1 Done Done Sent false
26070 7998 Acked Write 3 Done Done Sent false
26071 7999 Acked Write 1 Done Done Sent false
26072 8000 Acked Read 9 Done Done Sent false
26073 8001 Acked Flush 0 Done Done Sent false
26074 8002 Acked Read 4 Done Done Sent false
26075 8003 Acked Write 5 Done Done Sent false
26076 8004 Acked Write 8 Done Done Sent false
26077 8005 Acked Read 8 Done Done Sent false
26078 8006 Acked Read 6 Done Done Sent false
26079 8007 Acked Write 2 Done Done Sent false
26080 8008 Acked Read 10 Done Done Sent false
26081 8009 Acked Read 6 Done Done Sent false
26082 8010 Acked Read 2 Done Done Sent false
26083 8011 Acked Flush 0 Done Done Sent false
26084 8012 Acked Read 7 Done Done Sent false
26085 8013 Acked Read 3 Done Done Sent false
26086 8014 Acked Read 1 Done Done Sent false
26087 8015 Acked Read 2 Done Done Sent false
26088 8016 Acked Read 8 Done Done Sent false
26089 8017 Acked Read 4 Done Done Sent false
26090 8018 Acked Write 2 Done Done Sent false
26091 8019 Acked Read 5 Done Done Sent false
26092 8020 Acked Flush 0 Done Done Sent false
26093 8021 Acked Write 5 Done Done Sent false
26094 8022 Acked Read 9 Done Done Sent false
26095 8023 Acked Write 8 Done Done Sent false
26096 8024 Acked Write 1 Done Done Sent false
26097 8025 Acked Flush 0 Done Done Sent false
26098 8026 Acked Write 8 Done Done Sent false
26099 8027 Acked Read 1 Done Done Sent false
26100 8028 Acked Read 10 Done Done Sent false
26101 8029 Acked Read 8 Done Done Sent false
26102 8030 Acked Read 8 Done Done Sent false
26103 8031 Acked Read 2 Done Done Sent false
26104 8032 Acked Read 7 Done Done Sent false
26105 8033 Acked Write 1 Done Done Sent false
26106 8034 Acked Read 3 Done Done Sent false
26107 8035 Acked Flush 0 Done Done Sent false
26108 8036 Acked Read 8 Done Done Sent false
26109 8037 Acked Read 2 Done Done Sent false
26110 8038 Acked Read 8 Done Done Sent false
26111 8039 Acked Write 3 Done Done Sent false
26112 8040 Acked Write 6 Done Done Sent false
26113 8041 Acked Read 7 Done Done Sent false
26114 8042 Acked Read 7 Done Done Sent false
26115 8043 Acked Read 10 Done Done Sent false
26116 8044 Acked Read 7 Done Done Sent false
26117 8045 Acked Read 9 Done Done Sent false
26118 8046 Acked Read 4 Done Done Sent false
26119 8047 Acked Write 4 Done Done Sent false
26120 8048 Acked Read 7 Done Done Sent false
26121 8049 Acked Read 1 Done Done Sent false
26122 8050 Acked Write 6 Done Done Sent false
26123 8051 Acked Flush 0 Done Done Sent false
26124 8052 Acked Flush 0 Done Done Sent false
26125 8053 Acked Write 6 Done Done Sent false
26126 8054 Acked Read 9 Done Done Sent false
26127 8055 Acked Read 6 Done Done Sent false
26128 8056 Acked Read 3 Done Done Sent false
26129 8057 Acked Read 5 Done Done Sent false
26130 8058 Acked Write 10 Done Done Sent false
26131 8059 Acked Flush 0 Done Done Sent false
26132 8060 Acked Read 5 Done Done Sent false
26133 8061 Acked Flush 0 Done Done Sent false
26134 8062 Acked Read 9 Done Done Sent false
26135 8063 Acked Read 3 Done Done Sent false
26136 8064 Acked Write 2 Done Done Sent false
26137 8065 Acked Read 9 Done Done Sent false
26138 8066 Acked Read 7 Done Done Sent false
26139 8067 Acked Read 2 Done Done Sent false
26140 8068 Acked Write 1 Done Done Sent false
26141 8069 Acked Read 10 Done Done Sent false
26142 8070 Acked Flush 0 Done Done Sent false
26143 8071 Acked Read 9 Done Done Sent false
26144 8072 Acked Read 5 Done Done Sent false
26145 8073 Acked Flush 0 Done Done Sent false
26146 8074 Acked Read 5 Done Done Sent false
26147 8075 Acked Read 6 Done Done Sent false
26148 8076 Acked Read 9 Done Done Sent false
26149 8077 Acked Flush 0 Done Done Sent false
26150 8078 Acked Read 4 Done Done Sent false
26151 8079 Acked Flush 0 Done Done Sent false
26152 8080 Acked Write 7 Done Done Sent false
26153 8081 Acked Write 8 Done Done Sent false
26154 8082 Acked Read 9 Done Done Sent false
26155 8083 Acked Write 10 Done Done Sent false
26156 8084 Acked Read 7 Done Done Sent false
26157 8085 Acked Read 6 Done Done Sent false
26158 8086 Acked Write 1 Done Done Sent false
26159 8087 Acked Flush 0 Done Done Sent false
26160 8088 Acked Read 7 Done Done Sent false
26161 8089 Acked Flush 0 Done Done Sent false
26162 8090 Acked Flush 0 Done Done Sent false
26163 8091 Acked Read 8 Done Done Sent false
26164 8092 Acked Flush 0 Done Done Sent false
26165 8093 Acked Read 2 Done Done Sent false
26166 8094 Acked Read 6 Done Done Sent false
26167 8095 Acked Read 5 Done Done Sent false
26168 8096 Acked Read 2 Done Done Sent false
26169 8097 Acked Read 4 Done Done Sent false
26170 8098 Acked Read 2 Done Done Sent false
26171 8099 Acked Write 1 Done Done Sent false
26172 8100 Acked Read 10 Done Done Sent false
26173 8101 Acked Write 5 Done Done Sent false
26174 8102 Acked Write 7 Done Done Sent false
26175 8103 Acked Write 5 Done Done Sent false
26176 8104 Acked Read 8 Done Done Sent false
26177 8105 Acked Write 4 Done Done Sent false
26178 8106 Acked Read 9 Done Done Sent false
26179 8107 Acked Read 1 Done Done Sent false
26180 8108 Acked Write 4 Done Done Sent false
26181 8109 Acked Read 7 Done Done Sent false
26182 8110 Acked Write 1 Done Done Sent false
26183 8111 Acked Read 10 Done Done Sent false
26184 8112 Acked Read 1 Done Done Sent false
26185 8113 Acked Write 1 Done Done Sent false
26186 8114 Acked Read 5 Done Done Sent false
26187 8115 Acked Write 10 Done Done Sent false
26188 8116 Acked Read 4 Done Done Sent false
26189 8117 Acked Read 8 Done Done Sent false
26190 8118 Acked Read 10 Done Done Sent false
26191 8119 Acked Write 8 Done Done Sent false
26192 8120 Acked Read 7 Done Done Sent false
26193 8121 Acked Read 6 Done Done Sent false
26194 8122 Acked Write 7 Done Done Sent false
26195 8123 Acked Read 8 Done Done Sent false
26196 8124 Acked Flush 0 Done Done Sent false
26197 8125 Acked Flush 0 Done Done Sent false
26198 8126 Acked Read 1 Done Done Sent false
26199 8127 Acked Read 10 Done Done Sent false
26200 8128 Acked Write 3 Done Done Sent false
26201 8129 Acked Read 4 Done Done Sent false
26202 8130 Acked Read 2 Done Done Sent false
26203 8131 Acked Write 6 Done Done Sent false
26204 8132 Acked Write 9 Done Done Sent false
26205 8133 Acked Flush 0 Done Done Sent false
26206 8134 Acked Write 8 Done Done Sent false
26207 8135 Acked Flush 0 Done Done Sent false
26208 8136 Acked Write 5 Done Done Sent false
26209 8137 Acked Read 2 Done Done Sent false
26210 8138 Acked Write 4 Done Done Sent false
26211 8139 Acked Write 8 Done Done Sent false
26212 8140 Acked Read 10 Done Done Sent false
26213 8141 Acked Read 5 Done Done Sent false
26214 8142 Acked Write 8 Done Done Sent false
26215 8143 Acked Write 6 Done Done Sent false
26216 8144 Acked Read 9 Done Done Sent false
26217 8145 Acked Read 5 Done Done Sent false
26218 8146 Acked Write 5 Done Done Sent false
26219 8147 Acked Write 10 Done Done Sent false
26220 8148 Acked Flush 0 Done Done Sent false
26221 8149 Acked Read 4 Done Done Sent false
26222 8150 Acked Write 4 Done Done Sent false
26223 8151 Acked Write 7 Done Done Sent false
26224 8152 Acked Write 4 Done Done Sent false
26225 8153 Acked Write 6 Done Done Sent false
26226 8154 Acked Read 8 Done Done Sent false
26227 8155 Acked Write 9 Done Done Sent false
26228 8156 Acked Write 3 Done Done Sent false
26229 8157 Acked Write 8 Done Done Sent false
26230 8158 Acked Write 5 Done Done Sent false
26231 8159 Acked Read 5 Done Done Sent false
26232 8160 Acked Read 2 Done Done Sent false
26233 8161 Acked Write 1 Done Done Sent false
26234 8162 Acked Write 4 Done Done Sent false
26235 8163 Acked Flush 0 Done Done Sent false
26236 8164 Acked Read 7 Done Done Sent false
26237 8165 Acked Read 9 Done Done Sent false
26238 8166 Acked Read 3 Done Done Sent false
26239 8167 Acked Write 3 Done Done Sent false
26240 8168 Acked Flush 0 Done Done Sent false
26241 8169 Acked Read 2 Done Done Sent false
26242 8170 Acked Read 2 Done Done Sent false
26243 8171 Acked Read 7 Done Done Sent false
26244 8172 Acked Read 1 Done Done Sent false
26245 8173 Acked Write 3 Done Done Sent false
26246 8174 Acked Write 4 Done Done Sent false
26247 8175 Acked Read 2 Done Done Sent false
26248 8176 Acked Write 8 Done Done Sent false
26249 8177 Acked Write 10 Done Done Sent false
26250 8178 Acked Write 8 Done Done Sent false
26251 8179 Acked Write 8 Done Done Sent false
26252 8180 Acked Write 2 Done Done Sent false
26253 8181 Acked Write 10 Done Done Sent false
26254 8182 Acked Write 9 Done Done Sent false
26255 8183 Acked Write 7 Done Done Sent false
26256 8184 Acked Write 1 Done Done Sent false
26257 8185 Acked Flush 0 Done Done Sent false
26258 8186 Acked Read 2 Done Done Sent false
26259 8187 Acked Write 4 Done Done Sent false
26260 8188 Acked Write 2 Done Done Sent false
26261 8189 Acked Read 3 Done Done Sent false
26262 8190 Acked Read 9 Done Done Sent false
26263 8191 Acked Read 4 Done Done Sent false
26264 8192 Acked Flush 0 Done Done Sent false
26265 8193 Acked Write 3 Done Done Sent false
26266 8194 Acked Write 5 Done Done Sent false
26267 8195 Acked Write 4 Done Done Sent false
26268 8196 Acked Flush 0 Done Done Sent false
26269 8197 Acked Write 1 Done Done Sent false
26270 8198 Acked Read 5 Done Done Sent false
26271 8199 Acked Read 7 Done Done Sent false
26272 8200 Acked Write 8 Done Done Sent false
26273 8201 Acked Write 1 Done Done Sent false
26274 8202 Acked Read 1 Done Done Sent false
26275 8203 Acked Write 8 Done Done Sent false
26276 8204 Acked Read 9 Done Done Sent false
26277 8205 Acked Read 1 Done Done Sent false
26278 8206 Acked Read 5 Done Done Sent false
26279 8207 Acked Flush 0 Done Done Sent false
26280 8208 Acked Read 5 Done Done Sent false
26281 8209 Acked Flush 0 Done Done Sent false
26282 8210 Acked Flush 0 Done Done Sent false
26283 8211 Acked Write 10 Done Done Sent false
26284 8212 Acked Write 9 Done Done Sent false
26285 8213 Acked Flush 0 Done Done Sent false
26286 8214 Acked Write 6 Done Done Sent false
26287 8215 Acked Read 8 Done Done Sent false
26288 8216 Acked Read 5 Done Done Sent false
26289 8217 Acked Read 2 Done Done Sent false
26290 8218 Acked Write 5 Done Done Sent false
26291 8219 Acked Read 6 Done Done Sent false
26292 8220 Acked Read 7 Done Done Sent false
26293 8221 Acked Read 5 Done Done Sent false
26294 8222 Acked Write 2 Done Done Sent false
26295 8223 Acked Read 9 Done Done Sent false
26296 8224 Acked Write 1 Done Done Sent false
26297 8225 Acked Write 2 Done Done Sent false
26298 8226 Acked Write 7 Done Done Sent false
26299 8227 Acked Read 7 Done Done Sent false
26300 8228 Acked Write 6 Done Done Sent false
26301 8229 Acked Read 2 Done Done Sent false
26302 8230 Acked Flush 0 Done Done Sent false
26303 8231 Acked Flush 0 Done Done Sent false
26304 8232 Acked Read 10 Done Done Sent false
26305 8233 Acked Write 6 Done Done Sent false
26306 8234 Acked Write 4 Done Done Sent false
26307 8235 Acked Read 6 Done Done Sent false
26308 8236 Acked Write 8 Done Done Sent false
26309 8237 Acked Read 5 Done Done Sent false
26310 8238 Acked Read 2 Done Done Sent false
26311 8239 Acked Write 5 Done Done Sent false
26312 8240 Acked Write 6 Done Done Sent false
26313 8241 Acked Write 8 Done Done Sent false
26314 8242 Acked Write 6 Done Done Sent false
26315 8243 Acked Write 7 Done Done Sent false
26316 8244 Acked Read 2 Done Done Sent false
26317 STATES DS:0 DS:1 DS:2 TOTAL
26318 Sent 0 0 301 301
26319 Done 301 301 0 602
26320 Skipped 0 0 0 0
26321 Error 0 0 0 0
26322 Last Flush: 8231 8231 7943
26323 Downstairs last five completed: 7943 7942 7941 7940 7939
26324 Upstairs last five completed: 8244 8243 8242 8241 8240
26325 CLIENT: Up:0 ds:301 act:2
26326 [8830] got start action
26327 [8830] I am going to restart
26328 Starting downstairs at port 8830
26329 Make output file at "/tmp/dsc/downstairs-8830.txt"
26330 [8830][2] reports Starting
26331 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1226)
26332 [8830][2] reports Running
26333 ----------------------------------------------------------------
26334 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26335 States: Active Active Offline
26336 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26337 7944 Acked Flush 0 Done Done Sent false
26338 7945 Acked Flush 0 Done Done Sent false
26339 7946 Acked Write 7 Done Done Sent false
26340 7947 Acked Write 7 Done Done Sent false
26341 7948 Acked Read 7 Done Done Sent false
26342 7949 Acked Read 5 Done Done Sent false
26343 7950 Acked Write 10 Done Done Sent false
26344 7951 Acked Read 2 Done Done Sent false
26345 7952 Acked Read 4 Done Done Sent false
26346 7953 Acked Write 3 Done Done Sent false
26347 7954 Acked Read 9 Done Done Sent false
26348 7955 Acked Write 8 Done Done Sent false
26349 7956 Acked Read 9 Done Done Sent false
26350 7957 Acked Read 10 Done Done Sent false
26351 7958 Acked Write 1 Done Done Sent false
26352 7959 Acked Read 10 Done Done Sent false
26353 7960 Acked Read 4 Done Done Sent false
26354 7961 Acked Write 10 Done Done Sent false
26355 7962 Acked Read 6 Done Done Sent false
26356 7963 Acked Flush 0 Done Done Sent false
26357 7964 Acked Write 10 Done Done Sent false
26358 7965 Acked Write 9 Done Done Sent false
26359 7966 Acked Write 2 Done Done Sent false
26360 7967 Acked Read 7 Done Done Sent false
26361 7968 Acked Read 2 Done Done Sent false
26362 7969 Acked Read 4 Done Done Sent false
26363 7970 Acked Flush 0 Done Done Sent false
26364 7971 Acked Read 9 Done Done Sent false
26365 7972 Acked Write 7 Done Done Sent false
26366 7973 Acked Read 10 Done Done Sent false
26367 7974 Acked Write 3 Done Done Sent false
26368 7975 Acked Read 2 Done Done Sent false
26369 7976 Acked Read 10 Done Done Sent false
26370 7977 Acked Read 7 Done Done Sent false
26371 7978 Acked Flush 0 Done Done Sent false
26372 7979 Acked Read 5 Done Done Sent false
26373 7980 Acked Write 5 Done Done Sent false
26374 7981 Acked Read 3 Done Done Sent false
26375 7982 Acked Write 8 Done Done Sent false
26376 7983 Acked Read 5 Done Done Sent false
26377 7984 Acked Read 1 Done Done Sent false
26378 7985 Acked Write 10 Done Done Sent false
26379 7986 Acked Write 5 Done Done Sent false
26380 7987 Acked Flush 0 Done Done Sent false
26381 7988 Acked Write 4 Done Done Sent false
26382 7989 Acked Write 1 Done Done Sent false
26383 7990 Acked Read 10 Done Done Sent false
26384 7991 Acked Read 5 Done Done Sent false
26385 7992 Acked Read 10 Done Done Sent false
26386 7993 Acked Write 2 Done Done Sent false
26387 7994 Acked Read 10 Done Done Sent false
26388 7995 Acked Read 5 Done Done Sent false
26389 7996 Acked Write 2 Done Done Sent false
26390 7997 Acked Write 1 Done Done Sent false
26391 7998 Acked Write 3 Done Done Sent false
26392 7999 Acked Write 1 Done Done Sent false
26393 8000 Acked Read 9 Done Done Sent false
26394 8001 Acked Flush 0 Done Done Sent false
26395 8002 Acked Read 4 Done Done Sent false
26396 8003 Acked Write 5 Done Done Sent false
26397 8004 Acked Write 8 Done Done Sent false
26398 8005 Acked Read 8 Done Done Sent false
26399 8006 Acked Read 6 Done Done Sent false
26400 8007 Acked Write 2 Done Done Sent false
26401 8008 Acked Read 10 Done Done Sent false
26402 8009 Acked Read 6 Done Done Sent false
26403 8010 Acked Read 2 Done Done Sent false
26404 8011 Acked Flush 0 Done Done Sent false
26405 8012 Acked Read 7 Done Done Sent false
26406 8013 Acked Read 3 Done Done Sent false
26407 8014 Acked Read 1 Done Done Sent false
26408 8015 Acked Read 2 Done Done Sent false
26409 8016 Acked Read 8 Done Done Sent false
26410 8017 Acked Read 4 Done Done Sent false
26411 8018 Acked Write 2 Done Done Sent false
26412 8019 Acked Read 5 Done Done Sent false
26413 8020 Acked Flush 0 Done Done Sent false
26414 8021 Acked Write 5 Done Done Sent false
26415 8022 Acked Read 9 Done Done Sent false
26416 8023 Acked Write 8 Done Done Sent false
26417 8024 Acked Write 1 Done Done Sent false
26418 8025 Acked Flush 0 Done Done Sent false
26419 8026 Acked Write 8 Done Done Sent false
26420 8027 Acked Read 1 Done Done Sent false
26421 8028 Acked Read 10 Done Done Sent false
26422 8029 Acked Read 8 Done Done Sent false
26423 8030 Acked Read 8 Done Done Sent false
26424 8031 Acked Read 2 Done Done Sent false
26425 8032 Acked Read 7 Done Done Sent false
26426 8033 Acked Write 1 Done Done Sent false
26427 8034 Acked Read 3 Done Done Sent false
26428 8035 Acked Flush 0 Done Done Sent false
26429 8036 Acked Read 8 Done Done Sent false
26430 8037 Acked Read 2 Done Done Sent false
26431 8038 Acked Read 8 Done Done Sent false
26432 8039 Acked Write 3 Done Done Sent false
26433 8040 Acked Write 6 Done Done Sent false
26434 8041 Acked Read 7 Done Done Sent false
26435 8042 Acked Read 7 Done Done Sent false
26436 8043 Acked Read 10 Done Done Sent false
26437 8044 Acked Read 7 Done Done Sent false
26438 8045 Acked Read 9 Done Done Sent false
26439 8046 Acked Read 4 Done Done Sent false
26440 8047 Acked Write 4 Done Done Sent false
26441 8048 Acked Read 7 Done Done Sent false
26442 8049 Acked Read 1 Done Done Sent false
26443 8050 Acked Write 6 Done Done Sent false
26444 8051 Acked Flush 0 Done Done Sent false
26445 8052 Acked Flush 0 Done Done Sent false
26446 8053 Acked Write 6 Done Done Sent false
26447 8054 Acked Read 9 Done Done Sent false
26448 8055 Acked Read 6 Done Done Sent false
26449 8056 Acked Read 3 Done Done Sent false
26450 8057 Acked Read 5 Done Done Sent false
26451 8058 Acked Write 10 Done Done Sent false
26452 8059 Acked Flush 0 Done Done Sent false
26453 8060 Acked Read 5 Done Done Sent false
26454 8061 Acked Flush 0 Done Done Sent false
26455 8062 Acked Read 9 Done Done Sent false
26456 8063 Acked Read 3 Done Done Sent false
26457 8064 Acked Write 2 Done Done Sent false
26458 8065 Acked Read 9 Done Done Sent false
26459 8066 Acked Read 7 Done Done Sent false
26460 8067 Acked Read 2 Done Done Sent false
26461 8068 Acked Write 1 Done Done Sent false
26462 8069 Acked Read 10 Done Done Sent false
26463 8070 Acked Flush 0 Done Done Sent false
26464 8071 Acked Read 9 Done Done Sent false
26465 8072 Acked Read 5 Done Done Sent false
26466 8073 Acked Flush 0 Done Done Sent false
26467 8074 Acked Read 5 Done Done Sent false
26468 8075 Acked Read 6 Done Done Sent false
26469 8076 Acked Read 9 Done Done Sent false
26470 8077 Acked Flush 0 Done Done Sent false
26471 8078 Acked Read 4 Done Done Sent false
26472 8079 Acked Flush 0 Done Done Sent false
26473 8080 Acked Write 7 Done Done Sent false
26474 8081 Acked Write 8 Done Done Sent false
26475 8082 Acked Read 9 Done Done Sent false
26476 8083 Acked Write 10 Done Done Sent false
26477 8084 Acked Read 7 Done Done Sent false
26478 8085 Acked Read 6 Done Done Sent false
26479 8086 Acked Write 1 Done Done Sent false
26480 8087 Acked Flush 0 Done Done Sent false
26481 8088 Acked Read 7 Done Done Sent false
26482 8089 Acked Flush 0 Done Done Sent false
26483 8090 Acked Flush 0 Done Done Sent false
26484 8091 Acked Read 8 Done Done Sent false
26485 8092 Acked Flush 0 Done Done Sent false
26486 8093 Acked Read 2 Done Done Sent false
26487 8094 Acked Read 6 Done Done Sent false
26488 8095 Acked Read 5 Done Done Sent false
26489 8096 Acked Read 2 Done Done Sent false
26490 8097 Acked Read 4 Done Done Sent false
26491 8098 Acked Read 2 Done Done Sent false
26492 8099 Acked Write 1 Done Done Sent false
26493 8100 Acked Read 10 Done Done Sent false
26494 8101 Acked Write 5 Done Done Sent false
26495 8102 Acked Write 7 Done Done Sent false
26496 8103 Acked Write 5 Done Done Sent false
26497 8104 Acked Read 8 Done Done Sent false
26498 8105 Acked Write 4 Done Done Sent false
26499 8106 Acked Read 9 Done Done Sent false
26500 8107 Acked Read 1 Done Done Sent false
26501 8108 Acked Write 4 Done Done Sent false
26502 8109 Acked Read 7 Done Done Sent false
26503 8110 Acked Write 1 Done Done Sent false
26504 8111 Acked Read 10 Done Done Sent false
26505 8112 Acked Read 1 Done Done Sent false
26506 8113 Acked Write 1 Done Done Sent false
26507 8114 Acked Read 5 Done Done Sent false
26508 8115 Acked Write 10 Done Done Sent false
26509 8116 Acked Read 4 Done Done Sent false
26510 8117 Acked Read 8 Done Done Sent false
26511 8118 Acked Read 10 Done Done Sent false
26512 8119 Acked Write 8 Done Done Sent false
26513 8120 Acked Read 7 Done Done Sent false
26514 8121 Acked Read 6 Done Done Sent false
26515 8122 Acked Write 7 Done Done Sent false
26516 8123 Acked Read 8 Done Done Sent false
26517 8124 Acked Flush 0 Done Done Sent false
26518 8125 Acked Flush 0 Done Done Sent false
26519 8126 Acked Read 1 Done Done Sent false
26520 8127 Acked Read 10 Done Done Sent false
26521 8128 Acked Write 3 Done Done Sent false
26522 8129 Acked Read 4 Done Done Sent false
26523 8130 Acked Read 2 Done Done Sent false
26524 8131 Acked Write 6 Done Done Sent false
26525 8132 Acked Write 9 Done Done Sent false
26526 8133 Acked Flush 0 Done Done Sent false
26527 8134 Acked Write 8 Done Done Sent false
26528 8135 Acked Flush 0 Done Done Sent false
26529 8136 Acked Write 5 Done Done Sent false
26530 8137 Acked Read 2 Done Done Sent false
26531 8138 Acked Write 4 Done Done Sent false
26532 8139 Acked Write 8 Done Done Sent false
26533 8140 Acked Read 10 Done Done Sent false
26534 8141 Acked Read 5 Done Done Sent false
26535 8142 Acked Write 8 Done Done Sent false
26536 8143 Acked Write 6 Done Done Sent false
26537 8144 Acked Read 9 Done Done Sent false
26538 8145 Acked Read 5 Done Done Sent false
26539 8146 Acked Write 5 Done Done Sent false
26540 8147 Acked Write 10 Done Done Sent false
26541 8148 Acked Flush 0 Done Done Sent false
26542 8149 Acked Read 4 Done Done Sent false
26543 8150 Acked Write 4 Done Done Sent false
26544 8151 Acked Write 7 Done Done Sent false
26545 8152 Acked Write 4 Done Done Sent false
26546 8153 Acked Write 6 Done Done Sent false
26547 8154 Acked Read 8 Done Done Sent false
26548 8155 Acked Write 9 Done Done Sent false
26549 8156 Acked Write 3 Done Done Sent false
26550 8157 Acked Write 8 Done Done Sent false
26551 8158 Acked Write 5 Done Done Sent false
26552 8159 Acked Read 5 Done Done Sent false
26553 8160 Acked Read 2 Done Done Sent false
26554 8161 Acked Write 1 Done Done Sent false
26555 8162 Acked Write 4 Done Done Sent false
26556 8163 Acked Flush 0 Done Done Sent false
26557 8164 Acked Read 7 Done Done Sent false
26558 8165 Acked Read 9 Done Done Sent false
26559 8166 Acked Read 3 Done Done Sent false
26560 8167 Acked Write 3 Done Done Sent false
26561 8168 Acked Flush 0 Done Done Sent false
26562 8169 Acked Read 2 Done Done Sent false
26563 8170 Acked Read 2 Done Done Sent false
26564 8171 Acked Read 7 Done Done Sent false
26565 8172 Acked Read 1 Done Done Sent false
26566 8173 Acked Write 3 Done Done Sent false
26567 8174 Acked Write 4 Done Done Sent false
26568 8175 Acked Read 2 Done Done Sent false
26569 8176 Acked Write 8 Done Done Sent false
26570 8177 Acked Write 10 Done Done Sent false
26571 8178 Acked Write 8 Done Done Sent false
26572 8179 Acked Write 8 Done Done Sent false
26573 8180 Acked Write 2 Done Done Sent false
26574 8181 Acked Write 10 Done Done Sent false
26575 8182 Acked Write 9 Done Done Sent false
26576 8183 Acked Write 7 Done Done Sent false
26577 8184 Acked Write 1 Done Done Sent false
26578 8185 Acked Flush 0 Done Done Sent false
26579 8186 Acked Read 2 Done Done Sent false
26580 8187 Acked Write 4 Done Done Sent false
26581 8188 Acked Write 2 Done Done Sent false
26582 8189 Acked Read 3 Done Done Sent false
26583 8190 Acked Read 9 Done Done Sent false
26584 8191 Acked Read 4 Done Done Sent false
26585 8192 Acked Flush 0 Done Done Sent false
26586 8193 Acked Write 3 Done Done Sent false
26587 8194 Acked Write 5 Done Done Sent false
26588 8195 Acked Write 4 Done Done Sent false
26589 8196 Acked Flush 0 Done Done Sent false
26590 8197 Acked Write 1 Done Done Sent false
26591 8198 Acked Read 5 Done Done Sent false
26592 8199 Acked Read 7 Done Done Sent false
26593 8200 Acked Write 8 Done Done Sent false
26594 8201 Acked Write 1 Done Done Sent false
26595 8202 Acked Read 1 Done Done Sent false
26596 8203 Acked Write 8 Done Done Sent false
26597 8204 Acked Read 9 Done Done Sent false
26598 8205 Acked Read 1 Done Done Sent false
26599 8206 Acked Read 5 Done Done Sent false
26600 8207 Acked Flush 0 Done Done Sent false
26601 8208 Acked Read 5 Done Done Sent false
26602 8209 Acked Flush 0 Done Done Sent false
26603 8210 Acked Flush 0 Done Done Sent false
26604 8211 Acked Write 10 Done Done Sent false
26605 8212 Acked Write 9 Done Done Sent false
26606 8213 Acked Flush 0 Done Done Sent false
26607 8214 Acked Write 6 Done Done Sent false
26608 8215 Acked Read 8 Done Done Sent false
26609 8216 Acked Read 5 Done Done Sent false
26610 8217 Acked Read 2 Done Done Sent false
26611 8218 Acked Write 5 Done Done Sent false
26612 8219 Acked Read 6 Done Done Sent false
26613 8220 Acked Read 7 Done Done Sent false
26614 8221 Acked Read 5 Done Done Sent false
26615 8222 Acked Write 2 Done Done Sent false
26616 8223 Acked Read 9 Done Done Sent false
26617 8224 Acked Write 1 Done Done Sent false
26618 8225 Acked Write 2 Done Done Sent false
26619 8226 Acked Write 7 Done Done Sent false
26620 8227 Acked Read 7 Done Done Sent false
26621 8228 Acked Write 6 Done Done Sent false
26622 8229 Acked Read 2 Done Done Sent false
26623 8230 Acked Flush 0 Done Done Sent false
26624 8231 Acked Flush 0 Done Done Sent false
26625 8232 Acked Read 10 Done Done Sent false
26626 8233 Acked Write 6 Done Done Sent false
26627 8234 Acked Write 4 Done Done Sent false
26628 8235 Acked Read 6 Done Done Sent false
26629 8236 Acked Write 8 Done Done Sent false
26630 8237 Acked Read 5 Done Done Sent false
26631 8238 Acked Read 2 Done Done Sent false
26632 8239 Acked Write 5 Done Done Sent false
26633 8240 Acked Write 6 Done Done Sent false
26634 8241 Acked Write 8 Done Done Sent false
26635 8242 Acked Write 6 Done Done Sent false
26636 8243 Acked Write 7 Done Done Sent false
26637 8244 Acked Read 2 Done Done Sent false
26638 8245 Acked Flush 0 Done Done Sent false
26639 STATES DS:0 DS:1 DS:2 TOTAL
26640 Sent 0 0 302 302
26641 Done 302 302 0 604
26642 Skipped 0 0 0 0
26643 Error 0 0 0 0
26644 Last Flush: 8245 8245 7943
26645 Downstairs last five completed: 7943 7942 7941 7940 7939
26646 Upstairs last five completed: 8245 8244 8243 8242 8241
26647 CLIENT: Up:0 ds:302 act:2
266482025-01-13T15:57:22.825ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
266492025-01-13T15:57:22.825ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59438) = io task client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
266502025-01-13T15:57:22.827ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
266512025-01-13T15:57:22.827ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
266522025-01-13T15:57:22.827ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
266532025-01-13T15:57:22.827ZINFOcrucible: send last flush ID to this DS: 7943 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
266542025-01-13T15:57:22.827ZINFOcrucible: Replied this last flush ID: 7943 = downstairs client = 2 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
266552025-01-13T15:57:22.827ZINFOcrucible: [2] client re-new 302 jobs since flush 7943 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
266562025-01-13T15:57:22.828ZINFOcrucible: [2] Replayed 302 jobs since flush: 7943 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
26657 ----------------------------------------------------------------
26658 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
26659 Downstairs last five completed: 8245 8244 8243 8242 8241
26660 Upstairs last five completed: 8245 8244 8243 8242 8241
26661 CLIENT: Up:0 ds:0 act:3
26662 Replay: All jobs finished
26663 Main task has work to do, go find it
26664 got dsc Stop(1)
26665 stop 1
26666 Jan 13 15:57:25.070 INFO request completed, latency_us: 360, response_code: 204[8820] Got stop action so:false kr:false
26667 , uri: /stop/cid/1, method: POST, req_id: 57241440-fb9c-4d2e-ad55-b799952f90df, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
26668 Jan 13 15:57:25.071 INFO request completed, latency_us: 133, response_code: 200, uri: /state/cid/1, method: GET, req_id: 471770bc-6f89-4add-838f-30420c056bc6, remote_addr: 127.0.0{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:57:25.071450363Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
26669 .1:52422, local_addr: 127.0.0.1:9998
266702025-01-13T15:57:25.071ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
266712025-01-13T15:57:25.071ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
266722025-01-13T15:57:25.071ZINFOcrucible: client task is exiting = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
26673 [8820] Exited with: ExitStatus(unix_wait_status(9))
26674 [8820][1] reports Exit
26675 Jan 13 15:57:29.074 INFO request completed, latency_us: 343, response_code: 200, uri: /state/cid/1, method: GET, req_id: c4bf6c8d-dc56-453e-947a-4b582b80a66c, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
26676 001/300 Read block 2236 len 12288
26677 002/300 Flush
26678 003/300 Flush
26679 004/300 Read block 4251 len 36864
26680 005/300 Read block 919 len 12288
26681 006/300 Read block 3089 len 24576
26682 007/300 Flush
26683 008/300 Read block 2936 len 28672
26684 009/300 Read block 4208 len 12288
26685 010/300 Write block 303 len 16384 data: 5 5 5 4
26686 011/300 Write block 1362 len 8192 data: 4 4
26687 012/300 Read block 117 len 40960
26688 013/300 Read block 1900 len 28672
26689 014/300 Read block 1854 len 32768
26690 015/300 Read block 3430 len 24576
26691 016/300 Write block 4119 len 40960 data: 3 4 5 4 4 4 4 4 4 4
26692 017/300 Read block 4926 len 12288
26693 018/300 Read block 2413 len 40960
26694 019/300 Flush
26695 020/300 Write block 1367 len 8192 data: 3 4
26696 021/300 Flush
26697 022/300 Write block 2903 len 12288 data: 6 5 5
26698 023/300 Write block 1167 len 8192 data: 7 8
26699 024/300 Write block 117 len 16384 data: 7 7 8 10
26700 025/300 Write block 1924 len 36864 data: 7 7 7 5 5 6 7 7 5
26701 026/300 Read block 4254 len 24576
26702 027/300 Read block 3975 len 12288
26703 028/300 Read block 3244 len 36864
26704 029/300 Write block 1395 len 24576 data: 3 2 2 3 3 3
26705 030/300 Write block 167 len 12288 data: 5 6 6
26706 031/300 Write block 3948 len 12288 data: 3 3 4
26707 032/300 Read block 2250 len 40960
26708 033/300 Read block 873 len 20480
26709 034/300 Write block 4785 len 36864 data: 7 6 5 5 5 5 4 3 3
26710 035/300 Read block 982 len 28672
26711 036/300 Read block 625 len 20480
26712 037/300 Write block 1452 len 20480 data: 5 5 5 4 4
26713 038/300 Read block 2015 len 12288
26714 039/300 Read block 3737 len 24576
26715 040/300 Read block 586 len 24576
26716 041/300 Read block 2149 len 40960
26717 042/300 Write block 84 len 24576 data: 6 4 5 4 2 2
26718 043/300 Read block 3677 len 20480
26719 044/300 Write block 4387 len 32768 data: 7 7 6 5 6 6 5 5
26720 045/300 Write block 268 len 16384 data: 3 4 5 5
26721 046/300 Read block 3501 len 20480
26722 047/300 Write block 81 len 8192 data: 7 7
26723 048/300 Write block 300 len 28672 data: 3 2 2 6 6 6 5
26724 049/300 Flush
26725 050/300 Read block 4286 len 4096
26726 051/300 Read block 3073 len 36864
26727 052/300 Read block 1580 len 40960
26728 053/300 Read block 3887 len 20480
26729 054/300 Read block 4239 len 32768
26730 055/300 Write block 335 len 16384 data: 3 3 3 3
26731 056/300 Write block 1758 len 32768 data: 5 6 5 6 5 5 6 6
26732 057/300 Write block 577 len 28672 data: 8 7 6 6 5 5 4
26733 058/300 Write block 3721 len 20480 data: 5 7 7 6 4
26734 059/300 Write block 2016 len 12288 data: 3 3 4
26735 060/300 Read block 1105 len 12288
26736 061/300 Read block 404 len 24576
26737 062/300 Read block 3662 len 28672
26738 063/300 Read block 3878 len 32768
26739 064/300 Read block 2045 len 16384
26740 065/300 Read block 3202 len 12288
26741 066/300 Write block 3842 len 24576 data: 6 6 6 6 6 4
26742 067/300 Read block 3354 len 20480
26743 068/300 Write block 3192 len 4096 data: 12
26744 069/300 Read block 1675 len 36864
26745 070/300 Read block 1740 len 20480
26746 071/300 Read block 2688 len 8192
26747 072/300 Read block 1002 len 36864
26748 073/300 Write block 91 len 24576 data: 3 4 4 3 3 2
26749 074/300 Read block 493 len 12288
26750 075/300 Write block 4677 len 8192 data: 6 4
26751 076/300 Write block 4047 len 4096 data: 2
26752 077/300 Read block 1076 len 12288
26753 078/300 Write block 2611 len 16384 data: 7 7 6 5
26754 079/300 Read block 991 len 32768
26755 080/300 Write block 915 len 36864 data: 2 2 2 2 2 3 4 5 4
26756 081/300 Read block 2990 len 28672
26757 082/300 Read block 4545 len 28672
26758 083/300 Write block 95 len 20480 data: 4 3 2 2 2
26759 084/300 Write block 1093 len 16384 data: 5 4 4 4
26760 085/300 Read block 1940 len 24576
26761 086/300 Flush
26762 087/300 Read block 578 len 20480
26763 088/300 Read block 1516 len 16384
26764 089/300 Read block 588 len 40960
26765 090/300 Write block 1842 len 32768 data: 7 6 5 4 3 3 3 4
26766 091/300 Read block 1824 len 16384
26767 092/300 Read block 4010 len 24576
26768 093/300 Read block 4615 len 8192
26769 094/300 Write block 2160 len 24576 data: 3 4 5 5 4 4
26770 095/300 Write block 3615 len 20480 data: 5 5 5 3 2
26771 096/300 Write block 1625 len 8192 data: 6 6
26772 097/300 Read block 4251 len 16384
26773 098/300 Read block 3508 len 32768
26774 099/300 Read block 4983 len 40960
26775 100/300 Write block 3592 len 16384 data: 6 4 4 4
26776 101/300 Write block 3727 len 20480 data: 9 9 9 5 4
26777 102/300 Read block 940 len 8192
26778 103/300 Read block 903 len 16384
26779 104/300 Flush
26780 105/300 Write block 183 len 12288 data: 4 3 3
26781 106/300 Write block 2293 len 12288 data: 4 4 5
26782 107/300 Read block 1684 len 36864
26783 108/300 Read block 4546 len 24576
26784 109/300 Read block 3010 len 12288
26785 110/300 Read block 296 len 4096
26786 111/300 Read block 3118 len 28672
26787 112/300 Read block 764 len 20480
26788 113/300 Read block 77 len 24576
26789 114/300 Flush
26790 115/300 Write block 706 len 4096 data: 9
26791 116/300 Write block 1051 len 32768 data: 6 8 7 7 7 8 7 4
26792 117/300 Write block 4590 len 4096 data: 6
26793 118/300 Write block 3933 len 36864 data: 4 4 4 4 5 5 4 4 4
26794 119/300 Write block 1334 len 16384 data: 6 8 10 9
26795 120/300 Write block 4046 len 12288 data: 3 3 2
26796 121/300 Read block 979 len 8192
26797 122/300 Write block 2413 len 28672 data: 6 5 6 6 6 7 6
26798 123/300 Read block 1515 len 32768
26799 124/300 Read block 1166 len 16384
26800 125/300 Read block 970 len 12288
26801 126/300 Read block 4737 len 12288
26802 127/300 Flush
26803 128/300 Read block 4979 len 4096
26804 129/300 Read block 446 len 36864
26805 130/300 Read block 3244 len 36864
26806 131/300 Read block 1288 len 28672
26807 132/300 Read block 3144 len 16384
26808 133/300 Write block 4198 len 24576 data: 5 5 6 4 4 4
26809 134/300 Read block 7 len 4096
26810 135/300 Read block 2821 len 8192
26811 136/300 Write block 4323 len 36864 data: 4 5 6 5 6 4 4 6 6
26812 137/300 Read block 1632 len 28672
26813 138/300 Read block 825 len 16384
26814 139/300 Write block 4078 len 4096 data: 3
26815 140/300 Read block 1667 len 4096
26816 141/300 Read block 944 len 40960
26817 142/300 Read block 2167 len 4096
26818 143/300 Read block 4245 len 8192
26819 144/300 Read block 1433 len 8192
26820 145/300 Write block 3932 len 40960 data: 4 5 5 5 5 6 6 5 5 5
26821 146/300 Read block 320 len 8192
26822 147/300 Write block 89 len 20480 data: 3 2 4 5 5
26823 148/300 Read block 2114 len 36864
26824 149/300 Read block 3196 len 4096
26825 150/300 Flush
26826 151/300 Read block 3135 len 8192
26827 152/300 Flush
26828 153/300 Write block 610 len 28672 data: 4 3 3 4 3 4 4
26829 154/300 Write block 3236 len 28672 data: 4 4 4 3 3 4 4
26830 155/300 Write block 1330 len 28672 data: 5 4 5 5 7 9 11
26831 156/300 Write block 2640 len 4096 data: 7
26832 157/300 Read block 354 len 24576
26833 158/300 Write block 4793 len 40960 data: 4 2 3 4 5 4 4 6 5 5
26834 159/300 Read block 45 len 36864
26835 160/300 Read block 1784 len 20480
26836 161/300 Write block 3526 len 12288 data: 4 4 4
26837 162/300 Write block 94 len 36864 data: 4 5 4 3 3 3 4 4 5
26838 163/300 Read block 4260 len 12288
26839 164/300 Read block 2570 len 20480
26840 165/300 Read block 137 len 20480
26841 166/300 Read block 2125 len 40960
26842 167/300 Write block 4328 len 40960 data: 5 5 7 7 6 6 6 6 7 6
26843 168/300 Flush
26844 169/300 Read block 1996 len 32768
26845 170/300 Write block 1149 len 32768 data: 9 8 7 5 4 3 4 4
26846 171/300 Read block 1141 len 36864
26847 172/300 Write block 3683 len 40960 data: 5 5 2 3 3 4 3 2 2 2
26848 173/300 Read block 4565 len 8192
26849 174/300 Write block 549 len 12288 data: 4 4 3
26850 175/300 Read block 4671 len 20480
26851 176/300 Read block 1787 len 16384
26852 177/300 Flush
26853 178/300 Write block 998 len 40960 data: 5 5 4 4 3 4 4 4 4 4
26854 179/300 Read block 4394 len 36864
26855 180/300 Read block 4241 len 40960
26856 181/300 Read block 255 len 12288
26857 182/300 Flush
26858 183/300 Write block 1199 len 8192 data: 4 4
26859 184/300 Flush
26860 185/300 Write block 4076 len 36864 data: 3 5 4 4 4 4 4 3 3
26861 186/300 Read block 2526 len 40960
26862 187/300 Read block 694 len 16384
26863 188/300 Flush
26864 189/300 Read block 2711 len 12288
26865 190/300 Write block 2459 len 40960 data: 4 4 4 4 4 3 4 3 4 5
26866 191/300 Read block 2163 len 12288
26867 192/300 Read block 4920 len 16384
26868 193/300 Read block 3205 len 24576
26869 194/300 Write block 2339 len 28672 data: 8 6 6 7 7 8 8
26870 195/300 Write block 3801 len 32768 data: 4 5 5 7 5 4 5 5
26871 196/300 Read block 4829 len 36864
26872 197/300 Write block 1966 len 8192 data: 8 8
26873 198/300 Read block 1278 len 8192
26874 199/300 Write block 2176 len 16384 data: 9 9 8 8
26875 200/300 Write block 1398 len 16384 data: 4 4 4 3
26876 201/300 Read block 587 len 12288
26877 202/300 Write block 1997 len 36864 data: 5 5 3 3 5 6 6 6 6
26878 203/300 Flush
26879 204/300 Read block 1671 len 28672
26880 205/300 Write block 4350 len 8192 data: 5 5
26881 206/300 Read block 822 len 24576
26882 207/300 Write block 4650 len 28672 data: 2 2 2 2 3 3 5
26883 208/300 Write block 3265 len 28672 data: 8 6 7 7 9 10 9
26884 209/300 Write block 478 len 8192 data: 9 9
26885 210/300 Write block 1045 len 12288 data: 7 6 6
26886 211/300 Write block 4538 len 4096 data: 3
26887 212/300 Write block 742 len 16384 data: 4 6 5 5
26888 213/300 Read block 1712 len 20480
26889 214/300 Write block 4239 len 24576 data: 2 2 3 4 5 5
26890 215/300 Write block 2102 len 24576 data: 5 4 4 5 5 6
26891 216/300 Write block 527 len 28672 data: 6 4 5 5 6 5 4
26892 217/300 Read block 2288 len 12288
26893 218/300 Write block 594 len 36864 data: 3 4 4 5 4 5 5 5 5
26894 219/300 Read block 1214 len 40960
26895 220/300 Write block 1007 len 8192 data: 5 4
26896 221/300 Write block 1218 len 28672 data: 3 5 5 4 5 5 6
26897 222/300 Read block 2968 len 28672
26898 223/300 Write block 1827 len 36864 data: 7 6 5 4 5 6 6 7 6
26899 224/300 Read block 1946 len 8192
26900 225/300 Write block 4631 len 12288 data: 4 5 4
26901 226/300 Read block 1484 len 12288
26902 227/300 Flush
26903 228/300 Read block 381 len 20480
26904 229/300 Write block 4231 len 28672 data: 4 3 3 3 3 3 2
26905 230/300 Write block 4969 len 32768 data: 5 4 3 4 4 7 8 7
26906 231/300 Read block 4771 len 28672
26907 232/300 Read block 2274 len 20480
26908 233/300 Read block 3754 len 28672
26909 234/300 Write block 800 len 20480 data: 3 4 5 6 6
26910 235/300 Write block 2572 len 32768 data: 4 4 4 4 5 4 4 4
26911 236/300 Write block 2579 len 28672 data: 5 6 5 3 2 3 2
26912 237/300 Write block 4744 len 16384 data: 6 3 3 4
26913 238/300 Read block 2833 len 36864
26914 239/300 Write block 1273 len 28672 data: 6 6 5 4 5 6 4
26915 240/300 Read block 3614 len 16384
26916 241/300 Write block 1354 len 12288 data: 6 6 6
26917 242/300 Read block 263 len 28672
26918 243/300 Read block 2815 len 40960
26919 244/300 Write block 4945 len 36864 data: 5 4 5 5 5 5 5 5 4
26920 245/300 Flush
26921 246/300 Write block 3133 len 12288 data: 3 4 4
26922 247/300 Write block 586 len 16384 data: 2 3 5 4
26923 248/300 Write block 3799 len 28672 data: 4 5 5 6 6 8 6
26924 249/300 Write block 638 len 16384 data: 3 4 4 4
26925 250/300 Write block 2716 len 24576 data: 4 3 3 3 2 3
26926 251/300 Read block 2042 len 28672
26927 252/300 Read block 354 len 24576
26928 253/300 Read block 1170 len 28672
26929 254/300 Read block 3251 len 20480
26930 255/300 Flush
26931 256/300 Read block 4851 len 32768
26932 257/300 Flush
26933 258/300 Flush
26934 259/300 Flush
26935 260/300 Read block 2312 len 4096
26936 261/300 Write block 4056 len 32768 data: 7 7 8 4 4 4 3 3
26937 262/300 Write block 4931 len 4096 data: 10
26938 263/300 Read block 4641 len 4096
26939 264/300 Write block 310 len 28672 data: 3 3 4 4 4 2 3
26940 265/300 Write block 3305 len 12288 data: 5 4 4
26941 266/300 Write block 1899 len 12288 data: 6 4 4
26942 267/300 Write block 884 len 36864 data: 6 4 4 4 3 4 4 4 3
26943 268/300 Read block 430 len 40960
26944 269/300 Flush
26945 270/300 Read block 4991 len 16384
26946 271/300 Write block 188 len 8192 data: 4 2
26947 272/300 Read block 928 len 20480
26948 273/300 Write block 4873 len 4096 data: 4
26949 274/300 Write block 3862 len 28672 data: 6 5 4 4 3 3 4
26950 275/300 Read block 3909 len 28672
26951 276/300 Write block 2087 len 4096 data: 6
26952 277/300 Read block 1875 len 24576
26953 278/300 Write block 4773 len 8192 data: 5 5
26954 279/300 Write block 5 len 4096 data: 7
26955 280/300 Write block 784 len 36864 data: 8 6 6 4 3 3 3 3 3
26956 281/300 Read block 967 len 32768
26957 282/300 Flush
26958 283/300 Write block 4302 len 16384 data: 9 10 8 9
26959 284/300 Write block 1685 len 16384 data: 2 3 3 4
26960 285/300 Read block 3521 len 4096
26961 286/300 Flush
26962 287/300 Write block 2997 len 24576 data: 4 3 4 4 4 6
26963 288/300 Read block 3188 len 4096
26964 289/300 Write block 3864 len 24576 data: 5 5 4 4 5 5
26965 290/300 Write block 476 len 4096 data: 8
26966 291/300 Read block 1338 len 16384
26967 292/300 Read block 1083 len 8192
26968 293/300 Write block 3279 len 12288 data: 8 7 7
26969 294/300 Read block 1903 len 40960
26970 295/300 Read block 451 len 36864
26971 296/300 Write block 4095 len 4096 data: 4
26972 297/300 Flush
26973 298/300 Write block 2562 len 40960 data: 4 8 7 7 6 5 4 3 4 4
26974 299/300 Read block 1402 len 20480
26975 300/300 Flush
26976 301/300 Write block 4316 len 36864 data: 4 6 6 5 6 4 4 5 6
26977 Main task has work to do, go find it
26978 got dsc Start(1)
26979 start 1
26980 Jan 13 15:57:29.308 INFO request completed, latency_us: 190, response_code: 204, uri: /start/cid/1, method: POST, req_id: 1e52a175-effc-4c1e-8524-8fd29a4388ff, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
26981 Replay: started 1, returned:Ok(())
26982 ----------------------------------------------------------------
26983 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
26984 States: Active Offline Active
26985 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26986 8246 Acked Read 3 Done Sent Done false
26987 8247 Acked Flush 0 Done Sent Done false
26988 8248 Acked Flush 0 Done Sent Done false
26989 8249 Acked Read 9 Done Sent Done false
26990 8250 Acked Read 3 Done Sent Done false
26991 8251 Acked Read 6 Done Sent Done false
26992 8252 Acked Flush 0 Done Sent Done false
26993 8253 Acked Read 7 Done Sent Done false
26994 8254 Acked Read 3 Done Sent Done false
26995 8255 Acked Write 4 Done Sent Done false
26996 8256 Acked Write 2 Done Sent Done false
26997 8257 Acked Read 10 Done Sent Done false
26998 8258 Acked Read 7 Done Sent Done false
26999 8259 Acked Read 8 Done Sent Done false
27000 8260 Acked Read 6 Done Sent Done false
27001 8261 Acked Write 10 Done Sent Done false
27002 8262 Acked Read 3 Done Sent Done false
27003 8263 Acked Read 10 Done Sent Done false
27004 8264 Acked Flush 0 Done Sent Done false
27005 8265 Acked Write 2 Done Sent Done false
27006 8266 Acked Flush 0 Done Sent Done false
27007 8267 Acked Write 3 Done Sent Done false
27008 8268 Acked Write 2 Done Sent Done false
27009 8269 Acked Write 4 Done Sent Done false
27010 8270 Acked Write 9 Done Sent Done false
27011 8271 Acked Read 6 Done Sent Done false
27012 8272 Acked Read 3 Done Sent Done false
27013 8273 Acked Read 9 Done Sent Done false
27014 8274 Acked Write 6 Done Sent Done false
27015 8275 Acked Write 3 Done Sent Done false
27016 8276 Acked Write 3 Done Sent Done false
27017 8277 Acked Read 10 Done Sent Done false
27018 8278 Acked Read 5 Done Sent Done false
27019 8279 Acked Write 9 Done Sent Done false
27020 8280 Acked Read 7 Done Sent Done false
27021 8281 Acked Read 5 Done Sent Done false
27022 8282 Acked Write 5 Done Sent Done false
27023 8283 Acked Read 3 Done Sent Done false
27024 8284 Acked Read 6 Done Sent Done false
27025 8285 Acked Read 6 Done Sent Done false
27026 8286 Acked Read 10 Done Sent Done false
27027 8287 Acked Write 6 Done Sent Done false
27028 8288 Acked Read 5 Done Sent Done false
27029 8289 Acked Write 8 Done Sent Done false
27030 8290 Acked Write 4 Done Sent Done false
27031 8291 Acked Read 5 Done Sent Done false
27032 8292 Acked Write 2 Done Sent Done false
27033 8293 Acked Write 7 Done Sent Done false
27034 8294 Acked Flush 0 Done Sent Done false
27035 8295 Acked Read 1 Done Sent Done false
27036 8296 Acked Read 9 Done Sent Done false
27037 8297 Acked Read 10 Done Sent Done false
27038 8298 Acked Read 5 Done Sent Done false
27039 8299 Acked Read 8 Done Sent Done false
27040 8300 Acked Write 4 Done Sent Done false
27041 8301 Acked Write 8 Done Sent Done false
27042 8302 Acked Write 7 Done Sent Done false
27043 8303 Acked Write 5 Done Sent Done false
27044 8304 Acked Write 3 Done Sent Done false
27045 8305 Acked Read 3 Done Sent Done false
27046 8306 Acked Read 6 Done Sent Done false
27047 8307 Acked Read 7 Done Sent Done false
27048 8308 Acked Read 8 Done Sent Done false
27049 8309 Acked Read 4 Done Sent Done false
27050 8310 Acked Read 3 Done Sent Done false
27051 8311 Acked Write 6 Done Sent Done false
27052 8312 Acked Read 5 Done Sent Done false
27053 8313 Acked Write 1 Done Sent Done false
27054 8314 Acked Read 9 Done Sent Done false
27055 8315 Acked Read 5 Done Sent Done false
27056 8316 Acked Read 2 Done Sent Done false
27057 8317 Acked Read 9 Done Sent Done false
27058 8318 Acked Write 6 Done Sent Done false
27059 8319 Acked Read 3 Done Sent Done false
27060 8320 Acked Write 2 Done Sent Done false
27061 8321 Acked Write 1 Done Sent Done false
27062 8322 Acked Read 3 Done Sent Done false
27063 8323 Acked Write 4 Done Sent Done false
27064 8324 Acked Read 8 Done Sent Done false
27065 8325 Acked Write 9 Done Sent Done false
27066 8326 Acked Read 7 Done Sent Done false
27067 8327 Acked Read 7 Done Sent Done false
27068 8328 Acked Write 5 Done Sent Done false
27069 8329 Acked Write 4 Done Sent Done false
27070 8330 Acked Read 6 Done Sent Done false
27071 8331 Acked Flush 0 Done Sent Done false
27072 8332 Acked Read 5 Done Sent Done false
27073 8333 Acked Read 4 Done Sent Done false
27074 8334 Acked Read 10 Done Sent Done false
27075 8335 Acked Write 8 Done Sent Done false
27076 8336 Acked Read 4 Done Sent Done false
27077 8337 Acked Read 6 Done Sent Done false
27078 8338 Acked Read 2 Done Sent Done false
27079 8339 Acked Write 6 Done Sent Done false
27080 8340 Acked Write 5 Done Sent Done false
27081 8341 Acked Write 2 Done Sent Done false
27082 8342 Acked Read 4 Done Sent Done false
27083 8343 Acked Read 8 Done Sent Done false
27084 8344 Acked Read 10 Done Sent Done false
27085 8345 Acked Write 4 Done Sent Done false
27086 8346 Acked Write 5 Done Sent Done false
27087 8347 Acked Read 2 Done Sent Done false
27088 8348 Acked Read 4 Done Sent Done false
27089 8349 Acked Flush 0 Done Sent Done false
27090 8350 Acked Write 3 Done Sent Done false
27091 8351 Acked Write 3 Done Sent Done false
27092 8352 Acked Read 9 Done Sent Done false
27093 8353 Acked Read 6 Done Sent Done false
27094 8354 Acked Read 3 Done Sent Done false
27095 8355 Acked Read 1 Done Sent Done false
27096 8356 Acked Read 7 Done Sent Done false
27097 8357 Acked Read 5 Done Sent Done false
27098 8358 Acked Read 6 Done Sent Done false
27099 8359 Acked Flush 0 Done Sent Done false
27100 8360 Acked Write 1 Done Sent Done false
27101 8361 Acked Write 8 Done Sent Done false
27102 8362 Acked Write 1 Done Sent Done false
27103 8363 Acked Write 9 Done Sent Done false
27104 8364 Acked Write 4 Done Sent Done false
27105 8365 Acked Write 3 Done Sent Done false
27106 8366 Acked Read 2 Done Sent Done false
27107 8367 Acked Write 7 Done Sent Done false
27108 8368 Acked Read 8 Done Sent Done false
27109 8369 Acked Read 4 Done Sent Done false
27110 8370 Acked Read 3 Done Sent Done false
27111 8371 Acked Read 3 Done Sent Done false
27112 8372 Acked Flush 0 Done Sent Done false
27113 8373 Acked Read 1 Done Sent Done false
27114 8374 Acked Read 9 Done Sent Done false
27115 8375 Acked Read 9 Done Sent Done false
27116 8376 Acked Read 7 Done Sent Done false
27117 8377 Acked Read 4 Done Sent Done false
27118 8378 Acked Write 6 Done Sent Done false
27119 8379 Acked Read 1 Done Sent Done false
27120 8380 Acked Read 2 Done Sent Done false
27121 8381 Acked Write 9 Done Sent Done false
27122 8382 Acked Read 7 Done Sent Done false
27123 8383 Acked Read 4 Done Sent Done false
27124 8384 Acked Write 1 Done Sent Done false
27125 8385 Acked Read 1 Done Sent Done false
27126 8386 Acked Read 10 Done Sent Done false
27127 8387 Acked Read 1 Done Sent Done false
27128 8388 Acked Read 2 Done Sent Done false
27129 8389 Acked Read 2 Done Sent Done false
27130 8390 Acked Write 10 Done Sent Done false
27131 8391 Acked Read 2 Done Sent Done false
27132 8392 Acked Write 5 Done Sent Done false
27133 8393 Acked Read 9 Done Sent Done false
27134 8394 Acked Read 1 Done Sent Done false
27135 8395 Acked Flush 0 Done Sent Done false
27136 8396 Acked Read 2 Done Sent Done false
27137 8397 Acked Flush 0 Done Sent Done false
27138 8398 Acked Write 7 Done Sent Done false
27139 8399 Acked Write 7 Done Sent Done false
27140 8400 Acked Write 7 Done Sent Done false
27141 8401 Acked Write 1 Done Sent Done false
27142 8402 Acked Read 6 Done Sent Done false
27143 8403 Acked Write 10 Done Sent Done false
27144 8404 Acked Read 9 Done Sent Done false
27145 8405 Acked Read 5 Done Sent Done false
27146 8406 Acked Write 3 Done Sent Done false
27147 8407 Acked Write 9 Done Sent Done false
27148 8408 Acked Read 3 Done Sent Done false
27149 8409 Acked Read 5 Done Sent Done false
27150 8410 Acked Read 5 Done Sent Done false
27151 8411 Acked Read 10 Done Sent Done false
27152 8412 Acked Write 10 Done Sent Done false
27153 8413 Acked Flush 0 Done Sent Done false
27154 8414 Acked Read 8 Done Sent Done false
27155 8415 Acked Write 8 Done Sent Done false
27156 8416 Acked Read 9 Done Sent Done false
27157 8417 Acked Write 10 Done Sent Done false
27158 8418 Acked Read 2 Done Sent Done false
27159 8419 Acked Write 3 Done Sent Done false
27160 8420 Acked Read 5 Done Sent Done false
27161 8421 Acked Read 4 Done Sent Done false
27162 8422 Acked Flush 0 Done Sent Done false
27163 8423 Acked Write 10 Done Sent Done false
27164 8424 Acked Read 9 Done Sent Done false
27165 8425 Acked Read 10 Done Sent Done false
27166 8426 Acked Read 3 Done Sent Done false
27167 8427 Acked Flush 0 Done Sent Done false
27168 8428 Acked Write 2 Done Sent Done false
27169 8429 Acked Flush 0 Done Sent Done false
27170 8430 Acked Write 9 Done Sent Done false
27171 8431 Acked Read 10 Done Sent Done false
27172 8432 Acked Read 4 Done Sent Done false
27173 8433 Acked Flush 0 Done Sent Done false
27174 8434 Acked Read 3 Done Sent Done false
27175 8435 Acked Write 10 Done Sent Done false
27176 8436 Acked Read 3 Done Sent Done false
27177 8437 Acked Read 4 Done Sent Done false
27178 8438 Acked Read 6 Done Sent Done false
27179 8439 Acked Write 7 Done Sent Done false
27180 8440 Acked Write 8 Done Sent Done false
27181 8441 Acked Read 9 Done Sent Done false
27182 8442 Acked Write 2 Done Sent Done false
27183 8443 Acked Read 2 Done Sent Done false
27184 8444 Acked Write 4 Done Sent Done false
27185 8445 Acked Write 4 Done Sent Done false
27186 8446 Acked Read 3 Done Sent Done false
27187 8447 Acked Write 9 Done Sent Done false
27188 8448 Acked Flush 0 Done Sent Done false
27189 8449 Acked Read 7 Done Sent Done false
27190 8450 Acked Write 2 Done Sent Done false
27191 8451 Acked Read 6 Done Sent Done false
27192 8452 Acked Write 7 Done Sent Done false
27193 8453 Acked Write 7 Done Sent Done false
27194 8454 Acked Write 2 Done Sent Done false
27195 8455 Acked Write 3 Done Sent Done false
27196 8456 Acked Write 1 Done Sent Done false
27197 8457 Acked Write 4 Done Sent Done false
27198 8458 Acked Read 5 Done Sent Done false
27199 8459 Acked Write 6 Done Sent Done false
27200 8460 Acked Write 6 Done Sent Done false
27201 8461 Acked Write 7 Done Sent Done false
27202 8462 Acked Read 3 Done Sent Done false
27203 8463 Acked Write 9 Done Sent Done false
27204 8464 Acked Read 10 Done Sent Done false
27205 8465 Acked Write 2 Done Sent Done false
27206 8466 Acked Write 7 Done Sent Done false
27207 8467 Acked Read 7 Done Sent Done false
27208 8468 Acked Write 9 Done Sent Done false
27209 8469 Acked Read 2 Done Sent Done false
27210 8470 Acked Write 3 Done Sent Done false
27211 8471 Acked Read 3 Done Sent Done false
27212 8472 Acked Flush 0 Done Sent Done false
27213 8473 Acked Read 5 Done Sent Done false
27214 8474 Acked Write 7 Done Sent Done false
27215 8475 Acked Write 8 Done Sent Done false
27216 8476 Acked Read 7 Done Sent Done false
27217 8477 Acked Read 5 Done Sent Done false
27218 8478 Acked Read 7 Done Sent Done false
27219 8479 Acked Write 5 Done Sent Done false
27220 8480 Acked Write 8 Done Sent Done false
27221 8481 Acked Write 7 Done Sent Done false
27222 8482 Acked Write 4 Done Sent Done false
27223 8483 Acked Read 9 Done Sent Done false
27224 8484 Acked Write 7 Done Sent Done false
27225 8485 Acked Read 4 Done Sent Done false
27226 8486 Acked Write 3 Done Sent Done false
27227 8487 Acked Read 7 Done Sent Done false
27228 8488 Acked Read 10 Done Sent Done false
27229 8489 Acked Write 9 Done Sent Done false
27230 8490 Acked Flush 0 Done Sent Done false
27231 8491 Acked Write 3 Done Sent Done false
27232 8492 Acked Write 4 Done Sent Done false
27233 8493 Acked Write 7 Done Sent Done false
27234 8494 Acked Write 4 Done Sent Done false
27235 8495 Acked Write 6 Done Sent Done false
27236 8496 Acked Read 7 Done Sent Done false
27237 8497 Acked Read 6 Done Sent Done false
27238 8498 Acked Read 7 Done Sent Done false
27239 8499 Acked Read 5 Done Sent Done false
27240 8500 Acked Flush 0 Done Sent Done false
27241 8501 Acked Read 8 Done Sent Done false
27242 8502 Acked Flush 0 Done Sent Done false
27243 8503 Acked Flush 0 Done Sent Done false
27244 8504 Acked Flush 0 Done Sent Done false
27245 8505 Acked Read 1 Done Sent Done false
27246 8506 Acked Write 8 Done Sent Done false
27247 8507 Acked Write 1 Done Sent Done false
27248 8508 Acked Read 1 Done Sent Done false
27249 8509 Acked Write 7 Done Sent Done false
27250 8510 Acked Write 3 Done Sent Done false
27251 8511 Acked Write 3 Done Sent Done false
27252 8512 Acked Write 9 Done Sent Done false
27253 8513 Acked Read 10 Done Sent Done false
27254 8514 Acked Flush 0 Done Sent Done false
27255 8515 Acked Read 4 Done Sent Done false
27256 8516 Acked Write 2 Done Sent Done false
27257 8517 Acked Read 5 Done Sent Done false
27258 8518 Acked Write 1 Done Sent Done false
27259 8519 Acked Write 7 Done Sent Done false
27260 8520 Acked Read 7 Done Sent Done false
27261 8521 Acked Write 1 Done Sent Done false
27262 8522 Acked Read 6 Done Sent Done false
27263 8523 Acked Write 2 Done Sent Done false
27264 8524 Acked Write 1 Done Sent Done false
27265 8525 Acked Write 9 Done Sent Done false
27266 8526 Acked Read 8 Done Sent Done false
27267 8527 Acked Flush 0 Done Sent Done false
27268 8528 Acked Write 4 Done Sent Done false
27269 8529 Acked Write 4 Done Sent Done false
27270 8530 Acked Read 1 Done Sent Done false
27271 8531 Acked Flush 0 Done Sent Done false
27272 8532 Acked Write 6 Done Sent Done false
27273 8533 Acked Read 1 Done Sent Done false
27274 8534 Acked Write 6 Done Sent Done false
27275 8535 Acked Write 1 Done Sent Done false
27276 8536 Acked Read 4 Done Sent Done false
27277 8537 Acked Read 2 Done Sent Done false
27278 8538 Acked Write 3 Done Sent Done false
27279 8539 Acked Read 10 Done Sent Done false
27280 8540 Acked Read 9 Done Sent Done false
27281 8541 Acked Write 1 Done Sent Done false
27282 8542 Acked Flush 0 Done Sent Done false
27283 8543 Acked Write 10 Done Sent Done false
27284 8544 Acked Read 5 Done Sent Done false
27285 8545 Acked Flush 0 Done Sent Done false
27286 8546 Acked Write 9 Sent Sent Sent false
27287 STATES DS:0 DS:1 DS:2 TOTAL
27288 Sent 1 301 1 303
27289 Done 300 0 300 600
27290 Skipped 0 0 0 0
27291 Error 0 0 0 0
27292 Last Flush: 8545 8245 8545
27293 Downstairs last five completed: 8245 8244 8243 8242 8241
27294 Upstairs last five completed: 8546 8545 8544 8543 8542
27295 CLIENT: Up:0 ds:301 act:2
27296 [8820] got start action
27297 [8820] I am going to restart
27298 Starting downstairs at port 8820
27299 Make output file at "/tmp/dsc/downstairs-8820.txt"
27300 [8820][1] reports Starting
27301 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1227)
27302 [8820][1] reports Running
27303 ----------------------------------------------------------------
27304 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27305 States: Active Offline Active
27306 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27307 8246 Acked Read 3 Done Sent Done false
27308 8247 Acked Flush 0 Done Sent Done false
27309 8248 Acked Flush 0 Done Sent Done false
27310 8249 Acked Read 9 Done Sent Done false
27311 8250 Acked Read 3 Done Sent Done false
27312 8251 Acked Read 6 Done Sent Done false
27313 8252 Acked Flush 0 Done Sent Done false
27314 8253 Acked Read 7 Done Sent Done false
27315 8254 Acked Read 3 Done Sent Done false
27316 8255 Acked Write 4 Done Sent Done false
27317 8256 Acked Write 2 Done Sent Done false
27318 8257 Acked Read 10 Done Sent Done false
27319 8258 Acked Read 7 Done Sent Done false
27320 8259 Acked Read 8 Done Sent Done false
27321 8260 Acked Read 6 Done Sent Done false
27322 8261 Acked Write 10 Done Sent Done false
27323 8262 Acked Read 3 Done Sent Done false
27324 8263 Acked Read 10 Done Sent Done false
27325 8264 Acked Flush 0 Done Sent Done false
27326 8265 Acked Write 2 Done Sent Done false
27327 8266 Acked Flush 0 Done Sent Done false
27328 8267 Acked Write 3 Done Sent Done false
27329 8268 Acked Write 2 Done Sent Done false
27330 8269 Acked Write 4 Done Sent Done false
27331 8270 Acked Write 9 Done Sent Done false
27332 8271 Acked Read 6 Done Sent Done false
27333 8272 Acked Read 3 Done Sent Done false
27334 8273 Acked Read 9 Done Sent Done false
27335 8274 Acked Write 6 Done Sent Done false
27336 8275 Acked Write 3 Done Sent Done false
27337 8276 Acked Write 3 Done Sent Done false
27338 8277 Acked Read 10 Done Sent Done false
27339 8278 Acked Read 5 Done Sent Done false
27340 8279 Acked Write 9 Done Sent Done false
27341 8280 Acked Read 7 Done Sent Done false
27342 8281 Acked Read 5 Done Sent Done false
27343 8282 Acked Write 5 Done Sent Done false
27344 8283 Acked Read 3 Done Sent Done false
27345 8284 Acked Read 6 Done Sent Done false
27346 8285 Acked Read 6 Done Sent Done false
27347 8286 Acked Read 10 Done Sent Done false
27348 8287 Acked Write 6 Done Sent Done false
27349 8288 Acked Read 5 Done Sent Done false
27350 8289 Acked Write 8 Done Sent Done false
27351 8290 Acked Write 4 Done Sent Done false
27352 8291 Acked Read 5 Done Sent Done false
27353 8292 Acked Write 2 Done Sent Done false
27354 8293 Acked Write 7 Done Sent Done false
27355 8294 Acked Flush 0 Done Sent Done false
27356 8295 Acked Read 1 Done Sent Done false
27357 8296 Acked Read 9 Done Sent Done false
27358 8297 Acked Read 10 Done Sent Done false
27359 8298 Acked Read 5 Done Sent Done false
27360 8299 Acked Read 8 Done Sent Done false
27361 8300 Acked Write 4 Done Sent Done false
27362 8301 Acked Write 8 Done Sent Done false
27363 8302 Acked Write 7 Done Sent Done false
27364 8303 Acked Write 5 Done Sent Done false
27365 8304 Acked Write 3 Done Sent Done false
27366 8305 Acked Read 3 Done Sent Done false
27367 8306 Acked Read 6 Done Sent Done false
27368 8307 Acked Read 7 Done Sent Done false
27369 8308 Acked Read 8 Done Sent Done false
27370 8309 Acked Read 4 Done Sent Done false
27371 8310 Acked Read 3 Done Sent Done false
27372 8311 Acked Write 6 Done Sent Done false
27373 8312 Acked Read 5 Done Sent Done false
27374 8313 Acked Write 1 Done Sent Done false
27375 8314 Acked Read 9 Done Sent Done false
27376 8315 Acked Read 5 Done Sent Done false
27377 8316 Acked Read 2 Done Sent Done false
27378 8317 Acked Read 9 Done Sent Done false
27379 8318 Acked Write 6 Done Sent Done false
27380 8319 Acked Read 3 Done Sent Done false
27381 8320 Acked Write 2 Done Sent Done false
27382 8321 Acked Write 1 Done Sent Done false
27383 8322 Acked Read 3 Done Sent Done false
27384 8323 Acked Write 4 Done Sent Done false
27385 8324 Acked Read 8 Done Sent Done false
27386 8325 Acked Write 9 Done Sent Done false
27387 8326 Acked Read 7 Done Sent Done false
27388 8327 Acked Read 7 Done Sent Done false
27389 8328 Acked Write 5 Done Sent Done false
27390 8329 Acked Write 4 Done Sent Done false
27391 8330 Acked Read 6 Done Sent Done false
27392 8331 Acked Flush 0 Done Sent Done false
27393 8332 Acked Read 5 Done Sent Done false
27394 8333 Acked Read 4 Done Sent Done false
27395 8334 Acked Read 10 Done Sent Done false
27396 8335 Acked Write 8 Done Sent Done false
27397 8336 Acked Read 4 Done Sent Done false
27398 8337 Acked Read 6 Done Sent Done false
27399 8338 Acked Read 2 Done Sent Done false
27400 8339 Acked Write 6 Done Sent Done false
27401 8340 Acked Write 5 Done Sent Done false
27402 8341 Acked Write 2 Done Sent Done false
27403 8342 Acked Read 4 Done Sent Done false
27404 8343 Acked Read 8 Done Sent Done false
27405 8344 Acked Read 10 Done Sent Done false
27406 8345 Acked Write 4 Done Sent Done false
27407 8346 Acked Write 5 Done Sent Done false
27408 8347 Acked Read 2 Done Sent Done false
27409 8348 Acked Read 4 Done Sent Done false
27410 8349 Acked Flush 0 Done Sent Done false
27411 8350 Acked Write 3 Done Sent Done false
27412 8351 Acked Write 3 Done Sent Done false
27413 8352 Acked Read 9 Done Sent Done false
27414 8353 Acked Read 6 Done Sent Done false
27415 8354 Acked Read 3 Done Sent Done false
27416 8355 Acked Read 1 Done Sent Done false
27417 8356 Acked Read 7 Done Sent Done false
27418 8357 Acked Read 5 Done Sent Done false
27419 8358 Acked Read 6 Done Sent Done false
27420 8359 Acked Flush 0 Done Sent Done false
27421 8360 Acked Write 1 Done Sent Done false
27422 8361 Acked Write 8 Done Sent Done false
27423 8362 Acked Write 1 Done Sent Done false
27424 8363 Acked Write 9 Done Sent Done false
27425 8364 Acked Write 4 Done Sent Done false
27426 8365 Acked Write 3 Done Sent Done false
27427 8366 Acked Read 2 Done Sent Done false
27428 8367 Acked Write 7 Done Sent Done false
27429 8368 Acked Read 8 Done Sent Done false
27430 8369 Acked Read 4 Done Sent Done false
27431 8370 Acked Read 3 Done Sent Done false
27432 8371 Acked Read 3 Done Sent Done false
27433 8372 Acked Flush 0 Done Sent Done false
27434 8373 Acked Read 1 Done Sent Done false
27435 8374 Acked Read 9 Done Sent Done false
27436 8375 Acked Read 9 Done Sent Done false
27437 8376 Acked Read 7 Done Sent Done false
27438 8377 Acked Read 4 Done Sent Done false
27439 8378 Acked Write 6 Done Sent Done false
27440 8379 Acked Read 1 Done Sent Done false
27441 8380 Acked Read 2 Done Sent Done false
27442 8381 Acked Write 9 Done Sent Done false
27443 8382 Acked Read 7 Done Sent Done false
27444 8383 Acked Read 4 Done Sent Done false
27445 8384 Acked Write 1 Done Sent Done false
27446 8385 Acked Read 1 Done Sent Done false
27447 8386 Acked Read 10 Done Sent Done false
27448 8387 Acked Read 1 Done Sent Done false
27449 8388 Acked Read 2 Done Sent Done false
27450 8389 Acked Read 2 Done Sent Done false
27451 8390 Acked Write 10 Done Sent Done false
27452 8391 Acked Read 2 Done Sent Done false
27453 8392 Acked Write 5 Done Sent Done false
27454 8393 Acked Read 9 Done Sent Done false
27455 8394 Acked Read 1 Done Sent Done false
27456 8395 Acked Flush 0 Done Sent Done false
27457 8396 Acked Read 2 Done Sent Done false
27458 8397 Acked Flush 0 Done Sent Done false
27459 8398 Acked Write 7 Done Sent Done false
27460 8399 Acked Write 7 Done Sent Done false
27461 8400 Acked Write 7 Done Sent Done false
27462 8401 Acked Write 1 Done Sent Done false
27463 8402 Acked Read 6 Done Sent Done false
27464 8403 Acked Write 10 Done Sent Done false
27465 8404 Acked Read 9 Done Sent Done false
27466 8405 Acked Read 5 Done Sent Done false
27467 8406 Acked Write 3 Done Sent Done false
27468 8407 Acked Write 9 Done Sent Done false
27469 8408 Acked Read 3 Done Sent Done false
27470 8409 Acked Read 5 Done Sent Done false
27471 8410 Acked Read 5 Done Sent Done false
27472 8411 Acked Read 10 Done Sent Done false
27473 8412 Acked Write 10 Done Sent Done false
27474 8413 Acked Flush 0 Done Sent Done false
27475 8414 Acked Read 8 Done Sent Done false
27476 8415 Acked Write 8 Done Sent Done false
27477 8416 Acked Read 9 Done Sent Done false
27478 8417 Acked Write 10 Done Sent Done false
27479 8418 Acked Read 2 Done Sent Done false
27480 8419 Acked Write 3 Done Sent Done false
27481 8420 Acked Read 5 Done Sent Done false
27482 8421 Acked Read 4 Done Sent Done false
27483 8422 Acked Flush 0 Done Sent Done false
27484 8423 Acked Write 10 Done Sent Done false
27485 8424 Acked Read 9 Done Sent Done false
27486 8425 Acked Read 10 Done Sent Done false
27487 8426 Acked Read 3 Done Sent Done false
27488 8427 Acked Flush 0 Done Sent Done false
27489 8428 Acked Write 2 Done Sent Done false
27490 8429 Acked Flush 0 Done Sent Done false
27491 8430 Acked Write 9 Done Sent Done false
27492 8431 Acked Read 10 Done Sent Done false
27493 8432 Acked Read 4 Done Sent Done false
27494 8433 Acked Flush 0 Done Sent Done false
27495 8434 Acked Read 3 Done Sent Done false
27496 8435 Acked Write 10 Done Sent Done false
27497 8436 Acked Read 3 Done Sent Done false
27498 8437 Acked Read 4 Done Sent Done false
27499 8438 Acked Read 6 Done Sent Done false
27500 8439 Acked Write 7 Done Sent Done false
27501 8440 Acked Write 8 Done Sent Done false
27502 8441 Acked Read 9 Done Sent Done false
27503 8442 Acked Write 2 Done Sent Done false
27504 8443 Acked Read 2 Done Sent Done false
27505 8444 Acked Write 4 Done Sent Done false
27506 8445 Acked Write 4 Done Sent Done false
27507 8446 Acked Read 3 Done Sent Done false
27508 8447 Acked Write 9 Done Sent Done false
27509 8448 Acked Flush 0 Done Sent Done false
27510 8449 Acked Read 7 Done Sent Done false
27511 8450 Acked Write 2 Done Sent Done false
27512 8451 Acked Read 6 Done Sent Done false
27513 8452 Acked Write 7 Done Sent Done false
27514 8453 Acked Write 7 Done Sent Done false
27515 8454 Acked Write 2 Done Sent Done false
27516 8455 Acked Write 3 Done Sent Done false
27517 8456 Acked Write 1 Done Sent Done false
27518 8457 Acked Write 4 Done Sent Done false
27519 8458 Acked Read 5 Done Sent Done false
27520 8459 Acked Write 6 Done Sent Done false
27521 8460 Acked Write 6 Done Sent Done false
27522 8461 Acked Write 7 Done Sent Done false
27523 8462 Acked Read 3 Done Sent Done false
27524 8463 Acked Write 9 Done Sent Done false
27525 8464 Acked Read 10 Done Sent Done false
27526 8465 Acked Write 2 Done Sent Done false
27527 8466 Acked Write 7 Done Sent Done false
27528 8467 Acked Read 7 Done Sent Done false
27529 8468 Acked Write 9 Done Sent Done false
27530 8469 Acked Read 2 Done Sent Done false
27531 8470 Acked Write 3 Done Sent Done false
27532 8471 Acked Read 3 Done Sent Done false
27533 8472 Acked Flush 0 Done Sent Done false
27534 8473 Acked Read 5 Done Sent Done false
27535 8474 Acked Write 7 Done Sent Done false
27536 8475 Acked Write 8 Done Sent Done false
27537 8476 Acked Read 7 Done Sent Done false
27538 8477 Acked Read 5 Done Sent Done false
27539 8478 Acked Read 7 Done Sent Done false
27540 8479 Acked Write 5 Done Sent Done false
27541 8480 Acked Write 8 Done Sent Done false
27542 8481 Acked Write 7 Done Sent Done false
27543 8482 Acked Write 4 Done Sent Done false
27544 8483 Acked Read 9 Done Sent Done false
27545 8484 Acked Write 7 Done Sent Done false
27546 8485 Acked Read 4 Done Sent Done false
27547 8486 Acked Write 3 Done Sent Done false
27548 8487 Acked Read 7 Done Sent Done false
27549 8488 Acked Read 10 Done Sent Done false
27550 8489 Acked Write 9 Done Sent Done false
27551 8490 Acked Flush 0 Done Sent Done false
27552 8491 Acked Write 3 Done Sent Done false
27553 8492 Acked Write 4 Done Sent Done false
27554 8493 Acked Write 7 Done Sent Done false
27555 8494 Acked Write 4 Done Sent Done false
27556 8495 Acked Write 6 Done Sent Done false
27557 8496 Acked Read 7 Done Sent Done false
27558 8497 Acked Read 6 Done Sent Done false
27559 8498 Acked Read 7 Done Sent Done false
27560 8499 Acked Read 5 Done Sent Done false
27561 8500 Acked Flush 0 Done Sent Done false
27562 8501 Acked Read 8 Done Sent Done false
27563 8502 Acked Flush 0 Done Sent Done false
27564 8503 Acked Flush 0 Done Sent Done false
27565 8504 Acked Flush 0 Done Sent Done false
27566 8505 Acked Read 1 Done Sent Done false
27567 8506 Acked Write 8 Done Sent Done false
27568 8507 Acked Write 1 Done Sent Done false
27569 8508 Acked Read 1 Done Sent Done false
27570 8509 Acked Write 7 Done Sent Done false
27571 8510 Acked Write 3 Done Sent Done false
27572 8511 Acked Write 3 Done Sent Done false
27573 8512 Acked Write 9 Done Sent Done false
27574 8513 Acked Read 10 Done Sent Done false
27575 8514 Acked Flush 0 Done Sent Done false
27576 8515 Acked Read 4 Done Sent Done false
27577 8516 Acked Write 2 Done Sent Done false
27578 8517 Acked Read 5 Done Sent Done false
27579 8518 Acked Write 1 Done Sent Done false
27580 8519 Acked Write 7 Done Sent Done false
27581 8520 Acked Read 7 Done Sent Done false
27582 8521 Acked Write 1 Done Sent Done false
27583 8522 Acked Read 6 Done Sent Done false
27584 8523 Acked Write 2 Done Sent Done false
27585 8524 Acked Write 1 Done Sent Done false
27586 8525 Acked Write 9 Done Sent Done false
27587 8526 Acked Read 8 Done Sent Done false
27588 8527 Acked Flush 0 Done Sent Done false
27589 8528 Acked Write 4 Done Sent Done false
27590 8529 Acked Write 4 Done Sent Done false
27591 8530 Acked Read 1 Done Sent Done false
27592 8531 Acked Flush 0 Done Sent Done false
27593 8532 Acked Write 6 Done Sent Done false
27594 8533 Acked Read 1 Done Sent Done false
27595 8534 Acked Write 6 Done Sent Done false
27596 8535 Acked Write 1 Done Sent Done false
27597 8536 Acked Read 4 Done Sent Done false
27598 8537 Acked Read 2 Done Sent Done false
27599 8538 Acked Write 3 Done Sent Done false
27600 8539 Acked Read 10 Done Sent Done false
27601 8540 Acked Read 9 Done Sent Done false
27602 8541 Acked Write 1 Done Sent Done false
27603 8542 Acked Flush 0 Done Sent Done false
27604 8543 Acked Write 10 Done Sent Done false
27605 8544 Acked Read 5 Done Sent Done false
27606 8545 Acked Flush 0 Done Sent Done false
27607 8546 Acked Write 9 Done Sent Done false
27608 8547 Acked Flush 0 Done Sent Done false
27609 STATES DS:0 DS:1 DS:2 TOTAL
27610 Sent 0 302 0 302
27611 Done 302 0 302 604
27612 Skipped 0 0 0 0
27613 Error 0 0 0 0
27614 Last Flush: 8547 8245 8547
27615 Downstairs last five completed: 8245 8244 8243 8242 8241
27616 Upstairs last five completed: 8547 8546 8545 8544 8543
27617 CLIENT: Up:0 ds:302 act:2
276182025-01-13T15:57:35.073ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
276192025-01-13T15:57:35.073ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54261) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
276202025-01-13T15:57:35.074ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
276212025-01-13T15:57:35.075ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
276222025-01-13T15:57:35.075ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
276232025-01-13T15:57:35.075ZINFOcrucible: send last flush ID to this DS: 8245 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
276242025-01-13T15:57:35.075ZINFOcrucible: Replied this last flush ID: 8245 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
276252025-01-13T15:57:35.075ZINFOcrucible: [1] client re-new 302 jobs since flush 8245 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
276262025-01-13T15:57:35.076ZINFOcrucible: [1] Replayed 302 jobs since flush: 8245 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
27627 ----------------------------------------------------------------
27628 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
27629 Downstairs last five completed: 8547 8546 8545 8544 8543
27630 Upstairs last five completed: 8547 8546 8545 8544 8543
27631 CLIENT: Up:0 ds:0 act:3
27632 Replay: All jobs finished
27633 Main task has work to do, go find it
27634 got dsc Stop(0)
27635 stop 0
27636 [8810] Got stop action so:false kr:false
27637 Jan 13 15:57:37.314 INFO request completed, latency_us: 360, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 5e0dadd8-429b-4bf6-9da8-a97e74c20612, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
276382025-01-13T15:57:37.314ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
276392025-01-13T15:57:37.314ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
276402025-01-13T15:57:37.314ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
27641 Jan 13 15:57:37.314 INFO request completed, latency_us{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-13T15:57:37.314957141Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
27642 : 136, response_code: 200, uri: /state/cid/0, method: GET, req_id: 8615a14c-43e5-4bea-b456-2fe5cd07c65a, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
27643 [8810] Exited with: ExitStatus(unix_wait_status(9))
27644 [8810][0] reports Exit
27645 Jan 13 15:57:41.317 INFO request completed, latency_us: 340, response_code: 200, uri: /state/cid/0, method: GET, req_id: 78306fd5-c8da-48b9-9ea7-4421befa724f, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
27646 001/300 Write block 654 len 40960 data: 4 4 4 5 7 6 6 7 8 8
27647 002/300 Read block 3069 len 24576
27648 003/300 Write block 4051 len 24576 data: 5 8 7 7 7 8
27649 004/300 Read block 2338 len 36864
27650 005/300 Flush
27651 006/300 Write block 1838 len 40960 data: 7 7 7 6 8 7 6 5 4 4
27652 007/300 Read block 2733 len 36864
27653 008/300 Read block 1170 len 24576
27654 009/300 Flush
27655 010/300 Write block 689 len 24576 data: 5 4 4 4 4 5
27656 011/300 Write block 4492 len 24576 data: 2 2 2 3 4 7
27657 012/300 Read block 130 len 20480
27658 013/300 Read block 875 len 28672
27659 014/300 Read block 2055 len 36864
27660 015/300 Read block 989 len 8192
27661 016/300 Write block 4260 len 8192 data: 2 3
27662 017/300 Write block 1267 len 4096 data: 7
27663 018/300 Read block 1591 len 16384
27664 019/300 Write block 4403 len 40960 data: 4 6 5 7 7 7 5 5 5 5
27665 020/300 Write block 3690 len 8192 data: 3 3
27666 021/300 Read block 2032 len 40960
27667 022/300 Read block 1581 len 24576
27668 023/300 Read block 4580 len 24576
27669 024/300 Read block 3776 len 8192
27670 025/300 Flush
27671 026/300 Read block 3963 len 20480
27672 027/300 Write block 1281 len 20480 data: 5 4 4 4 3
27673 028/300 Flush
27674 029/300 Read block 4691 len 16384
27675 030/300 Read block 447 len 24576
27676 031/300 Flush
27677 032/300 Read block 654 len 28672
27678 033/300 Read block 3898 len 36864
27679 034/300 Read block 1117 len 16384
27680 035/300 Read block 1708 len 24576
27681 036/300 Read block 3393 len 40960
27682 037/300 Write block 3869 len 28672 data: 6 5 6 6 7 6 5
27683 038/300 Read block 778 len 4096
27684 039/300 Read block 3790 len 40960
27685 040/300 Flush
27686 041/300 Read block 1684 len 40960
27687 042/300 Read block 4850 len 12288
27688 043/300 Write block 4962 len 32768 data: 5 4 4 4 5 6 7 6
27689 044/300 Read block 4711 len 4096
27690 045/300 Write block 3306 len 28672 data: 5 5 5 4 4 5 5
27691 046/300 Read block 584 len 16384
27692 047/300 Flush
27693 048/300 Write block 863 len 32768 data: 7 7 6 7 7 7 8 7
27694 049/300 Read block 3973 len 40960
27695 050/300 Write block 2024 len 36864 data: 11 10 11 11 10 8 7 6 5
27696 051/300 Flush
27697 052/300 Read block 2134 len 16384
27698 053/300 Write block 671 len 8192 data: 6 4
27699 054/300 Write block 2273 len 20480 data: 6 7 7 7 7
27700 055/300 Read block 1325 len 4096
27701 056/300 Read block 914 len 36864
27702 057/300 Write block 1703 len 32768 data: 3 3 6 5 4 2 2 2
27703 058/300 Write block 1959 len 4096 data: 6
27704 059/300 Read block 3882 len 8192
27705 060/300 Read block 2647 len 16384
27706 061/300 Flush
27707 062/300 Write block 2369 len 16384 data: 4 4 3 4
27708 063/300 Flush
27709 064/300 Write block 2591 len 8192 data: 9 9
27710 065/300 Read block 3138 len 28672
27711 066/300 Write block 3719 len 24576 data: 4 3 6 8 8 7
27712 067/300 Read block 2829 len 40960
27713 068/300 Read block 2638 len 20480
27714 069/300 Write block 1455 len 28672 data: 5 5 3 3 3 3 3
27715 070/300 Write block 4628 len 8192 data: 5 4
27716 071/300 Flush
27717 072/300 Read block 2258 len 8192
27718 073/300 Write block 2307 len 32768 data: 3 5 6 7 6 4 4 5
27719 074/300 Read block 4096 len 36864
27720 075/300 Write block 3785 len 4096 data: 5
27721 076/300 Read block 3531 len 4096
27722 077/300 Read block 4340 len 16384
27723 078/300 Write block 2454 len 8192 data: 3 3
27724 079/300 Read block 821 len 4096
27725 080/300 Write block 4150 len 28672 data: 4 3 4 2 2 2 3
27726 081/300 Write block 1743 len 12288 data: 8 8 10
27727 082/300 Read block 1027 len 40960
27728 083/300 Write block 2912 len 4096 data: 5
27729 084/300 Write block 2993 len 4096 data: 6
27730 085/300 Flush
27731 086/300 Write block 380 len 8192 data: 7 7
27732 087/300 Flush
27733 088/300 Read block 1251 len 8192
27734 089/300 Read block 2137 len 24576
27735 090/300 Write block 1861 len 12288 data: 4 5 4
27736 091/300 Write block 1454 len 24576 data: 6 6 6 4 4 4
27737 092/300 Flush
27738 093/300 Flush
27739 094/300 Read block 3028 len 24576
27740 095/300 Read block 4651 len 32768
27741 096/300 Write block 14 len 24576 data: 3 3 3 3 4 3
27742 097/300 Read block 1465 len 28672
27743 098/300 Read block 1053 len 24576
27744 099/300 Write block 3558 len 8192 data: 6 6
27745 100/300 Read block 599 len 24576
27746 101/300 Write block 721 len 32768 data: 4 5 6 3 4 5 6 6
27747 102/300 Flush
27748 103/300 Write block 4836 len 4096 data: 7
27749 104/300 Write block 4065 len 16384 data: 4 4 3 3
27750 105/300 Read block 2987 len 20480
27751 106/300 Write block 1536 len 8192 data: 2 2
27752 107/300 Read block 691 len 32768
27753 108/300 Write block 2140 len 20480 data: 5 6 6 6 3
27754 109/300 Read block 4030 len 16384
27755 110/300 Write block 2696 len 40960 data: 3 7 9 9 8 8 8 8 7 7
27756 111/300 Write block 1991 len 28672 data: 4 4 5 6 5 5 6
27757 112/300 Flush
27758 113/300 Write block 4579 len 20480 data: 6 4 6 6 5
27759 114/300 Read block 2238 len 28672
27760 115/300 Write block 3247 len 24576 data: 6 6 6 5 4 3
27761 116/300 Read block 1372 len 28672
27762 117/300 Read block 4405 len 20480
27763 118/300 Write block 453 len 20480 data: 5 6 4 4 4
27764 119/300 Write block 1213 len 20480 data: 3 4 2 2 3
27765 120/300 Read block 1549 len 32768
27766 121/300 Write block 3178 len 8192 data: 6 5
27767 122/300 Read block 2432 len 32768
27768 123/300 Write block 2017 len 32768 data: 4 5 7 6 6 9 10 12
27769 124/300 Write block 1915 len 24576 data: 5 5 5 5 5 7
27770 125/300 Write block 1654 len 16384 data: 3 2 2 3
27771 126/300 Read block 3201 len 16384
27772 127/300 Read block 1975 len 4096
27773 128/300 Flush
27774 129/300 Write block 3157 len 24576 data: 3 3 4 4 4 5
27775 130/300 Write block 2722 len 32768 data: 4 5 7 5 6 6 5 4
27776 131/300 Write block 325 len 8192 data: 4 4
27777 132/300 Write block 1138 len 16384 data: 2 5 5 7
27778 133/300 Read block 2404 len 4096
27779 134/300 Read block 446 len 16384
27780 135/300 Write block 215 len 24576 data: 8 7 7 7 5 5
27781 136/300 Read block 1510 len 40960
27782 137/300 Read block 2358 len 32768
27783 138/300 Read block 1427 len 20480
27784 139/300 Read block 290 len 8192
27785 140/300 Read block 2152 len 4096
27786 141/300 Write block 1876 len 40960 data: 3 3 2 2 2 3 3 3 2 2
27787 142/300 Read block 506 len 28672
27788 143/300 Flush
27789 144/300 Read block 783 len 8192
27790 145/300 Read block 503 len 40960
27791 146/300 Flush
27792 147/300 Write block 916 len 4096 data: 3
27793 148/300 Write block 1443 len 28672 data: 6 5 3 3 3 5 5
27794 149/300 Read block 2942 len 12288
27795 150/300 Write block 480 len 36864 data: 7 7 5 5 6 6 6 5 2
27796 151/300 Write block 2795 len 36864 data: 4 4 5 6 5 6 4 5 6
27797 152/300 Read block 4473 len 32768
27798 153/300 Write block 1746 len 40960 data: 7 6 4 6 6 5 5 5 4 2
27799 154/300 Flush
27800 155/300 Write block 332 len 16384 data: 4 4 4 4
27801 156/300 Write block 4637 len 12288 data: 4 4 5
27802 157/300 Write block 2662 len 12288 data: 6 6 6
27803 158/300 Read block 652 len 40960
27804 159/300 Read block 3807 len 28672
27805 160/300 Write block 4139 len 36864 data: 3 3 3 2 2 2 3 4 4
27806 161/300 Read block 1246 len 12288
27807 162/300 Read block 1780 len 36864
27808 163/300 Read block 3702 len 28672
27809 164/300 Write block 3594 len 32768 data: 5 5 5 4 5 5 5 6
27810 165/300 Read block 1708 len 4096
27811 166/300 Write block 891 len 16384 data: 5 4 3 3
27812 167/300 Read block 4325 len 12288
27813 168/300 Read block 3959 len 32768
27814 169/300 Write block 4166 len 8192 data: 4 4
27815 170/300 Flush
27816 171/300 Flush
27817 172/300 Flush
27818 173/300 Write block 424 len 4096 data: 4
27819 174/300 Read block 1526 len 36864
27820 175/300 Read block 2552 len 32768
27821 176/300 Write block 3918 len 4096 data: 5
27822 177/300 Read block 1608 len 24576
27823 178/300 Read block 3838 len 20480
27824 179/300 Read block 4613 len 12288
27825 180/300 Flush
27826 181/300 Read block 592 len 28672
27827 182/300 Read block 561 len 28672
27828 183/300 Read block 3581 len 20480
27829 184/300 Read block 1671 len 8192
27830 185/300 Read block 4137 len 24576
27831 186/300 Flush
27832 187/300 Read block 3611 len 20480
27833 188/300 Write block 3906 len 36864 data: 4 4 4 3 3 4 3 5 4
27834 189/300 Read block 207 len 24576
27835 190/300 Write block 3064 len 32768 data: 5 5 5 6 5 5 6 5
27836 191/300 Write block 3345 len 36864 data: 4 4 5 7 8 7 8 6 6
27837 192/300 Flush
27838 193/300 Read block 2395 len 24576
27839 194/300 Read block 4017 len 40960
27840 195/300 Write block 1946 len 20480 data: 6 6 6 5 6
27841 196/300 Read block 3241 len 16384
27842 197/300 Write block 4127 len 36864 data: 5 5 4 3 3 3 3 3 3
27843 198/300 Write block 2323 len 24576 data: 4 4 2 2 2 3
27844 199/300 Read block 3041 len 16384
27845 200/300 Read block 1446 len 16384
27846 201/300 Write block 1463 len 8192 data: 3 4
27847 202/300 Write block 2703 len 4096 data: 9
27848 203/300 Read block 652 len 8192
27849 204/300 Read block 2271 len 16384
27850 205/300 Read block 4891 len 20480
27851 206/300 Read block 4012 len 8192
27852 207/300 Read block 1695 len 4096
27853 208/300 Read block 1440 len 36864
27854 209/300 Write block 3348 len 40960 data: 8 9 8 9 7 7 6 7 8 7
27855 210/300 Read block 68 len 12288
27856 211/300 Write block 4983 len 16384 data: 4 3 3 4
27857 212/300 Read block 3885 len 20480
27858 213/300 Write block 2019 len 24576 data: 8 7 7 10 11 13
27859 214/300 Write block 2443 len 36864 data: 6 6 4 4 4 5 4 4 4
27860 215/300 Read block 572 len 4096
27861 216/300 Read block 2644 len 12288
27862 217/300 Read block 3210 len 28672
27863 218/300 Read block 2090 len 40960
27864 219/300 Write block 1207 len 36864 data: 5 4 4 4 3 4 4 5 3
27865 220/300 Write block 2857 len 40960 data: 3 3 3 3 3 3 4 5 8 7
27866 221/300 Write block 1957 len 36864 data: 5 5 7 7 7 5 5 4 5
27867 222/300 Read block 2495 len 28672
27868 223/300 Read block 4456 len 20480
27869 224/300 Write block 2082 len 8192 data: 8 10
27870 225/300 Read block 4542 len 28672
27871 226/300 Write block 4734 len 4096 data: 10
27872 227/300 Read block 4439 len 4096
27873 228/300 Read block 4826 len 4096
27874 229/300 Read block 1961 len 24576
27875 230/300 Write block 2760 len 32768 data: 2 3 4 6 5 4 5 6
27876 231/300 Flush
27877 232/300 Read block 3152 len 32768
27878 233/300 Write block 1722 len 24576 data: 3 4 4 4 5 5
27879 234/300 Write block 3959 len 32768 data: 6 4 6 6 7 6 6 5
27880 235/300 Write block 2874 len 36864 data: 6 7 6 5 5 7 8 8 7
27881 236/300 Write block 171 len 40960 data: 6 8 8 7 6 6 4 3 5 4
27882 237/300 Write block 1849 len 8192 data: 5 6
27883 238/300 Write block 4401 len 32768 data: 4 5 5 7 6 8 8 8
27884 239/300 Read block 3605 len 12288
27885 240/300 Write block 1900 len 28672 data: 5 5 3 3 3 3 4
27886 241/300 Read block 1912 len 24576
27887 242/300 Read block 3818 len 24576
27888 243/300 Read block 4557 len 28672
27889 244/300 Read block 273 len 24576
27890 245/300 Read block 3900 len 32768
27891 246/300 Read block 2973 len 16384
27892 247/300 Write block 1018 len 40960 data: 6 6 8 8 9 7 8 7 9 8
27893 248/300 Write block 3406 len 16384 data: 5 6 7 8
27894 249/300 Write block 4137 len 4096 data: 2
27895 250/300 Read block 345 len 28672
27896 251/300 Read block 2990 len 40960
27897 252/300 Read block 1500 len 32768
27898 253/300 Read block 1832 len 24576
27899 254/300 Read block 3173 len 28672
27900 255/300 Read block 735 len 32768
27901 256/300 Write block 2161 len 16384 data: 5 6 6 5
27902 257/300 Write block 3646 len 24576 data: 6 5 2 2 2 2
27903 258/300 Read block 2464 len 24576
27904 259/300 Read block 1532 len 20480
27905 260/300 Write block 608 len 32768 data: 5 4 5 4 4 5 4 5
27906 261/300 Write block 2125 len 4096 data: 5
27907 262/300 Flush
27908 263/300 Read block 2563 len 20480
27909 264/300 Write block 4349 len 16384 data: 4 6 6 5
27910 265/300 Read block 948 len 40960
27911 266/300 Read block 3714 len 16384
27912 267/300 Write block 4447 len 8192 data: 6 9
27913 268/300 Read block 4439 len 12288
27914 269/300 Write block 833 len 4096 data: 8
27915 270/300 Write block 4593 len 4096 data: 5
27916 271/300 Read block 4428 len 28672
27917 272/300 Write block 4745 len 12288 data: 4 4 5
27918 273/300 Write block 4069 len 12288 data: 4 6 6
27919 274/300 Read block 607 len 32768
27920 275/300 Read block 1196 len 16384
27921 276/300 Flush
27922 277/300 Read block 142 len 12288
27923 278/300 Write block 3886 len 8192 data: 5 6
27924 279/300 Write block 1276 len 32768 data: 5 6 7 5 4 6 5 5
27925 280/300 Read block 897 len 36864
27926 281/300 Read block 1825 len 40960
27927 282/300 Read block 390 len 16384
27928 283/300 Read block 997 len 28672
27929 284/300 Read block 4343 len 20480
27930 285/300 Read block 3538 len 32768
27931 286/300 Read block 4781 len 40960
27932 287/300 Read block 3633 len 40960
27933 288/300 Write block 3868 len 12288 data: 6 7 6
27934 289/300 Write block 2140 len 4096 data: 6
27935 290/300 Flush
27936 291/300 Write block 2110 len 32768 data: 8 7 6 3 3 2 3 5
27937 292/300 Write block 4221 len 36864 data: 7 5 7 7 7 5 5 5 6
27938 293/300 Write block 2980 len 28672 data: 3 4 4 4 4 3 3
27939 294/300 Write block 2558 len 12288 data: 7 6 6
27940 295/300 Read block 3162 len 16384
27941 296/300 Read block 4020 len 36864
27942 297/300 Read block 2950 len 16384
27943 298/300 Read block 52 len 28672
27944 299/300 Write block 1861 len 28672 data: 5 6 5 4 5 5 4
27945 300/300 Write block 1412 len 16384 data: 8 7 7 8
27946 301/300 Write block 1210 len 36864 data: 5 4 5 5 6 4 3 4 4
27947 Main task has work to do, go find it
27948 got dsc Start(0)
27949 start 0
27950 Jan 13 15:57:41.559 INFO request completed, latency_us: 187, response_code: 204, uri: /start/cid/0, method: POST, req_id: bc6894c5-5c70-48ff-9275-5d40ed6a0a98, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
27951 Replay: started 0, returned:Ok(())
27952 ----------------------------------------------------------------
27953 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
27954 States: Offline Active Active
27955 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27956 8548 Acked Write 10 Sent Done Done false
27957 8549 Acked Read 6 Sent Done Done false
27958 8550 Acked Write 6 Sent Done Done false
27959 8551 Acked Read 9 Sent Done Done false
27960 8552 Acked Flush 0 Sent Done Done false
27961 8553 Acked Write 10 Sent Done Done false
27962 8554 Acked Read 9 Sent Done Done false
27963 8555 Acked Read 6 Sent Done Done false
27964 8556 Acked Flush 0 Sent Done Done false
27965 8557 Acked Write 6 Sent Done Done false
27966 8558 Acked Write 6 Sent Done Done false
27967 8559 Acked Read 5 Sent Done Done false
27968 8560 Acked Read 7 Sent Done Done false
27969 8561 Acked Read 9 Sent Done Done false
27970 8562 Acked Read 2 Sent Done Done false
27971 8563 Acked Write 2 Sent Done Done false
27972 8564 Acked Write 1 Sent Done Done false
27973 8565 Acked Read 4 Sent Done Done false
27974 8566 Acked Write 10 Sent Done Done false
27975 8567 Acked Write 2 Sent Done Done false
27976 8568 Acked Read 10 Sent Done Done false
27977 8569 Acked Read 6 Sent Done Done false
27978 8570 Acked Read 6 Sent Done Done false
27979 8571 Acked Read 2 Sent Done Done false
27980 8572 Acked Flush 0 Sent Done Done false
27981 8573 Acked Read 5 Sent Done Done false
27982 8574 Acked Write 5 Sent Done Done false
27983 8575 Acked Flush 0 Sent Done Done false
27984 8576 Acked Read 4 Sent Done Done false
27985 8577 Acked Read 6 Sent Done Done false
27986 8578 Acked Flush 0 Sent Done Done false
27987 8579 Acked Read 7 Sent Done Done false
27988 8580 Acked Read 9 Sent Done Done false
27989 8581 Acked Read 4 Sent Done Done false
27990 8582 Acked Read 6 Sent Done Done false
27991 8583 Acked Read 10 Sent Done Done false
27992 8584 Acked Write 7 Sent Done Done false
27993 8585 Acked Read 1 Sent Done Done false
27994 8586 Acked Read 10 Sent Done Done false
27995 8587 Acked Flush 0 Sent Done Done false
27996 8588 Acked Read 10 Sent Done Done false
27997 8589 Acked Read 3 Sent Done Done false
27998 8590 Acked Write 8 Sent Done Done false
27999 8591 Acked Read 1 Sent Done Done false
28000 8592 Acked Write 7 Sent Done Done false
28001 8593 Acked Read 4 Sent Done Done false
28002 8594 Acked Flush 0 Sent Done Done false
28003 8595 Acked Write 8 Sent Done Done false
28004 8596 Acked Read 10 Sent Done Done false
28005 8597 Acked Write 9 Sent Done Done false
28006 8598 Acked Flush 0 Sent Done Done false
28007 8599 Acked Read 4 Sent Done Done false
28008 8600 Acked Write 2 Sent Done Done false
28009 8601 Acked Write 5 Sent Done Done false
28010 8602 Acked Read 1 Sent Done Done false
28011 8603 Acked Read 9 Sent Done Done false
28012 8604 Acked Write 8 Sent Done Done false
28013 8605 Acked Write 1 Sent Done Done false
28014 8606 Acked Read 2 Sent Done Done false
28015 8607 Acked Read 4 Sent Done Done false
28016 8608 Acked Flush 0 Sent Done Done false
28017 8609 Acked Write 4 Sent Done Done false
28018 8610 Acked Flush 0 Sent Done Done false
28019 8611 Acked Write 2 Sent Done Done false
28020 8612 Acked Read 7 Sent Done Done false
28021 8613 Acked Write 6 Sent Done Done false
28022 8614 Acked Read 10 Sent Done Done false
28023 8615 Acked Read 5 Sent Done Done false
28024 8616 Acked Write 7 Sent Done Done false
28025 8617 Acked Write 2 Sent Done Done false
28026 8618 Acked Flush 0 Sent Done Done false
28027 8619 Acked Read 2 Sent Done Done false
28028 8620 Acked Write 8 Sent Done Done false
28029 8621 Acked Read 9 Sent Done Done false
28030 8622 Acked Write 1 Sent Done Done false
28031 8623 Acked Read 1 Sent Done Done false
28032 8624 Acked Read 4 Sent Done Done false
28033 8625 Acked Write 2 Sent Done Done false
28034 8626 Acked Read 1 Sent Done Done false
28035 8627 Acked Write 7 Sent Done Done false
28036 8628 Acked Write 3 Sent Done Done false
28037 8629 Acked Read 10 Sent Done Done false
28038 8630 Acked Write 1 Sent Done Done false
28039 8631 Acked Write 1 Sent Done Done false
28040 8632 Acked Flush 0 Sent Done Done false
28041 8633 Acked Write 2 Sent Done Done false
28042 8634 Acked Flush 0 Sent Done Done false
28043 8635 Acked Read 2 Sent Done Done false
28044 8636 Acked Read 6 Sent Done Done false
28045 8637 Acked Write 3 Sent Done Done false
28046 8638 Acked Write 6 Sent Done Done false
28047 8639 Acked Flush 0 Sent Done Done false
28048 8640 Acked Flush 0 Sent Done Done false
28049 8641 Acked Read 6 Sent Done Done false
28050 8642 Acked Read 8 Sent Done Done false
28051 8643 Acked Write 6 Sent Done Done false
28052 8644 Acked Read 7 Sent Done Done false
28053 8645 Acked Read 6 Sent Done Done false
28054 8646 Acked Write 2 Sent Done Done false
28055 8647 Acked Read 6 Sent Done Done false
28056 8648 Acked Write 8 Sent Done Done false
28057 8649 Acked Flush 0 Sent Done Done false
28058 8650 Acked Write 1 Sent Done Done false
28059 8651 Acked Write 4 Sent Done Done false
28060 8652 Acked Read 5 Sent Done Done false
28061 8653 Acked Write 2 Sent Done Done false
28062 8654 Acked Read 8 Sent Done Done false
28063 8655 Acked Write 5 Sent Done Done false
28064 8656 Acked Read 4 Sent Done Done false
28065 8657 Acked Write 10 Sent Done Done false
28066 8658 Acked Write 7 Sent Done Done false
28067 8659 Acked Flush 0 Sent Done Done false
28068 8660 Acked Write 5 Sent Done Done false
28069 8661 Acked Read 7 Sent Done Done false
28070 8662 Acked Write 6 Sent Done Done false
28071 8663 Acked Read 7 Sent Done Done false
28072 8664 Acked Read 5 Sent Done Done false
28073 8665 Acked Write 5 Sent Done Done false
28074 8666 Acked Write 5 Sent Done Done false
28075 8667 Acked Read 8 Sent Done Done false
28076 8668 Acked Write 2 Sent Done Done false
28077 8669 Acked Read 8 Sent Done Done false
28078 8670 Acked Write 8 Sent Done Done false
28079 8671 Acked Write 6 Sent Done Done false
28080 8672 Acked Write 4 Sent Done Done false
28081 8673 Acked Read 4 Sent Done Done false
28082 8674 Acked Read 1 Sent Done Done false
28083 8675 Acked Flush 0 Sent Done Done false
28084 8676 Acked Write 6 Sent Done Done false
28085 8677 Acked Write 8 Sent Done Done false
28086 8678 Acked Write 2 Sent Done Done false
28087 8679 Acked Write 4 Sent Done Done false
28088 8680 Acked Read 1 Sent Done Done false
28089 8681 Acked Read 4 Sent Done Done false
28090 8682 Acked Write 6 Sent Done Done false
28091 8683 Acked Read 10 Sent Done Done false
28092 8684 Acked Read 8 Sent Done Done false
28093 8685 Acked Read 5 Sent Done Done false
28094 8686 Acked Read 2 Sent Done Done false
28095 8687 Acked Read 1 Sent Done Done false
28096 8688 Acked Write 10 Sent Done Done false
28097 8689 Acked Read 7 Sent Done Done false
28098 8690 Acked Flush 0 Sent Done Done false
28099 8691 Acked Read 2 Sent Done Done false
28100 8692 Acked Read 10 Sent Done Done false
28101 8693 Acked Flush 0 Sent Done Done false
28102 8694 Acked Write 1 Sent Done Done false
28103 8695 Acked Write 7 Sent Done Done false
28104 8696 Acked Read 3 Sent Done Done false
28105 8697 Acked Write 9 Sent Done Done false
28106 8698 Acked Write 9 Sent Done Done false
28107 8699 Acked Read 8 Sent Done Done false
28108 8700 Acked Write 10 Sent Done Done false
28109 8701 Acked Flush 0 Sent Done Done false
28110 8702 Acked Write 4 Sent Done Done false
28111 8703 Acked Write 3 Sent Done Done false
28112 8704 Acked Write 3 Sent Done Done false
28113 8705 Acked Read 10 Sent Done Done false
28114 8706 Acked Read 7 Sent Done Done false
28115 8707 Acked Write 9 Sent Done Done false
28116 8708 Acked Read 3 Sent Done Done false
28117 8709 Acked Read 9 Sent Done Done false
28118 8710 Acked Read 7 Sent Done Done false
28119 8711 Acked Write 8 Sent Done Done false
28120 8712 Acked Read 1 Sent Done Done false
28121 8713 Acked Write 4 Sent Done Done false
28122 8714 Acked Read 3 Sent Done Done false
28123 8715 Acked Read 8 Sent Done Done false
28124 8716 Acked Write 2 Sent Done Done false
28125 8717 Acked Flush 0 Sent Done Done false
28126 8718 Acked Flush 0 Sent Done Done false
28127 8719 Acked Flush 0 Sent Done Done false
28128 8720 Acked Write 1 Sent Done Done false
28129 8721 Acked Read 9 Sent Done Done false
28130 8722 Acked Read 8 Sent Done Done false
28131 8723 Acked Write 1 Sent Done Done false
28132 8724 Acked Read 6 Sent Done Done false
28133 8725 Acked Read 5 Sent Done Done false
28134 8726 Acked Read 3 Sent Done Done false
28135 8727 Acked Flush 0 Sent Done Done false
28136 8728 Acked Read 7 Sent Done Done false
28137 8729 Acked Read 7 Sent Done Done false
28138 8730 Acked Read 5 Sent Done Done false
28139 8731 Acked Read 2 Sent Done Done false
28140 8732 Acked Read 6 Sent Done Done false
28141 8733 Acked Flush 0 Sent Done Done false
28142 8734 Acked Read 5 Sent Done Done false
28143 8735 Acked Write 9 Sent Done Done false
28144 8736 Acked Read 6 Sent Done Done false
28145 8737 Acked Write 8 Sent Done Done false
28146 8738 Acked Write 9 Sent Done Done false
28147 8739 Acked Flush 0 Sent Done Done false
28148 8740 Acked Read 6 Sent Done Done false
28149 8741 Acked Read 10 Sent Done Done false
28150 8742 Acked Write 5 Sent Done Done false
28151 8743 Acked Read 4 Sent Done Done false
28152 8744 Acked Write 9 Sent Done Done false
28153 8745 Acked Write 6 Sent Done Done false
28154 8746 Acked Read 4 Sent Done Done false
28155 8747 Acked Read 4 Sent Done Done false
28156 8748 Acked Write 2 Sent Done Done false
28157 8749 Acked Write 1 Sent Done Done false
28158 8750 Acked Read 2 Sent Done Done false
28159 8751 Acked Read 4 Sent Done Done false
28160 8752 Acked Read 5 Sent Done Done false
28161 8753 Acked Read 2 Sent Done Done false
28162 8754 Acked Read 1 Sent Done Done false
28163 8755 Acked Read 9 Sent Done Done false
28164 8756 Acked Write 10 Sent Done Done false
28165 8757 Acked Read 3 Sent Done Done false
28166 8758 Acked Write 4 Sent Done Done false
28167 8759 Acked Read 5 Sent Done Done false
28168 8760 Acked Write 6 Sent Done Done false
28169 8761 Acked Write 9 Sent Done Done false
28170 8762 Acked Read 1 Sent Done Done false
28171 8763 Acked Read 3 Sent Done Done false
28172 8764 Acked Read 7 Sent Done Done false
28173 8765 Acked Read 10 Sent Done Done false
28174 8766 Acked Write 9 Sent Done Done false
28175 8767 Acked Write 10 Sent Done Done false
28176 8768 Acked Write 9 Sent Done Done false
28177 8769 Acked Read 7 Sent Done Done false
28178 8770 Acked Read 5 Sent Done Done false
28179 8771 Acked Write 2 Sent Done Done false
28180 8772 Acked Read 7 Sent Done Done false
28181 8773 Acked Write 1 Sent Done Done false
28182 8774 Acked Read 1 Sent Done Done false
28183 8775 Acked Read 1 Sent Done Done false
28184 8776 Acked Read 6 Sent Done Done false
28185 8777 Acked Write 8 Sent Done Done false
28186 8778 Acked Flush 0 Sent Done Done false
28187 8779 Acked Read 8 Sent Done Done false
28188 8780 Acked Write 6 Sent Done Done false
28189 8781 Acked Write 8 Sent Done Done false
28190 8782 Acked Write 9 Sent Done Done false
28191 8783 Acked Write 10 Sent Done Done false
28192 8784 Acked Write 2 Sent Done Done false
28193 8785 Acked Write 8 Sent Done Done false
28194 8786 Acked Read 3 Sent Done Done false
28195 8787 Acked Write 7 Sent Done Done false
28196 8788 Acked Read 6 Sent Done Done false
28197 8789 Acked Read 6 Sent Done Done false
28198 8790 Acked Read 7 Sent Done Done false
28199 8791 Acked Read 6 Sent Done Done false
28200 8792 Acked Read 8 Sent Done Done false
28201 8793 Acked Read 4 Sent Done Done false
28202 8794 Acked Write 10 Sent Done Done false
28203 8795 Acked Write 4 Sent Done Done false
28204 8796 Acked Write 1 Sent Done Done false
28205 8797 Acked Read 7 Sent Done Done false
28206 8798 Acked Read 10 Sent Done Done false
28207 8799 Acked Read 8 Sent Done Done false
28208 8800 Acked Read 6 Sent Done Done false
28209 8801 Acked Read 7 Sent Done Done false
28210 8802 Acked Read 8 Sent Done Done false
28211 8803 Acked Write 4 Sent Done Done false
28212 8804 Acked Write 6 Sent Done Done false
28213 8805 Acked Read 6 Sent Done Done false
28214 8806 Acked Read 5 Sent Done Done false
28215 8807 Acked Write 8 Sent Done Done false
28216 8808 Acked Write 1 Sent Done Done false
28217 8809 Acked Flush 0 Sent Done Done false
28218 8810 Acked Read 5 Sent Done Done false
28219 8811 Acked Write 4 Sent Done Done false
28220 8812 Acked Read 10 Sent Done Done false
28221 8813 Acked Read 4 Sent Done Done false
28222 8814 Acked Write 2 Sent Done Done false
28223 8815 Acked Read 3 Sent Done Done false
28224 8816 Acked Write 1 Sent Done Done false
28225 8817 Acked Write 1 Sent Done Done false
28226 8818 Acked Read 7 Sent Done Done false
28227 8819 Acked Write 3 Sent Done Done false
28228 8820 Acked Write 3 Sent Done Done false
28229 8821 Acked Read 8 Sent Done Done false
28230 8822 Acked Read 4 Sent Done Done false
28231 8823 Acked Flush 0 Sent Done Done false
28232 8824 Acked Read 3 Sent Done Done false
28233 8825 Acked Write 2 Sent Done Done false
28234 8826 Acked Write 8 Sent Done Done false
28235 8827 Acked Read 9 Sent Done Done false
28236 8828 Acked Read 10 Sent Done Done false
28237 8829 Acked Read 4 Sent Done Done false
28238 8830 Acked Read 7 Sent Done Done false
28239 8831 Acked Read 5 Sent Done Done false
28240 8832 Acked Read 8 Sent Done Done false
28241 8833 Acked Read 10 Sent Done Done false
28242 8834 Acked Read 10 Sent Done Done false
28243 8835 Acked Write 3 Sent Done Done false
28244 8836 Acked Write 1 Sent Done Done false
28245 8837 Acked Flush 0 Sent Done Done false
28246 8838 Acked Write 8 Sent Done Done false
28247 8839 Acked Write 9 Sent Done Done false
28248 8840 Acked Write 7 Sent Done Done false
28249 8841 Acked Write 3 Sent Done Done false
28250 8842 Acked Read 4 Sent Done Done false
28251 8843 Acked Read 9 Sent Done Done false
28252 8844 Acked Read 4 Sent Done Done false
28253 8845 Acked Read 7 Sent Done Done false
28254 8846 Acked Write 7 Sent Done Done false
28255 8847 Acked Write 4 Sent Done Done false
28256 8848 Acked Write 9 Sent Done Done false
28257 STATES DS:0 DS:1 DS:2 TOTAL
28258 Sent 301 0 0 301
28259 Done 0 301 301 602
28260 Skipped 0 0 0 0
28261 Error 0 0 0 0
28262 Last Flush: 8547 8837 8837
28263 Downstairs last five completed: 8547 8546 8545 8544 8543
28264 Upstairs last five completed: 8848 8847 8846 8845 8844
28265 CLIENT: Up:0 ds:301 act:2
28266 [8810] got start action
28267 [8810] I am going to restart
28268 Starting downstairs at port 8810
28269 Make output file at "/tmp/dsc/downstairs-8810.txt"
28270 [8810][0] reports Starting
28271 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1228)
28272 [8810][0] reports Running
28273 ----------------------------------------------------------------
28274 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28275 States: Offline Active Active
28276 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28277 8548 Acked Write 10 Sent Done Done false
28278 8549 Acked Read 6 Sent Done Done false
28279 8550 Acked Write 6 Sent Done Done false
28280 8551 Acked Read 9 Sent Done Done false
28281 8552 Acked Flush 0 Sent Done Done false
28282 8553 Acked Write 10 Sent Done Done false
28283 8554 Acked Read 9 Sent Done Done false
28284 8555 Acked Read 6 Sent Done Done false
28285 8556 Acked Flush 0 Sent Done Done false
28286 8557 Acked Write 6 Sent Done Done false
28287 8558 Acked Write 6 Sent Done Done false
28288 8559 Acked Read 5 Sent Done Done false
28289 8560 Acked Read 7 Sent Done Done false
28290 8561 Acked Read 9 Sent Done Done false
28291 8562 Acked Read 2 Sent Done Done false
28292 8563 Acked Write 2 Sent Done Done false
28293 8564 Acked Write 1 Sent Done Done false
28294 8565 Acked Read 4 Sent Done Done false
28295 8566 Acked Write 10 Sent Done Done false
28296 8567 Acked Write 2 Sent Done Done false
28297 8568 Acked Read 10 Sent Done Done false
28298 8569 Acked Read 6 Sent Done Done false
28299 8570 Acked Read 6 Sent Done Done false
28300 8571 Acked Read 2 Sent Done Done false
28301 8572 Acked Flush 0 Sent Done Done false
28302 8573 Acked Read 5 Sent Done Done false
28303 8574 Acked Write 5 Sent Done Done false
28304 8575 Acked Flush 0 Sent Done Done false
28305 8576 Acked Read 4 Sent Done Done false
28306 8577 Acked Read 6 Sent Done Done false
28307 8578 Acked Flush 0 Sent Done Done false
28308 8579 Acked Read 7 Sent Done Done false
28309 8580 Acked Read 9 Sent Done Done false
28310 8581 Acked Read 4 Sent Done Done false
28311 8582 Acked Read 6 Sent Done Done false
28312 8583 Acked Read 10 Sent Done Done false
28313 8584 Acked Write 7 Sent Done Done false
28314 8585 Acked Read 1 Sent Done Done false
28315 8586 Acked Read 10 Sent Done Done false
28316 8587 Acked Flush 0 Sent Done Done false
28317 8588 Acked Read 10 Sent Done Done false
28318 8589 Acked Read 3 Sent Done Done false
28319 8590 Acked Write 8 Sent Done Done false
28320 8591 Acked Read 1 Sent Done Done false
28321 8592 Acked Write 7 Sent Done Done false
28322 8593 Acked Read 4 Sent Done Done false
28323 8594 Acked Flush 0 Sent Done Done false
28324 8595 Acked Write 8 Sent Done Done false
28325 8596 Acked Read 10 Sent Done Done false
28326 8597 Acked Write 9 Sent Done Done false
28327 8598 Acked Flush 0 Sent Done Done false
28328 8599 Acked Read 4 Sent Done Done false
28329 8600 Acked Write 2 Sent Done Done false
28330 8601 Acked Write 5 Sent Done Done false
28331 8602 Acked Read 1 Sent Done Done false
28332 8603 Acked Read 9 Sent Done Done false
28333 8604 Acked Write 8 Sent Done Done false
28334 8605 Acked Write 1 Sent Done Done false
28335 8606 Acked Read 2 Sent Done Done false
28336 8607 Acked Read 4 Sent Done Done false
28337 8608 Acked Flush 0 Sent Done Done false
28338 8609 Acked Write 4 Sent Done Done false
28339 8610 Acked Flush 0 Sent Done Done false
28340 8611 Acked Write 2 Sent Done Done false
28341 8612 Acked Read 7 Sent Done Done false
28342 8613 Acked Write 6 Sent Done Done false
28343 8614 Acked Read 10 Sent Done Done false
28344 8615 Acked Read 5 Sent Done Done false
28345 8616 Acked Write 7 Sent Done Done false
28346 8617 Acked Write 2 Sent Done Done false
28347 8618 Acked Flush 0 Sent Done Done false
28348 8619 Acked Read 2 Sent Done Done false
28349 8620 Acked Write 8 Sent Done Done false
28350 8621 Acked Read 9 Sent Done Done false
28351 8622 Acked Write 1 Sent Done Done false
28352 8623 Acked Read 1 Sent Done Done false
28353 8624 Acked Read 4 Sent Done Done false
28354 8625 Acked Write 2 Sent Done Done false
28355 8626 Acked Read 1 Sent Done Done false
28356 8627 Acked Write 7 Sent Done Done false
28357 8628 Acked Write 3 Sent Done Done false
28358 8629 Acked Read 10 Sent Done Done false
28359 8630 Acked Write 1 Sent Done Done false
28360 8631 Acked Write 1 Sent Done Done false
28361 8632 Acked Flush 0 Sent Done Done false
28362 8633 Acked Write 2 Sent Done Done false
28363 8634 Acked Flush 0 Sent Done Done false
28364 8635 Acked Read 2 Sent Done Done false
28365 8636 Acked Read 6 Sent Done Done false
28366 8637 Acked Write 3 Sent Done Done false
28367 8638 Acked Write 6 Sent Done Done false
28368 8639 Acked Flush 0 Sent Done Done false
28369 8640 Acked Flush 0 Sent Done Done false
28370 8641 Acked Read 6 Sent Done Done false
28371 8642 Acked Read 8 Sent Done Done false
28372 8643 Acked Write 6 Sent Done Done false
28373 8644 Acked Read 7 Sent Done Done false
28374 8645 Acked Read 6 Sent Done Done false
28375 8646 Acked Write 2 Sent Done Done false
28376 8647 Acked Read 6 Sent Done Done false
28377 8648 Acked Write 8 Sent Done Done false
28378 8649 Acked Flush 0 Sent Done Done false
28379 8650 Acked Write 1 Sent Done Done false
28380 8651 Acked Write 4 Sent Done Done false
28381 8652 Acked Read 5 Sent Done Done false
28382 8653 Acked Write 2 Sent Done Done false
28383 8654 Acked Read 8 Sent Done Done false
28384 8655 Acked Write 5 Sent Done Done false
28385 8656 Acked Read 4 Sent Done Done false
28386 8657 Acked Write 10 Sent Done Done false
28387 8658 Acked Write 7 Sent Done Done false
28388 8659 Acked Flush 0 Sent Done Done false
28389 8660 Acked Write 5 Sent Done Done false
28390 8661 Acked Read 7 Sent Done Done false
28391 8662 Acked Write 6 Sent Done Done false
28392 8663 Acked Read 7 Sent Done Done false
28393 8664 Acked Read 5 Sent Done Done false
28394 8665 Acked Write 5 Sent Done Done false
28395 8666 Acked Write 5 Sent Done Done false
28396 8667 Acked Read 8 Sent Done Done false
28397 8668 Acked Write 2 Sent Done Done false
28398 8669 Acked Read 8 Sent Done Done false
28399 8670 Acked Write 8 Sent Done Done false
28400 8671 Acked Write 6 Sent Done Done false
28401 8672 Acked Write 4 Sent Done Done false
28402 8673 Acked Read 4 Sent Done Done false
28403 8674 Acked Read 1 Sent Done Done false
28404 8675 Acked Flush 0 Sent Done Done false
28405 8676 Acked Write 6 Sent Done Done false
28406 8677 Acked Write 8 Sent Done Done false
28407 8678 Acked Write 2 Sent Done Done false
28408 8679 Acked Write 4 Sent Done Done false
28409 8680 Acked Read 1 Sent Done Done false
28410 8681 Acked Read 4 Sent Done Done false
28411 8682 Acked Write 6 Sent Done Done false
28412 8683 Acked Read 10 Sent Done Done false
28413 8684 Acked Read 8 Sent Done Done false
28414 8685 Acked Read 5 Sent Done Done false
28415 8686 Acked Read 2 Sent Done Done false
28416 8687 Acked Read 1 Sent Done Done false
28417 8688 Acked Write 10 Sent Done Done false
28418 8689 Acked Read 7 Sent Done Done false
28419 8690 Acked Flush 0 Sent Done Done false
28420 8691 Acked Read 2 Sent Done Done false
28421 8692 Acked Read 10 Sent Done Done false
28422 8693 Acked Flush 0 Sent Done Done false
28423 8694 Acked Write 1 Sent Done Done false
28424 8695 Acked Write 7 Sent Done Done false
28425 8696 Acked Read 3 Sent Done Done false
28426 8697 Acked Write 9 Sent Done Done false
28427 8698 Acked Write 9 Sent Done Done false
28428 8699 Acked Read 8 Sent Done Done false
28429 8700 Acked Write 10 Sent Done Done false
28430 8701 Acked Flush 0 Sent Done Done false
28431 8702 Acked Write 4 Sent Done Done false
28432 8703 Acked Write 3 Sent Done Done false
28433 8704 Acked Write 3 Sent Done Done false
28434 8705 Acked Read 10 Sent Done Done false
28435 8706 Acked Read 7 Sent Done Done false
28436 8707 Acked Write 9 Sent Done Done false
28437 8708 Acked Read 3 Sent Done Done false
28438 8709 Acked Read 9 Sent Done Done false
28439 8710 Acked Read 7 Sent Done Done false
28440 8711 Acked Write 8 Sent Done Done false
28441 8712 Acked Read 1 Sent Done Done false
28442 8713 Acked Write 4 Sent Done Done false
28443 8714 Acked Read 3 Sent Done Done false
28444 8715 Acked Read 8 Sent Done Done false
28445 8716 Acked Write 2 Sent Done Done false
28446 8717 Acked Flush 0 Sent Done Done false
28447 8718 Acked Flush 0 Sent Done Done false
28448 8719 Acked Flush 0 Sent Done Done false
28449 8720 Acked Write 1 Sent Done Done false
28450 8721 Acked Read 9 Sent Done Done false
28451 8722 Acked Read 8 Sent Done Done false
28452 8723 Acked Write 1 Sent Done Done false
28453 8724 Acked Read 6 Sent Done Done false
28454 8725 Acked Read 5 Sent Done Done false
28455 8726 Acked Read 3 Sent Done Done false
28456 8727 Acked Flush 0 Sent Done Done false
28457 8728 Acked Read 7 Sent Done Done false
28458 8729 Acked Read 7 Sent Done Done false
28459 8730 Acked Read 5 Sent Done Done false
28460 8731 Acked Read 2 Sent Done Done false
28461 8732 Acked Read 6 Sent Done Done false
28462 8733 Acked Flush 0 Sent Done Done false
28463 8734 Acked Read 5 Sent Done Done false
28464 8735 Acked Write 9 Sent Done Done false
28465 8736 Acked Read 6 Sent Done Done false
28466 8737 Acked Write 8 Sent Done Done false
28467 8738 Acked Write 9 Sent Done Done false
28468 8739 Acked Flush 0 Sent Done Done false
28469 8740 Acked Read 6 Sent Done Done false
28470 8741 Acked Read 10 Sent Done Done false
28471 8742 Acked Write 5 Sent Done Done false
28472 8743 Acked Read 4 Sent Done Done false
28473 8744 Acked Write 9 Sent Done Done false
28474 8745 Acked Write 6 Sent Done Done false
28475 8746 Acked Read 4 Sent Done Done false
28476 8747 Acked Read 4 Sent Done Done false
28477 8748 Acked Write 2 Sent Done Done false
28478 8749 Acked Write 1 Sent Done Done false
28479 8750 Acked Read 2 Sent Done Done false
28480 8751 Acked Read 4 Sent Done Done false
28481 8752 Acked Read 5 Sent Done Done false
28482 8753 Acked Read 2 Sent Done Done false
28483 8754 Acked Read 1 Sent Done Done false
28484 8755 Acked Read 9 Sent Done Done false
28485 8756 Acked Write 10 Sent Done Done false
28486 8757 Acked Read 3 Sent Done Done false
28487 8758 Acked Write 4 Sent Done Done false
28488 8759 Acked Read 5 Sent Done Done false
28489 8760 Acked Write 6 Sent Done Done false
28490 8761 Acked Write 9 Sent Done Done false
28491 8762 Acked Read 1 Sent Done Done false
28492 8763 Acked Read 3 Sent Done Done false
28493 8764 Acked Read 7 Sent Done Done false
28494 8765 Acked Read 10 Sent Done Done false
28495 8766 Acked Write 9 Sent Done Done false
28496 8767 Acked Write 10 Sent Done Done false
28497 8768 Acked Write 9 Sent Done Done false
28498 8769 Acked Read 7 Sent Done Done false
28499 8770 Acked Read 5 Sent Done Done false
28500 8771 Acked Write 2 Sent Done Done false
28501 8772 Acked Read 7 Sent Done Done false
28502 8773 Acked Write 1 Sent Done Done false
28503 8774 Acked Read 1 Sent Done Done false
28504 8775 Acked Read 1 Sent Done Done false
28505 8776 Acked Read 6 Sent Done Done false
28506 8777 Acked Write 8 Sent Done Done false
28507 8778 Acked Flush 0 Sent Done Done false
28508 8779 Acked Read 8 Sent Done Done false
28509 8780 Acked Write 6 Sent Done Done false
28510 8781 Acked Write 8 Sent Done Done false
28511 8782 Acked Write 9 Sent Done Done false
28512 8783 Acked Write 10 Sent Done Done false
28513 8784 Acked Write 2 Sent Done Done false
28514 8785 Acked Write 8 Sent Done Done false
28515 8786 Acked Read 3 Sent Done Done false
28516 8787 Acked Write 7 Sent Done Done false
28517 8788 Acked Read 6 Sent Done Done false
28518 8789 Acked Read 6 Sent Done Done false
28519 8790 Acked Read 7 Sent Done Done false
28520 8791 Acked Read 6 Sent Done Done false
28521 8792 Acked Read 8 Sent Done Done false
28522 8793 Acked Read 4 Sent Done Done false
28523 8794 Acked Write 10 Sent Done Done false
28524 8795 Acked Write 4 Sent Done Done false
28525 8796 Acked Write 1 Sent Done Done false
28526 8797 Acked Read 7 Sent Done Done false
28527 8798 Acked Read 10 Sent Done Done false
28528 8799 Acked Read 8 Sent Done Done false
28529 8800 Acked Read 6 Sent Done Done false
28530 8801 Acked Read 7 Sent Done Done false
28531 8802 Acked Read 8 Sent Done Done false
28532 8803 Acked Write 4 Sent Done Done false
28533 8804 Acked Write 6 Sent Done Done false
28534 8805 Acked Read 6 Sent Done Done false
28535 8806 Acked Read 5 Sent Done Done false
28536 8807 Acked Write 8 Sent Done Done false
28537 8808 Acked Write 1 Sent Done Done false
28538 8809 Acked Flush 0 Sent Done Done false
28539 8810 Acked Read 5 Sent Done Done false
28540 8811 Acked Write 4 Sent Done Done false
28541 8812 Acked Read 10 Sent Done Done false
28542 8813 Acked Read 4 Sent Done Done false
28543 8814 Acked Write 2 Sent Done Done false
28544 8815 Acked Read 3 Sent Done Done false
28545 8816 Acked Write 1 Sent Done Done false
28546 8817 Acked Write 1 Sent Done Done false
28547 8818 Acked Read 7 Sent Done Done false
28548 8819 Acked Write 3 Sent Done Done false
28549 8820 Acked Write 3 Sent Done Done false
28550 8821 Acked Read 8 Sent Done Done false
28551 8822 Acked Read 4 Sent Done Done false
28552 8823 Acked Flush 0 Sent Done Done false
28553 8824 Acked Read 3 Sent Done Done false
28554 8825 Acked Write 2 Sent Done Done false
28555 8826 Acked Write 8 Sent Done Done false
28556 8827 Acked Read 9 Sent Done Done false
28557 8828 Acked Read 10 Sent Done Done false
28558 8829 Acked Read 4 Sent Done Done false
28559 8830 Acked Read 7 Sent Done Done false
28560 8831 Acked Read 5 Sent Done Done false
28561 8832 Acked Read 8 Sent Done Done false
28562 8833 Acked Read 10 Sent Done Done false
28563 8834 Acked Read 10 Sent Done Done false
28564 8835 Acked Write 3 Sent Done Done false
28565 8836 Acked Write 1 Sent Done Done false
28566 8837 Acked Flush 0 Sent Done Done false
28567 8838 Acked Write 8 Sent Done Done false
28568 8839 Acked Write 9 Sent Done Done false
28569 8840 Acked Write 7 Sent Done Done false
28570 8841 Acked Write 3 Sent Done Done false
28571 8842 Acked Read 4 Sent Done Done false
28572 8843 Acked Read 9 Sent Done Done false
28573 8844 Acked Read 4 Sent Done Done false
28574 8845 Acked Read 7 Sent Done Done false
28575 8846 Acked Write 7 Sent Done Done false
28576 8847 Acked Write 4 Sent Done Done false
28577 8848 Acked Write 9 Sent Done Done false
28578 8849 Acked Flush 0 Sent Done Done false
28579 STATES DS:0 DS:1 DS:2 TOTAL
28580 Sent 302 0 0 302
28581 Done 0 302 302 604
28582 Skipped 0 0 0 0
28583 Error 0 0 0 0
28584 Last Flush: 8547 8849 8849
28585 Downstairs last five completed: 8547 8546 8545 8544 8543
28586 Upstairs last five completed: 8849 8848 8847 8846 8845
28587 CLIENT: Up:0 ds:302 act:2
285882025-01-13T15:57:47.316ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
285892025-01-13T15:57:47.316ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49952) = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
285902025-01-13T15:57:47.317ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
285912025-01-13T15:57:47.318ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
285922025-01-13T15:57:47.318ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
285932025-01-13T15:57:47.318ZINFOcrucible: send last flush ID to this DS: 8547 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
285942025-01-13T15:57:47.318ZINFOcrucible: Replied this last flush ID: 8547 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
285952025-01-13T15:57:47.318ZINFOcrucible: [0] client re-new 302 jobs since flush 8547 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
285962025-01-13T15:57:47.318ZINFOcrucible: [0] Replayed 302 jobs since flush: 8547 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
28597 ----------------------------------------------------------------
28598 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
28599 Downstairs last five completed: 8849 8848 8847 8846 8845
28600 Upstairs last five completed: 8849 8848 8847 8846 8845
28601 CLIENT: Up:0 ds:0 act:3
28602 Replay: All jobs finished
28603 Main task has work to do, go find it
28604 got dsc Stop(0)
28605 stop 0
28606 Jan 13 15:57:49.566 [8810] Got stop action so:false kr:false
28607 INFO request completed, latency_us: 491, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 7f3cc2e2-5964-4800-bea1-f757f18b0fd4, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
28608 Jan 13 15:57:49.566 INFO request completed, {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:57:49.566792183Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
28609 latency_us: 134, response_code: 200, uri: /state/cid/0, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:57:49.566853033Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
28610 method: GET, req_id: {"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:57:49.566881798Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
28611 8901ed3f-1fe5-4779-9304-1679e617b500, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
286122025-01-13T15:57:49.567ZINFOcrucible: client task is exiting = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
28613 [8810] Exited with: ExitStatus(unix_wait_status(9))
28614 [8810][0] reports Exit
28615 Jan 13 15:57:53.568 INFO request completed, latency_us: 266, response_code: 200, uri: /state/cid/0, method: GET, req_id: 26af63c6-3c25-4d55-b357-ca385b488ceb, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
28616 001/300 Read block 1934 len 32768
28617 002/300 Read block 2668 len 4096
28618 003/300 Flush
28619 004/300 Write block 756 len 20480 data: 8 8 6 7 9
28620 005/300 Read block 4958 len 36864
28621 006/300 Read block 1205 len 4096
28622 007/300 Write block 4841 len 32768 data: 8 7 6 6 6 4 5 3
28623 008/300 Read block 4420 len 40960
28624 009/300 Write block 180 len 4096 data: 5
28625 010/300 Write block 3826 len 36864 data: 6 7 6 7 7 7 6 5 3
28626 011/300 Read block 1067 len 8192
28627 012/300 Write block 3466 len 36864 data: 5 5 4 6 6 8 7 8 10
28628 013/300 Read block 454 len 12288
28629 014/300 Write block 3323 len 8192 data: 6 7
28630 015/300 Read block 4948 len 40960
28631 016/300 Write block 2987 len 28672 data: 3 4 4 4 4 6 7
28632 017/300 Flush
28633 018/300 Write block 1726 len 28672 data: 6 6 4 6 6 6 6
28634 019/300 Read block 1627 len 16384
28635 020/300 Read block 4613 len 16384
28636 021/300 Read block 4815 len 8192
28637 022/300 Read block 587 len 36864
28638 023/300 Read block 3137 len 40960
28639 024/300 Read block 2225 len 4096
28640 025/300 Read block 4752 len 8192
28641 026/300 Write block 3862 len 20480 data: 7 6 6 6 5
28642 027/300 Read block 2486 len 28672
28643 028/300 Read block 3040 len 8192
28644 029/300 Write block 2195 len 16384 data: 7 9 10 11
28645 030/300 Read block 333 len 28672
28646 031/300 Flush
28647 032/300 Write block 1623 len 4096 data: 8
28648 033/300 Read block 2723 len 12288
28649 034/300 Read block 1780 len 36864
28650 035/300 Read block 4778 len 4096
28651 036/300 Read block 1270 len 28672
28652 037/300 Write block 4940 len 32768 data: 6 6 5 6 7 6 5 6
28653 038/300 Read block 3806 len 20480
28654 039/300 Read block 1666 len 24576
28655 040/300 Read block 2314 len 8192
28656 041/300 Write block 1267 len 24576 data: 8 6 6 6 7 6
28657 042/300 Read block 769 len 16384
28658 043/300 Write block 375 len 16384 data: 10 10 10 10
28659 044/300 Flush
28660 045/300 Read block 792 len 36864
28661 046/300 Write block 4787 len 8192 data: 6 6
28662 047/300 Write block 1717 len 32768 data: 3 3 3 3 3 4 5 5
28663 048/300 Read block 2076 len 36864
28664 049/300 Write block 3575 len 8192 data: 2 4
28665 050/300 Write block 2750 len 20480 data: 9 9 9 8 7
28666 051/300 Read block 2674 len 40960
28667 052/300 Read block 4087 len 16384
28668 053/300 Read block 3467 len 8192
28669 054/300 Write block 2064 len 36864 data: 4 4 3 4 3 3 3 3 4
28670 055/300 Flush
28671 056/300 Flush
28672 057/300 Write block 1271 len 36864 data: 8 7 7 7 6 6 7 8 6
28673 058/300 Read block 3362 len 8192
28674 059/300 Read block 4629 len 36864
28675 060/300 Flush
28676 061/300 Write block 1820 len 28672 data: 5 5 4 5 5 5 6
28677 062/300 Read block 109 len 16384
28678 063/300 Read block 4023 len 4096
28679 064/300 Read block 3423 len 4096
28680 065/300 Read block 2424 len 20480
28681 066/300 Flush
28682 067/300 Flush
28683 068/300 Write block 1315 len 16384 data: 6 6 5 5
28684 069/300 Read block 442 len 8192
28685 070/300 Flush
28686 071/300 Write block 3592 len 40960 data: 7 5 6 6 6 5 6 6 6 7
28687 072/300 Read block 4281 len 20480
28688 073/300 Write block 2446 len 12288 data: 5 5 6
28689 074/300 Write block 430 len 36864 data: 6 6 6 5 4 5 4 4 4
28690 075/300 Read block 344 len 8192
28691 076/300 Read block 711 len 20480
28692 077/300 Write block 431 len 4096 data: 7
28693 078/300 Read block 1029 len 8192
28694 079/300 Flush
28695 080/300 Write block 282 len 20480 data: 7 8 9 11 11
28696 081/300 Read block 4543 len 16384
28697 082/300 Write block 2050 len 16384 data: 7 8 9 7
28698 083/300 Read block 101 len 12288
28699 084/300 Write block 3328 len 28672 data: 7 7 6 5 5 4 3
28700 085/300 Read block 4385 len 32768
28701 086/300 Read block 2601 len 40960
28702 087/300 Read block 2263 len 32768
28703 088/300 Write block 3685 len 40960 data: 3 4 4 5 4 4 4 3 2 2
28704 089/300 Read block 3529 len 32768
28705 090/300 Read block 2492 len 24576
28706 091/300 Write block 3118 len 16384 data: 5 5 3 5
28707 092/300 Write block 4948 len 16384 data: 6 6 6 6
28708 093/300 Read block 3050 len 12288
28709 094/300 Flush
28710 095/300 Read block 1677 len 8192
28711 096/300 Write block 1702 len 40960 data: 5 4 4 7 6 5 3 3 3 2
28712 097/300 Read block 2744 len 36864
28713 098/300 Read block 2410 len 20480
28714 099/300 Read block 4003 len 12288
28715 100/300 Write block 2996 len 12288 data: 6 5 4
28716 101/300 Flush
28717 102/300 Write block 4857 len 24576 data: 5 5 5 7 7 5
28718 103/300 Read block 945 len 28672
28719 104/300 Write block 3643 len 8192 data: 7 8
28720 105/300 Flush
28721 106/300 Read block 1041 len 8192
28722 107/300 Read block 2239 len 36864
28723 108/300 Read block 3300 len 40960
28724 109/300 Write block 2330 len 40960 data: 4 4 6 7 7 7 6 6 6 9
28725 110/300 Write block 29 len 28672 data: 9 9 7 5 5 6 6
28726 111/300 Write block 3121 len 28672 data: 6 6 5 5 5 6 5
28727 112/300 Write block 389 len 28672 data: 7 7 5 5 5 5 5
28728 113/300 Read block 589 len 40960
28729 114/300 Write block 2604 len 8192 data: 8 9
28730 115/300 Write block 1437 len 8192 data: 9 7
28731 116/300 Read block 3469 len 12288
28732 117/300 Write block 3024 len 20480 data: 10 8 8 10 10
28733 118/300 Read block 1185 len 40960
28734 119/300 Write block 4500 len 28672 data: 6 8 9 11 9 3 4
28735 120/300 Write block 689 len 8192 data: 6 5
28736 121/300 Read block 4332 len 4096
28737 122/300 Read block 2687 len 4096
28738 123/300 Read block 4263 len 28672
28739 124/300 Write block 4171 len 12288 data: 4 7 6
28740 125/300 Read block 747 len 12288
28741 126/300 Write block 3785 len 40960 data: 6 4 4 5 5 5 7 8 6 6
28742 127/300 Flush
28743 128/300 Read block 2985 len 36864
28744 129/300 Flush
28745 130/300 Read block 2833 len 20480
28746 131/300 Read block 2650 len 40960
28747 132/300 Write block 3614 len 28672 data: 5 6 6 6 4 3 3
28748 133/300 Write block 4315 len 36864 data: 3 5 7 7 6 7 5 5 6
28749 134/300 Write block 3284 len 20480 data: 7 7 6 6 7
28750 135/300 Read block 937 len 24576
28751 136/300 Read block 2787 len 36864
28752 137/300 Read block 4356 len 20480
28753 138/300 Read block 3113 len 28672
28754 139/300 Write block 2512 len 32768 data: 4 4 4 4 5 5 7 7
28755 140/300 Write block 2683 len 32768 data: 6 8 7 5 6 6 5 5
28756 141/300 Write block 2323 len 12288 data: 5 5 3
28757 142/300 Write block 2870 len 16384 data: 8 8 8 7
28758 143/300 Read block 2747 len 8192
28759 144/300 Read block 1617 len 12288
28760 145/300 Write block 384 len 32768 data: 5 5 7 8 7 8 8 6
28761 146/300 Read block 392 len 32768
28762 147/300 Write block 4809 len 32768 data: 3 3 4 4 3 3 5 6
28763 148/300 Read block 4489 len 32768
28764 149/300 Write block 3112 len 8192 data: 6 6
28765 150/300 Read block 317 len 32768
28766 151/300 Read block 4139 len 36864
28767 152/300 Read block 1960 len 8192
28768 153/300 Read block 218 len 4096
28769 154/300 Flush
28770 155/300 Read block 773 len 32768
28771 156/300 Read block 1392 len 28672
28772 157/300 Read block 3597 len 28672
28773 158/300 Write block 2314 len 24576 data: 6 5 5 6 5 5
28774 159/300 Read block 4047 len 4096
28775 160/300 Read block 308 len 28672
28776 161/300 Read block 2006 len 36864
28777 162/300 Read block 4266 len 28672
28778 163/300 Read block 3612 len 20480
28779 164/300 Write block 897 len 24576 data: 5 6 5 4 3 3
28780 165/300 Write block 4769 len 24576 data: 5 5 5 5 6 6
28781 166/300 Write block 482 len 24576 data: 6 6 7 7 7 6
28782 167/300 Read block 2227 len 4096
28783 168/300 Write block 2082 len 12288 data: 9 11 8
28784 169/300 Read block 2385 len 28672
28785 170/300 Read block 2813 len 40960
28786 171/300 Read block 3154 len 40960
28787 172/300 Write block 2500 len 12288 data: 4 5 5
28788 173/300 Read block 4724 len 28672
28789 174/300 Flush
28790 175/300 Read block 3574 len 32768
28791 176/300 Read block 545 len 4096
28792 177/300 Read block 2416 len 24576
28793 178/300 Write block 1730 len 20480 data: 7 7 7 6 4
28794 179/300 Flush
28795 180/300 Read block 2586 len 12288
28796 181/300 Write block 2745 len 32768 data: 4 4 7 8 9 10 10 10
28797 182/300 Read block 2589 len 40960
28798 183/300 Flush
28799 184/300 Read block 3724 len 32768
28800 185/300 Write block 2909 len 4096 data: 8
28801 186/300 Read block 3069 len 36864
28802 187/300 Write block 2874 len 4096 data: 7
28803 188/300 Read block 3201 len 4096
28804 189/300 Write block 4769 len 32768 data: 6 6 6 6 7 7 4 4
28805 190/300 Read block 4445 len 24576
28806 191/300 Read block 952 len 4096
28807 192/300 Write block 56 len 8192 data: 7 6
28808 193/300 Write block 2016 len 24576 data: 4 5 6 9 8 8
28809 194/300 Read block 1041 len 28672
28810 195/300 Write block 4087 len 32768 data: 4 4 3 3 3 4 4 4
28811 196/300 Write block 1460 len 4096 data: 4
28812 197/300 Write block 443 len 28672 data: 3 4 4 5 5 5 5
28813 198/300 Write block 4230 len 12288 data: 5 5 4
28814 199/300 Read block 1030 len 12288
28815 200/300 Write block 687 len 40960 data: 6 6 7 6 5 5 5 6 5 7
28816 201/300 Write block 3085 len 12288 data: 4 4 5
28817 202/300 Read block 3626 len 16384
28818 203/300 Write block 2389 len 12288 data: 3 4 5
28819 204/300 Write block 4512 len 36864 data: 7 6 9 8 8 8 8 8 6
28820 205/300 Read block 2747 len 4096
28821 206/300 Read block 4546 len 8192
28822 207/300 Write block 173 len 24576 data: 9 8 7 7 5 4
28823 208/300 Read block 3778 len 28672
28824 209/300 Read block 2435 len 16384
28825 210/300 Read block 1975 len 12288
28826 211/300 Read block 1724 len 16384
28827 212/300 Read block 1426 len 28672
28828 213/300 Read block 2083 len 36864
28829 214/300 Flush
28830 215/300 Write block 2461 len 28672 data: 5 5 5 4 5 4 5
28831 216/300 Read block 3154 len 32768
28832 217/300 Write block 1463 len 28672 data: 4 5 4 3 3 2 2
28833 218/300 Read block 4230 len 32768
28834 219/300 Write block 3342 len 4096 data: 3
28835 220/300 Read block 3930 len 28672
28836 221/300 Write block 4338 len 4096 data: 5
28837 222/300 Write block 1000 len 24576 data: 5 5 4 5 5 5
28838 223/300 Flush
28839 224/300 Read block 1759 len 36864
28840 225/300 Write block 3682 len 16384 data: 5 6 6 4
28841 226/300 Read block 3146 len 28672
28842 227/300 Read block 4424 len 12288
28843 228/300 Write block 550 len 28672 data: 5 4 2 2 2 2 2
28844 229/300 Read block 2738 len 20480
28845 230/300 Write block 4243 len 12288 data: 6 6 4
28846 231/300 Read block 1064 len 12288
28847 232/300 Write block 4267 len 40960 data: 6 8 8 9 9 8 7 7 8 6
28848 233/300 Read block 2285 len 4096
28849 234/300 Write block 405 len 16384 data: 5 4 6 4
28850 235/300 Read block 1850 len 20480
28851 236/300 Read block 1999 len 12288
28852 237/300 Read block 89 len 24576
28853 238/300 Read block 3719 len 40960
28854 239/300 Read block 2849 len 4096
28855 240/300 Flush
28856 241/300 Read block 906 len 24576
28857 242/300 Write block 2476 len 16384 data: 6 5 5 6
28858 243/300 Write block 3156 len 24576 data: 3 4 4 5 5 5
28859 244/300 Flush
28860 245/300 Write block 195 len 36864 data: 2 2 3 3 3 4 4 6 6
28861 246/300 Read block 289 len 32768
28862 247/300 Flush
28863 248/300 Read block 3432 len 24576
28864 249/300 Read block 1936 len 16384
28865 250/300 Flush
28866 251/300 Read block 1135 len 32768
28867 252/300 Write block 813 len 40960 data: 3 3 3 4 4 5 5 4 5 5
28868 253/300 Write block 729 len 24576 data: 6 7 7 8 6 5
28869 254/300 Read block 1697 len 32768
28870 255/300 Read block 1597 len 4096
28871 256/300 Read block 3409 len 40960
28872 257/300 Read block 473 len 8192
28873 258/300 Read block 4689 len 28672
28874 259/300 Write block 4400 len 24576 data: 7 5 6 6 8 7
28875 260/300 Read block 4601 len 28672
28876 261/300 Read block 2964 len 4096
28877 262/300 Read block 3447 len 12288
28878 263/300 Write block 4870 len 28672 data: 5 4 4 5 4 5 4
28879 264/300 Write block 2154 len 24576 data: 6 7 5 4 5 3
28880 265/300 Write block 805 len 20480 data: 5 5 4 5 5
28881 266/300 Write block 290 len 32768 data: 9 9 7 4 3 4 2 2
28882 267/300 Read block 3474 len 36864
28883 268/300 Read block 2096 len 28672
28884 269/300 Read block 2443 len 20480
28885 270/300 Flush
28886 271/300 Read block 2563 len 32768
28887 272/300 Read block 4784 len 28672
28888 273/300 Read block 1333 len 4096
28889 274/300 Read block 4119 len 24576
28890 275/300 Read block 1409 len 12288
28891 276/300 Read block 1216 len 32768
28892 277/300 Write block 153 len 28672 data: 6 7 7 6 5 5 3
28893 278/300 Write block 2198 len 36864 data: 12 9 7 6 5 6 7 8 7
28894 279/300 Flush
28895 280/300 Write block 3046 len 40960 data: 5 5 5 5 6 6 5 4 3 3
28896 281/300 Write block 3110 len 24576 data: 6 6 7 7 5 5
28897 282/300 Write block 645 len 24576 data: 5 6 4 3 5 5
28898 283/300 Read block 3367 len 12288
28899 284/300 Write block 2204 len 8192 data: 8 9
28900 285/300 Read block 3231 len 28672
28901 286/300 Read block 606 len 8192
28902 287/300 Read block 2465 len 16384
28903 288/300 Read block 3060 len 16384
28904 289/300 Read block 4433 len 40960
28905 290/300 Write block 3424 len 40960 data: 5 5 5 3 3 4 2 2 3 3
28906 291/300 Flush
28907 292/300 Write block 3846 len 32768 data: 7 5 5 5 6 7 6 7
28908 293/300 Write block 4855 len 4096 data: 4
28909 294/300 Read block 1226 len 12288
28910 295/300 Write block 1764 len 24576 data: 7 7 5 5 6 6
28911 296/300 Write block 2087 len 12288 data: 7 5 5
28912 297/300 Read block 4525 len 12288
28913 298/300 Write block 3568 len 4096 data: 4
28914 299/300 Write block 2180 len 8192 data: 6 5
28915 300/300 Read block 4069 len 32768
28916 301/300 Write block 1977 len 4096 data: 6
28917 Main task has work to do, go find it
28918 got dsc Start(0)
28919 start 0
28920 Jan 13 15:57:53.812 INFO request completed, latency_us: 179, response_code: 204, uri: /start/cid/0, method: POST, req_id: fe1fc39d-bd71-4a87-b737-9418fb96b12b, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
28921 Replay: started 0, returned:Ok(())
28922 ----------------------------------------------------------------
28923 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
28924 States: Offline Active Active
28925 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28926 8850 Acked Read 8 Sent Done Done false
28927 8851 Acked Read 1 Sent Done Done false
28928 8852 Acked Flush 0 Sent Done Done false
28929 8853 Acked Write 5 Sent Done Done false
28930 8854 Acked Read 9 Sent Done Done false
28931 8855 Acked Read 1 Sent Done Done false
28932 8856 Acked Write 8 Sent Done Done false
28933 8857 Acked Read 10 Sent Done Done false
28934 8858 Acked Write 1 Sent Done Done false
28935 8859 Acked Write 9 Sent Done Done false
28936 8860 Acked Read 2 Sent Done Done false
28937 8861 Acked Write 9 Sent Done Done false
28938 8862 Acked Read 3 Sent Done Done false
28939 8863 Acked Write 2 Sent Done Done false
28940 8864 Acked Read 10 Sent Done Done false
28941 8865 Acked Write 7 Sent Done Done false
28942 8866 Acked Flush 0 Sent Done Done false
28943 8867 Acked Write 7 Sent Done Done false
28944 8868 Acked Read 4 Sent Done Done false
28945 8869 Acked Read 4 Sent Done Done false
28946 8870 Acked Read 2 Sent Done Done false
28947 8871 Acked Read 9 Sent Done Done false
28948 8872 Acked Read 10 Sent Done Done false
28949 8873 Acked Read 1 Sent Done Done false
28950 8874 Acked Read 2 Sent Done Done false
28951 8875 Acked Write 5 Sent Done Done false
28952 8876 Acked Read 7 Sent Done Done false
28953 8877 Acked Read 2 Sent Done Done false
28954 8878 Acked Write 4 Sent Done Done false
28955 8879 Acked Read 7 Sent Done Done false
28956 8880 Acked Flush 0 Sent Done Done false
28957 8881 Acked Write 1 Sent Done Done false
28958 8882 Acked Read 3 Sent Done Done false
28959 8883 Acked Read 9 Sent Done Done false
28960 8884 Acked Read 1 Sent Done Done false
28961 8885 Acked Read 7 Sent Done Done false
28962 8886 Acked Write 8 Sent Done Done false
28963 8887 Acked Read 5 Sent Done Done false
28964 8888 Acked Read 6 Sent Done Done false
28965 8889 Acked Read 2 Sent Done Done false
28966 8890 Acked Write 6 Sent Done Done false
28967 8891 Acked Read 4 Sent Done Done false
28968 8892 Acked Write 4 Sent Done Done false
28969 8893 Acked Flush 0 Sent Done Done false
28970 8894 Acked Read 9 Sent Done Done false
28971 8895 Acked Write 2 Sent Done Done false
28972 8896 Acked Write 8 Sent Done Done false
28973 8897 Acked Read 9 Sent Done Done false
28974 8898 Acked Write 2 Sent Done Done false
28975 8899 Acked Write 5 Sent Done Done false
28976 8900 Acked Read 10 Sent Done Done false
28977 8901 Acked Read 4 Sent Done Done false
28978 8902 Acked Read 2 Sent Done Done false
28979 8903 Acked Write 9 Sent Done Done false
28980 8904 Acked Flush 0 Sent Done Done false
28981 8905 Acked Flush 0 Sent Done Done false
28982 8906 Acked Write 9 Sent Done Done false
28983 8907 Acked Read 2 Sent Done Done false
28984 8908 Acked Read 9 Sent Done Done false
28985 8909 Acked Flush 0 Sent Done Done false
28986 8910 Acked Write 7 Sent Done Done false
28987 8911 Acked Read 4 Sent Done Done false
28988 8912 Acked Read 1 Sent Done Done false
28989 8913 Acked Read 1 Sent Done Done false
28990 8914 Acked Read 5 Sent Done Done false
28991 8915 Acked Flush 0 Sent Done Done false
28992 8916 Acked Flush 0 Sent Done Done false
28993 8917 Acked Write 4 Sent Done Done false
28994 8918 Acked Read 2 Sent Done Done false
28995 8919 Acked Flush 0 Sent Done Done false
28996 8920 Acked Write 10 Sent Done Done false
28997 8921 Acked Read 5 Sent Done Done false
28998 8922 Acked Write 3 Sent Done Done false
28999 8923 Acked Write 9 Sent Done Done false
29000 8924 Acked Read 2 Sent Done Done false
29001 8925 Acked Read 5 Sent Done Done false
29002 8926 Acked Write 1 Sent Done Done false
29003 8927 Acked Read 2 Sent Done Done false
29004 8928 Acked Flush 0 Sent Done Done false
29005 8929 Acked Write 5 Sent Done Done false
29006 8930 Acked Read 4 Sent Done Done false
29007 8931 Acked Write 4 Sent Done Done false
29008 8932 Acked Read 3 Sent Done Done false
29009 8933 Acked Write 7 Sent Done Done false
29010 8934 Acked Read 8 Sent Done Done false
29011 8935 Acked Read 10 Sent Done Done false
29012 8936 Acked Read 8 Sent Done Done false
29013 8937 Acked Write 10 Sent Done Done false
29014 8938 Acked Read 8 Sent Done Done false
29015 8939 Acked Read 6 Sent Done Done false
29016 8940 Acked Write 4 Sent Done Done false
29017 8941 Acked Write 4 Sent Done Done false
29018 8942 Acked Read 3 Sent Done Done false
29019 8943 Acked Flush 0 Sent Done Done false
29020 8944 Acked Read 2 Sent Done Done false
29021 8945 Acked Write 10 Sent Done Done false
29022 8946 Acked Read 9 Sent Done Done false
29023 8947 Acked Read 5 Sent Done Done false
29024 8948 Acked Read 3 Sent Done Done false
29025 8949 Acked Write 3 Sent Done Done false
29026 8950 Acked Flush 0 Sent Done Done false
29027 8951 Acked Write 6 Sent Done Done false
29028 8952 Acked Read 7 Sent Done Done false
29029 8953 Acked Write 2 Sent Done Done false
29030 8954 Acked Flush 0 Sent Done Done false
29031 8955 Acked Read 2 Sent Done Done false
29032 8956 Acked Read 9 Sent Done Done false
29033 8957 Acked Read 10 Sent Done Done false
29034 8958 Acked Write 10 Sent Done Done false
29035 8959 Acked Write 7 Sent Done Done false
29036 8960 Acked Write 7 Sent Done Done false
29037 8961 Acked Write 7 Sent Done Done false
29038 8962 Acked Read 10 Sent Done Done false
29039 8963 Acked Write 2 Sent Done Done false
29040 8964 Acked Write 2 Sent Done Done false
29041 8965 Acked Read 3 Sent Done Done false
29042 8966 Acked Write 5 Sent Done Done false
29043 8967 Acked Read 10 Sent Done Done false
29044 8968 Acked Write 7 Sent Done Done false
29045 8969 Acked Write 2 Sent Done Done false
29046 8970 Acked Read 1 Sent Done Done false
29047 8971 Acked Read 1 Sent Done Done false
29048 8972 Acked Read 7 Sent Done Done false
29049 8973 Acked Write 3 Sent Done Done false
29050 8974 Acked Read 3 Sent Done Done false
29051 8975 Acked Write 10 Sent Done Done false
29052 8976 Acked Flush 0 Sent Done Done false
29053 8977 Acked Read 9 Sent Done Done false
29054 8978 Acked Flush 0 Sent Done Done false
29055 8979 Acked Read 5 Sent Done Done false
29056 8980 Acked Read 10 Sent Done Done false
29057 8981 Acked Write 7 Sent Done Done false
29058 8982 Acked Write 9 Sent Done Done false
29059 8983 Acked Write 5 Sent Done Done false
29060 8984 Acked Read 6 Sent Done Done false
29061 8985 Acked Read 9 Sent Done Done false
29062 8986 Acked Read 5 Sent Done Done false
29063 8987 Acked Read 7 Sent Done Done false
29064 8988 Acked Write 8 Sent Done Done false
29065 8989 Acked Write 8 Sent Done Done false
29066 8990 Acked Write 3 Sent Done Done false
29067 8991 Acked Write 4 Sent Done Done false
29068 8992 Acked Read 2 Sent Done Done false
29069 8993 Acked Read 3 Sent Done Done false
29070 8994 Acked Write 8 Sent Done Done false
29071 8995 Acked Read 8 Sent Done Done false
29072 8996 Acked Write 8 Sent Done Done false
29073 8997 Acked Read 8 Sent Done Done false
29074 8998 Acked Write 2 Sent Done Done false
29075 8999 Acked Read 8 Sent Done Done false
29076 9000 Acked Read 9 Sent Done Done false
29077 9001 Acked Read 2 Sent Done Done false
29078 9002 Acked Read 1 Sent Done Done false
29079 9003 Acked Flush 0 Sent Done Done false
29080 9004 Acked Read 8 Sent Done Done false
29081 9005 Acked Read 7 Sent Done Done false
29082 9006 Acked Read 7 Sent Done Done false
29083 9007 Acked Write 6 Sent Done Done false
29084 9008 Acked Read 1 Sent Done Done false
29085 9009 Acked Read 7 Sent Done Done false
29086 9010 Acked Read 9 Sent Done Done false
29087 9011 Acked Read 7 Sent Done Done false
29088 9012 Acked Read 5 Sent Done Done false
29089 9013 Acked Write 6 Sent Done Done false
29090 9014 Acked Write 6 Sent Done Done false
29091 9015 Acked Write 6 Sent Done Done false
29092 9016 Acked Read 1 Sent Done Done false
29093 9017 Acked Write 3 Sent Done Done false
29094 9018 Acked Read 7 Sent Done Done false
29095 9019 Acked Read 10 Sent Done Done false
29096 9020 Acked Read 10 Sent Done Done false
29097 9021 Acked Write 3 Sent Done Done false
29098 9022 Acked Read 7 Sent Done Done false
29099 9023 Acked Flush 0 Sent Done Done false
29100 9024 Acked Read 8 Sent Done Done false
29101 9025 Acked Read 1 Sent Done Done false
29102 9026 Acked Read 6 Sent Done Done false
29103 9027 Acked Write 5 Sent Done Done false
29104 9028 Acked Flush 0 Sent Done Done false
29105 9029 Acked Read 3 Sent Done Done false
29106 9030 Acked Write 8 Sent Done Done false
29107 9031 Acked Read 10 Sent Done Done false
29108 9032 Acked Flush 0 Sent Done Done false
29109 9033 Acked Read 8 Sent Done Done false
29110 9034 Acked Write 1 Sent Done Done false
29111 9035 Acked Read 9 Sent Done Done false
29112 9036 Acked Write 1 Sent Done Done false
29113 9037 Acked Read 1 Sent Done Done false
29114 9038 Acked Write 8 Sent Done Done false
29115 9039 Acked Read 6 Sent Done Done false
29116 9040 Acked Read 1 Sent Done Done false
29117 9041 Acked Write 2 Sent Done Done false
29118 9042 Acked Write 6 Sent Done Done false
29119 9043 Acked Read 7 Sent Done Done false
29120 9044 Acked Write 8 Sent Done Done false
29121 9045 Acked Write 1 Sent Done Done false
29122 9046 Acked Write 7 Sent Done Done false
29123 9047 Acked Write 3 Sent Done Done false
29124 9048 Acked Read 3 Sent Done Done false
29125 9049 Acked Write 10 Sent Done Done false
29126 9050 Acked Write 3 Sent Done Done false
29127 9051 Acked Read 4 Sent Done Done false
29128 9052 Acked Write 3 Sent Done Done false
29129 9053 Acked Write 9 Sent Done Done false
29130 9054 Acked Read 1 Sent Done Done false
29131 9055 Acked Read 2 Sent Done Done false
29132 9056 Acked Write 6 Sent Done Done false
29133 9057 Acked Read 7 Sent Done Done false
29134 9058 Acked Read 4 Sent Done Done false
29135 9059 Acked Read 3 Sent Done Done false
29136 9060 Acked Read 4 Sent Done Done false
29137 9061 Acked Read 7 Sent Done Done false
29138 9062 Acked Read 9 Sent Done Done false
29139 9063 Acked Flush 0 Sent Done Done false
29140 9064 Acked Write 7 Sent Done Done false
29141 9065 Acked Read 8 Sent Done Done false
29142 9066 Acked Write 7 Sent Done Done false
29143 9067 Acked Read 8 Sent Done Done false
29144 9068 Acked Write 1 Sent Done Done false
29145 9069 Acked Read 7 Sent Done Done false
29146 9070 Acked Write 1 Sent Done Done false
29147 9071 Acked Write 6 Sent Done Done false
29148 9072 Acked Flush 0 Sent Done Done false
29149 9073 Acked Read 9 Sent Done Done false
29150 9074 Acked Write 4 Sent Done Done false
29151 9075 Acked Read 7 Sent Done Done false
29152 9076 Acked Read 3 Sent Done Done false
29153 9077 Acked Write 7 Sent Done Done false
29154 9078 Acked Read 5 Sent Done Done false
29155 9079 Acked Write 3 Sent Done Done false
29156 9080 Acked Read 3 Sent Done Done false
29157 9081 Acked Write 10 Sent Done Done false
29158 9082 Acked Read 1 Sent Done Done false
29159 9083 Acked Write 4 Sent Done Done false
29160 9084 Acked Read 5 Sent Done Done false
29161 9085 Acked Read 3 Sent Done Done false
29162 9086 Acked Read 6 Sent Done Done false
29163 9087 Acked Read 10 Sent Done Done false
29164 9088 Acked Read 1 Sent Done Done false
29165 9089 Acked Flush 0 Sent Done Done false
29166 9090 Acked Read 6 Sent Done Done false
29167 9091 Acked Write 4 Sent Done Done false
29168 9092 Acked Write 6 Sent Done Done false
29169 9093 Acked Flush 0 Sent Done Done false
29170 9094 Acked Write 9 Sent Done Done false
29171 9095 Acked Read 8 Sent Done Done false
29172 9096 Acked Flush 0 Sent Done Done false
29173 9097 Acked Read 6 Sent Done Done false
29174 9098 Acked Read 4 Sent Done Done false
29175 9099 Acked Flush 0 Sent Done Done false
29176 9100 Acked Read 8 Sent Done Done false
29177 9101 Acked Write 10 Sent Done Done false
29178 9102 Acked Write 6 Sent Done Done false
29179 9103 Acked Read 8 Sent Done Done false
29180 9104 Acked Read 1 Sent Done Done false
29181 9105 Acked Read 10 Sent Done Done false
29182 9106 Acked Read 2 Sent Done Done false
29183 9107 Acked Read 7 Sent Done Done false
29184 9108 Acked Write 6 Sent Done Done false
29185 9109 Acked Read 7 Sent Done Done false
29186 9110 Acked Read 1 Sent Done Done false
29187 9111 Acked Read 3 Sent Done Done false
29188 9112 Acked Write 7 Sent Done Done false
29189 9113 Acked Write 6 Sent Done Done false
29190 9114 Acked Write 5 Sent Done Done false
29191 9115 Acked Write 8 Sent Done Done false
29192 9116 Acked Read 9 Sent Done Done false
29193 9117 Acked Read 7 Sent Done Done false
29194 9118 Acked Read 5 Sent Done Done false
29195 9119 Acked Flush 0 Sent Done Done false
29196 9120 Acked Read 8 Sent Done Done false
29197 9121 Acked Read 7 Sent Done Done false
29198 9122 Acked Read 1 Sent Done Done false
29199 9123 Acked Read 6 Sent Done Done false
29200 9124 Acked Read 3 Sent Done Done false
29201 9125 Acked Read 8 Sent Done Done false
29202 9126 Acked Write 7 Sent Done Done false
29203 9127 Acked Write 9 Sent Done Done false
29204 9128 Acked Flush 0 Sent Done Done false
29205 9129 Acked Write 10 Sent Done Done false
29206 9130 Acked Write 6 Sent Done Done false
29207 9131 Acked Write 6 Sent Done Done false
29208 9132 Acked Read 3 Sent Done Done false
29209 9133 Acked Write 2 Sent Done Done false
29210 9134 Acked Read 7 Sent Done Done false
29211 9135 Acked Read 2 Sent Done Done false
29212 9136 Acked Read 4 Sent Done Done false
29213 9137 Acked Read 4 Sent Done Done false
29214 9138 Acked Read 10 Sent Done Done false
29215 9139 Acked Write 10 Sent Done Done false
29216 9140 Acked Flush 0 Sent Done Done false
29217 9141 Acked Write 8 Sent Done Done false
29218 9142 Acked Write 1 Sent Done Done false
29219 9143 Acked Read 3 Sent Done Done false
29220 9144 Acked Write 6 Sent Done Done false
29221 9145 Acked Write 3 Sent Done Done false
29222 9146 Acked Read 3 Sent Done Done false
29223 9147 Acked Write 1 Sent Done Done false
29224 9148 Acked Write 2 Sent Done Done false
29225 9149 Acked Read 8 Sent Done Done false
29226 9150 Acked Write 1 Sent Done Done false
29227 STATES DS:0 DS:1 DS:2 TOTAL
29228 Sent 301 0 0 301
29229 Done 0 301 301 602
29230 Skipped 0 0 0 0
29231 Error 0 0 0 0
29232 Last Flush: 8849 9140 9140
29233 Downstairs last five completed: 8849 8848 8847 8846 8845
29234 Upstairs last five completed: 9150 9149 9148 9147 9146
29235 CLIENT: Up:0 ds:301 act:2
29236 [8810] got start action
29237 [8810] I am going to restart
29238 Starting downstairs at port 8810
29239 Make output file at "/tmp/dsc/downstairs-8810.txt"
29240 [8810][0] reports Starting
29241 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1229)
29242 [8810][0] reports Running
29243 ----------------------------------------------------------------
29244 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29245 States: Offline Active Active
29246 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29247 8850 Acked Read 8 Sent Done Done false
29248 8851 Acked Read 1 Sent Done Done false
29249 8852 Acked Flush 0 Sent Done Done false
29250 8853 Acked Write 5 Sent Done Done false
29251 8854 Acked Read 9 Sent Done Done false
29252 8855 Acked Read 1 Sent Done Done false
29253 8856 Acked Write 8 Sent Done Done false
29254 8857 Acked Read 10 Sent Done Done false
29255 8858 Acked Write 1 Sent Done Done false
29256 8859 Acked Write 9 Sent Done Done false
29257 8860 Acked Read 2 Sent Done Done false
29258 8861 Acked Write 9 Sent Done Done false
29259 8862 Acked Read 3 Sent Done Done false
29260 8863 Acked Write 2 Sent Done Done false
29261 8864 Acked Read 10 Sent Done Done false
29262 8865 Acked Write 7 Sent Done Done false
29263 8866 Acked Flush 0 Sent Done Done false
29264 8867 Acked Write 7 Sent Done Done false
29265 8868 Acked Read 4 Sent Done Done false
29266 8869 Acked Read 4 Sent Done Done false
29267 8870 Acked Read 2 Sent Done Done false
29268 8871 Acked Read 9 Sent Done Done false
29269 8872 Acked Read 10 Sent Done Done false
29270 8873 Acked Read 1 Sent Done Done false
29271 8874 Acked Read 2 Sent Done Done false
29272 8875 Acked Write 5 Sent Done Done false
29273 8876 Acked Read 7 Sent Done Done false
29274 8877 Acked Read 2 Sent Done Done false
29275 8878 Acked Write 4 Sent Done Done false
29276 8879 Acked Read 7 Sent Done Done false
29277 8880 Acked Flush 0 Sent Done Done false
29278 8881 Acked Write 1 Sent Done Done false
29279 8882 Acked Read 3 Sent Done Done false
29280 8883 Acked Read 9 Sent Done Done false
29281 8884 Acked Read 1 Sent Done Done false
29282 8885 Acked Read 7 Sent Done Done false
29283 8886 Acked Write 8 Sent Done Done false
29284 8887 Acked Read 5 Sent Done Done false
29285 8888 Acked Read 6 Sent Done Done false
29286 8889 Acked Read 2 Sent Done Done false
29287 8890 Acked Write 6 Sent Done Done false
29288 8891 Acked Read 4 Sent Done Done false
29289 8892 Acked Write 4 Sent Done Done false
29290 8893 Acked Flush 0 Sent Done Done false
29291 8894 Acked Read 9 Sent Done Done false
29292 8895 Acked Write 2 Sent Done Done false
29293 8896 Acked Write 8 Sent Done Done false
29294 8897 Acked Read 9 Sent Done Done false
29295 8898 Acked Write 2 Sent Done Done false
29296 8899 Acked Write 5 Sent Done Done false
29297 8900 Acked Read 10 Sent Done Done false
29298 8901 Acked Read 4 Sent Done Done false
29299 8902 Acked Read 2 Sent Done Done false
29300 8903 Acked Write 9 Sent Done Done false
29301 8904 Acked Flush 0 Sent Done Done false
29302 8905 Acked Flush 0 Sent Done Done false
29303 8906 Acked Write 9 Sent Done Done false
29304 8907 Acked Read 2 Sent Done Done false
29305 8908 Acked Read 9 Sent Done Done false
29306 8909 Acked Flush 0 Sent Done Done false
29307 8910 Acked Write 7 Sent Done Done false
29308 8911 Acked Read 4 Sent Done Done false
29309 8912 Acked Read 1 Sent Done Done false
29310 8913 Acked Read 1 Sent Done Done false
29311 8914 Acked Read 5 Sent Done Done false
29312 8915 Acked Flush 0 Sent Done Done false
29313 8916 Acked Flush 0 Sent Done Done false
29314 8917 Acked Write 4 Sent Done Done false
29315 8918 Acked Read 2 Sent Done Done false
29316 8919 Acked Flush 0 Sent Done Done false
29317 8920 Acked Write 10 Sent Done Done false
29318 8921 Acked Read 5 Sent Done Done false
29319 8922 Acked Write 3 Sent Done Done false
29320 8923 Acked Write 9 Sent Done Done false
29321 8924 Acked Read 2 Sent Done Done false
29322 8925 Acked Read 5 Sent Done Done false
29323 8926 Acked Write 1 Sent Done Done false
29324 8927 Acked Read 2 Sent Done Done false
29325 8928 Acked Flush 0 Sent Done Done false
29326 8929 Acked Write 5 Sent Done Done false
29327 8930 Acked Read 4 Sent Done Done false
29328 8931 Acked Write 4 Sent Done Done false
29329 8932 Acked Read 3 Sent Done Done false
29330 8933 Acked Write 7 Sent Done Done false
29331 8934 Acked Read 8 Sent Done Done false
29332 8935 Acked Read 10 Sent Done Done false
29333 8936 Acked Read 8 Sent Done Done false
29334 8937 Acked Write 10 Sent Done Done false
29335 8938 Acked Read 8 Sent Done Done false
29336 8939 Acked Read 6 Sent Done Done false
29337 8940 Acked Write 4 Sent Done Done false
29338 8941 Acked Write 4 Sent Done Done false
29339 8942 Acked Read 3 Sent Done Done false
29340 8943 Acked Flush 0 Sent Done Done false
29341 8944 Acked Read 2 Sent Done Done false
29342 8945 Acked Write 10 Sent Done Done false
29343 8946 Acked Read 9 Sent Done Done false
29344 8947 Acked Read 5 Sent Done Done false
29345 8948 Acked Read 3 Sent Done Done false
29346 8949 Acked Write 3 Sent Done Done false
29347 8950 Acked Flush 0 Sent Done Done false
29348 8951 Acked Write 6 Sent Done Done false
29349 8952 Acked Read 7 Sent Done Done false
29350 8953 Acked Write 2 Sent Done Done false
29351 8954 Acked Flush 0 Sent Done Done false
29352 8955 Acked Read 2 Sent Done Done false
29353 8956 Acked Read 9 Sent Done Done false
29354 8957 Acked Read 10 Sent Done Done false
29355 8958 Acked Write 10 Sent Done Done false
29356 8959 Acked Write 7 Sent Done Done false
29357 8960 Acked Write 7 Sent Done Done false
29358 8961 Acked Write 7 Sent Done Done false
29359 8962 Acked Read 10 Sent Done Done false
29360 8963 Acked Write 2 Sent Done Done false
29361 8964 Acked Write 2 Sent Done Done false
29362 8965 Acked Read 3 Sent Done Done false
29363 8966 Acked Write 5 Sent Done Done false
29364 8967 Acked Read 10 Sent Done Done false
29365 8968 Acked Write 7 Sent Done Done false
29366 8969 Acked Write 2 Sent Done Done false
29367 8970 Acked Read 1 Sent Done Done false
29368 8971 Acked Read 1 Sent Done Done false
29369 8972 Acked Read 7 Sent Done Done false
29370 8973 Acked Write 3 Sent Done Done false
29371 8974 Acked Read 3 Sent Done Done false
29372 8975 Acked Write 10 Sent Done Done false
29373 8976 Acked Flush 0 Sent Done Done false
29374 8977 Acked Read 9 Sent Done Done false
29375 8978 Acked Flush 0 Sent Done Done false
29376 8979 Acked Read 5 Sent Done Done false
29377 8980 Acked Read 10 Sent Done Done false
29378 8981 Acked Write 7 Sent Done Done false
29379 8982 Acked Write 9 Sent Done Done false
29380 8983 Acked Write 5 Sent Done Done false
29381 8984 Acked Read 6 Sent Done Done false
29382 8985 Acked Read 9 Sent Done Done false
29383 8986 Acked Read 5 Sent Done Done false
29384 8987 Acked Read 7 Sent Done Done false
29385 8988 Acked Write 8 Sent Done Done false
29386 8989 Acked Write 8 Sent Done Done false
29387 8990 Acked Write 3 Sent Done Done false
29388 8991 Acked Write 4 Sent Done Done false
29389 8992 Acked Read 2 Sent Done Done false
29390 8993 Acked Read 3 Sent Done Done false
29391 8994 Acked Write 8 Sent Done Done false
29392 8995 Acked Read 8 Sent Done Done false
29393 8996 Acked Write 8 Sent Done Done false
29394 8997 Acked Read 8 Sent Done Done false
29395 8998 Acked Write 2 Sent Done Done false
29396 8999 Acked Read 8 Sent Done Done false
29397 9000 Acked Read 9 Sent Done Done false
29398 9001 Acked Read 2 Sent Done Done false
29399 9002 Acked Read 1 Sent Done Done false
29400 9003 Acked Flush 0 Sent Done Done false
29401 9004 Acked Read 8 Sent Done Done false
29402 9005 Acked Read 7 Sent Done Done false
29403 9006 Acked Read 7 Sent Done Done false
29404 9007 Acked Write 6 Sent Done Done false
29405 9008 Acked Read 1 Sent Done Done false
29406 9009 Acked Read 7 Sent Done Done false
29407 9010 Acked Read 9 Sent Done Done false
29408 9011 Acked Read 7 Sent Done Done false
29409 9012 Acked Read 5 Sent Done Done false
29410 9013 Acked Write 6 Sent Done Done false
29411 9014 Acked Write 6 Sent Done Done false
29412 9015 Acked Write 6 Sent Done Done false
29413 9016 Acked Read 1 Sent Done Done false
29414 9017 Acked Write 3 Sent Done Done false
29415 9018 Acked Read 7 Sent Done Done false
29416 9019 Acked Read 10 Sent Done Done false
29417 9020 Acked Read 10 Sent Done Done false
29418 9021 Acked Write 3 Sent Done Done false
29419 9022 Acked Read 7 Sent Done Done false
29420 9023 Acked Flush 0 Sent Done Done false
29421 9024 Acked Read 8 Sent Done Done false
29422 9025 Acked Read 1 Sent Done Done false
29423 9026 Acked Read 6 Sent Done Done false
29424 9027 Acked Write 5 Sent Done Done false
29425 9028 Acked Flush 0 Sent Done Done false
29426 9029 Acked Read 3 Sent Done Done false
29427 9030 Acked Write 8 Sent Done Done false
29428 9031 Acked Read 10 Sent Done Done false
29429 9032 Acked Flush 0 Sent Done Done false
29430 9033 Acked Read 8 Sent Done Done false
29431 9034 Acked Write 1 Sent Done Done false
29432 9035 Acked Read 9 Sent Done Done false
29433 9036 Acked Write 1 Sent Done Done false
29434 9037 Acked Read 1 Sent Done Done false
29435 9038 Acked Write 8 Sent Done Done false
29436 9039 Acked Read 6 Sent Done Done false
29437 9040 Acked Read 1 Sent Done Done false
29438 9041 Acked Write 2 Sent Done Done false
29439 9042 Acked Write 6 Sent Done Done false
29440 9043 Acked Read 7 Sent Done Done false
29441 9044 Acked Write 8 Sent Done Done false
29442 9045 Acked Write 1 Sent Done Done false
29443 9046 Acked Write 7 Sent Done Done false
29444 9047 Acked Write 3 Sent Done Done false
29445 9048 Acked Read 3 Sent Done Done false
29446 9049 Acked Write 10 Sent Done Done false
29447 9050 Acked Write 3 Sent Done Done false
29448 9051 Acked Read 4 Sent Done Done false
29449 9052 Acked Write 3 Sent Done Done false
29450 9053 Acked Write 9 Sent Done Done false
29451 9054 Acked Read 1 Sent Done Done false
29452 9055 Acked Read 2 Sent Done Done false
29453 9056 Acked Write 6 Sent Done Done false
29454 9057 Acked Read 7 Sent Done Done false
29455 9058 Acked Read 4 Sent Done Done false
29456 9059 Acked Read 3 Sent Done Done false
29457 9060 Acked Read 4 Sent Done Done false
29458 9061 Acked Read 7 Sent Done Done false
29459 9062 Acked Read 9 Sent Done Done false
29460 9063 Acked Flush 0 Sent Done Done false
29461 9064 Acked Write 7 Sent Done Done false
29462 9065 Acked Read 8 Sent Done Done false
29463 9066 Acked Write 7 Sent Done Done false
29464 9067 Acked Read 8 Sent Done Done false
29465 9068 Acked Write 1 Sent Done Done false
29466 9069 Acked Read 7 Sent Done Done false
29467 9070 Acked Write 1 Sent Done Done false
29468 9071 Acked Write 6 Sent Done Done false
29469 9072 Acked Flush 0 Sent Done Done false
29470 9073 Acked Read 9 Sent Done Done false
29471 9074 Acked Write 4 Sent Done Done false
29472 9075 Acked Read 7 Sent Done Done false
29473 9076 Acked Read 3 Sent Done Done false
29474 9077 Acked Write 7 Sent Done Done false
29475 9078 Acked Read 5 Sent Done Done false
29476 9079 Acked Write 3 Sent Done Done false
29477 9080 Acked Read 3 Sent Done Done false
29478 9081 Acked Write 10 Sent Done Done false
29479 9082 Acked Read 1 Sent Done Done false
29480 9083 Acked Write 4 Sent Done Done false
29481 9084 Acked Read 5 Sent Done Done false
29482 9085 Acked Read 3 Sent Done Done false
29483 9086 Acked Read 6 Sent Done Done false
29484 9087 Acked Read 10 Sent Done Done false
29485 9088 Acked Read 1 Sent Done Done false
29486 9089 Acked Flush 0 Sent Done Done false
29487 9090 Acked Read 6 Sent Done Done false
29488 9091 Acked Write 4 Sent Done Done false
29489 9092 Acked Write 6 Sent Done Done false
29490 9093 Acked Flush 0 Sent Done Done false
29491 9094 Acked Write 9 Sent Done Done false
29492 9095 Acked Read 8 Sent Done Done false
29493 9096 Acked Flush 0 Sent Done Done false
29494 9097 Acked Read 6 Sent Done Done false
29495 9098 Acked Read 4 Sent Done Done false
29496 9099 Acked Flush 0 Sent Done Done false
29497 9100 Acked Read 8 Sent Done Done false
29498 9101 Acked Write 10 Sent Done Done false
29499 9102 Acked Write 6 Sent Done Done false
29500 9103 Acked Read 8 Sent Done Done false
29501 9104 Acked Read 1 Sent Done Done false
29502 9105 Acked Read 10 Sent Done Done false
29503 9106 Acked Read 2 Sent Done Done false
29504 9107 Acked Read 7 Sent Done Done false
29505 9108 Acked Write 6 Sent Done Done false
29506 9109 Acked Read 7 Sent Done Done false
29507 9110 Acked Read 1 Sent Done Done false
29508 9111 Acked Read 3 Sent Done Done false
29509 9112 Acked Write 7 Sent Done Done false
29510 9113 Acked Write 6 Sent Done Done false
29511 9114 Acked Write 5 Sent Done Done false
29512 9115 Acked Write 8 Sent Done Done false
29513 9116 Acked Read 9 Sent Done Done false
29514 9117 Acked Read 7 Sent Done Done false
29515 9118 Acked Read 5 Sent Done Done false
29516 9119 Acked Flush 0 Sent Done Done false
29517 9120 Acked Read 8 Sent Done Done false
29518 9121 Acked Read 7 Sent Done Done false
29519 9122 Acked Read 1 Sent Done Done false
29520 9123 Acked Read 6 Sent Done Done false
29521 9124 Acked Read 3 Sent Done Done false
29522 9125 Acked Read 8 Sent Done Done false
29523 9126 Acked Write 7 Sent Done Done false
29524 9127 Acked Write 9 Sent Done Done false
29525 9128 Acked Flush 0 Sent Done Done false
29526 9129 Acked Write 10 Sent Done Done false
29527 9130 Acked Write 6 Sent Done Done false
29528 9131 Acked Write 6 Sent Done Done false
29529 9132 Acked Read 3 Sent Done Done false
29530 9133 Acked Write 2 Sent Done Done false
29531 9134 Acked Read 7 Sent Done Done false
29532 9135 Acked Read 2 Sent Done Done false
29533 9136 Acked Read 4 Sent Done Done false
29534 9137 Acked Read 4 Sent Done Done false
29535 9138 Acked Read 10 Sent Done Done false
29536 9139 Acked Write 10 Sent Done Done false
29537 9140 Acked Flush 0 Sent Done Done false
29538 9141 Acked Write 8 Sent Done Done false
29539 9142 Acked Write 1 Sent Done Done false
29540 9143 Acked Read 3 Sent Done Done false
29541 9144 Acked Write 6 Sent Done Done false
29542 9145 Acked Write 3 Sent Done Done false
29543 9146 Acked Read 3 Sent Done Done false
29544 9147 Acked Write 1 Sent Done Done false
29545 9148 Acked Write 2 Sent Done Done false
29546 9149 Acked Read 8 Sent Done Done false
29547 9150 Acked Write 1 Sent Done Done false
29548 9151 Acked Flush 0 Sent Done Done false
29549 STATES DS:0 DS:1 DS:2 TOTAL
29550 Sent 302 0 0 302
29551 Done 0 302 302 604
29552 Skipped 0 0 0 0
29553 Error 0 0 0 0
29554 Last Flush: 8849 9151 9151
29555 Downstairs last five completed: 8849 8848 8847 8846 8845
29556 Upstairs last five completed: 9151 9150 9149 9148 9147
29557 CLIENT: Up:0 ds:302 act:2
295582025-01-13T15:57:59.567ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
295592025-01-13T15:57:59.567ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39075) = io task client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
295602025-01-13T15:57:59.569ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
295612025-01-13T15:57:59.569ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
295622025-01-13T15:57:59.569ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
295632025-01-13T15:57:59.569ZINFOcrucible: send last flush ID to this DS: 8849 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
295642025-01-13T15:57:59.570ZINFOcrucible: Replied this last flush ID: 8849 = downstairs client = 0 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
295652025-01-13T15:57:59.570ZINFOcrucible: [0] client re-new 302 jobs since flush 8849 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
295662025-01-13T15:57:59.570ZINFOcrucible: [0] Replayed 302 jobs since flush: 8849 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
29567 ----------------------------------------------------------------
29568 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
29569 Downstairs last five completed: 9151 9150 9149 9148 9147
29570 Upstairs last five completed: 9151 9150 9149 9148 9147
29571 CLIENT: Up:0 ds:0 act:3
29572 Replay: All jobs finished
29573 Main task has work to do, go find it
29574 got dsc Stop(1)
29575 stop 1
29576 Jan 13 15:58:01.820[8820] Got stop action so:false kr:false
29577 INFO request completed, latency_us: 662, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 30e11ade-6850-4e2d-8bf7-68e88f64c6c2, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
295782025-01-13T15:58:01.821ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
295792025-01-13T15:58:01.821ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
295802025-01-13T15:58:01.821ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
29581 Jan 13 15:58:01.821 INFO request completed, latency_us: 151, response_code: 200, uri: /state/cid/1, method: GET, req_id: 1f03b40c-2ded-4dfd-a8f1-4a38cdd1d086, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
295822025-01-13T15:58:01.821ZINFOcrucible: client task is exiting = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
29583 [8820] Exited with: ExitStatus(unix_wait_status(9))
29584 [8820][1] reports Exit
29585 Jan 13 15:58:05.823 INFO request completed, latency_us: 381, response_code: 200, uri: /state/cid/1, method: GET, req_id: 53617433-8199-43bb-bebc-b23660b7162e, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
29586 001/300 Read block 4351 len 12288
29587 002/300 Flush
29588 003/300 Read block 4842 len 20480
29589 004/300 Write block 2865 len 12288 data: 9 8 9
29590 005/300 Read block 4569 len 12288
29591 006/300 Read block 4864 len 12288
29592 007/300 Read block 3301 len 16384
29593 008/300 Read block 1735 len 28672
29594 009/300 Read block 865 len 28672
29595 010/300 Read block 4691 len 24576
29596 011/300 Write block 304 len 28672 data: 7 7 6 3 3 3 4
29597 012/300 Write block 3057 len 28672 data: 4 4 3 3 3 4 5
29598 013/300 Read block 341 len 24576
29599 014/300 Read block 2323 len 16384
29600 015/300 Write block 342 len 36864 data: 3 5 5 4 4 4 5 5 6
29601 016/300 Read block 1904 len 12288
29602 017/300 Read block 71 len 4096
29603 018/300 Read block 46 len 8192
29604 019/300 Read block 3796 len 8192
29605 020/300 Write block 1333 len 36864 data: 6 8 10 12 10 10 8 7 6
29606 021/300 Write block 2470 len 28672 data: 4 4 5 5 6 7 7
29607 022/300 Write block 1679 len 32768 data: 3 3 3 3 3 2 3 4
29608 023/300 Read block 2084 len 32768
29609 024/300 Read block 452 len 32768
29610 025/300 Read block 4422 len 24576
29611 026/300 Write block 4280 len 20480 data: 5 6 6 5 7
29612 027/300 Flush
29613 028/300 Read block 4373 len 36864
29614 029/300 Write block 4907 len 36864 data: 3 4 3 4 5 7 6 5 4
29615 030/300 Write block 4932 len 4096 data: 10
29616 031/300 Read block 4495 len 16384
29617 032/300 Read block 4424 len 8192
29618 033/300 Write block 2969 len 8192 data: 6 6
29619 034/300 Read block 3625 len 4096
29620 035/300 Flush
29621 036/300 Read block 4144 len 28672
29622 037/300 Write block 82 len 4096 data: 8
29623 038/300 Read block 695 len 12288
29624 039/300 Read block 483 len 24576
29625 040/300 Write block 3202 len 20480 data: 7 6 6 4 5
29626 041/300 Write block 2 len 40960 data: 4 5 7 8 6 5 5 7 5 4
29627 042/300 Read block 1381 len 16384
29628 043/300 Read block 2682 len 40960
29629 044/300 Write block 1411 len 24576 data: 7 9 8 8 9 6
29630 045/300 Read block 2058 len 20480
29631 046/300 Write block 3688 len 28672 data: 6 5 5 5 4 3 3
29632 047/300 Write block 2993 len 12288 data: 8 6 5
29633 048/300 Write block 1592 len 40960 data: 7 4 4 4 3 3 3 4 3 4
29634 049/300 Write block 712 len 4096 data: 5
29635 050/300 Read block 4445 len 20480
29636 051/300 Write block 1431 len 32768 data: 7 6 6 5 8 8 10 8
29637 052/300 Read block 4753 len 36864
29638 053/300 Read block 1615 len 12288
29639 054/300 Read block 607 len 16384
29640 055/300 Read block 1022 len 24576
29641 056/300 Write block 843 len 20480 data: 5 4 4 4 4
29642 057/300 Read block 4307 len 32768
29643 058/300 Read block 888 len 24576
29644 059/300 Write block 1569 len 40960 data: 5 7 7 6 5 5 5 5 7 6
29645 060/300 Read block 4921 len 32768
29646 061/300 Read block 4917 len 20480
29647 062/300 Read block 4757 len 32768
29648 063/300 Write block 2704 len 36864 data: 8 8 5 3 4 3 3 4 4
29649 064/300 Read block 1999 len 40960
29650 065/300 Read block 1271 len 4096
29651 066/300 Write block 1368 len 40960 data: 5 4 2 2 3 5 5 4 5 7
29652 067/300 Write block 317 len 16384 data: 4 5 4 4
29653 068/300 Write block 2416 len 28672 data: 7 7 8 7 4 3 6
29654 069/300 Flush
29655 070/300 Read block 1391 len 36864
29656 071/300 Write block 4943 len 12288 data: 7 8 7
29657 072/300 Read block 736 len 4096
29658 073/300 Read block 682 len 8192
29659 074/300 Read block 1577 len 36864
29660 075/300 Write block 871 len 4096 data: 5
29661 076/300 Write block 4865 len 16384 data: 7 9 8 6
29662 077/300 Flush
29663 078/300 Read block 3437 len 28672
29664 079/300 Write block 4301 len 12288 data: 10 10 11
29665 080/300 Write block 282 len 32768 data: 8 9 10 12 12 11 10 8
29666 081/300 Flush
29667 082/300 Read block 4803 len 32768
29668 083/300 Read block 4548 len 20480
29669 084/300 Write block 292 len 32768 data: 8 5 4 5 3 3 2 3
29670 085/300 Flush
29671 086/300 Write block 3538 len 8192 data: 3 3
29672 087/300 Write block 2977 len 32768 data: 6 4 3 4 5 5 5 5
29673 088/300 Read block 354 len 16384
29674 089/300 Read block 3331 len 32768
29675 090/300 Read block 242 len 12288
29676 091/300 Write block 629 len 4096 data: 4
29677 092/300 Write block 1985 len 28672 data: 5 5 4 4 3 3 5
29678 093/300 Read block 2155 len 28672
29679 094/300 Write block 2602 len 32768 data: 6 8 9 10 9 9 8 8
29680 095/300 Read block 2513 len 8192
29681 096/300 Write block 1973 len 8192 data: 7 7
29682 097/300 Read block 135 len 32768
29683 098/300 Write block 4310 len 16384 data: 5 4 3 3
29684 099/300 Write block 1442 len 20480 data: 8 7 6 4 4
29685 100/300 Read block 519 len 28672
29686 101/300 Write block 560 len 32768 data: 5 4 4 4 4 3 2 3
29687 102/300 Write block 1289 len 8192 data: 3 4
29688 103/300 Read block 2044 len 28672
29689 104/300 Read block 254 len 4096
29690 105/300 Write block 685 len 40960 data: 5 5 7 7 8 7 6 6 6 7
29691 106/300 Write block 1660 len 16384 data: 5 5 5 6
29692 107/300 Write block 1593 len 8192 data: 5 5
29693 108/300 Write block 3951 len 36864 data: 4 5 6 6 5 6 6 6 7
29694 109/300 Write block 1991 len 12288 data: 6 5 6
29695 110/300 Write block 3187 len 16384 data: 8 9 12 12
29696 111/300 Read block 4019 len 12288
29697 112/300 Read block 2250 len 36864
29698 113/300 Read block 3492 len 12288
29699 114/300 Read block 108 len 24576
29700 115/300 Flush
29701 116/300 Read block 2952 len 36864
29702 117/300 Read block 177 len 32768
29703 118/300 Read block 4333 len 8192
29704 119/300 Read block 3144 len 36864
29705 120/300 Write block 2921 len 12288 data: 3 3 2
29706 121/300 Flush
29707 122/300 Read block 2325 len 32768
29708 123/300 Read block 4130 len 20480
29709 124/300 Read block 1465 len 8192
29710 125/300 Read block 3603 len 12288
29711 126/300 Write block 90 len 40960 data: 3 5 6 6 5 6 5 4 4 4
29712 127/300 Write block 2444 len 32768 data: 7 5 6 6 7 5 5 5
29713 128/300 Write block 1745 len 20480 data: 11 8 7 5 7
29714 129/300 Write block 3727 len 40960 data: 10 10 10 6 5 3 3 4 4 5
29715 130/300 Write block 1852 len 36864 data: 6 7 7 8 8 8 6 3 3
29716 131/300 Write block 4167 len 36864 data: 5 4 4 4 5 8 7 5 6
29717 132/300 Read block 1881 len 32768
29718 133/300 Write block 1531 len 8192 data: 5 3
29719 134/300 Read block 1113 len 8192
29720 135/300 Read block 3751 len 28672
29721 136/300 Read block 4305 len 12288
29722 137/300 Read block 2630 len 32768
29723 138/300 Read block 1440 len 12288
29724 139/300 Read block 441 len 20480
29725 140/300 Read block 4955 len 40960
29726 141/300 Write block 513 len 16384 data: 7 6 6 6
29727 142/300 Write block 3817 len 8192 data: 6 7
29728 143/300 Read block 942 len 24576
29729 144/300 Read block 438 len 28672
29730 145/300 Write block 2002 len 24576 data: 7 7 7 7 6 6
29731 146/300 Write block 1322 len 32768 data: 5 4 4 5 6 6 7 5
29732 147/300 Read block 1490 len 16384
29733 148/300 Read block 41 len 24576
29734 149/300 Read block 513 len 20480
29735 150/300 Write block 1933 len 36864 data: 7 6 6 8 8 6 5 4 4
29736 151/300 Read block 4046 len 28672
29737 152/300 Read block 1265 len 12288
29738 153/300 Write block 1627 len 16384 data: 6 6 4 4
29739 154/300 Write block 1589 len 24576 data: 7 8 9 8 6 6
29740 155/300 Flush
29741 156/300 Flush
29742 157/300 Write block 1320 len 24576 data: 7 7 6 5 5 6
29743 158/300 Write block 1397 len 40960 data: 3 5 5 5 4 3 3 3 3 3
29744 159/300 Write block 3284 len 8192 data: 8 8
29745 160/300 Read block 1353 len 40960
29746 161/300 Flush
29747 162/300 Write block 4736 len 28672 data: 10 10 9 8 8 6 5
29748 163/300 Write block 16 len 8192 data: 4 4
29749 164/300 Read block 3809 len 4096
29750 165/300 Write block 3002 len 40960 data: 7 6 7 7 7 7 6 7 8 8
29751 166/300 Write block 3980 len 32768 data: 9 9 7 7 9 8 8 7
29752 167/300 Read block 4733 len 12288
29753 168/300 Read block 4434 len 20480
29754 169/300 Read block 4020 len 36864
29755 170/300 Flush
29756 171/300 Write block 99 len 12288 data: 5 5 5
29757 172/300 Write block 4354 len 40960 data: 5 5 5 5 5 6 2 2 2 3
29758 173/300 Read block 3173 len 12288
29759 174/300 Write block 912 len 32768 data: 3 2 2 3 4 3 3 3
29760 175/300 Write block 1501 len 12288 data: 7 7 5
29761 176/300 Read block 2559 len 32768
29762 177/300 Flush
29763 178/300 Read block 3138 len 20480
29764 179/300 Read block 2682 len 8192
29765 180/300 Read block 1515 len 8192
29766 181/300 Write block 1251 len 28672 data: 4 6 6 6 5 4 4
29767 182/300 Write block 4058 len 28672 data: 9 5 5 5 4 4 4
29768 183/300 Read block 211 len 16384
29769 184/300 Write block 1431 len 12288 data: 8 7 7
29770 185/300 Read block 4522 len 16384
29771 186/300 Write block 4900 len 20480 data: 3 3 4 3 3
29772 187/300 Read block 4903 len 40960
29773 188/300 Read block 308 len 40960
29774 189/300 Read block 3204 len 16384
29775 190/300 Read block 3358 len 12288
29776 191/300 Write block 2911 len 12288 data: 5 6 5
29777 192/300 Write block 481 len 4096 data: 8
29778 193/300 Read block 4413 len 36864
29779 194/300 Write block 4174 len 28672 data: 6 7 6 6 5 6 6
29780 195/300 Write block 42 len 24576 data: 4 6 6 6 5 5
29781 196/300 Read block 1983 len 20480
29782 197/300 Read block 4104 len 8192
29783 198/300 Flush
29784 199/300 Write block 1171 len 40960 data: 8 7 8 6 5 5 7 6 4 3
29785 200/300 Write block 1969 len 4096 data: 9
29786 201/300 Flush
29787 202/300 Read block 2200 len 28672
29788 203/300 Write block 1725 len 32768 data: 5 7 7 5 7 8 8 8
29789 204/300 Write block 1630 len 24576 data: 5 4 6 6 4 4
29790 205/300 Read block 4779 len 16384
29791 206/300 Read block 2607 len 40960
29792 207/300 Read block 2124 len 12288
29793 208/300 Read block 4117 len 28672
29794 209/300 Write block 1665 len 20480 data: 6 6 6 8 7
29795 210/300 Read block 362 len 12288
29796 211/300 Flush
29797 212/300 Flush
29798 213/300 Read block 965 len 32768
29799 214/300 Read block 3812 len 36864
29800 215/300 Read block 1623 len 28672
29801 216/300 Read block 2456 len 36864
29802 217/300 Read block 811 len 28672
29803 218/300 Write block 2772 len 20480 data: 3 4 2 2 3
29804 219/300 Read block 2398 len 16384
29805 220/300 Write block 3939 len 12288 data: 6 6 6
29806 221/300 Read block 3708 len 28672
29807 222/300 Write block 2614 len 12288 data: 6 4 4
29808 223/300 Read block 1658 len 8192
29809 224/300 Read block 2148 len 20480
29810 225/300 Write block 2303 len 36864 data: 3 3 3 3 4 6 7 8 7
29811 226/300 Read block 4215 len 28672
29812 227/300 Read block 2044 len 24576
29813 228/300 Read block 42 len 16384
29814 229/300 Read block 2098 len 4096
29815 230/300 Write block 2090 len 40960 data: 4 4 3 5 5 5 5 5 3 4
29816 231/300 Write block 3029 len 24576 data: 6 4 4 4 5 5
29817 232/300 Read block 2056 len 24576
29818 233/300 Flush
29819 234/300 Write block 972 len 20480 data: 2 3 5 4 4
29820 235/300 Read block 3836 len 28672
29821 236/300 Read block 680 len 28672
29822 237/300 Write block 2907 len 16384 data: 5 8 9 6
29823 238/300 Flush
29824 239/300 Write block 66 len 8192 data: 5 5
29825 240/300 Write block 516 len 36864 data: 7 5 5 4 6 5 6 7 7
29826 241/300 Read block 4788 len 12288
29827 242/300 Read block 4928 len 20480
29828 243/300 Write block 4774 len 12288 data: 8 5 5
29829 244/300 Read block 3414 len 12288
29830 245/300 Read block 1133 len 8192
29831 246/300 Write block 4122 len 28672 data: 5 5 5 5 5 6 6
29832 247/300 Read block 4413 len 40960
29833 248/300 Write block 2034 len 32768 data: 5 4 3 3 4 5 4 4
29834 249/300 Read block 3509 len 28672
29835 250/300 Write block 902 len 20480 data: 4 4 3 2 2
29836 251/300 Flush
29837 252/300 Write block 3469 len 8192 data: 7 7
29838 253/300 Read block 2495 len 40960
29839 254/300 Read block 1092 len 16384
29840 255/300 Write block 3166 len 16384 data: 5 6 6 6
29841 256/300 Read block 529 len 36864
29842 257/300 Write block 4248 len 32768 data: 3 4 6 6 6 5 4 3
29843 258/300 Read block 1037 len 28672
29844 259/300 Read block 730 len 4096
29845 260/300 Write block 1387 len 12288 data: 7 7 6
29846 261/300 Write block 2532 len 28672 data: 4 5 4 5 5 5 5
29847 262/300 Read block 840 len 36864
29848 263/300 Write block 4290 len 32768 data: 5 4 3 3 3 3 4 7
29849 264/300 Write block 2437 len 16384 data: 5 5 7 6
29850 265/300 Write block 1630 len 12288 data: 6 5 7
29851 266/300 Write block 334 len 28672 data: 5 5 4 4 4 4 4
29852 267/300 Write block 3956 len 20480 data: 7 7 7 8 5
29853 268/300 Write block 4058 len 24576 data: 10 6 6 6 5 5
29854 269/300 Write block 60 len 28672 data: 7 8 7 8 7 6 6
29855 270/300 Write block 586 len 8192 data: 3 4
29856 271/300 Write block 4656 len 28672 data: 6 5 5 5 5 6 5
29857 272/300 Write block 3391 len 24576 data: 7 7 7 7 7 7
29858 273/300 Write block 3405 len 8192 data: 4 6
29859 274/300 Flush
29860 275/300 Flush
29861 276/300 Read block 4504 len 8192
29862 277/300 Read block 4287 len 16384
29863 278/300 Read block 2204 len 4096
29864 279/300 Read block 2816 len 28672
29865 280/300 Write block 2995 len 40960 data: 6 7 6 5 5 5 5 8 7 8
29866 281/300 Read block 427 len 16384
29867 282/300 Write block 1082 len 12288 data: 7 8 8
29868 283/300 Read block 209 len 16384
29869 284/300 Write block 4975 len 32768 data: 9 8 4 3 4 5 5 5
29870 285/300 Read block 2310 len 40960
29871 286/300 Write block 3510 len 8192 data: 7 8
29872 287/300 Read block 1754 len 20480
29873 288/300 Write block 77 len 12288 data: 5 7 9
29874 289/300 Write block 4238 len 36864 data: 2 3 3 4 5 7 7 5 5
29875 290/300 Write block 4689 len 28672 data: 4 4 3 4 6 7 7
29876 291/300 Write block 2388 len 20480 data: 4 4 5 6 7
29877 292/300 Write block 3118 len 28672 data: 6 6 4 7 7 6 6
29878 293/300 Read block 3893 len 40960
29879 294/300 Write block 1640 len 8192 data: 5 4
29880 295/300 Read block 705 len 20480
29881 296/300 Write block 2101 len 40960 data: 6 6 5 5 6 6 7 8 9 9
29882 297/300 Write block 4587 len 16384 data: 4 4 6 7
29883 298/300 Read block 1529 len 8192
29884 299/300 Read block 2067 len 8192
29885 300/300 Read block 4880 len 36864
29886 301/300 Read block 1614 len 20480
29887 Main task has work to do, go find it
29888 got dsc Start(1)
29889 start 1
29890 Jan 13 15:58:06.062 INFO request completed, latency_us: 176, response_code: 204, uri: /start/cid/1, method: POST, req_id: 8973411b-ea80-4125-9408-e24c6f384e72, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
29891 Replay: started 1, returned:Ok(())
29892 ----------------------------------------------------------------
29893 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
29894 States: Active Offline Active
29895 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29896 9152 Acked Read 3 Done Sent Done false
29897 9153 Acked Flush 0 Done Sent Done false
29898 9154 Acked Read 5 Done Sent Done false
29899 9155 Acked Write 3 Done Sent Done false
29900 9156 Acked Read 3 Done Sent Done false
29901 9157 Acked Read 3 Done Sent Done false
29902 9158 Acked Read 4 Done Sent Done false
29903 9159 Acked Read 7 Done Sent Done false
29904 9160 Acked Read 7 Done Sent Done false
29905 9161 Acked Read 6 Done Sent Done false
29906 9162 Acked Write 7 Done Sent Done false
29907 9163 Acked Write 7 Done Sent Done false
29908 9164 Acked Read 6 Done Sent Done false
29909 9165 Acked Read 4 Done Sent Done false
29910 9166 Acked Write 9 Done Sent Done false
29911 9167 Acked Read 3 Done Sent Done false
29912 9168 Acked Read 1 Done Sent Done false
29913 9169 Acked Read 2 Done Sent Done false
29914 9170 Acked Read 2 Done Sent Done false
29915 9171 Acked Write 9 Done Sent Done false
29916 9172 Acked Write 7 Done Sent Done false
29917 9173 Acked Write 8 Done Sent Done false
29918 9174 Acked Read 8 Done Sent Done false
29919 9175 Acked Read 8 Done Sent Done false
29920 9176 Acked Read 6 Done Sent Done false
29921 9177 Acked Write 5 Done Sent Done false
29922 9178 Acked Flush 0 Done Sent Done false
29923 9179 Acked Read 9 Done Sent Done false
29924 9180 Acked Write 9 Done Sent Done false
29925 9181 Acked Write 1 Done Sent Done false
29926 9182 Acked Read 4 Done Sent Done false
29927 9183 Acked Read 2 Done Sent Done false
29928 9184 Acked Write 2 Done Sent Done false
29929 9185 Acked Read 1 Done Sent Done false
29930 9186 Acked Flush 0 Done Sent Done false
29931 9187 Acked Read 7 Done Sent Done false
29932 9188 Acked Write 1 Done Sent Done false
29933 9189 Acked Read 3 Done Sent Done false
29934 9190 Acked Read 6 Done Sent Done false
29935 9191 Acked Write 5 Done Sent Done false
29936 9192 Acked Write 10 Done Sent Done false
29937 9193 Acked Read 4 Done Sent Done false
29938 9194 Acked Read 10 Done Sent Done false
29939 9195 Acked Write 6 Done Sent Done false
29940 9196 Acked Read 5 Done Sent Done false
29941 9197 Acked Write 7 Done Sent Done false
29942 9198 Acked Write 3 Done Sent Done false
29943 9199 Acked Write 10 Done Sent Done false
29944 9200 Acked Write 1 Done Sent Done false
29945 9201 Acked Read 5 Done Sent Done false
29946 9202 Acked Write 8 Done Sent Done false
29947 9203 Acked Read 9 Done Sent Done false
29948 9204 Acked Read 3 Done Sent Done false
29949 9205 Acked Read 4 Done Sent Done false
29950 9206 Acked Read 6 Done Sent Done false
29951 9207 Acked Write 5 Done Sent Done false
29952 9208 Acked Read 8 Done Sent Done false
29953 9209 Acked Read 6 Done Sent Done false
29954 9210 Acked Write 10 Done Sent Done false
29955 9211 Acked Read 8 Done Sent Done false
29956 9212 Acked Read 5 Done Sent Done false
29957 9213 Acked Read 8 Done Sent Done false
29958 9214 Acked Write 9 Done Sent Done false
29959 9215 Acked Read 10 Done Sent Done false
29960 9216 Acked Read 1 Done Sent Done false
29961 9217 Acked Write 10 Done Sent Done false
29962 9218 Acked Write 4 Done Sent Done false
29963 9219 Acked Write 7 Done Sent Done false
29964 9220 Acked Flush 0 Done Sent Done false
29965 9221 Acked Read 9 Done Sent Done false
29966 9222 Acked Write 3 Done Sent Done false
29967 9223 Acked Read 1 Done Sent Done false
29968 9224 Acked Read 2 Done Sent Done false
29969 9225 Acked Read 9 Done Sent Done false
29970 9226 Acked Write 1 Done Sent Done false
29971 9227 Acked Write 4 Done Sent Done false
29972 9228 Acked Flush 0 Done Sent Done false
29973 9229 Acked Read 7 Done Sent Done false
29974 9230 Acked Write 3 Done Sent Done false
29975 9231 Acked Write 8 Done Sent Done false
29976 9232 Acked Flush 0 Done Sent Done false
29977 9233 Acked Read 8 Done Sent Done false
29978 9234 Acked Read 5 Done Sent Done false
29979 9235 Acked Write 8 Done Sent Done false
29980 9236 Acked Flush 0 Done Sent Done false
29981 9237 Acked Write 2 Done Sent Done false
29982 9238 Acked Write 8 Done Sent Done false
29983 9239 Acked Read 4 Done Sent Done false
29984 9240 Acked Read 8 Done Sent Done false
29985 9241 Acked Read 3 Done Sent Done false
29986 9242 Acked Write 1 Done Sent Done false
29987 9243 Acked Write 7 Done Sent Done false
29988 9244 Acked Read 7 Done Sent Done false
29989 9245 Acked Write 8 Done Sent Done false
29990 9246 Acked Read 2 Done Sent Done false
29991 9247 Acked Write 2 Done Sent Done false
29992 9248 Acked Read 8 Done Sent Done false
29993 9249 Acked Write 4 Done Sent Done false
29994 9250 Acked Write 5 Done Sent Done false
29995 9251 Acked Read 7 Done Sent Done false
29996 9252 Acked Write 8 Done Sent Done false
29997 9253 Acked Write 2 Done Sent Done false
29998 9254 Acked Read 7 Done Sent Done false
29999 9255 Acked Read 1 Done Sent Done false
30000 9256 Acked Write 10 Done Sent Done false
30001 9257 Acked Write 4 Done Sent Done false
30002 9258 Acked Write 2 Done Sent Done false
30003 9259 Acked Write 9 Done Sent Done false
30004 9260 Acked Write 3 Done Sent Done false
30005 9261 Acked Write 4 Done Sent Done false
30006 9262 Acked Read 3 Done Sent Done false
30007 9263 Acked Read 9 Done Sent Done false
30008 9264 Acked Read 3 Done Sent Done false
30009 9265 Acked Read 6 Done Sent Done false
30010 9266 Acked Flush 0 Done Sent Done false
30011 9267 Acked Read 9 Done Sent Done false
30012 9268 Acked Read 8 Done Sent Done false
30013 9269 Acked Read 2 Done Sent Done false
30014 9270 Acked Read 9 Done Sent Done false
30015 9271 Acked Write 3 Done Sent Done false
30016 9272 Acked Flush 0 Done Sent Done false
30017 9273 Acked Read 8 Done Sent Done false
30018 9274 Acked Read 5 Done Sent Done false
30019 9275 Acked Read 2 Done Sent Done false
30020 9276 Acked Read 3 Done Sent Done false
30021 9277 Acked Write 10 Done Sent Done false
30022 9278 Acked Write 8 Done Sent Done false
30023 9279 Acked Write 5 Done Sent Done false
30024 9280 Acked Write 10 Done Sent Done false
30025 9281 Acked Write 9 Done Sent Done false
30026 9282 Acked Write 9 Done Sent Done false
30027 9283 Acked Read 8 Done Sent Done false
30028 9284 Acked Write 2 Done Sent Done false
30029 9285 Acked Read 2 Done Sent Done false
30030 9286 Acked Read 7 Done Sent Done false
30031 9287 Acked Read 3 Done Sent Done false
30032 9288 Acked Read 8 Done Sent Done false
30033 9289 Acked Read 3 Done Sent Done false
30034 9290 Acked Read 5 Done Sent Done false
30035 9291 Acked Read 10 Done Sent Done false
30036 9292 Acked Write 4 Done Sent Done false
30037 9293 Acked Write 2 Done Sent Done false
30038 9294 Acked Read 6 Done Sent Done false
30039 9295 Acked Read 7 Done Sent Done false
30040 9296 Acked Write 6 Done Sent Done false
30041 9297 Acked Write 8 Done Sent Done false
30042 9298 Acked Read 4 Done Sent Done false
30043 9299 Acked Read 6 Done Sent Done false
30044 9300 Acked Read 5 Done Sent Done false
30045 9301 Acked Write 9 Done Sent Done false
30046 9302 Acked Read 7 Done Sent Done false
30047 9303 Acked Read 3 Done Sent Done false
30048 9304 Acked Write 4 Done Sent Done false
30049 9305 Acked Write 6 Done Sent Done false
30050 9306 Acked Flush 0 Done Sent Done false
30051 9307 Acked Flush 0 Done Sent Done false
30052 9308 Acked Write 6 Done Sent Done false
30053 9309 Acked Write 10 Done Sent Done false
30054 9310 Acked Write 2 Done Sent Done false
30055 9311 Acked Read 10 Done Sent Done false
30056 9312 Acked Flush 0 Done Sent Done false
30057 9313 Acked Write 7 Done Sent Done false
30058 9314 Acked Write 2 Done Sent Done false
30059 9315 Acked Read 1 Done Sent Done false
30060 9316 Acked Write 10 Done Sent Done false
30061 9317 Acked Write 8 Done Sent Done false
30062 9318 Acked Read 3 Done Sent Done false
30063 9319 Acked Read 5 Done Sent Done false
30064 9320 Acked Read 9 Done Sent Done false
30065 9321 Acked Flush 0 Done Sent Done false
30066 9322 Acked Write 3 Done Sent Done false
30067 9323 Acked Write 10 Done Sent Done false
30068 9324 Acked Read 3 Done Sent Done false
30069 9325 Acked Write 8 Done Sent Done false
30070 9326 Acked Write 3 Done Sent Done false
30071 9327 Acked Read 8 Done Sent Done false
30072 9328 Acked Flush 0 Done Sent Done false
30073 9329 Acked Read 5 Done Sent Done false
30074 9330 Acked Read 2 Done Sent Done false
30075 9331 Acked Read 2 Done Sent Done false
30076 9332 Acked Write 7 Done Sent Done false
30077 9333 Acked Write 7 Done Sent Done false
30078 9334 Acked Read 4 Done Sent Done false
30079 9335 Acked Write 3 Done Sent Done false
30080 9336 Acked Read 4 Done Sent Done false
30081 9337 Acked Write 5 Done Sent Done false
30082 9338 Acked Read 10 Done Sent Done false
30083 9339 Acked Read 10 Done Sent Done false
30084 9340 Acked Read 4 Done Sent Done false
30085 9341 Acked Read 3 Done Sent Done false
30086 9342 Acked Write 3 Done Sent Done false
30087 9343 Acked Write 1 Done Sent Done false
30088 9344 Acked Read 9 Done Sent Done false
30089 9345 Acked Write 7 Done Sent Done false
30090 9346 Acked Write 6 Done Sent Done false
30091 9347 Acked Read 5 Done Sent Done false
30092 9348 Acked Read 2 Done Sent Done false
30093 9349 Acked Flush 0 Done Sent Done false
30094 9350 Acked Write 10 Done Sent Done false
30095 9351 Acked Write 1 Done Sent Done false
30096 9352 Acked Flush 0 Done Sent Done false
30097 9353 Acked Read 7 Done Sent Done false
30098 9354 Acked Write 8 Done Sent Done false
30099 9355 Acked Write 6 Done Sent Done false
30100 9356 Acked Read 4 Done Sent Done false
30101 9357 Acked Read 10 Done Sent Done false
30102 9358 Acked Read 3 Done Sent Done false
30103 9359 Acked Read 7 Done Sent Done false
30104 9360 Acked Write 5 Done Sent Done false
30105 9361 Acked Read 3 Done Sent Done false
30106 9362 Acked Flush 0 Done Sent Done false
30107 9363 Acked Flush 0 Done Sent Done false
30108 9364 Acked Read 8 Done Sent Done false
30109 9365 Acked Read 9 Done Sent Done false
30110 9366 Acked Read 7 Done Sent Done false
30111 9367 Acked Read 9 Done Sent Done false
30112 9368 Acked Read 7 Done Sent Done false
30113 9369 Acked Write 5 Done Sent Done false
30114 9370 Acked Read 4 Done Sent Done false
30115 9371 Acked Write 3 Done Sent Done false
30116 9372 Acked Read 7 Done Sent Done false
30117 9373 Acked Write 3 Done Sent Done false
30118 9374 Acked Read 2 Done Sent Done false
30119 9375 Acked Read 5 Done Sent Done false
30120 9376 Acked Write 9 Done Sent Done false
30121 9377 Acked Read 7 Done Sent Done false
30122 9378 Acked Read 6 Done Sent Done false
30123 9379 Acked Read 4 Done Sent Done false
30124 9380 Acked Read 1 Done Sent Done false
30125 9381 Acked Write 10 Done Sent Done false
30126 9382 Acked Write 6 Done Sent Done false
30127 9383 Acked Read 6 Done Sent Done false
30128 9384 Acked Flush 0 Done Sent Done false
30129 9385 Acked Write 5 Done Sent Done false
30130 9386 Acked Read 7 Done Sent Done false
30131 9387 Acked Read 7 Done Sent Done false
30132 9388 Acked Write 4 Done Sent Done false
30133 9389 Acked Flush 0 Done Sent Done false
30134 9390 Acked Write 2 Done Sent Done false
30135 9391 Acked Write 9 Done Sent Done false
30136 9392 Acked Read 3 Done Sent Done false
30137 9393 Acked Read 5 Done Sent Done false
30138 9394 Acked Write 3 Done Sent Done false
30139 9395 Acked Read 3 Done Sent Done false
30140 9396 Acked Read 2 Done Sent Done false
30141 9397 Acked Write 7 Done Sent Done false
30142 9398 Acked Read 10 Done Sent Done false
30143 9399 Acked Write 8 Done Sent Done false
30144 9400 Acked Read 7 Done Sent Done false
30145 9401 Acked Write 5 Done Sent Done false
30146 9402 Acked Flush 0 Done Sent Done false
30147 9403 Acked Write 2 Done Sent Done false
30148 9404 Acked Read 10 Done Sent Done false
30149 9405 Acked Read 4 Done Sent Done false
30150 9406 Acked Write 4 Done Sent Done false
30151 9407 Acked Read 9 Done Sent Done false
30152 9408 Acked Write 8 Done Sent Done false
30153 9409 Acked Read 7 Done Sent Done false
30154 9410 Acked Read 1 Done Sent Done false
30155 9411 Acked Write 3 Done Sent Done false
30156 9412 Acked Write 7 Done Sent Done false
30157 9413 Acked Read 9 Done Sent Done false
30158 9414 Acked Write 8 Done Sent Done false
30159 9415 Acked Write 4 Done Sent Done false
30160 9416 Acked Write 3 Done Sent Done false
30161 9417 Acked Write 7 Done Sent Done false
30162 9418 Acked Write 5 Done Sent Done false
30163 9419 Acked Write 6 Done Sent Done false
30164 9420 Acked Write 7 Done Sent Done false
30165 9421 Acked Write 2 Done Sent Done false
30166 9422 Acked Write 7 Done Sent Done false
30167 9423 Acked Write 6 Done Sent Done false
30168 9424 Acked Write 2 Done Sent Done false
30169 9425 Acked Flush 0 Done Sent Done false
30170 9426 Acked Flush 0 Done Sent Done false
30171 9427 Acked Read 2 Done Sent Done false
30172 9428 Acked Read 4 Done Sent Done false
30173 9429 Acked Read 1 Done Sent Done false
30174 9430 Acked Read 7 Done Sent Done false
30175 9431 Acked Write 10 Done Sent Done false
30176 9432 Acked Read 4 Done Sent Done false
30177 9433 Acked Write 3 Done Sent Done false
30178 9434 Acked Read 4 Done Sent Done false
30179 9435 Acked Write 8 Done Sent Done false
30180 9436 Acked Read 10 Done Sent Done false
30181 9437 Acked Write 2 Done Sent Done false
30182 9438 Acked Read 5 Done Sent Done false
30183 9439 Acked Write 3 Done Sent Done false
30184 9440 Acked Write 9 Done Sent Done false
30185 9441 Acked Write 7 Done Sent Done false
30186 9442 Acked Write 5 Done Sent Done false
30187 9443 Acked Write 7 Done Sent Done false
30188 9444 Acked Read 10 Done Sent Done false
30189 9445 Acked Write 2 Done Sent Done false
30190 9446 Acked Read 5 Done Sent Done false
30191 9447 Acked Write 10 Done Sent Done false
30192 9448 Acked Write 4 Done Sent Done false
30193 9449 Acked Read 2 Done Sent Done false
30194 9450 Acked Read 2 Done Sent Done false
30195 9451 Acked Read 9 Done Sent Done false
30196 9452 Acked Read 5 Done Sent Done false
30197 STATES DS:0 DS:1 DS:2 TOTAL
30198 Sent 0 301 0 301
30199 Done 301 0 301 602
30200 Skipped 0 0 0 0
30201 Error 0 0 0 0
30202 Last Flush: 9426 9151 9426
30203 Downstairs last five completed: 9151 9150 9149 9148 9147
30204 Upstairs last five completed: 9452 9451 9450 9449 9448
30205 CLIENT: Up:0 ds:301 act:2
30206 [8820] got start action
30207 [8820] I am going to restart
30208 Starting downstairs at port 8820
30209 Make output file at "/tmp/dsc/downstairs-8820.txt"
30210 [8820][1] reports Starting
30211 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1230)
30212 [8820][1] reports Running
30213 ----------------------------------------------------------------
30214 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
30215 States: Active Offline Active
30216 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30217 9152 Acked Read 3 Done Sent Done false
30218 9153 Acked Flush 0 Done Sent Done false
30219 9154 Acked Read 5 Done Sent Done false
30220 9155 Acked Write 3 Done Sent Done false
30221 9156 Acked Read 3 Done Sent Done false
30222 9157 Acked Read 3 Done Sent Done false
30223 9158 Acked Read 4 Done Sent Done false
30224 9159 Acked Read 7 Done Sent Done false
30225 9160 Acked Read 7 Done Sent Done false
30226 9161 Acked Read 6 Done Sent Done false
30227 9162 Acked Write 7 Done Sent Done false
30228 9163 Acked Write 7 Done Sent Done false
30229 9164 Acked Read 6 Done Sent Done false
30230 9165 Acked Read 4 Done Sent Done false
30231 9166 Acked Write 9 Done Sent Done false
30232 9167 Acked Read 3 Done Sent Done false
30233 9168 Acked Read 1 Done Sent Done false
30234 9169 Acked Read 2 Done Sent Done false
30235 9170 Acked Read 2 Done Sent Done false
30236 9171 Acked Write 9 Done Sent Done false
30237 9172 Acked Write 7 Done Sent Done false
30238 9173 Acked Write 8 Done Sent Done false
30239 9174 Acked Read 8 Done Sent Done false
30240 9175 Acked Read 8 Done Sent Done false
30241 9176 Acked Read 6 Done Sent Done false
30242 9177 Acked Write 5 Done Sent Done false
30243 9178 Acked Flush 0 Done Sent Done false
30244 9179 Acked Read 9 Done Sent Done false
30245 9180 Acked Write 9 Done Sent Done false
30246 9181 Acked Write 1 Done Sent Done false
30247 9182 Acked Read 4 Done Sent Done false
30248 9183 Acked Read 2 Done Sent Done false
30249 9184 Acked Write 2 Done Sent Done false
30250 9185 Acked Read 1 Done Sent Done false
30251 9186 Acked Flush 0 Done Sent Done false
30252 9187 Acked Read 7 Done Sent Done false
30253 9188 Acked Write 1 Done Sent Done false
30254 9189 Acked Read 3 Done Sent Done false
30255 9190 Acked Read 6 Done Sent Done false
30256 9191 Acked Write 5 Done Sent Done false
30257 9192 Acked Write 10 Done Sent Done false
30258 9193 Acked Read 4 Done Sent Done false
30259 9194 Acked Read 10 Done Sent Done false
30260 9195 Acked Write 6 Done Sent Done false
30261 9196 Acked Read 5 Done Sent Done false
30262 9197 Acked Write 7 Done Sent Done false
30263 9198 Acked Write 3 Done Sent Done false
30264 9199 Acked Write 10 Done Sent Done false
30265 9200 Acked Write 1 Done Sent Done false
30266 9201 Acked Read 5 Done Sent Done false
30267 9202 Acked Write 8 Done Sent Done false
30268 9203 Acked Read 9 Done Sent Done false
30269 9204 Acked Read 3 Done Sent Done false
30270 9205 Acked Read 4 Done Sent Done false
30271 9206 Acked Read 6 Done Sent Done false
30272 9207 Acked Write 5 Done Sent Done false
30273 9208 Acked Read 8 Done Sent Done false
30274 9209 Acked Read 6 Done Sent Done false
30275 9210 Acked Write 10 Done Sent Done false
30276 9211 Acked Read 8 Done Sent Done false
30277 9212 Acked Read 5 Done Sent Done false
30278 9213 Acked Read 8 Done Sent Done false
30279 9214 Acked Write 9 Done Sent Done false
30280 9215 Acked Read 10 Done Sent Done false
30281 9216 Acked Read 1 Done Sent Done false
30282 9217 Acked Write 10 Done Sent Done false
30283 9218 Acked Write 4 Done Sent Done false
30284 9219 Acked Write 7 Done Sent Done false
30285 9220 Acked Flush 0 Done Sent Done false
30286 9221 Acked Read 9 Done Sent Done false
30287 9222 Acked Write 3 Done Sent Done false
30288 9223 Acked Read 1 Done Sent Done false
30289 9224 Acked Read 2 Done Sent Done false
30290 9225 Acked Read 9 Done Sent Done false
30291 9226 Acked Write 1 Done Sent Done false
30292 9227 Acked Write 4 Done Sent Done false
30293 9228 Acked Flush 0 Done Sent Done false
30294 9229 Acked Read 7 Done Sent Done false
30295 9230 Acked Write 3 Done Sent Done false
30296 9231 Acked Write 8 Done Sent Done false
30297 9232 Acked Flush 0 Done Sent Done false
30298 9233 Acked Read 8 Done Sent Done false
30299 9234 Acked Read 5 Done Sent Done false
30300 9235 Acked Write 8 Done Sent Done false
30301 9236 Acked Flush 0 Done Sent Done false
30302 9237 Acked Write 2 Done Sent Done false
30303 9238 Acked Write 8 Done Sent Done false
30304 9239 Acked Read 4 Done Sent Done false
30305 9240 Acked Read 8 Done Sent Done false
30306 9241 Acked Read 3 Done Sent Done false
30307 9242 Acked Write 1 Done Sent Done false
30308 9243 Acked Write 7 Done Sent Done false
30309 9244 Acked Read 7 Done Sent Done false
30310 9245 Acked Write 8 Done Sent Done false
30311 9246 Acked Read 2 Done Sent Done false
30312 9247 Acked Write 2 Done Sent Done false
30313 9248 Acked Read 8 Done Sent Done false
30314 9249 Acked Write 4 Done Sent Done false
30315 9250 Acked Write 5 Done Sent Done false
30316 9251 Acked Read 7 Done Sent Done false
30317 9252 Acked Write 8 Done Sent Done false
30318 9253 Acked Write 2 Done Sent Done false
30319 9254 Acked Read 7 Done Sent Done false
30320 9255 Acked Read 1 Done Sent Done false
30321 9256 Acked Write 10 Done Sent Done false
30322 9257 Acked Write 4 Done Sent Done false
30323 9258 Acked Write 2 Done Sent Done false
30324 9259 Acked Write 9 Done Sent Done false
30325 9260 Acked Write 3 Done Sent Done false
30326 9261 Acked Write 4 Done Sent Done false
30327 9262 Acked Read 3 Done Sent Done false
30328 9263 Acked Read 9 Done Sent Done false
30329 9264 Acked Read 3 Done Sent Done false
30330 9265 Acked Read 6 Done Sent Done false
30331 9266 Acked Flush 0 Done Sent Done false
30332 9267 Acked Read 9 Done Sent Done false
30333 9268 Acked Read 8 Done Sent Done false
30334 9269 Acked Read 2 Done Sent Done false
30335 9270 Acked Read 9 Done Sent Done false
30336 9271 Acked Write 3 Done Sent Done false
30337 9272 Acked Flush 0 Done Sent Done false
30338 9273 Acked Read 8 Done Sent Done false
30339 9274 Acked Read 5 Done Sent Done false
30340 9275 Acked Read 2 Done Sent Done false
30341 9276 Acked Read 3 Done Sent Done false
30342 9277 Acked Write 10 Done Sent Done false
30343 9278 Acked Write 8 Done Sent Done false
30344 9279 Acked Write 5 Done Sent Done false
30345 9280 Acked Write 10 Done Sent Done false
30346 9281 Acked Write 9 Done Sent Done false
30347 9282 Acked Write 9 Done Sent Done false
30348 9283 Acked Read 8 Done Sent Done false
30349 9284 Acked Write 2 Done Sent Done false
30350 9285 Acked Read 2 Done Sent Done false
30351 9286 Acked Read 7 Done Sent Done false
30352 9287 Acked Read 3 Done Sent Done false
30353 9288 Acked Read 8 Done Sent Done false
30354 9289 Acked Read 3 Done Sent Done false
30355 9290 Acked Read 5 Done Sent Done false
30356 9291 Acked Read 10 Done Sent Done false
30357 9292 Acked Write 4 Done Sent Done false
30358 9293 Acked Write 2 Done Sent Done false
30359 9294 Acked Read 6 Done Sent Done false
30360 9295 Acked Read 7 Done Sent Done false
30361 9296 Acked Write 6 Done Sent Done false
30362 9297 Acked Write 8 Done Sent Done false
30363 9298 Acked Read 4 Done Sent Done false
30364 9299 Acked Read 6 Done Sent Done false
30365 9300 Acked Read 5 Done Sent Done false
30366 9301 Acked Write 9 Done Sent Done false
30367 9302 Acked Read 7 Done Sent Done false
30368 9303 Acked Read 3 Done Sent Done false
30369 9304 Acked Write 4 Done Sent Done false
30370 9305 Acked Write 6 Done Sent Done false
30371 9306 Acked Flush 0 Done Sent Done false
30372 9307 Acked Flush 0 Done Sent Done false
30373 9308 Acked Write 6 Done Sent Done false
30374 9309 Acked Write 10 Done Sent Done false
30375 9310 Acked Write 2 Done Sent Done false
30376 9311 Acked Read 10 Done Sent Done false
30377 9312 Acked Flush 0 Done Sent Done false
30378 9313 Acked Write 7 Done Sent Done false
30379 9314 Acked Write 2 Done Sent Done false
30380 9315 Acked Read 1 Done Sent Done false
30381 9316 Acked Write 10 Done Sent Done false
30382 9317 Acked Write 8 Done Sent Done false
30383 9318 Acked Read 3 Done Sent Done false
30384 9319 Acked Read 5 Done Sent Done false
30385 9320 Acked Read 9 Done Sent Done false
30386 9321 Acked Flush 0 Done Sent Done false
30387 9322 Acked Write 3 Done Sent Done false
30388 9323 Acked Write 10 Done Sent Done false
30389 9324 Acked Read 3 Done Sent Done false
30390 9325 Acked Write 8 Done Sent Done false
30391 9326 Acked Write 3 Done Sent Done false
30392 9327 Acked Read 8 Done Sent Done false
30393 9328 Acked Flush 0 Done Sent Done false
30394 9329 Acked Read 5 Done Sent Done false
30395 9330 Acked Read 2 Done Sent Done false
30396 9331 Acked Read 2 Done Sent Done false
30397 9332 Acked Write 7 Done Sent Done false
30398 9333 Acked Write 7 Done Sent Done false
30399 9334 Acked Read 4 Done Sent Done false
30400 9335 Acked Write 3 Done Sent Done false
30401 9336 Acked Read 4 Done Sent Done false
30402 9337 Acked Write 5 Done Sent Done false
30403 9338 Acked Read 10 Done Sent Done false
30404 9339 Acked Read 10 Done Sent Done false
30405 9340 Acked Read 4 Done Sent Done false
30406 9341 Acked Read 3 Done Sent Done false
30407 9342 Acked Write 3 Done Sent Done false
30408 9343 Acked Write 1 Done Sent Done false
30409 9344 Acked Read 9 Done Sent Done false
30410 9345 Acked Write 7 Done Sent Done false
30411 9346 Acked Write 6 Done Sent Done false
30412 9347 Acked Read 5 Done Sent Done false
30413 9348 Acked Read 2 Done Sent Done false
30414 9349 Acked Flush 0 Done Sent Done false
30415 9350 Acked Write 10 Done Sent Done false
30416 9351 Acked Write 1 Done Sent Done false
30417 9352 Acked Flush 0 Done Sent Done false
30418 9353 Acked Read 7 Done Sent Done false
30419 9354 Acked Write 8 Done Sent Done false
30420 9355 Acked Write 6 Done Sent Done false
30421 9356 Acked Read 4 Done Sent Done false
30422 9357 Acked Read 10 Done Sent Done false
30423 9358 Acked Read 3 Done Sent Done false
30424 9359 Acked Read 7 Done Sent Done false
30425 9360 Acked Write 5 Done Sent Done false
30426 9361 Acked Read 3 Done Sent Done false
30427 9362 Acked Flush 0 Done Sent Done false
30428 9363 Acked Flush 0 Done Sent Done false
30429 9364 Acked Read 8 Done Sent Done false
30430 9365 Acked Read 9 Done Sent Done false
30431 9366 Acked Read 7 Done Sent Done false
30432 9367 Acked Read 9 Done Sent Done false
30433 9368 Acked Read 7 Done Sent Done false
30434 9369 Acked Write 5 Done Sent Done false
30435 9370 Acked Read 4 Done Sent Done false
30436 9371 Acked Write 3 Done Sent Done false
30437 9372 Acked Read 7 Done Sent Done false
30438 9373 Acked Write 3 Done Sent Done false
30439 9374 Acked Read 2 Done Sent Done false
30440 9375 Acked Read 5 Done Sent Done false
30441 9376 Acked Write 9 Done Sent Done false
30442 9377 Acked Read 7 Done Sent Done false
30443 9378 Acked Read 6 Done Sent Done false
30444 9379 Acked Read 4 Done Sent Done false
30445 9380 Acked Read 1 Done Sent Done false
30446 9381 Acked Write 10 Done Sent Done false
30447 9382 Acked Write 6 Done Sent Done false
30448 9383 Acked Read 6 Done Sent Done false
30449 9384 Acked Flush 0 Done Sent Done false
30450 9385 Acked Write 5 Done Sent Done false
30451 9386 Acked Read 7 Done Sent Done false
30452 9387 Acked Read 7 Done Sent Done false
30453 9388 Acked Write 4 Done Sent Done false
30454 9389 Acked Flush 0 Done Sent Done false
30455 9390 Acked Write 2 Done Sent Done false
30456 9391 Acked Write 9 Done Sent Done false
30457 9392 Acked Read 3 Done Sent Done false
30458 9393 Acked Read 5 Done Sent Done false
30459 9394 Acked Write 3 Done Sent Done false
30460 9395 Acked Read 3 Done Sent Done false
30461 9396 Acked Read 2 Done Sent Done false
30462 9397 Acked Write 7 Done Sent Done false
30463 9398 Acked Read 10 Done Sent Done false
30464 9399 Acked Write 8 Done Sent Done false
30465 9400 Acked Read 7 Done Sent Done false
30466 9401 Acked Write 5 Done Sent Done false
30467 9402 Acked Flush 0 Done Sent Done false
30468 9403 Acked Write 2 Done Sent Done false
30469 9404 Acked Read 10 Done Sent Done false
30470 9405 Acked Read 4 Done Sent Done false
30471 9406 Acked Write 4 Done Sent Done false
30472 9407 Acked Read 9 Done Sent Done false
30473 9408 Acked Write 8 Done Sent Done false
30474 9409 Acked Read 7 Done Sent Done false
30475 9410 Acked Read 1 Done Sent Done false
30476 9411 Acked Write 3 Done Sent Done false
30477 9412 Acked Write 7 Done Sent Done false
30478 9413 Acked Read 9 Done Sent Done false
30479 9414 Acked Write 8 Done Sent Done false
30480 9415 Acked Write 4 Done Sent Done false
30481 9416 Acked Write 3 Done Sent Done false
30482 9417 Acked Write 7 Done Sent Done false
30483 9418 Acked Write 5 Done Sent Done false
30484 9419 Acked Write 6 Done Sent Done false
30485 9420 Acked Write 7 Done Sent Done false
30486 9421 Acked Write 2 Done Sent Done false
30487 9422 Acked Write 7 Done Sent Done false
30488 9423 Acked Write 6 Done Sent Done false
30489 9424 Acked Write 2 Done Sent Done false
30490 9425 Acked Flush 0 Done Sent Done false
30491 9426 Acked Flush 0 Done Sent Done false
30492 9427 Acked Read 2 Done Sent Done false
30493 9428 Acked Read 4 Done Sent Done false
30494 9429 Acked Read 1 Done Sent Done false
30495 9430 Acked Read 7 Done Sent Done false
30496 9431 Acked Write 10 Done Sent Done false
30497 9432 Acked Read 4 Done Sent Done false
30498 9433 Acked Write 3 Done Sent Done false
30499 9434 Acked Read 4 Done Sent Done false
30500 9435 Acked Write 8 Done Sent Done false
30501 9436 Acked Read 10 Done Sent Done false
30502 9437 Acked Write 2 Done Sent Done false
30503 9438 Acked Read 5 Done Sent Done false
30504 9439 Acked Write 3 Done Sent Done false
30505 9440 Acked Write 9 Done Sent Done false
30506 9441 Acked Write 7 Done Sent Done false
30507 9442 Acked Write 5 Done Sent Done false
30508 9443 Acked Write 7 Done Sent Done false
30509 9444 Acked Read 10 Done Sent Done false
30510 9445 Acked Write 2 Done Sent Done false
30511 9446 Acked Read 5 Done Sent Done false
30512 9447 Acked Write 10 Done Sent Done false
30513 9448 Acked Write 4 Done Sent Done false
30514 9449 Acked Read 2 Done Sent Done false
30515 9450 Acked Read 2 Done Sent Done false
30516 9451 Acked Read 9 Done Sent Done false
30517 9452 Acked Read 5 Done Sent Done false
30518 9453 Acked Flush 0 Done Sent Done false
30519 STATES DS:0 DS:1 DS:2 TOTAL
30520 Sent 0 302 0 302
30521 Done 302 0 302 604
30522 Skipped 0 0 0 0
30523 Error 0 0 0 0
30524 Last Flush: 9453 9151 9453
30525 Downstairs last five completed: 9151 9150 9149 9148 9147
30526 Upstairs last five completed: 9453 9452 9451 9450 9449
30527 CLIENT: Up:0 ds:302 act:2
305282025-01-13T15:58:11.823ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
305292025-01-13T15:58:11.823ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:45817) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
305302025-01-13T15:58:11.824ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
305312025-01-13T15:58:11.825ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
305322025-01-13T15:58:11.825ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
305332025-01-13T15:58:11.825ZINFOcrucible: send last flush ID to this DS: 9151 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
305342025-01-13T15:58:11.825ZINFOcrucible: Replied this last flush ID: 9151 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
305352025-01-13T15:58:11.825ZINFOcrucible: [1] client re-new 302 jobs since flush 9151 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
305362025-01-13T15:58:11.826ZINFOcrucible: [1] Replayed 302 jobs since flush: 9151 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
30537 ----------------------------------------------------------------
30538 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
30539 Downstairs last five completed: 9453 9452 9451 9450 9449
30540 Upstairs last five completed: 9453 9452 9451 9450 9449
30541 CLIENT: Up:0 ds:0 act:3
30542 Replay: All jobs finished
30543 Main task has work to do, go find it
30544 got dsc Stop(1)
30545 stop 1
30546 Jan 13 15:58:14.069 INFO [8820] Got stop action so:false kr:false
30547 request completed, latency_us: 337, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 0adb0b9c-0a61-4cb6-92b6-1e043084e09f, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
305482025-01-13T15:58:14.069ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
30549 Jan 13 15:58:14.069 INFO request completed, latency_us: 108, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:58:14.06992433Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
30550 response_code: 200, uri: /state/cid/1, method: GET, req_id{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:58:14.069962224Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
30551 : bed3f613-6652-41fd-84f7-506438635715, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-13T15:58:14.069996658Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
30552 1:9998
30553 [8820] Exited with: ExitStatus(unix_wait_status(9))
30554 [8820][1] reports Exit
30555 Jan 13 15:58:18.071 INFO request completed, latency_us: 176, response_code: 200, uri: /state/cid/1, method: GET, req_id: 8a85ff7d-b39d-44db-ae3c-94d90906048f, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
30556 001/300 Read block 1666 len 32768
30557 002/300 Read block 3689 len 12288
30558 003/300 Write block 3892 len 36864 data: 4 5 6 6 6 6 6 8 7
30559 004/300 Read block 2394 len 28672
30560 005/300 Write block 501 len 8192 data: 6 7
30561 006/300 Read block 4495 len 16384
30562 007/300 Read block 3844 len 28672
30563 008/300 Flush
30564 009/300 Write block 865 len 40960 data: 7 8 8 8 9 8 6 5 5 6
30565 010/300 Read block 2821 len 12288
30566 011/300 Write block 4684 len 36864 data: 8 7 6 5 6 5 5 4 5
30567 012/300 Read block 4712 len 12288
30568 013/300 Read block 448 len 32768
30569 014/300 Read block 4702 len 12288
30570 015/300 Flush
30571 016/300 Write block 3492 len 20480 data: 8 6 5 5 4
30572 017/300 Write block 188 len 16384 data: 5 3 2 2
30573 018/300 Write block 769 len 40960 data: 8 8 6 6 8 8 8 6 8 6
30574 019/300 Read block 703 len 12288
30575 020/300 Read block 4683 len 32768
30576 021/300 Read block 1123 len 12288
30577 022/300 Write block 3252 len 28672 data: 4 3 3 2 4 3 3
30578 023/300 Write block 4452 len 40960 data: 7 6 7 7 4 3 3 3 3 5
30579 024/300 Flush
30580 025/300 Flush
30581 026/300 Flush
30582 027/300 Read block 371 len 36864
30583 028/300 Write block 3449 len 28672 data: 6 7 5 4 5 5 5
30584 029/300 Read block 1147 len 16384
30585 030/300 Read block 4648 len 24576
30586 031/300 Read block 3782 len 40960
30587 032/300 Read block 4953 len 36864
30588 033/300 Write block 1445 len 20480 data: 5 5 4 6 6
30589 034/300 Read block 487 len 8192
30590 035/300 Write block 1142 len 16384 data: 8 9 9 8
30591 036/300 Read block 858 len 28672
30592 037/300 Write block 2154 len 12288 data: 7 8 6
30593 038/300 Read block 3888 len 32768
30594 039/300 Write block 4183 len 16384 data: 7 7 9 7
30595 040/300 Read block 3601 len 8192
30596 041/300 Read block 2674 len 24576
30597 042/300 Read block 2768 len 28672
30598 043/300 Write block 2986 len 20480 data: 4 4 5 5 5
30599 044/300 Write block 2279 len 12288 data: 6 5 4
30600 045/300 Write block 4618 len 20480 data: 2 3 3 3 3
30601 046/300 Read block 2675 len 8192
30602 047/300 Read block 4510 len 24576
30603 048/300 Flush
30604 049/300 Read block 4251 len 28672
30605 050/300 Read block 852 len 8192
30606 051/300 Read block 2104 len 12288
30607 052/300 Write block 170 len 24576 data: 6 7 9 10 9 8
30608 053/300 Write block 1506 len 8192 data: 7 5
30609 054/300 Read block 545 len 16384
30610 055/300 Read block 2755 len 24576
30611 056/300 Write block 580 len 28672 data: 7 6 6 5 4 3 4
30612 057/300 Read block 2889 len 20480
30613 058/300 Write block 2964 len 8192 data: 6 6
30614 059/300 Write block 1055 len 4096 data: 8
30615 060/300 Read block 2636 len 16384
30616 061/300 Read block 524 len 8192
30617 062/300 Read block 4095 len 36864
30618 063/300 Read block 3087 len 28672
30619 064/300 Write block 565 len 32768 data: 4 3 4 3 3 3 5 5
30620 065/300 Read block 2389 len 16384
30621 066/300 Write block 4741 len 36864 data: 7 6 5 7 5 5 6 6 8
30622 067/300 Write block 261 len 24576 data: 3 3 2 2 2 3
30623 068/300 Read block 418 len 4096
30624 069/300 Read block 1211 len 16384
30625 070/300 Write block 910 len 4096 data: 4
30626 071/300 Write block 1630 len 4096 data: 7
30627 072/300 Read block 2903 len 4096
30628 073/300 Write block 3378 len 16384 data: 4 4 4 4
30629 074/300 Flush
30630 075/300 Read block 2506 len 8192
30631 076/300 Read block 1074 len 20480
30632 077/300 Read block 2315 len 24576
30633 078/300 Read block 529 len 24576
30634 079/300 Flush
30635 080/300 Read block 3683 len 32768
30636 081/300 Read block 2179 len 24576
30637 082/300 Write block 2436 len 32768 data: 5 6 6 8 7 6 6 7
30638 083/300 Read block 1390 len 8192
30639 084/300 Write block 4804 len 16384 data: 5 4 3 4
30640 085/300 Read block 3663 len 36864
30641 086/300 Write block 103 len 28672 data: 6 6 6 7 7 8 7
30642 087/300 Write block 2178 len 8192 data: 9 9
30643 088/300 Read block 937 len 4096
30644 089/300 Write block 3061 len 24576 data: 4 5 6 6 6 6
30645 090/300 Read block 1718 len 20480
30646 091/300 Flush
30647 092/300 Read block 1024 len 16384
30648 093/300 Write block 1000 len 20480 data: 6 6 5 6 6
30649 094/300 Write block 1410 len 24576 data: 7 8 10 9 9 10
30650 095/300 Read block 428 len 40960
30651 096/300 Read block 2205 len 32768
30652 097/300 Write block 299 len 20480 data: 4 4 3 3 7
30653 098/300 Read block 4626 len 16384
30654 099/300 Read block 217 len 40960
30655 100/300 Read block 359 len 4096
30656 101/300 Read block 1506 len 36864
30657 102/300 Write block 4781 len 8192 data: 4 7
30658 103/300 Write block 2539 len 24576 data: 5 4 4 5 5 6
30659 104/300 Read block 2308 len 8192
30660 105/300 Write block 3983 len 40960 data: 8 10 9 9 8 7 6 6 7 6
30661 106/300 Write block 4966 len 36864 data: 6 7 8 7 5 4 5 5 8
30662 107/300 Write block 2589 len 36864 data: 4 7 10 10 11 11 12 14 11
30663 108/300 Read block 3299 len 40960
30664 109/300 Write block 4903 len 16384 data: 4 4 3 3
30665 110/300 Write block 4029 len 28672 data: 8 7 7 7 6 6 6
30666 111/300 Read block 3151 len 32768
30667 112/300 Write block 4595 len 12288 data: 6 8 8
30668 113/300 Write block 4302 len 28672 data: 11 12 9 10 8 7 6
30669 114/300 Write block 2801 len 20480 data: 5 6 7 5 6
30670 115/300 Flush
30671 116/300 Read block 4895 len 28672
30672 117/300 Read block 4902 len 8192
30673 118/300 Write block 3918 len 36864 data: 6 5 4 7 5 5 5 4 3
30674 119/300 Read block 2545 len 16384
30675 120/300 Write block 2956 len 4096 data: 5
30676 121/300 Read block 3952 len 40960
30677 122/300 Read block 2906 len 20480
30678 123/300 Read block 2966 len 4096
30679 124/300 Write block 2446 len 24576 data: 7 7 8 6 6 6
30680 125/300 Read block 1337 len 36864
30681 126/300 Write block 106 len 24576 data: 8 8 9 8 7 6
30682 127/300 Read block 3950 len 32768
30683 128/300 Read block 655 len 40960
30684 129/300 Write block 4886 len 28672 data: 9 9 11 9 7 7 6
30685 130/300 Read block 4605 len 12288
30686 131/300 Write block 3601 len 36864 data: 8 6 5 5 4 4 4 3 4
30687 132/300 Read block 4978 len 28672
30688 133/300 Write block 3033 len 8192 data: 6 6
30689 134/300 Write block 183 len 12288 data: 5 4 4
30690 135/300 Write block 3356 len 4096 data: 9
30691 136/300 Write block 2638 len 20480 data: 7 7 8 8 7
30692 137/300 Write block 4023 len 28672 data: 3 3 3 3 4 4 9
30693 138/300 Read block 2474 len 36864
30694 139/300 Read block 442 len 4096
30695 140/300 Write block 939 len 40960 data: 5 5 6 9 10 11 10 12 11 7
30696 141/300 Read block 4862 len 24576
30697 142/300 Write block 3054 len 28672 data: 4 4 3 5 5 4 4
30698 143/300 Read block 1635 len 20480
30699 144/300 Read block 524 len 24576
30700 145/300 Read block 1346 len 20480
30701 146/300 Read block 2005 len 4096
30702 147/300 Flush
30703 148/300 Read block 4900 len 8192
30704 149/300 Read block 2896 len 4096
30705 150/300 Write block 329 len 20480 data: 7 7 5 5 5
30706 151/300 Write block 4867 len 32768 data: 9 7 7 6 5 5 6 5
30707 152/300 Read block 1512 len 12288
30708 153/300 Write block 4450 len 8192 data: 8 7
30709 154/300 Read block 2877 len 24576
30710 155/300 Write block 1935 len 20480 data: 7 9 9 7 6
30711 156/300 Read block 2214 len 8192
30712 157/300 Flush
30713 158/300 Write block 1991 len 28672 data: 7 6 7 7 6 6 7
30714 159/300 Write block 879 len 24576 data: 7 9 6 6 5 7
30715 160/300 Read block 4778 len 20480
30716 161/300 Write block 299 len 32768 data: 5 5 4 4 8 8 8 7
30717 162/300 Write block 1662 len 12288 data: 6 7 5
30718 163/300 Flush
30719 164/300 Read block 4694 len 36864
30720 165/300 Read block 4700 len 28672
30721 166/300 Read block 738 len 32768
30722 167/300 Read block 2386 len 40960
30723 168/300 Read block 2491 len 12288
30724 169/300 Write block 1738 len 24576 data: 3 5 5 7 7 9
30725 170/300 Read block 797 len 12288
30726 171/300 Write block 4945 len 20480 data: 8 6 7 7 7
30727 172/300 Write block 4155 len 4096 data: 3
30728 173/300 Read block 592 len 40960
30729 174/300 Write block 3470 len 40960 data: 8 9 8 9 11 8 7 5 3 3
30730 175/300 Flush
30731 176/300 Read block 269 len 20480
30732 177/300 Write block 2394 len 28672 data: 6 6 8 7 7 5 5
30733 178/300 Write block 4116 len 24576 data: 5 5 5 4 5 6
30734 179/300 Write block 292 len 32768 data: 9 6 5 6 4 4 3 6
30735 180/300 Write block 477 len 28672 data: 9 10 10 8 9 7 7
30736 181/300 Write block 3202 len 12288 data: 8 7 7
30737 182/300 Write block 124 len 40960 data: 7 8 7 7 7 6 7 6 6 6
30738 183/300 Read block 3603 len 4096
30739 184/300 Flush
30740 185/300 Read block 4844 len 28672
30741 186/300 Read block 4511 len 8192
30742 187/300 Read block 3825 len 40960
30743 188/300 Read block 1650 len 8192
30744 189/300 Read block 4757 len 20480
30745 190/300 Write block 2218 len 4096 data: 6
30746 191/300 Write block 4385 len 20480 data: 7 7 8 8 7
30747 192/300 Read block 3177 len 36864
30748 193/300 Read block 1633 len 36864
30749 194/300 Write block 3050 len 40960 data: 7 7 6 5 5 5 4 6 6 5
30750 195/300 Read block 1718 len 8192
30751 196/300 Write block 2317 len 16384 data: 7 6 6 5
30752 197/300 Write block 4668 len 4096 data: 6
30753 198/300 Write block 1297 len 24576 data: 5 5 6 6 6 7
30754 199/300 Flush
30755 200/300 Read block 807 len 28672
30756 201/300 Read block 3930 len 40960
30757 202/300 Read block 81 len 24576
30758 203/300 Write block 1491 len 32768 data: 5 5 7 8 7 6 6 10
30759 204/300 Read block 913 len 36864
30760 205/300 Flush
30761 206/300 Flush
30762 207/300 Write block 4848 len 32768 data: 4 6 5 6 4 3 3 5
30763 208/300 Write block 1829 len 4096 data: 6
30764 209/300 Read block 3834 len 28672
30765 210/300 Read block 4054 len 20480
30766 211/300 Write block 2794 len 28672 data: 5 5 5 6 7 6 7
30767 212/300 Read block 1345 len 20480
30768 213/300 Write block 2687 len 12288 data: 7 7 6
30769 214/300 Read block 2865 len 40960
30770 215/300 Read block 3828 len 28672
30771 216/300 Flush
30772 217/300 Write block 4720 len 12288 data: 3 4 4
30773 218/300 Write block 2674 len 24576 data: 3 3 3 5 5 8
30774 219/300 Write block 3492 len 36864 data: 9 7 6 6 5 3 3 4 4
30775 220/300 Read block 1646 len 4096
30776 221/300 Write block 2172 len 16384 data: 12 11 10 10
30777 222/300 Read block 3268 len 36864
30778 223/300 Write block 3857 len 20480 data: 7 8 7 6 6
30779 224/300 Read block 3223 len 24576
30780 225/300 Read block 4479 len 8192
30781 226/300 Write block 3447 len 20480 data: 5 5 7 8 6
30782 227/300 Read block 4963 len 4096
30783 228/300 Read block 1262 len 12288
30784 229/300 Flush
30785 230/300 Write block 2919 len 32768 data: 4 4 4 4 3 2 3 5
30786 231/300 Write block 1391 len 20480 data: 4 5 4 4 4
30787 232/300 Read block 1241 len 4096
30788 233/300 Flush
30789 234/300 Write block 1709 len 24576 data: 4 4 3 3 3 3
30790 235/300 Flush
30791 236/300 Read block 3575 len 36864
30792 237/300 Read block 1402 len 8192
30793 238/300 Write block 1766 len 4096 data: 6
30794 239/300 Flush
30795 240/300 Write block 2254 len 8192 data: 4 4
30796 241/300 Read block 3488 len 20480
30797 242/300 Read block 3744 len 24576
30798 243/300 Read block 1272 len 16384
30799 244/300 Write block 355 len 40960 data: 6 7 6 6 5 6 6 6 6 7
30800 245/300 Read block 1949 len 28672
30801 246/300 Write block 3276 len 28672 data: 9 9 8 9 8 8 6
30802 247/300 Write block 1508 len 28672 data: 5 6 6 5 4 4 5
30803 248/300 Read block 1029 len 36864
30804 249/300 Write block 229 len 28672 data: 5 6 5 4 4 4 3
30805 250/300 Read block 4718 len 8192
30806 251/300 Read block 3468 len 16384
30807 252/300 Read block 639 len 16384
30808 253/300 Read block 1982 len 20480
30809 254/300 Read block 3887 len 28672
30810 255/300 Write block 3268 len 20480 data: 8 10 11 10 10
30811 256/300 Write block 4888 len 20480 data: 12 10 8 8 7
30812 257/300 Read block 321 len 8192
30813 258/300 Write block 3953 len 12288 data: 7 7 6
30814 259/300 Read block 3393 len 36864
30815 260/300 Read block 3893 len 40960
30816 261/300 Read block 1038 len 36864
30817 262/300 Read block 609 len 16384
30818 263/300 Write block 1301 len 12288 data: 7 8 6
30819 264/300 Read block 3052 len 40960
30820 265/300 Write block 2003 len 20480 data: 8 8 8 7 7
30821 266/300 Write block 3350 len 8192 data: 9 10
30822 267/300 Read block 3476 len 20480
30823 268/300 Flush
30824 269/300 Write block 553 len 8192 data: 3 3
30825 270/300 Write block 4114 len 12288 data: 6 4 6
30826 271/300 Flush
30827 272/300 Read block 2742 len 12288
30828 273/300 Write block 3868 len 36864 data: 7 8 7 7 7 8 7 6 4
30829 274/300 Write block 2447 len 36864 data: 8 9 7 7 7 4 4 4 4
30830 275/300 Read block 1719 len 12288
30831 276/300 Read block 3954 len 20480
30832 277/300 Read block 4249 len 36864
30833 278/300 Read block 2662 len 28672
30834 279/300 Flush
30835 280/300 Read block 4682 len 32768
30836 281/300 Read block 1819 len 20480
30837 282/300 Write block 1231 len 36864 data: 4 2 2 3 4 4 6 6 6
30838 283/300 Flush
30839 284/300 Write block 2669 len 24576 data: 3 2 2 3 4 4
30840 285/300 Read block 973 len 8192
30841 286/300 Write block 1705 len 40960 data: 8 7 6 4 5 5 4 4 4 4
30842 287/300 Read block 2860 len 16384
30843 288/300 Read block 1262 len 4096
30844 289/300 Write block 781 len 8192 data: 7 7
30845 290/300 Read block 2544 len 12288
30846 291/300 Write block 3784 len 20480 data: 7 7 5 5 6
30847 292/300 Read block 2418 len 32768
30848 293/300 Read block 2534 len 24576
30849 294/300 Read block 1935 len 28672
30850 295/300 Read block 3537 len 28672
30851 296/300 Write block 4117 len 4096 data: 6
30852 297/300 Flush
30853 298/300 Read block 2184 len 24576
30854 299/300 Read block 2718 len 16384
30855 300/300 Write block 2275 len 16384 data: 8 8 8 6
30856 301/300 Write block 1080 len 8192 data: 8 6
30857 Main task has work to do, go find it
30858 got dsc Start(1)
30859 start 1
30860 Jan 13 15:58:18.312 INFO request completed, latency_us: 227, response_code: 204, uri: /start/cid/1, method: POST, req_id: 09492f53-b775-4944-a0af-3c0b894800f9, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
30861 Replay: started 1, returned:Ok(())
30862 ----------------------------------------------------------------
30863 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
30864 States: Active Offline Active
30865 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30866 9454 Acked Read 8 Done Sent Done false
30867 9455 Acked Read 3 Done Sent Done false
30868 9456 Acked Write 9 Done Sent Done false
30869 9457 Acked Read 7 Done Sent Done false
30870 9458 Acked Write 2 Done Sent Done false
30871 9459 Acked Read 4 Done Sent Done false
30872 9460 Acked Read 7 Done Sent Done false
30873 9461 Acked Flush 0 Done Sent Done false
30874 9462 Acked Write 10 Done Sent Done false
30875 9463 Acked Read 3 Done Sent Done false
30876 9464 Acked Write 9 Done Sent Done false
30877 9465 Acked Read 3 Done Sent Done false
30878 9466 Acked Read 8 Done Sent Done false
30879 9467 Acked Read 3 Done Sent Done false
30880 9468 Acked Flush 0 Done Sent Done false
30881 9469 Acked Write 5 Done Sent Done false
30882 9470 Acked Write 4 Done Sent Done false
30883 9471 Acked Write 10 Done Sent Done false
30884 9472 Acked Read 3 Done Sent Done false
30885 9473 Acked Read 8 Done Sent Done false
30886 9474 Acked Read 3 Done Sent Done false
30887 9475 Acked Write 7 Done Sent Done false
30888 9476 Acked Write 10 Done Sent Done false
30889 9477 Acked Flush 0 Done Sent Done false
30890 9478 Acked Flush 0 Done Sent Done false
30891 9479 Acked Flush 0 Done Sent Done false
30892 9480 Acked Read 9 Done Sent Done false
30893 9481 Acked Write 7 Done Sent Done false
30894 9482 Acked Read 4 Done Sent Done false
30895 9483 Acked Read 6 Done Sent Done false
30896 9484 Acked Read 10 Done Sent Done false
30897 9485 Acked Read 9 Done Sent Done false
30898 9486 Acked Write 5 Done Sent Done false
30899 9487 Acked Read 2 Done Sent Done false
30900 9488 Acked Write 4 Done Sent Done false
30901 9489 Acked Read 7 Done Sent Done false
30902 9490 Acked Write 3 Done Sent Done false
30903 9491 Acked Read 8 Done Sent Done false
30904 9492 Acked Write 4 Done Sent Done false
30905 9493 Acked Read 2 Done Sent Done false
30906 9494 Acked Read 6 Done Sent Done false
30907 9495 Acked Read 7 Done Sent Done false
30908 9496 Acked Write 5 Done Sent Done false
30909 9497 Acked Write 3 Done Sent Done false
30910 9498 Acked Write 5 Done Sent Done false
30911 9499 Acked Read 2 Done Sent Done false
30912 9500 Acked Read 6 Done Sent Done false
30913 9501 Acked Flush 0 Done Sent Done false
30914 9502 Acked Read 7 Done Sent Done false
30915 9503 Acked Read 2 Done Sent Done false
30916 9504 Acked Read 3 Done Sent Done false
30917 9505 Acked Write 6 Done Sent Done false
30918 9506 Acked Write 2 Done Sent Done false
30919 9507 Acked Read 4 Done Sent Done false
30920 9508 Acked Read 6 Done Sent Done false
30921 9509 Acked Write 7 Done Sent Done false
30922 9510 Acked Read 5 Done Sent Done false
30923 9511 Acked Write 2 Done Sent Done false
30924 9512 Acked Write 1 Done Sent Done false
30925 9513 Acked Read 4 Done Sent Done false
30926 9514 Acked Read 2 Done Sent Done false
30927 9515 Acked Read 9 Done Sent Done false
30928 9516 Acked Read 7 Done Sent Done false
30929 9517 Acked Write 8 Done Sent Done false
30930 9518 Acked Read 4 Done Sent Done false
30931 9519 Acked Write 9 Done Sent Done false
30932 9520 Acked Write 6 Done Sent Done false
30933 9521 Acked Read 1 Done Sent Done false
30934 9522 Acked Read 4 Done Sent Done false
30935 9523 Acked Write 1 Done Sent Done false
30936 9524 Acked Write 1 Done Sent Done false
30937 9525 Acked Read 1 Done Sent Done false
30938 9526 Acked Write 4 Done Sent Done false
30939 9527 Acked Flush 0 Done Sent Done false
30940 9528 Acked Read 2 Done Sent Done false
30941 9529 Acked Read 5 Done Sent Done false
30942 9530 Acked Read 6 Done Sent Done false
30943 9531 Acked Read 6 Done Sent Done false
30944 9532 Acked Flush 0 Done Sent Done false
30945 9533 Acked Read 8 Done Sent Done false
30946 9534 Acked Read 6 Done Sent Done false
30947 9535 Acked Write 8 Done Sent Done false
30948 9536 Acked Read 2 Done Sent Done false
30949 9537 Acked Write 4 Done Sent Done false
30950 9538 Acked Read 9 Done Sent Done false
30951 9539 Acked Write 7 Done Sent Done false
30952 9540 Acked Write 2 Done Sent Done false
30953 9541 Acked Read 1 Done Sent Done false
30954 9542 Acked Write 6 Done Sent Done false
30955 9543 Acked Read 5 Done Sent Done false
30956 9544 Acked Flush 0 Done Sent Done false
30957 9545 Acked Read 4 Done Sent Done false
30958 9546 Acked Write 5 Done Sent Done false
30959 9547 Acked Write 6 Done Sent Done false
30960 9548 Acked Read 10 Done Sent Done false
30961 9549 Acked Read 8 Done Sent Done false
30962 9550 Acked Write 5 Done Sent Done false
30963 9551 Acked Read 4 Done Sent Done false
30964 9552 Acked Read 10 Done Sent Done false
30965 9553 Acked Read 1 Done Sent Done false
30966 9554 Acked Read 9 Done Sent Done false
30967 9555 Acked Write 2 Done Sent Done false
30968 9556 Acked Write 6 Done Sent Done false
30969 9557 Acked Read 2 Done Sent Done false
30970 9558 Acked Write 10 Done Sent Done false
30971 9559 Acked Write 9 Done Sent Done false
30972 9560 Acked Write 9 Done Sent Done false
30973 9561 Acked Read 10 Done Sent Done false
30974 9562 Acked Write 4 Done Sent Done false
30975 9563 Acked Write 7 Done Sent Done false
30976 9564 Acked Read 8 Done Sent Done false
30977 9565 Acked Write 3 Done Sent Done false
30978 9566 Acked Write 7 Done Sent Done false
30979 9567 Acked Write 5 Done Sent Done false
30980 9568 Acked Flush 0 Done Sent Done false
30981 9569 Acked Read 7 Done Sent Done false
30982 9570 Acked Read 2 Done Sent Done false
30983 9571 Acked Write 9 Done Sent Done false
30984 9572 Acked Read 4 Done Sent Done false
30985 9573 Acked Write 1 Done Sent Done false
30986 9574 Acked Read 10 Done Sent Done false
30987 9575 Acked Read 5 Done Sent Done false
30988 9576 Acked Read 1 Done Sent Done false
30989 9577 Acked Write 6 Done Sent Done false
30990 9578 Acked Read 9 Done Sent Done false
30991 9579 Acked Write 6 Done Sent Done false
30992 9580 Acked Read 8 Done Sent Done false
30993 9581 Acked Read 10 Done Sent Done false
30994 9582 Acked Write 7 Done Sent Done false
30995 9583 Acked Read 3 Done Sent Done false
30996 9584 Acked Write 9 Done Sent Done false
30997 9585 Acked Read 7 Done Sent Done false
30998 9586 Acked Write 2 Done Sent Done false
30999 9587 Acked Write 3 Done Sent Done false
31000 9588 Acked Write 1 Done Sent Done false
31001 9589 Acked Write 5 Done Sent Done false
31002 9590 Acked Write 7 Done Sent Done false
31003 9591 Acked Read 9 Done Sent Done false
31004 9592 Acked Read 1 Done Sent Done false
31005 9593 Acked Write 10 Done Sent Done false
31006 9594 Acked Read 6 Done Sent Done false
31007 9595 Acked Write 7 Done Sent Done false
31008 9596 Acked Read 5 Done Sent Done false
31009 9597 Acked Read 6 Done Sent Done false
31010 9598 Acked Read 5 Done Sent Done false
31011 9599 Acked Read 1 Done Sent Done false
31012 9600 Acked Flush 0 Done Sent Done false
31013 9601 Acked Read 2 Done Sent Done false
31014 9602 Acked Read 1 Done Sent Done false
31015 9603 Acked Write 5 Done Sent Done false
31016 9604 Acked Write 8 Done Sent Done false
31017 9605 Acked Read 3 Done Sent Done false
31018 9606 Acked Write 2 Done Sent Done false
31019 9607 Acked Read 6 Done Sent Done false
31020 9608 Acked Write 5 Done Sent Done false
31021 9609 Acked Read 2 Done Sent Done false
31022 9610 Acked Flush 0 Done Sent Done false
31023 9611 Acked Write 7 Done Sent Done false
31024 9612 Acked Write 6 Done Sent Done false
31025 9613 Acked Read 5 Done Sent Done false
31026 9614 Acked Write 8 Done Sent Done false
31027 9615 Acked Write 3 Done Sent Done false
31028 9616 Acked Flush 0 Done Sent Done false
31029 9617 Acked Read 9 Done Sent Done false
31030 9618 Acked Read 7 Done Sent Done false
31031 9619 Acked Read 8 Done Sent Done false
31032 9620 Acked Read 10 Done Sent Done false
31033 9621 Acked Read 3 Done Sent Done false
31034 9622 Acked Write 6 Done Sent Done false
31035 9623 Acked Read 3 Done Sent Done false
31036 9624 Acked Write 5 Done Sent Done false
31037 9625 Acked Write 1 Done Sent Done false
31038 9626 Acked Read 10 Done Sent Done false
31039 9627 Acked Write 10 Done Sent Done false
31040 9628 Acked Flush 0 Done Sent Done false
31041 9629 Acked Read 5 Done Sent Done false
31042 9630 Acked Write 7 Done Sent Done false
31043 9631 Acked Write 6 Done Sent Done false
31044 9632 Acked Write 8 Done Sent Done false
31045 9633 Acked Write 7 Done Sent Done false
31046 9634 Acked Write 3 Done Sent Done false
31047 9635 Acked Write 10 Done Sent Done false
31048 9636 Acked Read 1 Done Sent Done false
31049 9637 Acked Flush 0 Done Sent Done false
31050 9638 Acked Read 7 Done Sent Done false
31051 9639 Acked Read 2 Done Sent Done false
31052 9640 Acked Read 10 Done Sent Done false
31053 9641 Acked Read 2 Done Sent Done false
31054 9642 Acked Read 5 Done Sent Done false
31055 9643 Acked Write 1 Done Sent Done false
31056 9644 Acked Write 5 Done Sent Done false
31057 9645 Acked Read 9 Done Sent Done false
31058 9646 Acked Read 9 Done Sent Done false
31059 9647 Acked Write 10 Done Sent Done false
31060 9648 Acked Read 2 Done Sent Done false
31061 9649 Acked Write 4 Done Sent Done false
31062 9650 Acked Write 1 Done Sent Done false
31063 9651 Acked Write 6 Done Sent Done false
31064 9652 Acked Flush 0 Done Sent Done false
31065 9653 Acked Read 7 Done Sent Done false
31066 9654 Acked Read 10 Done Sent Done false
31067 9655 Acked Read 6 Done Sent Done false
31068 9656 Acked Write 8 Done Sent Done false
31069 9657 Acked Read 9 Done Sent Done false
31070 9658 Acked Flush 0 Done Sent Done false
31071 9659 Acked Flush 0 Done Sent Done false
31072 9660 Acked Write 8 Done Sent Done false
31073 9661 Acked Write 1 Done Sent Done false
31074 9662 Acked Read 7 Done Sent Done false
31075 9663 Acked Read 5 Done Sent Done false
31076 9664 Acked Write 7 Done Sent Done false
31077 9665 Acked Read 5 Done Sent Done false
31078 9666 Acked Write 3 Done Sent Done false
31079 9667 Acked Read 10 Done Sent Done false
31080 9668 Acked Read 7 Done Sent Done false
31081 9669 Acked Flush 0 Done Sent Done false
31082 9670 Acked Write 3 Done Sent Done false
31083 9671 Acked Write 6 Done Sent Done false
31084 9672 Acked Write 9 Done Sent Done false
31085 9673 Acked Read 1 Done Sent Done false
31086 9674 Acked Write 4 Done Sent Done false
31087 9675 Acked Read 9 Done Sent Done false
31088 9676 Acked Write 5 Done Sent Done false
31089 9677 Acked Read 6 Done Sent Done false
31090 9678 Acked Read 2 Done Sent Done false
31091 9679 Acked Write 5 Done Sent Done false
31092 9680 Acked Read 1 Done Sent Done false
31093 9681 Acked Read 3 Done Sent Done false
31094 9682 Acked Flush 0 Done Sent Done false
31095 9683 Acked Write 8 Done Sent Done false
31096 9684 Acked Write 5 Done Sent Done false
31097 9685 Acked Read 1 Done Sent Done false
31098 9686 Acked Flush 0 Done Sent Done false
31099 9687 Acked Write 6 Done Sent Done false
31100 9688 Acked Flush 0 Done Sent Done false
31101 9689 Acked Read 9 Done Sent Done false
31102 9690 Acked Read 2 Done Sent Done false
31103 9691 Acked Write 1 Done Sent Done false
31104 9692 Acked Flush 0 Done Sent Done false
31105 9693 Acked Write 2 Done Sent Done false
31106 9694 Acked Read 5 Done Sent Done false
31107 9695 Acked Read 6 Done Sent Done false
31108 9696 Acked Read 4 Done Sent Done false
31109 9697 Acked Write 10 Done Sent Done false
31110 9698 Acked Read 7 Done Sent Done false
31111 9699 Acked Write 7 Done Sent Done false
31112 9700 Acked Write 7 Done Sent Done false
31113 9701 Acked Read 9 Done Sent Done false
31114 9702 Acked Write 7 Done Sent Done false
31115 9703 Acked Read 2 Done Sent Done false
31116 9704 Acked Read 4 Done Sent Done false
31117 9705 Acked Read 4 Done Sent Done false
31118 9706 Acked Read 5 Done Sent Done false
31119 9707 Acked Read 7 Done Sent Done false
31120 9708 Acked Write 5 Done Sent Done false
31121 9709 Acked Write 5 Done Sent Done false
31122 9710 Acked Read 2 Done Sent Done false
31123 9711 Acked Write 3 Done Sent Done false
31124 9712 Acked Read 9 Done Sent Done false
31125 9713 Acked Read 10 Done Sent Done false
31126 9714 Acked Read 9 Done Sent Done false
31127 9715 Acked Read 4 Done Sent Done false
31128 9716 Acked Write 3 Done Sent Done false
31129 9717 Acked Read 10 Done Sent Done false
31130 9718 Acked Write 5 Done Sent Done false
31131 9719 Acked Write 2 Done Sent Done false
31132 9720 Acked Read 5 Done Sent Done false
31133 9721 Acked Flush 0 Done Sent Done false
31134 9722 Acked Write 2 Done Sent Done false
31135 9723 Acked Write 3 Done Sent Done false
31136 9724 Acked Flush 0 Done Sent Done false
31137 9725 Acked Read 3 Done Sent Done false
31138 9726 Acked Write 9 Done Sent Done false
31139 9727 Acked Write 9 Done Sent Done false
31140 9728 Acked Read 3 Done Sent Done false
31141 9729 Acked Read 5 Done Sent Done false
31142 9730 Acked Read 9 Done Sent Done false
31143 9731 Acked Read 7 Done Sent Done false
31144 9732 Acked Flush 0 Done Sent Done false
31145 9733 Acked Read 8 Done Sent Done false
31146 9734 Acked Read 5 Done Sent Done false
31147 9735 Acked Write 9 Done Sent Done false
31148 9736 Acked Flush 0 Done Sent Done false
31149 9737 Acked Write 6 Done Sent Done false
31150 9738 Acked Read 2 Done Sent Done false
31151 9739 Acked Write 10 Done Sent Done false
31152 9740 Acked Read 4 Done Sent Done false
31153 9741 Acked Read 1 Done Sent Done false
31154 9742 Acked Write 2 Done Sent Done false
31155 9743 Acked Read 3 Done Sent Done false
31156 9744 Acked Write 5 Done Sent Done false
31157 9745 Acked Read 8 Done Sent Done false
31158 9746 Acked Read 6 Done Sent Done false
31159 9747 Acked Read 7 Done Sent Done false
31160 9748 Acked Read 7 Done Sent Done false
31161 9749 Acked Write 1 Done Sent Done false
31162 9750 Acked Flush 0 Done Sent Done false
31163 9751 Acked Read 6 Done Sent Done false
31164 9752 Acked Read 4 Done Sent Done false
31165 9753 Acked Write 4 Done Sent Done false
31166 9754 Acked Write 2 Done Sent Done false
31167 STATES DS:0 DS:1 DS:2 TOTAL
31168 Sent 0 301 0 301
31169 Done 301 0 301 602
31170 Skipped 0 0 0 0
31171 Error 0 0 0 0
31172 Last Flush: 9750 9453 9750
31173 Downstairs last five completed: 9453 9452 9451 9450 9449
31174 Upstairs last five completed: 9754 9753 9752 9751 9750
31175 CLIENT: Up:0 ds:301 act:2
31176 [8820] got start action
31177 [8820] I am going to restart
31178 Starting downstairs at port 8820
31179 Make output file at "/tmp/dsc/downstairs-8820.txt"
31180 [8820][1] reports Starting
31181 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1231)
31182 [8820][1] reports Running
31183 ----------------------------------------------------------------
31184 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31185 States: Active Offline Active
31186 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31187 9454 Acked Read 8 Done Sent Done false
31188 9455 Acked Read 3 Done Sent Done false
31189 9456 Acked Write 9 Done Sent Done false
31190 9457 Acked Read 7 Done Sent Done false
31191 9458 Acked Write 2 Done Sent Done false
31192 9459 Acked Read 4 Done Sent Done false
31193 9460 Acked Read 7 Done Sent Done false
31194 9461 Acked Flush 0 Done Sent Done false
31195 9462 Acked Write 10 Done Sent Done false
31196 9463 Acked Read 3 Done Sent Done false
31197 9464 Acked Write 9 Done Sent Done false
31198 9465 Acked Read 3 Done Sent Done false
31199 9466 Acked Read 8 Done Sent Done false
31200 9467 Acked Read 3 Done Sent Done false
31201 9468 Acked Flush 0 Done Sent Done false
31202 9469 Acked Write 5 Done Sent Done false
31203 9470 Acked Write 4 Done Sent Done false
31204 9471 Acked Write 10 Done Sent Done false
31205 9472 Acked Read 3 Done Sent Done false
31206 9473 Acked Read 8 Done Sent Done false
31207 9474 Acked Read 3 Done Sent Done false
31208 9475 Acked Write 7 Done Sent Done false
31209 9476 Acked Write 10 Done Sent Done false
31210 9477 Acked Flush 0 Done Sent Done false
31211 9478 Acked Flush 0 Done Sent Done false
31212 9479 Acked Flush 0 Done Sent Done false
31213 9480 Acked Read 9 Done Sent Done false
31214 9481 Acked Write 7 Done Sent Done false
31215 9482 Acked Read 4 Done Sent Done false
31216 9483 Acked Read 6 Done Sent Done false
31217 9484 Acked Read 10 Done Sent Done false
31218 9485 Acked Read 9 Done Sent Done false
31219 9486 Acked Write 5 Done Sent Done false
31220 9487 Acked Read 2 Done Sent Done false
31221 9488 Acked Write 4 Done Sent Done false
31222 9489 Acked Read 7 Done Sent Done false
31223 9490 Acked Write 3 Done Sent Done false
31224 9491 Acked Read 8 Done Sent Done false
31225 9492 Acked Write 4 Done Sent Done false
31226 9493 Acked Read 2 Done Sent Done false
31227 9494 Acked Read 6 Done Sent Done false
31228 9495 Acked Read 7 Done Sent Done false
31229 9496 Acked Write 5 Done Sent Done false
31230 9497 Acked Write 3 Done Sent Done false
31231 9498 Acked Write 5 Done Sent Done false
31232 9499 Acked Read 2 Done Sent Done false
31233 9500 Acked Read 6 Done Sent Done false
31234 9501 Acked Flush 0 Done Sent Done false
31235 9502 Acked Read 7 Done Sent Done false
31236 9503 Acked Read 2 Done Sent Done false
31237 9504 Acked Read 3 Done Sent Done false
31238 9505 Acked Write 6 Done Sent Done false
31239 9506 Acked Write 2 Done Sent Done false
31240 9507 Acked Read 4 Done Sent Done false
31241 9508 Acked Read 6 Done Sent Done false
31242 9509 Acked Write 7 Done Sent Done false
31243 9510 Acked Read 5 Done Sent Done false
31244 9511 Acked Write 2 Done Sent Done false
31245 9512 Acked Write 1 Done Sent Done false
31246 9513 Acked Read 4 Done Sent Done false
31247 9514 Acked Read 2 Done Sent Done false
31248 9515 Acked Read 9 Done Sent Done false
31249 9516 Acked Read 7 Done Sent Done false
31250 9517 Acked Write 8 Done Sent Done false
31251 9518 Acked Read 4 Done Sent Done false
31252 9519 Acked Write 9 Done Sent Done false
31253 9520 Acked Write 6 Done Sent Done false
31254 9521 Acked Read 1 Done Sent Done false
31255 9522 Acked Read 4 Done Sent Done false
31256 9523 Acked Write 1 Done Sent Done false
31257 9524 Acked Write 1 Done Sent Done false
31258 9525 Acked Read 1 Done Sent Done false
31259 9526 Acked Write 4 Done Sent Done false
31260 9527 Acked Flush 0 Done Sent Done false
31261 9528 Acked Read 2 Done Sent Done false
31262 9529 Acked Read 5 Done Sent Done false
31263 9530 Acked Read 6 Done Sent Done false
31264 9531 Acked Read 6 Done Sent Done false
31265 9532 Acked Flush 0 Done Sent Done false
31266 9533 Acked Read 8 Done Sent Done false
31267 9534 Acked Read 6 Done Sent Done false
31268 9535 Acked Write 8 Done Sent Done false
31269 9536 Acked Read 2 Done Sent Done false
31270 9537 Acked Write 4 Done Sent Done false
31271 9538 Acked Read 9 Done Sent Done false
31272 9539 Acked Write 7 Done Sent Done false
31273 9540 Acked Write 2 Done Sent Done false
31274 9541 Acked Read 1 Done Sent Done false
31275 9542 Acked Write 6 Done Sent Done false
31276 9543 Acked Read 5 Done Sent Done false
31277 9544 Acked Flush 0 Done Sent Done false
31278 9545 Acked Read 4 Done Sent Done false
31279 9546 Acked Write 5 Done Sent Done false
31280 9547 Acked Write 6 Done Sent Done false
31281 9548 Acked Read 10 Done Sent Done false
31282 9549 Acked Read 8 Done Sent Done false
31283 9550 Acked Write 5 Done Sent Done false
31284 9551 Acked Read 4 Done Sent Done false
31285 9552 Acked Read 10 Done Sent Done false
31286 9553 Acked Read 1 Done Sent Done false
31287 9554 Acked Read 9 Done Sent Done false
31288 9555 Acked Write 2 Done Sent Done false
31289 9556 Acked Write 6 Done Sent Done false
31290 9557 Acked Read 2 Done Sent Done false
31291 9558 Acked Write 10 Done Sent Done false
31292 9559 Acked Write 9 Done Sent Done false
31293 9560 Acked Write 9 Done Sent Done false
31294 9561 Acked Read 10 Done Sent Done false
31295 9562 Acked Write 4 Done Sent Done false
31296 9563 Acked Write 7 Done Sent Done false
31297 9564 Acked Read 8 Done Sent Done false
31298 9565 Acked Write 3 Done Sent Done false
31299 9566 Acked Write 7 Done Sent Done false
31300 9567 Acked Write 5 Done Sent Done false
31301 9568 Acked Flush 0 Done Sent Done false
31302 9569 Acked Read 7 Done Sent Done false
31303 9570 Acked Read 2 Done Sent Done false
31304 9571 Acked Write 9 Done Sent Done false
31305 9572 Acked Read 4 Done Sent Done false
31306 9573 Acked Write 1 Done Sent Done false
31307 9574 Acked Read 10 Done Sent Done false
31308 9575 Acked Read 5 Done Sent Done false
31309 9576 Acked Read 1 Done Sent Done false
31310 9577 Acked Write 6 Done Sent Done false
31311 9578 Acked Read 9 Done Sent Done false
31312 9579 Acked Write 6 Done Sent Done false
31313 9580 Acked Read 8 Done Sent Done false
31314 9581 Acked Read 10 Done Sent Done false
31315 9582 Acked Write 7 Done Sent Done false
31316 9583 Acked Read 3 Done Sent Done false
31317 9584 Acked Write 9 Done Sent Done false
31318 9585 Acked Read 7 Done Sent Done false
31319 9586 Acked Write 2 Done Sent Done false
31320 9587 Acked Write 3 Done Sent Done false
31321 9588 Acked Write 1 Done Sent Done false
31322 9589 Acked Write 5 Done Sent Done false
31323 9590 Acked Write 7 Done Sent Done false
31324 9591 Acked Read 9 Done Sent Done false
31325 9592 Acked Read 1 Done Sent Done false
31326 9593 Acked Write 10 Done Sent Done false
31327 9594 Acked Read 6 Done Sent Done false
31328 9595 Acked Write 7 Done Sent Done false
31329 9596 Acked Read 5 Done Sent Done false
31330 9597 Acked Read 6 Done Sent Done false
31331 9598 Acked Read 5 Done Sent Done false
31332 9599 Acked Read 1 Done Sent Done false
31333 9600 Acked Flush 0 Done Sent Done false
31334 9601 Acked Read 2 Done Sent Done false
31335 9602 Acked Read 1 Done Sent Done false
31336 9603 Acked Write 5 Done Sent Done false
31337 9604 Acked Write 8 Done Sent Done false
31338 9605 Acked Read 3 Done Sent Done false
31339 9606 Acked Write 2 Done Sent Done false
31340 9607 Acked Read 6 Done Sent Done false
31341 9608 Acked Write 5 Done Sent Done false
31342 9609 Acked Read 2 Done Sent Done false
31343 9610 Acked Flush 0 Done Sent Done false
31344 9611 Acked Write 7 Done Sent Done false
31345 9612 Acked Write 6 Done Sent Done false
31346 9613 Acked Read 5 Done Sent Done false
31347 9614 Acked Write 8 Done Sent Done false
31348 9615 Acked Write 3 Done Sent Done false
31349 9616 Acked Flush 0 Done Sent Done false
31350 9617 Acked Read 9 Done Sent Done false
31351 9618 Acked Read 7 Done Sent Done false
31352 9619 Acked Read 8 Done Sent Done false
31353 9620 Acked Read 10 Done Sent Done false
31354 9621 Acked Read 3 Done Sent Done false
31355 9622 Acked Write 6 Done Sent Done false
31356 9623 Acked Read 3 Done Sent Done false
31357 9624 Acked Write 5 Done Sent Done false
31358 9625 Acked Write 1 Done Sent Done false
31359 9626 Acked Read 10 Done Sent Done false
31360 9627 Acked Write 10 Done Sent Done false
31361 9628 Acked Flush 0 Done Sent Done false
31362 9629 Acked Read 5 Done Sent Done false
31363 9630 Acked Write 7 Done Sent Done false
31364 9631 Acked Write 6 Done Sent Done false
31365 9632 Acked Write 8 Done Sent Done false
31366 9633 Acked Write 7 Done Sent Done false
31367 9634 Acked Write 3 Done Sent Done false
31368 9635 Acked Write 10 Done Sent Done false
31369 9636 Acked Read 1 Done Sent Done false
31370 9637 Acked Flush 0 Done Sent Done false
31371 9638 Acked Read 7 Done Sent Done false
31372 9639 Acked Read 2 Done Sent Done false
31373 9640 Acked Read 10 Done Sent Done false
31374 9641 Acked Read 2 Done Sent Done false
31375 9642 Acked Read 5 Done Sent Done false
31376 9643 Acked Write 1 Done Sent Done false
31377 9644 Acked Write 5 Done Sent Done false
31378 9645 Acked Read 9 Done Sent Done false
31379 9646 Acked Read 9 Done Sent Done false
31380 9647 Acked Write 10 Done Sent Done false
31381 9648 Acked Read 2 Done Sent Done false
31382 9649 Acked Write 4 Done Sent Done false
31383 9650 Acked Write 1 Done Sent Done false
31384 9651 Acked Write 6 Done Sent Done false
31385 9652 Acked Flush 0 Done Sent Done false
31386 9653 Acked Read 7 Done Sent Done false
31387 9654 Acked Read 10 Done Sent Done false
31388 9655 Acked Read 6 Done Sent Done false
31389 9656 Acked Write 8 Done Sent Done false
31390 9657 Acked Read 9 Done Sent Done false
31391 9658 Acked Flush 0 Done Sent Done false
31392 9659 Acked Flush 0 Done Sent Done false
31393 9660 Acked Write 8 Done Sent Done false
31394 9661 Acked Write 1 Done Sent Done false
31395 9662 Acked Read 7 Done Sent Done false
31396 9663 Acked Read 5 Done Sent Done false
31397 9664 Acked Write 7 Done Sent Done false
31398 9665 Acked Read 5 Done Sent Done false
31399 9666 Acked Write 3 Done Sent Done false
31400 9667 Acked Read 10 Done Sent Done false
31401 9668 Acked Read 7 Done Sent Done false
31402 9669 Acked Flush 0 Done Sent Done false
31403 9670 Acked Write 3 Done Sent Done false
31404 9671 Acked Write 6 Done Sent Done false
31405 9672 Acked Write 9 Done Sent Done false
31406 9673 Acked Read 1 Done Sent Done false
31407 9674 Acked Write 4 Done Sent Done false
31408 9675 Acked Read 9 Done Sent Done false
31409 9676 Acked Write 5 Done Sent Done false
31410 9677 Acked Read 6 Done Sent Done false
31411 9678 Acked Read 2 Done Sent Done false
31412 9679 Acked Write 5 Done Sent Done false
31413 9680 Acked Read 1 Done Sent Done false
31414 9681 Acked Read 3 Done Sent Done false
31415 9682 Acked Flush 0 Done Sent Done false
31416 9683 Acked Write 8 Done Sent Done false
31417 9684 Acked Write 5 Done Sent Done false
31418 9685 Acked Read 1 Done Sent Done false
31419 9686 Acked Flush 0 Done Sent Done false
31420 9687 Acked Write 6 Done Sent Done false
31421 9688 Acked Flush 0 Done Sent Done false
31422 9689 Acked Read 9 Done Sent Done false
31423 9690 Acked Read 2 Done Sent Done false
31424 9691 Acked Write 1 Done Sent Done false
31425 9692 Acked Flush 0 Done Sent Done false
31426 9693 Acked Write 2 Done Sent Done false
31427 9694 Acked Read 5 Done Sent Done false
31428 9695 Acked Read 6 Done Sent Done false
31429 9696 Acked Read 4 Done Sent Done false
31430 9697 Acked Write 10 Done Sent Done false
31431 9698 Acked Read 7 Done Sent Done false
31432 9699 Acked Write 7 Done Sent Done false
31433 9700 Acked Write 7 Done Sent Done false
31434 9701 Acked Read 9 Done Sent Done false
31435 9702 Acked Write 7 Done Sent Done false
31436 9703 Acked Read 2 Done Sent Done false
31437 9704 Acked Read 4 Done Sent Done false
31438 9705 Acked Read 4 Done Sent Done false
31439 9706 Acked Read 5 Done Sent Done false
31440 9707 Acked Read 7 Done Sent Done false
31441 9708 Acked Write 5 Done Sent Done false
31442 9709 Acked Write 5 Done Sent Done false
31443 9710 Acked Read 2 Done Sent Done false
31444 9711 Acked Write 3 Done Sent Done false
31445 9712 Acked Read 9 Done Sent Done false
31446 9713 Acked Read 10 Done Sent Done false
31447 9714 Acked Read 9 Done Sent Done false
31448 9715 Acked Read 4 Done Sent Done false
31449 9716 Acked Write 3 Done Sent Done false
31450 9717 Acked Read 10 Done Sent Done false
31451 9718 Acked Write 5 Done Sent Done false
31452 9719 Acked Write 2 Done Sent Done false
31453 9720 Acked Read 5 Done Sent Done false
31454 9721 Acked Flush 0 Done Sent Done false
31455 9722 Acked Write 2 Done Sent Done false
31456 9723 Acked Write 3 Done Sent Done false
31457 9724 Acked Flush 0 Done Sent Done false
31458 9725 Acked Read 3 Done Sent Done false
31459 9726 Acked Write 9 Done Sent Done false
31460 9727 Acked Write 9 Done Sent Done false
31461 9728 Acked Read 3 Done Sent Done false
31462 9729 Acked Read 5 Done Sent Done false
31463 9730 Acked Read 9 Done Sent Done false
31464 9731 Acked Read 7 Done Sent Done false
31465 9732 Acked Flush 0 Done Sent Done false
31466 9733 Acked Read 8 Done Sent Done false
31467 9734 Acked Read 5 Done Sent Done false
31468 9735 Acked Write 9 Done Sent Done false
31469 9736 Acked Flush 0 Done Sent Done false
31470 9737 Acked Write 6 Done Sent Done false
31471 9738 Acked Read 2 Done Sent Done false
31472 9739 Acked Write 10 Done Sent Done false
31473 9740 Acked Read 4 Done Sent Done false
31474 9741 Acked Read 1 Done Sent Done false
31475 9742 Acked Write 2 Done Sent Done false
31476 9743 Acked Read 3 Done Sent Done false
31477 9744 Acked Write 5 Done Sent Done false
31478 9745 Acked Read 8 Done Sent Done false
31479 9746 Acked Read 6 Done Sent Done false
31480 9747 Acked Read 7 Done Sent Done false
31481 9748 Acked Read 7 Done Sent Done false
31482 9749 Acked Write 1 Done Sent Done false
31483 9750 Acked Flush 0 Done Sent Done false
31484 9751 Acked Read 6 Done Sent Done false
31485 9752 Acked Read 4 Done Sent Done false
31486 9753 Acked Write 4 Done Sent Done false
31487 9754 Acked Write 2 Done Sent Done false
31488 9755 Acked Flush 0 Done Sent Done false
31489 STATES DS:0 DS:1 DS:2 TOTAL
31490 Sent 0 302 0 302
31491 Done 302 0 302 604
31492 Skipped 0 0 0 0
31493 Error 0 0 0 0
31494 Last Flush: 9755 9453 9755
31495 Downstairs last five completed: 9453 9452 9451 9450 9449
31496 Upstairs last five completed: 9755 9754 9753 9752 9751
31497 CLIENT: Up:0 ds:302 act:2
314982025-01-13T15:58:24.070ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
314992025-01-13T15:58:24.071ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39236) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
315002025-01-13T15:58:24.072ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
315012025-01-13T15:58:24.072ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
315022025-01-13T15:58:24.072ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
315032025-01-13T15:58:24.072ZINFOcrucible: send last flush ID to this DS: 9453 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
315042025-01-13T15:58:24.073ZINFOcrucible: Replied this last flush ID: 9453 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
315052025-01-13T15:58:24.073ZINFOcrucible: [1] client re-new 302 jobs since flush 9453 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
315062025-01-13T15:58:24.073ZINFOcrucible: [1] Replayed 302 jobs since flush: 9453 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
31507 ----------------------------------------------------------------
31508 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
31509 Downstairs last five completed: 9755 9754 9753 9752 9751
31510 Upstairs last five completed: 9755 9754 9753 9752 9751
31511 CLIENT: Up:0 ds:0 act:3
31512 Replay: All jobs finished
31513 Main task has work to do, go find it
31514 got dsc Stop(1)
31515 stop 1
31516 [8820] Got stop action so:false kr:false
31517 Jan 13 15:58:26.317 INFO request completed, latency_us: 389, response_code: 204, uri: /stop/cid/1, method: POST, req_id: d7871716-0ed3-4d8c-bde5-d0e3d6219b5c, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
315182025-01-13T15:58:26.318ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
31519 Jan 13 15:58:26.318 INFO request completed{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:58:26.318660214Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
31520 , latency_us: 178, response_code: 200, uri: /state/cid/1, method: GET, req_id: c183e8ba-fb74-42aa-8120-32ad01a18e66, remote_addr: 127.0.0.1:52422, local_addr: 127{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:58:26.318704077Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
31521 .0.0.1:9998
315222025-01-13T15:58:26.318ZINFOcrucible: client task is exiting = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
31523 [8820] Exited with: ExitStatus(unix_wait_status(9))
31524 [8820][1] reports Exit
31525 Jan 13 15:58:30.321 INFO request completed, latency_us: 319, response_code: 200, uri: /state/cid/1, method: GET, req_id: 1b88e63d-bfeb-4852-b53a-3ea1b5d749f8, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
31526 001/300 Write block 2185 len 16384 data: 5 5 5 5
31527 002/300 Read block 1974 len 32768
31528 003/300 Read block 1890 len 20480
31529 004/300 Read block 1332 len 28672
31530 005/300 Read block 490 len 24576
31531 006/300 Read block 365 len 28672
31532 007/300 Write block 2343 len 4096 data: 8
31533 008/300 Flush
31534 009/300 Write block 4953 len 12288 data: 5 4 4
31535 010/300 Read block 4992 len 12288
31536 011/300 Read block 4319 len 32768
31537 012/300 Read block 3373 len 4096
31538 013/300 Write block 2937 len 40960 data: 8 9 8 8 7 6 5 4 4 5
31539 014/300 Write block 3318 len 28672 data: 6 7 7 8 7 7 8
31540 015/300 Write block 3580 len 28672 data: 4 4 7 7 6 5 5
31541 016/300 Write block 296 len 32768 data: 5 5 4 7 6 5 5 9
31542 017/300 Read block 2054 len 36864
31543 018/300 Read block 766 len 16384
31544 019/300 Flush
31545 020/300 Write block 2479 len 8192 data: 7 7
31546 021/300 Write block 959 len 36864 data: 4 8 9 9 8 6 6 5 6
31547 022/300 Write block 1854 len 12288 data: 8 9 9
31548 023/300 Read block 275 len 36864
31549 024/300 Write block 3091 len 32768 data: 4 4 5 5 6 6 5 6
31550 025/300 Write block 876 len 32768 data: 6 5 6 8 10 7 7 6
31551 026/300 Read block 2470 len 36864
31552 027/300 Write block 4659 len 36864 data: 6 6 7 6 6 6 6 6 6
31553 028/300 Read block 2055 len 24576
31554 029/300 Write block 4719 len 40960 data: 2 4 5 5 4 3 5 5 5 6
31555 030/300 Write block 1435 len 4096 data: 9
31556 031/300 Write block 4462 len 32768 data: 5 4 4 5 4 4 5 3
31557 032/300 Write block 1879 len 36864 data: 3 3 4 4 4 3 3 2 2
31558 033/300 Write block 1615 len 8192 data: 5 9
31559 034/300 Write block 2881 len 8192 data: 9 8
31560 035/300 Read block 2937 len 12288
31561 036/300 Read block 4894 len 24576
31562 037/300 Write block 3494 len 32768 data: 7 7 6 4 4 5 5 4
31563 038/300 Write block 3080 len 32768 data: 6 5 4 4 6 5 5 6
31564 039/300 Write block 1942 len 12288 data: 4 4 5
31565 040/300 Read block 3441 len 8192
31566 041/300 Read block 3103 len 24576
31567 042/300 Read block 3954 len 4096
31568 043/300 Write block 4510 len 28672 data: 6 7 8 7 10 9 9
31569 044/300 Flush
31570 045/300 Write block 2284 len 8192 data: 2 2
31571 046/300 Write block 4987 len 20480 data: 5 4 4 3 4
31572 047/300 Read block 56 len 36864
31573 048/300 Write block 1561 len 20480 data: 4 4 4 3 4
31574 049/300 Read block 1223 len 8192
31575 050/300 Write block 3230 len 28672 data: 9 8 8 7 6 5 5
31576 051/300 Write block 4265 len 16384 data: 3 6 7 9
31577 052/300 Read block 1149 len 28672
31578 053/300 Read block 2426 len 24576
31579 054/300 Read block 3159 len 28672
31580 055/300 Write block 1518 len 40960 data: 9 8 9 8 9 8 6 6 5 6
31581 056/300 Read block 272 len 20480
31582 057/300 Read block 3762 len 8192
31583 058/300 Write block 791 len 4096 data: 4
31584 059/300 Flush
31585 060/300 Write block 2938 len 40960 data: 10 9 9 8 7 6 5 5 6 5
31586 061/300 Write block 4487 len 28672 data: 3 3 3 2 2 3 3
31587 062/300 Write block 2308 len 32768 data: 7 8 9 8 5 5 7 6
31588 063/300 Read block 2854 len 16384
31589 064/300 Write block 1538 len 16384 data: 5 4 5 5
31590 065/300 Write block 640 len 36864 data: 5 5 4 4 4 6 7 5 4
31591 066/300 Flush
31592 067/300 Read block 3909 len 4096
31593 068/300 Write block 1052 len 4096 data: 9
31594 069/300 Read block 1059 len 40960
31595 070/300 Write block 4918 len 8192 data: 4 4
31596 071/300 Read block 1720 len 32768
31597 072/300 Read block 4241 len 28672
31598 073/300 Write block 1192 len 16384 data: 4 3 3 3
31599 074/300 Read block 3504 len 16384
31600 075/300 Write block 2436 len 32768 data: 6 7 7 9 8 7 7 8
31601 076/300 Write block 4954 len 8192 data: 5 5
31602 077/300 Read block 4178 len 12288
31603 078/300 Read block 658 len 8192
31604 079/300 Write block 2551 len 12288 data: 9 7 9
31605 080/300 Write block 2298 len 24576 data: 4 4 4 4 3 4
31606 081/300 Read block 578 len 8192
31607 082/300 Write block 3017 len 16384 data: 3 3 6 10
31608 083/300 Write block 385 len 20480 data: 6 8 9 8 9
31609 084/300 Write block 269 len 32768 data: 5 6 6 5 5 5 4 3
31610 085/300 Read block 3888 len 20480
31611 086/300 Write block 210 len 4096 data: 4
31612 087/300 Read block 2217 len 24576
31613 088/300 Write block 1980 len 12288 data: 5 5 6
31614 089/300 Write block 1716 len 20480 data: 3 4 4 4 4
31615 090/300 Write block 3327 len 36864 data: 9 8 8 7 6 6 5 4 3
31616 091/300 Read block 4488 len 16384
31617 092/300 Read block 4316 len 20480
31618 093/300 Read block 3596 len 28672
31619 094/300 Write block 928 len 4096 data: 4
31620 095/300 Read block 4110 len 12288
31621 096/300 Read block 3003 len 24576
31622 097/300 Read block 4178 len 12288
31623 098/300 Write block 4824 len 16384 data: 2 3 3 3
31624 099/300 Write block 3309 len 4096 data: 5
31625 100/300 Write block 3506 len 40960 data: 4 4 5 7 8 9 8 10 10 9
31626 101/300 Write block 1276 len 32768 data: 7 8 9 7 5 7 6 6
31627 102/300 Read block 291 len 12288
31628 103/300 Read block 4818 len 28672
31629 104/300 Write block 3498 len 12288 data: 5 6 6
31630 105/300 Write block 4532 len 16384 data: 10 7 5 4
31631 106/300 Read block 3757 len 4096
31632 107/300 Read block 2154 len 8192
31633 108/300 Read block 3211 len 20480
31634 109/300 Read block 1014 len 40960
31635 110/300 Write block 397 len 36864 data: 6 9 8 7 6 6 6 6 6
31636 111/300 Read block 721 len 36864
31637 112/300 Write block 3465 len 40960 data: 4 6 6 5 8 9 10 9 10 12
31638 113/300 Read block 2387 len 36864
31639 114/300 Write block 4384 len 40960 data: 7 8 8 9 9 8 6 7 7 6
31640 115/300 Write block 3430 len 40960 data: 3 3 4 4 3 4 4 5 5 5
31641 116/300 Read block 1652 len 40960
31642 117/300 Read block 2756 len 28672
31643 118/300 Write block 3804 len 12288 data: 9 7 5
31644 119/300 Flush
31645 120/300 Read block 2358 len 40960
31646 121/300 Write block 1540 len 40960 data: 6 6 6 5 5 4 4 5 4 7
31647 122/300 Flush
31648 123/300 Write block 3992 len 36864 data: 7 4 5 5 6 7 5 4 4
31649 124/300 Flush
31650 125/300 Write block 2894 len 12288 data: 8 5 3
31651 126/300 Read block 2317 len 24576
31652 127/300 Write block 4592 len 4096 data: 6
31653 128/300 Flush
31654 129/300 Read block 181 len 20480
31655 130/300 Read block 669 len 8192
31656 131/300 Read block 2930 len 36864
31657 132/300 Read block 3905 len 8192
31658 133/300 Write block 1328 len 36864 data: 8 6 6 5 6 7 9 11 13
31659 134/300 Write block 695 len 36864 data: 6 8 7 6 6 6 7 6 7
31660 135/300 Write block 1479 len 16384 data: 3 3 4 3
31661 136/300 Read block 1717 len 4096
31662 137/300 Read block 3519 len 4096
31663 138/300 Read block 2765 len 20480
31664 139/300 Read block 4659 len 12288
31665 140/300 Read block 3373 len 32768
31666 141/300 Read block 3001 len 20480
31667 142/300 Read block 2535 len 16384
31668 143/300 Write block 3518 len 20480 data: 6 7 4 4 4
31669 144/300 Read block 934 len 32768
31670 145/300 Write block 2640 len 32768 data: 9 9 8 8 8 5 6 6
31671 146/300 Read block 331 len 24576
31672 147/300 Write block 512 len 20480 data: 7 8 7 7 8
31673 148/300 Write block 2880 len 4096 data: 9
31674 149/300 Read block 4376 len 16384
31675 150/300 Read block 367 len 20480
31676 151/300 Write block 481 len 40960 data: 10 8 8 8 8 8 7 3 2 2
31677 152/300 Write block 258 len 24576 data: 2 3 3 4 4 3
31678 153/300 Read block 2085 len 8192
31679 154/300 Read block 4312 len 24576
31680 155/300 Read block 2707 len 12288
31681 156/300 Flush
31682 157/300 Write block 666 len 20480 data: 10 9 11 10 9
31683 158/300 Read block 3963 len 28672
31684 159/300 Write block 2807 len 16384 data: 6 6 6 8
31685 160/300 Write block 3149 len 16384 data: 3 3 4 4
31686 161/300 Write block 3850 len 20480 data: 7 8 7 8 9
31687 162/300 Read block 1880 len 4096
31688 163/300 Read block 3598 len 40960
31689 164/300 Write block 609 len 16384 data: 5 6 5 5
31690 165/300 Flush
31691 166/300 Read block 2612 len 36864
31692 167/300 Read block 4701 len 40960
31693 168/300 Read block 566 len 12288
31694 169/300 Read block 1846 len 4096
31695 170/300 Write block 4623 len 36864 data: 3 5 6 6 6 6 5 4 5
31696 171/300 Read block 2661 len 16384
31697 172/300 Read block 390 len 4096
31698 173/300 Write block 974 len 20480 data: 6 5 5 5 6
31699 174/300 Write block 4316 len 40960 data: 6 8 8 7 8 6 6 7 7 7
31700 175/300 Flush
31701 176/300 Read block 1544 len 16384
31702 177/300 Read block 1939 len 4096
31703 178/300 Read block 266 len 4096
31704 179/300 Write block 534 len 32768 data: 6 7 6 5 8 8 6 5
31705 180/300 Flush
31706 181/300 Read block 2890 len 4096
31707 182/300 Write block 2579 len 32768 data: 6 7 6 4 3 4 3 2
31708 183/300 Read block 1917 len 24576
31709 184/300 Flush
31710 185/300 Read block 2974 len 32768
31711 186/300 Read block 1859 len 36864
31712 187/300 Write block 768 len 24576 data: 8 9 9 7 7 9
31713 188/300 Flush
31714 189/300 Read block 4882 len 36864
31715 190/300 Write block 3900 len 4096 data: 8
31716 191/300 Read block 4522 len 32768
31717 192/300 Read block 1713 len 16384
31718 193/300 Read block 1888 len 28672
31719 194/300 Write block 1392 len 8192 data: 6 5
31720 195/300 Write block 3652 len 40960 data: 2 2 2 3 3 4 5 5 5 5
31721 196/300 Read block 2790 len 12288
31722 197/300 Flush
31723 198/300 Read block 3712 len 32768
31724 199/300 Read block 109 len 20480
31725 200/300 Read block 3875 len 40960
31726 201/300 Write block 765 len 16384 data: 8 8 8 9
31727 202/300 Read block 22 len 36864
31728 203/300 Flush
31729 204/300 Read block 3156 len 28672
31730 205/300 Write block 1542 len 4096 data: 7
31731 206/300 Write block 1520 len 8192 data: 10 9
31732 207/300 Read block 4353 len 12288
31733 208/300 Read block 1080 len 12288
31734 209/300 Flush
31735 210/300 Read block 2153 len 16384
31736 211/300 Write block 3332 len 24576 data: 7 6 5 4 4 3
31737 212/300 Write block 1995 len 24576 data: 7 7 8 6 4 4
31738 213/300 Read block 1424 len 8192
31739 214/300 Read block 3909 len 4096
31740 215/300 Read block 4352 len 32768
31741 216/300 Write block 3014 len 36864 data: 5 5 5 4 4 7 11 9 9
31742 217/300 Read block 53 len 32768
31743 218/300 Write block 2047 len 16384 data: 7 8 6 8
31744 219/300 Write block 1528 len 28672 data: 6 7 7 6 4 3 3
31745 220/300 Write block 1426 len 32768 data: 7 7 9 9 8 9 8 8
31746 221/300 Read block 3987 len 40960
31747 222/300 Write block 2939 len 12288 data: 10 10 9
31748 223/300 Write block 4434 len 36864 data: 3 4 5 5 5 6 5 6 6
31749 224/300 Flush
31750 225/300 Read block 2387 len 28672
31751 226/300 Read block 3877 len 8192
31752 227/300 Read block 4010 len 32768
31753 228/300 Read block 86 len 20480
31754 229/300 Flush
31755 230/300 Write block 3798 len 36864 data: 4 5 6 6 7 7 10 8 6
31756 231/300 Read block 1666 len 8192
31757 232/300 Read block 4750 len 20480
31758 233/300 Flush
31759 234/300 Write block 238 len 40960 data: 5 5 7 7 5 4 4 4 5 5
31760 235/300 Write block 769 len 36864 data: 10 10 8 8 10 9 9 7 9
31761 236/300 Read block 1309 len 24576
31762 237/300 Read block 1917 len 8192
31763 238/300 Read block 2967 len 24576
31764 239/300 Read block 2042 len 32768
31765 240/300 Flush
31766 241/300 Write block 4376 len 20480 data: 4 4 6 7 5
31767 242/300 Write block 2870 len 4096 data: 9
31768 243/300 Read block 4722 len 20480
31769 244/300 Read block 445 len 32768
31770 245/300 Read block 3956 len 4096
31771 246/300 Read block 66 len 4096
31772 247/300 Read block 3831 len 36864
31773 248/300 Read block 4574 len 28672
31774 249/300 Read block 4535 len 8192
31775 250/300 Write block 1242 len 36864 data: 6 6 5 4 4 6 6 5 3
31776 251/300 Write block 1479 len 32768 data: 4 4 5 4 3 2 2 3
31777 252/300 Write block 2234 len 20480 data: 5 4 4 4 3
31778 253/300 Write block 431 len 8192 data: 8 7
31779 254/300 Write block 4481 len 24576 data: 4 4 4 5 4 3
31780 255/300 Read block 2567 len 40960
31781 256/300 Write block 4696 len 36864 data: 5 5 4 3 2 2 3 3 3
31782 257/300 Read block 4541 len 8192
31783 258/300 Read block 1704 len 24576
31784 259/300 Read block 621 len 8192
31785 260/300 Read block 4238 len 20480
31786 261/300 Read block 1067 len 24576
31787 262/300 Read block 1483 len 20480
31788 263/300 Read block 1983 len 16384
31789 264/300 Flush
31790 265/300 Read block 4341 len 8192
31791 266/300 Read block 1593 len 12288
31792 267/300 Write block 1422 len 24576 data: 6 5 6 6 8 8
31793 268/300 Write block 105 len 20480 data: 7 9 9 10 9
31794 269/300 Flush
31795 270/300 Write block 1248 len 16384 data: 7 6 4 5
31796 271/300 Write block 4682 len 20480 data: 6 6 9 8 7
31797 272/300 Write block 2857 len 32768 data: 4 4 4 4 4 4 5 6
31798 273/300 Write block 2993 len 32768 data: 9 7 7 8 7 6 6 6
31799 274/300 Read block 2682 len 12288
31800 275/300 Write block 4313 len 24576 data: 4 3 4 7 9 9
31801 276/300 Write block 2856 len 28672 data: 3 5 5 5 5 5 5
31802 277/300 Flush
31803 278/300 Read block 2646 len 8192
31804 279/300 Read block 2760 len 12288
31805 280/300 Write block 4740 len 40960 data: 9 8 7 6 8 6 6 7 7 9
31806 281/300 Read block 3241 len 8192
31807 282/300 Write block 4024 len 4096 data: 4
31808 283/300 Read block 3415 len 16384
31809 284/300 Write block 3094 len 36864 data: 6 7 7 6 7 6 6 5 4
31810 285/300 Write block 1512 len 40960 data: 5 5 6 4 7 8 10 9 11 10
31811 286/300 Write block 1227 len 32768 data: 5 4 3 3 5 3 3 4
31812 287/300 Flush
31813 288/300 Write block 4153 len 20480 data: 3 3 4 4 4
31814 289/300 Read block 4698 len 8192
31815 290/300 Read block 4760 len 4096
31816 291/300 Read block 1611 len 4096
31817 292/300 Read block 591 len 36864
31818 293/300 Read block 1345 len 36864
31819 294/300 Read block 4468 len 4096
31820 295/300 Flush
31821 296/300 Read block 2135 len 36864
31822 297/300 Write block 4476 len 32768 data: 3 4 4 4 4 5 5 5
31823 298/300 Read block 399 len 40960
31824 299/300 Write block 1736 len 40960 data: 4 3 4 6 6 8 8 10 9 12
31825 300/300 Read block 2912 len 28672
31826 301/300 Write block 2965 len 32768 data: 7 6 6 6 7 7 7 7
31827 Main task has work to do, go find it
31828 got dsc Start(1)
31829 start 1
31830 Jan 13 15:58:30.556 INFO request completed, latency_us: 194, response_code: 204, uri: /start/cid/1, method: POST, req_id: 585481aa-90e3-4f7f-a6b2-0afbad261afc, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
31831 Replay: started 1, returned:Ok(())
31832 ----------------------------------------------------------------
31833 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
31834 States: Active Offline Active
31835 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31836 9756 Acked Write 4 Done Sent Done false
31837 9757 Acked Read 8 Done Sent Done false
31838 9758 Acked Read 5 Done Sent Done false
31839 9759 Acked Read 7 Done Sent Done false
31840 9760 Acked Read 6 Done Sent Done false
31841 9761 Acked Read 7 Done Sent Done false
31842 9762 Acked Write 1 Done Sent Done false
31843 9763 Acked Flush 0 Done Sent Done false
31844 9764 Acked Write 3 Done Sent Done false
31845 9765 Acked Read 3 Done Sent Done false
31846 9766 Acked Read 8 Done Sent Done false
31847 9767 Acked Read 1 Done Sent Done false
31848 9768 Acked Write 10 Done Sent Done false
31849 9769 Acked Write 7 Done Sent Done false
31850 9770 Acked Write 7 Done Sent Done false
31851 9771 Acked Write 8 Done Sent Done false
31852 9772 Acked Read 9 Done Sent Done false
31853 9773 Acked Read 4 Done Sent Done false
31854 9774 Acked Flush 0 Done Sent Done false
31855 9775 Acked Write 2 Done Sent Done false
31856 9776 Acked Write 9 Done Sent Done false
31857 9777 Acked Write 3 Done Sent Done false
31858 9778 Acked Read 9 Done Sent Done false
31859 9779 Acked Write 8 Done Sent Done false
31860 9780 Acked Write 8 Done Sent Done false
31861 9781 Acked Read 9 Done Sent Done false
31862 9782 Acked Write 9 Done Sent Done false
31863 9783 Acked Read 6 Done Sent Done false
31864 9784 Acked Write 10 Done Sent Done false
31865 9785 Acked Write 1 Done Sent Done false
31866 9786 Acked Write 8 Done Sent Done false
31867 9787 Acked Write 9 Done Sent Done false
31868 9788 Acked Write 2 Done Sent Done false
31869 9789 Acked Write 2 Done Sent Done false
31870 9790 Acked Read 3 Done Sent Done false
31871 9791 Acked Read 6 Done Sent Done false
31872 9792 Acked Write 8 Done Sent Done false
31873 9793 Acked Write 8 Done Sent Done false
31874 9794 Acked Write 3 Done Sent Done false
31875 9795 Acked Read 2 Done Sent Done false
31876 9796 Acked Read 6 Done Sent Done false
31877 9797 Acked Read 1 Done Sent Done false
31878 9798 Acked Write 7 Done Sent Done false
31879 9799 Acked Flush 0 Done Sent Done false
31880 9800 Acked Write 2 Done Sent Done false
31881 9801 Acked Write 5 Done Sent Done false
31882 9802 Acked Read 9 Done Sent Done false
31883 9803 Acked Write 5 Done Sent Done false
31884 9804 Acked Read 2 Done Sent Done false
31885 9805 Acked Write 7 Done Sent Done false
31886 9806 Acked Write 4 Done Sent Done false
31887 9807 Acked Read 7 Done Sent Done false
31888 9808 Acked Read 6 Done Sent Done false
31889 9809 Acked Read 7 Done Sent Done false
31890 9810 Acked Write 10 Done Sent Done false
31891 9811 Acked Read 5 Done Sent Done false
31892 9812 Acked Read 2 Done Sent Done false
31893 9813 Acked Write 1 Done Sent Done false
31894 9814 Acked Flush 0 Done Sent Done false
31895 9815 Acked Write 10 Done Sent Done false
31896 9816 Acked Write 7 Done Sent Done false
31897 9817 Acked Write 8 Done Sent Done false
31898 9818 Acked Read 4 Done Sent Done false
31899 9819 Acked Write 4 Done Sent Done false
31900 9820 Acked Write 9 Done Sent Done false
31901 9821 Acked Flush 0 Done Sent Done false
31902 9822 Acked Read 1 Done Sent Done false
31903 9823 Acked Write 1 Done Sent Done false
31904 9824 Acked Read 10 Done Sent Done false
31905 9825 Acked Write 2 Done Sent Done false
31906 9826 Acked Read 8 Done Sent Done false
31907 9827 Acked Read 7 Done Sent Done false
31908 9828 Acked Write 4 Done Sent Done false
31909 9829 Acked Read 4 Done Sent Done false
31910 9830 Acked Write 8 Done Sent Done false
31911 9831 Acked Write 2 Done Sent Done false
31912 9832 Acked Read 3 Done Sent Done false
31913 9833 Acked Read 2 Done Sent Done false
31914 9834 Acked Write 3 Done Sent Done false
31915 9835 Acked Write 6 Done Sent Done false
31916 9836 Acked Read 2 Done Sent Done false
31917 9837 Acked Write 4 Done Sent Done false
31918 9838 Acked Write 5 Done Sent Done false
31919 9839 Acked Write 8 Done Sent Done false
31920 9840 Acked Read 5 Done Sent Done false
31921 9841 Acked Write 1 Done Sent Done false
31922 9842 Acked Read 6 Done Sent Done false
31923 9843 Acked Write 3 Done Sent Done false
31924 9844 Acked Write 5 Done Sent Done false
31925 9845 Acked Write 9 Done Sent Done false
31926 9846 Acked Read 4 Done Sent Done false
31927 9847 Acked Read 5 Done Sent Done false
31928 9848 Acked Read 7 Done Sent Done false
31929 9849 Acked Write 1 Done Sent Done false
31930 9850 Acked Read 3 Done Sent Done false
31931 9851 Acked Read 6 Done Sent Done false
31932 9852 Acked Read 3 Done Sent Done false
31933 9853 Acked Write 4 Done Sent Done false
31934 9854 Acked Write 1 Done Sent Done false
31935 9855 Acked Write 10 Done Sent Done false
31936 9856 Acked Write 8 Done Sent Done false
31937 9857 Acked Read 3 Done Sent Done false
31938 9858 Acked Read 7 Done Sent Done false
31939 9859 Acked Write 3 Done Sent Done false
31940 9860 Acked Write 4 Done Sent Done false
31941 9861 Acked Read 1 Done Sent Done false
31942 9862 Acked Read 2 Done Sent Done false
31943 9863 Acked Read 5 Done Sent Done false
31944 9864 Acked Read 10 Done Sent Done false
31945 9865 Acked Write 9 Done Sent Done false
31946 9866 Acked Read 9 Done Sent Done false
31947 9867 Acked Write 10 Done Sent Done false
31948 9868 Acked Read 9 Done Sent Done false
31949 9869 Acked Write 10 Done Sent Done false
31950 9870 Acked Write 10 Done Sent Done false
31951 9871 Acked Read 10 Done Sent Done false
31952 9872 Acked Read 7 Done Sent Done false
31953 9873 Acked Write 3 Done Sent Done false
31954 9874 Acked Flush 0 Done Sent Done false
31955 9875 Acked Read 10 Done Sent Done false
31956 9876 Acked Write 10 Done Sent Done false
31957 9877 Acked Flush 0 Done Sent Done false
31958 9878 Acked Write 9 Done Sent Done false
31959 9879 Acked Flush 0 Done Sent Done false
31960 9880 Acked Write 3 Done Sent Done false
31961 9881 Acked Read 6 Done Sent Done false
31962 9882 Acked Write 1 Done Sent Done false
31963 9883 Acked Flush 0 Done Sent Done false
31964 9884 Acked Read 5 Done Sent Done false
31965 9885 Acked Read 2 Done Sent Done false
31966 9886 Acked Read 9 Done Sent Done false
31967 9887 Acked Read 2 Done Sent Done false
31968 9888 Acked Write 9 Done Sent Done false
31969 9889 Acked Write 9 Done Sent Done false
31970 9890 Acked Write 4 Done Sent Done false
31971 9891 Acked Read 1 Done Sent Done false
31972 9892 Acked Read 1 Done Sent Done false
31973 9893 Acked Read 5 Done Sent Done false
31974 9894 Acked Read 3 Done Sent Done false
31975 9895 Acked Read 8 Done Sent Done false
31976 9896 Acked Read 5 Done Sent Done false
31977 9897 Acked Read 4 Done Sent Done false
31978 9898 Acked Write 5 Done Sent Done false
31979 9899 Acked Read 8 Done Sent Done false
31980 9900 Acked Write 8 Done Sent Done false
31981 9901 Acked Read 6 Done Sent Done false
31982 9902 Acked Write 5 Done Sent Done false
31983 9903 Acked Write 1 Done Sent Done false
31984 9904 Acked Read 4 Done Sent Done false
31985 9905 Acked Read 5 Done Sent Done false
31986 9906 Acked Write 10 Done Sent Done false
31987 9907 Acked Write 6 Done Sent Done false
31988 9908 Acked Read 2 Done Sent Done false
31989 9909 Acked Read 6 Done Sent Done false
31990 9910 Acked Read 3 Done Sent Done false
31991 9911 Acked Flush 0 Done Sent Done false
31992 9912 Acked Write 5 Done Sent Done false
31993 9913 Acked Read 7 Done Sent Done false
31994 9914 Acked Write 4 Done Sent Done false
31995 9915 Acked Write 4 Done Sent Done false
31996 9916 Acked Write 5 Done Sent Done false
31997 9917 Acked Read 1 Done Sent Done false
31998 9918 Acked Read 10 Done Sent Done false
31999 9919 Acked Write 4 Done Sent Done false
32000 9920 Acked Flush 0 Done Sent Done false
32001 9921 Acked Read 9 Done Sent Done false
32002 9922 Acked Read 10 Done Sent Done false
32003 9923 Acked Read 3 Done Sent Done false
32004 9924 Acked Read 1 Done Sent Done false
32005 9925 Acked Write 9 Done Sent Done false
32006 9926 Acked Read 4 Done Sent Done false
32007 9927 Acked Read 1 Done Sent Done false
32008 9928 Acked Write 5 Done Sent Done false
32009 9929 Acked Write 10 Done Sent Done false
32010 9930 Acked Flush 0 Done Sent Done false
32011 9931 Acked Read 4 Done Sent Done false
32012 9932 Acked Read 1 Done Sent Done false
32013 9933 Acked Read 1 Done Sent Done false
32014 9934 Acked Write 8 Done Sent Done false
32015 9935 Acked Flush 0 Done Sent Done false
32016 9936 Acked Read 1 Done Sent Done false
32017 9937 Acked Write 8 Done Sent Done false
32018 9938 Acked Read 6 Done Sent Done false
32019 9939 Acked Flush 0 Done Sent Done false
32020 9940 Acked Read 8 Done Sent Done false
32021 9941 Acked Read 9 Done Sent Done false
32022 9942 Acked Write 6 Done Sent Done false
32023 9943 Acked Flush 0 Done Sent Done false
32024 9944 Acked Read 9 Done Sent Done false
32025 9945 Acked Write 1 Done Sent Done false
32026 9946 Acked Read 8 Done Sent Done false
32027 9947 Acked Read 4 Done Sent Done false
32028 9948 Acked Read 7 Done Sent Done false
32029 9949 Acked Write 2 Done Sent Done false
32030 9950 Acked Write 10 Done Sent Done false
32031 9951 Acked Read 3 Done Sent Done false
32032 9952 Acked Flush 0 Done Sent Done false
32033 9953 Acked Read 8 Done Sent Done false
32034 9954 Acked Read 5 Done Sent Done false
32035 9955 Acked Read 10 Done Sent Done false
32036 9956 Acked Write 4 Done Sent Done false
32037 9957 Acked Read 9 Done Sent Done false
32038 9958 Acked Flush 0 Done Sent Done false
32039 9959 Acked Read 7 Done Sent Done false
32040 9960 Acked Write 1 Done Sent Done false
32041 9961 Acked Write 2 Done Sent Done false
32042 9962 Acked Read 3 Done Sent Done false
32043 9963 Acked Read 3 Done Sent Done false
32044 9964 Acked Flush 0 Done Sent Done false
32045 9965 Acked Read 4 Done Sent Done false
32046 9966 Acked Write 6 Done Sent Done false
32047 9967 Acked Write 6 Done Sent Done false
32048 9968 Acked Read 2 Done Sent Done false
32049 9969 Acked Read 1 Done Sent Done false
32050 9970 Acked Read 8 Done Sent Done false
32051 9971 Acked Write 9 Done Sent Done false
32052 9972 Acked Read 8 Done Sent Done false
32053 9973 Acked Write 4 Done Sent Done false
32054 9974 Acked Write 7 Done Sent Done false
32055 9975 Acked Write 8 Done Sent Done false
32056 9976 Acked Read 10 Done Sent Done false
32057 9977 Acked Write 3 Done Sent Done false
32058 9978 Acked Write 9 Done Sent Done false
32059 9979 Acked Flush 0 Done Sent Done false
32060 9980 Acked Read 7 Done Sent Done false
32061 9981 Acked Read 2 Done Sent Done false
32062 9982 Acked Read 8 Done Sent Done false
32063 9983 Acked Read 5 Done Sent Done false
32064 9984 Acked Flush 0 Done Sent Done false
32065 9985 Acked Write 9 Done Sent Done false
32066 9986 Acked Read 2 Done Sent Done false
32067 9987 Acked Read 5 Done Sent Done false
32068 9988 Acked Flush 0 Done Sent Done false
32069 9989 Acked Write 10 Done Sent Done false
32070 9990 Acked Write 9 Done Sent Done false
32071 9991 Acked Read 6 Done Sent Done false
32072 9992 Acked Read 2 Done Sent Done false
32073 9993 Acked Read 6 Done Sent Done false
32074 9994 Acked Read 8 Done Sent Done false
32075 9995 Acked Flush 0 Done Sent Done false
32076 9996 Acked Write 5 Done Sent Done false
32077 9997 Acked Write 1 Done Sent Done false
32078 9998 Acked Read 5 Done Sent Done false
32079 9999 Acked Read 8 Done Sent Done false
32080 10000 Acked Read 1 Done Sent Done false
32081 10001 Acked Read 1 Done Sent Done false
32082 10002 Acked Read 9 Done Sent Done false
32083 10003 Acked Read 7 Done Sent Done false
32084 10004 Acked Read 2 Done Sent Done false
32085 10005 Acked Write 9 Done Sent Done false
32086 10006 Acked Write 8 Done Sent Done false
32087 10007 Acked Write 5 Done Sent Done false
32088 10008 Acked Write 2 Done Sent Done false
32089 10009 Acked Write 6 Done Sent Done false
32090 10010 Acked Read 10 Done Sent Done false
32091 10011 Acked Write 9 Done Sent Done false
32092 10012 Acked Read 2 Done Sent Done false
32093 10013 Acked Read 6 Done Sent Done false
32094 10014 Acked Read 2 Done Sent Done false
32095 10015 Acked Read 5 Done Sent Done false
32096 10016 Acked Read 6 Done Sent Done false
32097 10017 Acked Read 5 Done Sent Done false
32098 10018 Acked Read 4 Done Sent Done false
32099 10019 Acked Flush 0 Done Sent Done false
32100 10020 Acked Read 2 Done Sent Done false
32101 10021 Acked Read 3 Done Sent Done false
32102 10022 Acked Write 6 Done Sent Done false
32103 10023 Acked Write 5 Done Sent Done false
32104 10024 Acked Flush 0 Done Sent Done false
32105 10025 Acked Write 4 Done Sent Done false
32106 10026 Acked Write 5 Done Sent Done false
32107 10027 Acked Write 8 Done Sent Done false
32108 10028 Acked Write 8 Done Sent Done false
32109 10029 Acked Read 3 Done Sent Done false
32110 10030 Acked Write 6 Done Sent Done false
32111 10031 Acked Write 7 Done Sent Done false
32112 10032 Acked Flush 0 Done Sent Done false
32113 10033 Acked Read 2 Done Sent Done false
32114 10034 Acked Read 3 Done Sent Done false
32115 10035 Acked Write 10 Done Sent Done false
32116 10036 Acked Read 2 Done Sent Done false
32117 10037 Acked Write 1 Done Sent Done false
32118 10038 Acked Read 4 Done Sent Done false
32119 10039 Acked Write 9 Done Sent Done false
32120 10040 Acked Write 10 Done Sent Done false
32121 10041 Acked Write 8 Done Sent Done false
32122 10042 Acked Flush 0 Done Sent Done false
32123 10043 Acked Write 5 Done Sent Done false
32124 10044 Acked Read 2 Done Sent Done false
32125 10045 Acked Read 1 Done Sent Done false
32126 10046 Acked Read 1 Done Sent Done false
32127 10047 Acked Read 9 Done Sent Done false
32128 10048 Acked Read 9 Done Sent Done false
32129 10049 Acked Read 1 Done Sent Done false
32130 10050 Acked Flush 0 Done Sent Done false
32131 10051 Acked Read 9 Done Sent Done false
32132 10052 Acked Write 8 Done Sent Done false
32133 10053 Acked Read 10 Done Sent Done false
32134 10054 Acked Write 10 Done Sent Done false
32135 10055 Acked Read 7 Done Sent Done false
32136 10056 Acked Write 8 Done Sent Sent false
32137 STATES DS:0 DS:1 DS:2 TOTAL
32138 Sent 0 301 1 302
32139 Done 301 0 300 601
32140 Skipped 0 0 0 0
32141 Error 0 0 0 0
32142 Last Flush: 10050 9755 10050
32143 Downstairs last five completed: 9755 9754 9753 9752 9751
32144 Upstairs last five completed: 10056 10055 10054 10053 10052
32145 CLIENT: Up:0 ds:301 act:2
32146 [8820] got start action
32147 [8820] I am going to restart
32148 Starting downstairs at port 8820
32149 Make output file at "/tmp/dsc/downstairs-8820.txt"
32150 [8820][1] reports Starting
32151 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1232)
32152 [8820][1] reports Running
32153 ----------------------------------------------------------------
32154 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32155 States: Active Offline Active
32156 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32157 9756 Acked Write 4 Done Sent Done false
32158 9757 Acked Read 8 Done Sent Done false
32159 9758 Acked Read 5 Done Sent Done false
32160 9759 Acked Read 7 Done Sent Done false
32161 9760 Acked Read 6 Done Sent Done false
32162 9761 Acked Read 7 Done Sent Done false
32163 9762 Acked Write 1 Done Sent Done false
32164 9763 Acked Flush 0 Done Sent Done false
32165 9764 Acked Write 3 Done Sent Done false
32166 9765 Acked Read 3 Done Sent Done false
32167 9766 Acked Read 8 Done Sent Done false
32168 9767 Acked Read 1 Done Sent Done false
32169 9768 Acked Write 10 Done Sent Done false
32170 9769 Acked Write 7 Done Sent Done false
32171 9770 Acked Write 7 Done Sent Done false
32172 9771 Acked Write 8 Done Sent Done false
32173 9772 Acked Read 9 Done Sent Done false
32174 9773 Acked Read 4 Done Sent Done false
32175 9774 Acked Flush 0 Done Sent Done false
32176 9775 Acked Write 2 Done Sent Done false
32177 9776 Acked Write 9 Done Sent Done false
32178 9777 Acked Write 3 Done Sent Done false
32179 9778 Acked Read 9 Done Sent Done false
32180 9779 Acked Write 8 Done Sent Done false
32181 9780 Acked Write 8 Done Sent Done false
32182 9781 Acked Read 9 Done Sent Done false
32183 9782 Acked Write 9 Done Sent Done false
32184 9783 Acked Read 6 Done Sent Done false
32185 9784 Acked Write 10 Done Sent Done false
32186 9785 Acked Write 1 Done Sent Done false
32187 9786 Acked Write 8 Done Sent Done false
32188 9787 Acked Write 9 Done Sent Done false
32189 9788 Acked Write 2 Done Sent Done false
32190 9789 Acked Write 2 Done Sent Done false
32191 9790 Acked Read 3 Done Sent Done false
32192 9791 Acked Read 6 Done Sent Done false
32193 9792 Acked Write 8 Done Sent Done false
32194 9793 Acked Write 8 Done Sent Done false
32195 9794 Acked Write 3 Done Sent Done false
32196 9795 Acked Read 2 Done Sent Done false
32197 9796 Acked Read 6 Done Sent Done false
32198 9797 Acked Read 1 Done Sent Done false
32199 9798 Acked Write 7 Done Sent Done false
32200 9799 Acked Flush 0 Done Sent Done false
32201 9800 Acked Write 2 Done Sent Done false
32202 9801 Acked Write 5 Done Sent Done false
32203 9802 Acked Read 9 Done Sent Done false
32204 9803 Acked Write 5 Done Sent Done false
32205 9804 Acked Read 2 Done Sent Done false
32206 9805 Acked Write 7 Done Sent Done false
32207 9806 Acked Write 4 Done Sent Done false
32208 9807 Acked Read 7 Done Sent Done false
32209 9808 Acked Read 6 Done Sent Done false
32210 9809 Acked Read 7 Done Sent Done false
32211 9810 Acked Write 10 Done Sent Done false
32212 9811 Acked Read 5 Done Sent Done false
32213 9812 Acked Read 2 Done Sent Done false
32214 9813 Acked Write 1 Done Sent Done false
32215 9814 Acked Flush 0 Done Sent Done false
32216 9815 Acked Write 10 Done Sent Done false
32217 9816 Acked Write 7 Done Sent Done false
32218 9817 Acked Write 8 Done Sent Done false
32219 9818 Acked Read 4 Done Sent Done false
32220 9819 Acked Write 4 Done Sent Done false
32221 9820 Acked Write 9 Done Sent Done false
32222 9821 Acked Flush 0 Done Sent Done false
32223 9822 Acked Read 1 Done Sent Done false
32224 9823 Acked Write 1 Done Sent Done false
32225 9824 Acked Read 10 Done Sent Done false
32226 9825 Acked Write 2 Done Sent Done false
32227 9826 Acked Read 8 Done Sent Done false
32228 9827 Acked Read 7 Done Sent Done false
32229 9828 Acked Write 4 Done Sent Done false
32230 9829 Acked Read 4 Done Sent Done false
32231 9830 Acked Write 8 Done Sent Done false
32232 9831 Acked Write 2 Done Sent Done false
32233 9832 Acked Read 3 Done Sent Done false
32234 9833 Acked Read 2 Done Sent Done false
32235 9834 Acked Write 3 Done Sent Done false
32236 9835 Acked Write 6 Done Sent Done false
32237 9836 Acked Read 2 Done Sent Done false
32238 9837 Acked Write 4 Done Sent Done false
32239 9838 Acked Write 5 Done Sent Done false
32240 9839 Acked Write 8 Done Sent Done false
32241 9840 Acked Read 5 Done Sent Done false
32242 9841 Acked Write 1 Done Sent Done false
32243 9842 Acked Read 6 Done Sent Done false
32244 9843 Acked Write 3 Done Sent Done false
32245 9844 Acked Write 5 Done Sent Done false
32246 9845 Acked Write 9 Done Sent Done false
32247 9846 Acked Read 4 Done Sent Done false
32248 9847 Acked Read 5 Done Sent Done false
32249 9848 Acked Read 7 Done Sent Done false
32250 9849 Acked Write 1 Done Sent Done false
32251 9850 Acked Read 3 Done Sent Done false
32252 9851 Acked Read 6 Done Sent Done false
32253 9852 Acked Read 3 Done Sent Done false
32254 9853 Acked Write 4 Done Sent Done false
32255 9854 Acked Write 1 Done Sent Done false
32256 9855 Acked Write 10 Done Sent Done false
32257 9856 Acked Write 8 Done Sent Done false
32258 9857 Acked Read 3 Done Sent Done false
32259 9858 Acked Read 7 Done Sent Done false
32260 9859 Acked Write 3 Done Sent Done false
32261 9860 Acked Write 4 Done Sent Done false
32262 9861 Acked Read 1 Done Sent Done false
32263 9862 Acked Read 2 Done Sent Done false
32264 9863 Acked Read 5 Done Sent Done false
32265 9864 Acked Read 10 Done Sent Done false
32266 9865 Acked Write 9 Done Sent Done false
32267 9866 Acked Read 9 Done Sent Done false
32268 9867 Acked Write 10 Done Sent Done false
32269 9868 Acked Read 9 Done Sent Done false
32270 9869 Acked Write 10 Done Sent Done false
32271 9870 Acked Write 10 Done Sent Done false
32272 9871 Acked Read 10 Done Sent Done false
32273 9872 Acked Read 7 Done Sent Done false
32274 9873 Acked Write 3 Done Sent Done false
32275 9874 Acked Flush 0 Done Sent Done false
32276 9875 Acked Read 10 Done Sent Done false
32277 9876 Acked Write 10 Done Sent Done false
32278 9877 Acked Flush 0 Done Sent Done false
32279 9878 Acked Write 9 Done Sent Done false
32280 9879 Acked Flush 0 Done Sent Done false
32281 9880 Acked Write 3 Done Sent Done false
32282 9881 Acked Read 6 Done Sent Done false
32283 9882 Acked Write 1 Done Sent Done false
32284 9883 Acked Flush 0 Done Sent Done false
32285 9884 Acked Read 5 Done Sent Done false
32286 9885 Acked Read 2 Done Sent Done false
32287 9886 Acked Read 9 Done Sent Done false
32288 9887 Acked Read 2 Done Sent Done false
32289 9888 Acked Write 9 Done Sent Done false
32290 9889 Acked Write 9 Done Sent Done false
32291 9890 Acked Write 4 Done Sent Done false
32292 9891 Acked Read 1 Done Sent Done false
32293 9892 Acked Read 1 Done Sent Done false
32294 9893 Acked Read 5 Done Sent Done false
32295 9894 Acked Read 3 Done Sent Done false
32296 9895 Acked Read 8 Done Sent Done false
32297 9896 Acked Read 5 Done Sent Done false
32298 9897 Acked Read 4 Done Sent Done false
32299 9898 Acked Write 5 Done Sent Done false
32300 9899 Acked Read 8 Done Sent Done false
32301 9900 Acked Write 8 Done Sent Done false
32302 9901 Acked Read 6 Done Sent Done false
32303 9902 Acked Write 5 Done Sent Done false
32304 9903 Acked Write 1 Done Sent Done false
32305 9904 Acked Read 4 Done Sent Done false
32306 9905 Acked Read 5 Done Sent Done false
32307 9906 Acked Write 10 Done Sent Done false
32308 9907 Acked Write 6 Done Sent Done false
32309 9908 Acked Read 2 Done Sent Done false
32310 9909 Acked Read 6 Done Sent Done false
32311 9910 Acked Read 3 Done Sent Done false
32312 9911 Acked Flush 0 Done Sent Done false
32313 9912 Acked Write 5 Done Sent Done false
32314 9913 Acked Read 7 Done Sent Done false
32315 9914 Acked Write 4 Done Sent Done false
32316 9915 Acked Write 4 Done Sent Done false
32317 9916 Acked Write 5 Done Sent Done false
32318 9917 Acked Read 1 Done Sent Done false
32319 9918 Acked Read 10 Done Sent Done false
32320 9919 Acked Write 4 Done Sent Done false
32321 9920 Acked Flush 0 Done Sent Done false
32322 9921 Acked Read 9 Done Sent Done false
32323 9922 Acked Read 10 Done Sent Done false
32324 9923 Acked Read 3 Done Sent Done false
32325 9924 Acked Read 1 Done Sent Done false
32326 9925 Acked Write 9 Done Sent Done false
32327 9926 Acked Read 4 Done Sent Done false
32328 9927 Acked Read 1 Done Sent Done false
32329 9928 Acked Write 5 Done Sent Done false
32330 9929 Acked Write 10 Done Sent Done false
32331 9930 Acked Flush 0 Done Sent Done false
32332 9931 Acked Read 4 Done Sent Done false
32333 9932 Acked Read 1 Done Sent Done false
32334 9933 Acked Read 1 Done Sent Done false
32335 9934 Acked Write 8 Done Sent Done false
32336 9935 Acked Flush 0 Done Sent Done false
32337 9936 Acked Read 1 Done Sent Done false
32338 9937 Acked Write 8 Done Sent Done false
32339 9938 Acked Read 6 Done Sent Done false
32340 9939 Acked Flush 0 Done Sent Done false
32341 9940 Acked Read 8 Done Sent Done false
32342 9941 Acked Read 9 Done Sent Done false
32343 9942 Acked Write 6 Done Sent Done false
32344 9943 Acked Flush 0 Done Sent Done false
32345 9944 Acked Read 9 Done Sent Done false
32346 9945 Acked Write 1 Done Sent Done false
32347 9946 Acked Read 8 Done Sent Done false
32348 9947 Acked Read 4 Done Sent Done false
32349 9948 Acked Read 7 Done Sent Done false
32350 9949 Acked Write 2 Done Sent Done false
32351 9950 Acked Write 10 Done Sent Done false
32352 9951 Acked Read 3 Done Sent Done false
32353 9952 Acked Flush 0 Done Sent Done false
32354 9953 Acked Read 8 Done Sent Done false
32355 9954 Acked Read 5 Done Sent Done false
32356 9955 Acked Read 10 Done Sent Done false
32357 9956 Acked Write 4 Done Sent Done false
32358 9957 Acked Read 9 Done Sent Done false
32359 9958 Acked Flush 0 Done Sent Done false
32360 9959 Acked Read 7 Done Sent Done false
32361 9960 Acked Write 1 Done Sent Done false
32362 9961 Acked Write 2 Done Sent Done false
32363 9962 Acked Read 3 Done Sent Done false
32364 9963 Acked Read 3 Done Sent Done false
32365 9964 Acked Flush 0 Done Sent Done false
32366 9965 Acked Read 4 Done Sent Done false
32367 9966 Acked Write 6 Done Sent Done false
32368 9967 Acked Write 6 Done Sent Done false
32369 9968 Acked Read 2 Done Sent Done false
32370 9969 Acked Read 1 Done Sent Done false
32371 9970 Acked Read 8 Done Sent Done false
32372 9971 Acked Write 9 Done Sent Done false
32373 9972 Acked Read 8 Done Sent Done false
32374 9973 Acked Write 4 Done Sent Done false
32375 9974 Acked Write 7 Done Sent Done false
32376 9975 Acked Write 8 Done Sent Done false
32377 9976 Acked Read 10 Done Sent Done false
32378 9977 Acked Write 3 Done Sent Done false
32379 9978 Acked Write 9 Done Sent Done false
32380 9979 Acked Flush 0 Done Sent Done false
32381 9980 Acked Read 7 Done Sent Done false
32382 9981 Acked Read 2 Done Sent Done false
32383 9982 Acked Read 8 Done Sent Done false
32384 9983 Acked Read 5 Done Sent Done false
32385 9984 Acked Flush 0 Done Sent Done false
32386 9985 Acked Write 9 Done Sent Done false
32387 9986 Acked Read 2 Done Sent Done false
32388 9987 Acked Read 5 Done Sent Done false
32389 9988 Acked Flush 0 Done Sent Done false
32390 9989 Acked Write 10 Done Sent Done false
32391 9990 Acked Write 9 Done Sent Done false
32392 9991 Acked Read 6 Done Sent Done false
32393 9992 Acked Read 2 Done Sent Done false
32394 9993 Acked Read 6 Done Sent Done false
32395 9994 Acked Read 8 Done Sent Done false
32396 9995 Acked Flush 0 Done Sent Done false
32397 9996 Acked Write 5 Done Sent Done false
32398 9997 Acked Write 1 Done Sent Done false
32399 9998 Acked Read 5 Done Sent Done false
32400 9999 Acked Read 8 Done Sent Done false
32401 10000 Acked Read 1 Done Sent Done false
32402 10001 Acked Read 1 Done Sent Done false
32403 10002 Acked Read 9 Done Sent Done false
32404 10003 Acked Read 7 Done Sent Done false
32405 10004 Acked Read 2 Done Sent Done false
32406 10005 Acked Write 9 Done Sent Done false
32407 10006 Acked Write 8 Done Sent Done false
32408 10007 Acked Write 5 Done Sent Done false
32409 10008 Acked Write 2 Done Sent Done false
32410 10009 Acked Write 6 Done Sent Done false
32411 10010 Acked Read 10 Done Sent Done false
32412 10011 Acked Write 9 Done Sent Done false
32413 10012 Acked Read 2 Done Sent Done false
32414 10013 Acked Read 6 Done Sent Done false
32415 10014 Acked Read 2 Done Sent Done false
32416 10015 Acked Read 5 Done Sent Done false
32417 10016 Acked Read 6 Done Sent Done false
32418 10017 Acked Read 5 Done Sent Done false
32419 10018 Acked Read 4 Done Sent Done false
32420 10019 Acked Flush 0 Done Sent Done false
32421 10020 Acked Read 2 Done Sent Done false
32422 10021 Acked Read 3 Done Sent Done false
32423 10022 Acked Write 6 Done Sent Done false
32424 10023 Acked Write 5 Done Sent Done false
32425 10024 Acked Flush 0 Done Sent Done false
32426 10025 Acked Write 4 Done Sent Done false
32427 10026 Acked Write 5 Done Sent Done false
32428 10027 Acked Write 8 Done Sent Done false
32429 10028 Acked Write 8 Done Sent Done false
32430 10029 Acked Read 3 Done Sent Done false
32431 10030 Acked Write 6 Done Sent Done false
32432 10031 Acked Write 7 Done Sent Done false
32433 10032 Acked Flush 0 Done Sent Done false
32434 10033 Acked Read 2 Done Sent Done false
32435 10034 Acked Read 3 Done Sent Done false
32436 10035 Acked Write 10 Done Sent Done false
32437 10036 Acked Read 2 Done Sent Done false
32438 10037 Acked Write 1 Done Sent Done false
32439 10038 Acked Read 4 Done Sent Done false
32440 10039 Acked Write 9 Done Sent Done false
32441 10040 Acked Write 10 Done Sent Done false
32442 10041 Acked Write 8 Done Sent Done false
32443 10042 Acked Flush 0 Done Sent Done false
32444 10043 Acked Write 5 Done Sent Done false
32445 10044 Acked Read 2 Done Sent Done false
32446 10045 Acked Read 1 Done Sent Done false
32447 10046 Acked Read 1 Done Sent Done false
32448 10047 Acked Read 9 Done Sent Done false
32449 10048 Acked Read 9 Done Sent Done false
32450 10049 Acked Read 1 Done Sent Done false
32451 10050 Acked Flush 0 Done Sent Done false
32452 10051 Acked Read 9 Done Sent Done false
32453 10052 Acked Write 8 Done Sent Done false
32454 10053 Acked Read 10 Done Sent Done false
32455 10054 Acked Write 10 Done Sent Done false
32456 10055 Acked Read 7 Done Sent Done false
32457 10056 Acked Write 8 Done Sent Done false
32458 10057 Acked Flush 0 Done Sent Done false
32459 STATES DS:0 DS:1 DS:2 TOTAL
32460 Sent 0 302 0 302
32461 Done 302 0 302 604
32462 Skipped 0 0 0 0
32463 Error 0 0 0 0
32464 Last Flush: 10057 9755 10057
32465 Downstairs last five completed: 9755 9754 9753 9752 9751
32466 Upstairs last five completed: 10057 10056 10055 10054 10053
32467 CLIENT: Up:0 ds:302 act:2
324682025-01-13T15:58:36.320ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
324692025-01-13T15:58:36.320ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36216) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
324702025-01-13T15:58:36.322ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
324712025-01-13T15:58:36.322ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
324722025-01-13T15:58:36.322ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
324732025-01-13T15:58:36.322ZINFOcrucible: send last flush ID to this DS: 9755 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
324742025-01-13T15:58:36.323ZINFOcrucible: Replied this last flush ID: 9755 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
324752025-01-13T15:58:36.323ZINFOcrucible: [1] client re-new 302 jobs since flush 9755 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
324762025-01-13T15:58:36.323ZINFOcrucible: [1] Replayed 302 jobs since flush: 9755 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
32477 ----------------------------------------------------------------
32478 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
32479 Downstairs last five completed: 10057 10056 10055 10054 10053
32480 Upstairs last five completed: 10057 10056 10055 10054 10053
32481 CLIENT: Up:0 ds:0 act:3
32482 Replay: All jobs finished
32483 Main task has work to do, go find it
32484 got dsc Stop(1)
32485 stop 1
32486 Jan 13 15:58:38.562 INFO request completed[8820] Got stop action so:false kr:false
32487 , latency_us: 436, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 8d73162b-449d-4ab2-9f00-de24fa98dabd, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
32488 Jan 13 15:58:38.562 INFO request completed, latency_us: 122, response_code: 200, uri: /state/cid/1, method: GET, req_id{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-13T15:58:38.562739504Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"cd58429c-c491-4a1e-aa01-02db1d6a3d79"}
32489 : f826e995-d1b0-43cc-8e3e-2c0c6de42886, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
324902025-01-13T15:58:38.562ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
324912025-01-13T15:58:38.562ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
324922025-01-13T15:58:38.562ZINFOcrucible: client task is exiting = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
32493 [8820] Exited with: ExitStatus(unix_wait_status(9))
32494 [8820][1] reports Exit
32495 Jan 13 15:58:42.564 INFO request completed, latency_us: 321, response_code: 200, uri: /state/cid/1, method: GET, req_id: 74163b91-94a8-481c-aefc-feef50279d7a, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
32496 001/300 Read block 4576 len 24576
32497 002/300 Write block 1308 len 32768 data: 4 3 2 4 6 6 6 7
32498 003/300 Read block 1198 len 20480
32499 004/300 Read block 3625 len 28672
32500 005/300 Read block 2868 len 8192
32501 006/300 Write block 2413 len 4096 data: 7
32502 007/300 Read block 3484 len 20480
32503 008/300 Write block 1068 len 40960 data: 3 4 4 4 3 3 3 3 3 3
32504 009/300 Read block 791 len 20480
32505 010/300 Write block 1045 len 4096 data: 8
32506 011/300 Read block 201 len 32768
32507 012/300 Read block 2499 len 36864
32508 013/300 Write block 4154 len 36864 data: 4 5 5 5 4 3 3 4 3
32509 014/300 Read block 1817 len 40960
32510 015/300 Flush
32511 016/300 Read block 1895 len 8192
32512 017/300 Write block 4541 len 4096 data: 4
32513 018/300 Write block 3331 len 24576 data: 7 8 7 6 5 5
32514 019/300 Read block 1970 len 24576
32515 020/300 Read block 4812 len 12288
32516 021/300 Read block 3231 len 4096
32517 022/300 Read block 441 len 8192
32518 023/300 Read block 1555 len 20480
32519 024/300 Write block 721 len 16384 data: 5 6 7 4
32520 025/300 Write block 274 len 40960 data: 6 5 4 3 4 3 3 5 9 10
32521 026/300 Read block 2046 len 28672
32522 027/300 Flush
32523 028/300 Read block 2306 len 4096
32524 029/300 Read block 1262 len 24576
32525 030/300 Read block 2309 len 24576
32526 031/300 Write block 1217 len 4096 data: 5
32527 032/300 Read block 3049 len 8192
32528 033/300 Flush
32529 034/300 Write block 1380 len 8192 data: 11 9
32530 035/300 Write block 2510 len 4096 data: 2
32531 036/300 Write block 1078 len 40960 data: 7 7 9 7 8 9 9 9 9 8
32532 037/300 Read block 2340 len 12288
32533 038/300 Read block 38 len 16384
32534 039/300 Flush
32535 040/300 Read block 2836 len 16384
32536 041/300 Read block 4703 len 20480
32537 042/300 Write block 2122 len 8192 data: 8 7
32538 043/300 Write block 4144 len 28672 data: 3 4 5 5 4 4 5
32539 044/300 Read block 1150 len 4096
32540 045/300 Read block 513 len 32768
32541 046/300 Read block 2735 len 32768
32542 047/300 Write block 1889 len 32768 data: 2 2 4 5 6 6 6 4
32543 048/300 Read block 167 len 20480
32544 049/300 Write block 2355 len 36864 data: 4 4 4 5 4 4 2 2 3
32545 050/300 Write block 2053 len 4096 data: 8
32546 051/300 Write block 889 len 4096 data: 5
32547 052/300 Read block 1598 len 4096
32548 053/300 Write block 857 len 40960 data: 2 3 3 4 5 7 8 8 8 9
32549 054/300 Read block 2398 len 4096
32550 055/300 Read block 571 len 28672
32551 056/300 Read block 4688 len 8192
32552 057/300 Write block 130 len 32768 data: 8 7 7 7 6 8 6 7
32553 058/300 Write block 1690 len 12288 data: 2 2 2
32554 059/300 Read block 474 len 32768
32555 060/300 Write block 96 len 16384 data: 6 5 5 6
32556 061/300 Read block 4657 len 28672
32557 062/300 Read block 2275 len 40960
32558 063/300 Write block 1593 len 8192 data: 7 7
32559 064/300 Read block 2486 len 36864
32560 065/300 Read block 3052 len 28672
32561 066/300 Write block 3120 len 32768 data: 5 8 8 7 7 6 7 6
32562 067/300 Write block 3721 len 40960 data: 7 9 9 8 5 6 11 11 11 7
32563 068/300 Flush
32564 069/300 Write block 3471 len 12288 data: 11 10 11
32565 070/300 Write block 2922 len 12288 data: 5 4 3
32566 071/300 Read block 3271 len 36864
32567 072/300 Write block 552 len 32768 data: 3 4 4 3 3 4 5 4
32568 073/300 Read block 1133 len 8192
32569 074/300 Write block 2752 len 40960 data: 11 9 8 6 4 4 3 3 3 4
32570 075/300 Read block 3764 len 4096
32571 076/300 Write block 3307 len 16384 data: 6 6 6 5
32572 077/300 Write block 289 len 24576 data: 9 10 10 10 7 6
32573 078/300 Read block 1640 len 40960
32574 079/300 Read block 394 len 8192
32575 080/300 Write block 4342 len 16384 data: 5 7 6 7
32576 081/300 Read block 4214 len 20480
32577 082/300 Read block 2456 len 16384
32578 083/300 Write block 699 len 32768 data: 7 7 8 7 8 6 8 10
32579 084/300 Write block 2870 len 12288 data: 10 9 9
32580 085/300 Read block 2730 len 4096
32581 086/300 Read block 3655 len 32768
32582 087/300 Write block 2730 len 40960 data: 6 6 6 6 7 6 6 6 5 4
32583 088/300 Flush
32584 089/300 Write block 2201 len 40960 data: 7 6 7 9 10 8 6 7 9 9
32585 090/300 Write block 244 len 28672 data: 5 5 6 6 3 2 2
32586 091/300 Read block 4047 len 40960
32587 092/300 Read block 932 len 40960
32588 093/300 Read block 3019 len 40960
32589 094/300 Read block 1709 len 12288
32590 095/300 Write block 646 len 16384 data: 8 6 5 6
32591 096/300 Flush
32592 097/300 Write block 2577 len 12288 data: 5 5 7
32593 098/300 Write block 1884 len 36864 data: 4 4 3 3 2 3 3 5 6
32594 099/300 Write block 112 len 4096 data: 6
32595 100/300 Read block 511 len 20480
32596 101/300 Write block 4248 len 12288 data: 4 5 7
32597 102/300 Write block 3002 len 36864 data: 9 8 9 8 8 8 7 8 9
32598 103/300 Read block 3154 len 24576
32599 104/300 Flush
32600 105/300 Write block 1604 len 20480 data: 6 7 7 6 5
32601 106/300 Read block 2756 len 24576
32602 107/300 Read block 1691 len 12288
32603 108/300 Read block 4319 len 24576
32604 109/300 Write block 436 len 32768 data: 5 5 5 4 3 3 3 4
32605 110/300 Write block 4888 len 12288 data: 13 11 9
32606 111/300 Read block 2268 len 16384
32607 112/300 Read block 219 len 4096
32608 113/300 Read block 2652 len 40960
32609 114/300 Read block 209 len 40960
32610 115/300 Read block 612 len 36864
32611 116/300 Read block 3319 len 24576
32612 117/300 Read block 363 len 32768
32613 118/300 Read block 3084 len 16384
32614 119/300 Write block 1699 len 12288 data: 7 6 6
32615 120/300 Write block 1530 len 28672 data: 8 7 5 4 4 3 3
32616 121/300 Read block 1041 len 8192
32617 122/300 Write block 328 len 16384 data: 6 8 8 6
32618 123/300 Read block 1536 len 28672
32619 124/300 Read block 3766 len 20480
32620 125/300 Read block 2564 len 40960
32621 126/300 Write block 1968 len 16384 data: 9 10 8 9
32622 127/300 Write block 727 len 40960 data: 7 7 7 8 8 9 7 6 4 4
32623 128/300 Flush
32624 129/300 Flush
32625 130/300 Read block 665 len 40960
32626 131/300 Read block 440 len 32768
32627 132/300 Write block 2201 len 16384 data: 8 7 8 10
32628 133/300 Write block 4571 len 28672 data: 4 5 5 5 5 3 4
32629 134/300 Write block 4301 len 4096 data: 11
32630 135/300 Read block 3464 len 40960
32631 136/300 Read block 3343 len 12288
32632 137/300 Read block 326 len 32768
32633 138/300 Read block 3866 len 36864
32634 139/300 Write block 110 len 16384 data: 8 7 7 5
32635 140/300 Read block 671 len 4096
32636 141/300 Write block 2037 len 16384 data: 4 5 6 5
32637 142/300 Read block 3049 len 12288
32638 143/300 Read block 4 len 24576
32639 144/300 Write block 3474 len 24576 data: 13 9 8 6 4 4
32640 145/300 Read block 3590 len 24576
32641 146/300 Read block 671 len 16384
32642 147/300 Read block 1571 len 12288
32643 148/300 Read block 4937 len 32768
32644 149/300 Read block 3858 len 32768
32645 150/300 Read block 4000 len 32768
32646 151/300 Read block 3638 len 12288
32647 152/300 Write block 3582 len 12288 data: 8 8 7
32648 153/300 Write block 4736 len 4096 data: 11
32649 154/300 Read block 1149 len 16384
32650 155/300 Read block 4402 len 32768
32651 156/300 Write block 4541 len 40960 data: 5 4 5 5 5 6 7 7 6 5
32652 157/300 Read block 4198 len 16384
32653 158/300 Write block 3026 len 24576 data: 9 11 11 7 5 5
32654 159/300 Read block 4560 len 8192
32655 160/300 Read block 4601 len 20480
32656 161/300 Read block 1439 len 16384
32657 162/300 Write block 3289 len 12288 data: 6 6 5
32658 163/300 Read block 265 len 36864
32659 164/300 Write block 4766 len 12288 data: 4 3 4
32660 165/300 Read block 1343 len 40960
32661 166/300 Read block 1457 len 8192
32662 167/300 Write block 930 len 36864 data: 3 3 3 4 4 4 4 4 3
32663 168/300 Write block 4348 len 4096 data: 5
32664 169/300 Write block 4331 len 28672 data: 8 7 7 7 7 8 7
32665 170/300 Read block 2719 len 4096
32666 171/300 Read block 1932 len 16384
32667 172/300 Write block 940 len 28672 data: 6 7 10 11 12 11 13
32668 173/300 Read block 3659 len 16384
32669 174/300 Read block 2109 len 36864
32670 175/300 Write block 3253 len 16384 data: 4 4 3 5
32671 176/300 Read block 3310 len 28672
32672 177/300 Read block 4566 len 12288
32673 178/300 Write block 99 len 36864 data: 7 6 6 6 7 7 8 10 10
32674 179/300 Write block 4033 len 40960 data: 7 7 7 5 4 4 3 3 2 3
32675 180/300 Read block 1002 len 16384
32676 181/300 Read block 1742 len 40960
32677 182/300 Write block 882 len 8192 data: 8 7
32678 183/300 Write block 625 len 36864 data: 4 5 5 5 5 4 5 5 6
32679 184/300 Read block 3811 len 36864
32680 185/300 Write block 1152 len 36864 data: 6 5 4 5 5 4 3 3 4
32681 186/300 Write block 3006 len 16384 data: 9 9 8 9
32682 187/300 Write block 493 len 40960 data: 2 2 2 3 5 5 4 6 7 8
32683 188/300 Read block 3813 len 36864
32684 189/300 Read block 2817 len 20480
32685 190/300 Read block 2769 len 28672
32686 191/300 Write block 3850 len 4096 data: 8
32687 192/300 Read block 3673 len 12288
32688 193/300 Read block 2122 len 16384
32689 194/300 Read block 83 len 8192
32690 195/300 Read block 758 len 36864
32691 196/300 Write block 4409 len 16384 data: 6 6 6 6
32692 197/300 Write block 1533 len 28672 data: 5 5 4 4 3 6 5
32693 198/300 Read block 1181 len 28672
32694 199/300 Write block 2800 len 16384 data: 8 6 7 8
32695 200/300 Flush
32696 201/300 Read block 1677 len 8192
32697 202/300 Write block 2255 len 40960 data: 5 5 5 5 6 5 3 5 8 9
32698 203/300 Read block 2399 len 40960
32699 204/300 Read block 3296 len 36864
32700 205/300 Read block 2387 len 8192
32701 206/300 Flush
32702 207/300 Write block 237 len 24576 data: 5 6 6 8 8 6
32703 208/300 Read block 3147 len 16384
32704 209/300 Read block 3275 len 40960
32705 210/300 Read block 3288 len 12288
32706 211/300 Write block 4143 len 16384 data: 3 4 5 6
32707 212/300 Read block 4416 len 24576
32708 213/300 Read block 4794 len 16384
32709 214/300 Read block 3100 len 8192
32710 215/300 Read block 1655 len 40960
32711 216/300 Write block 368 len 40960 data: 7 5 5 6 6 8 7 11 11 11
32712 217/300 Read block 4200 len 36864
32713 218/300 Write block 1357 len 28672 data: 5 4 4 6 5 5 5
32714 219/300 Flush
32715 220/300 Flush
32716 221/300 Read block 3549 len 20480
32717 222/300 Read block 2205 len 40960
32718 223/300 Write block 526 len 36864 data: 5 7 5 6 6 7 6 5 7
32719 224/300 Read block 4933 len 12288
32720 225/300 Read block 1480 len 40960
32721 226/300 Read block 624 len 20480
32722 227/300 Flush
32723 228/300 Write block 2215 len 24576 data: 9 8 7 7 5 6
32724 229/300 Read block 4274 len 36864
32725 230/300 Read block 855 len 24576
32726 231/300 Read block 273 len 40960
32727 232/300 Read block 3038 len 12288
32728 233/300 Read block 2331 len 40960
32729 234/300 Read block 3456 len 4096
32730 235/300 Flush
32731 236/300 Read block 3942 len 24576
32732 237/300 Read block 86 len 28672
32733 238/300 Write block 2509 len 8192 data: 3 3
32734 239/300 Flush
32735 240/300 Write block 4962 len 24576 data: 6 5 5 5 7 8
32736 241/300 Write block 480 len 12288 data: 9 11 9
32737 242/300 Write block 3273 len 4096 data: 11
32738 243/300 Read block 732 len 16384
32739 244/300 Read block 3348 len 20480
32740 245/300 Read block 4157 len 24576
32741 246/300 Read block 4935 len 12288
32742 247/300 Read block 1246 len 4096
32743 248/300 Read block 10 len 24576
32744 249/300 Flush
32745 250/300 Write block 4412 len 20480 data: 7 4 2 2 2
32746 251/300 Read block 1455 len 40960
32747 252/300 Read block 4927 len 24576
32748 253/300 Read block 1911 len 12288
32749 254/300 Read block 4120 len 16384
32750 255/300 Flush
32751 256/300 Write block 4482 len 36864 data: 6 6 6 5 4 4 4 4 3
32752 257/300 Write block 405 len 16384 data: 7 5 7 5
32753 258/300 Read block 2973 len 8192
32754 259/300 Write block 2665 len 36864 data: 5 5 4 3 4 3 3 4 5
32755 260/300 Read block 3693 len 12288
32756 261/300 Write block 4630 len 32768 data: 5 6 6 5 3 3 4 5
32757 262/300 Read block 159 len 20480
32758 263/300 Write block 1654 len 32768 data: 4 3 3 4 4 4 6 6
32759 264/300 Read block 3303 len 4096
32760 265/300 Read block 1540 len 16384
32761 266/300 Read block 4152 len 36864
32762 267/300 Flush
32763 268/300 Write block 2303 len 40960 data: 5 4 4 4 5 8 9 10 9 6
32764 269/300 Read block 4039 len 12288
32765 270/300 Write block 2755 len 28672 data: 7 5 5 4 4 4 5
32766 271/300 Read block 4677 len 28672
32767 272/300 Read block 550 len 8192
32768 273/300 Read block 4990 len 32768
32769 274/300 Write block 1207 len 16384 data: 6 5 5 6
32770 275/300 Read block 4148 len 12288
32771 276/300 Write block 2194 len 4096 data: 6
32772 277/300 Write block 4939 len 20480 data: 4 7 7 6 8
32773 278/300 Read block 3735 len 8192
32774 279/300 Write block 2944 len 20480 data: 6 6 7 6 5
32775 280/300 Write block 4754 len 32768 data: 5 5 6 5 2 2 2 3
32776 281/300 Read block 3762 len 28672
32777 282/300 Write block 4605 len 12288 data: 7 6 6
32778 283/300 Read block 3024 len 4096
32779 284/300 Read block 3978 len 8192
32780 285/300 Write block 3944 len 32768 data: 4 3 3 3 4 4 5 5
32781 286/300 Write block 2845 len 32768 data: 5 4 7 7 5 5 5 5
32782 287/300 Flush
32783 288/300 Flush
32784 289/300 Write block 25 len 28672 data: 8 7 9 9 10 10 8
32785 290/300 Flush
32786 291/300 Write block 3879 len 16384 data: 4 4 4 4
32787 292/300 Write block 1027 len 32768 data: 9 7 6 7 5 5 5 5
32788 293/300 Write block 4573 len 16384 data: 6 6 6 4
32789 294/300 Write block 4394 len 4096 data: 6
32790 295/300 Flush
32791 296/300 Flush
32792 297/300 Flush
32793 298/300 Read block 638 len 12288
32794 299/300 Write block 382 len 20480 data: 5 6 6 7 9
32795 300/300 Read block 2251 len 32768
32796 301/300 Read block 2475 len 12288
32797 Main task has work to do, go find it
32798 got dsc Start(1)
32799 start 1
32800 Jan 13 15:58:42.813 INFO request completed, latency_us: 213, response_code: 204, uri: /start/cid/1, method: POST, req_id: b165200b-4212-4231-ad02-80b024314c9e, remote_addr: 127.0.0.1:52422, local_addr: 127.0.0.1:9998
32801 Replay: started 1, returned:Ok(())
32802 ----------------------------------------------------------------
32803 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
32804 States: Active Offline Active
32805 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32806 10058 Acked Read 6 Done Sent Done false
32807 10059 Acked Write 8 Done Sent Done false
32808 10060 Acked Read 5 Done Sent Done false
32809 10061 Acked Read 7 Done Sent Done false
32810 10062 Acked Read 2 Done Sent Done false
32811 10063 Acked Write 1 Done Sent Done false
32812 10064 Acked Read 5 Done Sent Done false
32813 10065 Acked Write 10 Done Sent Done false
32814 10066 Acked Read 5 Done Sent Done false
32815 10067 Acked Write 1 Done Sent Done false
32816 10068 Acked Read 8 Done Sent Done false
32817 10069 Acked Read 9 Done Sent Done false
32818 10070 Acked Write 9 Done Sent Done false
32819 10071 Acked Read 10 Done Sent Done false
32820 10072 Acked Flush 0 Done Sent Done false
32821 10073 Acked Read 2 Done Sent Done false
32822 10074 Acked Write 1 Done Sent Done false
32823 10075 Acked Write 6 Done Sent Done false
32824 10076 Acked Read 6 Done Sent Done false
32825 10077 Acked Read 3 Done Sent Done false
32826 10078 Acked Read 1 Done Sent Done false
32827 10079 Acked Read 2 Done Sent Done false
32828 10080 Acked Read 5 Done Sent Done false
32829 10081 Acked Write 4 Done Sent Done false
32830 10082 Acked Write 10 Done Sent Done false
32831 10083 Acked Read 7 Done Sent Done false
32832 10084 Acked Flush 0 Done Sent Done false
32833 10085 Acked Read 1 Done Sent Done false
32834 10086 Acked Read 6 Done Sent Done false
32835 10087 Acked Read 6 Done Sent Done false
32836 10088 Acked Write 1 Done Sent Done false
32837 10089 Acked Read 2 Done Sent Done false
32838 10090 Acked Flush 0 Done Sent Done false
32839 10091 Acked Write 2 Done Sent Done false
32840 10092 Acked Write 1 Done Sent Done false
32841 10093 Acked Write 10 Done Sent Done false
32842 10094 Acked Read 3 Done Sent Done false
32843 10095 Acked Read 4 Done Sent Done false
32844 10096 Acked Flush 0 Done Sent Done false
32845 10097 Acked Read 4 Done Sent Done false
32846 10098 Acked Read 5 Done Sent Done false
32847 10099 Acked Write 2 Done Sent Done false
32848 10100 Acked Write 7 Done Sent Done false
32849 10101 Acked Read 1 Done Sent Done false
32850 10102 Acked Read 8 Done Sent Done false
32851 10103 Acked Read 8 Done Sent Done false
32852 10104 Acked Write 8 Done Sent Done false
32853 10105 Acked Read 5 Done Sent Done false
32854 10106 Acked Write 9 Done Sent Done false
32855 10107 Acked Write 1 Done Sent Done false
32856 10108 Acked Write 1 Done Sent Done false
32857 10109 Acked Read 1 Done Sent Done false
32858 10110 Acked Write 10 Done Sent Done false
32859 10111 Acked Read 1 Done Sent Done false
32860 10112 Acked Read 7 Done Sent Done false
32861 10113 Acked Read 2 Done Sent Done false
32862 10114 Acked Write 8 Done Sent Done false
32863 10115 Acked Write 3 Done Sent Done false
32864 10116 Acked Read 8 Done Sent Done false
32865 10117 Acked Write 4 Done Sent Done false
32866 10118 Acked Read 7 Done Sent Done false
32867 10119 Acked Read 10 Done Sent Done false
32868 10120 Acked Write 2 Done Sent Done false
32869 10121 Acked Read 9 Done Sent Done false
32870 10122 Acked Read 7 Done Sent Done false
32871 10123 Acked Write 8 Done Sent Done false
32872 10124 Acked Write 10 Done Sent Done false
32873 10125 Acked Flush 0 Done Sent Done false
32874 10126 Acked Write 3 Done Sent Done false
32875 10127 Acked Write 3 Done Sent Done false
32876 10128 Acked Read 9 Done Sent Done false
32877 10129 Acked Write 8 Done Sent Done false
32878 10130 Acked Read 2 Done Sent Done false
32879 10131 Acked Write 10 Done Sent Done false
32880 10132 Acked Read 1 Done Sent Done false
32881 10133 Acked Write 4 Done Sent Done false
32882 10134 Acked Write 6 Done Sent Done false
32883 10135 Acked Read 10 Done Sent Done false
32884 10136 Acked Read 2 Done Sent Done false
32885 10137 Acked Write 4 Done Sent Done false
32886 10138 Acked Read 5 Done Sent Done false
32887 10139 Acked Read 4 Done Sent Done false
32888 10140 Acked Write 8 Done Sent Done false
32889 10141 Acked Write 3 Done Sent Done false
32890 10142 Acked Read 1 Done Sent Done false
32891 10143 Acked Read 8 Done Sent Done false
32892 10144 Acked Write 10 Done Sent Done false
32893 10145 Acked Flush 0 Done Sent Done false
32894 10146 Acked Write 10 Done Sent Done false
32895 10147 Acked Write 7 Done Sent Done false
32896 10148 Acked Read 10 Done Sent Done false
32897 10149 Acked Read 10 Done Sent Done false
32898 10150 Acked Read 10 Done Sent Done false
32899 10151 Acked Read 3 Done Sent Done false
32900 10152 Acked Write 4 Done Sent Done false
32901 10153 Acked Flush 0 Done Sent Done false
32902 10154 Acked Write 3 Done Sent Done false
32903 10155 Acked Write 9 Done Sent Done false
32904 10156 Acked Write 1 Done Sent Done false
32905 10157 Acked Read 5 Done Sent Done false
32906 10158 Acked Write 3 Done Sent Done false
32907 10159 Acked Write 9 Done Sent Done false
32908 10160 Acked Read 6 Done Sent Done false
32909 10161 Acked Flush 0 Done Sent Done false
32910 10162 Acked Write 5 Done Sent Done false
32911 10163 Acked Read 6 Done Sent Done false
32912 10164 Acked Read 3 Done Sent Done false
32913 10165 Acked Read 6 Done Sent Done false
32914 10166 Acked Write 8 Done Sent Done false
32915 10167 Acked Write 3 Done Sent Done false
32916 10168 Acked Read 4 Done Sent Done false
32917 10169 Acked Read 1 Done Sent Done false
32918 10170 Acked Read 10 Done Sent Done false
32919 10171 Acked Read 10 Done Sent Done false
32920 10172 Acked Read 9 Done Sent Done false
32921 10173 Acked Read 6 Done Sent Done false
32922 10174 Acked Read 8 Done Sent Done false
32923 10175 Acked Read 4 Done Sent Done false
32924 10176 Acked Write 3 Done Sent Done false
32925 10177 Acked Write 7 Done Sent Done false
32926 10178 Acked Read 2 Done Sent Done false
32927 10179 Acked Write 4 Done Sent Done false
32928 10180 Acked Read 7 Done Sent Done false
32929 10181 Acked Read 5 Done Sent Done false
32930 10182 Acked Read 10 Done Sent Done false
32931 10183 Acked Write 4 Done Sent Done false
32932 10184 Acked Write 10 Done Sent Done false
32933 10185 Acked Flush 0 Done Sent Done false
32934 10186 Acked Flush 0 Done Sent Done false
32935 10187 Acked Read 10 Done Sent Done false
32936 10188 Acked Read 8 Done Sent Done false
32937 10189 Acked Write 4 Done Sent Done false
32938 10190 Acked Write 7 Done Sent Done false
32939 10191 Acked Write 1 Done Sent Done false
32940 10192 Acked Read 10 Done Sent Done false
32941 10193 Acked Read 3 Done Sent Done false
32942 10194 Acked Read 8 Done Sent Done false
32943 10195 Acked Read 9 Done Sent Done false
32944 10196 Acked Write 4 Done Sent Done false
32945 10197 Acked Read 1 Done Sent Done false
32946 10198 Acked Write 4 Done Sent Done false
32947 10199 Acked Read 3 Done Sent Done false
32948 10200 Acked Read 6 Done Sent Done false
32949 10201 Acked Write 6 Done Sent Done false
32950 10202 Acked Read 6 Done Sent Done false
32951 10203 Acked Read 4 Done Sent Done false
32952 10204 Acked Read 3 Done Sent Done false
32953 10205 Acked Read 8 Done Sent Done false
32954 10206 Acked Read 8 Done Sent Done false
32955 10207 Acked Read 8 Done Sent Done false
32956 10208 Acked Read 3 Done Sent Done false
32957 10209 Acked Write 3 Done Sent Done false
32958 10210 Acked Write 1 Done Sent Done false
32959 10211 Acked Read 4 Done Sent Done false
32960 10212 Acked Read 8 Done Sent Done false
32961 10213 Acked Write 10 Done Sent Done false
32962 10214 Acked Read 4 Done Sent Done false
32963 10215 Acked Write 6 Done Sent Done false
32964 10216 Acked Read 2 Done Sent Done false
32965 10217 Acked Read 5 Done Sent Done false
32966 10218 Acked Read 4 Done Sent Done false
32967 10219 Acked Write 3 Done Sent Done false
32968 10220 Acked Read 9 Done Sent Done false
32969 10221 Acked Write 3 Done Sent Done false
32970 10222 Acked Read 10 Done Sent Done false
32971 10223 Acked Read 2 Done Sent Done false
32972 10224 Acked Write 9 Done Sent Done false
32973 10225 Acked Write 1 Done Sent Done false
32974 10226 Acked Write 7 Done Sent Done false
32975 10227 Acked Read 1 Done Sent Done false
32976 10228 Acked Read 4 Done Sent Done false
32977 10229 Acked Write 7 Done Sent Done false
32978 10230 Acked Read 4 Done Sent Done false
32979 10231 Acked Read 9 Done Sent Done false
32980 10232 Acked Write 4 Done Sent Done false
32981 10233 Acked Read 7 Done Sent Done false
32982 10234 Acked Read 3 Done Sent Done false
32983 10235 Acked Write 9 Done Sent Done false
32984 10236 Acked Write 10 Done Sent Done false
32985 10237 Acked Read 4 Done Sent Done false
32986 10238 Acked Read 10 Done Sent Done false
32987 10239 Acked Write 2 Done Sent Done false
32988 10240 Acked Write 9 Done Sent Done false
32989 10241 Acked Read 9 Done Sent Done false
32990 10242 Acked Write 9 Done Sent Done false
32991 10243 Acked Write 4 Done Sent Done false
32992 10244 Acked Write 10 Done Sent Done false
32993 10245 Acked Read 9 Done Sent Done false
32994 10246 Acked Read 5 Done Sent Done false
32995 10247 Acked Read 7 Done Sent Done false
32996 10248 Acked Write 1 Done Sent Done false
32997 10249 Acked Read 3 Done Sent Done false
32998 10250 Acked Read 4 Done Sent Done false
32999 10251 Acked Read 2 Done Sent Done false
33000 10252 Acked Read 9 Done Sent Done false
33001 10253 Acked Write 4 Done Sent Done false
33002 10254 Acked Write 7 Done Sent Done false
33003 10255 Acked Read 7 Done Sent Done false
33004 10256 Acked Write 4 Done Sent Done false
33005 10257 Acked Flush 0 Done Sent Done false
33006 10258 Acked Read 2 Done Sent Done false
33007 10259 Acked Write 10 Done Sent Done false
33008 10260 Acked Read 10 Done Sent Done false
33009 10261 Acked Read 9 Done Sent Done false
33010 10262 Acked Read 2 Done Sent Done false
33011 10263 Acked Flush 0 Done Sent Done false
33012 10264 Acked Write 6 Done Sent Done false
33013 10265 Acked Read 4 Done Sent Done false
33014 10266 Acked Read 10 Done Sent Done false
33015 10267 Acked Read 3 Done Sent Done false
33016 10268 Acked Write 4 Done Sent Done false
33017 10269 Acked Read 6 Done Sent Done false
33018 10270 Acked Read 4 Done Sent Done false
33019 10271 Acked Read 2 Done Sent Done false
33020 10272 Acked Read 10 Done Sent Done false
33021 10273 Acked Write 10 Done Sent Done false
33022 10274 Acked Read 9 Done Sent Done false
33023 10275 Acked Write 7 Done Sent Done false
33024 10276 Acked Flush 0 Done Sent Done false
33025 10277 Acked Flush 0 Done Sent Done false
33026 10278 Acked Read 5 Done Sent Done false
33027 10279 Acked Read 10 Done Sent Done false
33028 10280 Acked Write 9 Done Sent Done false
33029 10281 Acked Read 3 Done Sent Done false
33030 10282 Acked Read 10 Done Sent Done false
33031 10283 Acked Read 5 Done Sent Done false
33032 10284 Acked Flush 0 Done Sent Done false
33033 10285 Acked Write 6 Done Sent Done false
33034 10286 Acked Read 9 Done Sent Done false
33035 10287 Acked Read 6 Done Sent Done false
33036 10288 Acked Read 10 Done Sent Done false
33037 10289 Acked Read 3 Done Sent Done false
33038 10290 Acked Read 10 Done Sent Done false
33039 10291 Acked Read 1 Done Sent Done false
33040 10292 Acked Flush 0 Done Sent Done false
33041 10293 Acked Read 6 Done Sent Done false
33042 10294 Acked Read 7 Done Sent Done false
33043 10295 Acked Write 2 Done Sent Done false
33044 10296 Acked Flush 0 Done Sent Done false
33045 10297 Acked Write 6 Done Sent Done false
33046 10298 Acked Write 3 Done Sent Done false
33047 10299 Acked Write 1 Done Sent Done false
33048 10300 Acked Read 4 Done Sent Done false
33049 10301 Acked Read 5 Done Sent Done false
33050 10302 Acked Read 6 Done Sent Done false
33051 10303 Acked Read 3 Done Sent Done false
33052 10304 Acked Read 1 Done Sent Done false
33053 10305 Acked Read 6 Done Sent Done false
33054 10306 Acked Flush 0 Done Sent Done false
33055 10307 Acked Write 5 Done Sent Done false
33056 10308 Acked Read 10 Done Sent Done false
33057 10309 Acked Read 6 Done Sent Done false
33058 10310 Acked Read 3 Done Sent Done false
33059 10311 Acked Read 4 Done Sent Done false
33060 10312 Acked Flush 0 Done Sent Done false
33061 10313 Acked Write 9 Done Sent Done false
33062 10314 Acked Write 4 Done Sent Done false
33063 10315 Acked Read 2 Done Sent Done false
33064 10316 Acked Write 9 Done Sent Done false
33065 10317 Acked Read 3 Done Sent Done false
33066 10318 Acked Write 8 Done Sent Done false
33067 10319 Acked Read 5 Done Sent Done false
33068 10320 Acked Write 8 Done Sent Done false
33069 10321 Acked Read 1 Done Sent Done false
33070 10322 Acked Read 4 Done Sent Done false
33071 10323 Acked Read 9 Done Sent Done false
33072 10324 Acked Flush 0 Done Sent Done false
33073 10325 Acked Write 10 Done Sent Done false
33074 10326 Acked Read 3 Done Sent Done false
33075 10327 Acked Write 7 Done Sent Done false
33076 10328 Acked Read 7 Done Sent Done false
33077 10329 Acked Read 2 Done Sent Done false
33078 10330 Acked Read 8 Done Sent Done false
33079 10331 Acked Write 4 Done Sent Done false
33080 10332 Acked Read 3 Done Sent Done false
33081 10333 Acked Write 1 Done Sent Done false
33082 10334 Acked Write 5 Done Sent Done false
33083 10335 Acked Read 2 Done Sent Done false
33084 10336 Acked Write 5 Done Sent Done false
33085 10337 Acked Write 8 Done Sent Done false
33086 10338 Acked Read 7 Done Sent Done false
33087 10339 Acked Write 3 Done Sent Done false
33088 10340 Acked Read 1 Done Sent Done false
33089 10341 Acked Read 2 Done Sent Done false
33090 10342 Acked Write 8 Done Sent Done false
33091 10343 Acked Write 8 Done Sent Done false
33092 10344 Acked Flush 0 Done Sent Done false
33093 10345 Acked Flush 0 Done Sent Done false
33094 10346 Acked Write 7 Done Sent Done false
33095 10347 Acked Flush 0 Done Sent Done false
33096 10348 Acked Write 4 Done Sent Done false
33097 10349 Acked Write 8 Done Sent Done false
33098 10350 Acked Write 4 Done Sent Done false
33099 10351 Acked Write 1 Done Sent Done false
33100 10352 Acked Flush 0 Done Sent Done false
33101 10353 Acked Flush 0 Done Sent Done false
33102 10354 Acked Flush 0 Done Sent Done false
33103 10355 Acked Read 3 Done Sent Done false
33104 10356 Acked Write 5 Done Sent Done false
33105 10357 Acked Read 8 Done Sent Done false
33106 10358 Acked Read 3 Done Sent Done false
33107 STATES DS:0 DS:1 DS:2 TOTAL
33108 Sent 0 301 0 301
33109 Done 301 0 301 602
33110 Skipped 0 0 0 0
33111 Error 0 0 0 0
33112 Last Flush: 10354 10057 10354
33113 Downstairs last five completed: 10057 10056 10055 10054 10053
33114 Upstairs last five completed: 10358 10357 10356 10355 10354
33115 CLIENT: Up:0 ds:301 act:2
33116 [8820] got start action
33117 [8820] I am going to restart
33118 Starting downstairs at port 8820
33119 Make output file at "/tmp/dsc/downstairs-8820.txt"
33120 [8820][1] reports Starting
33121 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1233)
33122 [8820][1] reports Running
33123 ----------------------------------------------------------------
33124 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
33125 States: Active Offline Active
33126 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33127 10058 Acked Read 6 Done Sent Done false
33128 10059 Acked Write 8 Done Sent Done false
33129 10060 Acked Read 5 Done Sent Done false
33130 10061 Acked Read 7 Done Sent Done false
33131 10062 Acked Read 2 Done Sent Done false
33132 10063 Acked Write 1 Done Sent Done false
33133 10064 Acked Read 5 Done Sent Done false
33134 10065 Acked Write 10 Done Sent Done false
33135 10066 Acked Read 5 Done Sent Done false
33136 10067 Acked Write 1 Done Sent Done false
33137 10068 Acked Read 8 Done Sent Done false
33138 10069 Acked Read 9 Done Sent Done false
33139 10070 Acked Write 9 Done Sent Done false
33140 10071 Acked Read 10 Done Sent Done false
33141 10072 Acked Flush 0 Done Sent Done false
33142 10073 Acked Read 2 Done Sent Done false
33143 10074 Acked Write 1 Done Sent Done false
33144 10075 Acked Write 6 Done Sent Done false
33145 10076 Acked Read 6 Done Sent Done false
33146 10077 Acked Read 3 Done Sent Done false
33147 10078 Acked Read 1 Done Sent Done false
33148 10079 Acked Read 2 Done Sent Done false
33149 10080 Acked Read 5 Done Sent Done false
33150 10081 Acked Write 4 Done Sent Done false
33151 10082 Acked Write 10 Done Sent Done false
33152 10083 Acked Read 7 Done Sent Done false
33153 10084 Acked Flush 0 Done Sent Done false
33154 10085 Acked Read 1 Done Sent Done false
33155 10086 Acked Read 6 Done Sent Done false
33156 10087 Acked Read 6 Done Sent Done false
33157 10088 Acked Write 1 Done Sent Done false
33158 10089 Acked Read 2 Done Sent Done false
33159 10090 Acked Flush 0 Done Sent Done false
33160 10091 Acked Write 2 Done Sent Done false
33161 10092 Acked Write 1 Done Sent Done false
33162 10093 Acked Write 10 Done Sent Done false
33163 10094 Acked Read 3 Done Sent Done false
33164 10095 Acked Read 4 Done Sent Done false
33165 10096 Acked Flush 0 Done Sent Done false
33166 10097 Acked Read 4 Done Sent Done false
33167 10098 Acked Read 5 Done Sent Done false
33168 10099 Acked Write 2 Done Sent Done false
33169 10100 Acked Write 7 Done Sent Done false
33170 10101 Acked Read 1 Done Sent Done false
33171 10102 Acked Read 8 Done Sent Done false
33172 10103 Acked Read 8 Done Sent Done false
33173 10104 Acked Write 8 Done Sent Done false
33174 10105 Acked Read 5 Done Sent Done false
33175 10106 Acked Write 9 Done Sent Done false
33176 10107 Acked Write 1 Done Sent Done false
33177 10108 Acked Write 1 Done Sent Done false
33178 10109 Acked Read 1 Done Sent Done false
33179 10110 Acked Write 10 Done Sent Done false
33180 10111 Acked Read 1 Done Sent Done false
33181 10112 Acked Read 7 Done Sent Done false
33182 10113 Acked Read 2 Done Sent Done false
33183 10114 Acked Write 8 Done Sent Done false
33184 10115 Acked Write 3 Done Sent Done false
33185 10116 Acked Read 8 Done Sent Done false
33186 10117 Acked Write 4 Done Sent Done false
33187 10118 Acked Read 7 Done Sent Done false
33188 10119 Acked Read 10 Done Sent Done false
33189 10120 Acked Write 2 Done Sent Done false
33190 10121 Acked Read 9 Done Sent Done false
33191 10122 Acked Read 7 Done Sent Done false
33192 10123 Acked Write 8 Done Sent Done false
33193 10124 Acked Write 10 Done Sent Done false
33194 10125 Acked Flush 0 Done Sent Done false
33195 10126 Acked Write 3 Done Sent Done false
33196 10127 Acked Write 3 Done Sent Done false
33197 10128 Acked Read 9 Done Sent Done false
33198 10129 Acked Write 8 Done Sent Done false
33199 10130 Acked Read 2 Done Sent Done false
33200 10131 Acked Write 10 Done Sent Done false
33201 10132 Acked Read 1 Done Sent Done false
33202 10133 Acked Write 4 Done Sent Done false
33203 10134 Acked Write 6 Done Sent Done false
33204 10135 Acked Read 10 Done Sent Done false
33205 10136 Acked Read 2 Done Sent Done false
33206 10137 Acked Write 4 Done Sent Done false
33207 10138 Acked Read 5 Done Sent Done false
33208 10139 Acked Read 4 Done Sent Done false
33209 10140 Acked Write 8 Done Sent Done false
33210 10141 Acked Write 3 Done Sent Done false
33211 10142 Acked Read 1 Done Sent Done false
33212 10143 Acked Read 8 Done Sent Done false
33213 10144 Acked Write 10 Done Sent Done false
33214 10145 Acked Flush 0 Done Sent Done false
33215 10146 Acked Write 10 Done Sent Done false
33216 10147 Acked Write 7 Done Sent Done false
33217 10148 Acked Read 10 Done Sent Done false
33218 10149 Acked Read 10 Done Sent Done false
33219 10150 Acked Read 10 Done Sent Done false
33220 10151 Acked Read 3 Done Sent Done false
33221 10152 Acked Write 4 Done Sent Done false
33222 10153 Acked Flush 0 Done Sent Done false
33223 10154 Acked Write 3 Done Sent Done false
33224 10155 Acked Write 9 Done Sent Done false
33225 10156 Acked Write 1 Done Sent Done false
33226 10157 Acked Read 5 Done Sent Done false
33227 10158 Acked Write 3 Done Sent Done false
33228 10159 Acked Write 9 Done Sent Done false
33229 10160 Acked Read 6 Done Sent Done false
33230 10161 Acked Flush 0 Done Sent Done false
33231 10162 Acked Write 5 Done Sent Done false
33232 10163 Acked Read 6 Done Sent Done false
33233 10164 Acked Read 3 Done Sent Done false
33234 10165 Acked Read 6 Done Sent Done false
33235 10166 Acked Write 8 Done Sent Done false
33236 10167 Acked Write 3 Done Sent Done false
33237 10168 Acked Read 4 Done Sent Done false
33238 10169 Acked Read 1 Done Sent Done false
33239 10170 Acked Read 10 Done Sent Done false
33240 10171 Acked Read 10 Done Sent Done false
33241 10172 Acked Read 9 Done Sent Done false
33242 10173 Acked Read 6 Done Sent Done false
33243 10174 Acked Read 8 Done Sent Done false
33244 10175 Acked Read 4 Done Sent Done false
33245 10176 Acked Write 3 Done Sent Done false
33246 10177 Acked Write 7 Done Sent Done false
33247 10178 Acked Read 2 Done Sent Done false
33248 10179 Acked Write 4 Done Sent Done false
33249 10180 Acked Read 7 Done Sent Done false
33250 10181 Acked Read 5 Done Sent Done false
33251 10182 Acked Read 10 Done Sent Done false
33252 10183 Acked Write 4 Done Sent Done false
33253 10184 Acked Write 10 Done Sent Done false
33254 10185 Acked Flush 0 Done Sent Done false
33255 10186 Acked Flush 0 Done Sent Done false
33256 10187 Acked Read 10 Done Sent Done false
33257 10188 Acked Read 8 Done Sent Done false
33258 10189 Acked Write 4 Done Sent Done false
33259 10190 Acked Write 7 Done Sent Done false
33260 10191 Acked Write 1 Done Sent Done false
33261 10192 Acked Read 10 Done Sent Done false
33262 10193 Acked Read 3 Done Sent Done false
33263 10194 Acked Read 8 Done Sent Done false
33264 10195 Acked Read 9 Done Sent Done false
33265 10196 Acked Write 4 Done Sent Done false
33266 10197 Acked Read 1 Done Sent Done false
33267 10198 Acked Write 4 Done Sent Done false
33268 10199 Acked Read 3 Done Sent Done false
33269 10200 Acked Read 6 Done Sent Done false
33270 10201 Acked Write 6 Done Sent Done false
33271 10202 Acked Read 6 Done Sent Done false
33272 10203 Acked Read 4 Done Sent Done false
33273 10204 Acked Read 3 Done Sent Done false
33274 10205 Acked Read 8 Done Sent Done false
33275 10206 Acked Read 8 Done Sent Done false
33276 10207 Acked Read 8 Done Sent Done false
33277 10208 Acked Read 3 Done Sent Done false
33278 10209 Acked Write 3 Done Sent Done false
33279 10210 Acked Write 1 Done Sent Done false
33280 10211 Acked Read 4 Done Sent Done false
33281 10212 Acked Read 8 Done Sent Done false
33282 10213 Acked Write 10 Done Sent Done false
33283 10214 Acked Read 4 Done Sent Done false
33284 10215 Acked Write 6 Done Sent Done false
33285 10216 Acked Read 2 Done Sent Done false
33286 10217 Acked Read 5 Done Sent Done false
33287 10218 Acked Read 4 Done Sent Done false
33288 10219 Acked Write 3 Done Sent Done false
33289 10220 Acked Read 9 Done Sent Done false
33290 10221 Acked Write 3 Done Sent Done false
33291 10222 Acked Read 10 Done Sent Done false
33292 10223 Acked Read 2 Done Sent Done false
33293 10224 Acked Write 9 Done Sent Done false
33294 10225 Acked Write 1 Done Sent Done false
33295 10226 Acked Write 7 Done Sent Done false
33296 10227 Acked Read 1 Done Sent Done false
33297 10228 Acked Read 4 Done Sent Done false
33298 10229 Acked Write 7 Done Sent Done false
33299 10230 Acked Read 4 Done Sent Done false
33300 10231 Acked Read 9 Done Sent Done false
33301 10232 Acked Write 4 Done Sent Done false
33302 10233 Acked Read 7 Done Sent Done false
33303 10234 Acked Read 3 Done Sent Done false
33304 10235 Acked Write 9 Done Sent Done false
33305 10236 Acked Write 10 Done Sent Done false
33306 10237 Acked Read 4 Done Sent Done false
33307 10238 Acked Read 10 Done Sent Done false
33308 10239 Acked Write 2 Done Sent Done false
33309 10240 Acked Write 9 Done Sent Done false
33310 10241 Acked Read 9 Done Sent Done false
33311 10242 Acked Write 9 Done Sent Done false
33312 10243 Acked Write 4 Done Sent Done false
33313 10244 Acked Write 10 Done Sent Done false
33314 10245 Acked Read 9 Done Sent Done false
33315 10246 Acked Read 5 Done Sent Done false
33316 10247 Acked Read 7 Done Sent Done false
33317 10248 Acked Write 1 Done Sent Done false
33318 10249 Acked Read 3 Done Sent Done false
33319 10250 Acked Read 4 Done Sent Done false
33320 10251 Acked Read 2 Done Sent Done false
33321 10252 Acked Read 9 Done Sent Done false
33322 10253 Acked Write 4 Done Sent Done false
33323 10254 Acked Write 7 Done Sent Done false
33324 10255 Acked Read 7 Done Sent Done false
33325 10256 Acked Write 4 Done Sent Done false
33326 10257 Acked Flush 0 Done Sent Done false
33327 10258 Acked Read 2 Done Sent Done false
33328 10259 Acked Write 10 Done Sent Done false
33329 10260 Acked Read 10 Done Sent Done false
33330 10261 Acked Read 9 Done Sent Done false
33331 10262 Acked Read 2 Done Sent Done false
33332 10263 Acked Flush 0 Done Sent Done false
33333 10264 Acked Write 6 Done Sent Done false
33334 10265 Acked Read 4 Done Sent Done false
33335 10266 Acked Read 10 Done Sent Done false
33336 10267 Acked Read 3 Done Sent Done false
33337 10268 Acked Write 4 Done Sent Done false
33338 10269 Acked Read 6 Done Sent Done false
33339 10270 Acked Read 4 Done Sent Done false
33340 10271 Acked Read 2 Done Sent Done false
33341 10272 Acked Read 10 Done Sent Done false
33342 10273 Acked Write 10 Done Sent Done false
33343 10274 Acked Read 9 Done Sent Done false
33344 10275 Acked Write 7 Done Sent Done false
33345 10276 Acked Flush 0 Done Sent Done false
33346 10277 Acked Flush 0 Done Sent Done false
33347 10278 Acked Read 5 Done Sent Done false
33348 10279 Acked Read 10 Done Sent Done false
33349 10280 Acked Write 9 Done Sent Done false
33350 10281 Acked Read 3 Done Sent Done false
33351 10282 Acked Read 10 Done Sent Done false
33352 10283 Acked Read 5 Done Sent Done false
33353 10284 Acked Flush 0 Done Sent Done false
33354 10285 Acked Write 6 Done Sent Done false
33355 10286 Acked Read 9 Done Sent Done false
33356 10287 Acked Read 6 Done Sent Done false
33357 10288 Acked Read 10 Done Sent Done false
33358 10289 Acked Read 3 Done Sent Done false
33359 10290 Acked Read 10 Done Sent Done false
33360 10291 Acked Read 1 Done Sent Done false
33361 10292 Acked Flush 0 Done Sent Done false
33362 10293 Acked Read 6 Done Sent Done false
33363 10294 Acked Read 7 Done Sent Done false
33364 10295 Acked Write 2 Done Sent Done false
33365 10296 Acked Flush 0 Done Sent Done false
33366 10297 Acked Write 6 Done Sent Done false
33367 10298 Acked Write 3 Done Sent Done false
33368 10299 Acked Write 1 Done Sent Done false
33369 10300 Acked Read 4 Done Sent Done false
33370 10301 Acked Read 5 Done Sent Done false
33371 10302 Acked Read 6 Done Sent Done false
33372 10303 Acked Read 3 Done Sent Done false
33373 10304 Acked Read 1 Done Sent Done false
33374 10305 Acked Read 6 Done Sent Done false
33375 10306 Acked Flush 0 Done Sent Done false
33376 10307 Acked Write 5 Done Sent Done false
33377 10308 Acked Read 10 Done Sent Done false
33378 10309 Acked Read 6 Done Sent Done false
33379 10310 Acked Read 3 Done Sent Done false
33380 10311 Acked Read 4 Done Sent Done false
33381 10312 Acked Flush 0 Done Sent Done false
33382 10313 Acked Write 9 Done Sent Done false
33383 10314 Acked Write 4 Done Sent Done false
33384 10315 Acked Read 2 Done Sent Done false
33385 10316 Acked Write 9 Done Sent Done false
33386 10317 Acked Read 3 Done Sent Done false
33387 10318 Acked Write 8 Done Sent Done false
33388 10319 Acked Read 5 Done Sent Done false
33389 10320 Acked Write 8 Done Sent Done false
33390 10321 Acked Read 1 Done Sent Done false
33391 10322 Acked Read 4 Done Sent Done false
33392 10323 Acked Read 9 Done Sent Done false
33393 10324 Acked Flush 0 Done Sent Done false
33394 10325 Acked Write 10 Done Sent Done false
33395 10326 Acked Read 3 Done Sent Done false
33396 10327 Acked Write 7 Done Sent Done false
33397 10328 Acked Read 7 Done Sent Done false
33398 10329 Acked Read 2 Done Sent Done false
33399 10330 Acked Read 8 Done Sent Done false
33400 10331 Acked Write 4 Done Sent Done false
33401 10332 Acked Read 3 Done Sent Done false
33402 10333 Acked Write 1 Done Sent Done false
33403 10334 Acked Write 5 Done Sent Done false
33404 10335 Acked Read 2 Done Sent Done false
33405 10336 Acked Write 5 Done Sent Done false
33406 10337 Acked Write 8 Done Sent Done false
33407 10338 Acked Read 7 Done Sent Done false
33408 10339 Acked Write 3 Done Sent Done false
33409 10340 Acked Read 1 Done Sent Done false
33410 10341 Acked Read 2 Done Sent Done false
33411 10342 Acked Write 8 Done Sent Done false
33412 10343 Acked Write 8 Done Sent Done false
33413 10344 Acked Flush 0 Done Sent Done false
33414 10345 Acked Flush 0 Done Sent Done false
33415 10346 Acked Write 7 Done Sent Done false
33416 10347 Acked Flush 0 Done Sent Done false
33417 10348 Acked Write 4 Done Sent Done false
33418 10349 Acked Write 8 Done Sent Done false
33419 10350 Acked Write 4 Done Sent Done false
33420 10351 Acked Write 1 Done Sent Done false
33421 10352 Acked Flush 0 Done Sent Done false
33422 10353 Acked Flush 0 Done Sent Done false
33423 10354 Acked Flush 0 Done Sent Done false
33424 10355 Acked Read 3 Done Sent Done false
33425 10356 Acked Write 5 Done Sent Done false
33426 10357 Acked Read 8 Done Sent Done false
33427 10358 Acked Read 3 Done Sent Done false
33428 10359 Acked Flush 0 Done Sent Done false
33429 STATES DS:0 DS:1 DS:2 TOTAL
33430 Sent 0 302 0 302
33431 Done 302 0 302 604
33432 Skipped 0 0 0 0
33433 Error 0 0 0 0
33434 Last Flush: 10359 10057 10359
33435 Downstairs last five completed: 10057 10056 10055 10054 10053
33436 Upstairs last five completed: 10359 10358 10357 10356 10355
33437 CLIENT: Up:0 ds:302 act:2
334382025-01-13T15:58:48.564ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
334392025-01-13T15:58:48.564ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64518) = io task client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
334402025-01-13T15:58:48.565ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
334412025-01-13T15:58:48.565ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
334422025-01-13T15:58:48.565ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
334432025-01-13T15:58:48.565ZINFOcrucible: send last flush ID to this DS: 10057 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
334442025-01-13T15:58:48.565ZINFOcrucible: Replied this last flush ID: 10057 = downstairs client = 1 session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
334452025-01-13T15:58:48.565ZINFOcrucible: [1] client re-new 302 jobs since flush 10057 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
334462025-01-13T15:58:48.566ZINFOcrucible: [1] Replayed 302 jobs since flush: 10057 = downstairs session_id = cd58429c-c491-4a1e-aa01-02db1d6a3d79
33447 ----------------------------------------------------------------
33448 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
33449 Downstairs last five completed: 10359 10358 10357 10356 10355
33450 Upstairs last five completed: 10359 10358 10357 10356 10355
33451 CLIENT: Up:0 ds:0 act:3
33452 Replay: All jobs finished
33453 Test replay has completed
33454 Wrote out file "/tmp/test_replay_verify.log"
33455 CLIENT: Tests done. All submitted work has been ACK'd
33456 ----------------------------------------------------------------
33457 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
33458 Downstairs last five completed: 10359 10358 10357 10356 10355
33459 Upstairs last five completed: 10359 10358 10357 10356 10355
33460 CLIENT: Up:0 ds:0 act:3
33461 CLIENT: All jobs finished, all DS active.
33462 Replays:30 time: 7:00
33463 Do final verify
33464 Jan 13 15:58:50.897 INFO accepted connection, remote_addr: 127.0.0.1:47902, local_addr: 127.0.0.1:9998
33465 Jan 13 15:58:50.898 INFO request completed, latency_us: 216, response_code: 200, uri: /regioninfo, method: GET, req_id: d84a7824-7583-4e7e-97ad-ba2e839be8d8, remote_addr: 127.0.0.1:47902, local_addr: 127.0.0.1:9998
334662025-01-13T15:58:50.898ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
33467 Jan 13 15:58:50.899 INFO request completed, latency_us: 91, response_code: 200, uri: /regioncount, method: GET, req_id: a2d1c503-c8f8-4112-a3c1-9ba42c70fae8, remote_addr: 127.0.0.1:47902, local_addr: 127.0.0.1:9998
334682025-01-13T15:58:50.899ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
33469 Jan 13 15:58:50.899 INFO request completed, latency_us: 112, response_code: 200, uri: /port/cid/0, method: GET, req_id: 50c79236-df39-47ec-af89-2d406c456dd9, remote_addr: 127.0.0.1:47902, local_addr: 127.0.0.1:9998
33470 Jan 13 15:58:50.900 INFO request completed, latency_us: 96, response_code: 200, uri: /port/cid/1, method: GET, req_id: 6be3a0d1-653f-43f1-975a-f260aacfa3f4, remote_addr: 127.0.0.1:47902, local_addr: 127.0.0.1:9998
33471 Jan 13 15:58:50.900 INFO request completed, latency_us: 92, response_code: 200, uri: /port/cid/2, method: GET, req_id: 061ca80a-2e45-4939-9d55-84a35e9cff53, remote_addr: 127.0.0.1:47902, local_addr: 127.0.0.1:9998
334722025-01-13T15:58:50.901ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
334732025-01-13T15:58:50.901ZINFOcrucible: Upstairs starts
334742025-01-13T15:58:50.901ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "12c38e1c2ca92eb8e00f82331ef94fdbf7d3f8af", git_commit_timestamp: "2025-01-13T15:35:09.000000000Z", git_branch: "mkeeter/lean-mean-state-machine", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
334752025-01-13T15:58:50.901ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
334762025-01-13T15:58:50.901ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
334772025-01-13T15:58:50.901ZINFOcrucible: Crucible b147e4af-19c4-474e-9956-96c048f3fdaa has session id: 9a6992cb-b1f3-4a47-a4ad-150ebad7be24 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334782025-01-13T15:58:50.901ZINFOcrucible: Upstairs opts: Upstairs UUID: b147e4af-19c4-474e-9956-96c048f3fdaa, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334792025-01-13T15:58:50.901ZINFOcrucible: Crucible stats registered with UUID: b147e4af-19c4-474e-9956-96c048f3fdaa session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334802025-01-13T15:58:50.935ZINFOcrucible: Downstairs in volume = 3 task = crutest
334812025-01-13T15:58:50.935ZINFOcrucible: The guest has requested activation with gen:3
334822025-01-13T15:58:50.935ZINFOcrucible: b147e4af-19c4-474e-9956-96c048f3fdaa active request set session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334832025-01-13T15:58:50.935ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334842025-01-13T15:58:50.935ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334852025-01-13T15:58:50.935ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334862025-01-13T15:58:50.935ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334872025-01-13T15:58:50.935ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334882025-01-13T15:58:50.935ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334892025-01-13T15:58:50.935ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334902025-01-13T15:58:50.935ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334912025-01-13T15:58:50.935ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334922025-01-13T15:58:50.936ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57435) = io task client = 0 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334932025-01-13T15:58:50.936ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:37078) = io task client = 1 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334942025-01-13T15:58:50.936ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:37447) = io task client = 2 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334952025-01-13T15:58:50.936ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334962025-01-13T15:58:50.936ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334972025-01-13T15:58:50.936ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334982025-01-13T15:58:50.937ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
334992025-01-13T15:58:50.937ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335002025-01-13T15:58:50.937ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335012025-01-13T15:58:50.938ZINFOcrucible: Waiting for more clients to be ready session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335022025-01-13T15:58:50.938ZINFOcrucible: Waiting for more clients to be ready session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335032025-01-13T15:58:50.938ZINFOcrucible: [0]R flush_numbers[0..12]: [976, 964, 966, 980, 973, 968, 964, 962, 964, 964, 977, 964] = downstairs session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335042025-01-13T15:58:50.938ZINFOcrucible: [0]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335052025-01-13T15:58:50.938ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335062025-01-13T15:58:50.938ZINFOcrucible: [1]R flush_numbers[0..12]: [976, 964, 966, 980, 973, 968, 964, 962, 964, 964, 977, 964] = downstairs session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335072025-01-13T15:58:50.938ZINFOcrucible: [1]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335082025-01-13T15:58:50.938ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335092025-01-13T15:58:50.938ZINFOcrucible: [2]R flush_numbers[0..12]: [976, 964, 966, 980, 973, 968, 964, 962, 964, 964, 977, 964] = downstairs session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335102025-01-13T15:58:50.938ZINFOcrucible: [2]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335112025-01-13T15:58:50.938ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335122025-01-13T15:58:50.938ZINFOcrucible: Max found gen is 3 = downstairs session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335132025-01-13T15:58:50.938ZINFOcrucible: Generation requested: 3 >= found:3 = downstairs session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
33514 {"msg":"Next flush: 981","v":0,"name":"crucible","Wait for a query_work_queue command to finish before sending IO
33515 level":30,"time":"2025-01-13T15:58:50.938391993Z","hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1238,"":"downstairs","":"downstairs","session_id":"9a6992cb-b1f3-4a47-a4ad-150ebad7be24"}
335162025-01-13T15:58:50.938ZINFOcrucible: All extents match = downstairs session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335172025-01-13T15:58:50.938ZINFOcrucible: No downstairs reconciliation required session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335182025-01-13T15:58:50.938ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335192025-01-13T15:58:50.938ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335202025-01-13T15:58:50.938ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335212025-01-13T15:58:50.938ZINFOcrucible: All required reconciliation work is completed session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335222025-01-13T15:58:50.938ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
33523 {"msg":"b147e4af-19c4-474e-9956-96c048f3fdaa is now active with session: 9a6992cb-b1f3-4a47-a4ad-150ebad7be24","v":0,"name":"crucible","level":30,"time":"2025-01-13T15:58:50.938497786Z",Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
33524 "hostname":"w-01JHG5QMHED7MX9XRA6ZM80Q6G","pid":1238,"session_id":"9a6992cb-b1f3-4a47-a4ad-150ebad7be24"}
335252025-01-13T15:58:50.938ZINFOcrucible: Set Active after no reconciliation session_id = 9a6992cb-b1f3-4a47-a4ad-150ebad7be24
335262025-01-13T15:58:50.938ZINFOcrucible: The guest has finished waiting for activation with:3
33527 Loading write count information from file "/tmp/test_replay_verify.log"
33528 Read and Verify all blocks (0..5000 range:false)
33529 Wrote out file "/tmp/test_replay_verify.log"
33530 Verify test completed
33531 Wrote out file "/tmp/test_replay_verify.log"
33532 CLIENT: Tests done. All submitted work has been ACK'd
33533 ----------------------------------------------------------------
33534 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:20
33535 States: Active Active Active
33536 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33537 1000 Acked Read 256 Done Done Done false
33538 1001 Acked Read 256 Done Done Done false
33539 1002 Acked Read 256 Done Done Done false
33540 1003 Acked Read 256 Done Done Done false
33541 1004 Acked Read 256 Done Done Done false
33542 1005 Acked Read 256 Done Done Done false
33543 1006 Acked Read 256 Done Done Done false
33544 1007 Acked Read 256 Done Done Done false
33545 1008 Acked Read 256 Done Done Done false
33546 1009 Acked Read 256 Done Done Done false
33547 1010 Acked Read 256 Done Done Done false
33548 1011 Acked Read 256 Done Done Done false
33549 1012 Acked Read 256 Done Done Done false
33550 1013 Acked Read 256 Done Done Done false
33551 1014 Acked Read 256 Done Done Done false
33552 1015 Acked Read 256 Done Done Done false
33553 1016 Acked Read 256 Done Done Done false
33554 1017 Acked Read 256 Done Done Done false
33555 1018 Acked Read 136 Done Done Done false
33556 1019 Acked Read 256 Done Done Done false
33557 STATES DS:0 DS:1 DS:2 TOTAL
33558 Sent 0 0 0 0
33559 Done 20 20 20 60
33560 Skipped 0 0 0 0
33561 Error 0 0 0 0
33562 Last Flush: 0 0 0
33563 Downstairs last five completed:
33564 Upstairs last five completed: 1019 1018 1017 1016 1015
33565 CLIENT: Up:0 ds:20 act:3
33566 ----------------------------------------------------------------
33567 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:0
33568 Downstairs last five completed: 1020 1019 1018 1017 1016
33569 Upstairs last five completed: 1020 1019 1018 1017 1016
33570 CLIENT: Up:0 ds:0 act:3
33571 CLIENT: All crucible jobs finished, exiting program
33572 Jan 13 15:58:55.415 INFO accepted connection, remote_addr: 127.0.0.1:40532, local_addr: 127.0.0.1:9998
33573 Main task has work to do, go find it
33574 got dsc Shutdown
33575 Shutdown
33576 Jan 13 15:58:55.415 INFO request completed, latency_us: 150, response_code: 204, uri: /shutdown, method: POST, req_idShut it down
33577 :[8810] Disable keep_running
33578 5a6d47ca-478f-4207-bd52-a46e511dab59, remote_addr: [8810] Got stop action so:false kr:false
33579 127.0.0.1:40532, local_addr: 127.0.0.1:9998
33580 [8830] Disable keep_running
33581 [8830] Got stop action so:false kr:false
33582 [8820] Disable keep_running
33583 [8820] Got stop action so:false kr:false
33584 [8830] Exited with: ExitStatus(unix_wait_status(9))
33585 [8830][2] reports Exit
33586 [8820] Exited with: ExitStatus(unix_wait_status(9))
33587 [8810] Exited with: ExitStatus(unix_wait_status(9))
33588 [8820][1] reports Exit
33589 [8810][0] reports Exit
33590 January 13, 2025 at 03:59:00 PM UTC Test ends with 0