Max level shown:
1 starting January 17, 2025 at 09:10:58 PM UTC
2 Creating 3 downstairs regions
3 Creating region directory at: /var/tmp/dsc/region
4 Creating dsc directory at: /tmp/dsc
5 Downstairs region 0 created at /var/tmp/dsc/region/8810 in 0.25613973
6 Downstairs region 1 created at /var/tmp/dsc/region/8820 in 0.02158438
7 Downstairs region 2 created at /var/tmp/dsc/region/8830 in 0.021274593
8 Created 3 regions
9 Starting 3 downstairs
10 Using existing output directory "/tmp/dsc"
11 Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 })
12 start ds: 8810
13 start ds: 8820
14 start ds: 8830
15 start access at:127.0.0.1:9998
16 Starting downstairs at port 8810
17 Starting downstairs at port 8830
18 Starting downstairs at port 8820
19 Make output file at "/tmp/dsc/downstairs-8810.txt"
20 Make output file at "/tmp/dsc/downstairs-8820.txt"
21 Make output file at "/tmp/dsc/downstairs-8830.txt"
22 [8810][0] initial start wait reports Starting
23 [8830][2] initial start wait reports Starting
24 [8820][1] initial start wait reports Starting
25 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1190)
26 [8810][0] initial start wait reports Running
27 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1191)
28 [8820][1] initial start wait reports Running
29 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1192)
30 [8830][2] initial start wait reports Running
31 All downstairs are running
32 Jan 17 21:10:58.585 INFO listening, local_addr: 127.0.0.1:9998
33 Control access at:127.0.0.1:9998
34 Running initial fill
35 Jan 17 21:11:03.649 INFO accepted connection, remote_addr: 127.0.0.1:42542, local_addr: 127.0.0.1:9998
36 Jan 17 21:11:03.651 INFO request completed, latency_us: 384, response_code: 200, uri: /regioninfo, method: GET, req_id: 1a45ee55-d8ce-493d-8d7a-ab81ecaa6215, remote_addr: 127.0.0.1:42542, local_addr: 127.0.0.1:9998
37 Jan 17 21:11:03.653 INFO request completed, latency_us: 112, response_code: 200, uri: /regioncount, method: GET, req_id: d3fb4ca6-52a5-4c43-ac18-939914b4af8e, remote_addr: 127.0.0.1:42542, local_addr: 127.0.0.1:9998
382025-01-17T21:11:03.652ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
392025-01-17T21:11:03.653ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
40 Jan 17 21:11:03.653 INFO request completed, latency_us: 117, response_code: 200, uri: /port/cid/0, method: GET, req_id: 2a8c05d2-4111-4b57-ba2d-18a39c651bc4, remote_addr: 127.0.0.1:42542, local_addr: 127.0.0.1:9998
41 Jan 17 21:11:03.654 INFO request completed, latency_us: 85, response_code: 200, uri: /port/cid/1, method: GET, req_id: 25cac29b-fc30-427a-ba99-51b5e9cca841, remote_addr: 127.0.0.1:42542, local_addr: 127.0.0.1:9998
42 Jan 17 21:11:03.654 INFO request completed, latency_us: 84, response_code: 200, uri: /port/cid/2, method: GET, req_id: e0da01d5-b84d-4887-bcdc-2588e199803f, remote_addr: 127.0.0.1:42542, local_addr: 127.0.0.1:9998
432025-01-17T21:11:03.655ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
442025-01-17T21:11:03.655ZINFOcrucible: Upstairs starts
452025-01-17T21:11:03.655ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "abd5430f1508af0bc008143ea2566277ef714ec2", git_commit_timestamp: "2025-01-17T20:55:03.000000000Z", git_branch: "alan/tokio-to-140", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
462025-01-17T21:11:03.655ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
472025-01-17T21:11:03.655ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
482025-01-17T21:11:03.656ZINFOcrucible: Crucible 932dde3b-0ed8-49c2-8c69-f5eda33630d5 has session id: 34064a10-3f72-4b8f-a943-3111c8564a55 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
492025-01-17T21:11:03.656ZINFOcrucible: Upstairs opts: Upstairs UUID: 932dde3b-0ed8-49c2-8c69-f5eda33630d5, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
502025-01-17T21:11:03.656ZINFOcrucible: Crucible stats registered with UUID: 932dde3b-0ed8-49c2-8c69-f5eda33630d5 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
512025-01-17T21:11:03.656ZWARNcrucible: could not find Downstairs address for Nexus session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
522025-01-17T21:11:03.658ZINFOcrucible: Downstairs in volume = 3 task = crutest
532025-01-17T21:11:03.658ZINFOcrucible: The guest has requested activation with gen:1
542025-01-17T21:11:03.658ZINFOcrucible: 932dde3b-0ed8-49c2-8c69-f5eda33630d5 active request set session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
552025-01-17T21:11:03.658ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
562025-01-17T21:11:03.658ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
572025-01-17T21:11:03.658ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
582025-01-17T21:11:03.658ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
592025-01-17T21:11:03.658ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
602025-01-17T21:11:03.658ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
612025-01-17T21:11:03.658ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
622025-01-17T21:11:03.658ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
632025-01-17T21:11:03.658ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
642025-01-17T21:11:03.658ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54826) = io task client = 0 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
652025-01-17T21:11:03.658ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42605) = io task client = 1 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
662025-01-17T21:11:03.658ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:45816) = io task client = 2 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
672025-01-17T21:11:03.660ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
682025-01-17T21:11:03.660ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
692025-01-17T21:11:03.660ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
702025-01-17T21:11:03.661ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
712025-01-17T21:11:03.661ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
722025-01-17T21:11:03.661ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
732025-01-17T21:11:03.662ZINFOcrucible: Waiting for more clients to be ready session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
742025-01-17T21:11:03.662ZINFOcrucible: Waiting for more clients to be ready session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
752025-01-17T21:11:03.662ZINFOcrucible: [0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
762025-01-17T21:11:03.662ZINFOcrucible: [0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
772025-01-17T21:11:03.662ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
782025-01-17T21:11:03.662ZINFOcrucible: [1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
792025-01-17T21:11:03.662ZINFOcrucible: [1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
802025-01-17T21:11:03.662ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
812025-01-17T21:11:03.662ZINFOcrucible: [2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
822025-01-17T21:11:03.662ZINFOcrucible: [2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
832025-01-17T21:11:03.662ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
842025-01-17T21:11:03.662ZINFOcrucible: Max found gen is 1 = downstairs session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
852025-01-17T21:11:03.662ZINFOcrucible: Generation requested: 1 >= found:1 = downstairs session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
862025-01-17T21:11:03.662ZINFOcrucible: Next flush: 1 = downstairs session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
872025-01-17T21:11:03.662ZINFOcrucible: All extents match = downstairs session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
882025-01-17T21:11:03.662ZINFOcrucible: No downstairs reconciliation required session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
892025-01-17T21:11:03.662ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
902025-01-17T21:11:03.662ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
912025-01-17T21:11:03.662ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
922025-01-17T21:11:03.663ZINFOcrucible: All required reconciliation work is completed session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
932025-01-17T21:11:03.663ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
942025-01-17T21:11:03.663ZINFOcrucible: 932dde3b-0ed8-49c2-8c69-f5eda33630d5 is now active with session: 34064a10-3f72-4b8f-a943-3111c8564a55 session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
952025-01-17T21:11:03.663ZINFOcrucible: Set Active after no reconciliation session_id = 34064a10-3f72-4b8f-a943-3111c8564a55
962025-01-17T21:11:03.663ZINFOcrucible: The guest has finished waiting for activation with:1
97 Activate successful
98 Wait for a query_work_queue command to finish before sending IO
99 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
100 Fill test
101 Read and Verify all blocks (0..5000 range:false)
102 Wrote out file "/tmp/test_replay_verify.log"
103 CLIENT: Tests done. All submitted work has been ACK'd
104 ----------------------------------------------------------------
105 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:20
106 States: Active Active Active
107 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
108 1021 Acked Read 256 Done Done Done false
109 1022 Acked Read 256 Done Done Done false
110 1023 Acked Read 256 Done Done Done false
111 1024 Acked Read 256 Done Done Done false
112 1025 Acked Read 256 Done Done Done false
113 1026 Acked Read 256 Done Done Done false
114 1027 Acked Read 256 Done Done Done false
115 1028 Acked Read 256 Done Done Done false
116 1029 Acked Read 256 Done Done Done false
117 1030 Acked Read 256 Done Done Done false
118 1031 Acked Read 256 Done Done Done false
119 1032 Acked Read 256 Done Done Done false
120 1033 Acked Read 256 Done Done Done false
121 1034 Acked Read 256 Done Done Done false
122 1035 Acked Read 256 Done Done Done false
123 1036 Acked Read 256 Done Done Done false
124 1037 Acked Read 256 Done Done Done false
125 1038 Acked Read 256 Done Done Done false
126 1039 Acked Read 256 Done Done Done false
127 1040 Acked Read 136 Done Done Done false
128 STATES DS:0 DS:1 DS:2 TOTAL
129 Sent 0 0 0 0
130 Done 20 20 20 60
131 Skipped 0 0 0 0
132 Error 0 0 0 0
133 Last Flush: 1020 1020 1020
134 Downstairs last five completed: 1020 1019 1018 1017 1016
135 Upstairs last five completed: 1040 1039 1038 1037 1036
136 CLIENT: Up:0 ds:20 act:3
137 ----------------------------------------------------------------
138 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0
139 Downstairs last five completed: 1041 1040 1039 1038 1037
140 Upstairs last five completed: 1041 1040 1039 1038 1037
141 CLIENT: Up:0 ds:0 act:3
142 CLIENT: All crucible jobs finished, exiting program
143 Replay loop starts now January 17, 2025 at 09:11:08 PM UTC
144 Jan 17 21:11:08.304 INFO accepted connection, remote_addr: 127.0.0.1:37902, local_addr: 127.0.0.1:9998
145 Jan 17 21:11:08.305 INFO request completed, latency_us: 162, response_code: 200, uri: /regioninfo, method: GET, req_id: 03fa3c7a-4924-43c7-91db-ab0856248872, remote_addr: 127.0.0.1:37902, local_addr: 127.0.0.1:9998
1462025-01-17T21:11:08.306ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
147 Jan 17 21:11:08.306 INFO request completed, latency_us: 84, response_code: 200, uri: /regioncount, method: GET, req_id: bbaaf733-3074-4ce2-b768-e4d10f052371, remote_addr: 127.0.0.1:37902, local_addr: 127.0.0.1:9998
1482025-01-17T21:11:08.306ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
149 Jan 17 21:11:08.306 INFO request completed, latency_us: 123, response_code: 200, uri: /port/cid/0, method: GET, req_id: f90b6ab8-0713-4745-85b0-d6932a71b409, remote_addr: 127.0.0.1:37902, local_addr: 127.0.0.1:9998
150 Jan 17 21:11:08.307 INFO request completed, latency_us: 93, response_code: 200, uri: /port/cid/1, method: GET, req_id: 188eb9e6-6dc6-43f2-aac1-872a43de21d3, remote_addr: 127.0.0.1:37902, local_addr: 127.0.0.1:9998
151 Jan 17 21:11:08.308 INFO request completed, latency_us: 87, response_code: 200, uri: /port/cid/2, method: GET, req_id: e4027bd5-33f3-4437-a9f1-a344b894f2f8, remote_addr: 127.0.0.1:37902, local_addr: 127.0.0.1:9998
1522025-01-17T21:11:08.308ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
1532025-01-17T21:11:08.308ZINFOcrucible: Upstairs starts
1542025-01-17T21:11:08.308ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "abd5430f1508af0bc008143ea2566277ef714ec2", git_commit_timestamp: "2025-01-17T20:55:03.000000000Z", git_branch: "alan/tokio-to-140", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
1552025-01-17T21:11:08.308ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
1562025-01-17T21:11:08.308ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
1572025-01-17T21:11:08.308ZINFOcrucible: Crucible e4675be1-79d9-490c-97e8-a27d919503a0 has session id: 64f44984-0f34-4cfb-a401-a0494f2ea105 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1582025-01-17T21:11:08.309ZINFOcrucible: Upstairs opts: Upstairs UUID: e4675be1-79d9-490c-97e8-a27d919503a0, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1592025-01-17T21:11:08.309ZINFOcrucible: Crucible stats registered with UUID: e4675be1-79d9-490c-97e8-a27d919503a0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1602025-01-17T21:11:08.309ZWARNcrucible: could not find Downstairs address for Nexus session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1612025-01-17T21:11:08.309ZINFOcrucible: Downstairs in volume = 3 task = crutest
1622025-01-17T21:11:08.309ZINFOcrucible: The guest has requested activation with gen:2
1632025-01-17T21:11:08.310ZINFOcrucible: e4675be1-79d9-490c-97e8-a27d919503a0 active request set session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1642025-01-17T21:11:08.310ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1652025-01-17T21:11:08.310ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1662025-01-17T21:11:08.310ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1672025-01-17T21:11:08.310ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1682025-01-17T21:11:08.310ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1692025-01-17T21:11:08.310ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1702025-01-17T21:11:08.310ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1712025-01-17T21:11:08.310ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1722025-01-17T21:11:08.310ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1732025-01-17T21:11:08.310ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51550) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1742025-01-17T21:11:08.310ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58377) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1752025-01-17T21:11:08.310ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58579) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1762025-01-17T21:11:08.311ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1772025-01-17T21:11:08.311ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1782025-01-17T21:11:08.311ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1792025-01-17T21:11:08.311ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1802025-01-17T21:11:08.311ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1812025-01-17T21:11:08.311ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1822025-01-17T21:11:08.312ZINFOcrucible: Waiting for more clients to be ready session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1832025-01-17T21:11:08.312ZINFOcrucible: Waiting for more clients to be ready session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1842025-01-17T21:11:08.312ZINFOcrucible: [0]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1852025-01-17T21:11:08.312ZINFOcrucible: [0]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1862025-01-17T21:11:08.312ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1872025-01-17T21:11:08.312ZINFOcrucible: [1]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1882025-01-17T21:11:08.312ZINFOcrucible: [1]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1892025-01-17T21:11:08.312ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1902025-01-17T21:11:08.312ZINFOcrucible: [2]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1912025-01-17T21:11:08.312ZINFOcrucible: [2]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1922025-01-17T21:11:08.312ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1932025-01-17T21:11:08.312ZINFOcrucible: Max found gen is 2 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1942025-01-17T21:11:08.312ZINFOcrucible: Generation requested: 2 >= found:2 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
195 {"msg":"Next flush: 2","v":0,"name":"crucible","level":30,"time":"2025-01-17T21:11:08.312902155Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"downstairs","Activate successful
196 Wait for a query_work_queue command to finish before sending IO
197 ":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
1982025-01-17T21:11:08.312ZINFOcrucible: All extents match = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1992025-01-17T21:11:08.312ZINFOcrucible: No downstairs reconciliation required session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
2002025-01-17T21:11:08.313ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
2012025-01-17T21:11:08.313ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
2022025-01-17T21:11:08.313ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
203 {"msg":"All required reconciliation work is completed","v":0,"name":"crucible","Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
204 level":30,"time":"2025-01-17T21:11:08.313114885Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
2052025-01-17T21:11:08.313ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
2062025-01-17T21:11:08.313ZINFOcrucible: e4675be1-79d9-490c-97e8-a27d919503a0 is now active with session: 64f44984-0f34-4cfb-a401-a0494f2ea105 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
2072025-01-17T21:11:08.313ZINFOcrucible: Set Active after no reconciliation session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
2082025-01-17T21:11:08.313ZINFOcrucible: The guest has finished waiting for activation with:2
209 Loading write count information from file "/tmp/test_replay_verify.log"
210 Jan 17 21:11:08.346 INFO accepted connection, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
211 Main task has work to do, go find it
212 got dsc Stop(1)
213 stop 1
214 Jan 17 21:11:08.346 INFO request completed, latency_us: 167, response_code: 204, uri: /stop/cid/1, [8820] Got stop action so:false kr:false
215 method: POST, req_id: 3948fc73-89ed-4da0-bf44-271446f4daa8, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
2162025-01-17T21:11:08.347ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
2172025-01-17T21:11:08.347ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
2182025-01-17T21:11:08.347ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
2192025-01-17T21:11:08.347ZINFOcrucible: client task is exiting = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
220 Jan 17 21:11:08.347 INFO request completed, latency_us: 303, response_code: 200, uri: /state/cid/1, method: GET, req_id: a8d05514-afc7-4986-ad00-6427c1f1d9fd, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
221 [8820] Exited with: ExitStatus(unix_wait_status(9))
222 [8820][1] reports Exit
223 Jan 17 21:11:12.349 INFO request completed, latency_us: 212, response_code: 200, uri: /state/cid/1, method: GET, req_id: a84fba48-49c8-4e7b-bdbb-45a2a8d42094, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
224 001/300 Write block 1321 len 40960 data: 2 2 2 2 2 2 2 2 2 2
225 002/300 Write block 2305 len 28672 data: 2 2 2 2 2 2 2
226 003/300 Write block 4172 len 28672 data: 2 2 2 2 2 2 2
227 004/300 Read block 4990 len 40960
228 005/300 Read block 2260 len 8192
229 006/300 Write block 4963 len 16384 data: 2 2 2 2
230 007/300 Write block 3664 len 28672 data: 2 2 2 2 2 2 2
231 008/300 Write block 2802 len 40960 data: 2 2 2 2 2 2 2 2 2 2
232 009/300 Read block 1800 len 32768
233 010/300 Write block 2275 len 20480 data: 2 2 2 2 2
234 011/300 Read block 3078 len 8192
235 012/300 Write block 619 len 4096 data: 2
236 013/300 Write block 1981 len 4096 data: 2
237 014/300 Write block 2651 len 36864 data: 2 2 2 2 2 2 2 2 2
238 015/300 Flush
239 016/300 Read block 1717 len 16384
240 017/300 Read block 150 len 20480
241 018/300 Write block 2231 len 24576 data: 2 2 2 2 2 2
242 019/300 Flush
243 020/300 Flush
244 021/300 Write block 2767 len 28672 data: 2 2 2 2 2 2 2
245 022/300 Write block 3839 len 20480 data: 2 2 2 2 2
246 023/300 Write block 421 len 32768 data: 2 2 2 2 2 2 2 2
247 024/300 Read block 1459 len 36864
248 025/300 Read block 3126 len 36864
249 026/300 Write block 4322 len 24576 data: 2 2 2 2 2 2
250 027/300 Read block 3812 len 40960
251 028/300 Write block 666 len 28672 data: 2 2 2 2 2 2 2
252 029/300 Read block 250 len 16384
253 030/300 Flush
254 031/300 Read block 3792 len 12288
255 032/300 Read block 2267 len 16384
256 033/300 Read block 1805 len 20480
257 034/300 Read block 1017 len 28672
258 035/300 Write block 1949 len 36864 data: 2 2 2 2 2 2 2 2 2
259 036/300 Read block 4473 len 4096
260 037/300 Read block 1367 len 8192
261 038/300 Write block 1584 len 36864 data: 2 2 2 2 2 2 2 2 2
262 039/300 Write block 2645 len 8192 data: 2 2
263 040/300 Flush
264 041/300 Write block 2663 len 12288 data: 2 2 2
265 042/300 Read block 2437 len 20480
266 043/300 Read block 3855 len 8192
267 044/300 Read block 3526 len 40960
268 045/300 Read block 671 len 20480
269 046/300 Write block 1440 len 16384 data: 2 2 2 2
270 047/300 Read block 4150 len 16384
271 048/300 Flush
272 049/300 Write block 250 len 24576 data: 2 2 2 2 2 2
273 050/300 Write block 4297 len 28672 data: 2 2 2 2 2 2 2
274 051/300 Write block 3530 len 4096 data: 2
275 052/300 Read block 365 len 4096
276 053/300 Read block 891 len 32768
277 054/300 Read block 2949 len 24576
278 055/300 Read block 2591 len 32768
279 056/300 Write block 4762 len 40960 data: 2 2 2 2 2 2 2 2 2 2
280 057/300 Flush
281 058/300 Read block 3575 len 36864
282 059/300 Write block 2031 len 12288 data: 2 2 2
283 060/300 Write block 3933 len 16384 data: 2 2 2 2
284 061/300 Write block 3889 len 28672 data: 2 2 2 2 2 2 2
285 062/300 Write block 3586 len 4096 data: 2
286 063/300 Flush
287 064/300 Read block 2268 len 24576
288 065/300 Flush
289 066/300 Flush
290 067/300 Write block 969 len 16384 data: 2 2 2 2
291 068/300 Flush
292 069/300 Write block 334 len 36864 data: 2 2 2 2 2 2 2 2 2
293 070/300 Flush
294 071/300 Write block 342 len 28672 data: 3 2 2 2 2 2 2
295 072/300 Write block 1051 len 40960 data: 2 2 2 2 2 2 2 2 2 2
296 073/300 Write block 2739 len 36864 data: 2 2 2 2 2 2 2 2 2
297 074/300 Read block 2871 len 24576
298 075/300 Read block 4626 len 20480
299 076/300 Write block 3496 len 20480 data: 2 2 2 2 2
300 077/300 Read block 3593 len 36864
301 078/300 Write block 133 len 8192 data: 2 2
302 079/300 Read block 3125 len 12288
303 080/300 Write block 841 len 12288 data: 2 2 2
304 081/300 Write block 1184 len 36864 data: 2 2 2 2 2 2 2 2 2
305 082/300 Write block 173 len 40960 data: 2 2 2 2 2 2 2 2 2 2
306 083/300 Flush
307 084/300 Write block 1599 len 36864 data: 2 2 2 2 2 2 2 2 2
308 085/300 Write block 3984 len 40960 data: 2 2 2 2 2 2 2 2 2 2
309 086/300 Read block 173 len 40960
310 087/300 Write block 1132 len 28672 data: 2 2 2 2 2 2 2
311 088/300 Read block 642 len 4096
312 089/300 Read block 3777 len 24576
313 090/300 Read block 4746 len 20480
314 091/300 Read block 234 len 20480
315 092/300 Read block 3596 len 32768
316 093/300 Read block 1337 len 4096
317 094/300 Write block 4779 len 32768 data: 2 2 2 2 2 2 2 2
318 095/300 Write block 2681 len 24576 data: 2 2 2 2 2 2
319 096/300 Write block 4710 len 12288 data: 2 2 2
320 097/300 Flush
321 098/300 Read block 2637 len 36864
322 099/300 Read block 2217 len 4096
323 100/300 Write block 3392 len 16384 data: 2 2 2 2
324 101/300 Write block 4317 len 32768 data: 2 2 2 2 2 3 3 3
325 102/300 Read block 4075 len 12288
326 103/300 Read block 4036 len 8192
327 104/300 Read block 4806 len 40960
328 105/300 Write block 1092 len 32768 data: 2 2 2 2 2 2 2 2
329 106/300 Write block 760 len 28672 data: 2 2 2 2 2 2 2
330 107/300 Read block 4639 len 32768
331 108/300 Write block 2161 len 24576 data: 2 2 2 2 2 2
332 109/300 Read block 1844 len 8192
333 110/300 Read block 2994 len 20480
334 111/300 Write block 898 len 40960 data: 2 2 2 2 2 2 2 2 2 2
335 112/300 Flush
336 113/300 Write block 272 len 8192 data: 2 2
337 114/300 Write block 1136 len 4096 data: 3
338 115/300 Read block 2137 len 32768
339 116/300 Read block 1131 len 32768
340 117/300 Read block 3474 len 28672
341 118/300 Read block 4743 len 4096
342 119/300 Write block 835 len 36864 data: 2 2 2 2 2 2 3 3 3
343 120/300 Write block 61 len 32768 data: 2 2 2 2 2 2 2 2
344 121/300 Flush
345 122/300 Flush
346 123/300 Write block 2349 len 12288 data: 2 2 2
347 124/300 Read block 4539 len 36864
348 125/300 Read block 932 len 16384
349 126/300 Read block 528 len 28672
350 127/300 Write block 2411 len 4096 data: 2
351 128/300 Read block 3375 len 16384
352 129/300 Flush
353 130/300 Read block 3185 len 24576
354 131/300 Read block 971 len 8192
355 132/300 Write block 469 len 24576 data: 2 2 2 2 2 2
356 133/300 Read block 1036 len 32768
357 134/300 Flush
358 135/300 Flush
359 136/300 Write block 2024 len 12288 data: 2 2 2
360 137/300 Read block 2834 len 40960
361 138/300 Write block 2660 len 28672 data: 2 2 2 3 3 3 2
362 139/300 Write block 517 len 4096 data: 2
363 140/300 Read block 1327 len 16384
364 141/300 Read block 1934 len 40960
365 142/300 Write block 3286 len 36864 data: 2 2 2 2 2 2 2 2 2
366 143/300 Flush
367 144/300 Read block 3426 len 20480
368 145/300 Write block 4607 len 8192 data: 2 2
369 146/300 Read block 1158 len 32768
370 147/300 Read block 1002 len 20480
371 148/300 Write block 4000 len 40960 data: 2 2 2 2 2 2 2 2 2 2
372 149/300 Read block 2862 len 36864
373 150/300 Write block 1022 len 12288 data: 2 2 2
374 151/300 Write block 488 len 4096 data: 2
375 152/300 Read block 4359 len 32768
376 153/300 Read block 4411 len 12288
377 154/300 Read block 3044 len 16384
378 155/300 Read block 2514 len 12288
379 156/300 Write block 24 len 36864 data: 2 2 2 2 2 2 2 2 2
380 157/300 Write block 4588 len 8192 data: 2 2
381 158/300 Write block 3443 len 32768 data: 2 2 2 2 2 2 2 2
382 159/300 Read block 4950 len 24576
383 160/300 Write block 3037 len 28672 data: 2 2 2 2 2 2 2
384 161/300 Flush
385 162/300 Write block 1664 len 12288 data: 2 2 2
386 163/300 Read block 622 len 20480
387 164/300 Read block 521 len 40960
388 165/300 Write block 3099 len 16384 data: 2 2 2 2
389 166/300 Read block 4989 len 40960
390 167/300 Write block 1229 len 12288 data: 2 2 2
391 168/300 Read block 2169 len 4096
392 169/300 Write block 103 len 36864 data: 2 2 2 2 2 2 2 2 2
393 170/300 Write block 1295 len 16384 data: 2 2 2 2
394 171/300 Read block 1704 len 20480
395 172/300 Read block 1837 len 36864
396 173/300 Read block 2863 len 4096
397 174/300 Read block 4826 len 4096
398 175/300 Write block 4714 len 36864 data: 2 2 2 2 2 2 2 2 2
399 176/300 Read block 3975 len 36864
400 177/300 Flush
401 178/300 Read block 4862 len 40960
402 179/300 Read block 4839 len 8192
403 180/300 Write block 2466 len 36864 data: 2 2 2 2 2 2 2 2 2
404 181/300 Read block 4222 len 40960
405 182/300 Write block 2649 len 40960 data: 2 2 3 3 3 3 3 3 3 3
406 183/300 Read block 1000 len 24576
407 184/300 Read block 4656 len 28672
408 185/300 Read block 2129 len 24576
409 186/300 Read block 2957 len 32768
410 187/300 Read block 65 len 16384
411 188/300 Read block 2688 len 12288
412 189/300 Write block 4346 len 24576 data: 2 2 2 2 2 2
413 190/300 Read block 3595 len 28672
414 191/300 Read block 1581 len 16384
415 192/300 Write block 2382 len 16384 data: 2 2 2 2
416 193/300 Write block 4212 len 40960 data: 2 2 2 2 2 2 2 2 2 2
417 194/300 Write block 4454 len 4096 data: 2
418 195/300 Write block 4524 len 36864 data: 2 2 2 2 2 2 2 2 2
419 196/300 Read block 4883 len 40960
420 197/300 Write block 4174 len 8192 data: 3 3
421 198/300 Write block 934 len 28672 data: 2 2 2 2 2 2 2
422 199/300 Write block 630 len 20480 data: 2 2 2 2 2
423 200/300 Read block 505 len 12288
424 201/300 Write block 4416 len 40960 data: 2 2 2 2 2 2 2 2 2 2
425 202/300 Read block 3462 len 40960
426 203/300 Write block 280 len 40960 data: 2 2 2 2 2 2 2 2 2 2
427 204/300 Write block 1169 len 20480 data: 2 2 2 2 2
428 205/300 Flush
429 206/300 Write block 2096 len 24576 data: 2 2 2 2 2 2
430 207/300 Read block 2723 len 20480
431 208/300 Write block 4266 len 24576 data: 2 2 2 2 2 2
432 209/300 Read block 4174 len 12288
433 210/300 Read block 316 len 4096
434 211/300 Write block 3412 len 20480 data: 2 2 2 2 2
435 212/300 Read block 2893 len 32768
436 213/300 Read block 360 len 32768
437 214/300 Read block 3531 len 28672
438 215/300 Write block 3103 len 4096 data: 2
439 216/300 Read block 4065 len 8192
440 217/300 Read block 4072 len 36864
441 218/300 Write block 2258 len 36864 data: 2 2 2 2 2 2 2 2 2
442 219/300 Read block 581 len 36864
443 220/300 Flush
444 221/300 Write block 4955 len 36864 data: 2 2 2 2 2 2 2 2 3
445 222/300 Read block 3311 len 8192
446 223/300 Read block 4938 len 8192
447 224/300 Write block 969 len 40960 data: 3 3 3 3 2 2 2 2 2 2
448 225/300 Write block 3218 len 8192 data: 2 2
449 226/300 Write block 3960 len 4096 data: 2
450 227/300 Read block 3223 len 28672
451 228/300 Write block 2539 len 40960 data: 2 2 2 2 2 2 2 2 2 2
452 229/300 Read block 3711 len 4096
453 230/300 Write block 18 len 20480 data: 2 2 2 2 2
454 231/300 Flush
455 232/300 Write block 962 len 28672 data: 2 2 2 2 2 2 2
456 233/300 Read block 2168 len 36864
457 234/300 Write block 4395 len 40960 data: 2 2 2 2 2 2 2 2 2 2
458 235/300 Flush
459 236/300 Write block 3075 len 36864 data: 2 2 2 2 2 2 2 2 2
460 237/300 Read block 1823 len 40960
461 238/300 Write block 4347 len 8192 data: 3 3
462 239/300 Read block 742 len 12288
463 240/300 Read block 2501 len 20480
464 241/300 Flush
465 242/300 Read block 4633 len 40960
466 243/300 Write block 1795 len 36864 data: 2 2 2 2 2 2 2 2 2
467 244/300 Read block 1293 len 40960
468 245/300 Read block 4757 len 36864
469 246/300 Read block 4325 len 28672
470 247/300 Write block 208 len 24576 data: 2 2 2 2 2 2
471 248/300 Read block 2769 len 32768
472 249/300 Write block 3732 len 36864 data: 2 2 2 2 2 2 2 2 2
473 250/300 Write block 494 len 4096 data: 2
474 251/300 Read block 408 len 24576
475 252/300 Read block 252 len 28672
476 253/300 Write block 940 len 24576 data: 3 2 2 2 2 2
477 254/300 Write block 1395 len 32768 data: 2 2 2 2 2 2 2 2
478 255/300 Write block 3743 len 8192 data: 2 2
479 256/300 Flush
480 257/300 Write block 2440 len 40960 data: 2 2 2 2 2 2 2 2 2 2
481 258/300 Write block 1723 len 36864 data: 2 2 2 2 2 2 2 2 2
482 259/300 Write block 1531 len 8192 data: 2 2
483 260/300 Write block 3184 len 36864 data: 2 2 2 2 2 2 2 2 2
484 261/300 Flush
485 262/300 Write block 4035 len 12288 data: 2 2 2
486 263/300 Write block 4252 len 12288 data: 2 2 2
487 264/300 Read block 2949 len 32768
488 265/300 Write block 737 len 4096 data: 2
489 266/300 Flush
490 267/300 Read block 4663 len 12288
491 268/300 Read block 2177 len 12288
492 269/300 Write block 721 len 24576 data: 2 2 2 2 2 2
493 270/300 Write block 359 len 40960 data: 2 2 2 2 2 2 2 2 2 2
494 271/300 Read block 1080 len 32768
495 272/300 Write block 3823 len 16384 data: 2 2 2 2
496 273/300 Write block 1892 len 4096 data: 2
497 274/300 Read block 2942 len 40960
498 275/300 Write block 1975 len 32768 data: 2 2 2 2 2 2 3 2
499 276/300 Write block 3351 len 32768 data: 2 2 2 2 2 2 2 2
500 277/300 Read block 3252 len 36864
501 278/300 Write block 802 len 20480 data: 2 2 2 2 2
502 279/300 Read block 4427 len 24576
503 280/300 Read block 3789 len 20480
504 281/300 Write block 1201 len 28672 data: 2 2 2 2 2 2 2
505 282/300 Flush
506 283/300 Flush
507 284/300 Write block 4323 len 36864 data: 4 4 3 3 3 2 2 2 2
508 285/300 Read block 544 len 12288
509 286/300 Write block 4277 len 8192 data: 2 2
510 287/300 Read block 570 len 16384
511 288/300 Write block 34 len 4096 data: 2
512 289/300 Write block 3595 len 4096 data: 2
513 290/300 Read block 2736 len 16384
514 291/300 Read block 4291 len 20480
515 292/300 Write block 250 len 12288 data: 3 3 3
516 293/300 Write block 1784 len 40960 data: 2 2 2 2 2 2 2 2 2 2
517 294/300 Read block 1095 len 28672
518 295/300 Write block 3869 len 32768 data: 2 2 2 2 2 2 2 2
519 296/300 Write block 2264 len 36864 data: 3 3 3 2 2 2 2 2 2
520 297/300 Read block 468 len 36864
521 298/300 Read block 3219 len 8192
522 299/300 Read block 884 len 16384
523 300/300 Write block 1437 len 8192 data: 2 2
524 301/300 Read block 3206 len 4096
525 Main task has work to do, go find it
526 got dsc Start(1)
527 start 1
528 Jan 17 21:11:12.589 INFO request completed, latency_us: 206, response_code: 204, uri: /start/cid/1, method: POST, req_id: 01ef6433-986d-4d96-ba34-dc77e9d019c9, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
529 Replay: started 1, returned:Ok(())
530 ----------------------------------------------------------------
531 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
532 States: Active Offline Active
533 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
534 1000 Acked Write 10 Done Sent Done false
535 1001 Acked Write 7 Done Sent Done false
536 1002 Acked Write 7 Done Sent Done false
537 1003 Acked Read 10 Done Sent Done false
538 1004 Acked Read 2 Done Sent Done false
539 1005 Acked Write 4 Done Sent Done false
540 1006 Acked Write 7 Done Sent Done false
541 1007 Acked Write 10 Done Sent Done false
542 1008 Acked Read 8 Done Sent Done false
543 1009 Acked Write 5 Done Sent Done false
544 1010 Acked Read 2 Done Sent Done false
545 1011 Acked Write 1 Done Sent Done false
546 1012 Acked Write 1 Done Sent Done false
547 1013 Acked Write 9 Done Sent Done false
548 1014 Acked Flush 0 Done Sent Done false
549 1015 Acked Read 4 Done Sent Done false
550 1016 Acked Read 5 Done Sent Done false
551 1017 Acked Write 6 Done Sent Done false
552 1018 Acked Flush 0 Done Sent Done false
553 1019 Acked Flush 0 Done Sent Done false
554 1020 Acked Write 7 Done Sent Done false
555 1021 Acked Write 5 Done Sent Done false
556 1022 Acked Write 8 Done Sent Done false
557 1023 Acked Read 9 Done Sent Done false
558 1024 Acked Read 9 Done Sent Done false
559 1025 Acked Write 6 Done Sent Done false
560 1026 Acked Read 10 Done Sent Done false
561 1027 Acked Write 7 Done Sent Done false
562 1028 Acked Read 4 Done Sent Done false
563 1029 Acked Flush 0 Done Sent Done false
564 1030 Acked Read 3 Done Sent Done false
565 1031 Acked Read 4 Done Sent Done false
566 1032 Acked Read 5 Done Sent Done false
567 1033 Acked Read 7 Done Sent Done false
568 1034 Acked Write 9 Done Sent Done false
569 1035 Acked Read 1 Done Sent Done false
570 1036 Acked Read 2 Done Sent Done false
571 1037 Acked Write 9 Done Sent Done false
572 1038 Acked Write 2 Done Sent Done false
573 1039 Acked Flush 0 Done Sent Done false
574 1040 Acked Write 3 Done Sent Done false
575 1041 Acked Read 5 Done Sent Done false
576 1042 Acked Read 2 Done Sent Done false
577 1043 Acked Read 10 Done Sent Done false
578 1044 Acked Read 5 Done Sent Done false
579 1045 Acked Write 4 Done Sent Done false
580 1046 Acked Read 4 Done Sent Done false
581 1047 Acked Flush 0 Done Sent Done false
582 1048 Acked Write 6 Done Sent Done false
583 1049 Acked Write 7 Done Sent Done false
584 1050 Acked Write 1 Done Sent Done false
585 1051 Acked Read 1 Done Sent Done false
586 1052 Acked Read 8 Done Sent Done false
587 1053 Acked Read 6 Done Sent Done false
588 1054 Acked Read 8 Done Sent Done false
589 1055 Acked Write 10 Done Sent Done false
590 1056 Acked Flush 0 Done Sent Done false
591 1057 Acked Read 9 Done Sent Done false
592 1058 Acked Write 3 Done Sent Done false
593 1059 Acked Write 4 Done Sent Done false
594 1060 Acked Write 7 Done Sent Done false
595 1061 Acked Write 1 Done Sent Done false
596 1062 Acked Flush 0 Done Sent Done false
597 1063 Acked Read 6 Done Sent Done false
598 1064 Acked Flush 0 Done Sent Done false
599 1065 Acked Flush 0 Done Sent Done false
600 1066 Acked Write 4 Done Sent Done false
601 1067 Acked Flush 0 Done Sent Done false
602 1068 Acked Write 9 Done Sent Done false
603 1069 Acked Flush 0 Done Sent Done false
604 1070 Acked Write 7 Done Sent Done false
605 1071 Acked Write 10 Done Sent Done false
606 1072 Acked Write 9 Done Sent Done false
607 1073 Acked Read 6 Done Sent Done false
608 1074 Acked Read 5 Done Sent Done false
609 1075 Acked Write 5 Done Sent Done false
610 1076 Acked Read 9 Done Sent Done false
611 1077 Acked Write 2 Done Sent Done false
612 1078 Acked Read 3 Done Sent Done false
613 1079 Acked Write 3 Done Sent Done false
614 1080 Acked Write 9 Done Sent Done false
615 1081 Acked Write 10 Done Sent Done false
616 1082 Acked Flush 0 Done Sent Done false
617 1083 Acked Write 9 Done Sent Done false
618 1084 Acked Write 10 Done Sent Done false
619 1085 Acked Read 10 Done Sent Done false
620 1086 Acked Write 7 Done Sent Done false
621 1087 Acked Read 1 Done Sent Done false
622 1088 Acked Read 6 Done Sent Done false
623 1089 Acked Read 5 Done Sent Done false
624 1090 Acked Read 5 Done Sent Done false
625 1091 Acked Read 8 Done Sent Done false
626 1092 Acked Read 1 Done Sent Done false
627 1093 Acked Write 8 Done Sent Done false
628 1094 Acked Write 6 Done Sent Done false
629 1095 Acked Write 3 Done Sent Done false
630 1096 Acked Flush 0 Done Sent Done false
631 1097 Acked Read 9 Done Sent Done false
632 1098 Acked Read 1 Done Sent Done false
633 1099 Acked Write 4 Done Sent Done false
634 1100 Acked Write 8 Done Sent Done false
635 1101 Acked Read 3 Done Sent Done false
636 1102 Acked Read 2 Done Sent Done false
637 1103 Acked Read 10 Done Sent Done false
638 1104 Acked Write 8 Done Sent Done false
639 1105 Acked Write 7 Done Sent Done false
640 1106 Acked Read 8 Done Sent Done false
641 1107 Acked Write 6 Done Sent Done false
642 1108 Acked Read 2 Done Sent Done false
643 1109 Acked Read 5 Done Sent Done false
644 1110 Acked Write 10 Done Sent Done false
645 1111 Acked Flush 0 Done Sent Done false
646 1112 Acked Write 2 Done Sent Done false
647 1113 Acked Write 1 Done Sent Done false
648 1114 Acked Read 8 Done Sent Done false
649 1115 Acked Read 8 Done Sent Done false
650 1116 Acked Read 7 Done Sent Done false
651 1117 Acked Read 1 Done Sent Done false
652 1118 Acked Write 9 Done Sent Done false
653 1119 Acked Write 8 Done Sent Done false
654 1120 Acked Flush 0 Done Sent Done false
655 1121 Acked Flush 0 Done Sent Done false
656 1122 Acked Write 3 Done Sent Done false
657 1123 Acked Read 9 Done Sent Done false
658 1124 Acked Read 4 Done Sent Done false
659 1125 Acked Read 7 Done Sent Done false
660 1126 Acked Write 1 Done Sent Done false
661 1127 Acked Read 4 Done Sent Done false
662 1128 Acked Flush 0 Done Sent Done false
663 1129 Acked Read 6 Done Sent Done false
664 1130 Acked Read 2 Done Sent Done false
665 1131 Acked Write 6 Done Sent Done false
666 1132 Acked Read 8 Done Sent Done false
667 1133 Acked Flush 0 Done Sent Done false
668 1134 Acked Flush 0 Done Sent Done false
669 1135 Acked Write 3 Done Sent Done false
670 1136 Acked Read 10 Done Sent Done false
671 1137 Acked Write 7 Done Sent Done false
672 1138 Acked Write 1 Done Sent Done false
673 1139 Acked Read 4 Done Sent Done false
674 1140 Acked Read 10 Done Sent Done false
675 1141 Acked Write 9 Done Sent Done false
676 1142 Acked Flush 0 Done Sent Done false
677 1143 Acked Read 5 Done Sent Done false
678 1144 Acked Write 2 Done Sent Done false
679 1145 Acked Read 8 Done Sent Done false
680 1146 Acked Read 5 Done Sent Done false
681 1147 Acked Write 10 Done Sent Done false
682 1148 Acked Read 9 Done Sent Done false
683 1149 Acked Write 3 Done Sent Done false
684 1150 Acked Write 1 Done Sent Done false
685 1151 Acked Read 8 Done Sent Done false
686 1152 Acked Read 3 Done Sent Done false
687 1153 Acked Read 4 Done Sent Done false
688 1154 Acked Read 3 Done Sent Done false
689 1155 Acked Write 9 Done Sent Done false
690 1156 Acked Write 2 Done Sent Done false
691 1157 Acked Write 8 Done Sent Done false
692 1158 Acked Read 6 Done Sent Done false
693 1159 Acked Write 7 Done Sent Done false
694 1160 Acked Flush 0 Done Sent Done false
695 1161 Acked Write 3 Done Sent Done false
696 1162 Acked Read 5 Done Sent Done false
697 1163 Acked Read 10 Done Sent Done false
698 1164 Acked Write 4 Done Sent Done false
699 1165 Acked Read 10 Done Sent Done false
700 1166 Acked Write 3 Done Sent Done false
701 1167 Acked Read 1 Done Sent Done false
702 1168 Acked Write 9 Done Sent Done false
703 1169 Acked Write 4 Done Sent Done false
704 1170 Acked Read 5 Done Sent Done false
705 1171 Acked Read 9 Done Sent Done false
706 1172 Acked Read 1 Done Sent Done false
707 1173 Acked Read 1 Done Sent Done false
708 1174 Acked Write 9 Done Sent Done false
709 1175 Acked Read 9 Done Sent Done false
710 1176 Acked Flush 0 Done Sent Done false
711 1177 Acked Read 10 Done Sent Done false
712 1178 Acked Read 2 Done Sent Done false
713 1179 Acked Write 9 Done Sent Done false
714 1180 Acked Read 10 Done Sent Done false
715 1181 Acked Write 10 Done Sent Done false
716 1182 Acked Read 6 Done Sent Done false
717 1183 Acked Read 7 Done Sent Done false
718 1184 Acked Read 6 Done Sent Done false
719 1185 Acked Read 8 Done Sent Done false
720 1186 Acked Read 4 Done Sent Done false
721 1187 Acked Read 3 Done Sent Done false
722 1188 Acked Write 6 Done Sent Done false
723 1189 Acked Read 7 Done Sent Done false
724 1190 Acked Read 4 Done Sent Done false
725 1191 Acked Write 4 Done Sent Done false
726 1192 Acked Write 10 Done Sent Done false
727 1193 Acked Write 1 Done Sent Done false
728 1194 Acked Write 9 Done Sent Done false
729 1195 Acked Read 10 Done Sent Done false
730 1196 Acked Write 2 Done Sent Done false
731 1197 Acked Write 7 Done Sent Done false
732 1198 Acked Write 5 Done Sent Done false
733 1199 Acked Read 3 Done Sent Done false
734 1200 Acked Write 10 Done Sent Done false
735 1201 Acked Read 10 Done Sent Done false
736 1202 Acked Write 10 Done Sent Done false
737 1203 Acked Write 5 Done Sent Done false
738 1204 Acked Flush 0 Done Sent Done false
739 1205 Acked Write 6 Done Sent Done false
740 1206 Acked Read 5 Done Sent Done false
741 1207 Acked Write 6 Done Sent Done false
742 1208 Acked Read 3 Done Sent Done false
743 1209 Acked Read 1 Done Sent Done false
744 1210 Acked Write 5 Done Sent Done false
745 1211 Acked Read 8 Done Sent Done false
746 1212 Acked Read 8 Done Sent Done false
747 1213 Acked Read 7 Done Sent Done false
748 1214 Acked Write 1 Done Sent Done false
749 1215 Acked Read 2 Done Sent Done false
750 1216 Acked Read 9 Done Sent Done false
751 1217 Acked Write 9 Done Sent Done false
752 1218 Acked Read 9 Done Sent Done false
753 1219 Acked Flush 0 Done Sent Done false
754 1220 Acked Write 9 Done Sent Done false
755 1221 Acked Read 2 Done Sent Done false
756 1222 Acked Read 2 Done Sent Done false
757 1223 Acked Write 10 Done Sent Done false
758 1224 Acked Write 2 Done Sent Done false
759 1225 Acked Write 1 Done Sent Done false
760 1226 Acked Read 7 Done Sent Done false
761 1227 Acked Write 10 Done Sent Done false
762 1228 Acked Read 1 Done Sent Done false
763 1229 Acked Write 5 Done Sent Done false
764 1230 Acked Flush 0 Done Sent Done false
765 1231 Acked Write 7 Done Sent Done false
766 1232 Acked Read 9 Done Sent Done false
767 1233 Acked Write 10 Done Sent Done false
768 1234 Acked Flush 0 Done Sent Done false
769 1235 Acked Write 9 Done Sent Done false
770 1236 Acked Read 10 Done Sent Done false
771 1237 Acked Write 2 Done Sent Done false
772 1238 Acked Read 3 Done Sent Done false
773 1239 Acked Read 5 Done Sent Done false
774 1240 Acked Flush 0 Done Sent Done false
775 1241 Acked Read 10 Done Sent Done false
776 1242 Acked Write 9 Done Sent Done false
777 1243 Acked Read 10 Done Sent Done false
778 1244 Acked Read 9 Done Sent Done false
779 1245 Acked Read 7 Done Sent Done false
780 1246 Acked Write 6 Done Sent Done false
781 1247 Acked Read 8 Done Sent Done false
782 1248 Acked Write 9 Done Sent Done false
783 1249 Acked Write 1 Done Sent Done false
784 1250 Acked Read 6 Done Sent Done false
785 1251 Acked Read 7 Done Sent Done false
786 1252 Acked Write 6 Done Sent Done false
787 1253 Acked Write 8 Done Sent Done false
788 1254 Acked Write 2 Done Sent Done false
789 1255 Acked Flush 0 Done Sent Done false
790 1256 Acked Write 10 Done Sent Done false
791 1257 Acked Write 9 Done Sent Done false
792 1258 Acked Write 2 Done Sent Done false
793 1259 Acked Write 9 Done Sent Done false
794 1260 Acked Flush 0 Done Sent Done false
795 1261 Acked Write 3 Done Sent Done false
796 1262 Acked Write 3 Done Sent Done false
797 1263 Acked Read 8 Done Sent Done false
798 1264 Acked Write 1 Done Sent Done false
799 1265 Acked Flush 0 Done Sent Done false
800 1266 Acked Read 3 Done Sent Done false
801 1267 Acked Read 3 Done Sent Done false
802 1268 Acked Write 6 Done Sent Done false
803 1269 Acked Write 10 Done Sent Done false
804 1270 Acked Read 8 Done Sent Done false
805 1271 Acked Write 4 Done Sent Done false
806 1272 Acked Write 1 Done Sent Done false
807 1273 Acked Read 10 Done Sent Done false
808 1274 Acked Write 8 Done Sent Done false
809 1275 Acked Write 8 Done Sent Done false
810 1276 Acked Read 9 Done Sent Done false
811 1277 Acked Write 5 Done Sent Done false
812 1278 Acked Read 6 Done Sent Done false
813 1279 Acked Read 5 Done Sent Done false
814 1280 Acked Write 7 Done Sent Done false
815 1281 Acked Flush 0 Done Sent Done false
816 1282 Acked Flush 0 Done Sent Done false
817 1283 Acked Write 9 Done Sent Done false
818 1284 Acked Read 3 Done Sent Done false
819 1285 Acked Write 2 Done Sent Done false
820 1286 Acked Read 4 Done Sent Done false
821 1287 Acked Write 1 Done Sent Done false
822 1288 Acked Write 1 Done Sent Done false
823 1289 Acked Read 4 Done Sent Done false
824 1290 Acked Read 5 Done Sent Done false
825 1291 Acked Write 3 Done Sent Done false
826 1292 Acked Write 10 Done Sent Done false
827 1293 Acked Read 7 Done Sent Done false
828 1294 Acked Write 8 Done Sent Done false
829 1295 Acked Write 9 Done Sent Done false
830 1296 Acked Read 9 Done Sent Done false
831 1297 Acked Read 2 Done Sent Done false
832 1298 Acked Read 4 Done Sent Done false
833 1299 Acked Write 2 Done Sent Done false
834 1300 Acked Read 1 Done Sent Done false
835 STATES DS:0 DS:1 DS:2 TOTAL
836 Sent 0 301 0 301
837 Done 301 0 301 602
838 Skipped 0 0 0 0
839 Error 0 0 0 0
840 Last Flush: 1282 0 1282
841 Downstairs last five completed:
842 Upstairs last five completed: 1300 1299 1298 1297 1296
843 CLIENT: Up:0 ds:301 act:2
844 [8820] got start action
845 [8820] I am going to restart
846 Starting downstairs at port 8820
847 Make output file at "/tmp/dsc/downstairs-8820.txt"
848 [8820][1] reports Starting
849 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1201)
850 [8820][1] reports Running
851 ----------------------------------------------------------------
852 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
853 States: Active Offline Active
854 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
855 1000 Acked Write 10 Done Sent Done false
856 1001 Acked Write 7 Done Sent Done false
857 1002 Acked Write 7 Done Sent Done false
858 1003 Acked Read 10 Done Sent Done false
859 1004 Acked Read 2 Done Sent Done false
860 1005 Acked Write 4 Done Sent Done false
861 1006 Acked Write 7 Done Sent Done false
862 1007 Acked Write 10 Done Sent Done false
863 1008 Acked Read 8 Done Sent Done false
864 1009 Acked Write 5 Done Sent Done false
865 1010 Acked Read 2 Done Sent Done false
866 1011 Acked Write 1 Done Sent Done false
867 1012 Acked Write 1 Done Sent Done false
868 1013 Acked Write 9 Done Sent Done false
869 1014 Acked Flush 0 Done Sent Done false
870 1015 Acked Read 4 Done Sent Done false
871 1016 Acked Read 5 Done Sent Done false
872 1017 Acked Write 6 Done Sent Done false
873 1018 Acked Flush 0 Done Sent Done false
874 1019 Acked Flush 0 Done Sent Done false
875 1020 Acked Write 7 Done Sent Done false
876 1021 Acked Write 5 Done Sent Done false
877 1022 Acked Write 8 Done Sent Done false
878 1023 Acked Read 9 Done Sent Done false
879 1024 Acked Read 9 Done Sent Done false
880 1025 Acked Write 6 Done Sent Done false
881 1026 Acked Read 10 Done Sent Done false
882 1027 Acked Write 7 Done Sent Done false
883 1028 Acked Read 4 Done Sent Done false
884 1029 Acked Flush 0 Done Sent Done false
885 1030 Acked Read 3 Done Sent Done false
886 1031 Acked Read 4 Done Sent Done false
887 1032 Acked Read 5 Done Sent Done false
888 1033 Acked Read 7 Done Sent Done false
889 1034 Acked Write 9 Done Sent Done false
890 1035 Acked Read 1 Done Sent Done false
891 1036 Acked Read 2 Done Sent Done false
892 1037 Acked Write 9 Done Sent Done false
893 1038 Acked Write 2 Done Sent Done false
894 1039 Acked Flush 0 Done Sent Done false
895 1040 Acked Write 3 Done Sent Done false
896 1041 Acked Read 5 Done Sent Done false
897 1042 Acked Read 2 Done Sent Done false
898 1043 Acked Read 10 Done Sent Done false
899 1044 Acked Read 5 Done Sent Done false
900 1045 Acked Write 4 Done Sent Done false
901 1046 Acked Read 4 Done Sent Done false
902 1047 Acked Flush 0 Done Sent Done false
903 1048 Acked Write 6 Done Sent Done false
904 1049 Acked Write 7 Done Sent Done false
905 1050 Acked Write 1 Done Sent Done false
906 1051 Acked Read 1 Done Sent Done false
907 1052 Acked Read 8 Done Sent Done false
908 1053 Acked Read 6 Done Sent Done false
909 1054 Acked Read 8 Done Sent Done false
910 1055 Acked Write 10 Done Sent Done false
911 1056 Acked Flush 0 Done Sent Done false
912 1057 Acked Read 9 Done Sent Done false
913 1058 Acked Write 3 Done Sent Done false
914 1059 Acked Write 4 Done Sent Done false
915 1060 Acked Write 7 Done Sent Done false
916 1061 Acked Write 1 Done Sent Done false
917 1062 Acked Flush 0 Done Sent Done false
918 1063 Acked Read 6 Done Sent Done false
919 1064 Acked Flush 0 Done Sent Done false
920 1065 Acked Flush 0 Done Sent Done false
921 1066 Acked Write 4 Done Sent Done false
922 1067 Acked Flush 0 Done Sent Done false
923 1068 Acked Write 9 Done Sent Done false
924 1069 Acked Flush 0 Done Sent Done false
925 1070 Acked Write 7 Done Sent Done false
926 1071 Acked Write 10 Done Sent Done false
927 1072 Acked Write 9 Done Sent Done false
928 1073 Acked Read 6 Done Sent Done false
929 1074 Acked Read 5 Done Sent Done false
930 1075 Acked Write 5 Done Sent Done false
931 1076 Acked Read 9 Done Sent Done false
932 1077 Acked Write 2 Done Sent Done false
933 1078 Acked Read 3 Done Sent Done false
934 1079 Acked Write 3 Done Sent Done false
935 1080 Acked Write 9 Done Sent Done false
936 1081 Acked Write 10 Done Sent Done false
937 1082 Acked Flush 0 Done Sent Done false
938 1083 Acked Write 9 Done Sent Done false
939 1084 Acked Write 10 Done Sent Done false
940 1085 Acked Read 10 Done Sent Done false
941 1086 Acked Write 7 Done Sent Done false
942 1087 Acked Read 1 Done Sent Done false
943 1088 Acked Read 6 Done Sent Done false
944 1089 Acked Read 5 Done Sent Done false
945 1090 Acked Read 5 Done Sent Done false
946 1091 Acked Read 8 Done Sent Done false
947 1092 Acked Read 1 Done Sent Done false
948 1093 Acked Write 8 Done Sent Done false
949 1094 Acked Write 6 Done Sent Done false
950 1095 Acked Write 3 Done Sent Done false
951 1096 Acked Flush 0 Done Sent Done false
952 1097 Acked Read 9 Done Sent Done false
953 1098 Acked Read 1 Done Sent Done false
954 1099 Acked Write 4 Done Sent Done false
955 1100 Acked Write 8 Done Sent Done false
956 1101 Acked Read 3 Done Sent Done false
957 1102 Acked Read 2 Done Sent Done false
958 1103 Acked Read 10 Done Sent Done false
959 1104 Acked Write 8 Done Sent Done false
960 1105 Acked Write 7 Done Sent Done false
961 1106 Acked Read 8 Done Sent Done false
962 1107 Acked Write 6 Done Sent Done false
963 1108 Acked Read 2 Done Sent Done false
964 1109 Acked Read 5 Done Sent Done false
965 1110 Acked Write 10 Done Sent Done false
966 1111 Acked Flush 0 Done Sent Done false
967 1112 Acked Write 2 Done Sent Done false
968 1113 Acked Write 1 Done Sent Done false
969 1114 Acked Read 8 Done Sent Done false
970 1115 Acked Read 8 Done Sent Done false
971 1116 Acked Read 7 Done Sent Done false
972 1117 Acked Read 1 Done Sent Done false
973 1118 Acked Write 9 Done Sent Done false
974 1119 Acked Write 8 Done Sent Done false
975 1120 Acked Flush 0 Done Sent Done false
976 1121 Acked Flush 0 Done Sent Done false
977 1122 Acked Write 3 Done Sent Done false
978 1123 Acked Read 9 Done Sent Done false
979 1124 Acked Read 4 Done Sent Done false
980 1125 Acked Read 7 Done Sent Done false
981 1126 Acked Write 1 Done Sent Done false
982 1127 Acked Read 4 Done Sent Done false
983 1128 Acked Flush 0 Done Sent Done false
984 1129 Acked Read 6 Done Sent Done false
985 1130 Acked Read 2 Done Sent Done false
986 1131 Acked Write 6 Done Sent Done false
987 1132 Acked Read 8 Done Sent Done false
988 1133 Acked Flush 0 Done Sent Done false
989 1134 Acked Flush 0 Done Sent Done false
990 1135 Acked Write 3 Done Sent Done false
991 1136 Acked Read 10 Done Sent Done false
992 1137 Acked Write 7 Done Sent Done false
993 1138 Acked Write 1 Done Sent Done false
994 1139 Acked Read 4 Done Sent Done false
995 1140 Acked Read 10 Done Sent Done false
996 1141 Acked Write 9 Done Sent Done false
997 1142 Acked Flush 0 Done Sent Done false
998 1143 Acked Read 5 Done Sent Done false
999 1144 Acked Write 2 Done Sent Done false
1000 1145 Acked Read 8 Done Sent Done false
1001 1146 Acked Read 5 Done Sent Done false
1002 1147 Acked Write 10 Done Sent Done false
1003 1148 Acked Read 9 Done Sent Done false
1004 1149 Acked Write 3 Done Sent Done false
1005 1150 Acked Write 1 Done Sent Done false
1006 1151 Acked Read 8 Done Sent Done false
1007 1152 Acked Read 3 Done Sent Done false
1008 1153 Acked Read 4 Done Sent Done false
1009 1154 Acked Read 3 Done Sent Done false
1010 1155 Acked Write 9 Done Sent Done false
1011 1156 Acked Write 2 Done Sent Done false
1012 1157 Acked Write 8 Done Sent Done false
1013 1158 Acked Read 6 Done Sent Done false
1014 1159 Acked Write 7 Done Sent Done false
1015 1160 Acked Flush 0 Done Sent Done false
1016 1161 Acked Write 3 Done Sent Done false
1017 1162 Acked Read 5 Done Sent Done false
1018 1163 Acked Read 10 Done Sent Done false
1019 1164 Acked Write 4 Done Sent Done false
1020 1165 Acked Read 10 Done Sent Done false
1021 1166 Acked Write 3 Done Sent Done false
1022 1167 Acked Read 1 Done Sent Done false
1023 1168 Acked Write 9 Done Sent Done false
1024 1169 Acked Write 4 Done Sent Done false
1025 1170 Acked Read 5 Done Sent Done false
1026 1171 Acked Read 9 Done Sent Done false
1027 1172 Acked Read 1 Done Sent Done false
1028 1173 Acked Read 1 Done Sent Done false
1029 1174 Acked Write 9 Done Sent Done false
1030 1175 Acked Read 9 Done Sent Done false
1031 1176 Acked Flush 0 Done Sent Done false
1032 1177 Acked Read 10 Done Sent Done false
1033 1178 Acked Read 2 Done Sent Done false
1034 1179 Acked Write 9 Done Sent Done false
1035 1180 Acked Read 10 Done Sent Done false
1036 1181 Acked Write 10 Done Sent Done false
1037 1182 Acked Read 6 Done Sent Done false
1038 1183 Acked Read 7 Done Sent Done false
1039 1184 Acked Read 6 Done Sent Done false
1040 1185 Acked Read 8 Done Sent Done false
1041 1186 Acked Read 4 Done Sent Done false
1042 1187 Acked Read 3 Done Sent Done false
1043 1188 Acked Write 6 Done Sent Done false
1044 1189 Acked Read 7 Done Sent Done false
1045 1190 Acked Read 4 Done Sent Done false
1046 1191 Acked Write 4 Done Sent Done false
1047 1192 Acked Write 10 Done Sent Done false
1048 1193 Acked Write 1 Done Sent Done false
1049 1194 Acked Write 9 Done Sent Done false
1050 1195 Acked Read 10 Done Sent Done false
1051 1196 Acked Write 2 Done Sent Done false
1052 1197 Acked Write 7 Done Sent Done false
1053 1198 Acked Write 5 Done Sent Done false
1054 1199 Acked Read 3 Done Sent Done false
1055 1200 Acked Write 10 Done Sent Done false
1056 1201 Acked Read 10 Done Sent Done false
1057 1202 Acked Write 10 Done Sent Done false
1058 1203 Acked Write 5 Done Sent Done false
1059 1204 Acked Flush 0 Done Sent Done false
1060 1205 Acked Write 6 Done Sent Done false
1061 1206 Acked Read 5 Done Sent Done false
1062 1207 Acked Write 6 Done Sent Done false
1063 1208 Acked Read 3 Done Sent Done false
1064 1209 Acked Read 1 Done Sent Done false
1065 1210 Acked Write 5 Done Sent Done false
1066 1211 Acked Read 8 Done Sent Done false
1067 1212 Acked Read 8 Done Sent Done false
1068 1213 Acked Read 7 Done Sent Done false
1069 1214 Acked Write 1 Done Sent Done false
1070 1215 Acked Read 2 Done Sent Done false
1071 1216 Acked Read 9 Done Sent Done false
1072 1217 Acked Write 9 Done Sent Done false
1073 1218 Acked Read 9 Done Sent Done false
1074 1219 Acked Flush 0 Done Sent Done false
1075 1220 Acked Write 9 Done Sent Done false
1076 1221 Acked Read 2 Done Sent Done false
1077 1222 Acked Read 2 Done Sent Done false
1078 1223 Acked Write 10 Done Sent Done false
1079 1224 Acked Write 2 Done Sent Done false
1080 1225 Acked Write 1 Done Sent Done false
1081 1226 Acked Read 7 Done Sent Done false
1082 1227 Acked Write 10 Done Sent Done false
1083 1228 Acked Read 1 Done Sent Done false
1084 1229 Acked Write 5 Done Sent Done false
1085 1230 Acked Flush 0 Done Sent Done false
1086 1231 Acked Write 7 Done Sent Done false
1087 1232 Acked Read 9 Done Sent Done false
1088 1233 Acked Write 10 Done Sent Done false
1089 1234 Acked Flush 0 Done Sent Done false
1090 1235 Acked Write 9 Done Sent Done false
1091 1236 Acked Read 10 Done Sent Done false
1092 1237 Acked Write 2 Done Sent Done false
1093 1238 Acked Read 3 Done Sent Done false
1094 1239 Acked Read 5 Done Sent Done false
1095 1240 Acked Flush 0 Done Sent Done false
1096 1241 Acked Read 10 Done Sent Done false
1097 1242 Acked Write 9 Done Sent Done false
1098 1243 Acked Read 10 Done Sent Done false
1099 1244 Acked Read 9 Done Sent Done false
1100 1245 Acked Read 7 Done Sent Done false
1101 1246 Acked Write 6 Done Sent Done false
1102 1247 Acked Read 8 Done Sent Done false
1103 1248 Acked Write 9 Done Sent Done false
1104 1249 Acked Write 1 Done Sent Done false
1105 1250 Acked Read 6 Done Sent Done false
1106 1251 Acked Read 7 Done Sent Done false
1107 1252 Acked Write 6 Done Sent Done false
1108 1253 Acked Write 8 Done Sent Done false
1109 1254 Acked Write 2 Done Sent Done false
1110 1255 Acked Flush 0 Done Sent Done false
1111 1256 Acked Write 10 Done Sent Done false
1112 1257 Acked Write 9 Done Sent Done false
1113 1258 Acked Write 2 Done Sent Done false
1114 1259 Acked Write 9 Done Sent Done false
1115 1260 Acked Flush 0 Done Sent Done false
1116 1261 Acked Write 3 Done Sent Done false
1117 1262 Acked Write 3 Done Sent Done false
1118 1263 Acked Read 8 Done Sent Done false
1119 1264 Acked Write 1 Done Sent Done false
1120 1265 Acked Flush 0 Done Sent Done false
1121 1266 Acked Read 3 Done Sent Done false
1122 1267 Acked Read 3 Done Sent Done false
1123 1268 Acked Write 6 Done Sent Done false
1124 1269 Acked Write 10 Done Sent Done false
1125 1270 Acked Read 8 Done Sent Done false
1126 1271 Acked Write 4 Done Sent Done false
1127 1272 Acked Write 1 Done Sent Done false
1128 1273 Acked Read 10 Done Sent Done false
1129 1274 Acked Write 8 Done Sent Done false
1130 1275 Acked Write 8 Done Sent Done false
1131 1276 Acked Read 9 Done Sent Done false
1132 1277 Acked Write 5 Done Sent Done false
1133 1278 Acked Read 6 Done Sent Done false
1134 1279 Acked Read 5 Done Sent Done false
1135 1280 Acked Write 7 Done Sent Done false
1136 1281 Acked Flush 0 Done Sent Done false
1137 1282 Acked Flush 0 Done Sent Done false
1138 1283 Acked Write 9 Done Sent Done false
1139 1284 Acked Read 3 Done Sent Done false
1140 1285 Acked Write 2 Done Sent Done false
1141 1286 Acked Read 4 Done Sent Done false
1142 1287 Acked Write 1 Done Sent Done false
1143 1288 Acked Write 1 Done Sent Done false
1144 1289 Acked Read 4 Done Sent Done false
1145 1290 Acked Read 5 Done Sent Done false
1146 1291 Acked Write 3 Done Sent Done false
1147 1292 Acked Write 10 Done Sent Done false
1148 1293 Acked Read 7 Done Sent Done false
1149 1294 Acked Write 8 Done Sent Done false
1150 1295 Acked Write 9 Done Sent Done false
1151 1296 Acked Read 9 Done Sent Done false
1152 1297 Acked Read 2 Done Sent Done false
1153 1298 Acked Read 4 Done Sent Done false
1154 1299 Acked Write 2 Done Sent Done false
1155 1300 Acked Read 1 Done Sent Done false
1156 1301 Acked Flush 0 Done Sent Done false
1157 STATES DS:0 DS:1 DS:2 TOTAL
1158 Sent 0 302 0 302
1159 Done 302 0 302 604
1160 Skipped 0 0 0 0
1161 Error 0 0 0 0
1162 Last Flush: 1301 0 1301
1163 Downstairs last five completed:
1164 Upstairs last five completed: 1301 1300 1299 1298 1297
1165 CLIENT: Up:0 ds:302 act:2
11662025-01-17T21:11:18.349ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
11672025-01-17T21:11:18.349ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:60405) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
11682025-01-17T21:11:18.350ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
11692025-01-17T21:11:18.350ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
11702025-01-17T21:11:18.350ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
11712025-01-17T21:11:18.351ZINFOcrucible: send last flush ID to this DS: 0 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
11722025-01-17T21:11:18.351ZINFOcrucible: Replied this last flush ID: 0 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
11732025-01-17T21:11:18.351ZINFOcrucible: [1] client re-new 302 jobs since flush 0 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
11742025-01-17T21:11:18.351ZINFOcrucible: [1] Replayed 302 jobs since flush: 0 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1175 ----------------------------------------------------------------
1176 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
1177 Downstairs last five completed: 1301 1300 1299 1298 1297
1178 Upstairs last five completed: 1301 1300 1299 1298 1297
1179 CLIENT: Up:0 ds:0 act:3
1180 Replay: All jobs finished
1181 Main task has work to do, go find it
1182 got dsc Stop(1)
1183 stop 1
1184 Jan 17 21:11:20.594 INFO request completed[8820] Got stop action so:false kr:false
1185 , latency_us: 323, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 72ab4056-ffe9-4007-b180-0511f3ff6061, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
11862025-01-17T21:11:20.594ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
11872025-01-17T21:11:20.594ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
11882025-01-17T21:11:20.594ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1189 Jan 17 21:11:20.594 INFO request completed, latency_us: 134, response_code: 200, uri: /state/cid/1, method: GET, req_id: 89cde498-e2fd-4936-859c-4f961f2d5eac, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
11902025-01-17T21:11:20.594ZINFOcrucible: client task is exiting = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
1191 [8820] Exited with: ExitStatus(unix_wait_status(9))
1192 [8820][1] reports Exit
1193 Jan 17 21:11:24.596 INFO request completed, latency_us: 219, response_code: 200, uri: /state/cid/1, method: GET, req_id: 5ad70cb6-f00f-4ce6-8cfd-9eeaa5df6c3f, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
1194 001/300 Read block 4016 len 20480
1195 002/300 Write block 1391 len 12288 data: 2 2 2
1196 003/300 Read block 4756 len 36864
1197 004/300 Read block 2770 len 24576
1198 005/300 Flush
1199 006/300 Read block 3553 len 32768
1200 007/300 Read block 1534 len 16384
1201 008/300 Read block 314 len 40960
1202 009/300 Read block 432 len 12288
1203 010/300 Write block 1912 len 28672 data: 2 2 2 2 2 2 2
1204 011/300 Write block 4444 len 8192 data: 2 2
1205 012/300 Write block 1230 len 32768 data: 3 3 2 2 2 2 2 2
1206 013/300 Read block 2329 len 36864
1207 014/300 Read block 190 len 36864
1208 015/300 Write block 1989 len 16384 data: 2 2 2 2
1209 016/300 Write block 4306 len 16384 data: 2 2 2 2
1210 017/300 Write block 1950 len 24576 data: 3 3 3 3 3 3
1211 018/300 Read block 462 len 28672
1212 019/300 Read block 3180 len 8192
1213 020/300 Read block 53 len 40960
1214 021/300 Read block 4023 len 36864
1215 022/300 Read block 1794 len 4096
1216 023/300 Read block 4471 len 12288
1217 024/300 Read block 4199 len 28672
1218 025/300 Write block 1500 len 12288 data: 2 2 2
1219 026/300 Read block 1652 len 12288
1220 027/300 Write block 3199 len 36864 data: 2 2 2 2 2 2 2 2 2
1221 028/300 Read block 3480 len 8192
1222 029/300 Read block 2219 len 40960
1223 030/300 Flush
1224 031/300 Read block 3455 len 16384
1225 032/300 Write block 2159 len 20480 data: 2 2 3 3 3
1226 033/300 Read block 4804 len 28672
1227 034/300 Write block 3501 len 12288 data: 2 2 2
1228 035/300 Write block 4828 len 4096 data: 2
1229 036/300 Write block 1095 len 40960 data: 3 3 3 3 3 2 2 2 2 2
1230 037/300 Read block 1252 len 16384
1231 038/300 Write block 3335 len 28672 data: 2 2 2 2 2 2 2
1232 039/300 Read block 2129 len 32768
1233 040/300 Read block 1149 len 4096
1234 041/300 Write block 4794 len 12288 data: 2 2 2
1235 042/300 Write block 2106 len 16384 data: 2 2 2 2
1236 043/300 Write block 3005 len 24576 data: 2 2 2 2 2 2
1237 044/300 Read block 1087 len 12288
1238 045/300 Read block 3145 len 4096
1239 046/300 Write block 3445 len 16384 data: 3 3 3 3
1240 047/300 Write block 1434 len 24576 data: 2 2 2 3 3 2
1241 048/300 Read block 1587 len 36864
1242 049/300 Read block 368 len 4096
1243 050/300 Write block 1358 len 16384 data: 2 2 2 2
1244 051/300 Write block 4118 len 24576 data: 2 2 2 2 2 2
1245 052/300 Write block 4781 len 4096 data: 3
1246 053/300 Flush
1247 054/300 Read block 760 len 32768
1248 055/300 Write block 2591 len 36864 data: 2 2 2 2 2 2 2 2 2
1249 056/300 Read block 1437 len 4096
1250 057/300 Write block 1114 len 20480 data: 2 2 2 2 2
1251 058/300 Flush
1252 059/300 Flush
1253 060/300 Read block 847 len 4096
1254 061/300 Write block 4952 len 4096 data: 2
1255 062/300 Read block 4492 len 20480
1256 063/300 Read block 4441 len 32768
1257 064/300 Write block 3471 len 20480 data: 2 2 2 2 2
1258 065/300 Write block 3754 len 12288 data: 2 2 2
1259 066/300 Write block 2887 len 36864 data: 2 2 2 2 2 2 2 2 2
1260 067/300 Read block 2974 len 4096
1261 068/300 Write block 50 len 20480 data: 2 2 2 2 2
1262 069/300 Write block 2987 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1263 070/300 Read block 4389 len 40960
1264 071/300 Read block 4597 len 20480
1265 072/300 Read block 1200 len 28672
1266 073/300 Read block 1001 len 12288
1267 074/300 Read block 1869 len 12288
1268 075/300 Write block 423 len 8192 data: 3 3
1269 076/300 Read block 54 len 8192
1270 077/300 Read block 574 len 40960
1271 078/300 Read block 1577 len 4096
1272 079/300 Read block 4476 len 28672
1273 080/300 Write block 3829 len 8192 data: 2 2
1274 081/300 Write block 1500 len 28672 data: 3 3 3 2 2 2 2
1275 082/300 Write block 2693 len 8192 data: 2 2
1276 083/300 Write block 2583 len 28672 data: 2 2 2 2 2 2 2
1277 084/300 Read block 1817 len 20480
1278 085/300 Write block 1882 len 28672 data: 2 2 2 2 2 2 2
1279 086/300 Read block 4560 len 40960
1280 087/300 Write block 4532 len 28672 data: 3 2 2 2 2 2 2
1281 088/300 Read block 2897 len 28672
1282 089/300 Read block 2322 len 20480
1283 090/300 Write block 3609 len 20480 data: 2 2 2 2 2
1284 091/300 Flush
1285 092/300 Write block 1731 len 20480 data: 3 2 2 2 2
1286 093/300 Write block 3711 len 28672 data: 2 2 2 2 2 2 2
1287 094/300 Read block 1570 len 24576
1288 095/300 Write block 694 len 24576 data: 2 2 2 2 2 2
1289 096/300 Read block 756 len 4096
1290 097/300 Write block 2381 len 32768 data: 2 3 3 3 3 2 2 2
1291 098/300 Read block 1668 len 12288
1292 099/300 Read block 4814 len 24576
1293 100/300 Read block 1695 len 12288
1294 101/300 Flush
1295 102/300 Read block 3280 len 36864
1296 103/300 Write block 2011 len 12288 data: 2 2 2
1297 104/300 Write block 572 len 16384 data: 2 2 2 2
1298 105/300 Read block 972 len 40960
1299 106/300 Read block 3347 len 8192
1300 107/300 Write block 2372 len 36864 data: 2 2 2 2 2 2 2 2 2
1301 108/300 Read block 2105 len 4096
1302 109/300 Flush
1303 110/300 Read block 3007 len 24576
1304 111/300 Read block 95 len 40960
1305 112/300 Write block 2881 len 24576 data: 2 2 2 2 2 2
1306 113/300 Write block 3523 len 20480 data: 2 2 2 2 2
1307 114/300 Flush
1308 115/300 Read block 4056 len 4096
1309 116/300 Write block 3082 len 24576 data: 3 3 2 2 2 2
1310 117/300 Write block 2964 len 4096 data: 2
1311 118/300 Read block 3923 len 12288
1312 119/300 Write block 580 len 32768 data: 2 2 2 2 2 2 2 2
1313 120/300 Read block 4206 len 32768
1314 121/300 Write block 4651 len 16384 data: 2 2 2 2
1315 122/300 Read block 2701 len 36864
1316 123/300 Write block 1481 len 28672 data: 2 2 2 2 2 2 2
1317 124/300 Write block 4181 len 24576 data: 2 2 2 2 2 2
1318 125/300 Read block 2321 len 32768
1319 126/300 Read block 2687 len 40960
1320 127/300 Read block 1054 len 8192
1321 128/300 Write block 4975 len 28672 data: 2 2 2 2 2 2 2
1322 129/300 Read block 2638 len 8192
1323 130/300 Read block 911 len 8192
1324 131/300 Write block 930 len 32768 data: 2 2 2 2 3 3 3 3
1325 132/300 Read block 1347 len 32768
1326 133/300 Read block 4925 len 24576
1327 134/300 Read block 4980 len 8192
1328 135/300 Read block 3316 len 32768
1329 136/300 Read block 4454 len 8192
1330 137/300 Read block 3765 len 32768
1331 138/300 Read block 2047 len 20480
1332 139/300 Write block 454 len 20480 data: 2 2 2 2 2
1333 140/300 Write block 1473 len 12288 data: 2 2 2
1334 141/300 Write block 1554 len 28672 data: 2 2 2 2 2 2 2
1335 142/300 Read block 2197 len 20480
1336 143/300 Read block 1670 len 16384
1337 144/300 Read block 4240 len 12288
1338 145/300 Read block 2419 len 28672
1339 146/300 Write block 1051 len 36864 data: 3 3 3 3 3 3 3 3 3
1340 147/300 Write block 2204 len 16384 data: 2 2 2 2
1341 148/300 Read block 2348 len 36864
1342 149/300 Write block 3192 len 40960 data: 3 2 2 2 2 2 2 3 3 3
1343 150/300 Read block 4461 len 32768
1344 151/300 Write block 3838 len 24576 data: 2 3 3 3 3 3
1345 152/300 Write block 1128 len 4096 data: 2
1346 153/300 Read block 727 len 28672
1347 154/300 Read block 4884 len 40960
1348 155/300 Read block 2520 len 24576
1349 156/300 Write block 2911 len 16384 data: 2 2 2 2
1350 157/300 Write block 1787 len 36864 data: 3 3 3 3 3 3 3 2 3
1351 158/300 Read block 1590 len 36864
1352 159/300 Read block 193 len 4096
1353 160/300 Read block 2175 len 32768
1354 161/300 Write block 1084 len 40960 data: 2 2 2 2 2 2 2 2 3 3
1355 162/300 Write block 2599 len 36864 data: 3 2 2 2 2 2 2 2 2
1356 163/300 Read block 1270 len 8192
1357 164/300 Write block 3862 len 36864 data: 2 2 2 2 2 2 2 3 3
1358 165/300 Read block 1926 len 28672
1359 166/300 Write block 145 len 20480 data: 2 2 2 2 2
1360 167/300 Read block 1776 len 40960
1361 168/300 Write block 3761 len 24576 data: 2 2 2 2 2 2
1362 169/300 Read block 763 len 4096
1363 170/300 Write block 2310 len 36864 data: 3 3 2 2 2 2 2 2 2
1364 171/300 Read block 1243 len 4096
1365 172/300 Read block 1013 len 28672
1366 173/300 Read block 977 len 36864
1367 174/300 Read block 1345 len 20480
1368 175/300 Write block 2015 len 24576 data: 2 2 2 2 2 2
1369 176/300 Write block 2371 len 12288 data: 2 3 3
1370 177/300 Write block 1886 len 24576 data: 3 3 3 2 2 2
1371 178/300 Read block 1042 len 28672
1372 179/300 Read block 1138 len 24576
1373 180/300 Read block 3317 len 36864
1374 181/300 Flush
1375 182/300 Write block 2786 len 8192 data: 2 2
1376 183/300 Read block 3164 len 24576
1377 184/300 Write block 640 len 24576 data: 2 2 2 2 2 2
1378 185/300 Flush
1379 186/300 Write block 1548 len 4096 data: 2
1380 187/300 Write block 1355 len 16384 data: 2 2 2 3
1381 188/300 Flush
1382 189/300 Read block 2961 len 16384
1383 190/300 Read block 322 len 36864
1384 191/300 Write block 1195 len 12288 data: 2 2 2
1385 192/300 Read block 1343 len 16384
1386 193/300 Write block 989 len 28672 data: 2 2 2 2 2 2 2
1387 194/300 Read block 4808 len 28672
1388 195/300 Read block 2795 len 32768
1389 196/300 Read block 3988 len 4096
1390 197/300 Write block 4870 len 24576 data: 2 2 2 2 2 2
1391 198/300 Write block 813 len 20480 data: 2 2 2 2 2
1392 199/300 Read block 4966 len 4096
1393 200/300 Write block 96 len 36864 data: 2 2 2 2 2 2 2 3 3
1394 201/300 Write block 4704 len 12288 data: 2 2 2
1395 202/300 Flush
1396 203/300 Write block 1544 len 20480 data: 2 2 2 2 3
1397 204/300 Read block 1653 len 4096
1398 205/300 Read block 2167 len 24576
1399 206/300 Read block 1768 len 40960
1400 207/300 Read block 1086 len 20480
1401 208/300 Read block 688 len 28672
1402 209/300 Write block 4635 len 20480 data: 2 2 2 2 2
1403 210/300 Read block 4002 len 40960
1404 211/300 Read block 493 len 20480
1405 212/300 Write block 531 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1406 213/300 Read block 4003 len 8192
1407 214/300 Read block 3031 len 8192
1408 215/300 Read block 1037 len 24576
1409 216/300 Read block 2640 len 4096
1410 217/300 Read block 3357 len 12288
1411 218/300 Read block 2510 len 16384
1412 219/300 Read block 2550 len 40960
1413 220/300 Write block 955 len 32768 data: 2 2 2 2 2 2 2 3
1414 221/300 Read block 1433 len 12288
1415 222/300 Read block 3108 len 16384
1416 223/300 Read block 1096 len 24576
1417 224/300 Read block 2388 len 4096
1418 225/300 Read block 2628 len 28672
1419 226/300 Write block 1026 len 20480 data: 2 2 2 2 2
1420 227/300 Read block 682 len 12288
1421 228/300 Flush
1422 229/300 Read block 1866 len 36864
1423 230/300 Read block 1481 len 8192
1424 231/300 Flush
1425 232/300 Read block 4946 len 12288
1426 233/300 Read block 2618 len 24576
1427 234/300 Read block 3876 len 20480
1428 235/300 Write block 2872 len 40960 data: 2 2 2 2 2 2 2 2 2 3
1429 236/300 Write block 1518 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1430 237/300 Write block 3902 len 16384 data: 2 2 2 2
1431 238/300 Flush
1432 239/300 Write block 206 len 32768 data: 2 2 3 3 3 3 3 3
1433 240/300 Flush
1434 241/300 Write block 3314 len 24576 data: 2 2 2 2 2 2
1435 242/300 Flush
1436 243/300 Write block 4371 len 32768 data: 2 2 2 2 2 2 2 2
1437 244/300 Write block 4173 len 28672 data: 3 4 4 3 3 3 2
1438 245/300 Write block 3028 len 4096 data: 2
1439 246/300 Write block 270 len 24576 data: 2 2 3 3 2 2
1440 247/300 Read block 4692 len 4096
1441 248/300 Read block 4766 len 20480
1442 249/300 Read block 4847 len 4096
1443 250/300 Read block 4055 len 20480
1444 251/300 Read block 313 len 8192
1445 252/300 Read block 4097 len 40960
1446 253/300 Write block 1511 len 4096 data: 2
1447 254/300 Write block 2294 len 28672 data: 2 2 2 2 2 2 2
1448 255/300 Write block 4077 len 32768 data: 2 2 2 2 2 2 2 2
1449 256/300 Read block 4315 len 36864
1450 257/300 Write block 2844 len 12288 data: 2 2 2
1451 258/300 Write block 1861 len 16384 data: 2 2 2 2
1452 259/300 Read block 1634 len 32768
1453 260/300 Read block 1530 len 12288
1454 261/300 Write block 4273 len 32768 data: 2 2 2 2 3 3 2 2
1455 262/300 Write block 4004 len 24576 data: 3 3 3 3 3 3
1456 263/300 Read block 3902 len 24576
1457 264/300 Read block 3220 len 4096
1458 265/300 Flush
1459 266/300 Write block 4860 len 4096 data: 2
1460 267/300 Write block 3616 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1461 268/300 Flush
1462 269/300 Flush
1463 270/300 Read block 139 len 12288
1464 271/300 Read block 2484 len 8192
1465 272/300 Read block 485 len 28672
1466 273/300 Read block 1273 len 36864
1467 274/300 Write block 3771 len 4096 data: 2
1468 275/300 Read block 4085 len 28672
1469 276/300 Write block 1900 len 20480 data: 2 2 2 2 2
1470 277/300 Read block 132 len 16384
1471 278/300 Write block 2344 len 20480 data: 2 2 2 2 2
1472 279/300 Write block 313 len 28672 data: 2 2 2 2 2 2 2
1473 280/300 Read block 2224 len 4096
1474 281/300 Flush
1475 282/300 Read block 4096 len 32768
1476 283/300 Read block 4206 len 40960
1477 284/300 Write block 1958 len 12288 data: 2 2 2
1478 285/300 Read block 854 len 36864
1479 286/300 Read block 4378 len 4096
1480 287/300 Read block 1972 len 20480
1481 288/300 Flush
1482 289/300 Read block 1506 len 32768
1483 290/300 Read block 1901 len 40960
1484 291/300 Write block 2512 len 20480 data: 2 2 2 2 2
1485 292/300 Write block 418 len 24576 data: 2 2 2 3 3 4
1486 293/300 Write block 2396 len 28672 data: 2 2 2 2 2 2 2
1487 294/300 Write block 1867 len 4096 data: 2
1488 295/300 Read block 1051 len 28672
1489 296/300 Write block 4802 len 36864 data: 2 2 2 2 2 2 2 2 2
1490 297/300 Read block 130 len 12288
1491 298/300 Read block 4789 len 40960
1492 299/300 Write block 729 len 40960 data: 2 2 2 2 2 2 2 2 3 2
1493 300/300 Write block 3766 len 8192 data: 3 2
1494 301/300 Read block 2897 len 4096
1495 Main task has work to do, go find it
1496 got dsc Start(1)
1497 start 1
1498 Jan 17 21:11:24.839 INFO request completed, latency_us: 205, response_code: 204, uri: /start/cid/1, method: POST, req_id: 29eac573-bec2-43ca-9eba-f6fbdbd6b0be, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
1499 Replay: started 1, returned:Ok(())
1500 ----------------------------------------------------------------
1501 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
1502 States: Active Offline Active
1503 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1504 1302 Acked Read 5 Done Sent Done false
1505 1303 Acked Write 3 Done Sent Done false
1506 1304 Acked Read 9 Done Sent Done false
1507 1305 Acked Read 6 Done Sent Done false
1508 1306 Acked Flush 0 Done Sent Done false
1509 1307 Acked Read 8 Done Sent Done false
1510 1308 Acked Read 4 Done Sent Done false
1511 1309 Acked Read 10 Done Sent Done false
1512 1310 Acked Read 3 Done Sent Done false
1513 1311 Acked Write 7 Done Sent Done false
1514 1312 Acked Write 2 Done Sent Done false
1515 1313 Acked Write 8 Done Sent Done false
1516 1314 Acked Read 9 Done Sent Done false
1517 1315 Acked Read 9 Done Sent Done false
1518 1316 Acked Write 4 Done Sent Done false
1519 1317 Acked Write 4 Done Sent Done false
1520 1318 Acked Write 6 Done Sent Done false
1521 1319 Acked Read 7 Done Sent Done false
1522 1320 Acked Read 2 Done Sent Done false
1523 1321 Acked Read 10 Done Sent Done false
1524 1322 Acked Read 9 Done Sent Done false
1525 1323 Acked Read 1 Done Sent Done false
1526 1324 Acked Read 3 Done Sent Done false
1527 1325 Acked Read 7 Done Sent Done false
1528 1326 Acked Write 3 Done Sent Done false
1529 1327 Acked Read 3 Done Sent Done false
1530 1328 Acked Write 9 Done Sent Done false
1531 1329 Acked Read 2 Done Sent Done false
1532 1330 Acked Read 10 Done Sent Done false
1533 1331 Acked Flush 0 Done Sent Done false
1534 1332 Acked Read 4 Done Sent Done false
1535 1333 Acked Write 5 Done Sent Done false
1536 1334 Acked Read 7 Done Sent Done false
1537 1335 Acked Write 3 Done Sent Done false
1538 1336 Acked Write 1 Done Sent Done false
1539 1337 Acked Write 10 Done Sent Done false
1540 1338 Acked Read 4 Done Sent Done false
1541 1339 Acked Write 7 Done Sent Done false
1542 1340 Acked Read 8 Done Sent Done false
1543 1341 Acked Read 1 Done Sent Done false
1544 1342 Acked Write 3 Done Sent Done false
1545 1343 Acked Write 4 Done Sent Done false
1546 1344 Acked Write 6 Done Sent Done false
1547 1345 Acked Read 3 Done Sent Done false
1548 1346 Acked Read 1 Done Sent Done false
1549 1347 Acked Write 4 Done Sent Done false
1550 1348 Acked Write 6 Done Sent Done false
1551 1349 Acked Read 9 Done Sent Done false
1552 1350 Acked Read 1 Done Sent Done false
1553 1351 Acked Write 4 Done Sent Done false
1554 1352 Acked Write 6 Done Sent Done false
1555 1353 Acked Write 1 Done Sent Done false
1556 1354 Acked Flush 0 Done Sent Done false
1557 1355 Acked Read 8 Done Sent Done false
1558 1356 Acked Write 9 Done Sent Done false
1559 1357 Acked Read 1 Done Sent Done false
1560 1358 Acked Write 5 Done Sent Done false
1561 1359 Acked Flush 0 Done Sent Done false
1562 1360 Acked Flush 0 Done Sent Done false
1563 1361 Acked Read 1 Done Sent Done false
1564 1362 Acked Write 1 Done Sent Done false
1565 1363 Acked Read 5 Done Sent Done false
1566 1364 Acked Read 8 Done Sent Done false
1567 1365 Acked Write 5 Done Sent Done false
1568 1366 Acked Write 3 Done Sent Done false
1569 1367 Acked Write 9 Done Sent Done false
1570 1368 Acked Read 1 Done Sent Done false
1571 1369 Acked Write 5 Done Sent Done false
1572 1370 Acked Write 10 Done Sent Done false
1573 1371 Acked Read 10 Done Sent Done false
1574 1372 Acked Read 5 Done Sent Done false
1575 1373 Acked Read 7 Done Sent Done false
1576 1374 Acked Read 3 Done Sent Done false
1577 1375 Acked Read 3 Done Sent Done false
1578 1376 Acked Write 2 Done Sent Done false
1579 1377 Acked Read 2 Done Sent Done false
1580 1378 Acked Read 10 Done Sent Done false
1581 1379 Acked Read 1 Done Sent Done false
1582 1380 Acked Read 7 Done Sent Done false
1583 1381 Acked Write 2 Done Sent Done false
1584 1382 Acked Write 7 Done Sent Done false
1585 1383 Acked Write 2 Done Sent Done false
1586 1384 Acked Write 7 Done Sent Done false
1587 1385 Acked Read 5 Done Sent Done false
1588 1386 Acked Write 7 Done Sent Done false
1589 1387 Acked Read 10 Done Sent Done false
1590 1388 Acked Write 7 Done Sent Done false
1591 1389 Acked Read 7 Done Sent Done false
1592 1390 Acked Read 5 Done Sent Done false
1593 1391 Acked Write 5 Done Sent Done false
1594 1392 Acked Flush 0 Done Sent Done false
1595 1393 Acked Write 5 Done Sent Done false
1596 1394 Acked Write 7 Done Sent Done false
1597 1395 Acked Read 6 Done Sent Done false
1598 1396 Acked Write 6 Done Sent Done false
1599 1397 Acked Read 1 Done Sent Done false
1600 1398 Acked Write 8 Done Sent Done false
1601 1399 Acked Read 3 Done Sent Done false
1602 1400 Acked Read 6 Done Sent Done false
1603 1401 Acked Read 3 Done Sent Done false
1604 1402 Acked Flush 0 Done Sent Done false
1605 1403 Acked Read 9 Done Sent Done false
1606 1404 Acked Write 3 Done Sent Done false
1607 1405 Acked Write 4 Done Sent Done false
1608 1406 Acked Read 10 Done Sent Done false
1609 1407 Acked Read 2 Done Sent Done false
1610 1408 Acked Write 9 Done Sent Done false
1611 1409 Acked Read 1 Done Sent Done false
1612 1410 Acked Flush 0 Done Sent Done false
1613 1411 Acked Read 6 Done Sent Done false
1614 1412 Acked Read 10 Done Sent Done false
1615 1413 Acked Write 6 Done Sent Done false
1616 1414 Acked Write 5 Done Sent Done false
1617 1415 Acked Flush 0 Done Sent Done false
1618 1416 Acked Read 1 Done Sent Done false
1619 1417 Acked Write 6 Done Sent Done false
1620 1418 Acked Write 1 Done Sent Done false
1621 1419 Acked Read 3 Done Sent Done false
1622 1420 Acked Write 8 Done Sent Done false
1623 1421 Acked Read 8 Done Sent Done false
1624 1422 Acked Write 4 Done Sent Done false
1625 1423 Acked Read 9 Done Sent Done false
1626 1424 Acked Write 7 Done Sent Done false
1627 1425 Acked Write 6 Done Sent Done false
1628 1426 Acked Read 8 Done Sent Done false
1629 1427 Acked Read 10 Done Sent Done false
1630 1428 Acked Read 2 Done Sent Done false
1631 1429 Acked Write 7 Done Sent Done false
1632 1430 Acked Read 2 Done Sent Done false
1633 1431 Acked Read 2 Done Sent Done false
1634 1432 Acked Write 8 Done Sent Done false
1635 1433 Acked Read 8 Done Sent Done false
1636 1434 Acked Read 6 Done Sent Done false
1637 1435 Acked Read 2 Done Sent Done false
1638 1436 Acked Read 8 Done Sent Done false
1639 1437 Acked Read 2 Done Sent Done false
1640 1438 Acked Read 8 Done Sent Done false
1641 1439 Acked Read 5 Done Sent Done false
1642 1440 Acked Write 5 Done Sent Done false
1643 1441 Acked Write 3 Done Sent Done false
1644 1442 Acked Write 7 Done Sent Done false
1645 1443 Acked Read 5 Done Sent Done false
1646 1444 Acked Read 4 Done Sent Done false
1647 1445 Acked Read 3 Done Sent Done false
1648 1446 Acked Read 7 Done Sent Done false
1649 1447 Acked Write 9 Done Sent Done false
1650 1448 Acked Write 4 Done Sent Done false
1651 1449 Acked Read 9 Done Sent Done false
1652 1450 Acked Write 10 Done Sent Done false
1653 1451 Acked Read 8 Done Sent Done false
1654 1452 Acked Write 6 Done Sent Done false
1655 1453 Acked Write 1 Done Sent Done false
1656 1454 Acked Read 7 Done Sent Done false
1657 1455 Acked Read 10 Done Sent Done false
1658 1456 Acked Read 6 Done Sent Done false
1659 1457 Acked Write 4 Done Sent Done false
1660 1458 Acked Write 9 Done Sent Done false
1661 1459 Acked Read 9 Done Sent Done false
1662 1460 Acked Read 1 Done Sent Done false
1663 1461 Acked Read 8 Done Sent Done false
1664 1462 Acked Write 10 Done Sent Done false
1665 1463 Acked Write 9 Done Sent Done false
1666 1464 Acked Read 2 Done Sent Done false
1667 1465 Acked Write 9 Done Sent Done false
1668 1466 Acked Read 7 Done Sent Done false
1669 1467 Acked Write 5 Done Sent Done false
1670 1468 Acked Read 10 Done Sent Done false
1671 1469 Acked Write 6 Done Sent Done false
1672 1470 Acked Read 1 Done Sent Done false
1673 1471 Acked Write 9 Done Sent Done false
1674 1472 Acked Read 1 Done Sent Done false
1675 1473 Acked Read 7 Done Sent Done false
1676 1474 Acked Read 9 Done Sent Done false
1677 1475 Acked Read 5 Done Sent Done false
1678 1476 Acked Write 6 Done Sent Done false
1679 1477 Acked Write 3 Done Sent Done false
1680 1478 Acked Write 6 Done Sent Done false
1681 1479 Acked Read 7 Done Sent Done false
1682 1480 Acked Read 6 Done Sent Done false
1683 1481 Acked Read 9 Done Sent Done false
1684 1482 Acked Flush 0 Done Sent Done false
1685 1483 Acked Write 2 Done Sent Done false
1686 1484 Acked Read 6 Done Sent Done false
1687 1485 Acked Write 6 Done Sent Done false
1688 1486 Acked Flush 0 Done Sent Done false
1689 1487 Acked Write 1 Done Sent Done false
1690 1488 Acked Write 4 Done Sent Done false
1691 1489 Acked Flush 0 Done Sent Done false
1692 1490 Acked Read 4 Done Sent Done false
1693 1491 Acked Read 9 Done Sent Done false
1694 1492 Acked Write 3 Done Sent Done false
1695 1493 Acked Read 4 Done Sent Done false
1696 1494 Acked Write 7 Done Sent Done false
1697 1495 Acked Read 7 Done Sent Done false
1698 1496 Acked Read 8 Done Sent Done false
1699 1497 Acked Read 1 Done Sent Done false
1700 1498 Acked Write 6 Done Sent Done false
1701 1499 Acked Write 5 Done Sent Done false
1702 1500 Acked Read 1 Done Sent Done false
1703 1501 Acked Write 9 Done Sent Done false
1704 1502 Acked Write 3 Done Sent Done false
1705 1503 Acked Flush 0 Done Sent Done false
1706 1504 Acked Write 5 Done Sent Done false
1707 1505 Acked Read 1 Done Sent Done false
1708 1506 Acked Read 6 Done Sent Done false
1709 1507 Acked Read 10 Done Sent Done false
1710 1508 Acked Read 5 Done Sent Done false
1711 1509 Acked Read 7 Done Sent Done false
1712 1510 Acked Write 5 Done Sent Done false
1713 1511 Acked Read 10 Done Sent Done false
1714 1512 Acked Read 5 Done Sent Done false
1715 1513 Acked Write 10 Done Sent Done false
1716 1514 Acked Read 2 Done Sent Done false
1717 1515 Acked Read 2 Done Sent Done false
1718 1516 Acked Read 6 Done Sent Done false
1719 1517 Acked Read 1 Done Sent Done false
1720 1518 Acked Read 3 Done Sent Done false
1721 1519 Acked Read 4 Done Sent Done false
1722 1520 Acked Read 10 Done Sent Done false
1723 1521 Acked Write 8 Done Sent Done false
1724 1522 Acked Read 3 Done Sent Done false
1725 1523 Acked Read 4 Done Sent Done false
1726 1524 Acked Read 6 Done Sent Done false
1727 1525 Acked Read 1 Done Sent Done false
1728 1526 Acked Read 7 Done Sent Done false
1729 1527 Acked Write 5 Done Sent Done false
1730 1528 Acked Read 3 Done Sent Done false
1731 1529 Acked Flush 0 Done Sent Done false
1732 1530 Acked Read 9 Done Sent Done false
1733 1531 Acked Read 2 Done Sent Done false
1734 1532 Acked Flush 0 Done Sent Done false
1735 1533 Acked Read 3 Done Sent Done false
1736 1534 Acked Read 6 Done Sent Done false
1737 1535 Acked Read 5 Done Sent Done false
1738 1536 Acked Write 10 Done Sent Done false
1739 1537 Acked Write 10 Done Sent Done false
1740 1538 Acked Write 4 Done Sent Done false
1741 1539 Acked Flush 0 Done Sent Done false
1742 1540 Acked Write 8 Done Sent Done false
1743 1541 Acked Flush 0 Done Sent Done false
1744 1542 Acked Write 6 Done Sent Done false
1745 1543 Acked Flush 0 Done Sent Done false
1746 1544 Acked Write 8 Done Sent Done false
1747 1545 Acked Write 7 Done Sent Done false
1748 1546 Acked Write 1 Done Sent Done false
1749 1547 Acked Write 6 Done Sent Done false
1750 1548 Acked Read 1 Done Sent Done false
1751 1549 Acked Read 5 Done Sent Done false
1752 1550 Acked Read 1 Done Sent Done false
1753 1551 Acked Read 5 Done Sent Done false
1754 1552 Acked Read 2 Done Sent Done false
1755 1553 Acked Read 10 Done Sent Done false
1756 1554 Acked Write 1 Done Sent Done false
1757 1555 Acked Write 7 Done Sent Done false
1758 1556 Acked Write 8 Done Sent Done false
1759 1557 Acked Read 9 Done Sent Done false
1760 1558 Acked Write 3 Done Sent Done false
1761 1559 Acked Write 4 Done Sent Done false
1762 1560 Acked Read 8 Done Sent Done false
1763 1561 Acked Read 3 Done Sent Done false
1764 1562 Acked Write 8 Done Sent Done false
1765 1563 Acked Write 6 Done Sent Done false
1766 1564 Acked Read 6 Done Sent Done false
1767 1565 Acked Read 1 Done Sent Done false
1768 1566 Acked Flush 0 Done Sent Done false
1769 1567 Acked Write 1 Done Sent Done false
1770 1568 Acked Write 10 Done Sent Done false
1771 1569 Acked Flush 0 Done Sent Done false
1772 1570 Acked Flush 0 Done Sent Done false
1773 1571 Acked Read 3 Done Sent Done false
1774 1572 Acked Read 2 Done Sent Done false
1775 1573 Acked Read 7 Done Sent Done false
1776 1574 Acked Read 9 Done Sent Done false
1777 1575 Acked Write 1 Done Sent Done false
1778 1576 Acked Read 7 Done Sent Done false
1779 1577 Acked Write 5 Done Sent Done false
1780 1578 Acked Read 4 Done Sent Done false
1781 1579 Acked Write 5 Done Sent Done false
1782 1580 Acked Write 7 Done Sent Done false
1783 1581 Acked Read 1 Done Sent Done false
1784 1582 Acked Flush 0 Done Sent Done false
1785 1583 Acked Read 8 Done Sent Done false
1786 1584 Acked Read 10 Done Sent Done false
1787 1585 Acked Write 3 Done Sent Done false
1788 1586 Acked Read 9 Done Sent Done false
1789 1587 Acked Read 1 Done Sent Done false
1790 1588 Acked Read 5 Done Sent Done false
1791 1589 Acked Flush 0 Done Sent Done false
1792 1590 Acked Read 8 Done Sent Done false
1793 1591 Acked Read 10 Done Sent Done false
1794 1592 Acked Write 5 Done Sent Done false
1795 1593 Acked Write 6 Done Sent Done false
1796 1594 Acked Write 7 Done Sent Done false
1797 1595 Acked Write 1 Done Sent Done false
1798 1596 Acked Read 7 Done Sent Done false
1799 1597 Acked Write 9 Done Sent Done false
1800 1598 Acked Read 3 Done Sent Done false
1801 1599 Acked Read 10 Done Sent Done false
1802 1600 Acked Write 10 Done Sent Done false
1803 1601 Acked Write 2 Done Sent Done false
1804 1602 Acked Read 1 Done Sent Done false
1805 STATES DS:0 DS:1 DS:2 TOTAL
1806 Sent 0 301 0 301
1807 Done 301 0 301 602
1808 Skipped 0 0 0 0
1809 Error 0 0 0 0
1810 Last Flush: 1589 1301 1589
1811 Downstairs last five completed: 1301 1300 1299 1298 1297
1812 Upstairs last five completed: 1602 1601 1600 1599 1598
1813 CLIENT: Up:0 ds:301 act:2
1814 [8820] got start action
1815 [8820] I am going to restart
1816 Starting downstairs at port 8820
1817 Make output file at "/tmp/dsc/downstairs-8820.txt"
1818 [8820][1] reports Starting
1819 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1202)
1820 [8820][1] reports Running
1821 ----------------------------------------------------------------
1822 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
1823 States: Active Offline Active
1824 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1825 1302 Acked Read 5 Done Sent Done false
1826 1303 Acked Write 3 Done Sent Done false
1827 1304 Acked Read 9 Done Sent Done false
1828 1305 Acked Read 6 Done Sent Done false
1829 1306 Acked Flush 0 Done Sent Done false
1830 1307 Acked Read 8 Done Sent Done false
1831 1308 Acked Read 4 Done Sent Done false
1832 1309 Acked Read 10 Done Sent Done false
1833 1310 Acked Read 3 Done Sent Done false
1834 1311 Acked Write 7 Done Sent Done false
1835 1312 Acked Write 2 Done Sent Done false
1836 1313 Acked Write 8 Done Sent Done false
1837 1314 Acked Read 9 Done Sent Done false
1838 1315 Acked Read 9 Done Sent Done false
1839 1316 Acked Write 4 Done Sent Done false
1840 1317 Acked Write 4 Done Sent Done false
1841 1318 Acked Write 6 Done Sent Done false
1842 1319 Acked Read 7 Done Sent Done false
1843 1320 Acked Read 2 Done Sent Done false
1844 1321 Acked Read 10 Done Sent Done false
1845 1322 Acked Read 9 Done Sent Done false
1846 1323 Acked Read 1 Done Sent Done false
1847 1324 Acked Read 3 Done Sent Done false
1848 1325 Acked Read 7 Done Sent Done false
1849 1326 Acked Write 3 Done Sent Done false
1850 1327 Acked Read 3 Done Sent Done false
1851 1328 Acked Write 9 Done Sent Done false
1852 1329 Acked Read 2 Done Sent Done false
1853 1330 Acked Read 10 Done Sent Done false
1854 1331 Acked Flush 0 Done Sent Done false
1855 1332 Acked Read 4 Done Sent Done false
1856 1333 Acked Write 5 Done Sent Done false
1857 1334 Acked Read 7 Done Sent Done false
1858 1335 Acked Write 3 Done Sent Done false
1859 1336 Acked Write 1 Done Sent Done false
1860 1337 Acked Write 10 Done Sent Done false
1861 1338 Acked Read 4 Done Sent Done false
1862 1339 Acked Write 7 Done Sent Done false
1863 1340 Acked Read 8 Done Sent Done false
1864 1341 Acked Read 1 Done Sent Done false
1865 1342 Acked Write 3 Done Sent Done false
1866 1343 Acked Write 4 Done Sent Done false
1867 1344 Acked Write 6 Done Sent Done false
1868 1345 Acked Read 3 Done Sent Done false
1869 1346 Acked Read 1 Done Sent Done false
1870 1347 Acked Write 4 Done Sent Done false
1871 1348 Acked Write 6 Done Sent Done false
1872 1349 Acked Read 9 Done Sent Done false
1873 1350 Acked Read 1 Done Sent Done false
1874 1351 Acked Write 4 Done Sent Done false
1875 1352 Acked Write 6 Done Sent Done false
1876 1353 Acked Write 1 Done Sent Done false
1877 1354 Acked Flush 0 Done Sent Done false
1878 1355 Acked Read 8 Done Sent Done false
1879 1356 Acked Write 9 Done Sent Done false
1880 1357 Acked Read 1 Done Sent Done false
1881 1358 Acked Write 5 Done Sent Done false
1882 1359 Acked Flush 0 Done Sent Done false
1883 1360 Acked Flush 0 Done Sent Done false
1884 1361 Acked Read 1 Done Sent Done false
1885 1362 Acked Write 1 Done Sent Done false
1886 1363 Acked Read 5 Done Sent Done false
1887 1364 Acked Read 8 Done Sent Done false
1888 1365 Acked Write 5 Done Sent Done false
1889 1366 Acked Write 3 Done Sent Done false
1890 1367 Acked Write 9 Done Sent Done false
1891 1368 Acked Read 1 Done Sent Done false
1892 1369 Acked Write 5 Done Sent Done false
1893 1370 Acked Write 10 Done Sent Done false
1894 1371 Acked Read 10 Done Sent Done false
1895 1372 Acked Read 5 Done Sent Done false
1896 1373 Acked Read 7 Done Sent Done false
1897 1374 Acked Read 3 Done Sent Done false
1898 1375 Acked Read 3 Done Sent Done false
1899 1376 Acked Write 2 Done Sent Done false
1900 1377 Acked Read 2 Done Sent Done false
1901 1378 Acked Read 10 Done Sent Done false
1902 1379 Acked Read 1 Done Sent Done false
1903 1380 Acked Read 7 Done Sent Done false
1904 1381 Acked Write 2 Done Sent Done false
1905 1382 Acked Write 7 Done Sent Done false
1906 1383 Acked Write 2 Done Sent Done false
1907 1384 Acked Write 7 Done Sent Done false
1908 1385 Acked Read 5 Done Sent Done false
1909 1386 Acked Write 7 Done Sent Done false
1910 1387 Acked Read 10 Done Sent Done false
1911 1388 Acked Write 7 Done Sent Done false
1912 1389 Acked Read 7 Done Sent Done false
1913 1390 Acked Read 5 Done Sent Done false
1914 1391 Acked Write 5 Done Sent Done false
1915 1392 Acked Flush 0 Done Sent Done false
1916 1393 Acked Write 5 Done Sent Done false
1917 1394 Acked Write 7 Done Sent Done false
1918 1395 Acked Read 6 Done Sent Done false
1919 1396 Acked Write 6 Done Sent Done false
1920 1397 Acked Read 1 Done Sent Done false
1921 1398 Acked Write 8 Done Sent Done false
1922 1399 Acked Read 3 Done Sent Done false
1923 1400 Acked Read 6 Done Sent Done false
1924 1401 Acked Read 3 Done Sent Done false
1925 1402 Acked Flush 0 Done Sent Done false
1926 1403 Acked Read 9 Done Sent Done false
1927 1404 Acked Write 3 Done Sent Done false
1928 1405 Acked Write 4 Done Sent Done false
1929 1406 Acked Read 10 Done Sent Done false
1930 1407 Acked Read 2 Done Sent Done false
1931 1408 Acked Write 9 Done Sent Done false
1932 1409 Acked Read 1 Done Sent Done false
1933 1410 Acked Flush 0 Done Sent Done false
1934 1411 Acked Read 6 Done Sent Done false
1935 1412 Acked Read 10 Done Sent Done false
1936 1413 Acked Write 6 Done Sent Done false
1937 1414 Acked Write 5 Done Sent Done false
1938 1415 Acked Flush 0 Done Sent Done false
1939 1416 Acked Read 1 Done Sent Done false
1940 1417 Acked Write 6 Done Sent Done false
1941 1418 Acked Write 1 Done Sent Done false
1942 1419 Acked Read 3 Done Sent Done false
1943 1420 Acked Write 8 Done Sent Done false
1944 1421 Acked Read 8 Done Sent Done false
1945 1422 Acked Write 4 Done Sent Done false
1946 1423 Acked Read 9 Done Sent Done false
1947 1424 Acked Write 7 Done Sent Done false
1948 1425 Acked Write 6 Done Sent Done false
1949 1426 Acked Read 8 Done Sent Done false
1950 1427 Acked Read 10 Done Sent Done false
1951 1428 Acked Read 2 Done Sent Done false
1952 1429 Acked Write 7 Done Sent Done false
1953 1430 Acked Read 2 Done Sent Done false
1954 1431 Acked Read 2 Done Sent Done false
1955 1432 Acked Write 8 Done Sent Done false
1956 1433 Acked Read 8 Done Sent Done false
1957 1434 Acked Read 6 Done Sent Done false
1958 1435 Acked Read 2 Done Sent Done false
1959 1436 Acked Read 8 Done Sent Done false
1960 1437 Acked Read 2 Done Sent Done false
1961 1438 Acked Read 8 Done Sent Done false
1962 1439 Acked Read 5 Done Sent Done false
1963 1440 Acked Write 5 Done Sent Done false
1964 1441 Acked Write 3 Done Sent Done false
1965 1442 Acked Write 7 Done Sent Done false
1966 1443 Acked Read 5 Done Sent Done false
1967 1444 Acked Read 4 Done Sent Done false
1968 1445 Acked Read 3 Done Sent Done false
1969 1446 Acked Read 7 Done Sent Done false
1970 1447 Acked Write 9 Done Sent Done false
1971 1448 Acked Write 4 Done Sent Done false
1972 1449 Acked Read 9 Done Sent Done false
1973 1450 Acked Write 10 Done Sent Done false
1974 1451 Acked Read 8 Done Sent Done false
1975 1452 Acked Write 6 Done Sent Done false
1976 1453 Acked Write 1 Done Sent Done false
1977 1454 Acked Read 7 Done Sent Done false
1978 1455 Acked Read 10 Done Sent Done false
1979 1456 Acked Read 6 Done Sent Done false
1980 1457 Acked Write 4 Done Sent Done false
1981 1458 Acked Write 9 Done Sent Done false
1982 1459 Acked Read 9 Done Sent Done false
1983 1460 Acked Read 1 Done Sent Done false
1984 1461 Acked Read 8 Done Sent Done false
1985 1462 Acked Write 10 Done Sent Done false
1986 1463 Acked Write 9 Done Sent Done false
1987 1464 Acked Read 2 Done Sent Done false
1988 1465 Acked Write 9 Done Sent Done false
1989 1466 Acked Read 7 Done Sent Done false
1990 1467 Acked Write 5 Done Sent Done false
1991 1468 Acked Read 10 Done Sent Done false
1992 1469 Acked Write 6 Done Sent Done false
1993 1470 Acked Read 1 Done Sent Done false
1994 1471 Acked Write 9 Done Sent Done false
1995 1472 Acked Read 1 Done Sent Done false
1996 1473 Acked Read 7 Done Sent Done false
1997 1474 Acked Read 9 Done Sent Done false
1998 1475 Acked Read 5 Done Sent Done false
1999 1476 Acked Write 6 Done Sent Done false
2000 1477 Acked Write 3 Done Sent Done false
2001 1478 Acked Write 6 Done Sent Done false
2002 1479 Acked Read 7 Done Sent Done false
2003 1480 Acked Read 6 Done Sent Done false
2004 1481 Acked Read 9 Done Sent Done false
2005 1482 Acked Flush 0 Done Sent Done false
2006 1483 Acked Write 2 Done Sent Done false
2007 1484 Acked Read 6 Done Sent Done false
2008 1485 Acked Write 6 Done Sent Done false
2009 1486 Acked Flush 0 Done Sent Done false
2010 1487 Acked Write 1 Done Sent Done false
2011 1488 Acked Write 4 Done Sent Done false
2012 1489 Acked Flush 0 Done Sent Done false
2013 1490 Acked Read 4 Done Sent Done false
2014 1491 Acked Read 9 Done Sent Done false
2015 1492 Acked Write 3 Done Sent Done false
2016 1493 Acked Read 4 Done Sent Done false
2017 1494 Acked Write 7 Done Sent Done false
2018 1495 Acked Read 7 Done Sent Done false
2019 1496 Acked Read 8 Done Sent Done false
2020 1497 Acked Read 1 Done Sent Done false
2021 1498 Acked Write 6 Done Sent Done false
2022 1499 Acked Write 5 Done Sent Done false
2023 1500 Acked Read 1 Done Sent Done false
2024 1501 Acked Write 9 Done Sent Done false
2025 1502 Acked Write 3 Done Sent Done false
2026 1503 Acked Flush 0 Done Sent Done false
2027 1504 Acked Write 5 Done Sent Done false
2028 1505 Acked Read 1 Done Sent Done false
2029 1506 Acked Read 6 Done Sent Done false
2030 1507 Acked Read 10 Done Sent Done false
2031 1508 Acked Read 5 Done Sent Done false
2032 1509 Acked Read 7 Done Sent Done false
2033 1510 Acked Write 5 Done Sent Done false
2034 1511 Acked Read 10 Done Sent Done false
2035 1512 Acked Read 5 Done Sent Done false
2036 1513 Acked Write 10 Done Sent Done false
2037 1514 Acked Read 2 Done Sent Done false
2038 1515 Acked Read 2 Done Sent Done false
2039 1516 Acked Read 6 Done Sent Done false
2040 1517 Acked Read 1 Done Sent Done false
2041 1518 Acked Read 3 Done Sent Done false
2042 1519 Acked Read 4 Done Sent Done false
2043 1520 Acked Read 10 Done Sent Done false
2044 1521 Acked Write 8 Done Sent Done false
2045 1522 Acked Read 3 Done Sent Done false
2046 1523 Acked Read 4 Done Sent Done false
2047 1524 Acked Read 6 Done Sent Done false
2048 1525 Acked Read 1 Done Sent Done false
2049 1526 Acked Read 7 Done Sent Done false
2050 1527 Acked Write 5 Done Sent Done false
2051 1528 Acked Read 3 Done Sent Done false
2052 1529 Acked Flush 0 Done Sent Done false
2053 1530 Acked Read 9 Done Sent Done false
2054 1531 Acked Read 2 Done Sent Done false
2055 1532 Acked Flush 0 Done Sent Done false
2056 1533 Acked Read 3 Done Sent Done false
2057 1534 Acked Read 6 Done Sent Done false
2058 1535 Acked Read 5 Done Sent Done false
2059 1536 Acked Write 10 Done Sent Done false
2060 1537 Acked Write 10 Done Sent Done false
2061 1538 Acked Write 4 Done Sent Done false
2062 1539 Acked Flush 0 Done Sent Done false
2063 1540 Acked Write 8 Done Sent Done false
2064 1541 Acked Flush 0 Done Sent Done false
2065 1542 Acked Write 6 Done Sent Done false
2066 1543 Acked Flush 0 Done Sent Done false
2067 1544 Acked Write 8 Done Sent Done false
2068 1545 Acked Write 7 Done Sent Done false
2069 1546 Acked Write 1 Done Sent Done false
2070 1547 Acked Write 6 Done Sent Done false
2071 1548 Acked Read 1 Done Sent Done false
2072 1549 Acked Read 5 Done Sent Done false
2073 1550 Acked Read 1 Done Sent Done false
2074 1551 Acked Read 5 Done Sent Done false
2075 1552 Acked Read 2 Done Sent Done false
2076 1553 Acked Read 10 Done Sent Done false
2077 1554 Acked Write 1 Done Sent Done false
2078 1555 Acked Write 7 Done Sent Done false
2079 1556 Acked Write 8 Done Sent Done false
2080 1557 Acked Read 9 Done Sent Done false
2081 1558 Acked Write 3 Done Sent Done false
2082 1559 Acked Write 4 Done Sent Done false
2083 1560 Acked Read 8 Done Sent Done false
2084 1561 Acked Read 3 Done Sent Done false
2085 1562 Acked Write 8 Done Sent Done false
2086 1563 Acked Write 6 Done Sent Done false
2087 1564 Acked Read 6 Done Sent Done false
2088 1565 Acked Read 1 Done Sent Done false
2089 1566 Acked Flush 0 Done Sent Done false
2090 1567 Acked Write 1 Done Sent Done false
2091 1568 Acked Write 10 Done Sent Done false
2092 1569 Acked Flush 0 Done Sent Done false
2093 1570 Acked Flush 0 Done Sent Done false
2094 1571 Acked Read 3 Done Sent Done false
2095 1572 Acked Read 2 Done Sent Done false
2096 1573 Acked Read 7 Done Sent Done false
2097 1574 Acked Read 9 Done Sent Done false
2098 1575 Acked Write 1 Done Sent Done false
2099 1576 Acked Read 7 Done Sent Done false
2100 1577 Acked Write 5 Done Sent Done false
2101 1578 Acked Read 4 Done Sent Done false
2102 1579 Acked Write 5 Done Sent Done false
2103 1580 Acked Write 7 Done Sent Done false
2104 1581 Acked Read 1 Done Sent Done false
2105 1582 Acked Flush 0 Done Sent Done false
2106 1583 Acked Read 8 Done Sent Done false
2107 1584 Acked Read 10 Done Sent Done false
2108 1585 Acked Write 3 Done Sent Done false
2109 1586 Acked Read 9 Done Sent Done false
2110 1587 Acked Read 1 Done Sent Done false
2111 1588 Acked Read 5 Done Sent Done false
2112 1589 Acked Flush 0 Done Sent Done false
2113 1590 Acked Read 8 Done Sent Done false
2114 1591 Acked Read 10 Done Sent Done false
2115 1592 Acked Write 5 Done Sent Done false
2116 1593 Acked Write 6 Done Sent Done false
2117 1594 Acked Write 7 Done Sent Done false
2118 1595 Acked Write 1 Done Sent Done false
2119 1596 Acked Read 7 Done Sent Done false
2120 1597 Acked Write 9 Done Sent Done false
2121 1598 Acked Read 3 Done Sent Done false
2122 1599 Acked Read 10 Done Sent Done false
2123 1600 Acked Write 10 Done Sent Done false
2124 1601 Acked Write 2 Done Sent Done false
2125 1602 Acked Read 1 Done Sent Done false
2126 1603 Acked Flush 0 Done Sent Done false
2127 STATES DS:0 DS:1 DS:2 TOTAL
2128 Sent 0 302 0 302
2129 Done 302 0 302 604
2130 Skipped 0 0 0 0
2131 Error 0 0 0 0
2132 Last Flush: 1603 1301 1603
2133 Downstairs last five completed: 1301 1300 1299 1298 1297
2134 Upstairs last five completed: 1603 1602 1601 1600 1599
2135 CLIENT: Up:0 ds:302 act:2
21362025-01-17T21:11:30.595ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
21372025-01-17T21:11:30.595ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:53254) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
21382025-01-17T21:11:30.596ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
21392025-01-17T21:11:30.597ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
21402025-01-17T21:11:30.597ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
21412025-01-17T21:11:30.597ZINFOcrucible: send last flush ID to this DS: 1301 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
21422025-01-17T21:11:30.597ZINFOcrucible: Replied this last flush ID: 1301 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
21432025-01-17T21:11:30.597ZINFOcrucible: [1] client re-new 302 jobs since flush 1301 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
21442025-01-17T21:11:30.598ZINFOcrucible: [1] Replayed 302 jobs since flush: 1301 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
2145 ----------------------------------------------------------------
2146 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
2147 Downstairs last five completed: 1603 1602 1601 1600 1599
2148 Upstairs last five completed: 1603 1602 1601 1600 1599
2149 CLIENT: Up:0 ds:0 act:3
2150 Replay: All jobs finished
2151 Main task has work to do, go find it
2152 got dsc Stop(2)
2153 stop 2
2154 [8830] Got stop action so:false kr:false
2155 Jan 17 21:11:32.843 INFO request completed, latency_us: 358, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 617872a1-ab87-425a-8aa4-8bf6833d99dd, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
21562025-01-17T21:11:32.844ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
21572025-01-17T21:11:32.844ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
21582025-01-17T21:11:32.844ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
2159 Jan 17 21:11:32.844 INFO request completed, latency_us: 151, response_code: 200, uri: /state/cid/2, method: GET, req_id: ad926074-89b2-4de6-affa-ed6218eb5055, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
21602025-01-17T21:11:32.844ZINFOcrucible: client task is exiting = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
2161 [8830] Exited with: ExitStatus(unix_wait_status(9))
2162 [8830][2] reports Exit
2163 Jan 17 21:11:36.847 INFO request completed, latency_us: 196, response_code: 200, uri: /state/cid/2, method: GET, req_id: 87560ca2-634f-4435-89c4-9c918f40a6b1, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
2164 001/300 Read block 1623 len 24576
2165 002/300 Write block 3539 len 28672 data: 2 2 2 2 2 2 2
2166 003/300 Write block 1709 len 16384 data: 2 2 2 2
2167 004/300 Read block 334 len 28672
2168 005/300 Write block 942 len 40960 data: 3 3 3 3 2 2 2 2 2 2
2169 006/300 Read block 1563 len 12288
2170 007/300 Read block 3968 len 40960
2171 008/300 Read block 3393 len 24576
2172 009/300 Read block 4656 len 4096
2173 010/300 Read block 1279 len 32768
2174 011/300 Write block 3223 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2175 012/300 Read block 2589 len 20480
2176 013/300 Write block 880 len 12288 data: 2 2 2
2177 014/300 Write block 789 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2178 015/300 Read block 2240 len 20480
2179 016/300 Read block 3214 len 4096
2180 017/300 Read block 454 len 16384
2181 018/300 Read block 2491 len 24576
2182 019/300 Write block 3807 len 36864 data: 2 2 2 2 2 2 2 2 2
2183 020/300 Write block 1307 len 36864 data: 2 2 2 2 2 2 2 2 2
2184 021/300 Read block 242 len 4096
2185 022/300 Write block 1064 len 4096 data: 2
2186 023/300 Read block 4099 len 20480
2187 024/300 Read block 2261 len 32768
2188 025/300 Write block 2249 len 40960 data: 2 2 2 2 2 2 2 2 2 3
2189 026/300 Read block 1428 len 32768
2190 027/300 Write block 2242 len 32768 data: 2 2 2 2 2 2 2 3
2191 028/300 Read block 2880 len 28672
2192 029/300 Write block 1453 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2193 030/300 Write block 1679 len 8192 data: 2 2
2194 031/300 Read block 123 len 20480
2195 032/300 Write block 4170 len 40960 data: 2 2 3 4 5 5 4 4 4 3
2196 033/300 Read block 568 len 4096
2197 034/300 Write block 69 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2198 035/300 Read block 4127 len 12288
2199 036/300 Read block 4274 len 16384
2200 037/300 Write block 784 len 20480 data: 2 2 2 2 2
2201 038/300 Write block 2477 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2202 039/300 Read block 2200 len 24576
2203 040/300 Read block 2082 len 40960
2204 041/300 Write block 4097 len 36864 data: 2 2 2 2 2 2 2 2 2
2205 042/300 Write block 1873 len 16384 data: 2 2 2 2
2206 043/300 Write block 4268 len 28672 data: 3 3 3 3 2 3 3
2207 044/300 Read block 2235 len 32768
2208 045/300 Read block 2716 len 32768
2209 046/300 Write block 3157 len 32768 data: 2 2 2 2 2 2 2 2
2210 047/300 Read block 102 len 28672
2211 048/300 Read block 771 len 36864
2212 049/300 Read block 3840 len 4096
2213 050/300 Read block 3271 len 36864
2214 051/300 Read block 1805 len 4096
2215 052/300 Read block 3478 len 36864
2216 053/300 Write block 1568 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2217 054/300 Write block 3643 len 24576 data: 2 2 2 2 2 2
2218 055/300 Read block 1475 len 16384
2219 056/300 Write block 3874 len 20480 data: 3 3 3 2 2
2220 057/300 Write block 2404 len 12288 data: 2 2 2
2221 058/300 Read block 4318 len 16384
2222 059/300 Write block 4688 len 36864 data: 2 2 2 2 2 2 2 2 2
2223 060/300 Write block 4085 len 36864 data: 2 2 2 2 2 2 2 2 2
2224 061/300 Flush
2225 062/300 Write block 4455 len 32768 data: 2 2 2 2 2 2 2 2
2226 063/300 Read block 1007 len 20480
2227 064/300 Write block 2773 len 28672 data: 3 2 2 2 2 2 2
2228 065/300 Read block 3061 len 36864
2229 066/300 Write block 2772 len 28672 data: 3 4 3 3 3 3 3
2230 067/300 Read block 4463 len 40960
2231 068/300 Read block 61 len 40960
2232 069/300 Write block 4294 len 40960 data: 2 2 2 3 3 3 3 3 3 3
2233 070/300 Read block 4109 len 8192
2234 071/300 Flush
2235 072/300 Read block 2963 len 32768
2236 073/300 Flush
2237 074/300 Read block 4386 len 4096
2238 075/300 Write block 4655 len 4096 data: 2
2239 076/300 Read block 1499 len 8192
2240 077/300 Read block 851 len 12288
2241 078/300 Write block 4677 len 36864 data: 2 2 2 2 2 2 2 2 2
2242 079/300 Read block 3667 len 36864
2243 080/300 Read block 4718 len 24576
2244 081/300 Read block 792 len 16384
2245 082/300 Read block 2461 len 12288
2246 083/300 Read block 14 len 16384
2247 084/300 Read block 162 len 24576
2248 085/300 Write block 4651 len 20480 data: 3 3 3 3 3
2249 086/300 Write block 1192 len 20480 data: 3 2 2 3 3
2250 087/300 Write block 2631 len 4096 data: 2
2251 088/300 Read block 3649 len 20480
2252 089/300 Read block 904 len 4096
2253 090/300 Write block 2221 len 20480 data: 2 2 2 2 2
2254 091/300 Read block 3140 len 8192
2255 092/300 Read block 4115 len 36864
2256 093/300 Read block 3283 len 32768
2257 094/300 Read block 662 len 36864
2258 095/300 Write block 1612 len 36864 data: 2 2 2 2 2 2 2 2 2
2259 096/300 Write block 2407 len 8192 data: 2 2
2260 097/300 Read block 1541 len 8192
2261 098/300 Write block 3445 len 36864 data: 4 4 4 4 3 3 2 2 2
2262 099/300 Read block 652 len 32768
2263 100/300 Write block 2835 len 4096 data: 2
2264 101/300 Read block 957 len 20480
2265 102/300 Write block 3296 len 32768 data: 2 2 2 2 2 2 2 2
2266 103/300 Read block 3180 len 20480
2267 104/300 Read block 3904 len 20480
2268 105/300 Read block 376 len 20480
2269 106/300 Write block 718 len 28672 data: 2 2 2 3 3 3 3
2270 107/300 Flush
2271 108/300 Read block 669 len 8192
2272 109/300 Read block 3131 len 4096
2273 110/300 Write block 2063 len 32768 data: 2 2 2 2 2 2 2 2
2274 111/300 Read block 3016 len 32768
2275 112/300 Flush
2276 113/300 Read block 2297 len 36864
2277 114/300 Read block 279 len 36864
2278 115/300 Write block 1701 len 36864 data: 2 2 2 2 2 2 2 2 3
2279 116/300 Read block 2671 len 24576
2280 117/300 Write block 99 len 28672 data: 3 3 3 3 4 4 3
2281 118/300 Flush
2282 119/300 Read block 1409 len 20480
2283 120/300 Read block 1292 len 8192
2284 121/300 Flush
2285 122/300 Flush
2286 123/300 Read block 4774 len 32768
2287 124/300 Read block 2420 len 28672
2288 125/300 Flush
2289 126/300 Read block 2127 len 24576
2290 127/300 Flush
2291 128/300 Read block 4697 len 16384
2292 129/300 Write block 2706 len 12288 data: 2 2 2
2293 130/300 Write block 3128 len 4096 data: 2
2294 131/300 Write block 2980 len 16384 data: 2 2 2 2
2295 132/300 Flush
2296 133/300 Flush
2297 134/300 Write block 2035 len 28672 data: 2 2 2 2 2 2 2
2298 135/300 Write block 2082 len 36864 data: 2 2 2 2 2 2 2 2 2
2299 136/300 Write block 525 len 32768 data: 2 2 2 2 2 2 3 3
2300 137/300 Write block 1376 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2301 138/300 Write block 4039 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2302 139/300 Read block 3986 len 16384
2303 140/300 Write block 1184 len 12288 data: 3 3 3
2304 141/300 Write block 1375 len 20480 data: 2 3 3 3 3
2305 142/300 Read block 1815 len 40960
2306 143/300 Write block 4773 len 4096 data: 2
2307 144/300 Write block 3262 len 12288 data: 2 2 2
2308 145/300 Write block 4293 len 36864 data: 2 3 3 3 4 4 4 4 4
2309 146/300 Write block 1694 len 28672 data: 2 2 2 2 2 2 2
2310 147/300 Write block 4170 len 8192 data: 3 3
2311 148/300 Write block 3186 len 24576 data: 3 3 3 3 3 3
2312 149/300 Read block 122 len 40960
2313 150/300 Read block 3794 len 12288
2314 151/300 Write block 807 len 40960 data: 2 2 2 2 2 2 3 3 3 3
2315 152/300 Flush
2316 153/300 Flush
2317 154/300 Write block 886 len 16384 data: 2 2 2 2
2318 155/300 Read block 1895 len 28672
2319 156/300 Read block 277 len 8192
2320 157/300 Flush
2321 158/300 Write block 2824 len 36864 data: 2 2 2 2 2 2 2 2 2
2322 159/300 Read block 1851 len 40960
2323 160/300 Write block 541 len 8192 data: 2 2
2324 161/300 Read block 3237 len 4096
2325 162/300 Flush
2326 163/300 Flush
2327 164/300 Write block 2706 len 8192 data: 3 3
2328 165/300 Write block 1156 len 24576 data: 2 2 2 2 2 2
2329 166/300 Flush
2330 167/300 Read block 4010 len 40960
2331 168/300 Write block 1355 len 20480 data: 3 3 3 4 3
2332 169/300 Write block 2993 len 28672 data: 3 3 3 3 2 2 2
2333 170/300 Write block 2846 len 4096 data: 3
2334 171/300 Read block 4574 len 16384
2335 172/300 Read block 2674 len 32768
2336 173/300 Read block 4652 len 12288
2337 174/300 Flush
2338 175/300 Read block 1193 len 24576
2339 176/300 Write block 2553 len 8192 data: 2 2
2340 177/300 Read block 2570 len 8192
2341 178/300 Read block 1595 len 32768
2342 179/300 Read block 3984 len 12288
2343 180/300 Write block 3059 len 28672 data: 2 2 2 2 2 2 2
2344 181/300 Write block 4170 len 20480 data: 4 4 4 5 6
2345 182/300 Read block 4448 len 16384
2346 183/300 Write block 790 len 16384 data: 3 3 3 3
2347 184/300 Flush
2348 185/300 Read block 569 len 16384
2349 186/300 Read block 2734 len 20480
2350 187/300 Read block 3533 len 28672
2351 188/300 Write block 4756 len 4096 data: 2
2352 189/300 Write block 1328 len 8192 data: 3 3
2353 190/300 Write block 1313 len 20480 data: 3 3 3 2 2
2354 191/300 Read block 2360 len 28672
2355 192/300 Write block 108 len 24576 data: 3 3 3 3 2 2
2356 193/300 Read block 2546 len 32768
2357 194/300 Write block 888 len 12288 data: 3 3 2
2358 195/300 Read block 4504 len 32768
2359 196/300 Read block 602 len 12288
2360 197/300 Read block 4856 len 24576
2361 198/300 Read block 367 len 8192
2362 199/300 Read block 1396 len 8192
2363 200/300 Read block 961 len 4096
2364 201/300 Read block 453 len 8192
2365 202/300 Read block 467 len 24576
2366 203/300 Write block 2069 len 4096 data: 3
2367 204/300 Write block 3136 len 20480 data: 2 2 2 2 2
2368 205/300 Write block 1002 len 4096 data: 2
2369 206/300 Write block 3085 len 20480 data: 3 3 3 2 2
2370 207/300 Write block 1383 len 40960 data: 3 3 3 2 2 2 2 2 3 3
2371 208/300 Flush
2372 209/300 Read block 2182 len 24576
2373 210/300 Read block 2478 len 20480
2374 211/300 Write block 502 len 4096 data: 2
2375 212/300 Read block 3769 len 32768
2376 213/300 Read block 1881 len 24576
2377 214/300 Write block 195 len 12288 data: 2 2 2
2378 215/300 Write block 2388 len 40960 data: 3 2 2 2 2 2 2 2 3 3
2379 216/300 Read block 1363 len 8192
2380 217/300 Read block 2850 len 32768
2381 218/300 Read block 3447 len 32768
2382 219/300 Write block 3235 len 16384 data: 2 2 2 2
2383 220/300 Write block 3930 len 20480 data: 2 2 2 3 3
2384 221/300 Write block 2487 len 8192 data: 2 2
2385 222/300 Write block 3316 len 20480 data: 3 3 3 3 2
2386 223/300 Read block 1285 len 32768
2387 224/300 Write block 4733 len 36864 data: 2 2 2 2 2 2 2 2 2
2388 225/300 Write block 4155 len 24576 data: 2 2 2 2 2 2
2389 226/300 Read block 4942 len 36864
2390 227/300 Write block 792 len 4096 data: 4
2391 228/300 Write block 3745 len 12288 data: 2 2 2
2392 229/300 Read block 4255 len 12288
2393 230/300 Read block 4452 len 16384
2394 231/300 Read block 436 len 32768
2395 232/300 Write block 549 len 24576 data: 2 2 2 2 2 2
2396 233/300 Read block 3170 len 16384
2397 234/300 Read block 3316 len 16384
2398 235/300 Write block 3837 len 32768 data: 2 3 4 4 4 4 4 2
2399 236/300 Read block 565 len 4096
2400 237/300 Write block 4933 len 12288 data: 2 2 2
2401 238/300 Read block 161 len 16384
2402 239/300 Flush
2403 240/300 Read block 1339 len 12288
2404 241/300 Read block 2395 len 16384
2405 242/300 Write block 2586 len 28672 data: 3 3 3 3 2 3 3
2406 243/300 Read block 854 len 12288
2407 244/300 Read block 3608 len 40960
2408 245/300 Write block 59 len 16384 data: 2 2 3 3
2409 246/300 Read block 1209 len 8192
2410 247/300 Read block 1281 len 16384
2411 248/300 Write block 1426 len 24576 data: 2 2 2 2 2 2
2412 249/300 Write block 171 len 40960 data: 2 2 3 3 3 3 3 3 3 3
2413 250/300 Read block 907 len 32768
2414 251/300 Read block 2420 len 8192
2415 252/300 Read block 1926 len 8192
2416 253/300 Write block 389 len 12288 data: 2 2 2
2417 254/300 Flush
2418 255/300 Read block 3732 len 32768
2419 256/300 Read block 1568 len 40960
2420 257/300 Write block 3101 len 40960 data: 3 3 3 2 2 2 2 2 2 2
2421 258/300 Flush
2422 259/300 Read block 1514 len 20480
2423 260/300 Read block 1467 len 20480
2424 261/300 Read block 863 len 12288
2425 262/300 Read block 3286 len 28672
2426 263/300 Read block 4061 len 36864
2427 264/300 Read block 2477 len 20480
2428 265/300 Read block 2955 len 8192
2429 266/300 Read block 4090 len 12288
2430 267/300 Read block 1241 len 28672
2431 268/300 Write block 2645 len 4096 data: 3
2432 269/300 Read block 2916 len 36864
2433 270/300 Write block 1013 len 20480 data: 2 2 2 2 2
2434 271/300 Write block 4265 len 40960 data: 2 3 3 4 4 4 4 3 4 4
2435 272/300 Read block 1118 len 16384
2436 273/300 Read block 110 len 24576
2437 274/300 Write block 3079 len 36864 data: 3 3 3 4 4 3 4 4 4
2438 275/300 Read block 3336 len 4096
2439 276/300 Read block 2603 len 24576
2440 277/300 Read block 4015 len 40960
2441 278/300 Flush
2442 279/300 Flush
2443 280/300 Write block 3965 len 12288 data: 2 2 2
2444 281/300 Write block 707 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2445 282/300 Write block 2329 len 20480 data: 2 2 2 2 2
2446 283/300 Write block 2550 len 36864 data: 2 2 2 3 3 2 2 2 2
2447 284/300 Read block 360 len 12288
2448 285/300 Read block 2037 len 4096
2449 286/300 Read block 2702 len 36864
2450 287/300 Read block 1926 len 20480
2451 288/300 Write block 1326 len 32768 data: 3 3 4 4 3 2 2 2
2452 289/300 Read block 306 len 8192
2453 290/300 Read block 629 len 12288
2454 291/300 Read block 500 len 24576
2455 292/300 Read block 1127 len 16384
2456 293/300 Write block 3875 len 16384 data: 4 4 3 3
2457 294/300 Read block 4980 len 32768
2458 295/300 Read block 877 len 8192
2459 296/300 Flush
2460 297/300 Read block 1468 len 20480
2461 298/300 Read block 4749 len 12288
2462 299/300 Read block 720 len 36864
2463 300/300 Write block 172 len 20480 data: 3 4 4 4 4
2464 301/300 Flush
2465 Main task has work to do, go find it
2466 got dsc Start(2)
2467 start 2
2468 Jan 17 21:11:37.091 INFO request completed, latency_us: 362, response_code: 204, uri: /start/cid/2, method: POST, req_id: d0bf2e5c-2845-4c4d-a299-39587cefdc1b, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
2469 Replay: started 2, returned:Ok(())
2470 ----------------------------------------------------------------
2471 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
2472 States: Active Active Offline
2473 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2474 1604 Acked Read 6 Done Done Sent false
2475 1605 Acked Write 7 Done Done Sent false
2476 1606 Acked Write 4 Done Done Sent false
2477 1607 Acked Read 7 Done Done Sent false
2478 1608 Acked Write 10 Done Done Sent false
2479 1609 Acked Read 3 Done Done Sent false
2480 1610 Acked Read 10 Done Done Sent false
2481 1611 Acked Read 6 Done Done Sent false
2482 1612 Acked Read 1 Done Done Sent false
2483 1613 Acked Read 8 Done Done Sent false
2484 1614 Acked Write 10 Done Done Sent false
2485 1615 Acked Read 5 Done Done Sent false
2486 1616 Acked Write 3 Done Done Sent false
2487 1617 Acked Write 10 Done Done Sent false
2488 1618 Acked Read 5 Done Done Sent false
2489 1619 Acked Read 1 Done Done Sent false
2490 1620 Acked Read 4 Done Done Sent false
2491 1621 Acked Read 6 Done Done Sent false
2492 1622 Acked Write 9 Done Done Sent false
2493 1623 Acked Write 9 Done Done Sent false
2494 1624 Acked Read 1 Done Done Sent false
2495 1625 Acked Write 1 Done Done Sent false
2496 1626 Acked Read 5 Done Done Sent false
2497 1627 Acked Read 8 Done Done Sent false
2498 1628 Acked Write 10 Done Done Sent false
2499 1629 Acked Read 8 Done Done Sent false
2500 1630 Acked Write 8 Done Done Sent false
2501 1631 Acked Read 7 Done Done Sent false
2502 1632 Acked Write 10 Done Done Sent false
2503 1633 Acked Write 2 Done Done Sent false
2504 1634 Acked Read 5 Done Done Sent false
2505 1635 Acked Write 10 Done Done Sent false
2506 1636 Acked Read 1 Done Done Sent false
2507 1637 Acked Write 10 Done Done Sent false
2508 1638 Acked Read 3 Done Done Sent false
2509 1639 Acked Read 4 Done Done Sent false
2510 1640 Acked Write 5 Done Done Sent false
2511 1641 Acked Write 10 Done Done Sent false
2512 1642 Acked Read 6 Done Done Sent false
2513 1643 Acked Read 10 Done Done Sent false
2514 1644 Acked Write 9 Done Done Sent false
2515 1645 Acked Write 4 Done Done Sent false
2516 1646 Acked Write 7 Done Done Sent false
2517 1647 Acked Read 8 Done Done Sent false
2518 1648 Acked Read 8 Done Done Sent false
2519 1649 Acked Write 8 Done Done Sent false
2520 1650 Acked Read 7 Done Done Sent false
2521 1651 Acked Read 9 Done Done Sent false
2522 1652 Acked Read 1 Done Done Sent false
2523 1653 Acked Read 9 Done Done Sent false
2524 1654 Acked Read 1 Done Done Sent false
2525 1655 Acked Read 9 Done Done Sent false
2526 1656 Acked Write 10 Done Done Sent false
2527 1657 Acked Write 6 Done Done Sent false
2528 1658 Acked Read 4 Done Done Sent false
2529 1659 Acked Write 5 Done Done Sent false
2530 1660 Acked Write 3 Done Done Sent false
2531 1661 Acked Read 4 Done Done Sent false
2532 1662 Acked Write 9 Done Done Sent false
2533 1663 Acked Write 9 Done Done Sent false
2534 1664 Acked Flush 0 Done Done Sent false
2535 1665 Acked Write 8 Done Done Sent false
2536 1666 Acked Read 5 Done Done Sent false
2537 1667 Acked Write 7 Done Done Sent false
2538 1668 Acked Read 9 Done Done Sent false
2539 1669 Acked Write 7 Done Done Sent false
2540 1670 Acked Read 10 Done Done Sent false
2541 1671 Acked Read 10 Done Done Sent false
2542 1672 Acked Write 10 Done Done Sent false
2543 1673 Acked Read 2 Done Done Sent false
2544 1674 Acked Flush 0 Done Done Sent false
2545 1675 Acked Read 8 Done Done Sent false
2546 1676 Acked Flush 0 Done Done Sent false
2547 1677 Acked Read 1 Done Done Sent false
2548 1678 Acked Write 1 Done Done Sent false
2549 1679 Acked Read 2 Done Done Sent false
2550 1680 Acked Read 3 Done Done Sent false
2551 1681 Acked Write 9 Done Done Sent false
2552 1682 Acked Read 9 Done Done Sent false
2553 1683 Acked Read 6 Done Done Sent false
2554 1684 Acked Read 4 Done Done Sent false
2555 1685 Acked Read 3 Done Done Sent false
2556 1686 Acked Read 4 Done Done Sent false
2557 1687 Acked Read 6 Done Done Sent false
2558 1688 Acked Write 5 Done Done Sent false
2559 1689 Acked Write 5 Done Done Sent false
2560 1690 Acked Write 1 Done Done Sent false
2561 1691 Acked Read 5 Done Done Sent false
2562 1692 Acked Read 1 Done Done Sent false
2563 1693 Acked Write 5 Done Done Sent false
2564 1694 Acked Read 2 Done Done Sent false
2565 1695 Acked Read 9 Done Done Sent false
2566 1696 Acked Read 8 Done Done Sent false
2567 1697 Acked Read 9 Done Done Sent false
2568 1698 Acked Write 9 Done Done Sent false
2569 1699 Acked Write 2 Done Done Sent false
2570 1700 Acked Read 2 Done Done Sent false
2571 1701 Acked Write 9 Done Done Sent false
2572 1702 Acked Read 8 Done Done Sent false
2573 1703 Acked Write 1 Done Done Sent false
2574 1704 Acked Read 5 Done Done Sent false
2575 1705 Acked Write 8 Done Done Sent false
2576 1706 Acked Read 5 Done Done Sent false
2577 1707 Acked Read 5 Done Done Sent false
2578 1708 Acked Read 5 Done Done Sent false
2579 1709 Acked Write 7 Done Done Sent false
2580 1710 Acked Flush 0 Done Done Sent false
2581 1711 Acked Read 2 Done Done Sent false
2582 1712 Acked Read 1 Done Done Sent false
2583 1713 Acked Write 8 Done Done Sent false
2584 1714 Acked Read 8 Done Done Sent false
2585 1715 Acked Flush 0 Done Done Sent false
2586 1716 Acked Read 9 Done Done Sent false
2587 1717 Acked Read 9 Done Done Sent false
2588 1718 Acked Write 9 Done Done Sent false
2589 1719 Acked Read 6 Done Done Sent false
2590 1720 Acked Write 7 Done Done Sent false
2591 1721 Acked Flush 0 Done Done Sent false
2592 1722 Acked Read 5 Done Done Sent false
2593 1723 Acked Read 2 Done Done Sent false
2594 1724 Acked Flush 0 Done Done Sent false
2595 1725 Acked Flush 0 Done Done Sent false
2596 1726 Acked Read 8 Done Done Sent false
2597 1727 Acked Read 7 Done Done Sent false
2598 1728 Acked Flush 0 Done Done Sent false
2599 1729 Acked Read 6 Done Done Sent false
2600 1730 Acked Flush 0 Done Done Sent false
2601 1731 Acked Read 4 Done Done Sent false
2602 1732 Acked Write 3 Done Done Sent false
2603 1733 Acked Write 1 Done Done Sent false
2604 1734 Acked Write 4 Done Done Sent false
2605 1735 Acked Flush 0 Done Done Sent false
2606 1736 Acked Flush 0 Done Done Sent false
2607 1737 Acked Write 7 Done Done Sent false
2608 1738 Acked Write 9 Done Done Sent false
2609 1739 Acked Write 8 Done Done Sent false
2610 1740 Acked Write 10 Done Done Sent false
2611 1741 Acked Write 10 Done Done Sent false
2612 1742 Acked Read 4 Done Done Sent false
2613 1743 Acked Write 3 Done Done Sent false
2614 1744 Acked Write 5 Done Done Sent false
2615 1745 Acked Read 10 Done Done Sent false
2616 1746 Acked Write 1 Done Done Sent false
2617 1747 Acked Write 3 Done Done Sent false
2618 1748 Acked Write 9 Done Done Sent false
2619 1749 Acked Write 7 Done Done Sent false
2620 1750 Acked Write 2 Done Done Sent false
2621 1751 Acked Write 6 Done Done Sent false
2622 1752 Acked Read 10 Done Done Sent false
2623 1753 Acked Read 3 Done Done Sent false
2624 1754 Acked Write 10 Done Done Sent false
2625 1755 Acked Flush 0 Done Done Sent false
2626 1756 Acked Flush 0 Done Done Sent false
2627 1757 Acked Write 4 Done Done Sent false
2628 1758 Acked Read 7 Done Done Sent false
2629 1759 Acked Read 2 Done Done Sent false
2630 1760 Acked Flush 0 Done Done Sent false
2631 1761 Acked Write 9 Done Done Sent false
2632 1762 Acked Read 10 Done Done Sent false
2633 1763 Acked Write 2 Done Done Sent false
2634 1764 Acked Read 1 Done Done Sent false
2635 1765 Acked Flush 0 Done Done Sent false
2636 1766 Acked Flush 0 Done Done Sent false
2637 1767 Acked Write 2 Done Done Sent false
2638 1768 Acked Write 6 Done Done Sent false
2639 1769 Acked Flush 0 Done Done Sent false
2640 1770 Acked Read 10 Done Done Sent false
2641 1771 Acked Write 5 Done Done Sent false
2642 1772 Acked Write 7 Done Done Sent false
2643 1773 Acked Write 1 Done Done Sent false
2644 1774 Acked Read 4 Done Done Sent false
2645 1775 Acked Read 8 Done Done Sent false
2646 1776 Acked Read 3 Done Done Sent false
2647 1777 Acked Flush 0 Done Done Sent false
2648 1778 Acked Read 6 Done Done Sent false
2649 1779 Acked Write 2 Done Done Sent false
2650 1780 Acked Read 2 Done Done Sent false
2651 1781 Acked Read 8 Done Done Sent false
2652 1782 Acked Read 3 Done Done Sent false
2653 1783 Acked Write 7 Done Done Sent false
2654 1784 Acked Write 5 Done Done Sent false
2655 1785 Acked Read 4 Done Done Sent false
2656 1786 Acked Write 4 Done Done Sent false
2657 1787 Acked Flush 0 Done Done Sent false
2658 1788 Acked Read 4 Done Done Sent false
2659 1789 Acked Read 5 Done Done Sent false
2660 1790 Acked Read 7 Done Done Sent false
2661 1791 Acked Write 1 Done Done Sent false
2662 1792 Acked Write 2 Done Done Sent false
2663 1793 Acked Write 5 Done Done Sent false
2664 1794 Acked Read 7 Done Done Sent false
2665 1795 Acked Write 6 Done Done Sent false
2666 1796 Acked Read 8 Done Done Sent false
2667 1797 Acked Write 3 Done Done Sent false
2668 1798 Acked Read 8 Done Done Sent false
2669 1799 Acked Read 3 Done Done Sent false
2670 1800 Acked Read 6 Done Done Sent false
2671 1801 Acked Read 2 Done Done Sent false
2672 1802 Acked Read 2 Done Done Sent false
2673 1803 Acked Read 1 Done Done Sent false
2674 1804 Acked Read 2 Done Done Sent false
2675 1805 Acked Read 6 Done Done Sent false
2676 1806 Acked Write 1 Done Done Sent false
2677 1807 Acked Write 5 Done Done Sent false
2678 1808 Acked Write 1 Done Done Sent false
2679 1809 Acked Write 5 Done Done Sent false
2680 1810 Acked Write 10 Done Done Sent false
2681 1811 Acked Flush 0 Done Done Sent false
2682 1812 Acked Read 6 Done Done Sent false
2683 1813 Acked Read 5 Done Done Sent false
2684 1814 Acked Write 1 Done Done Sent false
2685 1815 Acked Read 8 Done Done Sent false
2686 1816 Acked Read 6 Done Done Sent false
2687 1817 Acked Write 3 Done Done Sent false
2688 1818 Acked Write 10 Done Done Sent false
2689 1819 Acked Read 2 Done Done Sent false
2690 1820 Acked Read 8 Done Done Sent false
2691 1821 Acked Read 8 Done Done Sent false
2692 1822 Acked Write 4 Done Done Sent false
2693 1823 Acked Write 5 Done Done Sent false
2694 1824 Acked Write 2 Done Done Sent false
2695 1825 Acked Write 5 Done Done Sent false
2696 1826 Acked Read 8 Done Done Sent false
2697 1827 Acked Write 9 Done Done Sent false
2698 1828 Acked Write 6 Done Done Sent false
2699 1829 Acked Read 9 Done Done Sent false
2700 1830 Acked Write 1 Done Done Sent false
2701 1831 Acked Write 3 Done Done Sent false
2702 1832 Acked Read 3 Done Done Sent false
2703 1833 Acked Read 4 Done Done Sent false
2704 1834 Acked Read 8 Done Done Sent false
2705 1835 Acked Write 6 Done Done Sent false
2706 1836 Acked Read 4 Done Done Sent false
2707 1837 Acked Read 4 Done Done Sent false
2708 1838 Acked Write 8 Done Done Sent false
2709 1839 Acked Read 1 Done Done Sent false
2710 1840 Acked Write 3 Done Done Sent false
2711 1841 Acked Read 4 Done Done Sent false
2712 1842 Acked Flush 0 Done Done Sent false
2713 1843 Acked Read 3 Done Done Sent false
2714 1844 Acked Read 4 Done Done Sent false
2715 1845 Acked Write 7 Done Done Sent false
2716 1846 Acked Read 3 Done Done Sent false
2717 1847 Acked Read 10 Done Done Sent false
2718 1848 Acked Write 4 Done Done Sent false
2719 1849 Acked Read 2 Done Done Sent false
2720 1850 Acked Read 4 Done Done Sent false
2721 1851 Acked Write 6 Done Done Sent false
2722 1852 Acked Write 10 Done Done Sent false
2723 1853 Acked Read 8 Done Done Sent false
2724 1854 Acked Read 2 Done Done Sent false
2725 1855 Acked Read 2 Done Done Sent false
2726 1856 Acked Write 3 Done Done Sent false
2727 1857 Acked Flush 0 Done Done Sent false
2728 1858 Acked Read 8 Done Done Sent false
2729 1859 Acked Read 10 Done Done Sent false
2730 1860 Acked Write 10 Done Done Sent false
2731 1861 Acked Flush 0 Done Done Sent false
2732 1862 Acked Read 5 Done Done Sent false
2733 1863 Acked Read 5 Done Done Sent false
2734 1864 Acked Read 3 Done Done Sent false
2735 1865 Acked Read 7 Done Done Sent false
2736 1866 Acked Read 9 Done Done Sent false
2737 1867 Acked Read 5 Done Done Sent false
2738 1868 Acked Read 2 Done Done Sent false
2739 1869 Acked Read 3 Done Done Sent false
2740 1870 Acked Read 7 Done Done Sent false
2741 1871 Acked Write 1 Done Done Sent false
2742 1872 Acked Read 9 Done Done Sent false
2743 1873 Acked Write 5 Done Done Sent false
2744 1874 Acked Write 10 Done Done Sent false
2745 1875 Acked Read 4 Done Done Sent false
2746 1876 Acked Read 6 Done Done Sent false
2747 1877 Acked Write 9 Done Done Sent false
2748 1878 Acked Read 1 Done Done Sent false
2749 1879 Acked Read 6 Done Done Sent false
2750 1880 Acked Read 10 Done Done Sent false
2751 1881 Acked Flush 0 Done Done Sent false
2752 1882 Acked Flush 0 Done Done Sent false
2753 1883 Acked Write 3 Done Done Sent false
2754 1884 Acked Write 10 Done Done Sent false
2755 1885 Acked Write 5 Done Done Sent false
2756 1886 Acked Write 9 Done Done Sent false
2757 1887 Acked Read 3 Done Done Sent false
2758 1888 Acked Read 1 Done Done Sent false
2759 1889 Acked Read 9 Done Done Sent false
2760 1890 Acked Read 5 Done Done Sent false
2761 1891 Acked Write 8 Done Done Sent false
2762 1892 Acked Read 2 Done Done Sent false
2763 1893 Acked Read 3 Done Done Sent false
2764 1894 Acked Read 6 Done Done Sent false
2765 1895 Acked Read 4 Done Done Sent false
2766 1896 Acked Write 4 Done Done Sent false
2767 1897 Acked Read 8 Done Done Sent false
2768 1898 Acked Read 2 Done Done Sent false
2769 1899 Acked Flush 0 Done Done Sent false
2770 1900 Acked Read 5 Done Done Sent false
2771 1901 Acked Read 3 Done Done Sent false
2772 1902 Acked Read 9 Done Done Sent false
2773 1903 Acked Write 5 Done Done Sent false
2774 1904 Acked Flush 0 Done Done Sent false
2775 STATES DS:0 DS:1 DS:2 TOTAL
2776 Sent 0 0 301 301
2777 Done 301 301 0 602
2778 Skipped 0 0 0 0
2779 Error 0 0 0 0
2780 Last Flush: 1904 1904 1603
2781 Downstairs last five completed: 1603 1602 1601 1600 1599
2782 Upstairs last five completed: 1904 1903 1902 1901 1900
2783 CLIENT: Up:0 ds:301 act:2
2784 [8830] got start action
2785 [8830] I am going to restart
2786 Starting downstairs at port 8830
2787 Make output file at "/tmp/dsc/downstairs-8830.txt"
2788 [8830][2] reports Starting
2789 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1203)
2790 [8830][2] reports Running
2791 ----------------------------------------------------------------
2792 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
2793 States: Active Active Offline
2794 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2795 1604 Acked Read 6 Done Done Sent false
2796 1605 Acked Write 7 Done Done Sent false
2797 1606 Acked Write 4 Done Done Sent false
2798 1607 Acked Read 7 Done Done Sent false
2799 1608 Acked Write 10 Done Done Sent false
2800 1609 Acked Read 3 Done Done Sent false
2801 1610 Acked Read 10 Done Done Sent false
2802 1611 Acked Read 6 Done Done Sent false
2803 1612 Acked Read 1 Done Done Sent false
2804 1613 Acked Read 8 Done Done Sent false
2805 1614 Acked Write 10 Done Done Sent false
2806 1615 Acked Read 5 Done Done Sent false
2807 1616 Acked Write 3 Done Done Sent false
2808 1617 Acked Write 10 Done Done Sent false
2809 1618 Acked Read 5 Done Done Sent false
2810 1619 Acked Read 1 Done Done Sent false
2811 1620 Acked Read 4 Done Done Sent false
2812 1621 Acked Read 6 Done Done Sent false
2813 1622 Acked Write 9 Done Done Sent false
2814 1623 Acked Write 9 Done Done Sent false
2815 1624 Acked Read 1 Done Done Sent false
2816 1625 Acked Write 1 Done Done Sent false
2817 1626 Acked Read 5 Done Done Sent false
2818 1627 Acked Read 8 Done Done Sent false
2819 1628 Acked Write 10 Done Done Sent false
2820 1629 Acked Read 8 Done Done Sent false
2821 1630 Acked Write 8 Done Done Sent false
2822 1631 Acked Read 7 Done Done Sent false
2823 1632 Acked Write 10 Done Done Sent false
2824 1633 Acked Write 2 Done Done Sent false
2825 1634 Acked Read 5 Done Done Sent false
2826 1635 Acked Write 10 Done Done Sent false
2827 1636 Acked Read 1 Done Done Sent false
2828 1637 Acked Write 10 Done Done Sent false
2829 1638 Acked Read 3 Done Done Sent false
2830 1639 Acked Read 4 Done Done Sent false
2831 1640 Acked Write 5 Done Done Sent false
2832 1641 Acked Write 10 Done Done Sent false
2833 1642 Acked Read 6 Done Done Sent false
2834 1643 Acked Read 10 Done Done Sent false
2835 1644 Acked Write 9 Done Done Sent false
2836 1645 Acked Write 4 Done Done Sent false
2837 1646 Acked Write 7 Done Done Sent false
2838 1647 Acked Read 8 Done Done Sent false
2839 1648 Acked Read 8 Done Done Sent false
2840 1649 Acked Write 8 Done Done Sent false
2841 1650 Acked Read 7 Done Done Sent false
2842 1651 Acked Read 9 Done Done Sent false
2843 1652 Acked Read 1 Done Done Sent false
2844 1653 Acked Read 9 Done Done Sent false
2845 1654 Acked Read 1 Done Done Sent false
2846 1655 Acked Read 9 Done Done Sent false
2847 1656 Acked Write 10 Done Done Sent false
2848 1657 Acked Write 6 Done Done Sent false
2849 1658 Acked Read 4 Done Done Sent false
2850 1659 Acked Write 5 Done Done Sent false
2851 1660 Acked Write 3 Done Done Sent false
2852 1661 Acked Read 4 Done Done Sent false
2853 1662 Acked Write 9 Done Done Sent false
2854 1663 Acked Write 9 Done Done Sent false
2855 1664 Acked Flush 0 Done Done Sent false
2856 1665 Acked Write 8 Done Done Sent false
2857 1666 Acked Read 5 Done Done Sent false
2858 1667 Acked Write 7 Done Done Sent false
2859 1668 Acked Read 9 Done Done Sent false
2860 1669 Acked Write 7 Done Done Sent false
2861 1670 Acked Read 10 Done Done Sent false
2862 1671 Acked Read 10 Done Done Sent false
2863 1672 Acked Write 10 Done Done Sent false
2864 1673 Acked Read 2 Done Done Sent false
2865 1674 Acked Flush 0 Done Done Sent false
2866 1675 Acked Read 8 Done Done Sent false
2867 1676 Acked Flush 0 Done Done Sent false
2868 1677 Acked Read 1 Done Done Sent false
2869 1678 Acked Write 1 Done Done Sent false
2870 1679 Acked Read 2 Done Done Sent false
2871 1680 Acked Read 3 Done Done Sent false
2872 1681 Acked Write 9 Done Done Sent false
2873 1682 Acked Read 9 Done Done Sent false
2874 1683 Acked Read 6 Done Done Sent false
2875 1684 Acked Read 4 Done Done Sent false
2876 1685 Acked Read 3 Done Done Sent false
2877 1686 Acked Read 4 Done Done Sent false
2878 1687 Acked Read 6 Done Done Sent false
2879 1688 Acked Write 5 Done Done Sent false
2880 1689 Acked Write 5 Done Done Sent false
2881 1690 Acked Write 1 Done Done Sent false
2882 1691 Acked Read 5 Done Done Sent false
2883 1692 Acked Read 1 Done Done Sent false
2884 1693 Acked Write 5 Done Done Sent false
2885 1694 Acked Read 2 Done Done Sent false
2886 1695 Acked Read 9 Done Done Sent false
2887 1696 Acked Read 8 Done Done Sent false
2888 1697 Acked Read 9 Done Done Sent false
2889 1698 Acked Write 9 Done Done Sent false
2890 1699 Acked Write 2 Done Done Sent false
2891 1700 Acked Read 2 Done Done Sent false
2892 1701 Acked Write 9 Done Done Sent false
2893 1702 Acked Read 8 Done Done Sent false
2894 1703 Acked Write 1 Done Done Sent false
2895 1704 Acked Read 5 Done Done Sent false
2896 1705 Acked Write 8 Done Done Sent false
2897 1706 Acked Read 5 Done Done Sent false
2898 1707 Acked Read 5 Done Done Sent false
2899 1708 Acked Read 5 Done Done Sent false
2900 1709 Acked Write 7 Done Done Sent false
2901 1710 Acked Flush 0 Done Done Sent false
2902 1711 Acked Read 2 Done Done Sent false
2903 1712 Acked Read 1 Done Done Sent false
2904 1713 Acked Write 8 Done Done Sent false
2905 1714 Acked Read 8 Done Done Sent false
2906 1715 Acked Flush 0 Done Done Sent false
2907 1716 Acked Read 9 Done Done Sent false
2908 1717 Acked Read 9 Done Done Sent false
2909 1718 Acked Write 9 Done Done Sent false
2910 1719 Acked Read 6 Done Done Sent false
2911 1720 Acked Write 7 Done Done Sent false
2912 1721 Acked Flush 0 Done Done Sent false
2913 1722 Acked Read 5 Done Done Sent false
2914 1723 Acked Read 2 Done Done Sent false
2915 1724 Acked Flush 0 Done Done Sent false
2916 1725 Acked Flush 0 Done Done Sent false
2917 1726 Acked Read 8 Done Done Sent false
2918 1727 Acked Read 7 Done Done Sent false
2919 1728 Acked Flush 0 Done Done Sent false
2920 1729 Acked Read 6 Done Done Sent false
2921 1730 Acked Flush 0 Done Done Sent false
2922 1731 Acked Read 4 Done Done Sent false
2923 1732 Acked Write 3 Done Done Sent false
2924 1733 Acked Write 1 Done Done Sent false
2925 1734 Acked Write 4 Done Done Sent false
2926 1735 Acked Flush 0 Done Done Sent false
2927 1736 Acked Flush 0 Done Done Sent false
2928 1737 Acked Write 7 Done Done Sent false
2929 1738 Acked Write 9 Done Done Sent false
2930 1739 Acked Write 8 Done Done Sent false
2931 1740 Acked Write 10 Done Done Sent false
2932 1741 Acked Write 10 Done Done Sent false
2933 1742 Acked Read 4 Done Done Sent false
2934 1743 Acked Write 3 Done Done Sent false
2935 1744 Acked Write 5 Done Done Sent false
2936 1745 Acked Read 10 Done Done Sent false
2937 1746 Acked Write 1 Done Done Sent false
2938 1747 Acked Write 3 Done Done Sent false
2939 1748 Acked Write 9 Done Done Sent false
2940 1749 Acked Write 7 Done Done Sent false
2941 1750 Acked Write 2 Done Done Sent false
2942 1751 Acked Write 6 Done Done Sent false
2943 1752 Acked Read 10 Done Done Sent false
2944 1753 Acked Read 3 Done Done Sent false
2945 1754 Acked Write 10 Done Done Sent false
2946 1755 Acked Flush 0 Done Done Sent false
2947 1756 Acked Flush 0 Done Done Sent false
2948 1757 Acked Write 4 Done Done Sent false
2949 1758 Acked Read 7 Done Done Sent false
2950 1759 Acked Read 2 Done Done Sent false
2951 1760 Acked Flush 0 Done Done Sent false
2952 1761 Acked Write 9 Done Done Sent false
2953 1762 Acked Read 10 Done Done Sent false
2954 1763 Acked Write 2 Done Done Sent false
2955 1764 Acked Read 1 Done Done Sent false
2956 1765 Acked Flush 0 Done Done Sent false
2957 1766 Acked Flush 0 Done Done Sent false
2958 1767 Acked Write 2 Done Done Sent false
2959 1768 Acked Write 6 Done Done Sent false
2960 1769 Acked Flush 0 Done Done Sent false
2961 1770 Acked Read 10 Done Done Sent false
2962 1771 Acked Write 5 Done Done Sent false
2963 1772 Acked Write 7 Done Done Sent false
2964 1773 Acked Write 1 Done Done Sent false
2965 1774 Acked Read 4 Done Done Sent false
2966 1775 Acked Read 8 Done Done Sent false
2967 1776 Acked Read 3 Done Done Sent false
2968 1777 Acked Flush 0 Done Done Sent false
2969 1778 Acked Read 6 Done Done Sent false
2970 1779 Acked Write 2 Done Done Sent false
2971 1780 Acked Read 2 Done Done Sent false
2972 1781 Acked Read 8 Done Done Sent false
2973 1782 Acked Read 3 Done Done Sent false
2974 1783 Acked Write 7 Done Done Sent false
2975 1784 Acked Write 5 Done Done Sent false
2976 1785 Acked Read 4 Done Done Sent false
2977 1786 Acked Write 4 Done Done Sent false
2978 1787 Acked Flush 0 Done Done Sent false
2979 1788 Acked Read 4 Done Done Sent false
2980 1789 Acked Read 5 Done Done Sent false
2981 1790 Acked Read 7 Done Done Sent false
2982 1791 Acked Write 1 Done Done Sent false
2983 1792 Acked Write 2 Done Done Sent false
2984 1793 Acked Write 5 Done Done Sent false
2985 1794 Acked Read 7 Done Done Sent false
2986 1795 Acked Write 6 Done Done Sent false
2987 1796 Acked Read 8 Done Done Sent false
2988 1797 Acked Write 3 Done Done Sent false
2989 1798 Acked Read 8 Done Done Sent false
2990 1799 Acked Read 3 Done Done Sent false
2991 1800 Acked Read 6 Done Done Sent false
2992 1801 Acked Read 2 Done Done Sent false
2993 1802 Acked Read 2 Done Done Sent false
2994 1803 Acked Read 1 Done Done Sent false
2995 1804 Acked Read 2 Done Done Sent false
2996 1805 Acked Read 6 Done Done Sent false
2997 1806 Acked Write 1 Done Done Sent false
2998 1807 Acked Write 5 Done Done Sent false
2999 1808 Acked Write 1 Done Done Sent false
3000 1809 Acked Write 5 Done Done Sent false
3001 1810 Acked Write 10 Done Done Sent false
3002 1811 Acked Flush 0 Done Done Sent false
3003 1812 Acked Read 6 Done Done Sent false
3004 1813 Acked Read 5 Done Done Sent false
3005 1814 Acked Write 1 Done Done Sent false
3006 1815 Acked Read 8 Done Done Sent false
3007 1816 Acked Read 6 Done Done Sent false
3008 1817 Acked Write 3 Done Done Sent false
3009 1818 Acked Write 10 Done Done Sent false
3010 1819 Acked Read 2 Done Done Sent false
3011 1820 Acked Read 8 Done Done Sent false
3012 1821 Acked Read 8 Done Done Sent false
3013 1822 Acked Write 4 Done Done Sent false
3014 1823 Acked Write 5 Done Done Sent false
3015 1824 Acked Write 2 Done Done Sent false
3016 1825 Acked Write 5 Done Done Sent false
3017 1826 Acked Read 8 Done Done Sent false
3018 1827 Acked Write 9 Done Done Sent false
3019 1828 Acked Write 6 Done Done Sent false
3020 1829 Acked Read 9 Done Done Sent false
3021 1830 Acked Write 1 Done Done Sent false
3022 1831 Acked Write 3 Done Done Sent false
3023 1832 Acked Read 3 Done Done Sent false
3024 1833 Acked Read 4 Done Done Sent false
3025 1834 Acked Read 8 Done Done Sent false
3026 1835 Acked Write 6 Done Done Sent false
3027 1836 Acked Read 4 Done Done Sent false
3028 1837 Acked Read 4 Done Done Sent false
3029 1838 Acked Write 8 Done Done Sent false
3030 1839 Acked Read 1 Done Done Sent false
3031 1840 Acked Write 3 Done Done Sent false
3032 1841 Acked Read 4 Done Done Sent false
3033 1842 Acked Flush 0 Done Done Sent false
3034 1843 Acked Read 3 Done Done Sent false
3035 1844 Acked Read 4 Done Done Sent false
3036 1845 Acked Write 7 Done Done Sent false
3037 1846 Acked Read 3 Done Done Sent false
3038 1847 Acked Read 10 Done Done Sent false
3039 1848 Acked Write 4 Done Done Sent false
3040 1849 Acked Read 2 Done Done Sent false
3041 1850 Acked Read 4 Done Done Sent false
3042 1851 Acked Write 6 Done Done Sent false
3043 1852 Acked Write 10 Done Done Sent false
3044 1853 Acked Read 8 Done Done Sent false
3045 1854 Acked Read 2 Done Done Sent false
3046 1855 Acked Read 2 Done Done Sent false
3047 1856 Acked Write 3 Done Done Sent false
3048 1857 Acked Flush 0 Done Done Sent false
3049 1858 Acked Read 8 Done Done Sent false
3050 1859 Acked Read 10 Done Done Sent false
3051 1860 Acked Write 10 Done Done Sent false
3052 1861 Acked Flush 0 Done Done Sent false
3053 1862 Acked Read 5 Done Done Sent false
3054 1863 Acked Read 5 Done Done Sent false
3055 1864 Acked Read 3 Done Done Sent false
3056 1865 Acked Read 7 Done Done Sent false
3057 1866 Acked Read 9 Done Done Sent false
3058 1867 Acked Read 5 Done Done Sent false
3059 1868 Acked Read 2 Done Done Sent false
3060 1869 Acked Read 3 Done Done Sent false
3061 1870 Acked Read 7 Done Done Sent false
3062 1871 Acked Write 1 Done Done Sent false
3063 1872 Acked Read 9 Done Done Sent false
3064 1873 Acked Write 5 Done Done Sent false
3065 1874 Acked Write 10 Done Done Sent false
3066 1875 Acked Read 4 Done Done Sent false
3067 1876 Acked Read 6 Done Done Sent false
3068 1877 Acked Write 9 Done Done Sent false
3069 1878 Acked Read 1 Done Done Sent false
3070 1879 Acked Read 6 Done Done Sent false
3071 1880 Acked Read 10 Done Done Sent false
3072 1881 Acked Flush 0 Done Done Sent false
3073 1882 Acked Flush 0 Done Done Sent false
3074 1883 Acked Write 3 Done Done Sent false
3075 1884 Acked Write 10 Done Done Sent false
3076 1885 Acked Write 5 Done Done Sent false
3077 1886 Acked Write 9 Done Done Sent false
3078 1887 Acked Read 3 Done Done Sent false
3079 1888 Acked Read 1 Done Done Sent false
3080 1889 Acked Read 9 Done Done Sent false
3081 1890 Acked Read 5 Done Done Sent false
3082 1891 Acked Write 8 Done Done Sent false
3083 1892 Acked Read 2 Done Done Sent false
3084 1893 Acked Read 3 Done Done Sent false
3085 1894 Acked Read 6 Done Done Sent false
3086 1895 Acked Read 4 Done Done Sent false
3087 1896 Acked Write 4 Done Done Sent false
3088 1897 Acked Read 8 Done Done Sent false
3089 1898 Acked Read 2 Done Done Sent false
3090 1899 Acked Flush 0 Done Done Sent false
3091 1900 Acked Read 5 Done Done Sent false
3092 1901 Acked Read 3 Done Done Sent false
3093 1902 Acked Read 9 Done Done Sent false
3094 1903 Acked Write 5 Done Done Sent false
3095 1904 Acked Flush 0 Done Done Sent false
3096 STATES DS:0 DS:1 DS:2 TOTAL
3097 Sent 0 0 301 301
3098 Done 301 301 0 602
3099 Skipped 0 0 0 0
3100 Error 0 0 0 0
3101 Last Flush: 1904 1904 1603
3102 Downstairs last five completed: 1603 1602 1601 1600 1599
3103 Upstairs last five completed: 1904 1903 1902 1901 1900
3104 CLIENT: Up:0 ds:301 act:2
31052025-01-17T21:11:42.846ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
31062025-01-17T21:11:42.846ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39892) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
31072025-01-17T21:11:42.847ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
31082025-01-17T21:11:42.848ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
31092025-01-17T21:11:42.848ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
31102025-01-17T21:11:42.848ZINFOcrucible: send last flush ID to this DS: 1603 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
31112025-01-17T21:11:42.848ZINFOcrucible: Replied this last flush ID: 1603 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
31122025-01-17T21:11:42.848ZINFOcrucible: [2] client re-new 301 jobs since flush 1603 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
31132025-01-17T21:11:42.848ZINFOcrucible: [2] Replayed 301 jobs since flush: 1603 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
3114 ----------------------------------------------------------------
3115 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
3116 Downstairs last five completed: 1904 1903 1902 1901 1900
3117 Upstairs last five completed: 1904 1903 1902 1901 1900
3118 CLIENT: Up:0 ds:0 act:3
3119 Replay: All jobs finished
3120 Main task has work to do, go find it
3121 got dsc Stop(1)
3122 stop 1
3123 Jan 17 21:11:45.096 INFO request completed, latency_us: 322, response_code: 204[8820] Got stop action so:false kr:false
3124 , uri: /stop/cid/1, method: POST, req_id: f48b5d60-8b74-4c9a-a95b-9c76be46c004, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
3125 Jan 17 21:11:45.097 INFO request completed{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:11:45.097594568Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
3126 , latency_us: 110, response_code: 200, uri: /state/cid/1, method: GET{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:11:45.097658468Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
3127 , req_id: 66fa9a06-e560-442a-9721-04f84522074c, remote_addr: 127.0.0{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:11:45.097682645Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
3128 .1:57191, local_addr: 127.0.0.1:9998
31292025-01-17T21:11:45.097ZINFOcrucible: client task is exiting = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
3130 [8820] Exited with: ExitStatus(unix_wait_status(9))
3131 [8820][1] reports Exit
3132 Jan 17 21:11:49.099 INFO request completed, latency_us: 221, response_code: 200, uri: /state/cid/1, method: GET, req_id: 8e2db49c-7ce7-4d4e-91bd-189c54ec9fa5, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
3133 001/300 Flush
3134 002/300 Flush
3135 003/300 Write block 4450 len 4096 data: 2
3136 004/300 Write block 101 len 40960 data: 4 4 5 5 4 3 3 4 4 4
3137 005/300 Flush
3138 006/300 Flush
3139 007/300 Read block 4714 len 36864
3140 008/300 Read block 380 len 4096
3141 009/300 Write block 2916 len 12288 data: 2 2 2
3142 010/300 Flush
3143 011/300 Read block 3686 len 28672
3144 012/300 Read block 3184 len 20480
3145 013/300 Read block 2297 len 20480
3146 014/300 Write block 919 len 20480 data: 2 2 2 2 2
3147 015/300 Flush
3148 016/300 Write block 1336 len 32768 data: 2 2 2 2 2 2 2 2
3149 017/300 Write block 4328 len 8192 data: 3 3
3150 018/300 Write block 246 len 4096 data: 2
3151 019/300 Read block 859 len 24576
3152 020/300 Read block 4068 len 12288
3153 021/300 Read block 2525 len 4096
3154 022/300 Read block 1427 len 36864
3155 023/300 Write block 1305 len 20480 data: 2 2 3 3 3
3156 024/300 Write block 1537 len 20480 data: 2 2 2 2 2
3157 025/300 Read block 1926 len 20480
3158 026/300 Read block 4421 len 32768
3159 027/300 Flush
3160 028/300 Read block 2495 len 40960
3161 029/300 Flush
3162 030/300 Read block 2086 len 16384
3163 031/300 Write block 292 len 32768 data: 2 2 2 2 2 2 2 2
3164 032/300 Write block 989 len 20480 data: 3 3 3 3 3
3165 033/300 Write block 4952 len 8192 data: 3 2
3166 034/300 Read block 2354 len 20480
3167 035/300 Write block 3929 len 28672 data: 2 3 3 3 4 4 3
3168 036/300 Write block 3881 len 24576 data: 2 2 2 2 2 2
3169 037/300 Write block 4211 len 16384 data: 2 3 3 3
3170 038/300 Write block 2393 len 20480 data: 3 3 3 4 4
3171 039/300 Read block 2552 len 24576
3172 040/300 Write block 2967 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3173 041/300 Write block 1825 len 32768 data: 2 2 2 2 2 2 2 2
3174 042/300 Write block 2644 len 16384 data: 2 4 3 2
3175 043/300 Read block 1427 len 36864
3176 044/300 Write block 3173 len 32768 data: 2 2 2 2 2 2 2 2
3177 045/300 Read block 4916 len 32768
3178 046/300 Read block 3255 len 20480
3179 047/300 Read block 2760 len 32768
3180 048/300 Read block 2825 len 36864
3181 049/300 Read block 4123 len 36864
3182 050/300 Read block 4924 len 20480
3183 051/300 Write block 608 len 8192 data: 2 2
3184 052/300 Flush
3185 053/300 Read block 3759 len 8192
3186 054/300 Write block 4393 len 28672 data: 2 2 3 3 3 3 3
3187 055/300 Flush
3188 056/300 Read block 459 len 4096
3189 057/300 Write block 2968 len 8192 data: 3 3
3190 058/300 Read block 4368 len 12288
3191 059/300 Flush
3192 060/300 Read block 2150 len 12288
3193 061/300 Read block 1454 len 16384
3194 062/300 Write block 2708 len 4096 data: 3
3195 063/300 Write block 422 len 12288 data: 4 5 4
3196 064/300 Flush
3197 065/300 Write block 3190 len 32768 data: 4 4 4 3 3 3 3 3
3198 066/300 Read block 227 len 4096
3199 067/300 Write block 2864 len 36864 data: 2 2 2 2 2 2 2 2 3
3200 068/300 Flush
3201 069/300 Write block 4556 len 8192 data: 2 2
3202 070/300 Read block 195 len 4096
3203 071/300 Write block 4697 len 40960 data: 2 2 2 2 2 2 2 3 3 3
3204 072/300 Write block 3768 len 8192 data: 2 2
3205 073/300 Read block 3025 len 12288
3206 074/300 Read block 1990 len 24576
3207 075/300 Read block 4029 len 16384
3208 076/300 Flush
3209 077/300 Write block 4235 len 20480 data: 2 2 2 2 2
3210 078/300 Write block 1709 len 8192 data: 4 3
3211 079/300 Read block 1018 len 28672
3212 080/300 Read block 2766 len 16384
3213 081/300 Flush
3214 082/300 Write block 536 len 4096 data: 3
3215 083/300 Write block 2022 len 16384 data: 2 2 3 3
3216 084/300 Write block 1899 len 16384 data: 2 3 3 3
3217 085/300 Read block 1557 len 12288
3218 086/300 Read block 1630 len 36864
3219 087/300 Read block 3597 len 16384
3220 088/300 Write block 3635 len 4096 data: 2
3221 089/300 Write block 4907 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3222 090/300 Read block 652 len 40960
3223 091/300 Read block 3646 len 16384
3224 092/300 Write block 1562 len 28672 data: 2 2 2 2 2 2 3
3225 093/300 Read block 4601 len 40960
3226 094/300 Read block 1354 len 24576
3227 095/300 Read block 4171 len 8192
3228 096/300 Flush
3229 097/300 Read block 2098 len 36864
3230 098/300 Write block 2325 len 36864 data: 2 2 2 2 3 3 3 3 3
3231 099/300 Flush
3232 100/300 Read block 3876 len 40960
3233 101/300 Write block 327 len 4096 data: 2
3234 102/300 Read block 435 len 8192
3235 103/300 Read block 1767 len 40960
3236 104/300 Read block 4978 len 40960
3237 105/300 Read block 1586 len 4096
3238 106/300 Read block 2509 len 24576
3239 107/300 Write block 1823 len 40960 data: 2 2 3 3 3 3 3 3 3 3
3240 108/300 Write block 1834 len 8192 data: 2 2
3241 109/300 Write block 4137 len 32768 data: 2 2 2 2 2 2 2 2
3242 110/300 Read block 889 len 32768
3243 111/300 Read block 3164 len 12288
3244 112/300 Read block 519 len 8192
3245 113/300 Write block 307 len 8192 data: 2 2
3246 114/300 Read block 68 len 20480
3247 115/300 Write block 3901 len 28672 data: 2 3 3 3 3 2 2
3248 116/300 Write block 2044 len 36864 data: 2 2 2 2 2 2 2 2 2
3249 117/300 Read block 883 len 36864
3250 118/300 Read block 513 len 36864
3251 119/300 Flush
3252 120/300 Write block 4208 len 16384 data: 2 2 2 3
3253 121/300 Read block 3515 len 20480
3254 122/300 Write block 1807 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3255 123/300 Write block 3211 len 36864 data: 2 2 2 2 2 2 2 3 3
3256 124/300 Read block 1699 len 32768
3257 125/300 Read block 4251 len 40960
3258 126/300 Write block 1675 len 24576 data: 2 2 2 2 3 3
3259 127/300 Write block 4860 len 28672 data: 3 2 2 2 2 2 2
3260 128/300 Write block 1841 len 28672 data: 2 2 2 2 2 2 2
3261 129/300 Read block 3246 len 16384
3262 130/300 Write block 2360 len 4096 data: 2
3263 131/300 Write block 4033 len 16384 data: 2 2 3 3
3264 132/300 Read block 1708 len 36864
3265 133/300 Read block 2537 len 4096
3266 134/300 Read block 726 len 8192
3267 135/300 Read block 2669 len 12288
3268 136/300 Write block 2957 len 8192 data: 2 2
3269 137/300 Flush
3270 138/300 Write block 1118 len 20480 data: 3 2 2 2 2
3271 139/300 Write block 2363 len 16384 data: 2 2 2 2
3272 140/300 Write block 2465 len 8192 data: 2 3
3273 141/300 Read block 3785 len 36864
3274 142/300 Read block 2377 len 32768
3275 143/300 Write block 1530 len 28672 data: 2 3 3 2 2 2 2
3276 144/300 Read block 4500 len 12288
3277 145/300 Read block 2917 len 40960
3278 146/300 Write block 3896 len 32768 data: 2 2 2 2 2 3 4 4
3279 147/300 Read block 2056 len 24576
3280 148/300 Read block 3233 len 20480
3281 149/300 Read block 4093 len 20480
3282 150/300 Write block 186 len 20480 data: 2 2 2 2 2
3283 151/300 Read block 3759 len 24576
3284 152/300 Flush
3285 153/300 Read block 1996 len 40960
3286 154/300 Read block 1300 len 40960
3287 155/300 Read block 2453 len 32768
3288 156/300 Read block 4740 len 40960
3289 157/300 Read block 568 len 24576
3290 158/300 Read block 1667 len 28672
3291 159/300 Read block 4821 len 20480
3292 160/300 Read block 456 len 12288
3293 161/300 Read block 384 len 16384
3294 162/300 Flush
3295 163/300 Write block 2065 len 32768 data: 3 3 3 3 4 3 2 2
3296 164/300 Read block 4082 len 8192
3297 165/300 Write block 2510 len 40960 data: 2 2 3 3 3 3 3 2 2 2
3298 166/300 Flush
3299 167/300 Read block 2376 len 32768
3300 168/300 Write block 4640 len 12288 data: 2 2 2
3301 169/300 Read block 3387 len 12288
3302 170/300 Write block 1057 len 40960 data: 4 4 4 3 2 2 2 3 2 2
3303 171/300 Read block 2548 len 28672
3304 172/300 Write block 4058 len 32768 data: 2 2 2 2 2 2 2 2
3305 173/300 Flush
3306 174/300 Read block 3377 len 36864
3307 175/300 Write block 2401 len 40960 data: 3 3 2 3 3 3 3 3 2 2
3308 176/300 Read block 1295 len 8192
3309 177/300 Flush
3310 178/300 Read block 2115 len 8192
3311 179/300 Write block 4740 len 8192 data: 3 3
3312 180/300 Write block 3439 len 36864 data: 2 2 2 2 3 3 5 5 5
3313 181/300 Read block 572 len 12288
3314 182/300 Read block 215 len 24576
3315 183/300 Flush
3316 184/300 Read block 3900 len 8192
3317 185/300 Read block 898 len 36864
3318 186/300 Read block 1112 len 20480
3319 187/300 Read block 2272 len 32768
3320 188/300 Write block 4497 len 8192 data: 2 2
3321 189/300 Read block 1894 len 4096
3322 190/300 Read block 1594 len 20480
3323 191/300 Read block 4830 len 8192
3324 192/300 Write block 4575 len 8192 data: 2 2
3325 193/300 Write block 1658 len 40960 data: 2 2 2 2 2 2 3 3 3 2
3326 194/300 Write block 4776 len 40960 data: 2 2 2 3 3 4 3 3 3 3
3327 195/300 Write block 205 len 12288 data: 2 3 3
3328 196/300 Write block 1964 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3329 197/300 Read block 4797 len 20480
3330 198/300 Read block 1207 len 28672
3331 199/300 Read block 704 len 40960
3332 200/300 Flush
3333 201/300 Write block 1046 len 20480 data: 2 2 2 2 2
3334 202/300 Read block 4035 len 20480
3335 203/300 Write block 1821 len 28672 data: 2 2 3 3 4 4 4
3336 204/300 Write block 2398 len 32768 data: 3 3 3 4 4 3 4 4
3337 205/300 Write block 1504 len 20480 data: 3 3 3 2 2
3338 206/300 Flush
3339 207/300 Write block 2472 len 4096 data: 3
3340 208/300 Write block 2852 len 8192 data: 2 2
3341 209/300 Write block 1853 len 36864 data: 2 2 2 2 2 2 2 2 3
3342 210/300 Write block 2828 len 40960 data: 3 3 3 3 3 2 2 3 2 2
3343 211/300 Write block 856 len 20480 data: 2 2 2 2 2
3344 212/300 Write block 624 len 16384 data: 2 2 2 2
3345 213/300 Write block 198 len 36864 data: 2 2 2 2 2 2 2 3 4
3346 214/300 Read block 2194 len 28672
3347 215/300 Read block 3397 len 20480
3348 216/300 Read block 2994 len 24576
3349 217/300 Write block 4177 len 8192 data: 5 5
3350 218/300 Read block 4680 len 24576
3351 219/300 Read block 3223 len 8192
3352 220/300 Read block 537 len 4096
3353 221/300 Read block 4746 len 20480
3354 222/300 Read block 1070 len 8192
3355 223/300 Write block 1157 len 12288 data: 3 3 3
3356 224/300 Write block 4397 len 4096 data: 4
3357 225/300 Read block 1476 len 40960
3358 226/300 Read block 3089 len 4096
3359 227/300 Write block 229 len 28672 data: 2 2 2 2 2 2 2
3360 228/300 Read block 4303 len 24576
3361 229/300 Read block 1721 len 12288
3362 230/300 Write block 1484 len 32768 data: 3 3 3 3 2 2 2 2
3363 231/300 Read block 953 len 16384
3364 232/300 Write block 586 len 4096 data: 3
3365 233/300 Write block 3620 len 16384 data: 3 3 3 3
3366 234/300 Read block 4364 len 20480
3367 235/300 Read block 664 len 28672
3368 236/300 Read block 4496 len 40960
3369 237/300 Flush
3370 238/300 Read block 2565 len 4096
3371 239/300 Write block 4544 len 16384 data: 2 2 2 2
3372 240/300 Read block 4898 len 4096
3373 241/300 Write block 4730 len 28672 data: 2 2 2 3 3 3 3
3374 242/300 Write block 4693 len 8192 data: 3 3
3375 243/300 Read block 2915 len 32768
3376 244/300 Read block 4214 len 40960
3377 245/300 Read block 3259 len 24576
3378 246/300 Write block 462 len 28672 data: 2 2 2 2 2 2 2
3379 247/300 Write block 26 len 16384 data: 3 3 3 3
3380 248/300 Write block 3496 len 36864 data: 3 3 3 3 3 3 3 3 2
3381 249/300 Read block 2460 len 24576
3382 250/300 Read block 4928 len 32768
3383 251/300 Write block 2322 len 4096 data: 2
3384 252/300 Write block 2490 len 36864 data: 2 2 2 2 2 2 2 2 2
3385 253/300 Write block 121 len 8192 data: 2 2
3386 254/300 Read block 1978 len 16384
3387 255/300 Read block 4506 len 28672
3388 256/300 Flush
3389 257/300 Flush
3390 258/300 Read block 1344 len 12288
3391 259/300 Write block 2600 len 4096 data: 3
3392 260/300 Write block 2757 len 4096 data: 2
3393 261/300 Read block 3672 len 8192
3394 262/300 Write block 2707 len 20480 data: 4 4 2 2 2
3395 263/300 Read block 4241 len 8192
3396 264/300 Write block 4760 len 36864 data: 2 2 3 3 3 3 3 3 3
3397 265/300 Read block 745 len 16384
3398 266/300 Write block 3406 len 20480 data: 2 2 2 2 2
3399 267/300 Write block 1168 len 12288 data: 2 3 3
3400 268/300 Read block 3231 len 8192
3401 269/300 Write block 1147 len 12288 data: 2 2 2
3402 270/300 Write block 2050 len 28672 data: 3 3 3 2 2 2 2
3403 271/300 Write block 871 len 40960 data: 2 2 2 2 2 2 2 2 2 3
3404 272/300 Write block 2376 len 16384 data: 3 3 3 3
3405 273/300 Flush
3406 274/300 Write block 4531 len 36864 data: 3 4 3 3 3 3 3 3 2
3407 275/300 Write block 3908 len 28672 data: 2 2 2 2 2 2 2
3408 276/300 Read block 2599 len 24576
3409 277/300 Write block 3124 len 8192 data: 2 2
3410 278/300 Read block 902 len 24576
3411 279/300 Read block 3591 len 28672
3412 280/300 Read block 229 len 8192
3413 281/300 Read block 4901 len 16384
3414 282/300 Write block 1239 len 4096 data: 2
3415 283/300 Read block 581 len 12288
3416 284/300 Write block 3788 len 4096 data: 2
3417 285/300 Write block 3361 len 4096 data: 2
3418 286/300 Write block 1900 len 28672 data: 4 4 4 3 3 2 2
3419 287/300 Write block 1835 len 28672 data: 3 2 2 2 2 2 3
3420 288/300 Write block 149 len 36864 data: 3 2 2 2 2 2 2 2 2
3421 289/300 Write block 880 len 36864 data: 4 3 3 2 2 2 3 3 4
3422 290/300 Read block 2061 len 16384
3423 291/300 Read block 442 len 16384
3424 292/300 Read block 4259 len 8192
3425 293/300 Read block 3602 len 40960
3426 294/300 Write block 3807 len 8192 data: 3 3
3427 295/300 Read block 3313 len 28672
3428 296/300 Read block 3993 len 40960
3429 297/300 Read block 4683 len 24576
3430 298/300 Read block 3433 len 36864
3431 299/300 Read block 3507 len 20480
3432 300/300 Read block 2770 len 20480
3433 301/300 Read block 3492 len 36864
3434 Main task has work to do, go find it
3435 got dsc Start(1)
3436 start 1
3437 Jan 17 21:11:49.337 INFO request completed, latency_us: 205, response_code: 204, uri: /start/cid/1, method: POST, req_id: 3213aba1-d341-4ee6-be8e-905cc5a765e3, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
3438 Replay: started 1, returned:Ok(())
3439 ----------------------------------------------------------------
3440 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
3441 States: Active Offline Active
3442 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3443 1905 Acked Flush 0 Done Sent Done false
3444 1906 Acked Flush 0 Done Sent Done false
3445 1907 Acked Write 1 Done Sent Done false
3446 1908 Acked Write 10 Done Sent Done false
3447 1909 Acked Flush 0 Done Sent Done false
3448 1910 Acked Flush 0 Done Sent Done false
3449 1911 Acked Read 9 Done Sent Done false
3450 1912 Acked Read 1 Done Sent Done false
3451 1913 Acked Write 3 Done Sent Done false
3452 1914 Acked Flush 0 Done Sent Done false
3453 1915 Acked Read 7 Done Sent Done false
3454 1916 Acked Read 5 Done Sent Done false
3455 1917 Acked Read 5 Done Sent Done false
3456 1918 Acked Write 5 Done Sent Done false
3457 1919 Acked Flush 0 Done Sent Done false
3458 1920 Acked Write 8 Done Sent Done false
3459 1921 Acked Write 2 Done Sent Done false
3460 1922 Acked Write 1 Done Sent Done false
3461 1923 Acked Read 6 Done Sent Done false
3462 1924 Acked Read 3 Done Sent Done false
3463 1925 Acked Read 1 Done Sent Done false
3464 1926 Acked Read 9 Done Sent Done false
3465 1927 Acked Write 5 Done Sent Done false
3466 1928 Acked Write 5 Done Sent Done false
3467 1929 Acked Read 5 Done Sent Done false
3468 1930 Acked Read 8 Done Sent Done false
3469 1931 Acked Flush 0 Done Sent Done false
3470 1932 Acked Read 10 Done Sent Done false
3471 1933 Acked Flush 0 Done Sent Done false
3472 1934 Acked Read 4 Done Sent Done false
3473 1935 Acked Write 8 Done Sent Done false
3474 1936 Acked Write 5 Done Sent Done false
3475 1937 Acked Write 2 Done Sent Done false
3476 1938 Acked Read 5 Done Sent Done false
3477 1939 Acked Write 7 Done Sent Done false
3478 1940 Acked Write 6 Done Sent Done false
3479 1941 Acked Write 4 Done Sent Done false
3480 1942 Acked Write 5 Done Sent Done false
3481 1943 Acked Read 6 Done Sent Done false
3482 1944 Acked Write 10 Done Sent Done false
3483 1945 Acked Write 8 Done Sent Done false
3484 1946 Acked Write 4 Done Sent Done false
3485 1947 Acked Read 9 Done Sent Done false
3486 1948 Acked Write 8 Done Sent Done false
3487 1949 Acked Read 8 Done Sent Done false
3488 1950 Acked Read 5 Done Sent Done false
3489 1951 Acked Read 8 Done Sent Done false
3490 1952 Acked Read 9 Done Sent Done false
3491 1953 Acked Read 9 Done Sent Done false
3492 1954 Acked Read 5 Done Sent Done false
3493 1955 Acked Write 2 Done Sent Done false
3494 1956 Acked Flush 0 Done Sent Done false
3495 1957 Acked Read 2 Done Sent Done false
3496 1958 Acked Write 7 Done Sent Done false
3497 1959 Acked Flush 0 Done Sent Done false
3498 1960 Acked Read 1 Done Sent Done false
3499 1961 Acked Write 2 Done Sent Done false
3500 1962 Acked Read 3 Done Sent Done false
3501 1963 Acked Flush 0 Done Sent Done false
3502 1964 Acked Read 3 Done Sent Done false
3503 1965 Acked Read 4 Done Sent Done false
3504 1966 Acked Write 1 Done Sent Done false
3505 1967 Acked Write 3 Done Sent Done false
3506 1968 Acked Flush 0 Done Sent Done false
3507 1969 Acked Write 8 Done Sent Done false
3508 1970 Acked Read 1 Done Sent Done false
3509 1971 Acked Write 9 Done Sent Done false
3510 1972 Acked Flush 0 Done Sent Done false
3511 1973 Acked Write 2 Done Sent Done false
3512 1974 Acked Read 1 Done Sent Done false
3513 1975 Acked Write 10 Done Sent Done false
3514 1976 Acked Write 2 Done Sent Done false
3515 1977 Acked Read 3 Done Sent Done false
3516 1978 Acked Read 6 Done Sent Done false
3517 1979 Acked Read 4 Done Sent Done false
3518 1980 Acked Flush 0 Done Sent Done false
3519 1981 Acked Write 5 Done Sent Done false
3520 1982 Acked Write 2 Done Sent Done false
3521 1983 Acked Read 7 Done Sent Done false
3522 1984 Acked Read 4 Done Sent Done false
3523 1985 Acked Flush 0 Done Sent Done false
3524 1986 Acked Write 1 Done Sent Done false
3525 1987 Acked Write 4 Done Sent Done false
3526 1988 Acked Write 4 Done Sent Done false
3527 1989 Acked Read 3 Done Sent Done false
3528 1990 Acked Read 9 Done Sent Done false
3529 1991 Acked Read 4 Done Sent Done false
3530 1992 Acked Write 1 Done Sent Done false
3531 1993 Acked Write 10 Done Sent Done false
3532 1994 Acked Read 10 Done Sent Done false
3533 1995 Acked Read 4 Done Sent Done false
3534 1996 Acked Write 7 Done Sent Done false
3535 1997 Acked Read 10 Done Sent Done false
3536 1998 Acked Read 6 Done Sent Done false
3537 1999 Acked Read 2 Done Sent Done false
3538 2000 Acked Flush 0 Done Sent Done false
3539 2001 Acked Read 9 Done Sent Done false
3540 2002 Acked Write 9 Done Sent Done false
3541 2003 Acked Flush 0 Done Sent Done false
3542 2004 Acked Read 10 Done Sent Done false
3543 2005 Acked Write 1 Done Sent Done false
3544 2006 Acked Read 2 Done Sent Done false
3545 2007 Acked Read 10 Done Sent Done false
3546 2008 Acked Read 10 Done Sent Done false
3547 2009 Acked Read 1 Done Sent Done false
3548 2010 Acked Read 6 Done Sent Done false
3549 2011 Acked Write 10 Done Sent Done false
3550 2012 Acked Write 2 Done Sent Done false
3551 2013 Acked Write 8 Done Sent Done false
3552 2014 Acked Read 8 Done Sent Done false
3553 2015 Acked Read 3 Done Sent Done false
3554 2016 Acked Read 2 Done Sent Done false
3555 2017 Acked Write 2 Done Sent Done false
3556 2018 Acked Read 5 Done Sent Done false
3557 2019 Acked Write 7 Done Sent Done false
3558 2020 Acked Write 9 Done Sent Done false
3559 2021 Acked Read 9 Done Sent Done false
3560 2022 Acked Read 9 Done Sent Done false
3561 2023 Acked Flush 0 Done Sent Done false
3562 2024 Acked Write 4 Done Sent Done false
3563 2025 Acked Read 5 Done Sent Done false
3564 2026 Acked Write 10 Done Sent Done false
3565 2027 Acked Write 9 Done Sent Done false
3566 2028 Acked Read 8 Done Sent Done false
3567 2029 Acked Read 10 Done Sent Done false
3568 2030 Acked Write 6 Done Sent Done false
3569 2031 Acked Write 7 Done Sent Done false
3570 2032 Acked Write 7 Done Sent Done false
3571 2033 Acked Read 4 Done Sent Done false
3572 2034 Acked Write 1 Done Sent Done false
3573 2035 Acked Write 4 Done Sent Done false
3574 2036 Acked Read 9 Done Sent Done false
3575 2037 Acked Read 1 Done Sent Done false
3576 2038 Acked Read 2 Done Sent Done false
3577 2039 Acked Read 3 Done Sent Done false
3578 2040 Acked Write 2 Done Sent Done false
3579 2041 Acked Flush 0 Done Sent Done false
3580 2042 Acked Write 5 Done Sent Done false
3581 2043 Acked Write 4 Done Sent Done false
3582 2044 Acked Write 2 Done Sent Done false
3583 2045 Acked Read 9 Done Sent Done false
3584 2046 Acked Read 8 Done Sent Done false
3585 2047 Acked Write 7 Done Sent Done false
3586 2048 Acked Read 3 Done Sent Done false
3587 2049 Acked Read 10 Done Sent Done false
3588 2050 Acked Write 8 Done Sent Done false
3589 2051 Acked Read 6 Done Sent Done false
3590 2052 Acked Read 5 Done Sent Done false
3591 2053 Acked Read 5 Done Sent Done false
3592 2054 Acked Write 5 Done Sent Done false
3593 2055 Acked Read 6 Done Sent Done false
3594 2056 Acked Flush 0 Done Sent Done false
3595 2057 Acked Read 10 Done Sent Done false
3596 2058 Acked Read 10 Done Sent Done false
3597 2059 Acked Read 8 Done Sent Done false
3598 2060 Acked Read 10 Done Sent Done false
3599 2061 Acked Read 6 Done Sent Done false
3600 2062 Acked Read 7 Done Sent Done false
3601 2063 Acked Read 5 Done Sent Done false
3602 2064 Acked Read 3 Done Sent Done false
3603 2065 Acked Read 4 Done Sent Done false
3604 2066 Acked Flush 0 Done Sent Done false
3605 2067 Acked Write 8 Done Sent Done false
3606 2068 Acked Read 2 Done Sent Done false
3607 2069 Acked Write 10 Done Sent Done false
3608 2070 Acked Flush 0 Done Sent Done false
3609 2071 Acked Read 8 Done Sent Done false
3610 2072 Acked Write 3 Done Sent Done false
3611 2073 Acked Read 3 Done Sent Done false
3612 2074 Acked Write 10 Done Sent Done false
3613 2075 Acked Read 7 Done Sent Done false
3614 2076 Acked Write 8 Done Sent Done false
3615 2077 Acked Flush 0 Done Sent Done false
3616 2078 Acked Read 9 Done Sent Done false
3617 2079 Acked Write 10 Done Sent Done false
3618 2080 Acked Read 2 Done Sent Done false
3619 2081 Acked Flush 0 Done Sent Done false
3620 2082 Acked Read 2 Done Sent Done false
3621 2083 Acked Write 2 Done Sent Done false
3622 2084 Acked Write 9 Done Sent Done false
3623 2085 Acked Read 3 Done Sent Done false
3624 2086 Acked Read 6 Done Sent Done false
3625 2087 Acked Flush 0 Done Sent Done false
3626 2088 Acked Read 2 Done Sent Done false
3627 2089 Acked Read 9 Done Sent Done false
3628 2090 Acked Read 5 Done Sent Done false
3629 2091 Acked Read 8 Done Sent Done false
3630 2092 Acked Write 2 Done Sent Done false
3631 2093 Acked Read 1 Done Sent Done false
3632 2094 Acked Read 5 Done Sent Done false
3633 2095 Acked Read 2 Done Sent Done false
3634 2096 Acked Write 2 Done Sent Done false
3635 2097 Acked Write 10 Done Sent Done false
3636 2098 Acked Write 10 Done Sent Done false
3637 2099 Acked Write 3 Done Sent Done false
3638 2100 Acked Write 10 Done Sent Done false
3639 2101 Acked Read 5 Done Sent Done false
3640 2102 Acked Read 7 Done Sent Done false
3641 2103 Acked Read 10 Done Sent Done false
3642 2104 Acked Flush 0 Done Sent Done false
3643 2105 Acked Write 5 Done Sent Done false
3644 2106 Acked Read 5 Done Sent Done false
3645 2107 Acked Write 7 Done Sent Done false
3646 2108 Acked Write 8 Done Sent Done false
3647 2109 Acked Write 5 Done Sent Done false
3648 2110 Acked Flush 0 Done Sent Done false
3649 2111 Acked Write 1 Done Sent Done false
3650 2112 Acked Write 2 Done Sent Done false
3651 2113 Acked Write 9 Done Sent Done false
3652 2114 Acked Write 10 Done Sent Done false
3653 2115 Acked Write 5 Done Sent Done false
3654 2116 Acked Write 4 Done Sent Done false
3655 2117 Acked Write 9 Done Sent Done false
3656 2118 Acked Read 7 Done Sent Done false
3657 2119 Acked Read 5 Done Sent Done false
3658 2120 Acked Read 6 Done Sent Done false
3659 2121 Acked Write 2 Done Sent Done false
3660 2122 Acked Read 6 Done Sent Done false
3661 2123 Acked Read 2 Done Sent Done false
3662 2124 Acked Read 1 Done Sent Done false
3663 2125 Acked Read 5 Done Sent Done false
3664 2126 Acked Read 2 Done Sent Done false
3665 2127 Acked Write 3 Done Sent Done false
3666 2128 Acked Write 1 Done Sent Done false
3667 2129 Acked Read 10 Done Sent Done false
3668 2130 Acked Read 1 Done Sent Done false
3669 2131 Acked Write 7 Done Sent Done false
3670 2132 Acked Read 6 Done Sent Done false
3671 2133 Acked Read 3 Done Sent Done false
3672 2134 Acked Write 8 Done Sent Done false
3673 2135 Acked Read 4 Done Sent Done false
3674 2136 Acked Write 1 Done Sent Done false
3675 2137 Acked Write 4 Done Sent Done false
3676 2138 Acked Read 5 Done Sent Done false
3677 2139 Acked Read 7 Done Sent Done false
3678 2140 Acked Read 10 Done Sent Done false
3679 2141 Acked Flush 0 Done Sent Done false
3680 2142 Acked Read 1 Done Sent Done false
3681 2143 Acked Write 4 Done Sent Done false
3682 2144 Acked Read 1 Done Sent Done false
3683 2145 Acked Write 7 Done Sent Done false
3684 2146 Acked Write 2 Done Sent Done false
3685 2147 Acked Read 8 Done Sent Done false
3686 2148 Acked Read 10 Done Sent Done false
3687 2149 Acked Read 6 Done Sent Done false
3688 2150 Acked Write 7 Done Sent Done false
3689 2151 Acked Write 4 Done Sent Done false
3690 2152 Acked Write 9 Done Sent Done false
3691 2153 Acked Read 6 Done Sent Done false
3692 2154 Acked Read 8 Done Sent Done false
3693 2155 Acked Write 1 Done Sent Done false
3694 2156 Acked Write 9 Done Sent Done false
3695 2157 Acked Write 2 Done Sent Done false
3696 2158 Acked Read 4 Done Sent Done false
3697 2159 Acked Read 7 Done Sent Done false
3698 2160 Acked Flush 0 Done Sent Done false
3699 2161 Acked Flush 0 Done Sent Done false
3700 2162 Acked Read 3 Done Sent Done false
3701 2163 Acked Write 1 Done Sent Done false
3702 2164 Acked Write 1 Done Sent Done false
3703 2165 Acked Read 2 Done Sent Done false
3704 2166 Acked Write 5 Done Sent Done false
3705 2167 Acked Read 2 Done Sent Done false
3706 2168 Acked Write 9 Done Sent Done false
3707 2169 Acked Read 4 Done Sent Done false
3708 2170 Acked Write 5 Done Sent Done false
3709 2171 Acked Write 3 Done Sent Done false
3710 2172 Acked Read 2 Done Sent Done false
3711 2173 Acked Write 3 Done Sent Done false
3712 2174 Acked Write 7 Done Sent Done false
3713 2175 Acked Write 10 Done Sent Done false
3714 2176 Acked Write 4 Done Sent Done false
3715 2177 Acked Flush 0 Done Sent Done false
3716 2178 Acked Write 9 Done Sent Done false
3717 2179 Acked Write 7 Done Sent Done false
3718 2180 Acked Read 6 Done Sent Done false
3719 2181 Acked Write 2 Done Sent Done false
3720 2182 Acked Read 6 Done Sent Done false
3721 2183 Acked Read 7 Done Sent Done false
3722 2184 Acked Read 2 Done Sent Done false
3723 2185 Acked Read 4 Done Sent Done false
3724 2186 Acked Write 1 Done Sent Done false
3725 2187 Acked Read 3 Done Sent Done false
3726 2188 Acked Write 1 Done Sent Done false
3727 2189 Acked Write 1 Done Sent Done false
3728 2190 Acked Write 7 Done Sent Done false
3729 2191 Acked Write 7 Done Sent Done false
3730 2192 Acked Write 9 Done Sent Done false
3731 2193 Acked Write 9 Done Sent Done false
3732 2194 Acked Read 4 Done Sent Done false
3733 2195 Acked Read 4 Done Sent Done false
3734 2196 Acked Read 2 Done Sent Done false
3735 2197 Acked Read 10 Done Sent Done false
3736 2198 Acked Write 2 Done Sent Done false
3737 2199 Acked Read 7 Done Sent Done false
3738 2200 Acked Read 10 Done Sent Done false
3739 2201 Acked Read 6 Done Sent Done false
3740 2202 Acked Read 9 Done Sent Done false
3741 2203 Acked Read 5 Done Sent Done false
3742 2204 Acked Read 5 Done Sent Done false
3743 2205 Acked Read 9 Done Sent Done false
3744 STATES DS:0 DS:1 DS:2 TOTAL
3745 Sent 0 301 0 301
3746 Done 301 0 301 602
3747 Skipped 0 0 0 0
3748 Error 0 0 0 0
3749 Last Flush: 2177 1904 2177
3750 Downstairs last five completed: 1904 1903 1902 1901 1900
3751 Upstairs last five completed: 2205 2204 2203 2202 2201
3752 CLIENT: Up:0 ds:301 act:2
3753 [8820] got start action
3754 [8820] I am going to restart
3755 Starting downstairs at port 8820
3756 Make output file at "/tmp/dsc/downstairs-8820.txt"
3757 [8820][1] reports Starting
3758 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1204)
3759 [8820][1] reports Running
3760 ----------------------------------------------------------------
3761 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
3762 States: Active Offline Active
3763 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3764 1905 Acked Flush 0 Done Sent Done false
3765 1906 Acked Flush 0 Done Sent Done false
3766 1907 Acked Write 1 Done Sent Done false
3767 1908 Acked Write 10 Done Sent Done false
3768 1909 Acked Flush 0 Done Sent Done false
3769 1910 Acked Flush 0 Done Sent Done false
3770 1911 Acked Read 9 Done Sent Done false
3771 1912 Acked Read 1 Done Sent Done false
3772 1913 Acked Write 3 Done Sent Done false
3773 1914 Acked Flush 0 Done Sent Done false
3774 1915 Acked Read 7 Done Sent Done false
3775 1916 Acked Read 5 Done Sent Done false
3776 1917 Acked Read 5 Done Sent Done false
3777 1918 Acked Write 5 Done Sent Done false
3778 1919 Acked Flush 0 Done Sent Done false
3779 1920 Acked Write 8 Done Sent Done false
3780 1921 Acked Write 2 Done Sent Done false
3781 1922 Acked Write 1 Done Sent Done false
3782 1923 Acked Read 6 Done Sent Done false
3783 1924 Acked Read 3 Done Sent Done false
3784 1925 Acked Read 1 Done Sent Done false
3785 1926 Acked Read 9 Done Sent Done false
3786 1927 Acked Write 5 Done Sent Done false
3787 1928 Acked Write 5 Done Sent Done false
3788 1929 Acked Read 5 Done Sent Done false
3789 1930 Acked Read 8 Done Sent Done false
3790 1931 Acked Flush 0 Done Sent Done false
3791 1932 Acked Read 10 Done Sent Done false
3792 1933 Acked Flush 0 Done Sent Done false
3793 1934 Acked Read 4 Done Sent Done false
3794 1935 Acked Write 8 Done Sent Done false
3795 1936 Acked Write 5 Done Sent Done false
3796 1937 Acked Write 2 Done Sent Done false
3797 1938 Acked Read 5 Done Sent Done false
3798 1939 Acked Write 7 Done Sent Done false
3799 1940 Acked Write 6 Done Sent Done false
3800 1941 Acked Write 4 Done Sent Done false
3801 1942 Acked Write 5 Done Sent Done false
3802 1943 Acked Read 6 Done Sent Done false
3803 1944 Acked Write 10 Done Sent Done false
3804 1945 Acked Write 8 Done Sent Done false
3805 1946 Acked Write 4 Done Sent Done false
3806 1947 Acked Read 9 Done Sent Done false
3807 1948 Acked Write 8 Done Sent Done false
3808 1949 Acked Read 8 Done Sent Done false
3809 1950 Acked Read 5 Done Sent Done false
3810 1951 Acked Read 8 Done Sent Done false
3811 1952 Acked Read 9 Done Sent Done false
3812 1953 Acked Read 9 Done Sent Done false
3813 1954 Acked Read 5 Done Sent Done false
3814 1955 Acked Write 2 Done Sent Done false
3815 1956 Acked Flush 0 Done Sent Done false
3816 1957 Acked Read 2 Done Sent Done false
3817 1958 Acked Write 7 Done Sent Done false
3818 1959 Acked Flush 0 Done Sent Done false
3819 1960 Acked Read 1 Done Sent Done false
3820 1961 Acked Write 2 Done Sent Done false
3821 1962 Acked Read 3 Done Sent Done false
3822 1963 Acked Flush 0 Done Sent Done false
3823 1964 Acked Read 3 Done Sent Done false
3824 1965 Acked Read 4 Done Sent Done false
3825 1966 Acked Write 1 Done Sent Done false
3826 1967 Acked Write 3 Done Sent Done false
3827 1968 Acked Flush 0 Done Sent Done false
3828 1969 Acked Write 8 Done Sent Done false
3829 1970 Acked Read 1 Done Sent Done false
3830 1971 Acked Write 9 Done Sent Done false
3831 1972 Acked Flush 0 Done Sent Done false
3832 1973 Acked Write 2 Done Sent Done false
3833 1974 Acked Read 1 Done Sent Done false
3834 1975 Acked Write 10 Done Sent Done false
3835 1976 Acked Write 2 Done Sent Done false
3836 1977 Acked Read 3 Done Sent Done false
3837 1978 Acked Read 6 Done Sent Done false
3838 1979 Acked Read 4 Done Sent Done false
3839 1980 Acked Flush 0 Done Sent Done false
3840 1981 Acked Write 5 Done Sent Done false
3841 1982 Acked Write 2 Done Sent Done false
3842 1983 Acked Read 7 Done Sent Done false
3843 1984 Acked Read 4 Done Sent Done false
3844 1985 Acked Flush 0 Done Sent Done false
3845 1986 Acked Write 1 Done Sent Done false
3846 1987 Acked Write 4 Done Sent Done false
3847 1988 Acked Write 4 Done Sent Done false
3848 1989 Acked Read 3 Done Sent Done false
3849 1990 Acked Read 9 Done Sent Done false
3850 1991 Acked Read 4 Done Sent Done false
3851 1992 Acked Write 1 Done Sent Done false
3852 1993 Acked Write 10 Done Sent Done false
3853 1994 Acked Read 10 Done Sent Done false
3854 1995 Acked Read 4 Done Sent Done false
3855 1996 Acked Write 7 Done Sent Done false
3856 1997 Acked Read 10 Done Sent Done false
3857 1998 Acked Read 6 Done Sent Done false
3858 1999 Acked Read 2 Done Sent Done false
3859 2000 Acked Flush 0 Done Sent Done false
3860 2001 Acked Read 9 Done Sent Done false
3861 2002 Acked Write 9 Done Sent Done false
3862 2003 Acked Flush 0 Done Sent Done false
3863 2004 Acked Read 10 Done Sent Done false
3864 2005 Acked Write 1 Done Sent Done false
3865 2006 Acked Read 2 Done Sent Done false
3866 2007 Acked Read 10 Done Sent Done false
3867 2008 Acked Read 10 Done Sent Done false
3868 2009 Acked Read 1 Done Sent Done false
3869 2010 Acked Read 6 Done Sent Done false
3870 2011 Acked Write 10 Done Sent Done false
3871 2012 Acked Write 2 Done Sent Done false
3872 2013 Acked Write 8 Done Sent Done false
3873 2014 Acked Read 8 Done Sent Done false
3874 2015 Acked Read 3 Done Sent Done false
3875 2016 Acked Read 2 Done Sent Done false
3876 2017 Acked Write 2 Done Sent Done false
3877 2018 Acked Read 5 Done Sent Done false
3878 2019 Acked Write 7 Done Sent Done false
3879 2020 Acked Write 9 Done Sent Done false
3880 2021 Acked Read 9 Done Sent Done false
3881 2022 Acked Read 9 Done Sent Done false
3882 2023 Acked Flush 0 Done Sent Done false
3883 2024 Acked Write 4 Done Sent Done false
3884 2025 Acked Read 5 Done Sent Done false
3885 2026 Acked Write 10 Done Sent Done false
3886 2027 Acked Write 9 Done Sent Done false
3887 2028 Acked Read 8 Done Sent Done false
3888 2029 Acked Read 10 Done Sent Done false
3889 2030 Acked Write 6 Done Sent Done false
3890 2031 Acked Write 7 Done Sent Done false
3891 2032 Acked Write 7 Done Sent Done false
3892 2033 Acked Read 4 Done Sent Done false
3893 2034 Acked Write 1 Done Sent Done false
3894 2035 Acked Write 4 Done Sent Done false
3895 2036 Acked Read 9 Done Sent Done false
3896 2037 Acked Read 1 Done Sent Done false
3897 2038 Acked Read 2 Done Sent Done false
3898 2039 Acked Read 3 Done Sent Done false
3899 2040 Acked Write 2 Done Sent Done false
3900 2041 Acked Flush 0 Done Sent Done false
3901 2042 Acked Write 5 Done Sent Done false
3902 2043 Acked Write 4 Done Sent Done false
3903 2044 Acked Write 2 Done Sent Done false
3904 2045 Acked Read 9 Done Sent Done false
3905 2046 Acked Read 8 Done Sent Done false
3906 2047 Acked Write 7 Done Sent Done false
3907 2048 Acked Read 3 Done Sent Done false
3908 2049 Acked Read 10 Done Sent Done false
3909 2050 Acked Write 8 Done Sent Done false
3910 2051 Acked Read 6 Done Sent Done false
3911 2052 Acked Read 5 Done Sent Done false
3912 2053 Acked Read 5 Done Sent Done false
3913 2054 Acked Write 5 Done Sent Done false
3914 2055 Acked Read 6 Done Sent Done false
3915 2056 Acked Flush 0 Done Sent Done false
3916 2057 Acked Read 10 Done Sent Done false
3917 2058 Acked Read 10 Done Sent Done false
3918 2059 Acked Read 8 Done Sent Done false
3919 2060 Acked Read 10 Done Sent Done false
3920 2061 Acked Read 6 Done Sent Done false
3921 2062 Acked Read 7 Done Sent Done false
3922 2063 Acked Read 5 Done Sent Done false
3923 2064 Acked Read 3 Done Sent Done false
3924 2065 Acked Read 4 Done Sent Done false
3925 2066 Acked Flush 0 Done Sent Done false
3926 2067 Acked Write 8 Done Sent Done false
3927 2068 Acked Read 2 Done Sent Done false
3928 2069 Acked Write 10 Done Sent Done false
3929 2070 Acked Flush 0 Done Sent Done false
3930 2071 Acked Read 8 Done Sent Done false
3931 2072 Acked Write 3 Done Sent Done false
3932 2073 Acked Read 3 Done Sent Done false
3933 2074 Acked Write 10 Done Sent Done false
3934 2075 Acked Read 7 Done Sent Done false
3935 2076 Acked Write 8 Done Sent Done false
3936 2077 Acked Flush 0 Done Sent Done false
3937 2078 Acked Read 9 Done Sent Done false
3938 2079 Acked Write 10 Done Sent Done false
3939 2080 Acked Read 2 Done Sent Done false
3940 2081 Acked Flush 0 Done Sent Done false
3941 2082 Acked Read 2 Done Sent Done false
3942 2083 Acked Write 2 Done Sent Done false
3943 2084 Acked Write 9 Done Sent Done false
3944 2085 Acked Read 3 Done Sent Done false
3945 2086 Acked Read 6 Done Sent Done false
3946 2087 Acked Flush 0 Done Sent Done false
3947 2088 Acked Read 2 Done Sent Done false
3948 2089 Acked Read 9 Done Sent Done false
3949 2090 Acked Read 5 Done Sent Done false
3950 2091 Acked Read 8 Done Sent Done false
3951 2092 Acked Write 2 Done Sent Done false
3952 2093 Acked Read 1 Done Sent Done false
3953 2094 Acked Read 5 Done Sent Done false
3954 2095 Acked Read 2 Done Sent Done false
3955 2096 Acked Write 2 Done Sent Done false
3956 2097 Acked Write 10 Done Sent Done false
3957 2098 Acked Write 10 Done Sent Done false
3958 2099 Acked Write 3 Done Sent Done false
3959 2100 Acked Write 10 Done Sent Done false
3960 2101 Acked Read 5 Done Sent Done false
3961 2102 Acked Read 7 Done Sent Done false
3962 2103 Acked Read 10 Done Sent Done false
3963 2104 Acked Flush 0 Done Sent Done false
3964 2105 Acked Write 5 Done Sent Done false
3965 2106 Acked Read 5 Done Sent Done false
3966 2107 Acked Write 7 Done Sent Done false
3967 2108 Acked Write 8 Done Sent Done false
3968 2109 Acked Write 5 Done Sent Done false
3969 2110 Acked Flush 0 Done Sent Done false
3970 2111 Acked Write 1 Done Sent Done false
3971 2112 Acked Write 2 Done Sent Done false
3972 2113 Acked Write 9 Done Sent Done false
3973 2114 Acked Write 10 Done Sent Done false
3974 2115 Acked Write 5 Done Sent Done false
3975 2116 Acked Write 4 Done Sent Done false
3976 2117 Acked Write 9 Done Sent Done false
3977 2118 Acked Read 7 Done Sent Done false
3978 2119 Acked Read 5 Done Sent Done false
3979 2120 Acked Read 6 Done Sent Done false
3980 2121 Acked Write 2 Done Sent Done false
3981 2122 Acked Read 6 Done Sent Done false
3982 2123 Acked Read 2 Done Sent Done false
3983 2124 Acked Read 1 Done Sent Done false
3984 2125 Acked Read 5 Done Sent Done false
3985 2126 Acked Read 2 Done Sent Done false
3986 2127 Acked Write 3 Done Sent Done false
3987 2128 Acked Write 1 Done Sent Done false
3988 2129 Acked Read 10 Done Sent Done false
3989 2130 Acked Read 1 Done Sent Done false
3990 2131 Acked Write 7 Done Sent Done false
3991 2132 Acked Read 6 Done Sent Done false
3992 2133 Acked Read 3 Done Sent Done false
3993 2134 Acked Write 8 Done Sent Done false
3994 2135 Acked Read 4 Done Sent Done false
3995 2136 Acked Write 1 Done Sent Done false
3996 2137 Acked Write 4 Done Sent Done false
3997 2138 Acked Read 5 Done Sent Done false
3998 2139 Acked Read 7 Done Sent Done false
3999 2140 Acked Read 10 Done Sent Done false
4000 2141 Acked Flush 0 Done Sent Done false
4001 2142 Acked Read 1 Done Sent Done false
4002 2143 Acked Write 4 Done Sent Done false
4003 2144 Acked Read 1 Done Sent Done false
4004 2145 Acked Write 7 Done Sent Done false
4005 2146 Acked Write 2 Done Sent Done false
4006 2147 Acked Read 8 Done Sent Done false
4007 2148 Acked Read 10 Done Sent Done false
4008 2149 Acked Read 6 Done Sent Done false
4009 2150 Acked Write 7 Done Sent Done false
4010 2151 Acked Write 4 Done Sent Done false
4011 2152 Acked Write 9 Done Sent Done false
4012 2153 Acked Read 6 Done Sent Done false
4013 2154 Acked Read 8 Done Sent Done false
4014 2155 Acked Write 1 Done Sent Done false
4015 2156 Acked Write 9 Done Sent Done false
4016 2157 Acked Write 2 Done Sent Done false
4017 2158 Acked Read 4 Done Sent Done false
4018 2159 Acked Read 7 Done Sent Done false
4019 2160 Acked Flush 0 Done Sent Done false
4020 2161 Acked Flush 0 Done Sent Done false
4021 2162 Acked Read 3 Done Sent Done false
4022 2163 Acked Write 1 Done Sent Done false
4023 2164 Acked Write 1 Done Sent Done false
4024 2165 Acked Read 2 Done Sent Done false
4025 2166 Acked Write 5 Done Sent Done false
4026 2167 Acked Read 2 Done Sent Done false
4027 2168 Acked Write 9 Done Sent Done false
4028 2169 Acked Read 4 Done Sent Done false
4029 2170 Acked Write 5 Done Sent Done false
4030 2171 Acked Write 3 Done Sent Done false
4031 2172 Acked Read 2 Done Sent Done false
4032 2173 Acked Write 3 Done Sent Done false
4033 2174 Acked Write 7 Done Sent Done false
4034 2175 Acked Write 10 Done Sent Done false
4035 2176 Acked Write 4 Done Sent Done false
4036 2177 Acked Flush 0 Done Sent Done false
4037 2178 Acked Write 9 Done Sent Done false
4038 2179 Acked Write 7 Done Sent Done false
4039 2180 Acked Read 6 Done Sent Done false
4040 2181 Acked Write 2 Done Sent Done false
4041 2182 Acked Read 6 Done Sent Done false
4042 2183 Acked Read 7 Done Sent Done false
4043 2184 Acked Read 2 Done Sent Done false
4044 2185 Acked Read 4 Done Sent Done false
4045 2186 Acked Write 1 Done Sent Done false
4046 2187 Acked Read 3 Done Sent Done false
4047 2188 Acked Write 1 Done Sent Done false
4048 2189 Acked Write 1 Done Sent Done false
4049 2190 Acked Write 7 Done Sent Done false
4050 2191 Acked Write 7 Done Sent Done false
4051 2192 Acked Write 9 Done Sent Done false
4052 2193 Acked Write 9 Done Sent Done false
4053 2194 Acked Read 4 Done Sent Done false
4054 2195 Acked Read 4 Done Sent Done false
4055 2196 Acked Read 2 Done Sent Done false
4056 2197 Acked Read 10 Done Sent Done false
4057 2198 Acked Write 2 Done Sent Done false
4058 2199 Acked Read 7 Done Sent Done false
4059 2200 Acked Read 10 Done Sent Done false
4060 2201 Acked Read 6 Done Sent Done false
4061 2202 Acked Read 9 Done Sent Done false
4062 2203 Acked Read 5 Done Sent Done false
4063 2204 Acked Read 5 Done Sent Done false
4064 2205 Acked Read 9 Done Sent Done false
4065 2206 Acked Flush 0 Done Sent Done false
4066 STATES DS:0 DS:1 DS:2 TOTAL
4067 Sent 0 302 0 302
4068 Done 302 0 302 604
4069 Skipped 0 0 0 0
4070 Error 0 0 0 0
4071 Last Flush: 2206 1904 2206
4072 Downstairs last five completed: 1904 1903 1902 1901 1900
4073 Upstairs last five completed: 2206 2205 2204 2203 2202
4074 CLIENT: Up:0 ds:302 act:2
40752025-01-17T21:11:55.098ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
40762025-01-17T21:11:55.098ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:45417) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
40772025-01-17T21:11:55.100ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
40782025-01-17T21:11:55.100ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
40792025-01-17T21:11:55.100ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
40802025-01-17T21:11:55.100ZINFOcrucible: send last flush ID to this DS: 1904 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
40812025-01-17T21:11:55.100ZINFOcrucible: Replied this last flush ID: 1904 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
40822025-01-17T21:11:55.100ZINFOcrucible: [1] client re-new 302 jobs since flush 1904 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
40832025-01-17T21:11:55.101ZINFOcrucible: [1] Replayed 302 jobs since flush: 1904 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
4084 ----------------------------------------------------------------
4085 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
4086 Downstairs last five completed: 2206 2205 2204 2203 2202
4087 Upstairs last five completed: 2206 2205 2204 2203 2202
4088 CLIENT: Up:0 ds:0 act:3
4089 Replay: All jobs finished
4090 Main task has work to do, go find it
4091 got dsc Stop(2)
4092 Jan 17 21:11:57.342 INFO stop 2
4093 request completed, latency_us: 317, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 5ef95e91-0b35-46f4-805f-29581366006b, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
4094 [8830] Got stop action so:false kr:false
4095 Jan 17 21:11:57.342 INFO request completed, latency_us: 180, response_code: 200, uri: /state/cid/2, method: GET, req_id: ce1139aa-f4bc-4ac2-a29f-8151463bce52, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
40962025-01-17T21:11:57.342ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
40972025-01-17T21:11:57.342ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
40982025-01-17T21:11:57.342ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
40992025-01-17T21:11:57.343ZINFOcrucible: client task is exiting = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
4100 [8830] Exited with: ExitStatus(unix_wait_status(9))
4101 [8830][2] reports Exit
4102 Jan 17 21:12:01.344 INFO request completed, latency_us: 214, response_code: 200, uri: /state/cid/2, method: GET, req_id: ee5a4734-2da2-4f54-8092-12c384040f42, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
4103 001/300 Read block 3858 len 16384
4104 002/300 Write block 4542 len 20480 data: 2 2 3 3 3
4105 003/300 Read block 2417 len 32768
4106 004/300 Read block 55 len 8192
4107 005/300 Read block 3125 len 32768
4108 006/300 Read block 4280 len 36864
4109 007/300 Flush
4110 008/300 Read block 1813 len 16384
4111 009/300 Write block 4418 len 4096 data: 3
4112 010/300 Read block 3535 len 40960
4113 011/300 Write block 2780 len 8192 data: 2 2
4114 012/300 Read block 4639 len 32768
4115 013/300 Read block 4686 len 24576
4116 014/300 Write block 895 len 32768 data: 2 2 2 3 3 3 3 3
4117 015/300 Write block 3747 len 16384 data: 3 2 2 2
4118 016/300 Read block 2932 len 16384
4119 017/300 Write block 2183 len 32768 data: 2 2 2 2 2 2 2 2
4120 018/300 Read block 359 len 4096
4121 019/300 Flush
4122 020/300 Read block 4052 len 4096
4123 021/300 Read block 2912 len 36864
4124 022/300 Read block 4771 len 4096
4125 023/300 Flush
4126 024/300 Read block 4270 len 24576
4127 025/300 Read block 3502 len 12288
4128 026/300 Read block 2935 len 24576
4129 027/300 Read block 4274 len 24576
4130 028/300 Write block 690 len 4096 data: 2
4131 029/300 Flush
4132 030/300 Write block 2418 len 16384 data: 2 2 2 2
4133 031/300 Read block 2554 len 24576
4134 032/300 Read block 1866 len 24576
4135 033/300 Write block 1442 len 4096 data: 3
4136 034/300 Read block 1778 len 28672
4137 035/300 Read block 612 len 12288
4138 036/300 Write block 2085 len 16384 data: 3 3 3 3
4139 037/300 Read block 3835 len 8192
4140 038/300 Read block 3697 len 24576
4141 039/300 Read block 2312 len 8192
4142 040/300 Write block 1997 len 16384 data: 2 2 2 2
4143 041/300 Write block 3115 len 36864 data: 2 2 2 2 2 2 2 2 2
4144 042/300 Write block 3990 len 28672 data: 3 3 3 3 2 2 2
4145 043/300 Flush
4146 044/300 Read block 3620 len 16384
4147 045/300 Write block 1741 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4148 046/300 Write block 4733 len 20480 data: 4 4 4 4 3
4149 047/300 Write block 3423 len 8192 data: 2 2
4150 048/300 Write block 496 len 16384 data: 2 2 2 2
4151 049/300 Read block 4627 len 32768
4152 050/300 Read block 1938 len 8192
4153 051/300 Write block 1225 len 32768 data: 2 2 2 2 3 4 4 3
4154 052/300 Flush
4155 053/300 Read block 1646 len 32768
4156 054/300 Read block 2551 len 4096
4157 055/300 Read block 1945 len 24576
4158 056/300 Write block 1055 len 40960 data: 4 4 5 5 5 4 3 3 3 4
4159 057/300 Read block 1579 len 36864
4160 058/300 Read block 507 len 16384
4161 059/300 Write block 2944 len 24576 data: 2 2 2 2 2 2
4162 060/300 Write block 1177 len 36864 data: 2 2 2 2 2 2 2 4 4
4163 061/300 Write block 3946 len 32768 data: 2 2 2 2 2 2 2 2
4164 062/300 Read block 4314 len 28672
4165 063/300 Read block 3342 len 20480
4166 064/300 Read block 4821 len 36864
4167 065/300 Read block 2856 len 20480
4168 066/300 Read block 1727 len 16384
4169 067/300 Read block 242 len 4096
4170 068/300 Write block 4530 len 28672 data: 3 4 5 4 4 4 4
4171 069/300 Write block 2349 len 36864 data: 3 3 3 2 2 2 2 2 2
4172 070/300 Write block 727 len 16384 data: 2 2 3 3
4173 071/300 Write block 1155 len 24576 data: 2 3 4 4 4 3
4174 072/300 Read block 290 len 20480
4175 073/300 Write block 1302 len 40960 data: 2 2 2 3 3 4 4 4 3 3
4176 074/300 Write block 4774 len 4096 data: 2
4177 075/300 Read block 217 len 28672
4178 076/300 Read block 678 len 20480
4179 077/300 Read block 4122 len 16384
4180 078/300 Read block 572 len 20480
4181 079/300 Read block 656 len 8192
4182 080/300 Write block 2526 len 16384 data: 2 2 2 2
4183 081/300 Read block 4298 len 12288
4184 082/300 Write block 1035 len 12288 data: 2 2 2
4185 083/300 Write block 1203 len 8192 data: 3 3
4186 084/300 Read block 2655 len 36864
4187 085/300 Read block 3477 len 4096
4188 086/300 Read block 4654 len 36864
4189 087/300 Write block 3892 len 24576 data: 3 3 3 3 3 3
4190 088/300 Write block 3851 len 36864 data: 2 2 2 2 2 2 2 2 2
4191 089/300 Read block 1307 len 28672
4192 090/300 Flush
4193 091/300 Write block 1030 len 4096 data: 3
4194 092/300 Write block 392 len 16384 data: 2 2 2 2
4195 093/300 Read block 4508 len 32768
4196 094/300 Read block 3337 len 20480
4197 095/300 Write block 2062 len 28672 data: 2 3 3 4 4 4 4
4198 096/300 Read block 3182 len 12288
4199 097/300 Read block 1461 len 16384
4200 098/300 Write block 521 len 20480 data: 2 2 2 2 3
4201 099/300 Read block 1594 len 8192
4202 100/300 Read block 3796 len 40960
4203 101/300 Read block 1863 len 36864
4204 102/300 Read block 1135 len 4096
4205 103/300 Read block 895 len 32768
4206 104/300 Flush
4207 105/300 Read block 979 len 36864
4208 106/300 Read block 1334 len 28672
4209 107/300 Write block 3603 len 12288 data: 2 2 2
4210 108/300 Read block 4939 len 28672
4211 109/300 Write block 3221 len 24576 data: 2 2 3 3 3 3
4212 110/300 Write block 3417 len 12288 data: 2 2 2
4213 111/300 Write block 4118 len 8192 data: 3 3
4214 112/300 Flush
4215 113/300 Read block 2337 len 4096
4216 114/300 Write block 1198 len 8192 data: 2 2
4217 115/300 Write block 957 len 24576 data: 3 3 3 3 3 4
4218 116/300 Write block 2537 len 8192 data: 2 2
4219 117/300 Read block 2717 len 16384
4220 118/300 Write block 4434 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4221 119/300 Write block 1193 len 8192 data: 3 3
4222 120/300 Read block 3597 len 4096
4223 121/300 Write block 2085 len 32768 data: 4 4 4 4 3 3 2 2
4224 122/300 Flush
4225 123/300 Read block 181 len 8192
4226 124/300 Write block 1072 len 12288 data: 2 2 2
4227 125/300 Read block 4093 len 24576
4228 126/300 Read block 3098 len 12288
4229 127/300 Write block 898 len 4096 data: 4
4230 128/300 Write block 3104 len 8192 data: 3 3
4231 129/300 Read block 3204 len 8192
4232 130/300 Write block 2648 len 32768 data: 2 3 3 4 4 4 4 4
4233 131/300 Read block 2041 len 32768
4234 132/300 Read block 3897 len 20480
4235 133/300 Read block 2040 len 28672
4236 134/300 Read block 4664 len 24576
4237 135/300 Write block 3379 len 28672 data: 2 2 2 2 2 2 2
4238 136/300 Read block 3980 len 20480
4239 137/300 Write block 824 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4240 138/300 Write block 911 len 40960 data: 2 2 2 2 2 2 2 2 3 3
4241 139/300 Write block 637 len 16384 data: 2 2 2 3
4242 140/300 Read block 4038 len 36864
4243 141/300 Read block 1318 len 12288
4244 142/300 Write block 390 len 32768 data: 3 3 3 3 3 3 2 2
4245 143/300 Flush
4246 144/300 Write block 2805 len 20480 data: 3 3 3 3 3
4247 145/300 Write block 2797 len 12288 data: 2 2 2
4248 146/300 Read block 857 len 16384
4249 147/300 Write block 2460 len 36864 data: 2 2 2 2 2 3 4 3 3
4250 148/300 Write block 1023 len 24576 data: 3 3 2 3 3 3
4251 149/300 Read block 3853 len 24576
4252 150/300 Read block 587 len 4096
4253 151/300 Write block 3127 len 28672 data: 2 3 2 2 2 2 2
4254 152/300 Read block 1434 len 4096
4255 153/300 Read block 1801 len 8192
4256 154/300 Flush
4257 155/300 Read block 2563 len 16384
4258 156/300 Write block 3054 len 16384 data: 2 2 2 2
4259 157/300 Flush
4260 158/300 Read block 2848 len 40960
4261 159/300 Write block 1523 len 40960 data: 3 3 3 3 3 2 2 3 4 4
4262 160/300 Flush
4263 161/300 Read block 1074 len 24576
4264 162/300 Read block 2340 len 36864
4265 163/300 Flush
4266 164/300 Flush
4267 165/300 Write block 4137 len 16384 data: 3 3 3 3
4268 166/300 Read block 4460 len 32768
4269 167/300 Flush
4270 168/300 Read block 4208 len 40960
4271 169/300 Write block 3360 len 8192 data: 2 3
4272 170/300 Write block 4879 len 8192 data: 2 2
4273 171/300 Flush
4274 172/300 Write block 682 len 28672 data: 2 2 2 2 2 2 2
4275 173/300 Write block 4261 len 12288 data: 2 2 2
4276 174/300 Read block 1112 len 8192
4277 175/300 Write block 2386 len 12288 data: 3 3 4
4278 176/300 Read block 3860 len 16384
4279 177/300 Read block 4853 len 40960
4280 178/300 Flush
4281 179/300 Write block 2514 len 32768 data: 4 4 4 3 3 3 2 2
4282 180/300 Write block 2976 len 4096 data: 3
4283 181/300 Read block 4434 len 28672
4284 182/300 Read block 1974 len 28672
4285 183/300 Write block 2557 len 16384 data: 3 3 2 2
4286 184/300 Read block 2577 len 16384
4287 185/300 Flush
4288 186/300 Read block 697 len 32768
4289 187/300 Write block 99 len 4096 data: 4
4290 188/300 Read block 2678 len 8192
4291 189/300 Read block 1022 len 36864
4292 190/300 Write block 2583 len 32768 data: 3 3 3 4 4 4 4 3
4293 191/300 Read block 846 len 8192
4294 192/300 Flush
4295 193/300 Read block 2344 len 36864
4296 194/300 Read block 638 len 24576
4297 195/300 Read block 4188 len 16384
4298 196/300 Write block 4100 len 20480 data: 3 3 3 3 3
4299 197/300 Write block 397 len 36864 data: 3 2 2 2 2 2 2 2 2
4300 198/300 Write block 2560 len 36864 data: 3 2 2 2 2 2 2 2 2
4301 199/300 Read block 623 len 24576
4302 200/300 Write block 4235 len 12288 data: 3 3 3
4303 201/300 Read block 2608 len 12288
4304 202/300 Write block 3597 len 4096 data: 2
4305 203/300 Flush
4306 204/300 Flush
4307 205/300 Read block 1022 len 40960
4308 206/300 Read block 25 len 32768
4309 207/300 Read block 2566 len 24576
4310 208/300 Read block 3979 len 4096
4311 209/300 Read block 1496 len 16384
4312 210/300 Read block 317 len 12288
4313 211/300 Write block 1517 len 36864 data: 2 3 3 3 3 3 4 4 4
4314 212/300 Read block 4675 len 32768
4315 213/300 Read block 4855 len 32768
4316 214/300 Write block 1065 len 4096 data: 3
4317 215/300 Read block 3325 len 32768
4318 216/300 Flush
4319 217/300 Read block 3456 len 16384
4320 218/300 Flush
4321 219/300 Flush
4322 220/300 Flush
4323 221/300 Read block 4230 len 24576
4324 222/300 Read block 441 len 32768
4325 223/300 Read block 3958 len 4096
4326 224/300 Read block 4214 len 16384
4327 225/300 Read block 595 len 36864
4328 226/300 Write block 1884 len 20480 data: 3 3 4 4 4
4329 227/300 Read block 1147 len 32768
4330 228/300 Write block 3643 len 36864 data: 3 3 3 3 3 3 2 2 2
4331 229/300 Read block 309 len 24576
4332 230/300 Write block 2029 len 20480 data: 2 2 3 3 3
4333 231/300 Read block 3346 len 28672
4334 232/300 Write block 27 len 32768 data: 4 4 4 3 3 3 2 3
4335 233/300 Read block 4232 len 36864
4336 234/300 Read block 782 len 32768
4337 235/300 Read block 1736 len 32768
4338 236/300 Read block 1397 len 16384
4339 237/300 Write block 3608 len 20480 data: 2 3 3 3 3
4340 238/300 Read block 1684 len 36864
4341 239/300 Write block 3917 len 12288 data: 2 2 2
4342 240/300 Write block 2483 len 40960 data: 3 3 3 3 3 3 2 3 3 3
4343 241/300 Read block 2849 len 20480
4344 242/300 Read block 3309 len 32768
4345 243/300 Read block 325 len 12288
4346 244/300 Write block 72 len 40960 data: 3 3 3 3 3 3 3 2 2 2
4347 245/300 Flush
4348 246/300 Write block 1880 len 24576 data: 2 2 3 3 4 4
4349 247/300 Read block 2149 len 8192
4350 248/300 Read block 2571 len 16384
4351 249/300 Read block 1582 len 12288
4352 250/300 Read block 1872 len 4096
4353 251/300 Write block 900 len 8192 data: 4 4
4354 252/300 Write block 3959 len 28672 data: 2 3 2 2 2 2 3
4355 253/300 Flush
4356 254/300 Write block 4215 len 12288 data: 3 3 3
4357 255/300 Write block 2622 len 12288 data: 2 2 2
4358 256/300 Flush
4359 257/300 Read block 1205 len 16384
4360 258/300 Read block 2432 len 28672
4361 259/300 Read block 1268 len 32768
4362 260/300 Read block 3038 len 8192
4363 261/300 Read block 208 len 24576
4364 262/300 Write block 1578 len 32768 data: 2 2 2 2 2 2 3 3
4365 263/300 Read block 2528 len 24576
4366 264/300 Read block 2553 len 36864
4367 265/300 Flush
4368 266/300 Read block 443 len 40960
4369 267/300 Read block 4200 len 40960
4370 268/300 Read block 2514 len 36864
4371 269/300 Write block 1582 len 12288 data: 3 3 4
4372 270/300 Write block 3140 len 40960 data: 3 2 2 2 2 2 2 2 2 2
4373 271/300 Write block 1342 len 40960 data: 3 3 2 2 2 2 2 2 2 2
4374 272/300 Write block 4942 len 20480 data: 2 2 2 2 2
4375 273/300 Write block 2748 len 32768 data: 2 2 2 2 2 2 2 2
4376 274/300 Read block 1155 len 12288
4377 275/300 Write block 4261 len 12288 data: 3 3 3
4378 276/300 Read block 65 len 20480
4379 277/300 Write block 3825 len 32768 data: 3 3 2 2 3 3 2 2
4380 278/300 Read block 690 len 24576
4381 279/300 Read block 1364 len 32768
4382 280/300 Read block 886 len 16384
4383 281/300 Write block 3612 len 4096 data: 4
4384 282/300 Read block 343 len 28672
4385 283/300 Read block 3096 len 28672
4386 284/300 Flush
4387 285/300 Write block 4658 len 20480 data: 2 2 2 2 2
4388 286/300 Read block 4206 len 8192
4389 287/300 Write block 955 len 12288 data: 3 3 4
4390 288/300 Write block 4776 len 16384 data: 3 3 3 4
4391 289/300 Write block 4686 len 28672 data: 2 2 3 3 3 3 3
4392 290/300 Write block 4820 len 20480 data: 2 2 2 2 2
4393 291/300 Read block 342 len 36864
4394 292/300 Read block 4031 len 16384
4395 293/300 Read block 1364 len 4096
4396 294/300 Read block 3289 len 40960
4397 295/300 Read block 1458 len 20480
4398 296/300 Read block 2278 len 28672
4399 297/300 Write block 4155 len 32768 data: 3 3 3 3 3 3 2 2
4400 298/300 Write block 4006 len 24576 data: 4 4 4 4 2 2
4401 299/300 Write block 3523 len 4096 data: 3
4402 300/300 Write block 4672 len 36864 data: 2 2 2 2 2 3 3 3 3
4403 301/300 Read block 233 len 4096
4404 Main task has work to do, go find it
4405 got dsc Start(2)
4406 start 2
4407 Jan 17 21:12:01.591 INFO request completed, latency_us: 209, response_code: 204, uri: /start/cid/2, method: POST, req_id: bc11be1a-a567-4590-a8ad-bf4a40d21f9d, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
4408 Replay: started 2, returned:Ok(())
4409 ----------------------------------------------------------------
4410 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
4411 States: Active Active Offline
4412 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4413 2207 Acked Read 4 Done Done Sent false
4414 2208 Acked Write 5 Done Done Sent false
4415 2209 Acked Read 8 Done Done Sent false
4416 2210 Acked Read 2 Done Done Sent false
4417 2211 Acked Read 8 Done Done Sent false
4418 2212 Acked Read 9 Done Done Sent false
4419 2213 Acked Flush 0 Done Done Sent false
4420 2214 Acked Read 4 Done Done Sent false
4421 2215 Acked Write 1 Done Done Sent false
4422 2216 Acked Read 10 Done Done Sent false
4423 2217 Acked Write 2 Done Done Sent false
4424 2218 Acked Read 8 Done Done Sent false
4425 2219 Acked Read 6 Done Done Sent false
4426 2220 Acked Write 8 Done Done Sent false
4427 2221 Acked Write 4 Done Done Sent false
4428 2222 Acked Read 4 Done Done Sent false
4429 2223 Acked Write 8 Done Done Sent false
4430 2224 Acked Read 1 Done Done Sent false
4431 2225 Acked Flush 0 Done Done Sent false
4432 2226 Acked Read 1 Done Done Sent false
4433 2227 Acked Read 9 Done Done Sent false
4434 2228 Acked Read 1 Done Done Sent false
4435 2229 Acked Flush 0 Done Done Sent false
4436 2230 Acked Read 6 Done Done Sent false
4437 2231 Acked Read 3 Done Done Sent false
4438 2232 Acked Read 6 Done Done Sent false
4439 2233 Acked Read 6 Done Done Sent false
4440 2234 Acked Write 1 Done Done Sent false
4441 2235 Acked Flush 0 Done Done Sent false
4442 2236 Acked Write 4 Done Done Sent false
4443 2237 Acked Read 6 Done Done Sent false
4444 2238 Acked Read 6 Done Done Sent false
4445 2239 Acked Write 1 Done Done Sent false
4446 2240 Acked Read 7 Done Done Sent false
4447 2241 Acked Read 3 Done Done Sent false
4448 2242 Acked Write 4 Done Done Sent false
4449 2243 Acked Read 2 Done Done Sent false
4450 2244 Acked Read 6 Done Done Sent false
4451 2245 Acked Read 2 Done Done Sent false
4452 2246 Acked Write 4 Done Done Sent false
4453 2247 Acked Write 9 Done Done Sent false
4454 2248 Acked Write 7 Done Done Sent false
4455 2249 Acked Flush 0 Done Done Sent false
4456 2250 Acked Read 4 Done Done Sent false
4457 2251 Acked Write 10 Done Done Sent false
4458 2252 Acked Write 5 Done Done Sent false
4459 2253 Acked Write 2 Done Done Sent false
4460 2254 Acked Write 4 Done Done Sent false
4461 2255 Acked Read 8 Done Done Sent false
4462 2256 Acked Read 2 Done Done Sent false
4463 2257 Acked Write 8 Done Done Sent false
4464 2258 Acked Flush 0 Done Done Sent false
4465 2259 Acked Read 8 Done Done Sent false
4466 2260 Acked Read 1 Done Done Sent false
4467 2261 Acked Read 6 Done Done Sent false
4468 2262 Acked Write 10 Done Done Sent false
4469 2263 Acked Read 9 Done Done Sent false
4470 2264 Acked Read 4 Done Done Sent false
4471 2265 Acked Write 6 Done Done Sent false
4472 2266 Acked Write 9 Done Done Sent false
4473 2267 Acked Write 8 Done Done Sent false
4474 2268 Acked Read 7 Done Done Sent false
4475 2269 Acked Read 5 Done Done Sent false
4476 2270 Acked Read 9 Done Done Sent false
4477 2271 Acked Read 5 Done Done Sent false
4478 2272 Acked Read 4 Done Done Sent false
4479 2273 Acked Read 1 Done Done Sent false
4480 2274 Acked Write 7 Done Done Sent false
4481 2275 Acked Write 9 Done Done Sent false
4482 2276 Acked Write 4 Done Done Sent false
4483 2277 Acked Write 6 Done Done Sent false
4484 2278 Acked Read 5 Done Done Sent false
4485 2279 Acked Write 10 Done Done Sent false
4486 2280 Acked Write 1 Done Done Sent false
4487 2281 Acked Read 7 Done Done Sent false
4488 2282 Acked Read 5 Done Done Sent false
4489 2283 Acked Read 4 Done Done Sent false
4490 2284 Acked Read 5 Done Done Sent false
4491 2285 Acked Read 2 Done Done Sent false
4492 2286 Acked Write 4 Done Done Sent false
4493 2287 Acked Read 3 Done Done Sent false
4494 2288 Acked Write 3 Done Done Sent false
4495 2289 Acked Write 2 Done Done Sent false
4496 2290 Acked Read 9 Done Done Sent false
4497 2291 Acked Read 1 Done Done Sent false
4498 2292 Acked Read 9 Done Done Sent false
4499 2293 Acked Write 6 Done Done Sent false
4500 2294 Acked Write 9 Done Done Sent false
4501 2295 Acked Read 7 Done Done Sent false
4502 2296 Acked Flush 0 Done Done Sent false
4503 2297 Acked Write 1 Done Done Sent false
4504 2298 Acked Write 4 Done Done Sent false
4505 2299 Acked Read 8 Done Done Sent false
4506 2300 Acked Read 5 Done Done Sent false
4507 2301 Acked Write 7 Done Done Sent false
4508 2302 Acked Read 3 Done Done Sent false
4509 2303 Acked Read 4 Done Done Sent false
4510 2304 Acked Write 5 Done Done Sent false
4511 2305 Acked Read 2 Done Done Sent false
4512 2306 Acked Read 10 Done Done Sent false
4513 2307 Acked Read 9 Done Done Sent false
4514 2308 Acked Read 1 Done Done Sent false
4515 2309 Acked Read 8 Done Done Sent false
4516 2310 Acked Flush 0 Done Done Sent false
4517 2311 Acked Read 9 Done Done Sent false
4518 2312 Acked Read 7 Done Done Sent false
4519 2313 Acked Write 3 Done Done Sent false
4520 2314 Acked Read 7 Done Done Sent false
4521 2315 Acked Write 6 Done Done Sent false
4522 2316 Acked Write 3 Done Done Sent false
4523 2317 Acked Write 2 Done Done Sent false
4524 2318 Acked Flush 0 Done Done Sent false
4525 2319 Acked Read 1 Done Done Sent false
4526 2320 Acked Write 2 Done Done Sent false
4527 2321 Acked Write 6 Done Done Sent false
4528 2322 Acked Write 2 Done Done Sent false
4529 2323 Acked Read 4 Done Done Sent false
4530 2324 Acked Write 10 Done Done Sent false
4531 2325 Acked Write 2 Done Done Sent false
4532 2326 Acked Read 1 Done Done Sent false
4533 2327 Acked Write 8 Done Done Sent false
4534 2328 Acked Flush 0 Done Done Sent false
4535 2329 Acked Read 2 Done Done Sent false
4536 2330 Acked Write 3 Done Done Sent false
4537 2331 Acked Read 6 Done Done Sent false
4538 2332 Acked Read 3 Done Done Sent false
4539 2333 Acked Write 1 Done Done Sent false
4540 2334 Acked Write 2 Done Done Sent false
4541 2335 Acked Read 2 Done Done Sent false
4542 2336 Acked Write 8 Done Done Sent false
4543 2337 Acked Read 8 Done Done Sent false
4544 2338 Acked Read 5 Done Done Sent false
4545 2339 Acked Read 7 Done Done Sent false
4546 2340 Acked Read 6 Done Done Sent false
4547 2341 Acked Write 7 Done Done Sent false
4548 2342 Acked Read 5 Done Done Sent false
4549 2343 Acked Write 10 Done Done Sent false
4550 2344 Acked Write 10 Done Done Sent false
4551 2345 Acked Write 4 Done Done Sent false
4552 2346 Acked Read 9 Done Done Sent false
4553 2347 Acked Read 3 Done Done Sent false
4554 2348 Acked Write 8 Done Done Sent false
4555 2349 Acked Flush 0 Done Done Sent false
4556 2350 Acked Write 5 Done Done Sent false
4557 2351 Acked Write 3 Done Done Sent false
4558 2352 Acked Read 4 Done Done Sent false
4559 2353 Acked Write 9 Done Done Sent false
4560 2354 Acked Write 6 Done Done Sent false
4561 2355 Acked Read 6 Done Done Sent false
4562 2356 Acked Read 1 Done Done Sent false
4563 2357 Acked Write 7 Done Done Sent false
4564 2358 Acked Read 1 Done Done Sent false
4565 2359 Acked Read 2 Done Done Sent false
4566 2360 Acked Flush 0 Done Done Sent false
4567 2361 Acked Read 4 Done Done Sent false
4568 2362 Acked Write 4 Done Done Sent false
4569 2363 Acked Flush 0 Done Done Sent false
4570 2364 Acked Read 10 Done Done Sent false
4571 2365 Acked Write 10 Done Done Sent false
4572 2366 Acked Flush 0 Done Done Sent false
4573 2367 Acked Read 6 Done Done Sent false
4574 2368 Acked Read 9 Done Done Sent false
4575 2369 Acked Flush 0 Done Done Sent false
4576 2370 Acked Flush 0 Done Done Sent false
4577 2371 Acked Write 4 Done Done Sent false
4578 2372 Acked Read 8 Done Done Sent false
4579 2373 Acked Flush 0 Done Done Sent false
4580 2374 Acked Read 10 Done Done Sent false
4581 2375 Acked Write 2 Done Done Sent false
4582 2376 Acked Write 2 Done Done Sent false
4583 2377 Acked Flush 0 Done Done Sent false
4584 2378 Acked Write 7 Done Done Sent false
4585 2379 Acked Write 3 Done Done Sent false
4586 2380 Acked Read 2 Done Done Sent false
4587 2381 Acked Write 3 Done Done Sent false
4588 2382 Acked Read 4 Done Done Sent false
4589 2383 Acked Read 10 Done Done Sent false
4590 2384 Acked Flush 0 Done Done Sent false
4591 2385 Acked Write 8 Done Done Sent false
4592 2386 Acked Write 1 Done Done Sent false
4593 2387 Acked Read 7 Done Done Sent false
4594 2388 Acked Read 7 Done Done Sent false
4595 2389 Acked Write 4 Done Done Sent false
4596 2390 Acked Read 4 Done Done Sent false
4597 2391 Acked Flush 0 Done Done Sent false
4598 2392 Acked Read 8 Done Done Sent false
4599 2393 Acked Write 1 Done Done Sent false
4600 2394 Acked Read 2 Done Done Sent false
4601 2395 Acked Read 9 Done Done Sent false
4602 2396 Acked Write 8 Done Done Sent false
4603 2397 Acked Read 2 Done Done Sent false
4604 2398 Acked Flush 0 Done Done Sent false
4605 2399 Acked Read 9 Done Done Sent false
4606 2400 Acked Read 6 Done Done Sent false
4607 2401 Acked Read 4 Done Done Sent false
4608 2402 Acked Write 5 Done Done Sent false
4609 2403 Acked Write 9 Done Done Sent false
4610 2404 Acked Write 9 Done Done Sent false
4611 2405 Acked Read 6 Done Done Sent false
4612 2406 Acked Write 3 Done Done Sent false
4613 2407 Acked Read 3 Done Done Sent false
4614 2408 Acked Write 1 Done Done Sent false
4615 2409 Acked Flush 0 Done Done Sent false
4616 2410 Acked Flush 0 Done Done Sent false
4617 2411 Acked Read 10 Done Done Sent false
4618 2412 Acked Read 8 Done Done Sent false
4619 2413 Acked Read 6 Done Done Sent false
4620 2414 Acked Read 1 Done Done Sent false
4621 2415 Acked Read 4 Done Done Sent false
4622 2416 Acked Read 3 Done Done Sent false
4623 2417 Acked Write 9 Done Done Sent false
4624 2418 Acked Read 8 Done Done Sent false
4625 2419 Acked Read 8 Done Done Sent false
4626 2420 Acked Write 1 Done Done Sent false
4627 2421 Acked Read 8 Done Done Sent false
4628 2422 Acked Flush 0 Done Done Sent false
4629 2423 Acked Read 4 Done Done Sent false
4630 2424 Acked Flush 0 Done Done Sent false
4631 2425 Acked Flush 0 Done Done Sent false
4632 2426 Acked Flush 0 Done Done Sent false
4633 2427 Acked Read 6 Done Done Sent false
4634 2428 Acked Read 8 Done Done Sent false
4635 2429 Acked Read 1 Done Done Sent false
4636 2430 Acked Read 4 Done Done Sent false
4637 2431 Acked Read 9 Done Done Sent false
4638 2432 Acked Write 5 Done Done Sent false
4639 2433 Acked Read 8 Done Done Sent false
4640 2434 Acked Write 9 Done Done Sent false
4641 2435 Acked Read 6 Done Done Sent false
4642 2436 Acked Write 5 Done Done Sent false
4643 2437 Acked Read 7 Done Done Sent false
4644 2438 Acked Write 8 Done Done Sent false
4645 2439 Acked Read 9 Done Done Sent false
4646 2440 Acked Read 8 Done Done Sent false
4647 2441 Acked Read 8 Done Done Sent false
4648 2442 Acked Read 4 Done Done Sent false
4649 2443 Acked Write 5 Done Done Sent false
4650 2444 Acked Read 9 Done Done Sent false
4651 2445 Acked Write 3 Done Done Sent false
4652 2446 Acked Write 10 Done Done Sent false
4653 2447 Acked Read 5 Done Done Sent false
4654 2448 Acked Read 8 Done Done Sent false
4655 2449 Acked Read 3 Done Done Sent false
4656 2450 Acked Write 10 Done Done Sent false
4657 2451 Acked Flush 0 Done Done Sent false
4658 2452 Acked Write 6 Done Done Sent false
4659 2453 Acked Read 2 Done Done Sent false
4660 2454 Acked Read 4 Done Done Sent false
4661 2455 Acked Read 3 Done Done Sent false
4662 2456 Acked Read 1 Done Done Sent false
4663 2457 Acked Write 2 Done Done Sent false
4664 2458 Acked Write 7 Done Done Sent false
4665 2459 Acked Flush 0 Done Done Sent false
4666 2460 Acked Write 3 Done Done Sent false
4667 2461 Acked Write 3 Done Done Sent false
4668 2462 Acked Flush 0 Done Done Sent false
4669 2463 Acked Read 4 Done Done Sent false
4670 2464 Acked Read 7 Done Done Sent false
4671 2465 Acked Read 8 Done Done Sent false
4672 2466 Acked Read 2 Done Done Sent false
4673 2467 Acked Read 6 Done Done Sent false
4674 2468 Acked Write 8 Done Done Sent false
4675 2469 Acked Read 6 Done Done Sent false
4676 2470 Acked Read 9 Done Done Sent false
4677 2471 Acked Flush 0 Done Done Sent false
4678 2472 Acked Read 10 Done Done Sent false
4679 2473 Acked Read 10 Done Done Sent false
4680 2474 Acked Read 9 Done Done Sent false
4681 2475 Acked Write 3 Done Done Sent false
4682 2476 Acked Write 10 Done Done Sent false
4683 2477 Acked Write 10 Done Done Sent false
4684 2478 Acked Write 5 Done Done Sent false
4685 2479 Acked Write 8 Done Done Sent false
4686 2480 Acked Read 3 Done Done Sent false
4687 2481 Acked Write 3 Done Done Sent false
4688 2482 Acked Read 5 Done Done Sent false
4689 2483 Acked Write 8 Done Done Sent false
4690 2484 Acked Read 6 Done Done Sent false
4691 2485 Acked Read 8 Done Done Sent false
4692 2486 Acked Read 4 Done Done Sent false
4693 2487 Acked Write 1 Done Done Sent false
4694 2488 Acked Read 7 Done Done Sent false
4695 2489 Acked Read 7 Done Done Sent false
4696 2490 Acked Flush 0 Done Done Sent false
4697 2491 Acked Write 5 Done Done Sent false
4698 2492 Acked Read 2 Done Done Sent false
4699 2493 Acked Write 3 Done Done Sent false
4700 2494 Acked Write 4 Done Done Sent false
4701 2495 Acked Write 7 Done Done Sent false
4702 2496 Acked Write 5 Done Done Sent false
4703 2497 Acked Read 9 Done Done Sent false
4704 2498 Acked Read 4 Done Done Sent false
4705 2499 Acked Read 1 Done Done Sent false
4706 2500 Acked Read 10 Done Done Sent false
4707 2501 Acked Read 5 Done Done Sent false
4708 2502 Acked Read 7 Done Done Sent false
4709 2503 Acked Write 8 Done Done Sent false
4710 2504 Acked Write 6 Done Done Sent false
4711 2505 Acked Write 1 Done Done Sent false
4712 2506 Acked Write 9 Done Done Sent false
4713 2507 Acked Read 1 Done Done Sent false
4714 STATES DS:0 DS:1 DS:2 TOTAL
4715 Sent 0 0 301 301
4716 Done 301 301 0 602
4717 Skipped 0 0 0 0
4718 Error 0 0 0 0
4719 Last Flush: 2490 2490 2206
4720 Downstairs last five completed: 2206 2205 2204 2203 2202
4721 Upstairs last five completed: 2507 2506 2505 2504 2503
4722 CLIENT: Up:0 ds:301 act:2
4723 [8830] got start action
4724 [8830] I am going to restart
4725 Starting downstairs at port 8830
4726 Make output file at "/tmp/dsc/downstairs-8830.txt"
4727 [8830][2] reports Starting
4728 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1205)
4729 [8830][2] reports Running
4730 ----------------------------------------------------------------
4731 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
4732 States: Active Active Offline
4733 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4734 2207 Acked Read 4 Done Done Sent false
4735 2208 Acked Write 5 Done Done Sent false
4736 2209 Acked Read 8 Done Done Sent false
4737 2210 Acked Read 2 Done Done Sent false
4738 2211 Acked Read 8 Done Done Sent false
4739 2212 Acked Read 9 Done Done Sent false
4740 2213 Acked Flush 0 Done Done Sent false
4741 2214 Acked Read 4 Done Done Sent false
4742 2215 Acked Write 1 Done Done Sent false
4743 2216 Acked Read 10 Done Done Sent false
4744 2217 Acked Write 2 Done Done Sent false
4745 2218 Acked Read 8 Done Done Sent false
4746 2219 Acked Read 6 Done Done Sent false
4747 2220 Acked Write 8 Done Done Sent false
4748 2221 Acked Write 4 Done Done Sent false
4749 2222 Acked Read 4 Done Done Sent false
4750 2223 Acked Write 8 Done Done Sent false
4751 2224 Acked Read 1 Done Done Sent false
4752 2225 Acked Flush 0 Done Done Sent false
4753 2226 Acked Read 1 Done Done Sent false
4754 2227 Acked Read 9 Done Done Sent false
4755 2228 Acked Read 1 Done Done Sent false
4756 2229 Acked Flush 0 Done Done Sent false
4757 2230 Acked Read 6 Done Done Sent false
4758 2231 Acked Read 3 Done Done Sent false
4759 2232 Acked Read 6 Done Done Sent false
4760 2233 Acked Read 6 Done Done Sent false
4761 2234 Acked Write 1 Done Done Sent false
4762 2235 Acked Flush 0 Done Done Sent false
4763 2236 Acked Write 4 Done Done Sent false
4764 2237 Acked Read 6 Done Done Sent false
4765 2238 Acked Read 6 Done Done Sent false
4766 2239 Acked Write 1 Done Done Sent false
4767 2240 Acked Read 7 Done Done Sent false
4768 2241 Acked Read 3 Done Done Sent false
4769 2242 Acked Write 4 Done Done Sent false
4770 2243 Acked Read 2 Done Done Sent false
4771 2244 Acked Read 6 Done Done Sent false
4772 2245 Acked Read 2 Done Done Sent false
4773 2246 Acked Write 4 Done Done Sent false
4774 2247 Acked Write 9 Done Done Sent false
4775 2248 Acked Write 7 Done Done Sent false
4776 2249 Acked Flush 0 Done Done Sent false
4777 2250 Acked Read 4 Done Done Sent false
4778 2251 Acked Write 10 Done Done Sent false
4779 2252 Acked Write 5 Done Done Sent false
4780 2253 Acked Write 2 Done Done Sent false
4781 2254 Acked Write 4 Done Done Sent false
4782 2255 Acked Read 8 Done Done Sent false
4783 2256 Acked Read 2 Done Done Sent false
4784 2257 Acked Write 8 Done Done Sent false
4785 2258 Acked Flush 0 Done Done Sent false
4786 2259 Acked Read 8 Done Done Sent false
4787 2260 Acked Read 1 Done Done Sent false
4788 2261 Acked Read 6 Done Done Sent false
4789 2262 Acked Write 10 Done Done Sent false
4790 2263 Acked Read 9 Done Done Sent false
4791 2264 Acked Read 4 Done Done Sent false
4792 2265 Acked Write 6 Done Done Sent false
4793 2266 Acked Write 9 Done Done Sent false
4794 2267 Acked Write 8 Done Done Sent false
4795 2268 Acked Read 7 Done Done Sent false
4796 2269 Acked Read 5 Done Done Sent false
4797 2270 Acked Read 9 Done Done Sent false
4798 2271 Acked Read 5 Done Done Sent false
4799 2272 Acked Read 4 Done Done Sent false
4800 2273 Acked Read 1 Done Done Sent false
4801 2274 Acked Write 7 Done Done Sent false
4802 2275 Acked Write 9 Done Done Sent false
4803 2276 Acked Write 4 Done Done Sent false
4804 2277 Acked Write 6 Done Done Sent false
4805 2278 Acked Read 5 Done Done Sent false
4806 2279 Acked Write 10 Done Done Sent false
4807 2280 Acked Write 1 Done Done Sent false
4808 2281 Acked Read 7 Done Done Sent false
4809 2282 Acked Read 5 Done Done Sent false
4810 2283 Acked Read 4 Done Done Sent false
4811 2284 Acked Read 5 Done Done Sent false
4812 2285 Acked Read 2 Done Done Sent false
4813 2286 Acked Write 4 Done Done Sent false
4814 2287 Acked Read 3 Done Done Sent false
4815 2288 Acked Write 3 Done Done Sent false
4816 2289 Acked Write 2 Done Done Sent false
4817 2290 Acked Read 9 Done Done Sent false
4818 2291 Acked Read 1 Done Done Sent false
4819 2292 Acked Read 9 Done Done Sent false
4820 2293 Acked Write 6 Done Done Sent false
4821 2294 Acked Write 9 Done Done Sent false
4822 2295 Acked Read 7 Done Done Sent false
4823 2296 Acked Flush 0 Done Done Sent false
4824 2297 Acked Write 1 Done Done Sent false
4825 2298 Acked Write 4 Done Done Sent false
4826 2299 Acked Read 8 Done Done Sent false
4827 2300 Acked Read 5 Done Done Sent false
4828 2301 Acked Write 7 Done Done Sent false
4829 2302 Acked Read 3 Done Done Sent false
4830 2303 Acked Read 4 Done Done Sent false
4831 2304 Acked Write 5 Done Done Sent false
4832 2305 Acked Read 2 Done Done Sent false
4833 2306 Acked Read 10 Done Done Sent false
4834 2307 Acked Read 9 Done Done Sent false
4835 2308 Acked Read 1 Done Done Sent false
4836 2309 Acked Read 8 Done Done Sent false
4837 2310 Acked Flush 0 Done Done Sent false
4838 2311 Acked Read 9 Done Done Sent false
4839 2312 Acked Read 7 Done Done Sent false
4840 2313 Acked Write 3 Done Done Sent false
4841 2314 Acked Read 7 Done Done Sent false
4842 2315 Acked Write 6 Done Done Sent false
4843 2316 Acked Write 3 Done Done Sent false
4844 2317 Acked Write 2 Done Done Sent false
4845 2318 Acked Flush 0 Done Done Sent false
4846 2319 Acked Read 1 Done Done Sent false
4847 2320 Acked Write 2 Done Done Sent false
4848 2321 Acked Write 6 Done Done Sent false
4849 2322 Acked Write 2 Done Done Sent false
4850 2323 Acked Read 4 Done Done Sent false
4851 2324 Acked Write 10 Done Done Sent false
4852 2325 Acked Write 2 Done Done Sent false
4853 2326 Acked Read 1 Done Done Sent false
4854 2327 Acked Write 8 Done Done Sent false
4855 2328 Acked Flush 0 Done Done Sent false
4856 2329 Acked Read 2 Done Done Sent false
4857 2330 Acked Write 3 Done Done Sent false
4858 2331 Acked Read 6 Done Done Sent false
4859 2332 Acked Read 3 Done Done Sent false
4860 2333 Acked Write 1 Done Done Sent false
4861 2334 Acked Write 2 Done Done Sent false
4862 2335 Acked Read 2 Done Done Sent false
4863 2336 Acked Write 8 Done Done Sent false
4864 2337 Acked Read 8 Done Done Sent false
4865 2338 Acked Read 5 Done Done Sent false
4866 2339 Acked Read 7 Done Done Sent false
4867 2340 Acked Read 6 Done Done Sent false
4868 2341 Acked Write 7 Done Done Sent false
4869 2342 Acked Read 5 Done Done Sent false
4870 2343 Acked Write 10 Done Done Sent false
4871 2344 Acked Write 10 Done Done Sent false
4872 2345 Acked Write 4 Done Done Sent false
4873 2346 Acked Read 9 Done Done Sent false
4874 2347 Acked Read 3 Done Done Sent false
4875 2348 Acked Write 8 Done Done Sent false
4876 2349 Acked Flush 0 Done Done Sent false
4877 2350 Acked Write 5 Done Done Sent false
4878 2351 Acked Write 3 Done Done Sent false
4879 2352 Acked Read 4 Done Done Sent false
4880 2353 Acked Write 9 Done Done Sent false
4881 2354 Acked Write 6 Done Done Sent false
4882 2355 Acked Read 6 Done Done Sent false
4883 2356 Acked Read 1 Done Done Sent false
4884 2357 Acked Write 7 Done Done Sent false
4885 2358 Acked Read 1 Done Done Sent false
4886 2359 Acked Read 2 Done Done Sent false
4887 2360 Acked Flush 0 Done Done Sent false
4888 2361 Acked Read 4 Done Done Sent false
4889 2362 Acked Write 4 Done Done Sent false
4890 2363 Acked Flush 0 Done Done Sent false
4891 2364 Acked Read 10 Done Done Sent false
4892 2365 Acked Write 10 Done Done Sent false
4893 2366 Acked Flush 0 Done Done Sent false
4894 2367 Acked Read 6 Done Done Sent false
4895 2368 Acked Read 9 Done Done Sent false
4896 2369 Acked Flush 0 Done Done Sent false
4897 2370 Acked Flush 0 Done Done Sent false
4898 2371 Acked Write 4 Done Done Sent false
4899 2372 Acked Read 8 Done Done Sent false
4900 2373 Acked Flush 0 Done Done Sent false
4901 2374 Acked Read 10 Done Done Sent false
4902 2375 Acked Write 2 Done Done Sent false
4903 2376 Acked Write 2 Done Done Sent false
4904 2377 Acked Flush 0 Done Done Sent false
4905 2378 Acked Write 7 Done Done Sent false
4906 2379 Acked Write 3 Done Done Sent false
4907 2380 Acked Read 2 Done Done Sent false
4908 2381 Acked Write 3 Done Done Sent false
4909 2382 Acked Read 4 Done Done Sent false
4910 2383 Acked Read 10 Done Done Sent false
4911 2384 Acked Flush 0 Done Done Sent false
4912 2385 Acked Write 8 Done Done Sent false
4913 2386 Acked Write 1 Done Done Sent false
4914 2387 Acked Read 7 Done Done Sent false
4915 2388 Acked Read 7 Done Done Sent false
4916 2389 Acked Write 4 Done Done Sent false
4917 2390 Acked Read 4 Done Done Sent false
4918 2391 Acked Flush 0 Done Done Sent false
4919 2392 Acked Read 8 Done Done Sent false
4920 2393 Acked Write 1 Done Done Sent false
4921 2394 Acked Read 2 Done Done Sent false
4922 2395 Acked Read 9 Done Done Sent false
4923 2396 Acked Write 8 Done Done Sent false
4924 2397 Acked Read 2 Done Done Sent false
4925 2398 Acked Flush 0 Done Done Sent false
4926 2399 Acked Read 9 Done Done Sent false
4927 2400 Acked Read 6 Done Done Sent false
4928 2401 Acked Read 4 Done Done Sent false
4929 2402 Acked Write 5 Done Done Sent false
4930 2403 Acked Write 9 Done Done Sent false
4931 2404 Acked Write 9 Done Done Sent false
4932 2405 Acked Read 6 Done Done Sent false
4933 2406 Acked Write 3 Done Done Sent false
4934 2407 Acked Read 3 Done Done Sent false
4935 2408 Acked Write 1 Done Done Sent false
4936 2409 Acked Flush 0 Done Done Sent false
4937 2410 Acked Flush 0 Done Done Sent false
4938 2411 Acked Read 10 Done Done Sent false
4939 2412 Acked Read 8 Done Done Sent false
4940 2413 Acked Read 6 Done Done Sent false
4941 2414 Acked Read 1 Done Done Sent false
4942 2415 Acked Read 4 Done Done Sent false
4943 2416 Acked Read 3 Done Done Sent false
4944 2417 Acked Write 9 Done Done Sent false
4945 2418 Acked Read 8 Done Done Sent false
4946 2419 Acked Read 8 Done Done Sent false
4947 2420 Acked Write 1 Done Done Sent false
4948 2421 Acked Read 8 Done Done Sent false
4949 2422 Acked Flush 0 Done Done Sent false
4950 2423 Acked Read 4 Done Done Sent false
4951 2424 Acked Flush 0 Done Done Sent false
4952 2425 Acked Flush 0 Done Done Sent false
4953 2426 Acked Flush 0 Done Done Sent false
4954 2427 Acked Read 6 Done Done Sent false
4955 2428 Acked Read 8 Done Done Sent false
4956 2429 Acked Read 1 Done Done Sent false
4957 2430 Acked Read 4 Done Done Sent false
4958 2431 Acked Read 9 Done Done Sent false
4959 2432 Acked Write 5 Done Done Sent false
4960 2433 Acked Read 8 Done Done Sent false
4961 2434 Acked Write 9 Done Done Sent false
4962 2435 Acked Read 6 Done Done Sent false
4963 2436 Acked Write 5 Done Done Sent false
4964 2437 Acked Read 7 Done Done Sent false
4965 2438 Acked Write 8 Done Done Sent false
4966 2439 Acked Read 9 Done Done Sent false
4967 2440 Acked Read 8 Done Done Sent false
4968 2441 Acked Read 8 Done Done Sent false
4969 2442 Acked Read 4 Done Done Sent false
4970 2443 Acked Write 5 Done Done Sent false
4971 2444 Acked Read 9 Done Done Sent false
4972 2445 Acked Write 3 Done Done Sent false
4973 2446 Acked Write 10 Done Done Sent false
4974 2447 Acked Read 5 Done Done Sent false
4975 2448 Acked Read 8 Done Done Sent false
4976 2449 Acked Read 3 Done Done Sent false
4977 2450 Acked Write 10 Done Done Sent false
4978 2451 Acked Flush 0 Done Done Sent false
4979 2452 Acked Write 6 Done Done Sent false
4980 2453 Acked Read 2 Done Done Sent false
4981 2454 Acked Read 4 Done Done Sent false
4982 2455 Acked Read 3 Done Done Sent false
4983 2456 Acked Read 1 Done Done Sent false
4984 2457 Acked Write 2 Done Done Sent false
4985 2458 Acked Write 7 Done Done Sent false
4986 2459 Acked Flush 0 Done Done Sent false
4987 2460 Acked Write 3 Done Done Sent false
4988 2461 Acked Write 3 Done Done Sent false
4989 2462 Acked Flush 0 Done Done Sent false
4990 2463 Acked Read 4 Done Done Sent false
4991 2464 Acked Read 7 Done Done Sent false
4992 2465 Acked Read 8 Done Done Sent false
4993 2466 Acked Read 2 Done Done Sent false
4994 2467 Acked Read 6 Done Done Sent false
4995 2468 Acked Write 8 Done Done Sent false
4996 2469 Acked Read 6 Done Done Sent false
4997 2470 Acked Read 9 Done Done Sent false
4998 2471 Acked Flush 0 Done Done Sent false
4999 2472 Acked Read 10 Done Done Sent false
5000 2473 Acked Read 10 Done Done Sent false
5001 2474 Acked Read 9 Done Done Sent false
5002 2475 Acked Write 3 Done Done Sent false
5003 2476 Acked Write 10 Done Done Sent false
5004 2477 Acked Write 10 Done Done Sent false
5005 2478 Acked Write 5 Done Done Sent false
5006 2479 Acked Write 8 Done Done Sent false
5007 2480 Acked Read 3 Done Done Sent false
5008 2481 Acked Write 3 Done Done Sent false
5009 2482 Acked Read 5 Done Done Sent false
5010 2483 Acked Write 8 Done Done Sent false
5011 2484 Acked Read 6 Done Done Sent false
5012 2485 Acked Read 8 Done Done Sent false
5013 2486 Acked Read 4 Done Done Sent false
5014 2487 Acked Write 1 Done Done Sent false
5015 2488 Acked Read 7 Done Done Sent false
5016 2489 Acked Read 7 Done Done Sent false
5017 2490 Acked Flush 0 Done Done Sent false
5018 2491 Acked Write 5 Done Done Sent false
5019 2492 Acked Read 2 Done Done Sent false
5020 2493 Acked Write 3 Done Done Sent false
5021 2494 Acked Write 4 Done Done Sent false
5022 2495 Acked Write 7 Done Done Sent false
5023 2496 Acked Write 5 Done Done Sent false
5024 2497 Acked Read 9 Done Done Sent false
5025 2498 Acked Read 4 Done Done Sent false
5026 2499 Acked Read 1 Done Done Sent false
5027 2500 Acked Read 10 Done Done Sent false
5028 2501 Acked Read 5 Done Done Sent false
5029 2502 Acked Read 7 Done Done Sent false
5030 2503 Acked Write 8 Done Done Sent false
5031 2504 Acked Write 6 Done Done Sent false
5032 2505 Acked Write 1 Done Done Sent false
5033 2506 Acked Write 9 Done Done Sent false
5034 2507 Acked Read 1 Done Done Sent false
5035 2508 Acked Flush 0 Done Done Sent false
5036 STATES DS:0 DS:1 DS:2 TOTAL
5037 Sent 0 0 302 302
5038 Done 302 302 0 604
5039 Skipped 0 0 0 0
5040 Error 0 0 0 0
5041 Last Flush: 2508 2508 2206
5042 Downstairs last five completed: 2206 2205 2204 2203 2202
5043 Upstairs last five completed: 2508 2507 2506 2505 2504
5044 CLIENT: Up:0 ds:302 act:2
50452025-01-17T21:12:07.343ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
50462025-01-17T21:12:07.343ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40131) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
50472025-01-17T21:12:07.345ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
50482025-01-17T21:12:07.345ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
50492025-01-17T21:12:07.345ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
50502025-01-17T21:12:07.345ZINFOcrucible: send last flush ID to this DS: 2206 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
50512025-01-17T21:12:07.345ZINFOcrucible: Replied this last flush ID: 2206 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
50522025-01-17T21:12:07.345ZINFOcrucible: [2] client re-new 302 jobs since flush 2206 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
50532025-01-17T21:12:07.346ZINFOcrucible: [2] Replayed 302 jobs since flush: 2206 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
5054 ----------------------------------------------------------------
5055 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
5056 Downstairs last five completed: 2508 2507 2506 2505 2504
5057 Upstairs last five completed: 2508 2507 2506 2505 2504
5058 CLIENT: Up:0 ds:0 act:3
5059 Replay: All jobs finished
5060 Main task has work to do, go find it
5061 got dsc Stop(1)
5062 stop 1
5063 [8820] Got stop action so:false kr:false
5064 Jan 17 21:12:09.597 INFO request completed, latency_us: 341, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 26f7adf2-d2af-4e42-8158-b05ff86b5971, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
50652025-01-17T21:12:09.597ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
50662025-01-17T21:12:09.597ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
50672025-01-17T21:12:09.597ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
50682025-01-17T21:12:09.597ZINFOcrucible: client task is exiting = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
5069 Jan 17 21:12:09.597 INFO request completed, latency_us: 155, response_code: 200, uri: /state/cid/1, method: GET, req_id: af0a4b67-9e31-4894-b11b-a44fd4eee3db, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
5070 [8820] Exited with: ExitStatus(unix_wait_status(9))
5071 [8820][1] reports Exit
5072 Jan 17 21:12:13.600 INFO request completed, latency_us: 211, response_code: 200, uri: /state/cid/1, method: GET, req_id: b2192de5-fab7-4fc8-b553-32fc5be1263a, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
5073 001/300 Read block 735 len 28672
5074 002/300 Read block 4704 len 24576
5075 003/300 Write block 1912 len 24576 data: 3 3 3 3 3 3
5076 004/300 Write block 4514 len 20480 data: 2 2 2 2 2
5077 005/300 Flush
5078 006/300 Write block 1149 len 36864 data: 3 2 2 2 2 2 3 4 5
5079 007/300 Write block 2864 len 20480 data: 3 3 3 3 3
5080 008/300 Read block 4634 len 32768
5081 009/300 Read block 2245 len 8192
5082 010/300 Read block 1428 len 28672
5083 011/300 Read block 2665 len 12288
5084 012/300 Read block 3621 len 40960
5085 013/300 Flush
5086 014/300 Read block 330 len 12288
5087 015/300 Read block 4819 len 40960
5088 016/300 Write block 22 len 16384 data: 3 2 3 3
5089 017/300 Read block 2585 len 36864
5090 018/300 Flush
5091 019/300 Write block 1969 len 12288 data: 3 3 3
5092 020/300 Read block 2862 len 40960
5093 021/300 Write block 435 len 32768 data: 2 2 2 2 2 2 2 2
5094 022/300 Read block 2370 len 12288
5095 023/300 Flush
5096 024/300 Read block 3744 len 32768
5097 025/300 Flush
5098 026/300 Read block 4255 len 28672
5099 027/300 Read block 2244 len 12288
5100 028/300 Read block 1954 len 8192
5101 029/300 Write block 116 len 8192 data: 2 2
5102 030/300 Write block 2426 len 28672 data: 2 2 2 2 2 2 2
5103 031/300 Read block 1360 len 20480
5104 032/300 Read block 1758 len 20480
5105 033/300 Write block 766 len 4096 data: 3
5106 034/300 Read block 2089 len 12288
5107 035/300 Flush
5108 036/300 Read block 695 len 4096
5109 037/300 Read block 3474 len 32768
5110 038/300 Read block 1682 len 20480
5111 039/300 Write block 1742 len 16384 data: 3 3 3 3
5112 040/300 Read block 2565 len 32768
5113 041/300 Write block 4935 len 40960 data: 3 2 2 2 2 2 2 3 3 3
5114 042/300 Read block 4777 len 24576
5115 043/300 Read block 3943 len 4096
5116 044/300 Read block 4726 len 24576
5117 045/300 Read block 4988 len 28672
5118 046/300 Read block 2280 len 16384
5119 047/300 Read block 2710 len 20480
5120 048/300 Read block 1722 len 8192
5121 049/300 Read block 1145 len 28672
5122 050/300 Write block 3391 len 4096 data: 2
5123 051/300 Read block 4819 len 16384
5124 052/300 Flush
5125 053/300 Read block 1101 len 32768
5126 054/300 Write block 4376 len 24576 data: 3 3 3 2 2 2
5127 055/300 Read block 2954 len 40960
5128 056/300 Write block 2807 len 40960 data: 4 4 4 3 3 2 2 2 2 2
5129 057/300 Read block 1279 len 28672
5130 058/300 Read block 3972 len 16384
5131 059/300 Write block 115 len 40960 data: 2 3 3 2 2 2 3 3 2 2
5132 060/300 Flush
5133 061/300 Flush
5134 062/300 Read block 2788 len 12288
5135 063/300 Write block 1054 len 36864 data: 4 5 5 6 6 6 5 4 4
5136 064/300 Flush
5137 065/300 Read block 4510 len 40960
5138 066/300 Write block 3465 len 40960 data: 2 2 2 2 2 2 3 3 3 3
5139 067/300 Write block 3501 len 16384 data: 4 4 4 3
5140 068/300 Read block 1321 len 28672
5141 069/300 Read block 1510 len 32768
5142 070/300 Write block 868 len 4096 data: 2
5143 071/300 Read block 2341 len 28672
5144 072/300 Flush
5145 073/300 Write block 3611 len 32768 data: 4 5 3 2 2 3 3 3
5146 074/300 Read block 3751 len 16384
5147 075/300 Read block 1098 len 28672
5148 076/300 Write block 195 len 28672 data: 3 3 3 3 3 3 3
5149 077/300 Write block 116 len 40960 data: 4 4 3 3 3 4 4 3 3 2
5150 078/300 Read block 254 len 4096
5151 079/300 Read block 2181 len 28672
5152 080/300 Flush
5153 081/300 Read block 2282 len 32768
5154 082/300 Read block 2585 len 36864
5155 083/300 Read block 1882 len 40960
5156 084/300 Read block 2162 len 24576
5157 085/300 Read block 2579 len 36864
5158 086/300 Flush
5159 087/300 Write block 4339 len 12288 data: 2 2 2
5160 088/300 Read block 4552 len 24576
5161 089/300 Read block 3072 len 32768
5162 090/300 Read block 418 len 24576
5163 091/300 Write block 4412 len 8192 data: 2 2
5164 092/300 Read block 2919 len 20480
5165 093/300 Flush
5166 094/300 Read block 558 len 32768
5167 095/300 Read block 3378 len 28672
5168 096/300 Read block 957 len 28672
5169 097/300 Read block 2306 len 36864
5170 098/300 Read block 199 len 36864
5171 099/300 Write block 398 len 28672 data: 3 3 3 3 3 3 3
5172 100/300 Read block 2367 len 24576
5173 101/300 Read block 1411 len 16384
5174 102/300 Write block 3915 len 16384 data: 2 2 3 3
5175 103/300 Read block 4286 len 36864
5176 104/300 Write block 4932 len 4096 data: 2
5177 105/300 Flush
5178 106/300 Read block 1705 len 36864
5179 107/300 Flush
5180 108/300 Write block 15 len 28672 data: 2 2 2 3 3 3 3
5181 109/300 Write block 4848 len 20480 data: 2 2 2 2 2
5182 110/300 Read block 4930 len 8192
5183 111/300 Write block 2727 len 20480 data: 2 2 2 2 2
5184 112/300 Read block 3426 len 16384
5185 113/300 Read block 857 len 32768
5186 114/300 Write block 245 len 16384 data: 2 3 2 2
5187 115/300 Write block 1583 len 8192 data: 4 5
5188 116/300 Write block 2478 len 32768 data: 3 3 3 3 3 4 4 4
5189 117/300 Read block 1011 len 40960
5190 118/300 Read block 2002 len 24576
5191 119/300 Read block 2676 len 32768
5192 120/300 Read block 1441 len 20480
5193 121/300 Read block 2219 len 28672
5194 122/300 Read block 4690 len 36864
5195 123/300 Write block 2227 len 20480 data: 2 2 2 2 3
5196 124/300 Write block 4507 len 28672 data: 2 2 2 2 2 2 2
5197 125/300 Read block 1812 len 16384
5198 126/300 Flush
5199 127/300 Write block 1082 len 36864 data: 2 2 3 3 3 3 3 3 3
5200 128/300 Write block 2625 len 20480 data: 2 2 2 2 2
5201 129/300 Write block 2498 len 4096 data: 3
5202 130/300 Write block 1486 len 4096 data: 4
5203 131/300 Write block 616 len 28672 data: 2 2 2 3 2 2 2
5204 132/300 Write block 156 len 16384 data: 3 3 2 2
5205 133/300 Read block 4014 len 36864
5206 134/300 Flush
5207 135/300 Read block 3007 len 32768
5208 136/300 Read block 253 len 40960
5209 137/300 Read block 2104 len 28672
5210 138/300 Write block 1022 len 32768 data: 3 4 4 3 4 4 4 3
5211 139/300 Write block 922 len 32768 data: 3 3 2 2 2 2 2 2
5212 140/300 Read block 2464 len 40960
5213 141/300 Flush
5214 142/300 Read block 2683 len 32768
5215 143/300 Write block 3931 len 24576 data: 4 4 5 5 4 3
5216 144/300 Read block 3804 len 16384
5217 145/300 Write block 155 len 24576 data: 3 4 4 3 3 2
5218 146/300 Read block 4647 len 4096
5219 147/300 Flush
5220 148/300 Read block 3313 len 32768
5221 149/300 Read block 3671 len 12288
5222 150/300 Read block 3276 len 32768
5223 151/300 Read block 203 len 16384
5224 152/300 Write block 1748 len 20480 data: 3 3 3 2 2
5225 153/300 Read block 651 len 4096
5226 154/300 Read block 889 len 16384
5227 155/300 Read block 4031 len 36864
5228 156/300 Read block 144 len 20480
5229 157/300 Read block 230 len 8192
5230 158/300 Read block 586 len 20480
5231 159/300 Write block 2312 len 12288 data: 3 3 3
5232 160/300 Read block 3406 len 20480
5233 161/300 Write block 4922 len 20480 data: 2 2 2 2 2
5234 162/300 Read block 2356 len 40960
5235 163/300 Flush
5236 164/300 Write block 3673 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5237 165/300 Read block 1542 len 4096
5238 166/300 Read block 4607 len 28672
5239 167/300 Write block 3726 len 8192 data: 2 2
5240 168/300 Write block 2358 len 4096 data: 2
5241 169/300 Read block 3451 len 32768
5242 170/300 Write block 3237 len 28672 data: 3 3 2 2 2 2 2
5243 171/300 Read block 861 len 20480
5244 172/300 Write block 3333 len 32768 data: 2 2 3 3 3 3 3 3
5245 173/300 Read block 94 len 12288
5246 174/300 Write block 2948 len 4096 data: 3
5247 175/300 Read block 3718 len 32768
5248 176/300 Write block 1108 len 28672 data: 2 2 2 2 2 2 3
5249 177/300 Read block 3099 len 20480
5250 178/300 Write block 4416 len 12288 data: 3 3 4
5251 179/300 Read block 2134 len 12288
5252 180/300 Write block 2637 len 12288 data: 2 2 2
5253 181/300 Flush
5254 182/300 Read block 2731 len 24576
5255 183/300 Flush
5256 184/300 Write block 1351 len 20480 data: 3 2 2 2 4
5257 185/300 Read block 1146 len 12288
5258 186/300 Flush
5259 187/300 Write block 2177 len 16384 data: 2 2 2 2
5260 188/300 Write block 3883 len 28672 data: 3 3 3 3 2 2 3
5261 189/300 Read block 2121 len 16384
5262 190/300 Read block 4298 len 24576
5263 191/300 Write block 1940 len 20480 data: 2 2 2 2 2
5264 192/300 Write block 4621 len 20480 data: 2 2 2 2 2
5265 193/300 Write block 4014 len 12288 data: 2 2 2
5266 194/300 Read block 1735 len 16384
5267 195/300 Read block 4392 len 4096
5268 196/300 Write block 1190 len 8192 data: 3 3
5269 197/300 Read block 3917 len 36864
5270 198/300 Read block 1058 len 28672
5271 199/300 Read block 4309 len 4096
5272 200/300 Write block 529 len 12288 data: 3 3 4
5273 201/300 Read block 572 len 4096
5274 202/300 Flush
5275 203/300 Write block 4882 len 4096 data: 2
5276 204/300 Read block 781 len 16384
5277 205/300 Write block 1497 len 8192 data: 2 2
5278 206/300 Flush
5279 207/300 Read block 232 len 32768
5280 208/300 Read block 1692 len 16384
5281 209/300 Read block 107 len 8192
5282 210/300 Flush
5283 211/300 Read block 2968 len 20480
5284 212/300 Read block 3639 len 24576
5285 213/300 Write block 3996 len 12288 data: 3 2 2
5286 214/300 Write block 2680 len 16384 data: 2 3 3 3
5287 215/300 Write block 2106 len 16384 data: 3 3 3 3
5288 216/300 Read block 3778 len 36864
5289 217/300 Write block 1665 len 8192 data: 4 4
5290 218/300 Read block 1253 len 8192
5291 219/300 Read block 2639 len 36864
5292 220/300 Read block 3013 len 8192
5293 221/300 Read block 3164 len 32768
5294 222/300 Read block 553 len 8192
5295 223/300 Flush
5296 224/300 Flush
5297 225/300 Write block 3526 len 12288 data: 3 3 2
5298 226/300 Read block 314 len 32768
5299 227/300 Read block 2519 len 8192
5300 228/300 Read block 2923 len 4096
5301 229/300 Read block 864 len 20480
5302 230/300 Read block 4624 len 24576
5303 231/300 Read block 525 len 36864
5304 232/300 Read block 4139 len 36864
5305 233/300 Write block 2145 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5306 234/300 Flush
5307 235/300 Read block 479 len 4096
5308 236/300 Read block 101 len 24576
5309 237/300 Flush
5310 238/300 Read block 881 len 32768
5311 239/300 Read block 1288 len 40960
5312 240/300 Write block 4977 len 12288 data: 3 3 3
5313 241/300 Read block 395 len 28672
5314 242/300 Flush
5315 243/300 Write block 4636 len 16384 data: 3 3 3 3
5316 244/300 Read block 4732 len 16384
5317 245/300 Write block 1250 len 32768 data: 2 2 2 2 2 2 2 2
5318 246/300 Read block 264 len 32768
5319 247/300 Write block 2169 len 36864 data: 2 2 2 2 2 2 2 2 3
5320 248/300 Read block 762 len 8192
5321 249/300 Write block 1062 len 28672 data: 5 4 5 4 3 2 2
5322 250/300 Flush
5323 251/300 Flush
5324 252/300 Read block 1701 len 8192
5325 253/300 Flush
5326 254/300 Flush
5327 255/300 Read block 2478 len 36864
5328 256/300 Flush
5329 257/300 Write block 449 len 12288 data: 2 2 2
5330 258/300 Write block 3810 len 40960 data: 3 3 3 3 3 3 2 2 2 2
5331 259/300 Write block 4146 len 4096 data: 2
5332 260/300 Write block 3537 len 36864 data: 2 2 3 3 3 3 3 3 3
5333 261/300 Read block 2692 len 36864
5334 262/300 Read block 4853 len 20480
5335 263/300 Read block 1353 len 12288
5336 264/300 Read block 410 len 40960
5337 265/300 Flush
5338 266/300 Write block 3166 len 4096 data: 2
5339 267/300 Write block 2238 len 16384 data: 2 2 2 2
5340 268/300 Read block 3412 len 20480
5341 269/300 Write block 698 len 24576 data: 3 3 2 2 2 2
5342 270/300 Read block 2374 len 8192
5343 271/300 Read block 3198 len 36864
5344 272/300 Write block 4676 len 24576 data: 3 4 4 4 4 3
5345 273/300 Write block 1708 len 12288 data: 3 5 4
5346 274/300 Write block 2167 len 36864 data: 2 2 3 3 3 3 3 3 3
5347 275/300 Write block 901 len 12288 data: 5 4 3
5348 276/300 Read block 3910 len 32768
5349 277/300 Flush
5350 278/300 Read block 578 len 32768
5351 279/300 Write block 2988 len 32768 data: 3 3 3 3 3 4 4 4
5352 280/300 Write block 4762 len 4096 data: 4
5353 281/300 Write block 2120 len 32768 data: 2 2 2 2 2 2 2 2
5354 282/300 Read block 2166 len 20480
5355 283/300 Write block 3783 len 32768 data: 2 2 2 2 2 3 2 2
5356 284/300 Read block 2531 len 4096
5357 285/300 Write block 2813 len 20480 data: 3 3 3 3 2
5358 286/300 Read block 2508 len 28672
5359 287/300 Read block 1623 len 4096
5360 288/300 Read block 371 len 8192
5361 289/300 Flush
5362 290/300 Write block 1360 len 16384 data: 3 3 2 2
5363 291/300 Write block 4609 len 24576 data: 2 2 2 2 2 2
5364 292/300 Write block 348 len 4096 data: 3
5365 293/300 Read block 4504 len 24576
5366 294/300 Read block 2905 len 28672
5367 295/300 Write block 1268 len 20480 data: 2 2 2 2 2
5368 296/300 Read block 1200 len 16384
5369 297/300 Write block 2318 len 28672 data: 3 2 2 2 3 2 2
5370 298/300 Read block 2069 len 28672
5371 299/300 Flush
5372 300/300 Read block 3360 len 4096
5373 301/300 Read block 1589 len 24576
5374 Main task has work to do, go find it
5375 got dsc Start(1)
5376 start 1
5377 Jan 17 21:12:13.850 INFO request completed, latency_us: 209, response_code: 204, uri: /start/cid/1, method: POST, req_id: 4b799d01-142c-4b28-915b-7cb507127ba2, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
5378 Replay: started 1, returned:Ok(())
5379 ----------------------------------------------------------------
5380 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
5381 States: Active Offline Active
5382 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5383 2509 Acked Read 7 Done Sent Done false
5384 2510 Acked Read 6 Done Sent Done false
5385 2511 Acked Write 6 Done Sent Done false
5386 2512 Acked Write 5 Done Sent Done false
5387 2513 Acked Flush 0 Done Sent Done false
5388 2514 Acked Write 9 Done Sent Done false
5389 2515 Acked Write 5 Done Sent Done false
5390 2516 Acked Read 8 Done Sent Done false
5391 2517 Acked Read 2 Done Sent Done false
5392 2518 Acked Read 7 Done Sent Done false
5393 2519 Acked Read 3 Done Sent Done false
5394 2520 Acked Read 10 Done Sent Done false
5395 2521 Acked Flush 0 Done Sent Done false
5396 2522 Acked Read 3 Done Sent Done false
5397 2523 Acked Read 10 Done Sent Done false
5398 2524 Acked Write 4 Done Sent Done false
5399 2525 Acked Read 9 Done Sent Done false
5400 2526 Acked Flush 0 Done Sent Done false
5401 2527 Acked Write 3 Done Sent Done false
5402 2528 Acked Read 10 Done Sent Done false
5403 2529 Acked Write 8 Done Sent Done false
5404 2530 Acked Read 3 Done Sent Done false
5405 2531 Acked Flush 0 Done Sent Done false
5406 2532 Acked Read 8 Done Sent Done false
5407 2533 Acked Flush 0 Done Sent Done false
5408 2534 Acked Read 7 Done Sent Done false
5409 2535 Acked Read 3 Done Sent Done false
5410 2536 Acked Read 2 Done Sent Done false
5411 2537 Acked Write 2 Done Sent Done false
5412 2538 Acked Write 7 Done Sent Done false
5413 2539 Acked Read 5 Done Sent Done false
5414 2540 Acked Read 5 Done Sent Done false
5415 2541 Acked Write 1 Done Sent Done false
5416 2542 Acked Read 3 Done Sent Done false
5417 2543 Acked Flush 0 Done Sent Done false
5418 2544 Acked Read 1 Done Sent Done false
5419 2545 Acked Read 8 Done Sent Done false
5420 2546 Acked Read 5 Done Sent Done false
5421 2547 Acked Write 4 Done Sent Done false
5422 2548 Acked Read 8 Done Sent Done false
5423 2549 Acked Write 10 Done Sent Done false
5424 2550 Acked Read 6 Done Sent Done false
5425 2551 Acked Read 1 Done Sent Done false
5426 2552 Acked Read 6 Done Sent Done false
5427 2553 Acked Read 7 Done Sent Done false
5428 2554 Acked Read 4 Done Sent Done false
5429 2555 Acked Read 5 Done Sent Done false
5430 2556 Acked Read 2 Done Sent Done false
5431 2557 Acked Read 7 Done Sent Done false
5432 2558 Acked Write 1 Done Sent Done false
5433 2559 Acked Read 4 Done Sent Done false
5434 2560 Acked Flush 0 Done Sent Done false
5435 2561 Acked Read 8 Done Sent Done false
5436 2562 Acked Write 6 Done Sent Done false
5437 2563 Acked Read 10 Done Sent Done false
5438 2564 Acked Write 10 Done Sent Done false
5439 2565 Acked Read 7 Done Sent Done false
5440 2566 Acked Read 4 Done Sent Done false
5441 2567 Acked Write 10 Done Sent Done false
5442 2568 Acked Flush 0 Done Sent Done false
5443 2569 Acked Flush 0 Done Sent Done false
5444 2570 Acked Read 3 Done Sent Done false
5445 2571 Acked Write 9 Done Sent Done false
5446 2572 Acked Flush 0 Done Sent Done false
5447 2573 Acked Read 10 Done Sent Done false
5448 2574 Acked Write 10 Done Sent Done false
5449 2575 Acked Write 4 Done Sent Done false
5450 2576 Acked Read 7 Done Sent Done false
5451 2577 Acked Read 8 Done Sent Done false
5452 2578 Acked Write 1 Done Sent Done false
5453 2579 Acked Read 7 Done Sent Done false
5454 2580 Acked Flush 0 Done Sent Done false
5455 2581 Acked Write 8 Done Sent Done false
5456 2582 Acked Read 4 Done Sent Done false
5457 2583 Acked Read 7 Done Sent Done false
5458 2584 Acked Write 7 Done Sent Done false
5459 2585 Acked Write 10 Done Sent Done false
5460 2586 Acked Read 1 Done Sent Done false
5461 2587 Acked Read 7 Done Sent Done false
5462 2588 Acked Flush 0 Done Sent Done false
5463 2589 Acked Read 8 Done Sent Done false
5464 2590 Acked Read 9 Done Sent Done false
5465 2591 Acked Read 10 Done Sent Done false
5466 2592 Acked Read 6 Done Sent Done false
5467 2593 Acked Read 9 Done Sent Done false
5468 2594 Acked Flush 0 Done Sent Done false
5469 2595 Acked Write 3 Done Sent Done false
5470 2596 Acked Read 6 Done Sent Done false
5471 2597 Acked Read 8 Done Sent Done false
5472 2598 Acked Read 6 Done Sent Done false
5473 2599 Acked Write 2 Done Sent Done false
5474 2600 Acked Read 5 Done Sent Done false
5475 2601 Acked Flush 0 Done Sent Done false
5476 2602 Acked Read 8 Done Sent Done false
5477 2603 Acked Read 7 Done Sent Done false
5478 2604 Acked Read 7 Done Sent Done false
5479 2605 Acked Read 9 Done Sent Done false
5480 2606 Acked Read 9 Done Sent Done false
5481 2607 Acked Write 7 Done Sent Done false
5482 2608 Acked Read 6 Done Sent Done false
5483 2609 Acked Read 4 Done Sent Done false
5484 2610 Acked Write 4 Done Sent Done false
5485 2611 Acked Read 9 Done Sent Done false
5486 2612 Acked Write 1 Done Sent Done false
5487 2613 Acked Flush 0 Done Sent Done false
5488 2614 Acked Read 9 Done Sent Done false
5489 2615 Acked Flush 0 Done Sent Done false
5490 2616 Acked Write 7 Done Sent Done false
5491 2617 Acked Write 5 Done Sent Done false
5492 2618 Acked Read 2 Done Sent Done false
5493 2619 Acked Write 5 Done Sent Done false
5494 2620 Acked Read 4 Done Sent Done false
5495 2621 Acked Read 8 Done Sent Done false
5496 2622 Acked Write 4 Done Sent Done false
5497 2623 Acked Write 2 Done Sent Done false
5498 2624 Acked Write 8 Done Sent Done false
5499 2625 Acked Read 10 Done Sent Done false
5500 2626 Acked Read 6 Done Sent Done false
5501 2627 Acked Read 8 Done Sent Done false
5502 2628 Acked Read 5 Done Sent Done false
5503 2629 Acked Read 7 Done Sent Done false
5504 2630 Acked Read 9 Done Sent Done false
5505 2631 Acked Write 5 Done Sent Done false
5506 2632 Acked Write 7 Done Sent Done false
5507 2633 Acked Read 4 Done Sent Done false
5508 2634 Acked Flush 0 Done Sent Done false
5509 2635 Acked Write 9 Done Sent Done false
5510 2636 Acked Write 5 Done Sent Done false
5511 2637 Acked Write 1 Done Sent Done false
5512 2638 Acked Write 1 Done Sent Done false
5513 2639 Acked Write 7 Done Sent Done false
5514 2640 Acked Write 4 Done Sent Done false
5515 2641 Acked Read 9 Done Sent Done false
5516 2642 Acked Flush 0 Done Sent Done false
5517 2643 Acked Read 8 Done Sent Done false
5518 2644 Acked Read 10 Done Sent Done false
5519 2645 Acked Read 7 Done Sent Done false
5520 2646 Acked Write 8 Done Sent Done false
5521 2647 Acked Write 8 Done Sent Done false
5522 2648 Acked Read 10 Done Sent Done false
5523 2649 Acked Flush 0 Done Sent Done false
5524 2650 Acked Read 8 Done Sent Done false
5525 2651 Acked Write 6 Done Sent Done false
5526 2652 Acked Read 4 Done Sent Done false
5527 2653 Acked Write 6 Done Sent Done false
5528 2654 Acked Read 1 Done Sent Done false
5529 2655 Acked Flush 0 Done Sent Done false
5530 2656 Acked Read 8 Done Sent Done false
5531 2657 Acked Read 3 Done Sent Done false
5532 2658 Acked Read 8 Done Sent Done false
5533 2659 Acked Read 4 Done Sent Done false
5534 2660 Acked Write 5 Done Sent Done false
5535 2661 Acked Read 1 Done Sent Done false
5536 2662 Acked Read 4 Done Sent Done false
5537 2663 Acked Read 9 Done Sent Done false
5538 2664 Acked Read 5 Done Sent Done false
5539 2665 Acked Read 2 Done Sent Done false
5540 2666 Acked Read 5 Done Sent Done false
5541 2667 Acked Write 3 Done Sent Done false
5542 2668 Acked Read 5 Done Sent Done false
5543 2669 Acked Write 5 Done Sent Done false
5544 2670 Acked Read 10 Done Sent Done false
5545 2671 Acked Flush 0 Done Sent Done false
5546 2672 Acked Write 10 Done Sent Done false
5547 2673 Acked Read 1 Done Sent Done false
5548 2674 Acked Read 7 Done Sent Done false
5549 2675 Acked Write 2 Done Sent Done false
5550 2676 Acked Write 1 Done Sent Done false
5551 2677 Acked Read 8 Done Sent Done false
5552 2678 Acked Write 7 Done Sent Done false
5553 2679 Acked Read 5 Done Sent Done false
5554 2680 Acked Write 8 Done Sent Done false
5555 2681 Acked Read 3 Done Sent Done false
5556 2682 Acked Write 1 Done Sent Done false
5557 2683 Acked Read 8 Done Sent Done false
5558 2684 Acked Write 7 Done Sent Done false
5559 2685 Acked Read 5 Done Sent Done false
5560 2686 Acked Write 3 Done Sent Done false
5561 2687 Acked Read 3 Done Sent Done false
5562 2688 Acked Write 3 Done Sent Done false
5563 2689 Acked Flush 0 Done Sent Done false
5564 2690 Acked Read 6 Done Sent Done false
5565 2691 Acked Flush 0 Done Sent Done false
5566 2692 Acked Write 5 Done Sent Done false
5567 2693 Acked Read 3 Done Sent Done false
5568 2694 Acked Flush 0 Done Sent Done false
5569 2695 Acked Write 4 Done Sent Done false
5570 2696 Acked Write 7 Done Sent Done false
5571 2697 Acked Read 4 Done Sent Done false
5572 2698 Acked Read 6 Done Sent Done false
5573 2699 Acked Write 5 Done Sent Done false
5574 2700 Acked Write 5 Done Sent Done false
5575 2701 Acked Write 3 Done Sent Done false
5576 2702 Acked Read 4 Done Sent Done false
5577 2703 Acked Read 1 Done Sent Done false
5578 2704 Acked Write 2 Done Sent Done false
5579 2705 Acked Read 9 Done Sent Done false
5580 2706 Acked Read 7 Done Sent Done false
5581 2707 Acked Read 1 Done Sent Done false
5582 2708 Acked Write 3 Done Sent Done false
5583 2709 Acked Read 1 Done Sent Done false
5584 2710 Acked Flush 0 Done Sent Done false
5585 2711 Acked Write 1 Done Sent Done false
5586 2712 Acked Read 4 Done Sent Done false
5587 2713 Acked Write 2 Done Sent Done false
5588 2714 Acked Flush 0 Done Sent Done false
5589 2715 Acked Read 8 Done Sent Done false
5590 2716 Acked Read 4 Done Sent Done false
5591 2717 Acked Read 2 Done Sent Done false
5592 2718 Acked Flush 0 Done Sent Done false
5593 2719 Acked Read 5 Done Sent Done false
5594 2720 Acked Read 6 Done Sent Done false
5595 2721 Acked Write 3 Done Sent Done false
5596 2722 Acked Write 4 Done Sent Done false
5597 2723 Acked Write 4 Done Sent Done false
5598 2724 Acked Read 9 Done Sent Done false
5599 2725 Acked Write 2 Done Sent Done false
5600 2726 Acked Read 2 Done Sent Done false
5601 2727 Acked Read 9 Done Sent Done false
5602 2728 Acked Read 2 Done Sent Done false
5603 2729 Acked Read 8 Done Sent Done false
5604 2730 Acked Read 2 Done Sent Done false
5605 2731 Acked Flush 0 Done Sent Done false
5606 2732 Acked Flush 0 Done Sent Done false
5607 2733 Acked Write 3 Done Sent Done false
5608 2734 Acked Read 8 Done Sent Done false
5609 2735 Acked Read 2 Done Sent Done false
5610 2736 Acked Read 1 Done Sent Done false
5611 2737 Acked Read 5 Done Sent Done false
5612 2738 Acked Read 6 Done Sent Done false
5613 2739 Acked Read 9 Done Sent Done false
5614 2740 Acked Read 9 Done Sent Done false
5615 2741 Acked Write 10 Done Sent Done false
5616 2742 Acked Flush 0 Done Sent Done false
5617 2743 Acked Read 1 Done Sent Done false
5618 2744 Acked Read 6 Done Sent Done false
5619 2745 Acked Flush 0 Done Sent Done false
5620 2746 Acked Read 8 Done Sent Done false
5621 2747 Acked Read 10 Done Sent Done false
5622 2748 Acked Write 3 Done Sent Done false
5623 2749 Acked Read 7 Done Sent Done false
5624 2750 Acked Flush 0 Done Sent Done false
5625 2751 Acked Write 4 Done Sent Done false
5626 2752 Acked Read 4 Done Sent Done false
5627 2753 Acked Write 8 Done Sent Done false
5628 2754 Acked Read 8 Done Sent Done false
5629 2755 Acked Write 9 Done Sent Done false
5630 2756 Acked Read 2 Done Sent Done false
5631 2757 Acked Write 7 Done Sent Done false
5632 2758 Acked Flush 0 Done Sent Done false
5633 2759 Acked Flush 0 Done Sent Done false
5634 2760 Acked Read 2 Done Sent Done false
5635 2761 Acked Flush 0 Done Sent Done false
5636 2762 Acked Flush 0 Done Sent Done false
5637 2763 Acked Read 9 Done Sent Done false
5638 2764 Acked Flush 0 Done Sent Done false
5639 2765 Acked Write 3 Done Sent Done false
5640 2766 Acked Write 10 Done Sent Done false
5641 2767 Acked Write 1 Done Sent Done false
5642 2768 Acked Write 9 Done Sent Done false
5643 2769 Acked Read 9 Done Sent Done false
5644 2770 Acked Read 5 Done Sent Done false
5645 2771 Acked Read 3 Done Sent Done false
5646 2772 Acked Read 10 Done Sent Done false
5647 2773 Acked Flush 0 Done Sent Done false
5648 2774 Acked Write 1 Done Sent Done false
5649 2775 Acked Write 4 Done Sent Done false
5650 2776 Acked Read 5 Done Sent Done false
5651 2777 Acked Write 6 Done Sent Done false
5652 2778 Acked Read 2 Done Sent Done false
5653 2779 Acked Read 9 Done Sent Done false
5654 2780 Acked Write 6 Done Sent Done false
5655 2781 Acked Write 3 Done Sent Done false
5656 2782 Acked Write 9 Done Sent Done false
5657 2783 Acked Write 3 Done Sent Done false
5658 2784 Acked Read 8 Done Sent Done false
5659 2785 Acked Flush 0 Done Sent Done false
5660 2786 Acked Read 8 Done Sent Done false
5661 2787 Acked Write 8 Done Sent Done false
5662 2788 Acked Write 1 Done Sent Done false
5663 2789 Acked Write 8 Done Sent Done false
5664 2790 Acked Read 5 Done Sent Done false
5665 2791 Acked Write 8 Done Sent Done false
5666 2792 Acked Read 1 Done Sent Done false
5667 2793 Acked Write 5 Done Sent Done false
5668 2794 Acked Read 7 Done Sent Done false
5669 2795 Acked Read 1 Done Sent Done false
5670 2796 Acked Read 2 Done Sent Done false
5671 2797 Acked Flush 0 Done Sent Done false
5672 2798 Acked Write 4 Done Sent Done false
5673 2799 Acked Write 6 Done Sent Done false
5674 2800 Acked Write 1 Done Sent Done false
5675 2801 Acked Read 6 Done Sent Done false
5676 2802 Acked Read 7 Done Sent Done false
5677 2803 Acked Write 5 Done Sent Done false
5678 2804 Acked Read 4 Done Sent Done false
5679 2805 Acked Write 7 Done Sent Done false
5680 2806 Acked Read 7 Done Sent Done false
5681 2807 Acked Flush 0 Done Sent Done false
5682 2808 Acked Read 1 Done Sent Done false
5683 2809 Acked Read 6 Done Sent Done false
5684 STATES DS:0 DS:1 DS:2 TOTAL
5685 Sent 0 301 0 301
5686 Done 301 0 301 602
5687 Skipped 0 0 0 0
5688 Error 0 0 0 0
5689 Last Flush: 2807 2508 2807
5690 Downstairs last five completed: 2508 2507 2506 2505 2504
5691 Upstairs last five completed: 2809 2808 2807 2806 2805
5692 CLIENT: Up:0 ds:301 act:2
5693 [8820] got start action
5694 [8820] I am going to restart
5695 Starting downstairs at port 8820
5696 Make output file at "/tmp/dsc/downstairs-8820.txt"
5697 [8820][1] reports Starting
5698 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1206)
5699 [8820][1] reports Running
5700 ----------------------------------------------------------------
5701 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
5702 States: Active Offline Active
5703 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5704 2509 Acked Read 7 Done Sent Done false
5705 2510 Acked Read 6 Done Sent Done false
5706 2511 Acked Write 6 Done Sent Done false
5707 2512 Acked Write 5 Done Sent Done false
5708 2513 Acked Flush 0 Done Sent Done false
5709 2514 Acked Write 9 Done Sent Done false
5710 2515 Acked Write 5 Done Sent Done false
5711 2516 Acked Read 8 Done Sent Done false
5712 2517 Acked Read 2 Done Sent Done false
5713 2518 Acked Read 7 Done Sent Done false
5714 2519 Acked Read 3 Done Sent Done false
5715 2520 Acked Read 10 Done Sent Done false
5716 2521 Acked Flush 0 Done Sent Done false
5717 2522 Acked Read 3 Done Sent Done false
5718 2523 Acked Read 10 Done Sent Done false
5719 2524 Acked Write 4 Done Sent Done false
5720 2525 Acked Read 9 Done Sent Done false
5721 2526 Acked Flush 0 Done Sent Done false
5722 2527 Acked Write 3 Done Sent Done false
5723 2528 Acked Read 10 Done Sent Done false
5724 2529 Acked Write 8 Done Sent Done false
5725 2530 Acked Read 3 Done Sent Done false
5726 2531 Acked Flush 0 Done Sent Done false
5727 2532 Acked Read 8 Done Sent Done false
5728 2533 Acked Flush 0 Done Sent Done false
5729 2534 Acked Read 7 Done Sent Done false
5730 2535 Acked Read 3 Done Sent Done false
5731 2536 Acked Read 2 Done Sent Done false
5732 2537 Acked Write 2 Done Sent Done false
5733 2538 Acked Write 7 Done Sent Done false
5734 2539 Acked Read 5 Done Sent Done false
5735 2540 Acked Read 5 Done Sent Done false
5736 2541 Acked Write 1 Done Sent Done false
5737 2542 Acked Read 3 Done Sent Done false
5738 2543 Acked Flush 0 Done Sent Done false
5739 2544 Acked Read 1 Done Sent Done false
5740 2545 Acked Read 8 Done Sent Done false
5741 2546 Acked Read 5 Done Sent Done false
5742 2547 Acked Write 4 Done Sent Done false
5743 2548 Acked Read 8 Done Sent Done false
5744 2549 Acked Write 10 Done Sent Done false
5745 2550 Acked Read 6 Done Sent Done false
5746 2551 Acked Read 1 Done Sent Done false
5747 2552 Acked Read 6 Done Sent Done false
5748 2553 Acked Read 7 Done Sent Done false
5749 2554 Acked Read 4 Done Sent Done false
5750 2555 Acked Read 5 Done Sent Done false
5751 2556 Acked Read 2 Done Sent Done false
5752 2557 Acked Read 7 Done Sent Done false
5753 2558 Acked Write 1 Done Sent Done false
5754 2559 Acked Read 4 Done Sent Done false
5755 2560 Acked Flush 0 Done Sent Done false
5756 2561 Acked Read 8 Done Sent Done false
5757 2562 Acked Write 6 Done Sent Done false
5758 2563 Acked Read 10 Done Sent Done false
5759 2564 Acked Write 10 Done Sent Done false
5760 2565 Acked Read 7 Done Sent Done false
5761 2566 Acked Read 4 Done Sent Done false
5762 2567 Acked Write 10 Done Sent Done false
5763 2568 Acked Flush 0 Done Sent Done false
5764 2569 Acked Flush 0 Done Sent Done false
5765 2570 Acked Read 3 Done Sent Done false
5766 2571 Acked Write 9 Done Sent Done false
5767 2572 Acked Flush 0 Done Sent Done false
5768 2573 Acked Read 10 Done Sent Done false
5769 2574 Acked Write 10 Done Sent Done false
5770 2575 Acked Write 4 Done Sent Done false
5771 2576 Acked Read 7 Done Sent Done false
5772 2577 Acked Read 8 Done Sent Done false
5773 2578 Acked Write 1 Done Sent Done false
5774 2579 Acked Read 7 Done Sent Done false
5775 2580 Acked Flush 0 Done Sent Done false
5776 2581 Acked Write 8 Done Sent Done false
5777 2582 Acked Read 4 Done Sent Done false
5778 2583 Acked Read 7 Done Sent Done false
5779 2584 Acked Write 7 Done Sent Done false
5780 2585 Acked Write 10 Done Sent Done false
5781 2586 Acked Read 1 Done Sent Done false
5782 2587 Acked Read 7 Done Sent Done false
5783 2588 Acked Flush 0 Done Sent Done false
5784 2589 Acked Read 8 Done Sent Done false
5785 2590 Acked Read 9 Done Sent Done false
5786 2591 Acked Read 10 Done Sent Done false
5787 2592 Acked Read 6 Done Sent Done false
5788 2593 Acked Read 9 Done Sent Done false
5789 2594 Acked Flush 0 Done Sent Done false
5790 2595 Acked Write 3 Done Sent Done false
5791 2596 Acked Read 6 Done Sent Done false
5792 2597 Acked Read 8 Done Sent Done false
5793 2598 Acked Read 6 Done Sent Done false
5794 2599 Acked Write 2 Done Sent Done false
5795 2600 Acked Read 5 Done Sent Done false
5796 2601 Acked Flush 0 Done Sent Done false
5797 2602 Acked Read 8 Done Sent Done false
5798 2603 Acked Read 7 Done Sent Done false
5799 2604 Acked Read 7 Done Sent Done false
5800 2605 Acked Read 9 Done Sent Done false
5801 2606 Acked Read 9 Done Sent Done false
5802 2607 Acked Write 7 Done Sent Done false
5803 2608 Acked Read 6 Done Sent Done false
5804 2609 Acked Read 4 Done Sent Done false
5805 2610 Acked Write 4 Done Sent Done false
5806 2611 Acked Read 9 Done Sent Done false
5807 2612 Acked Write 1 Done Sent Done false
5808 2613 Acked Flush 0 Done Sent Done false
5809 2614 Acked Read 9 Done Sent Done false
5810 2615 Acked Flush 0 Done Sent Done false
5811 2616 Acked Write 7 Done Sent Done false
5812 2617 Acked Write 5 Done Sent Done false
5813 2618 Acked Read 2 Done Sent Done false
5814 2619 Acked Write 5 Done Sent Done false
5815 2620 Acked Read 4 Done Sent Done false
5816 2621 Acked Read 8 Done Sent Done false
5817 2622 Acked Write 4 Done Sent Done false
5818 2623 Acked Write 2 Done Sent Done false
5819 2624 Acked Write 8 Done Sent Done false
5820 2625 Acked Read 10 Done Sent Done false
5821 2626 Acked Read 6 Done Sent Done false
5822 2627 Acked Read 8 Done Sent Done false
5823 2628 Acked Read 5 Done Sent Done false
5824 2629 Acked Read 7 Done Sent Done false
5825 2630 Acked Read 9 Done Sent Done false
5826 2631 Acked Write 5 Done Sent Done false
5827 2632 Acked Write 7 Done Sent Done false
5828 2633 Acked Read 4 Done Sent Done false
5829 2634 Acked Flush 0 Done Sent Done false
5830 2635 Acked Write 9 Done Sent Done false
5831 2636 Acked Write 5 Done Sent Done false
5832 2637 Acked Write 1 Done Sent Done false
5833 2638 Acked Write 1 Done Sent Done false
5834 2639 Acked Write 7 Done Sent Done false
5835 2640 Acked Write 4 Done Sent Done false
5836 2641 Acked Read 9 Done Sent Done false
5837 2642 Acked Flush 0 Done Sent Done false
5838 2643 Acked Read 8 Done Sent Done false
5839 2644 Acked Read 10 Done Sent Done false
5840 2645 Acked Read 7 Done Sent Done false
5841 2646 Acked Write 8 Done Sent Done false
5842 2647 Acked Write 8 Done Sent Done false
5843 2648 Acked Read 10 Done Sent Done false
5844 2649 Acked Flush 0 Done Sent Done false
5845 2650 Acked Read 8 Done Sent Done false
5846 2651 Acked Write 6 Done Sent Done false
5847 2652 Acked Read 4 Done Sent Done false
5848 2653 Acked Write 6 Done Sent Done false
5849 2654 Acked Read 1 Done Sent Done false
5850 2655 Acked Flush 0 Done Sent Done false
5851 2656 Acked Read 8 Done Sent Done false
5852 2657 Acked Read 3 Done Sent Done false
5853 2658 Acked Read 8 Done Sent Done false
5854 2659 Acked Read 4 Done Sent Done false
5855 2660 Acked Write 5 Done Sent Done false
5856 2661 Acked Read 1 Done Sent Done false
5857 2662 Acked Read 4 Done Sent Done false
5858 2663 Acked Read 9 Done Sent Done false
5859 2664 Acked Read 5 Done Sent Done false
5860 2665 Acked Read 2 Done Sent Done false
5861 2666 Acked Read 5 Done Sent Done false
5862 2667 Acked Write 3 Done Sent Done false
5863 2668 Acked Read 5 Done Sent Done false
5864 2669 Acked Write 5 Done Sent Done false
5865 2670 Acked Read 10 Done Sent Done false
5866 2671 Acked Flush 0 Done Sent Done false
5867 2672 Acked Write 10 Done Sent Done false
5868 2673 Acked Read 1 Done Sent Done false
5869 2674 Acked Read 7 Done Sent Done false
5870 2675 Acked Write 2 Done Sent Done false
5871 2676 Acked Write 1 Done Sent Done false
5872 2677 Acked Read 8 Done Sent Done false
5873 2678 Acked Write 7 Done Sent Done false
5874 2679 Acked Read 5 Done Sent Done false
5875 2680 Acked Write 8 Done Sent Done false
5876 2681 Acked Read 3 Done Sent Done false
5877 2682 Acked Write 1 Done Sent Done false
5878 2683 Acked Read 8 Done Sent Done false
5879 2684 Acked Write 7 Done Sent Done false
5880 2685 Acked Read 5 Done Sent Done false
5881 2686 Acked Write 3 Done Sent Done false
5882 2687 Acked Read 3 Done Sent Done false
5883 2688 Acked Write 3 Done Sent Done false
5884 2689 Acked Flush 0 Done Sent Done false
5885 2690 Acked Read 6 Done Sent Done false
5886 2691 Acked Flush 0 Done Sent Done false
5887 2692 Acked Write 5 Done Sent Done false
5888 2693 Acked Read 3 Done Sent Done false
5889 2694 Acked Flush 0 Done Sent Done false
5890 2695 Acked Write 4 Done Sent Done false
5891 2696 Acked Write 7 Done Sent Done false
5892 2697 Acked Read 4 Done Sent Done false
5893 2698 Acked Read 6 Done Sent Done false
5894 2699 Acked Write 5 Done Sent Done false
5895 2700 Acked Write 5 Done Sent Done false
5896 2701 Acked Write 3 Done Sent Done false
5897 2702 Acked Read 4 Done Sent Done false
5898 2703 Acked Read 1 Done Sent Done false
5899 2704 Acked Write 2 Done Sent Done false
5900 2705 Acked Read 9 Done Sent Done false
5901 2706 Acked Read 7 Done Sent Done false
5902 2707 Acked Read 1 Done Sent Done false
5903 2708 Acked Write 3 Done Sent Done false
5904 2709 Acked Read 1 Done Sent Done false
5905 2710 Acked Flush 0 Done Sent Done false
5906 2711 Acked Write 1 Done Sent Done false
5907 2712 Acked Read 4 Done Sent Done false
5908 2713 Acked Write 2 Done Sent Done false
5909 2714 Acked Flush 0 Done Sent Done false
5910 2715 Acked Read 8 Done Sent Done false
5911 2716 Acked Read 4 Done Sent Done false
5912 2717 Acked Read 2 Done Sent Done false
5913 2718 Acked Flush 0 Done Sent Done false
5914 2719 Acked Read 5 Done Sent Done false
5915 2720 Acked Read 6 Done Sent Done false
5916 2721 Acked Write 3 Done Sent Done false
5917 2722 Acked Write 4 Done Sent Done false
5918 2723 Acked Write 4 Done Sent Done false
5919 2724 Acked Read 9 Done Sent Done false
5920 2725 Acked Write 2 Done Sent Done false
5921 2726 Acked Read 2 Done Sent Done false
5922 2727 Acked Read 9 Done Sent Done false
5923 2728 Acked Read 2 Done Sent Done false
5924 2729 Acked Read 8 Done Sent Done false
5925 2730 Acked Read 2 Done Sent Done false
5926 2731 Acked Flush 0 Done Sent Done false
5927 2732 Acked Flush 0 Done Sent Done false
5928 2733 Acked Write 3 Done Sent Done false
5929 2734 Acked Read 8 Done Sent Done false
5930 2735 Acked Read 2 Done Sent Done false
5931 2736 Acked Read 1 Done Sent Done false
5932 2737 Acked Read 5 Done Sent Done false
5933 2738 Acked Read 6 Done Sent Done false
5934 2739 Acked Read 9 Done Sent Done false
5935 2740 Acked Read 9 Done Sent Done false
5936 2741 Acked Write 10 Done Sent Done false
5937 2742 Acked Flush 0 Done Sent Done false
5938 2743 Acked Read 1 Done Sent Done false
5939 2744 Acked Read 6 Done Sent Done false
5940 2745 Acked Flush 0 Done Sent Done false
5941 2746 Acked Read 8 Done Sent Done false
5942 2747 Acked Read 10 Done Sent Done false
5943 2748 Acked Write 3 Done Sent Done false
5944 2749 Acked Read 7 Done Sent Done false
5945 2750 Acked Flush 0 Done Sent Done false
5946 2751 Acked Write 4 Done Sent Done false
5947 2752 Acked Read 4 Done Sent Done false
5948 2753 Acked Write 8 Done Sent Done false
5949 2754 Acked Read 8 Done Sent Done false
5950 2755 Acked Write 9 Done Sent Done false
5951 2756 Acked Read 2 Done Sent Done false
5952 2757 Acked Write 7 Done Sent Done false
5953 2758 Acked Flush 0 Done Sent Done false
5954 2759 Acked Flush 0 Done Sent Done false
5955 2760 Acked Read 2 Done Sent Done false
5956 2761 Acked Flush 0 Done Sent Done false
5957 2762 Acked Flush 0 Done Sent Done false
5958 2763 Acked Read 9 Done Sent Done false
5959 2764 Acked Flush 0 Done Sent Done false
5960 2765 Acked Write 3 Done Sent Done false
5961 2766 Acked Write 10 Done Sent Done false
5962 2767 Acked Write 1 Done Sent Done false
5963 2768 Acked Write 9 Done Sent Done false
5964 2769 Acked Read 9 Done Sent Done false
5965 2770 Acked Read 5 Done Sent Done false
5966 2771 Acked Read 3 Done Sent Done false
5967 2772 Acked Read 10 Done Sent Done false
5968 2773 Acked Flush 0 Done Sent Done false
5969 2774 Acked Write 1 Done Sent Done false
5970 2775 Acked Write 4 Done Sent Done false
5971 2776 Acked Read 5 Done Sent Done false
5972 2777 Acked Write 6 Done Sent Done false
5973 2778 Acked Read 2 Done Sent Done false
5974 2779 Acked Read 9 Done Sent Done false
5975 2780 Acked Write 6 Done Sent Done false
5976 2781 Acked Write 3 Done Sent Done false
5977 2782 Acked Write 9 Done Sent Done false
5978 2783 Acked Write 3 Done Sent Done false
5979 2784 Acked Read 8 Done Sent Done false
5980 2785 Acked Flush 0 Done Sent Done false
5981 2786 Acked Read 8 Done Sent Done false
5982 2787 Acked Write 8 Done Sent Done false
5983 2788 Acked Write 1 Done Sent Done false
5984 2789 Acked Write 8 Done Sent Done false
5985 2790 Acked Read 5 Done Sent Done false
5986 2791 Acked Write 8 Done Sent Done false
5987 2792 Acked Read 1 Done Sent Done false
5988 2793 Acked Write 5 Done Sent Done false
5989 2794 Acked Read 7 Done Sent Done false
5990 2795 Acked Read 1 Done Sent Done false
5991 2796 Acked Read 2 Done Sent Done false
5992 2797 Acked Flush 0 Done Sent Done false
5993 2798 Acked Write 4 Done Sent Done false
5994 2799 Acked Write 6 Done Sent Done false
5995 2800 Acked Write 1 Done Sent Done false
5996 2801 Acked Read 6 Done Sent Done false
5997 2802 Acked Read 7 Done Sent Done false
5998 2803 Acked Write 5 Done Sent Done false
5999 2804 Acked Read 4 Done Sent Done false
6000 2805 Acked Write 7 Done Sent Done false
6001 2806 Acked Read 7 Done Sent Done false
6002 2807 Acked Flush 0 Done Sent Done false
6003 2808 Acked Read 1 Done Sent Done false
6004 2809 Acked Read 6 Done Sent Done false
6005 2810 Acked Flush 0 Done Sent Done false
6006 STATES DS:0 DS:1 DS:2 TOTAL
6007 Sent 0 302 0 302
6008 Done 302 0 302 604
6009 Skipped 0 0 0 0
6010 Error 0 0 0 0
6011 Last Flush: 2810 2508 2810
6012 Downstairs last five completed: 2508 2507 2506 2505 2504
6013 Upstairs last five completed: 2810 2809 2808 2807 2806
6014 CLIENT: Up:0 ds:302 act:2
60152025-01-17T21:12:19.599ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
60162025-01-17T21:12:19.599ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:34596) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
60172025-01-17T21:12:19.600ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
60182025-01-17T21:12:19.601ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
60192025-01-17T21:12:19.601ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
60202025-01-17T21:12:19.601ZINFOcrucible: send last flush ID to this DS: 2508 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
60212025-01-17T21:12:19.601ZINFOcrucible: Replied this last flush ID: 2508 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
60222025-01-17T21:12:19.601ZINFOcrucible: [1] client re-new 302 jobs since flush 2508 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
60232025-01-17T21:12:19.601ZINFOcrucible: [1] Replayed 302 jobs since flush: 2508 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
6024 ----------------------------------------------------------------
6025 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
6026 Downstairs last five completed: 2810 2809 2808 2807 2806
6027 Upstairs last five completed: 2810 2809 2808 2807 2806
6028 CLIENT: Up:0 ds:0 act:3
6029 Replay: All jobs finished
6030 Main task has work to do, go find it
6031 got dsc Stop(0)
6032 Jan 17 21:12:21.856stop 0
6033 INFO request completed, latency_us: 339, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 0ae7df67-2d79-4385-9ff2-06ac701b08c7, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
6034 [8810] Got stop action so:false kr:false
6035 Jan 17 21:12:21.856 INFO request completed, latency_us: 121, response_code: 200, uri: /state/cid/0, method: GET, req_id: 50a05186-1eed-4850-9ebc-253fc1733f38, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
60362025-01-17T21:12:21.857ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
60372025-01-17T21:12:21.857ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
60382025-01-17T21:12:21.857ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
60392025-01-17T21:12:21.857ZINFOcrucible: client task is exiting = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
6040 [8810] Exited with: ExitStatus(unix_wait_status(9))
6041 [8810][0] reports Exit
6042 Jan 17 21:12:25.859 INFO request completed, latency_us: 200, response_code: 200, uri: /state/cid/0, method: GET, req_id: 6cc41fa6-cf0f-41b9-b4dc-bcc19cd8b597, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
6043 001/300 Read block 1127 len 8192
6044 002/300 Write block 3832 len 20480 data: 3 2 2 2 2
6045 003/300 Read block 853 len 28672
6046 004/300 Write block 1375 len 36864 data: 3 4 4 4 4 3 3 3 4
6047 005/300 Write block 3444 len 20480 data: 4 6 6 6 5
6048 006/300 Write block 2800 len 8192 data: 2 2
6049 007/300 Read block 547 len 4096
6050 008/300 Write block 1790 len 36864 data: 4 4 4 4 3 4 3 3 3
6051 009/300 Read block 2997 len 40960
6052 010/300 Write block 4184 len 40960 data: 3 3 3 2 2 2 2 2 2 2
6053 011/300 Write block 4350 len 12288 data: 3 3 2
6054 012/300 Write block 4435 len 28672 data: 3 3 3 3 3 3 3
6055 013/300 Read block 663 len 12288
6056 014/300 Write block 951 len 24576 data: 3 2 2 2 4 4
6057 015/300 Flush
6058 016/300 Flush
6059 017/300 Read block 1724 len 20480
6060 018/300 Read block 2318 len 16384
6061 019/300 Flush
6062 020/300 Write block 1397 len 12288 data: 3 3 3
6063 021/300 Write block 1138 len 8192 data: 3 2
6064 022/300 Read block 4758 len 8192
6065 023/300 Write block 980 len 4096 data: 2
6066 024/300 Read block 4319 len 12288
6067 025/300 Write block 1756 len 12288 data: 2 2 2
6068 026/300 Write block 3150 len 28672 data: 2 2 2 2 2 2 2
6069 027/300 Write block 24 len 36864 data: 4 4 4 5 5 5 4 4 4
6070 028/300 Read block 652 len 4096
6071 029/300 Read block 1805 len 4096
6072 030/300 Write block 4752 len 24576 data: 2 2 2 2 3 2
6073 031/300 Read block 4549 len 36864
6074 032/300 Flush
6075 033/300 Read block 1475 len 28672
6076 034/300 Flush
6077 035/300 Read block 2465 len 8192
6078 036/300 Write block 2050 len 8192 data: 4 4
6079 037/300 Write block 3254 len 20480 data: 2 2 2 2 2
6080 038/300 Write block 4874 len 36864 data: 3 3 2 2 2 3 3 2 3
6081 039/300 Read block 4388 len 4096
6082 040/300 Write block 4025 len 8192 data: 2 2
6083 041/300 Write block 3294 len 16384 data: 3 2 3 3
6084 042/300 Read block 96 len 12288
6085 043/300 Write block 708 len 16384 data: 3 3 3 3
6086 044/300 Flush
6087 045/300 Write block 4446 len 28672 data: 2 2 2 2 3 2 2
6088 046/300 Flush
6089 047/300 Read block 18 len 24576
6090 048/300 Write block 3165 len 28672 data: 2 3 2 2 2 2 2
6091 049/300 Write block 836 len 20480 data: 3 3 3 3 3
6092 050/300 Write block 1706 len 20480 data: 3 3 4 6 5
6093 051/300 Read block 3058 len 20480
6094 052/300 Read block 4984 len 40960
6095 053/300 Read block 1230 len 12288
6096 054/300 Write block 1118 len 36864 data: 4 3 3 3 3 2 2 2 2
6097 055/300 Read block 4687 len 12288
6098 056/300 Read block 835 len 24576
6099 057/300 Read block 3168 len 36864
6100 058/300 Read block 1382 len 12288
6101 059/300 Read block 4232 len 12288
6102 060/300 Read block 4466 len 4096
6103 061/300 Read block 1844 len 20480
6104 062/300 Read block 2609 len 40960
6105 063/300 Read block 196 len 24576
6106 064/300 Write block 4138 len 36864 data: 4 4 4 3 3 3 3 2 3
6107 065/300 Write block 3446 len 24576 data: 7 7 6 4 4 3
6108 066/300 Flush
6109 067/300 Read block 3082 len 32768
6110 068/300 Read block 4695 len 32768
6111 069/300 Write block 510 len 36864 data: 2 2 2 2 2 2 2 3 2
6112 070/300 Read block 1207 len 36864
6113 071/300 Read block 4913 len 20480
6114 072/300 Write block 230 len 16384 data: 3 3 3 3
6115 073/300 Flush
6116 074/300 Write block 4121 len 8192 data: 3 3
6117 075/300 Read block 2733 len 8192
6118 076/300 Read block 1154 len 16384
6119 077/300 Write block 1579 len 32768 data: 3 3 3 4 5 6 4 3
6120 078/300 Read block 2857 len 20480
6121 079/300 Write block 4581 len 4096 data: 2
6122 080/300 Write block 2161 len 4096 data: 4
6123 081/300 Write block 4152 len 28672 data: 2 2 2 4 4 4 4
6124 082/300 Write block 3931 len 32768 data: 5 5 6 6 5 4 2 2
6125 083/300 Write block 4175 len 4096 data: 6
6126 084/300 Write block 4220 len 24576 data: 3 3 2 2 2 2
6127 085/300 Write block 980 len 40960 data: 3 2 2 2 2 2 2 2 2 4
6128 086/300 Read block 4963 len 20480
6129 087/300 Flush
6130 088/300 Write block 4545 len 40960 data: 4 4 3 2 2 2 2 2 2 2
6131 089/300 Read block 4679 len 40960
6132 090/300 Read block 536 len 32768
6133 091/300 Read block 1485 len 16384
6134 092/300 Write block 3792 len 12288 data: 2 2 2
6135 093/300 Read block 3522 len 40960
6136 094/300 Read block 2025 len 12288
6137 095/300 Write block 2418 len 24576 data: 3 3 3 3 2 2
6138 096/300 Read block 2092 len 16384
6139 097/300 Write block 3908 len 20480 data: 3 3 3 3 3
6140 098/300 Write block 946 len 4096 data: 3
6141 099/300 Read block 1090 len 16384
6142 100/300 Write block 2887 len 8192 data: 3 3
6143 101/300 Read block 4781 len 36864
6144 102/300 Read block 2892 len 12288
6145 103/300 Read block 2192 len 28672
6146 104/300 Read block 1141 len 16384
6147 105/300 Flush
6148 106/300 Write block 2346 len 24576 data: 3 3 3 4 4 4
6149 107/300 Write block 3481 len 20480 data: 2 2 2 2 2
6150 108/300 Write block 473 len 12288 data: 3 3 2
6151 109/300 Read block 185 len 16384
6152 110/300 Flush
6153 111/300 Read block 3129 len 8192
6154 112/300 Read block 1594 len 12288
6155 113/300 Read block 4361 len 28672
6156 114/300 Read block 2767 len 36864
6157 115/300 Read block 3219 len 12288
6158 116/300 Flush
6159 117/300 Read block 2474 len 12288
6160 118/300 Write block 4468 len 40960 data: 2 2 2 2 2 2 2 2 2 2
6161 119/300 Read block 3043 len 12288
6162 120/300 Read block 2300 len 40960
6163 121/300 Write block 2008 len 24576 data: 2 2 2 3 3 3
6164 122/300 Read block 2621 len 16384
6165 123/300 Read block 4881 len 40960
6166 124/300 Write block 4011 len 12288 data: 3 2 2
6167 125/300 Write block 3706 len 16384 data: 2 2 2 2
6168 126/300 Read block 3382 len 12288
6169 127/300 Write block 356 len 4096 data: 2
6170 128/300 Read block 1766 len 4096
6171 129/300 Read block 3620 len 4096
6172 130/300 Write block 559 len 20480 data: 2 2 2 2 2
6173 131/300 Read block 4248 len 12288
6174 132/300 Write block 2346 len 4096 data: 4
6175 133/300 Write block 2130 len 16384 data: 2 2 2 2
6176 134/300 Read block 2978 len 4096
6177 135/300 Read block 441 len 12288
6178 136/300 Read block 1716 len 40960
6179 137/300 Read block 1707 len 8192
6180 138/300 Flush
6181 139/300 Read block 4810 len 28672
6182 140/300 Write block 3585 len 24576 data: 2 3 2 2 2 2
6183 141/300 Write block 862 len 12288 data: 2 2 2
6184 142/300 Write block 4779 len 40960 data: 5 4 5 4 4 4 4 3 2 2
6185 143/300 Read block 532 len 40960
6186 144/300 Write block 4202 len 28672 data: 2 2 2 2 2 2 3
6187 145/300 Write block 2631 len 36864 data: 3 2 2 2 2 2 3 3 3
6188 146/300 Write block 684 len 20480 data: 3 3 3 3 3
6189 147/300 Write block 385 len 40960 data: 2 2 2 2 3 4 4 4 4 4
6190 148/300 Read block 3882 len 32768
6191 149/300 Read block 804 len 24576
6192 150/300 Read block 252 len 28672
6193 151/300 Write block 4145 len 36864 data: 3 4 2 2 2 2 2 3 3
6194 152/300 Read block 1511 len 4096
6195 153/300 Flush
6196 154/300 Read block 4119 len 28672
6197 155/300 Write block 4675 len 4096 data: 3
6198 156/300 Read block 2305 len 32768
6199 157/300 Read block 643 len 40960
6200 158/300 Read block 190 len 8192
6201 159/300 Write block 958 len 4096 data: 4
6202 160/300 Read block 446 len 12288
6203 161/300 Write block 4022 len 4096 data: 2
6204 162/300 Write block 2586 len 28672 data: 5 5 5 5 4 4 4
6205 163/300 Read block 2800 len 16384
6206 164/300 Flush
6207 165/300 Write block 2234 len 4096 data: 3
6208 166/300 Write block 3201 len 20480 data: 4 3 3 3 3
6209 167/300 Write block 4913 len 12288 data: 3 3 3
6210 168/300 Write block 4185 len 40960 data: 4 4 3 3 3 3 3 3 3 2
6211 169/300 Read block 2960 len 4096
6212 170/300 Read block 1628 len 36864
6213 171/300 Write block 4547 len 24576 data: 4 3 3 3 3 3
6214 172/300 Write block 3855 len 4096 data: 3
6215 173/300 Write block 347 len 16384 data: 3 4 2 2
6216 174/300 Read block 3942 len 16384
6217 175/300 Read block 2642 len 20480
6218 176/300 Write block 686 len 32768 data: 4 4 4 2 3 2 2 2
6219 177/300 Read block 3821 len 24576
6220 178/300 Read block 2846 len 32768
6221 179/300 Write block 4459 len 40960 data: 3 3 3 3 2 2 2 2 2 3
6222 180/300 Read block 829 len 12288
6223 181/300 Write block 2674 len 16384 data: 2 2 2 2
6224 182/300 Read block 192 len 32768
6225 183/300 Flush
6226 184/300 Read block 674 len 16384
6227 185/300 Read block 2790 len 32768
6228 186/300 Read block 832 len 4096
6229 187/300 Write block 643 len 8192 data: 3 3
6230 188/300 Write block 3825 len 4096 data: 4
6231 189/300 Read block 3927 len 32768
6232 190/300 Read block 680 len 40960
6233 191/300 Read block 2635 len 8192
6234 192/300 Write block 2148 len 40960 data: 3 3 3 3 3 3 3 2 2 2
6235 193/300 Read block 1044 len 4096
6236 194/300 Read block 103 len 4096
6237 195/300 Flush
6238 196/300 Flush
6239 197/300 Read block 308 len 28672
6240 198/300 Read block 395 len 16384
6241 199/300 Flush
6242 200/300 Read block 880 len 20480
6243 201/300 Read block 4560 len 12288
6244 202/300 Read block 3765 len 28672
6245 203/300 Write block 3234 len 20480 data: 2 3 3 4 4
6246 204/300 Write block 1148 len 4096 data: 3
6247 205/300 Read block 2315 len 20480
6248 206/300 Flush
6249 207/300 Write block 4728 len 24576 data: 2 2 3 3 3 5
6250 208/300 Read block 1467 len 20480
6251 209/300 Read block 1368 len 12288
6252 210/300 Read block 2523 len 32768
6253 211/300 Write block 260 len 8192 data: 2 2
6254 212/300 Read block 1698 len 28672
6255 213/300 Write block 4843 len 32768 data: 2 2 2 2 2 3 3 3
6256 214/300 Read block 3266 len 32768
6257 215/300 Read block 3693 len 4096
6258 216/300 Write block 2318 len 8192 data: 4 3
6259 217/300 Read block 1995 len 4096
6260 218/300 Read block 4382 len 20480
6261 219/300 Read block 3114 len 36864
6262 220/300 Read block 3063 len 12288
6263 221/300 Read block 2940 len 20480
6264 222/300 Write block 1738 len 40960 data: 2 2 2 3 4 4 4 4 3 3
6265 223/300 Read block 3510 len 28672
6266 224/300 Write block 2278 len 28672 data: 3 3 2 2 2 2 2
6267 225/300 Write block 3767 len 40960 data: 3 3 3 2 3 2 2 2 2 2
6268 226/300 Write block 918 len 28672 data: 3 4 4 3 4 4 3
6269 227/300 Read block 4112 len 40960
6270 228/300 Write block 1176 len 4096 data: 2
6271 229/300 Write block 142 len 32768 data: 2 2 2 3 3 3 3 4
6272 230/300 Read block 2900 len 36864
6273 231/300 Read block 73 len 20480
6274 232/300 Read block 3336 len 36864
6275 233/300 Read block 3619 len 40960
6276 234/300 Write block 12 len 12288 data: 2 2 2
6277 235/300 Write block 2487 len 20480 data: 4 4 3 4 4
6278 236/300 Write block 1668 len 24576 data: 2 2 2 2 2 2
6279 237/300 Write block 1628 len 4096 data: 2
6280 238/300 Write block 1904 len 4096 data: 4
6281 239/300 Read block 4049 len 12288
6282 240/300 Read block 3154 len 40960
6283 241/300 Write block 786 len 40960 data: 3 3 3 3 4 4 5 4 3 3
6284 242/300 Write block 4054 len 28672 data: 2 2 2 2 3 3 3
6285 243/300 Read block 4838 len 36864
6286 244/300 Read block 4823 len 20480
6287 245/300 Write block 756 len 4096 data: 2
6288 246/300 Write block 325 len 28672 data: 2 2 3 2 2 2 2
6289 247/300 Read block 1571 len 12288
6290 248/300 Read block 3742 len 20480
6291 249/300 Flush
6292 250/300 Flush
6293 251/300 Read block 1802 len 36864
6294 252/300 Read block 3861 len 24576
6295 253/300 Write block 1951 len 28672 data: 4 4 4 4 4 3 3
6296 254/300 Read block 4643 len 32768
6297 255/300 Write block 1765 len 28672 data: 2 2 2 2 2 2 2
6298 256/300 Write block 398 len 36864 data: 4 4 4 4 4 4 4 3 2
6299 257/300 Read block 795 len 24576
6300 258/300 Read block 2938 len 8192
6301 259/300 Read block 424 len 24576
6302 260/300 Write block 1964 len 24576 data: 3 3 3 3 3 4
6303 261/300 Write block 53 len 4096 data: 3
6304 262/300 Write block 407 len 24576 data: 2 2 2 2 2 2
6305 263/300 Read block 2138 len 4096
6306 264/300 Write block 3620 len 8192 data: 4 4
6307 265/300 Write block 1267 len 24576 data: 2 3 3 3 3 3
6308 266/300 Write block 611 len 20480 data: 2 2 2 2 2
6309 267/300 Flush
6310 268/300 Read block 1640 len 32768
6311 269/300 Write block 4110 len 16384 data: 2 2 2 2
6312 270/300 Read block 4110 len 16384
6313 271/300 Read block 318 len 32768
6314 272/300 Write block 1683 len 24576 data: 2 2 2 2 2 2
6315 273/300 Write block 4599 len 20480 data: 2 2 2 2 2
6316 274/300 Read block 1554 len 36864
6317 275/300 Read block 684 len 20480
6318 276/300 Write block 2926 len 16384 data: 2 2 2 2
6319 277/300 Read block 4166 len 20480
6320 278/300 Write block 3278 len 36864 data: 2 2 2 2 2 2 2 2 3
6321 279/300 Write block 4286 len 28672 data: 2 2 2 2 2 2 2
6322 280/300 Write block 3242 len 20480 data: 3 3 2 2 2
6323 281/300 Read block 3469 len 16384
6324 282/300 Write block 3035 len 8192 data: 2 2
6325 283/300 Read block 4146 len 16384
6326 284/300 Read block 360 len 4096
6327 285/300 Read block 1389 len 8192
6328 286/300 Write block 2627 len 16384 data: 3 3 3 2
6329 287/300 Read block 286 len 36864
6330 288/300 Write block 3221 len 32768 data: 3 3 4 4 4 4 3 3
6331 289/300 Read block 1078 len 12288
6332 290/300 Write block 4429 len 4096 data: 2
6333 291/300 Read block 4359 len 32768
6334 292/300 Read block 3210 len 28672
6335 293/300 Read block 1934 len 12288
6336 294/300 Write block 122 len 8192 data: 5 4
6337 295/300 Read block 2753 len 40960
6338 296/300 Read block 4772 len 8192
6339 297/300 Read block 930 len 16384
6340 298/300 Read block 2435 len 32768
6341 299/300 Read block 2953 len 4096
6342 300/300 Read block 3701 len 4096
6343 301/300 Write block 818 len 28672 data: 2 2 2 2 2 2 3
6344 Main task has work to do, go find it
6345 got dsc Start(0)
6346 start 0
6347 Jan 17 21:12:26.094 INFO request completed, latency_us: 221, response_code: 204, uri: /start/cid/0, method: POST, req_id: 940c8a5b-f405-4179-9a44-5ca8fb92992a, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
6348 Replay: started 0, returned:Ok(())
6349 ----------------------------------------------------------------
6350 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6351 States: Offline Active Active
6352 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6353 2811 Acked Read 2 Sent Done Done false
6354 2812 Acked Write 5 Sent Done Done false
6355 2813 Acked Read 7 Sent Done Done false
6356 2814 Acked Write 9 Sent Done Done false
6357 2815 Acked Write 5 Sent Done Done false
6358 2816 Acked Write 2 Sent Done Done false
6359 2817 Acked Read 1 Sent Done Done false
6360 2818 Acked Write 9 Sent Done Done false
6361 2819 Acked Read 10 Sent Done Done false
6362 2820 Acked Write 10 Sent Done Done false
6363 2821 Acked Write 3 Sent Done Done false
6364 2822 Acked Write 7 Sent Done Done false
6365 2823 Acked Read 3 Sent Done Done false
6366 2824 Acked Write 6 Sent Done Done false
6367 2825 Acked Flush 0 Sent Done Done false
6368 2826 Acked Flush 0 Sent Done Done false
6369 2827 Acked Read 5 Sent Done Done false
6370 2828 Acked Read 4 Sent Done Done false
6371 2829 Acked Flush 0 Sent Done Done false
6372 2830 Acked Write 3 Sent Done Done false
6373 2831 Acked Write 2 Sent Done Done false
6374 2832 Acked Read 2 Sent Done Done false
6375 2833 Acked Write 1 Sent Done Done false
6376 2834 Acked Read 3 Sent Done Done false
6377 2835 Acked Write 3 Sent Done Done false
6378 2836 Acked Write 7 Sent Done Done false
6379 2837 Acked Write 9 Sent Done Done false
6380 2838 Acked Read 1 Sent Done Done false
6381 2839 Acked Read 1 Sent Done Done false
6382 2840 Acked Write 6 Sent Done Done false
6383 2841 Acked Read 9 Sent Done Done false
6384 2842 Acked Flush 0 Sent Done Done false
6385 2843 Acked Read 7 Sent Done Done false
6386 2844 Acked Flush 0 Sent Done Done false
6387 2845 Acked Read 2 Sent Done Done false
6388 2846 Acked Write 2 Sent Done Done false
6389 2847 Acked Write 5 Sent Done Done false
6390 2848 Acked Write 9 Sent Done Done false
6391 2849 Acked Read 1 Sent Done Done false
6392 2850 Acked Write 2 Sent Done Done false
6393 2851 Acked Write 4 Sent Done Done false
6394 2852 Acked Read 3 Sent Done Done false
6395 2853 Acked Write 4 Sent Done Done false
6396 2854 Acked Flush 0 Sent Done Done false
6397 2855 Acked Write 7 Sent Done Done false
6398 2856 Acked Flush 0 Sent Done Done false
6399 2857 Acked Read 6 Sent Done Done false
6400 2858 Acked Write 7 Sent Done Done false
6401 2859 Acked Write 5 Sent Done Done false
6402 2860 Acked Write 5 Sent Done Done false
6403 2861 Acked Read 5 Sent Done Done false
6404 2862 Acked Read 10 Sent Done Done false
6405 2863 Acked Read 3 Sent Done Done false
6406 2864 Acked Write 9 Sent Done Done false
6407 2865 Acked Read 3 Sent Done Done false
6408 2866 Acked Read 6 Sent Done Done false
6409 2867 Acked Read 9 Sent Done Done false
6410 2868 Acked Read 3 Sent Done Done false
6411 2869 Acked Read 3 Sent Done Done false
6412 2870 Acked Read 1 Sent Done Done false
6413 2871 Acked Read 5 Sent Done Done false
6414 2872 Acked Read 10 Sent Done Done false
6415 2873 Acked Read 6 Sent Done Done false
6416 2874 Acked Write 9 Sent Done Done false
6417 2875 Acked Write 6 Sent Done Done false
6418 2876 Acked Flush 0 Sent Done Done false
6419 2877 Acked Read 8 Sent Done Done false
6420 2878 Acked Read 8 Sent Done Done false
6421 2879 Acked Write 9 Sent Done Done false
6422 2880 Acked Read 9 Sent Done Done false
6423 2881 Acked Read 5 Sent Done Done false
6424 2882 Acked Write 4 Sent Done Done false
6425 2883 Acked Flush 0 Sent Done Done false
6426 2884 Acked Write 2 Sent Done Done false
6427 2885 Acked Read 2 Sent Done Done false
6428 2886 Acked Read 4 Sent Done Done false
6429 2887 Acked Write 8 Sent Done Done false
6430 2888 Acked Read 5 Sent Done Done false
6431 2889 Acked Write 1 Sent Done Done false
6432 2890 Acked Write 1 Sent Done Done false
6433 2891 Acked Write 7 Sent Done Done false
6434 2892 Acked Write 8 Sent Done Done false
6435 2893 Acked Write 1 Sent Done Done false
6436 2894 Acked Write 6 Sent Done Done false
6437 2895 Acked Write 10 Sent Done Done false
6438 2896 Acked Read 5 Sent Done Done false
6439 2897 Acked Flush 0 Sent Done Done false
6440 2898 Acked Write 10 Sent Done Done false
6441 2899 Acked Read 10 Sent Done Done false
6442 2900 Acked Read 8 Sent Done Done false
6443 2901 Acked Read 4 Sent Done Done false
6444 2902 Acked Write 3 Sent Done Done false
6445 2903 Acked Read 10 Sent Done Done false
6446 2904 Acked Read 3 Sent Done Done false
6447 2905 Acked Write 6 Sent Done Done false
6448 2906 Acked Read 4 Sent Done Done false
6449 2907 Acked Write 5 Sent Done Done false
6450 2908 Acked Write 1 Sent Done Done false
6451 2909 Acked Read 4 Sent Done Done false
6452 2910 Acked Write 2 Sent Done Done false
6453 2911 Acked Read 9 Sent Done Done false
6454 2912 Acked Read 3 Sent Done Done false
6455 2913 Acked Read 7 Sent Done Done false
6456 2914 Acked Read 4 Sent Done Done false
6457 2915 Acked Flush 0 Sent Done Done false
6458 2916 Acked Write 6 Sent Done Done false
6459 2917 Acked Write 5 Sent Done Done false
6460 2918 Acked Write 3 Sent Done Done false
6461 2919 Acked Read 4 Sent Done Done false
6462 2920 Acked Flush 0 Sent Done Done false
6463 2921 Acked Read 2 Sent Done Done false
6464 2922 Acked Read 3 Sent Done Done false
6465 2923 Acked Read 7 Sent Done Done false
6466 2924 Acked Read 9 Sent Done Done false
6467 2925 Acked Read 3 Sent Done Done false
6468 2926 Acked Flush 0 Sent Done Done false
6469 2927 Acked Read 3 Sent Done Done false
6470 2928 Acked Write 10 Sent Done Done false
6471 2929 Acked Read 3 Sent Done Done false
6472 2930 Acked Read 10 Sent Done Done false
6473 2931 Acked Write 6 Sent Done Done false
6474 2932 Acked Read 4 Sent Done Done false
6475 2933 Acked Read 10 Sent Done Done false
6476 2934 Acked Write 3 Sent Done Done false
6477 2935 Acked Write 4 Sent Done Done false
6478 2936 Acked Read 3 Sent Done Done false
6479 2937 Acked Write 1 Sent Done Done false
6480 2938 Acked Read 1 Sent Done Done false
6481 2939 Acked Read 1 Sent Done Done false
6482 2940 Acked Write 5 Sent Done Done false
6483 2941 Acked Read 3 Sent Done Done false
6484 2942 Acked Write 1 Sent Done Done false
6485 2943 Acked Write 4 Sent Done Done false
6486 2944 Acked Read 1 Sent Done Done false
6487 2945 Acked Read 3 Sent Done Done false
6488 2946 Acked Read 10 Sent Done Done false
6489 2947 Acked Read 2 Sent Done Done false
6490 2948 Acked Flush 0 Sent Done Done false
6491 2949 Acked Read 7 Sent Done Done false
6492 2950 Acked Write 6 Sent Done Done false
6493 2951 Acked Write 3 Sent Done Done false
6494 2952 Acked Write 10 Sent Done Done false
6495 2953 Acked Read 10 Sent Done Done false
6496 2954 Acked Write 7 Sent Done Done false
6497 2955 Acked Write 9 Sent Done Done false
6498 2956 Acked Write 5 Sent Done Done false
6499 2957 Acked Write 10 Sent Done Done false
6500 2958 Acked Read 8 Sent Done Done false
6501 2959 Acked Read 6 Sent Done Done false
6502 2960 Acked Read 7 Sent Done Done false
6503 2961 Acked Write 9 Sent Done Done false
6504 2962 Acked Read 1 Sent Done Done false
6505 2963 Acked Flush 0 Sent Done Done false
6506 2964 Acked Read 7 Sent Done Done false
6507 2965 Acked Write 1 Sent Done Done false
6508 2966 Acked Read 8 Sent Done Done false
6509 2967 Acked Read 10 Sent Done Done false
6510 2968 Acked Read 2 Sent Done Done false
6511 2969 Acked Write 1 Sent Done Done false
6512 2970 Acked Read 3 Sent Done Done false
6513 2971 Acked Write 1 Sent Done Done false
6514 2972 Acked Write 7 Sent Done Done false
6515 2973 Acked Read 4 Sent Done Done false
6516 2974 Acked Flush 0 Sent Done Done false
6517 2975 Acked Write 1 Sent Done Done false
6518 2976 Acked Write 5 Sent Done Done false
6519 2977 Acked Write 3 Sent Done Done false
6520 2978 Acked Write 10 Sent Done Done false
6521 2979 Acked Read 1 Sent Done Done false
6522 2980 Acked Read 9 Sent Done Done false
6523 2981 Acked Write 6 Sent Done Done false
6524 2982 Acked Write 1 Sent Done Done false
6525 2983 Acked Write 4 Sent Done Done false
6526 2984 Acked Read 4 Sent Done Done false
6527 2985 Acked Read 5 Sent Done Done false
6528 2986 Acked Write 8 Sent Done Done false
6529 2987 Acked Read 6 Sent Done Done false
6530 2988 Acked Read 8 Sent Done Done false
6531 2989 Acked Write 10 Sent Done Done false
6532 2990 Acked Read 3 Sent Done Done false
6533 2991 Acked Write 4 Sent Done Done false
6534 2992 Acked Read 8 Sent Done Done false
6535 2993 Acked Flush 0 Sent Done Done false
6536 2994 Acked Read 4 Sent Done Done false
6537 2995 Acked Read 8 Sent Done Done false
6538 2996 Acked Read 1 Sent Done Done false
6539 2997 Acked Write 2 Sent Done Done false
6540 2998 Acked Write 1 Sent Done Done false
6541 2999 Acked Read 8 Sent Done Done false
6542 3000 Acked Read 10 Sent Done Done false
6543 3001 Acked Read 2 Sent Done Done false
6544 3002 Acked Write 10 Sent Done Done false
6545 3003 Acked Read 1 Sent Done Done false
6546 3004 Acked Read 1 Sent Done Done false
6547 3005 Acked Flush 0 Sent Done Done false
6548 3006 Acked Flush 0 Sent Done Done false
6549 3007 Acked Read 7 Sent Done Done false
6550 3008 Acked Read 4 Sent Done Done false
6551 3009 Acked Flush 0 Sent Done Done false
6552 3010 Acked Read 5 Sent Done Done false
6553 3011 Acked Read 3 Sent Done Done false
6554 3012 Acked Read 7 Sent Done Done false
6555 3013 Acked Write 5 Sent Done Done false
6556 3014 Acked Write 1 Sent Done Done false
6557 3015 Acked Read 5 Sent Done Done false
6558 3016 Acked Flush 0 Sent Done Done false
6559 3017 Acked Write 6 Sent Done Done false
6560 3018 Acked Read 5 Sent Done Done false
6561 3019 Acked Read 3 Sent Done Done false
6562 3020 Acked Read 8 Sent Done Done false
6563 3021 Acked Write 2 Sent Done Done false
6564 3022 Acked Read 7 Sent Done Done false
6565 3023 Acked Write 8 Sent Done Done false
6566 3024 Acked Read 8 Sent Done Done false
6567 3025 Acked Read 1 Sent Done Done false
6568 3026 Acked Write 2 Sent Done Done false
6569 3027 Acked Read 1 Sent Done Done false
6570 3028 Acked Read 5 Sent Done Done false
6571 3029 Acked Read 9 Sent Done Done false
6572 3030 Acked Read 3 Sent Done Done false
6573 3031 Acked Read 5 Sent Done Done false
6574 3032 Acked Write 10 Sent Done Done false
6575 3033 Acked Read 7 Sent Done Done false
6576 3034 Acked Write 7 Sent Done Done false
6577 3035 Acked Write 10 Sent Done Done false
6578 3036 Acked Write 7 Sent Done Done false
6579 3037 Acked Read 10 Sent Done Done false
6580 3038 Acked Write 1 Sent Done Done false
6581 3039 Acked Write 8 Sent Done Done false
6582 3040 Acked Read 9 Sent Done Done false
6583 3041 Acked Read 5 Sent Done Done false
6584 3042 Acked Read 9 Sent Done Done false
6585 3043 Acked Read 10 Sent Done Done false
6586 3044 Acked Write 3 Sent Done Done false
6587 3045 Acked Write 5 Sent Done Done false
6588 3046 Acked Write 6 Sent Done Done false
6589 3047 Acked Write 1 Sent Done Done false
6590 3048 Acked Write 1 Sent Done Done false
6591 3049 Acked Read 3 Sent Done Done false
6592 3050 Acked Read 10 Sent Done Done false
6593 3051 Acked Write 10 Sent Done Done false
6594 3052 Acked Write 7 Sent Done Done false
6595 3053 Acked Read 9 Sent Done Done false
6596 3054 Acked Read 5 Sent Done Done false
6597 3055 Acked Write 1 Sent Done Done false
6598 3056 Acked Write 7 Sent Done Done false
6599 3057 Acked Read 3 Sent Done Done false
6600 3058 Acked Read 5 Sent Done Done false
6601 3059 Acked Flush 0 Sent Done Done false
6602 3060 Acked Flush 0 Sent Done Done false
6603 3061 Acked Read 9 Sent Done Done false
6604 3062 Acked Read 6 Sent Done Done false
6605 3063 Acked Write 7 Sent Done Done false
6606 3064 Acked Read 8 Sent Done Done false
6607 3065 Acked Write 7 Sent Done Done false
6608 3066 Acked Write 9 Sent Done Done false
6609 3067 Acked Read 6 Sent Done Done false
6610 3068 Acked Read 2 Sent Done Done false
6611 3069 Acked Read 6 Sent Done Done false
6612 3070 Acked Write 6 Sent Done Done false
6613 3071 Acked Write 1 Sent Done Done false
6614 3072 Acked Write 6 Sent Done Done false
6615 3073 Acked Read 1 Sent Done Done false
6616 3074 Acked Write 2 Sent Done Done false
6617 3075 Acked Write 6 Sent Done Done false
6618 3076 Acked Write 5 Sent Done Done false
6619 3077 Acked Flush 0 Sent Done Done false
6620 3078 Acked Read 8 Sent Done Done false
6621 3079 Acked Write 4 Sent Done Done false
6622 3080 Acked Read 4 Sent Done Done false
6623 3081 Acked Read 8 Sent Done Done false
6624 3082 Acked Write 6 Sent Done Done false
6625 3083 Acked Write 5 Sent Done Done false
6626 3084 Acked Read 9 Sent Done Done false
6627 3085 Acked Read 5 Sent Done Done false
6628 3086 Acked Write 4 Sent Done Done false
6629 3087 Acked Read 5 Sent Done Done false
6630 3088 Acked Write 9 Sent Done Done false
6631 3089 Acked Write 7 Sent Done Done false
6632 3090 Acked Write 5 Sent Done Done false
6633 3091 Acked Read 4 Sent Done Done false
6634 3092 Acked Write 2 Sent Done Done false
6635 3093 Acked Read 4 Sent Done Done false
6636 3094 Acked Read 1 Sent Done Done false
6637 3095 Acked Read 2 Sent Done Done false
6638 3096 Acked Write 4 Sent Done Done false
6639 3097 Acked Read 9 Sent Done Done false
6640 3098 Acked Write 8 Sent Done Done false
6641 3099 Acked Read 3 Sent Done Done false
6642 3100 Acked Write 1 Sent Done Done false
6643 3101 Acked Read 8 Sent Done Done false
6644 3102 Acked Read 7 Sent Done Done false
6645 3103 Acked Read 3 Sent Done Done false
6646 3104 Acked Write 2 Sent Done Done false
6647 3105 Acked Read 10 Sent Done Done false
6648 3106 Acked Read 2 Sent Done Done false
6649 3107 Acked Read 4 Sent Done Done false
6650 3108 Acked Read 8 Sent Done Done false
6651 3109 Acked Read 1 Sent Done Done false
6652 3110 Acked Read 1 Sent Done Done false
6653 3111 Acked Write 7 Sent Done Done false
6654 STATES DS:0 DS:1 DS:2 TOTAL
6655 Sent 301 0 0 301
6656 Done 0 301 301 602
6657 Skipped 0 0 0 0
6658 Error 0 0 0 0
6659 Last Flush: 2810 3077 3077
6660 Downstairs last five completed: 2810 2809 2808 2807 2806
6661 Upstairs last five completed: 3111 3110 3109 3108 3107
6662 CLIENT: Up:0 ds:301 act:2
6663 [8810] got start action
6664 [8810] I am going to restart
6665 Starting downstairs at port 8810
6666 Make output file at "/tmp/dsc/downstairs-8810.txt"
6667 [8810][0] reports Starting
6668 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1207)
6669 [8810][0] reports Running
6670 ----------------------------------------------------------------
6671 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6672 States: Offline Active Active
6673 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6674 2811 Acked Read 2 Sent Done Done false
6675 2812 Acked Write 5 Sent Done Done false
6676 2813 Acked Read 7 Sent Done Done false
6677 2814 Acked Write 9 Sent Done Done false
6678 2815 Acked Write 5 Sent Done Done false
6679 2816 Acked Write 2 Sent Done Done false
6680 2817 Acked Read 1 Sent Done Done false
6681 2818 Acked Write 9 Sent Done Done false
6682 2819 Acked Read 10 Sent Done Done false
6683 2820 Acked Write 10 Sent Done Done false
6684 2821 Acked Write 3 Sent Done Done false
6685 2822 Acked Write 7 Sent Done Done false
6686 2823 Acked Read 3 Sent Done Done false
6687 2824 Acked Write 6 Sent Done Done false
6688 2825 Acked Flush 0 Sent Done Done false
6689 2826 Acked Flush 0 Sent Done Done false
6690 2827 Acked Read 5 Sent Done Done false
6691 2828 Acked Read 4 Sent Done Done false
6692 2829 Acked Flush 0 Sent Done Done false
6693 2830 Acked Write 3 Sent Done Done false
6694 2831 Acked Write 2 Sent Done Done false
6695 2832 Acked Read 2 Sent Done Done false
6696 2833 Acked Write 1 Sent Done Done false
6697 2834 Acked Read 3 Sent Done Done false
6698 2835 Acked Write 3 Sent Done Done false
6699 2836 Acked Write 7 Sent Done Done false
6700 2837 Acked Write 9 Sent Done Done false
6701 2838 Acked Read 1 Sent Done Done false
6702 2839 Acked Read 1 Sent Done Done false
6703 2840 Acked Write 6 Sent Done Done false
6704 2841 Acked Read 9 Sent Done Done false
6705 2842 Acked Flush 0 Sent Done Done false
6706 2843 Acked Read 7 Sent Done Done false
6707 2844 Acked Flush 0 Sent Done Done false
6708 2845 Acked Read 2 Sent Done Done false
6709 2846 Acked Write 2 Sent Done Done false
6710 2847 Acked Write 5 Sent Done Done false
6711 2848 Acked Write 9 Sent Done Done false
6712 2849 Acked Read 1 Sent Done Done false
6713 2850 Acked Write 2 Sent Done Done false
6714 2851 Acked Write 4 Sent Done Done false
6715 2852 Acked Read 3 Sent Done Done false
6716 2853 Acked Write 4 Sent Done Done false
6717 2854 Acked Flush 0 Sent Done Done false
6718 2855 Acked Write 7 Sent Done Done false
6719 2856 Acked Flush 0 Sent Done Done false
6720 2857 Acked Read 6 Sent Done Done false
6721 2858 Acked Write 7 Sent Done Done false
6722 2859 Acked Write 5 Sent Done Done false
6723 2860 Acked Write 5 Sent Done Done false
6724 2861 Acked Read 5 Sent Done Done false
6725 2862 Acked Read 10 Sent Done Done false
6726 2863 Acked Read 3 Sent Done Done false
6727 2864 Acked Write 9 Sent Done Done false
6728 2865 Acked Read 3 Sent Done Done false
6729 2866 Acked Read 6 Sent Done Done false
6730 2867 Acked Read 9 Sent Done Done false
6731 2868 Acked Read 3 Sent Done Done false
6732 2869 Acked Read 3 Sent Done Done false
6733 2870 Acked Read 1 Sent Done Done false
6734 2871 Acked Read 5 Sent Done Done false
6735 2872 Acked Read 10 Sent Done Done false
6736 2873 Acked Read 6 Sent Done Done false
6737 2874 Acked Write 9 Sent Done Done false
6738 2875 Acked Write 6 Sent Done Done false
6739 2876 Acked Flush 0 Sent Done Done false
6740 2877 Acked Read 8 Sent Done Done false
6741 2878 Acked Read 8 Sent Done Done false
6742 2879 Acked Write 9 Sent Done Done false
6743 2880 Acked Read 9 Sent Done Done false
6744 2881 Acked Read 5 Sent Done Done false
6745 2882 Acked Write 4 Sent Done Done false
6746 2883 Acked Flush 0 Sent Done Done false
6747 2884 Acked Write 2 Sent Done Done false
6748 2885 Acked Read 2 Sent Done Done false
6749 2886 Acked Read 4 Sent Done Done false
6750 2887 Acked Write 8 Sent Done Done false
6751 2888 Acked Read 5 Sent Done Done false
6752 2889 Acked Write 1 Sent Done Done false
6753 2890 Acked Write 1 Sent Done Done false
6754 2891 Acked Write 7 Sent Done Done false
6755 2892 Acked Write 8 Sent Done Done false
6756 2893 Acked Write 1 Sent Done Done false
6757 2894 Acked Write 6 Sent Done Done false
6758 2895 Acked Write 10 Sent Done Done false
6759 2896 Acked Read 5 Sent Done Done false
6760 2897 Acked Flush 0 Sent Done Done false
6761 2898 Acked Write 10 Sent Done Done false
6762 2899 Acked Read 10 Sent Done Done false
6763 2900 Acked Read 8 Sent Done Done false
6764 2901 Acked Read 4 Sent Done Done false
6765 2902 Acked Write 3 Sent Done Done false
6766 2903 Acked Read 10 Sent Done Done false
6767 2904 Acked Read 3 Sent Done Done false
6768 2905 Acked Write 6 Sent Done Done false
6769 2906 Acked Read 4 Sent Done Done false
6770 2907 Acked Write 5 Sent Done Done false
6771 2908 Acked Write 1 Sent Done Done false
6772 2909 Acked Read 4 Sent Done Done false
6773 2910 Acked Write 2 Sent Done Done false
6774 2911 Acked Read 9 Sent Done Done false
6775 2912 Acked Read 3 Sent Done Done false
6776 2913 Acked Read 7 Sent Done Done false
6777 2914 Acked Read 4 Sent Done Done false
6778 2915 Acked Flush 0 Sent Done Done false
6779 2916 Acked Write 6 Sent Done Done false
6780 2917 Acked Write 5 Sent Done Done false
6781 2918 Acked Write 3 Sent Done Done false
6782 2919 Acked Read 4 Sent Done Done false
6783 2920 Acked Flush 0 Sent Done Done false
6784 2921 Acked Read 2 Sent Done Done false
6785 2922 Acked Read 3 Sent Done Done false
6786 2923 Acked Read 7 Sent Done Done false
6787 2924 Acked Read 9 Sent Done Done false
6788 2925 Acked Read 3 Sent Done Done false
6789 2926 Acked Flush 0 Sent Done Done false
6790 2927 Acked Read 3 Sent Done Done false
6791 2928 Acked Write 10 Sent Done Done false
6792 2929 Acked Read 3 Sent Done Done false
6793 2930 Acked Read 10 Sent Done Done false
6794 2931 Acked Write 6 Sent Done Done false
6795 2932 Acked Read 4 Sent Done Done false
6796 2933 Acked Read 10 Sent Done Done false
6797 2934 Acked Write 3 Sent Done Done false
6798 2935 Acked Write 4 Sent Done Done false
6799 2936 Acked Read 3 Sent Done Done false
6800 2937 Acked Write 1 Sent Done Done false
6801 2938 Acked Read 1 Sent Done Done false
6802 2939 Acked Read 1 Sent Done Done false
6803 2940 Acked Write 5 Sent Done Done false
6804 2941 Acked Read 3 Sent Done Done false
6805 2942 Acked Write 1 Sent Done Done false
6806 2943 Acked Write 4 Sent Done Done false
6807 2944 Acked Read 1 Sent Done Done false
6808 2945 Acked Read 3 Sent Done Done false
6809 2946 Acked Read 10 Sent Done Done false
6810 2947 Acked Read 2 Sent Done Done false
6811 2948 Acked Flush 0 Sent Done Done false
6812 2949 Acked Read 7 Sent Done Done false
6813 2950 Acked Write 6 Sent Done Done false
6814 2951 Acked Write 3 Sent Done Done false
6815 2952 Acked Write 10 Sent Done Done false
6816 2953 Acked Read 10 Sent Done Done false
6817 2954 Acked Write 7 Sent Done Done false
6818 2955 Acked Write 9 Sent Done Done false
6819 2956 Acked Write 5 Sent Done Done false
6820 2957 Acked Write 10 Sent Done Done false
6821 2958 Acked Read 8 Sent Done Done false
6822 2959 Acked Read 6 Sent Done Done false
6823 2960 Acked Read 7 Sent Done Done false
6824 2961 Acked Write 9 Sent Done Done false
6825 2962 Acked Read 1 Sent Done Done false
6826 2963 Acked Flush 0 Sent Done Done false
6827 2964 Acked Read 7 Sent Done Done false
6828 2965 Acked Write 1 Sent Done Done false
6829 2966 Acked Read 8 Sent Done Done false
6830 2967 Acked Read 10 Sent Done Done false
6831 2968 Acked Read 2 Sent Done Done false
6832 2969 Acked Write 1 Sent Done Done false
6833 2970 Acked Read 3 Sent Done Done false
6834 2971 Acked Write 1 Sent Done Done false
6835 2972 Acked Write 7 Sent Done Done false
6836 2973 Acked Read 4 Sent Done Done false
6837 2974 Acked Flush 0 Sent Done Done false
6838 2975 Acked Write 1 Sent Done Done false
6839 2976 Acked Write 5 Sent Done Done false
6840 2977 Acked Write 3 Sent Done Done false
6841 2978 Acked Write 10 Sent Done Done false
6842 2979 Acked Read 1 Sent Done Done false
6843 2980 Acked Read 9 Sent Done Done false
6844 2981 Acked Write 6 Sent Done Done false
6845 2982 Acked Write 1 Sent Done Done false
6846 2983 Acked Write 4 Sent Done Done false
6847 2984 Acked Read 4 Sent Done Done false
6848 2985 Acked Read 5 Sent Done Done false
6849 2986 Acked Write 8 Sent Done Done false
6850 2987 Acked Read 6 Sent Done Done false
6851 2988 Acked Read 8 Sent Done Done false
6852 2989 Acked Write 10 Sent Done Done false
6853 2990 Acked Read 3 Sent Done Done false
6854 2991 Acked Write 4 Sent Done Done false
6855 2992 Acked Read 8 Sent Done Done false
6856 2993 Acked Flush 0 Sent Done Done false
6857 2994 Acked Read 4 Sent Done Done false
6858 2995 Acked Read 8 Sent Done Done false
6859 2996 Acked Read 1 Sent Done Done false
6860 2997 Acked Write 2 Sent Done Done false
6861 2998 Acked Write 1 Sent Done Done false
6862 2999 Acked Read 8 Sent Done Done false
6863 3000 Acked Read 10 Sent Done Done false
6864 3001 Acked Read 2 Sent Done Done false
6865 3002 Acked Write 10 Sent Done Done false
6866 3003 Acked Read 1 Sent Done Done false
6867 3004 Acked Read 1 Sent Done Done false
6868 3005 Acked Flush 0 Sent Done Done false
6869 3006 Acked Flush 0 Sent Done Done false
6870 3007 Acked Read 7 Sent Done Done false
6871 3008 Acked Read 4 Sent Done Done false
6872 3009 Acked Flush 0 Sent Done Done false
6873 3010 Acked Read 5 Sent Done Done false
6874 3011 Acked Read 3 Sent Done Done false
6875 3012 Acked Read 7 Sent Done Done false
6876 3013 Acked Write 5 Sent Done Done false
6877 3014 Acked Write 1 Sent Done Done false
6878 3015 Acked Read 5 Sent Done Done false
6879 3016 Acked Flush 0 Sent Done Done false
6880 3017 Acked Write 6 Sent Done Done false
6881 3018 Acked Read 5 Sent Done Done false
6882 3019 Acked Read 3 Sent Done Done false
6883 3020 Acked Read 8 Sent Done Done false
6884 3021 Acked Write 2 Sent Done Done false
6885 3022 Acked Read 7 Sent Done Done false
6886 3023 Acked Write 8 Sent Done Done false
6887 3024 Acked Read 8 Sent Done Done false
6888 3025 Acked Read 1 Sent Done Done false
6889 3026 Acked Write 2 Sent Done Done false
6890 3027 Acked Read 1 Sent Done Done false
6891 3028 Acked Read 5 Sent Done Done false
6892 3029 Acked Read 9 Sent Done Done false
6893 3030 Acked Read 3 Sent Done Done false
6894 3031 Acked Read 5 Sent Done Done false
6895 3032 Acked Write 10 Sent Done Done false
6896 3033 Acked Read 7 Sent Done Done false
6897 3034 Acked Write 7 Sent Done Done false
6898 3035 Acked Write 10 Sent Done Done false
6899 3036 Acked Write 7 Sent Done Done false
6900 3037 Acked Read 10 Sent Done Done false
6901 3038 Acked Write 1 Sent Done Done false
6902 3039 Acked Write 8 Sent Done Done false
6903 3040 Acked Read 9 Sent Done Done false
6904 3041 Acked Read 5 Sent Done Done false
6905 3042 Acked Read 9 Sent Done Done false
6906 3043 Acked Read 10 Sent Done Done false
6907 3044 Acked Write 3 Sent Done Done false
6908 3045 Acked Write 5 Sent Done Done false
6909 3046 Acked Write 6 Sent Done Done false
6910 3047 Acked Write 1 Sent Done Done false
6911 3048 Acked Write 1 Sent Done Done false
6912 3049 Acked Read 3 Sent Done Done false
6913 3050 Acked Read 10 Sent Done Done false
6914 3051 Acked Write 10 Sent Done Done false
6915 3052 Acked Write 7 Sent Done Done false
6916 3053 Acked Read 9 Sent Done Done false
6917 3054 Acked Read 5 Sent Done Done false
6918 3055 Acked Write 1 Sent Done Done false
6919 3056 Acked Write 7 Sent Done Done false
6920 3057 Acked Read 3 Sent Done Done false
6921 3058 Acked Read 5 Sent Done Done false
6922 3059 Acked Flush 0 Sent Done Done false
6923 3060 Acked Flush 0 Sent Done Done false
6924 3061 Acked Read 9 Sent Done Done false
6925 3062 Acked Read 6 Sent Done Done false
6926 3063 Acked Write 7 Sent Done Done false
6927 3064 Acked Read 8 Sent Done Done false
6928 3065 Acked Write 7 Sent Done Done false
6929 3066 Acked Write 9 Sent Done Done false
6930 3067 Acked Read 6 Sent Done Done false
6931 3068 Acked Read 2 Sent Done Done false
6932 3069 Acked Read 6 Sent Done Done false
6933 3070 Acked Write 6 Sent Done Done false
6934 3071 Acked Write 1 Sent Done Done false
6935 3072 Acked Write 6 Sent Done Done false
6936 3073 Acked Read 1 Sent Done Done false
6937 3074 Acked Write 2 Sent Done Done false
6938 3075 Acked Write 6 Sent Done Done false
6939 3076 Acked Write 5 Sent Done Done false
6940 3077 Acked Flush 0 Sent Done Done false
6941 3078 Acked Read 8 Sent Done Done false
6942 3079 Acked Write 4 Sent Done Done false
6943 3080 Acked Read 4 Sent Done Done false
6944 3081 Acked Read 8 Sent Done Done false
6945 3082 Acked Write 6 Sent Done Done false
6946 3083 Acked Write 5 Sent Done Done false
6947 3084 Acked Read 9 Sent Done Done false
6948 3085 Acked Read 5 Sent Done Done false
6949 3086 Acked Write 4 Sent Done Done false
6950 3087 Acked Read 5 Sent Done Done false
6951 3088 Acked Write 9 Sent Done Done false
6952 3089 Acked Write 7 Sent Done Done false
6953 3090 Acked Write 5 Sent Done Done false
6954 3091 Acked Read 4 Sent Done Done false
6955 3092 Acked Write 2 Sent Done Done false
6956 3093 Acked Read 4 Sent Done Done false
6957 3094 Acked Read 1 Sent Done Done false
6958 3095 Acked Read 2 Sent Done Done false
6959 3096 Acked Write 4 Sent Done Done false
6960 3097 Acked Read 9 Sent Done Done false
6961 3098 Acked Write 8 Sent Done Done false
6962 3099 Acked Read 3 Sent Done Done false
6963 3100 Acked Write 1 Sent Done Done false
6964 3101 Acked Read 8 Sent Done Done false
6965 3102 Acked Read 7 Sent Done Done false
6966 3103 Acked Read 3 Sent Done Done false
6967 3104 Acked Write 2 Sent Done Done false
6968 3105 Acked Read 10 Sent Done Done false
6969 3106 Acked Read 2 Sent Done Done false
6970 3107 Acked Read 4 Sent Done Done false
6971 3108 Acked Read 8 Sent Done Done false
6972 3109 Acked Read 1 Sent Done Done false
6973 3110 Acked Read 1 Sent Done Done false
6974 3111 Acked Write 7 Sent Done Done false
6975 3112 Acked Flush 0 Sent Done Done false
6976 STATES DS:0 DS:1 DS:2 TOTAL
6977 Sent 302 0 0 302
6978 Done 0 302 302 604
6979 Skipped 0 0 0 0
6980 Error 0 0 0 0
6981 Last Flush: 2810 3112 3112
6982 Downstairs last five completed: 2810 2809 2808 2807 2806
6983 Upstairs last five completed: 3112 3111 3110 3109 3108
6984 CLIENT: Up:0 ds:302 act:2
69852025-01-17T21:12:31.857ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
69862025-01-17T21:12:31.857ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:33685) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
69872025-01-17T21:12:31.858ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
69882025-01-17T21:12:31.859ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
69892025-01-17T21:12:31.859ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
69902025-01-17T21:12:31.859ZINFOcrucible: send last flush ID to this DS: 2810 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
69912025-01-17T21:12:31.859ZINFOcrucible: Replied this last flush ID: 2810 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
69922025-01-17T21:12:31.859ZINFOcrucible: [0] client re-new 302 jobs since flush 2810 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
69932025-01-17T21:12:31.860ZINFOcrucible: [0] Replayed 302 jobs since flush: 2810 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
6994 ----------------------------------------------------------------
6995 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
6996 Downstairs last five completed: 3112 3111 3110 3109 3108
6997 Upstairs last five completed: 3112 3111 3110 3109 3108
6998 CLIENT: Up:0 ds:0 act:3
6999 Replay: All jobs finished
7000 Main task has work to do, go find it
7001 got dsc Stop(0)
7002 stop 0
7003 Jan 17 21:12:34.099 [8810] Got stop action so:false kr:false
7004 INFO request completed, latency_us: 324, response_code: 204, uri: /stop/cid/0, method: POST, req_id: b0138350-9f9b-4e21-b888-7b9184b906d7, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
70052025-01-17T21:12:34.100ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
7006 Jan 17 21:12:34.100 INFO request completed, latency_us: 131, response_code: {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:12:34.100240848Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
7007 200, uri: /state/cid/0, method: GET, {"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:12:34.100281552Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
7008 req_id: e395c776-7587-4132-b40e-fa181e99c554, remote_addr:{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-17T21:12:34.100311258Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
7009 127.0.0.1:57191, local_addr: 127.0.0.1:9998
7010 [8810] Exited with: ExitStatus(unix_wait_status(9))
7011 [8810][0] reports Exit
7012 Jan 17 21:12:38.102 INFO request completed, latency_us: 196, response_code: 200, uri: /state/cid/0, method: GET, req_id: 13d1166c-8c6f-459b-9ce1-e3ae9605ed2c, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
7013 001/300 Read block 2906 len 32768
7014 002/300 Read block 4568 len 12288
7015 003/300 Flush
7016 004/300 Read block 608 len 32768
7017 005/300 Read block 4139 len 16384
7018 006/300 Read block 2838 len 20480
7019 007/300 Flush
7020 008/300 Write block 585 len 32768 data: 3 4 3 2 2 2 2 2
7021 009/300 Write block 3515 len 40960 data: 2 2 2 2 2 2 2 2 4 3
7022 010/300 Write block 1505 len 20480 data: 4 4 3 3 2
7023 011/300 Read block 1993 len 4096
7024 012/300 Read block 1144 len 32768
7025 013/300 Flush
7026 014/300 Read block 4783 len 36864
7027 015/300 Read block 3094 len 32768
7028 016/300 Read block 2254 len 16384
7029 017/300 Write block 2373 len 36864 data: 4 3 3 4 4 4 4 3 3
7030 018/300 Read block 3013 len 4096
7031 019/300 Write block 1367 len 12288 data: 2 2 2
7032 020/300 Flush
7033 021/300 Read block 1682 len 16384
7034 022/300 Read block 1677 len 4096
7035 023/300 Read block 1524 len 12288
7036 024/300 Flush
7037 025/300 Read block 3605 len 4096
7038 026/300 Read block 578 len 28672
7039 027/300 Write block 3393 len 32768 data: 3 3 3 2 2 2 2 2
7040 028/300 Write block 3172 len 40960 data: 2 3 3 3 3 3 3 3 3 2
7041 029/300 Flush
7042 030/300 Flush
7043 031/300 Read block 1683 len 12288
7044 032/300 Write block 4714 len 4096 data: 3
7045 033/300 Read block 2979 len 4096
7046 034/300 Write block 2406 len 28672 data: 4 4 4 3 3 3 2
7047 035/300 Write block 2826 len 20480 data: 3 3 4 4 4
7048 036/300 Write block 3319 len 28672 data: 4 3 2 2 2 2 2
7049 037/300 Read block 2497 len 24576
7050 038/300 Read block 2236 len 12288
7051 039/300 Read block 339 len 32768
7052 040/300 Read block 4953 len 20480
7053 041/300 Read block 2448 len 16384
7054 042/300 Read block 1990 len 20480
7055 043/300 Read block 2974 len 24576
7056 044/300 Read block 3050 len 20480
7057 045/300 Write block 647 len 32768 data: 2 2 2 2 2 2 2 2
7058 046/300 Read block 3999 len 4096
7059 047/300 Write block 298 len 20480 data: 3 3 2 2 2
7060 048/300 Flush
7061 049/300 Read block 4349 len 36864
7062 050/300 Flush
7063 051/300 Write block 4843 len 16384 data: 3 3 3 3
7064 052/300 Write block 4345 len 4096 data: 2
7065 053/300 Write block 1877 len 28672 data: 2 2 2 3 3 4 4
7066 054/300 Flush
7067 055/300 Write block 2537 len 28672 data: 3 3 3 3 3 3 3
7068 056/300 Write block 3495 len 8192 data: 2 4
7069 057/300 Write block 4671 len 36864 data: 2 3 3 3 4 4 5 5 5
7070 058/300 Flush
7071 059/300 Flush
7072 060/300 Write block 2305 len 28672 data: 3 3 3 3 3 4 4
7073 061/300 Write block 2072 len 32768 data: 3 2 2 2 2 2 2 2
7074 062/300 Read block 1198 len 36864
7075 063/300 Write block 2886 len 28672 data: 3 4 4 3 3 3 3
7076 064/300 Read block 2441 len 16384
7077 065/300 Read block 3100 len 40960
7078 066/300 Write block 1581 len 36864 data: 4 5 6 7 5 4 3 3 3
7079 067/300 Read block 4409 len 4096
7080 068/300 Read block 3231 len 20480
7081 069/300 Read block 4977 len 20480
7082 070/300 Read block 512 len 16384
7083 071/300 Write block 381 len 8192 data: 2 2
7084 072/300 Flush
7085 073/300 Write block 386 len 32768 data: 3 3 3 4 5 5 5 5
7086 074/300 Write block 1472 len 4096 data: 2
7087 075/300 Flush
7088 076/300 Read block 253 len 20480
7089 077/300 Write block 4330 len 40960 data: 3 3 2 2 2 2 2 2 2 3
7090 078/300 Read block 2842 len 40960
7091 079/300 Write block 4196 len 4096 data: 2
7092 080/300 Read block 2110 len 24576
7093 081/300 Flush
7094 082/300 Write block 2865 len 24576 data: 4 4 4 4 3 3
7095 083/300 Read block 4473 len 12288
7096 084/300 Read block 3216 len 20480
7097 085/300 Flush
7098 086/300 Flush
7099 087/300 Read block 1082 len 4096
7100 088/300 Read block 1302 len 32768
7101 089/300 Read block 4439 len 40960
7102 090/300 Read block 3569 len 20480
7103 091/300 Read block 4838 len 28672
7104 092/300 Read block 4906 len 20480
7105 093/300 Read block 2279 len 24576
7106 094/300 Read block 4297 len 24576
7107 095/300 Read block 2661 len 40960
7108 096/300 Read block 728 len 32768
7109 097/300 Write block 1920 len 36864 data: 2 2 2 2 2 2 2 2 2
7110 098/300 Write block 4363 len 36864 data: 2 2 2 2 2 2 2 2 3
7111 099/300 Read block 2151 len 36864
7112 100/300 Write block 4362 len 40960 data: 2 3 3 3 3 3 3 3 3 4
7113 101/300 Write block 777 len 24576 data: 2 2 2 2 2 2
7114 102/300 Read block 3677 len 20480
7115 103/300 Read block 4435 len 12288
7116 104/300 Read block 1773 len 16384
7117 105/300 Read block 2049 len 12288
7118 106/300 Write block 2486 len 12288 data: 4 5 5
7119 107/300 Write block 3778 len 20480 data: 2 2 2 2 2
7120 108/300 Read block 586 len 4096
7121 109/300 Write block 2162 len 36864 data: 4 4 3 3 3 3 3 4 4
7122 110/300 Read block 4950 len 32768
7123 111/300 Write block 3043 len 4096 data: 3
7124 112/300 Write block 734 len 36864 data: 3 3 3 4 3 2 2 2 2
7125 113/300 Write block 3509 len 32768 data: 2 2 2 2 2 2 3 3
7126 114/300 Write block 2229 len 16384 data: 3 3 4 3
7127 115/300 Read block 4745 len 40960
7128 116/300 Read block 1395 len 32768
7129 117/300 Write block 2288 len 32768 data: 2 2 2 2 2 2 3 3
7130 118/300 Write block 151 len 36864 data: 3 3 3 3 4 5 5 4 4
7131 119/300 Write block 4744 len 32768 data: 2 2 2 2 2 2 2 2
7132 120/300 Flush
7133 121/300 Read block 3453 len 32768
7134 122/300 Write block 2047 len 20480 data: 3 3 3 5 5
7135 123/300 Write block 184 len 20480 data: 2 2 3 3 3
7136 124/300 Read block 2005 len 36864
7137 125/300 Flush
7138 126/300 Read block 2755 len 32768
7139 127/300 Read block 326 len 20480
7140 128/300 Write block 3550 len 24576 data: 2 2 2 2 2 2
7141 129/300 Write block 1714 len 32768 data: 2 2 2 2 2 2 2 2
7142 130/300 Read block 2552 len 40960
7143 131/300 Read block 3413 len 16384
7144 132/300 Read block 409 len 16384
7145 133/300 Read block 2716 len 40960
7146 134/300 Read block 102 len 4096
7147 135/300 Read block 1794 len 24576
7148 136/300 Read block 509 len 8192
7149 137/300 Write block 4029 len 12288 data: 2 2 2
7150 138/300 Read block 1550 len 28672
7151 139/300 Read block 2537 len 20480
7152 140/300 Write block 1330 len 8192 data: 4 3
7153 141/300 Write block 1108 len 8192 data: 3 3
7154 142/300 Read block 2873 len 12288
7155 143/300 Write block 4662 len 40960 data: 3 2 2 2 2 2 2 2 2 3
7156 144/300 Read block 3522 len 20480
7157 145/300 Write block 825 len 40960 data: 3 3 3 3 3 3 3 3 3 2
7158 146/300 Read block 4735 len 4096
7159 147/300 Read block 87 len 8192
7160 148/300 Read block 1119 len 32768
7161 149/300 Read block 2710 len 12288
7162 150/300 Read block 3315 len 28672
7163 151/300 Write block 1061 len 20480 data: 5 6 5 6 5
7164 152/300 Read block 3521 len 40960
7165 153/300 Read block 15 len 8192
7166 154/300 Read block 4113 len 12288
7167 155/300 Write block 4321 len 28672 data: 3 4 5 5 4 4 4
7168 156/300 Read block 584 len 28672
7169 157/300 Write block 159 len 4096 data: 5
7170 158/300 Write block 646 len 8192 data: 2 3
7171 159/300 Write block 3704 len 32768 data: 2 2 3 3 3 3 2 3
7172 160/300 Read block 2290 len 40960
7173 161/300 Read block 901 len 16384
7174 162/300 Read block 2585 len 28672
7175 163/300 Flush
7176 164/300 Flush
7177 165/300 Write block 4195 len 40960 data: 2 3 2 2 2 2 2 3 3 3
7178 166/300 Read block 815 len 16384
7179 167/300 Flush
7180 168/300 Read block 3752 len 16384
7181 169/300 Read block 2685 len 8192
7182 170/300 Read block 4617 len 16384
7183 171/300 Write block 4315 len 40960 data: 2 2 3 3 3 3 4 5 6 6
7184 172/300 Write block 603 len 28672 data: 2 2 2 2 2 3 3
7185 173/300 Write block 2053 len 20480 data: 3 3 3 3 2
7186 174/300 Write block 72 len 24576 data: 4 4 4 4 4 4
7187 175/300 Read block 345 len 32768
7188 176/300 Read block 2193 len 32768
7189 177/300 Read block 3709 len 12288
7190 178/300 Write block 3010 len 24576 data: 3 2 2 2 2 2
7191 179/300 Write block 716 len 4096 data: 3
7192 180/300 Read block 3440 len 8192
7193 181/300 Write block 265 len 8192 data: 2 2
7194 182/300 Write block 4702 len 20480 data: 3 3 4 4 4
7195 183/300 Write block 3003 len 24576 data: 2 2 3 3 3 3
7196 184/300 Read block 4211 len 36864
7197 185/300 Read block 3692 len 28672
7198 186/300 Write block 1909 len 16384 data: 2 2 2 4
7199 187/300 Write block 4806 len 40960 data: 3 3 3 3 3 2 2 2 2 2
7200 188/300 Read block 2808 len 16384
7201 189/300 Write block 4874 len 20480 data: 4 4 3 3 3
7202 190/300 Read block 8 len 20480
7203 191/300 Read block 2956 len 40960
7204 192/300 Read block 881 len 12288
7205 193/300 Read block 547 len 16384
7206 194/300 Write block 2264 len 16384 data: 4 4 4 3
7207 195/300 Write block 4118 len 28672 data: 4 4 3 4 4 3 2
7208 196/300 Read block 3461 len 8192
7209 197/300 Write block 4066 len 20480 data: 2 2 2 2 2
7210 198/300 Write block 1873 len 8192 data: 3 3
7211 199/300 Read block 4623 len 20480
7212 200/300 Write block 3094 len 24576 data: 2 2 2 2 2 3
7213 201/300 Write block 2227 len 20480 data: 3 3 4 4 5
7214 202/300 Read block 4880 len 28672
7215 203/300 Write block 687 len 40960 data: 5 5 3 4 3 3 3 3 3 3
7216 204/300 Read block 3197 len 4096
7217 205/300 Read block 2599 len 24576
7218 206/300 Read block 644 len 24576
7219 207/300 Read block 4836 len 40960
7220 208/300 Write block 2812 len 4096 data: 3
7221 209/300 Read block 2853 len 40960
7222 210/300 Read block 4408 len 8192
7223 211/300 Write block 2671 len 4096 data: 2
7224 212/300 Read block 2224 len 32768
7225 213/300 Read block 3997 len 8192
7226 214/300 Read block 105 len 16384
7227 215/300 Flush
7228 216/300 Flush
7229 217/300 Write block 3104 len 8192 data: 4 4
7230 218/300 Write block 4449 len 32768 data: 3 4 3 3 2 3 3 3
7231 219/300 Read block 440 len 24576
7232 220/300 Read block 3132 len 12288
7233 221/300 Flush
7234 222/300 Read block 3249 len 28672
7235 223/300 Read block 1137 len 36864
7236 224/300 Write block 1861 len 20480 data: 4 3 3 3 2
7237 225/300 Write block 186 len 8192 data: 4 4
7238 226/300 Read block 2389 len 32768
7239 227/300 Read block 4813 len 16384
7240 228/300 Write block 2727 len 24576 data: 3 3 3 3 3 2
7241 229/300 Flush
7242 230/300 Write block 984 len 4096 data: 3
7243 231/300 Read block 3078 len 16384
7244 232/300 Flush
7245 233/300 Read block 3907 len 16384
7246 234/300 Write block 4356 len 8192 data: 2 2
7247 235/300 Read block 3691 len 28672
7248 236/300 Write block 4785 len 20480 data: 5 4 3 3 2
7249 237/300 Read block 174 len 8192
7250 238/300 Read block 427 len 16384
7251 239/300 Read block 2874 len 36864
7252 240/300 Read block 4244 len 36864
7253 241/300 Write block 4870 len 24576 data: 3 3 3 3 5 5
7254 242/300 Read block 3336 len 40960
7255 243/300 Write block 1127 len 12288 data: 2 3 2
7256 244/300 Flush
7257 245/300 Write block 2068 len 8192 data: 5 5
7258 246/300 Read block 736 len 24576
7259 247/300 Read block 548 len 20480
7260 248/300 Flush
7261 249/300 Read block 4177 len 8192
7262 250/300 Read block 4677 len 40960
7263 251/300 Flush
7264 252/300 Write block 4751 len 36864 data: 3 3 3 3 3 4 3 2 2
7265 253/300 Read block 1644 len 28672
7266 254/300 Read block 281 len 8192
7267 255/300 Read block 3357 len 40960
7268 256/300 Write block 4217 len 4096 data: 4
7269 257/300 Read block 1008 len 16384
7270 258/300 Read block 1102 len 36864
7271 259/300 Read block 1035 len 4096
7272 260/300 Write block 3224 len 28672 data: 5 5 5 4 4 3 3
7273 261/300 Write block 2473 len 24576 data: 3 3 2 2 3 4
7274 262/300 Read block 1500 len 24576
7275 263/300 Read block 4213 len 20480
7276 264/300 Flush
7277 265/300 Write block 382 len 16384 data: 3 2 2 3
7278 266/300 Read block 4925 len 24576
7279 267/300 Flush
7280 268/300 Write block 3936 len 32768 data: 5 3 3 2 2 2 2 2
7281 269/300 Write block 2173 len 8192 data: 4 4
7282 270/300 Flush
7283 271/300 Read block 3149 len 24576
7284 272/300 Write block 2896 len 36864 data: 2 2 2 2 2 2 2 2 2
7285 273/300 Read block 3757 len 4096
7286 274/300 Write block 4060 len 40960 data: 4 3 3 3 3 3 3 3 3 3
7287 275/300 Write block 2628 len 28672 data: 4 4 3 4 3 3 3
7288 276/300 Read block 2097 len 4096
7289 277/300 Write block 1592 len 20480 data: 3 2 2 2 2
7290 278/300 Write block 3512 len 16384 data: 3 3 3 4
7291 279/300 Write block 87 len 40960 data: 2 2 2 2 2 2 2 2 2 3
7292 280/300 Read block 3765 len 36864
7293 281/300 Write block 269 len 12288 data: 2 3 3
7294 282/300 Flush
7295 283/300 Read block 4421 len 12288
7296 284/300 Flush
7297 285/300 Read block 1644 len 4096
7298 286/300 Write block 2325 len 8192 data: 3 3
7299 287/300 Read block 232 len 32768
7300 288/300 Read block 3808 len 24576
7301 289/300 Flush
7302 290/300 Write block 812 len 8192 data: 3 4
7303 291/300 Flush
7304 292/300 Write block 3140 len 28672 data: 4 3 3 3 3 3 3
7305 293/300 Read block 4279 len 20480
7306 294/300 Write block 4185 len 40960 data: 5 5 4 4 4 4 4 4 4 3
7307 295/300 Read block 3254 len 4096
7308 296/300 Read block 2798 len 20480
7309 297/300 Write block 422 len 28672 data: 5 6 5 3 3 3 3
7310 298/300 Read block 4509 len 40960
7311 299/300 Write block 929 len 40960 data: 3 3 3 3 3 4 4 4 4 3
7312 300/300 Read block 1045 len 16384
7313 301/300 Flush
7314 Main task has work to do, go find it
7315 got dsc Start(0)
7316 start 0
7317 Jan 17 21:12:38.346 INFO request completed, latency_us: 211, response_code: 204, uri: /start/cid/0, method: POST, req_id: 77455969-0913-49c4-95fb-e073402bc193, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
7318 Replay: started 0, returned:Ok(())
7319 ----------------------------------------------------------------
7320 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
7321 States: Offline Active Active
7322 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7323 3113 Acked Read 8 Sent Done Done false
7324 3114 Acked Read 3 Sent Done Done false
7325 3115 Acked Flush 0 Sent Done Done false
7326 3116 Acked Read 8 Sent Done Done false
7327 3117 Acked Read 4 Sent Done Done false
7328 3118 Acked Read 5 Sent Done Done false
7329 3119 Acked Flush 0 Sent Done Done false
7330 3120 Acked Write 8 Sent Done Done false
7331 3121 Acked Write 10 Sent Done Done false
7332 3122 Acked Write 5 Sent Done Done false
7333 3123 Acked Read 1 Sent Done Done false
7334 3124 Acked Read 8 Sent Done Done false
7335 3125 Acked Flush 0 Sent Done Done false
7336 3126 Acked Read 9 Sent Done Done false
7337 3127 Acked Read 8 Sent Done Done false
7338 3128 Acked Read 4 Sent Done Done false
7339 3129 Acked Write 9 Sent Done Done false
7340 3130 Acked Read 1 Sent Done Done false
7341 3131 Acked Write 3 Sent Done Done false
7342 3132 Acked Flush 0 Sent Done Done false
7343 3133 Acked Read 4 Sent Done Done false
7344 3134 Acked Read 1 Sent Done Done false
7345 3135 Acked Read 3 Sent Done Done false
7346 3136 Acked Flush 0 Sent Done Done false
7347 3137 Acked Read 1 Sent Done Done false
7348 3138 Acked Read 7 Sent Done Done false
7349 3139 Acked Write 8 Sent Done Done false
7350 3140 Acked Write 10 Sent Done Done false
7351 3141 Acked Flush 0 Sent Done Done false
7352 3142 Acked Flush 0 Sent Done Done false
7353 3143 Acked Read 3 Sent Done Done false
7354 3144 Acked Write 1 Sent Done Done false
7355 3145 Acked Read 1 Sent Done Done false
7356 3146 Acked Write 7 Sent Done Done false
7357 3147 Acked Write 5 Sent Done Done false
7358 3148 Acked Write 7 Sent Done Done false
7359 3149 Acked Read 6 Sent Done Done false
7360 3150 Acked Read 3 Sent Done Done false
7361 3151 Acked Read 8 Sent Done Done false
7362 3152 Acked Read 5 Sent Done Done false
7363 3153 Acked Read 4 Sent Done Done false
7364 3154 Acked Read 5 Sent Done Done false
7365 3155 Acked Read 6 Sent Done Done false
7366 3156 Acked Read 5 Sent Done Done false
7367 3157 Acked Write 8 Sent Done Done false
7368 3158 Acked Read 1 Sent Done Done false
7369 3159 Acked Write 5 Sent Done Done false
7370 3160 Acked Flush 0 Sent Done Done false
7371 3161 Acked Read 9 Sent Done Done false
7372 3162 Acked Flush 0 Sent Done Done false
7373 3163 Acked Write 4 Sent Done Done false
7374 3164 Acked Write 1 Sent Done Done false
7375 3165 Acked Write 7 Sent Done Done false
7376 3166 Acked Flush 0 Sent Done Done false
7377 3167 Acked Write 7 Sent Done Done false
7378 3168 Acked Write 2 Sent Done Done false
7379 3169 Acked Write 9 Sent Done Done false
7380 3170 Acked Flush 0 Sent Done Done false
7381 3171 Acked Flush 0 Sent Done Done false
7382 3172 Acked Write 7 Sent Done Done false
7383 3173 Acked Write 8 Sent Done Done false
7384 3174 Acked Read 9 Sent Done Done false
7385 3175 Acked Write 7 Sent Done Done false
7386 3176 Acked Read 4 Sent Done Done false
7387 3177 Acked Read 10 Sent Done Done false
7388 3178 Acked Write 9 Sent Done Done false
7389 3179 Acked Read 1 Sent Done Done false
7390 3180 Acked Read 5 Sent Done Done false
7391 3181 Acked Read 5 Sent Done Done false
7392 3182 Acked Read 4 Sent Done Done false
7393 3183 Acked Write 2 Sent Done Done false
7394 3184 Acked Flush 0 Sent Done Done false
7395 3185 Acked Write 8 Sent Done Done false
7396 3186 Acked Write 1 Sent Done Done false
7397 3187 Acked Flush 0 Sent Done Done false
7398 3188 Acked Read 5 Sent Done Done false
7399 3189 Acked Write 10 Sent Done Done false
7400 3190 Acked Read 10 Sent Done Done false
7401 3191 Acked Write 1 Sent Done Done false
7402 3192 Acked Read 6 Sent Done Done false
7403 3193 Acked Flush 0 Sent Done Done false
7404 3194 Acked Write 6 Sent Done Done false
7405 3195 Acked Read 3 Sent Done Done false
7406 3196 Acked Read 5 Sent Done Done false
7407 3197 Acked Flush 0 Sent Done Done false
7408 3198 Acked Flush 0 Sent Done Done false
7409 3199 Acked Read 1 Sent Done Done false
7410 3200 Acked Read 8 Sent Done Done false
7411 3201 Acked Read 10 Sent Done Done false
7412 3202 Acked Read 5 Sent Done Done false
7413 3203 Acked Read 7 Sent Done Done false
7414 3204 Acked Read 5 Sent Done Done false
7415 3205 Acked Read 6 Sent Done Done false
7416 3206 Acked Read 6 Sent Done Done false
7417 3207 Acked Read 10 Sent Done Done false
7418 3208 Acked Read 8 Sent Done Done false
7419 3209 Acked Write 9 Sent Done Done false
7420 3210 Acked Write 9 Sent Done Done false
7421 3211 Acked Read 9 Sent Done Done false
7422 3212 Acked Write 10 Sent Done Done false
7423 3213 Acked Write 6 Sent Done Done false
7424 3214 Acked Read 5 Sent Done Done false
7425 3215 Acked Read 3 Sent Done Done false
7426 3216 Acked Read 4 Sent Done Done false
7427 3217 Acked Read 3 Sent Done Done false
7428 3218 Acked Write 3 Sent Done Done false
7429 3219 Acked Write 5 Sent Done Done false
7430 3220 Acked Read 1 Sent Done Done false
7431 3221 Acked Write 9 Sent Done Done false
7432 3222 Acked Read 8 Sent Done Done false
7433 3223 Acked Write 1 Sent Done Done false
7434 3224 Acked Write 9 Sent Done Done false
7435 3225 Acked Write 8 Sent Done Done false
7436 3226 Acked Write 4 Sent Done Done false
7437 3227 Acked Read 10 Sent Done Done false
7438 3228 Acked Read 8 Sent Done Done false
7439 3229 Acked Write 8 Sent Done Done false
7440 3230 Acked Write 9 Sent Done Done false
7441 3231 Acked Write 8 Sent Done Done false
7442 3232 Acked Flush 0 Sent Done Done false
7443 3233 Acked Read 8 Sent Done Done false
7444 3234 Acked Write 5 Sent Done Done false
7445 3235 Acked Write 5 Sent Done Done false
7446 3236 Acked Read 9 Sent Done Done false
7447 3237 Acked Flush 0 Sent Done Done false
7448 3238 Acked Read 8 Sent Done Done false
7449 3239 Acked Read 5 Sent Done Done false
7450 3240 Acked Write 6 Sent Done Done false
7451 3241 Acked Write 8 Sent Done Done false
7452 3242 Acked Read 10 Sent Done Done false
7453 3243 Acked Read 4 Sent Done Done false
7454 3244 Acked Read 4 Sent Done Done false
7455 3245 Acked Read 10 Sent Done Done false
7456 3246 Acked Read 1 Sent Done Done false
7457 3247 Acked Read 6 Sent Done Done false
7458 3248 Acked Read 2 Sent Done Done false
7459 3249 Acked Write 3 Sent Done Done false
7460 3250 Acked Read 7 Sent Done Done false
7461 3251 Acked Read 5 Sent Done Done false
7462 3252 Acked Write 2 Sent Done Done false
7463 3253 Acked Write 2 Sent Done Done false
7464 3254 Acked Read 3 Sent Done Done false
7465 3255 Acked Write 10 Sent Done Done false
7466 3256 Acked Read 5 Sent Done Done false
7467 3257 Acked Write 10 Sent Done Done false
7468 3258 Acked Read 1 Sent Done Done false
7469 3259 Acked Read 2 Sent Done Done false
7470 3260 Acked Read 8 Sent Done Done false
7471 3261 Acked Read 3 Sent Done Done false
7472 3262 Acked Read 7 Sent Done Done false
7473 3263 Acked Write 5 Sent Done Done false
7474 3264 Acked Read 10 Sent Done Done false
7475 3265 Acked Read 2 Sent Done Done false
7476 3266 Acked Read 3 Sent Done Done false
7477 3267 Acked Write 7 Sent Done Done false
7478 3268 Acked Read 7 Sent Done Done false
7479 3269 Acked Write 1 Sent Done Done false
7480 3270 Acked Write 2 Sent Done Done false
7481 3271 Acked Write 8 Sent Done Done false
7482 3272 Acked Read 10 Sent Done Done false
7483 3273 Acked Read 4 Sent Done Done false
7484 3274 Acked Read 7 Sent Done Done false
7485 3275 Acked Flush 0 Sent Done Done false
7486 3276 Acked Flush 0 Sent Done Done false
7487 3277 Acked Write 10 Sent Done Done false
7488 3278 Acked Read 4 Sent Done Done false
7489 3279 Acked Flush 0 Sent Done Done false
7490 3280 Acked Read 4 Sent Done Done false
7491 3281 Acked Read 2 Sent Done Done false
7492 3282 Acked Read 4 Sent Done Done false
7493 3283 Acked Write 10 Sent Done Done false
7494 3284 Acked Write 7 Sent Done Done false
7495 3285 Acked Write 5 Sent Done Done false
7496 3286 Acked Write 6 Sent Done Done false
7497 3287 Acked Read 8 Sent Done Done false
7498 3288 Acked Read 8 Sent Done Done false
7499 3289 Acked Read 3 Sent Done Done false
7500 3290 Acked Write 6 Sent Done Done false
7501 3291 Acked Write 1 Sent Done Done false
7502 3292 Acked Read 2 Sent Done Done false
7503 3293 Acked Write 2 Sent Done Done false
7504 3294 Acked Write 5 Sent Done Done false
7505 3295 Acked Write 6 Sent Done Done false
7506 3296 Acked Read 9 Sent Done Done false
7507 3297 Acked Read 7 Sent Done Done false
7508 3298 Acked Write 4 Sent Done Done false
7509 3299 Acked Write 10 Sent Done Done false
7510 3300 Acked Read 4 Sent Done Done false
7511 3301 Acked Write 5 Sent Done Done false
7512 3302 Acked Read 5 Sent Done Done false
7513 3303 Acked Read 10 Sent Done Done false
7514 3304 Acked Read 3 Sent Done Done false
7515 3305 Acked Read 4 Sent Done Done false
7516 3306 Acked Write 4 Sent Done Done false
7517 3307 Acked Write 7 Sent Done Done false
7518 3308 Acked Read 2 Sent Done Done false
7519 3309 Acked Write 5 Sent Done Done false
7520 3310 Acked Write 2 Sent Done Done false
7521 3311 Acked Read 5 Sent Done Done false
7522 3312 Acked Write 6 Sent Done Done false
7523 3313 Acked Write 5 Sent Done Done false
7524 3314 Acked Read 7 Sent Done Done false
7525 3315 Acked Write 10 Sent Done Done false
7526 3316 Acked Read 1 Sent Done Done false
7527 3317 Acked Read 6 Sent Done Done false
7528 3318 Acked Read 6 Sent Done Done false
7529 3319 Acked Read 10 Sent Done Done false
7530 3320 Acked Write 1 Sent Done Done false
7531 3321 Acked Read 10 Sent Done Done false
7532 3322 Acked Read 2 Sent Done Done false
7533 3323 Acked Write 1 Sent Done Done false
7534 3324 Acked Read 8 Sent Done Done false
7535 3325 Acked Read 2 Sent Done Done false
7536 3326 Acked Read 4 Sent Done Done false
7537 3327 Acked Flush 0 Sent Done Done false
7538 3328 Acked Flush 0 Sent Done Done false
7539 3329 Acked Write 2 Sent Done Done false
7540 3330 Acked Write 8 Sent Done Done false
7541 3331 Acked Read 6 Sent Done Done false
7542 3332 Acked Read 3 Sent Done Done false
7543 3333 Acked Flush 0 Sent Done Done false
7544 3334 Acked Read 7 Sent Done Done false
7545 3335 Acked Read 9 Sent Done Done false
7546 3336 Acked Write 5 Sent Done Done false
7547 3337 Acked Write 2 Sent Done Done false
7548 3338 Acked Read 8 Sent Done Done false
7549 3339 Acked Read 4 Sent Done Done false
7550 3340 Acked Write 6 Sent Done Done false
7551 3341 Acked Flush 0 Sent Done Done false
7552 3342 Acked Write 1 Sent Done Done false
7553 3343 Acked Read 4 Sent Done Done false
7554 3344 Acked Flush 0 Sent Done Done false
7555 3345 Acked Read 4 Sent Done Done false
7556 3346 Acked Write 2 Sent Done Done false
7557 3347 Acked Read 7 Sent Done Done false
7558 3348 Acked Write 5 Sent Done Done false
7559 3349 Acked Read 2 Sent Done Done false
7560 3350 Acked Read 4 Sent Done Done false
7561 3351 Acked Read 9 Sent Done Done false
7562 3352 Acked Read 9 Sent Done Done false
7563 3353 Acked Write 6 Sent Done Done false
7564 3354 Acked Read 10 Sent Done Done false
7565 3355 Acked Write 3 Sent Done Done false
7566 3356 Acked Flush 0 Sent Done Done false
7567 3357 Acked Write 2 Sent Done Done false
7568 3358 Acked Read 6 Sent Done Done false
7569 3359 Acked Read 5 Sent Done Done false
7570 3360 Acked Flush 0 Sent Done Done false
7571 3361 Acked Read 2 Sent Done Done false
7572 3362 Acked Read 10 Sent Done Done false
7573 3363 Acked Flush 0 Sent Done Done false
7574 3364 Acked Write 9 Sent Done Done false
7575 3365 Acked Read 7 Sent Done Done false
7576 3366 Acked Read 2 Sent Done Done false
7577 3367 Acked Read 10 Sent Done Done false
7578 3368 Acked Write 1 Sent Done Done false
7579 3369 Acked Read 4 Sent Done Done false
7580 3370 Acked Read 9 Sent Done Done false
7581 3371 Acked Read 1 Sent Done Done false
7582 3372 Acked Write 7 Sent Done Done false
7583 3373 Acked Write 6 Sent Done Done false
7584 3374 Acked Read 6 Sent Done Done false
7585 3375 Acked Read 5 Sent Done Done false
7586 3376 Acked Flush 0 Sent Done Done false
7587 3377 Acked Write 4 Sent Done Done false
7588 3378 Acked Read 6 Sent Done Done false
7589 3379 Acked Flush 0 Sent Done Done false
7590 3380 Acked Write 8 Sent Done Done false
7591 3381 Acked Write 2 Sent Done Done false
7592 3382 Acked Flush 0 Sent Done Done false
7593 3383 Acked Read 6 Sent Done Done false
7594 3384 Acked Write 9 Sent Done Done false
7595 3385 Acked Read 1 Sent Done Done false
7596 3386 Acked Write 10 Sent Done Done false
7597 3387 Acked Write 7 Sent Done Done false
7598 3388 Acked Read 1 Sent Done Done false
7599 3389 Acked Write 5 Sent Done Done false
7600 3390 Acked Write 4 Sent Done Done false
7601 3391 Acked Write 10 Sent Done Done false
7602 3392 Acked Read 9 Sent Done Done false
7603 3393 Acked Write 3 Sent Done Done false
7604 3394 Acked Flush 0 Sent Done Done false
7605 3395 Acked Read 3 Sent Done Done false
7606 3396 Acked Flush 0 Sent Done Done false
7607 3397 Acked Read 1 Sent Done Done false
7608 3398 Acked Write 2 Sent Done Done false
7609 3399 Acked Read 8 Sent Done Done false
7610 3400 Acked Read 6 Sent Done Done false
7611 3401 Acked Flush 0 Sent Done Done false
7612 3402 Acked Write 2 Sent Done Done false
7613 3403 Acked Flush 0 Sent Done Done false
7614 3404 Acked Write 7 Sent Done Done false
7615 3405 Acked Read 5 Sent Done Done false
7616 3406 Acked Write 10 Sent Done Done false
7617 3407 Acked Read 1 Sent Done Done false
7618 3408 Acked Read 5 Sent Done Done false
7619 3409 Acked Write 7 Sent Done Done false
7620 3410 Acked Read 10 Sent Done Done false
7621 3411 Acked Write 10 Sent Done Done false
7622 3412 Acked Read 4 Sent Done Done false
7623 3413 Acked Flush 0 Sent Done Done false
7624 STATES DS:0 DS:1 DS:2 TOTAL
7625 Sent 301 0 0 301
7626 Done 0 301 301 602
7627 Skipped 0 0 0 0
7628 Error 0 0 0 0
7629 Last Flush: 3112 3413 3413
7630 Downstairs last five completed: 3112 3111 3110 3109 3108
7631 Upstairs last five completed: 3413 3412 3411 3410 3409
7632 CLIENT: Up:0 ds:301 act:2
7633 ----------------------------------------------------------------
7634 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
7635 States: Offline Active Active
7636 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7637 3113 Acked Read 8 Sent Done Done false
7638 3114 Acked Read 3 Sent Done Done false
7639 3115 Acked Flush 0 Sent Done Done false
7640 3116 Acked Read 8 Sent Done Done false
7641 3117 Acked Read 4 Sent Done Done false
7642 3118 Acked Read 5 Sent Done Done false
7643 3119 Acked Flush 0 Sent Done Done false
7644 3120 Acked Write 8 Sent Done Done false
7645 3121 Acked Write 10 Sent Done Done false
7646 3122 Acked Write 5 Sent Done Done false
7647 3123 Acked Read 1 Sent Done Done false
7648 3124 Acked Read 8 Sent Done Done false
7649 3125 Acked Flush 0 Sent Done Done false
7650 3126 Acked Read 9 Sent Done Done false
7651 3127 Acked Read 8 Sent Done Done false
7652 3128 Acked Read 4 Sent Done Done false
7653 3129 Acked Write 9 Sent Done Done false
7654 3130 Acked Read 1 Sent Done Done false
7655 3131 Acked Write 3 Sent Done Done false
7656 3132 Acked Flush 0 Sent Done Done false
7657 3133 Acked Read 4 Sent Done Done false
7658 3134 Acked Read 1 Sent Done Done false
7659 3135 Acked Read 3 Sent Done Done false
7660 3136 Acked Flush 0 Sent Done Done false
7661 3137 Acked Read 1 Sent Done Done false
7662 3138 Acked Read 7 Sent Done Done false
7663 3139 Acked Write 8 Sent Done Done false
7664 3140 Acked Write 10 Sent Done Done false
7665 3141 Acked Flush 0 Sent Done Done false
7666 3142 Acked Flush 0 Sent Done Done false
7667 3143 Acked Read 3 Sent Done Done false
7668 3144 Acked Write 1 Sent Done Done false
7669 3145 Acked Read 1 Sent Done Done false
7670 3146 Acked Write 7 Sent Done Done false
7671 3147 Acked Write 5 Sent Done Done false
7672 3148 Acked Write 7 Sent Done Done false
7673 3149 Acked Read 6 Sent Done Done false
7674 3150 Acked Read 3 Sent Done Done false
7675 3151 Acked Read 8 Sent Done Done false
7676 3152 Acked Read 5 Sent Done Done false
7677 3153 Acked Read 4 Sent Done Done false
7678 3154 Acked Read 5 Sent Done Done false
7679 3155 Acked Read 6 Sent Done Done false
7680 3156 Acked Read 5 Sent Done Done false
7681 3157 Acked Write 8 Sent Done Done false
7682 3158 Acked Read 1 Sent Done Done false
7683 3159 Acked Write 5 Sent Done Done false
7684 3160 Acked Flush 0 Sent Done Done false
7685 3161 Acked Read 9 Sent Done Done false
7686 3162 Acked Flush 0 Sent Done Done false
7687 3163 Acked Write 4 Sent Done Done false
7688 3164 Acked Write 1 Sent Done Done false
7689 3165 Acked Write 7 Sent Done Done false
7690 3166 Acked Flush 0 Sent Done Done false
7691 3167 Acked Write 7 Sent Done Done false
7692 3168 Acked Write 2 Sent Done Done false
7693 3169 Acked Write 9 Sent Done Done false
7694 3170 Acked Flush 0 Sent Done Done false
7695 3171 Acked Flush 0 Sent Done Done false
7696 3172 Acked Write 7 Sent Done Done false
7697 3173 Acked Write 8 Sent Done Done false
7698 3174 Acked Read 9 Sent Done Done false
7699 3175 Acked Write 7 Sent Done Done false
7700 3176 Acked Read 4 Sent Done Done false
7701 3177 Acked Read 10 Sent Done Done false
7702 3178 Acked Write 9 Sent Done Done false
7703 3179 Acked Read 1 Sent Done Done false
7704 3180 Acked Read 5 Sent Done Done false
7705 3181 Acked Read 5 Sent Done Done false
7706 3182 Acked Read 4 Sent Done Done false
7707 3183 Acked Write 2 Sent Done Done false
7708 3184 Acked Flush 0 Sent Done Done false
7709 3185 Acked Write 8 Sent Done Done false
7710 3186 Acked Write 1 Sent Done Done false
7711 3187 Acked Flush 0 Sent Done Done false
7712 3188 Acked Read 5 Sent Done Done false
7713 3189 Acked Write 10 Sent Done Done false
7714 3190 Acked Read 10 Sent Done Done false
7715 3191 Acked Write 1 Sent Done Done false
7716 3192 Acked Read 6 Sent Done Done false
7717 3193 Acked Flush 0 Sent Done Done false
7718 3194 Acked Write 6 Sent Done Done false
7719 3195 Acked Read 3 Sent Done Done false
7720 3196 Acked Read 5 Sent Done Done false
7721 3197 Acked Flush 0 Sent Done Done false
7722 3198 Acked Flush 0 Sent Done Done false
7723 3199 Acked Read 1 Sent Done Done false
7724 3200 Acked Read 8 Sent Done Done false
7725 3201 Acked Read 10 Sent Done Done false
7726 3202 Acked Read 5 Sent Done Done false
7727 3203 Acked Read 7 Sent Done Done false
7728 3204 Acked Read 5 Sent Done Done false
7729 3205 Acked Read 6 Sent Done Done false
7730 3206 Acked Read 6 Sent Done Done false
7731 3207 Acked Read 10 Sent Done Done false
7732 3208 Acked Read 8 Sent Done Done false
7733 3209 Acked Write 9 Sent Done Done false
7734 3210 Acked Write 9 Sent Done Done false
7735 3211 Acked Read 9 Sent Done Done false
7736 3212 Acked Write 10 Sent Done Done false
7737 3213 Acked Write 6 Sent Done Done false
7738 3214 Acked Read 5 Sent Done Done false
7739 3215 Acked Read 3 Sent Done Done false
7740 3216 Acked Read 4 Sent Done Done false
7741 3217 Acked Read 3 Sent Done Done false
7742 3218 Acked Write 3 Sent Done Done false
7743 3219 Acked Write 5 Sent Done Done false
7744 3220 Acked Read 1 Sent Done Done false
7745 3221 Acked Write 9 Sent Done Done false
7746 3222 Acked Read 8 Sent Done Done false
7747 3223 Acked Write 1 Sent Done Done false
7748 3224 Acked Write 9 Sent Done Done false
7749 3225 Acked Write 8 Sent Done Done false
7750 3226 Acked Write 4 Sent Done Done false
7751 3227 Acked Read 10 Sent Done Done false
7752 3228 Acked Read 8 Sent Done Done false
7753 3229 Acked Write 8 Sent Done Done false
7754 3230 Acked Write 9 Sent Done Done false
7755 3231 Acked Write 8 Sent Done Done false
7756 3232 Acked Flush 0 Sent Done Done false
7757 3233 Acked Read 8 Sent Done Done false
7758 3234 Acked Write 5 Sent Done Done false
7759 3235 Acked Write 5 Sent Done Done false
7760 3236 Acked Read 9 Sent Done Done false
7761 3237 Acked Flush 0 Sent Done Done false
7762 3238 Acked Read 8 Sent Done Done false
7763 3239 Acked Read 5 Sent Done Done false
7764 3240 Acked Write 6 Sent Done Done false
7765 3241 Acked Write 8 Sent Done Done false
7766 3242 Acked Read 10 Sent Done Done false
7767 3243 Acked Read 4 Sent Done Done false
7768 3244 Acked Read 4 Sent Done Done false
7769 3245 Acked Read 10 Sent Done Done false
7770 3246 Acked Read 1 Sent Done Done false
7771 3247 Acked Read 6 Sent Done Done false
7772 3248 Acked Read 2 Sent Done Done false
7773 3249 Acked Write 3 Sent Done Done false
7774 3250 Acked Read 7 Sent Done Done false
7775 3251 Acked Read 5 Sent Done Done false
7776 3252 Acked Write 2 Sent Done Done false
7777 3253 Acked Write 2 Sent Done Done false
7778 3254 Acked Read 3 Sent Done Done false
7779 3255 Acked Write 10 Sent Done Done false
7780 3256 Acked Read 5 Sent Done Done false
7781 3257 Acked Write 10 Sent Done Done false
7782 3258 Acked Read 1 Sent Done Done false
7783 3259 Acked Read 2 Sent Done Done false
7784 3260 Acked Read 8 Sent Done Done false
7785 3261 Acked Read 3 Sent Done Done false
7786 3262 Acked Read 7 Sent Done Done false
7787 3263 Acked Write 5 Sent Done Done false
7788 3264 Acked Read 10 Sent Done Done false
7789 3265 Acked Read 2 Sent Done Done false
7790 3266 Acked Read 3 Sent Done Done false
7791 3267 Acked Write 7 Sent Done Done false
7792 3268 Acked Read 7 Sent Done Done false
7793 3269 Acked Write 1 Sent Done Done false
7794 3270 Acked Write 2 Sent Done Done false
7795 3271 Acked Write 8 Sent Done Done false
7796 3272 Acked Read 10 Sent Done Done false
7797 3273 Acked Read 4 Sent Done Done false
7798 3274 Acked Read 7 Sent Done Done false
7799 3275 Acked Flush 0 Sent Done Done false
7800 3276 Acked Flush 0 Sent Done Done false
7801 3277 Acked Write 10 Sent Done Done false
7802 3278 Acked Read 4 Sent Done Done false
7803 3279 Acked Flush 0 Sent Done Done false
7804 3280 Acked Read 4 Sent Done Done false
7805 3281 Acked Read 2 Sent Done Done false
7806 3282 Acked Read 4 Sent Done Done false
7807 3283 Acked Write 10 Sent Done Done false
7808 3284 Acked Write 7 Sent Done Done false
7809 3285 Acked Write 5 Sent Done Done false
7810 3286 Acked Write 6 Sent Done Done false
7811 3287 Acked Read 8 Sent Done Done false
7812 3288 Acked Read 8 Sent Done Done false
7813 3289 Acked Read 3 Sent Done Done false
7814 3290 Acked Write 6 Sent Done Done false
7815 3291 Acked Write 1 Sent Done Done false
7816 3292 Acked Read 2 Sent Done Done false
7817 3293 Acked Write 2 Sent Done Done false
7818 3294 Acked Write 5 Sent Done Done false
7819 3295 Acked Write 6 Sent Done Done false
7820 3296 Acked Read 9 Sent Done Done false
7821 3297 Acked Read 7 Sent Done Done false
7822 3298 Acked Write 4 Sent Done Done false
7823 3299 Acked Write 10 Sent Done Done false
7824 3300 Acked Read 4 Sent Done Done false
7825 3301 Acked Write 5 Sent Done Done false
7826 3302 Acked Read 5 Sent Done Done false
7827 3303 Acked Read 10 Sent Done Done false
7828 3304 Acked Read 3 Sent Done Done false
7829 3305 Acked Read 4 Sent Done Done false
7830 3306 Acked Write 4 Sent Done Done false
7831 3307 Acked Write 7 Sent Done Done false
7832 3308 Acked Read 2 Sent Done Done false
7833 3309 Acked Write 5 Sent Done Done false
7834 3310 Acked Write 2 Sent Done Done false
7835 3311 Acked Read 5 Sent Done Done false
7836 3312 Acked Write 6 Sent Done Done false
7837 3313 Acked Write 5 Sent Done Done false
7838 3314 Acked Read 7 Sent Done Done false
7839 3315 Acked Write 10 Sent Done Done false
7840 3316 Acked Read 1 Sent Done Done false
7841 3317 Acked Read 6 Sent Done Done false
7842 3318 Acked Read 6 Sent Done Done false
7843 3319 Acked Read 10 Sent Done Done false
7844 3320 Acked Write 1 Sent Done Done false
7845 3321 Acked Read 10 Sent Done Done false
7846 3322 Acked Read 2 Sent Done Done false
7847 3323 Acked Write 1 Sent Done Done false
7848 3324 Acked Read 8 Sent Done Done false
7849 3325 Acked Read 2 Sent Done Done false
7850 3326 Acked Read 4 Sent Done Done false
7851 3327 Acked Flush 0 Sent Done Done false
7852 3328 Acked Flush 0 Sent Done Done false
7853 3329 Acked Write 2 Sent Done Done false
7854 3330 Acked Write 8 Sent Done Done false
7855 3331 Acked Read 6 Sent Done Done false
7856 3332 Acked Read 3 Sent Done Done false
7857 3333 Acked Flush 0 Sent Done Done false
7858 3334 Acked Read 7 Sent Done Done false
7859 3335 Acked Read 9 Sent Done Done false
7860 3336 Acked Write 5 Sent Done Done false
7861 3337 Acked Write 2 Sent Done Done false
7862 3338 Acked Read 8 Sent Done Done false
7863 3339 Acked Read 4 Sent Done Done false
7864 3340 Acked Write 6 Sent Done Done false
7865 3341 Acked Flush 0 Sent Done Done false
7866 3342 Acked Write 1 Sent Done Done false
7867 3343 Acked Read 4 Sent Done Done false
7868 3344 Acked Flush 0 Sent Done Done false
7869 3345 Acked Read 4 Sent Done Done false
7870 3346 Acked Write 2 Sent Done Done false
7871 3347 Acked Read 7 Sent Done Done false
7872 3348 Acked Write 5 Sent Done Done false
7873 3349 Acked Read 2 Sent Done Done false
7874 3350 Acked Read 4 Sent Done Done false
7875 3351 Acked Read 9 Sent Done Done false
7876 3352 Acked Read 9 Sent Done Done false
7877 3353 Acked Write 6 Sent Done Done false
7878 3354 Acked Read 10 Sent Done Done false
7879 3355 Acked Write 3 Sent Done Done false
7880 3356 Acked Flush 0 Sent Done Done false
7881 3357 Acked Write 2 Sent Done Done false
7882 3358 Acked Read 6 Sent Done Done false
7883 3359 Acked Read 5 Sent Done Done false
7884 3360 Acked Flush 0 Sent Done Done false
7885 3361 Acked Read 2 Sent Done Done false
7886 3362 Acked Read 10 Sent Done Done false
7887 3363 Acked Flush 0 Sent Done Done false
7888 3364 Acked Write 9 Sent Done Done false
7889 3365 Acked Read 7 Sent Done Done false
7890 3366 Acked Read 2 Sent Done Done false
7891 3367 Acked Read 10 Sent Done Done false
7892 3368 Acked Write 1 Sent Done Done false
7893 3369 Acked Read 4 Sent Done Done false
7894 3370 Acked Read 9 Sent Done Done false
7895 3371 Acked Read 1 Sent Done Done false
7896 3372 Acked Write 7 Sent Done Done false
7897 3373 Acked Write 6 Sent Done Done false
7898 3374 Acked Read 6 Sent Done Done false
7899 3375 Acked Read 5 Sent Done Done false
7900 3376 Acked Flush 0 Sent Done Done false
7901 3377 Acked Write 4 Sent Done Done false
7902 3378 Acked Read 6 Sent Done Done false
7903 3379 Acked Flush 0 Sent Done Done false
7904 3380 Acked Write 8 Sent Done Done false
7905 3381 Acked Write 2 Sent Done Done false
7906 3382 Acked Flush 0 Sent Done Done false
7907 3383 Acked Read 6 Sent Done Done false
7908 3384 Acked Write 9 Sent Done Done false
7909 3385 Acked Read 1 Sent Done Done false
7910 3386 Acked Write 10 Sent Done Done false
7911 3387 Acked Write 7 Sent Done Done false
7912 3388 Acked Read 1 Sent Done Done false
7913 3389 Acked Write 5 Sent Done Done false
7914 3390 Acked Write 4 Sent Done Done false
7915 3391 Acked Write 10 Sent Done Done false
7916 3392 Acked Read 9 Sent Done Done false
7917 3393 Acked Write 3 Sent Done Done false
7918 3394 Acked Flush 0 Sent Done Done false
7919 3395 Acked Read 3 Sent Done Done false
7920 3396 Acked Flush 0 Sent Done Done false
7921 3397 Acked Read 1 Sent Done Done false
7922 3398 Acked Write 2 Sent Done Done false
7923 3399 Acked Read 8 Sent Done Done false
7924 3400 Acked Read 6 Sent Done Done false
7925 3401 Acked Flush 0 Sent Done Done false
7926 3402 Acked Write 2 Sent Done Done false
7927 3403 Acked Flush 0 Sent Done Done false
7928 3404 Acked Write 7 Sent Done Done false
7929 3405 Acked Read 5 Sent Done Done false
7930 3406 Acked Write 10 Sent Done Done false
7931 3407 Acked Read 1 Sent Done Done false
7932 3408 Acked Read 5 Sent Done Done false
7933 3409 Acked Write 7 Sent Done Done false
7934 3410 Acked Read 10 Sent Done Done false
7935 3411 Acked Write 10 Sent Done Done false
7936 3412 Acked Read 4 Sent Done Done false
7937 3413 Acked Flush 0 Sent Done Done false
7938 STATES DS:0 DS:1 DS:2 TOTAL
7939 Sent 301 0 0 301
7940 Done 0 301 301 602
7941 Skipped 0 0 0 0
7942 Error 0 0 0 0
7943 Last Flush: 3112 3413 3413
7944 Downstairs last five completed: 3112 3111 3110 3109 3108
7945 Upstairs last five completed: 3413 3412 3411 3410 3409
7946 CLIENT: Up:0 ds:301 act:2
79472025-01-17T21:12:44.100ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
79482025-01-17T21:12:44.101ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
79492025-01-17T21:12:44.101ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
79502025-01-17T21:12:44.101ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
79512025-01-17T21:12:44.101ZINFOcrucible: client task is exiting = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
7952 ----------------------------------------------------------------
7953 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
7954 States: Offline Active Active
7955 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7956 3113 Acked Read 8 Sent Done Done false
7957 3114 Acked Read 3 Sent Done Done false
7958 3115 Acked Flush 0 Sent Done Done false
7959 3116 Acked Read 8 Sent Done Done false
7960 3117 Acked Read 4 Sent Done Done false
7961 3118 Acked Read 5 Sent Done Done false
7962 3119 Acked Flush 0 Sent Done Done false
7963 3120 Acked Write 8 Sent Done Done false
7964 3121 Acked Write 10 Sent Done Done false
7965 3122 Acked Write 5 Sent Done Done false
7966 3123 Acked Read 1 Sent Done Done false
7967 3124 Acked Read 8 Sent Done Done false
7968 3125 Acked Flush 0 Sent Done Done false
7969 3126 Acked Read 9 Sent Done Done false
7970 3127 Acked Read 8 Sent Done Done false
7971 3128 Acked Read 4 Sent Done Done false
7972 3129 Acked Write 9 Sent Done Done false
7973 3130 Acked Read 1 Sent Done Done false
7974 3131 Acked Write 3 Sent Done Done false
7975 3132 Acked Flush 0 Sent Done Done false
7976 3133 Acked Read 4 Sent Done Done false
7977 3134 Acked Read 1 Sent Done Done false
7978 3135 Acked Read 3 Sent Done Done false
7979 3136 Acked Flush 0 Sent Done Done false
7980 3137 Acked Read 1 Sent Done Done false
7981 3138 Acked Read 7 Sent Done Done false
7982 3139 Acked Write 8 Sent Done Done false
7983 3140 Acked Write 10 Sent Done Done false
7984 3141 Acked Flush 0 Sent Done Done false
7985 3142 Acked Flush 0 Sent Done Done false
7986 3143 Acked Read 3 Sent Done Done false
7987 3144 Acked Write 1 Sent Done Done false
7988 3145 Acked Read 1 Sent Done Done false
7989 3146 Acked Write 7 Sent Done Done false
7990 3147 Acked Write 5 Sent Done Done false
7991 3148 Acked Write 7 Sent Done Done false
7992 3149 Acked Read 6 Sent Done Done false
7993 3150 Acked Read 3 Sent Done Done false
7994 3151 Acked Read 8 Sent Done Done false
7995 3152 Acked Read 5 Sent Done Done false
7996 3153 Acked Read 4 Sent Done Done false
7997 3154 Acked Read 5 Sent Done Done false
7998 3155 Acked Read 6 Sent Done Done false
7999 3156 Acked Read 5 Sent Done Done false
8000 3157 Acked Write 8 Sent Done Done false
8001 3158 Acked Read 1 Sent Done Done false
8002 3159 Acked Write 5 Sent Done Done false
8003 3160 Acked Flush 0 Sent Done Done false
8004 3161 Acked Read 9 Sent Done Done false
8005 3162 Acked Flush 0 Sent Done Done false
8006 3163 Acked Write 4 Sent Done Done false
8007 3164 Acked Write 1 Sent Done Done false
8008 3165 Acked Write 7 Sent Done Done false
8009 3166 Acked Flush 0 Sent Done Done false
8010 3167 Acked Write 7 Sent Done Done false
8011 3168 Acked Write 2 Sent Done Done false
8012 3169 Acked Write 9 Sent Done Done false
8013 3170 Acked Flush 0 Sent Done Done false
8014 3171 Acked Flush 0 Sent Done Done false
8015 3172 Acked Write 7 Sent Done Done false
8016 3173 Acked Write 8 Sent Done Done false
8017 3174 Acked Read 9 Sent Done Done false
8018 3175 Acked Write 7 Sent Done Done false
8019 3176 Acked Read 4 Sent Done Done false
8020 3177 Acked Read 10 Sent Done Done false
8021 3178 Acked Write 9 Sent Done Done false
8022 3179 Acked Read 1 Sent Done Done false
8023 3180 Acked Read 5 Sent Done Done false
8024 3181 Acked Read 5 Sent Done Done false
8025 3182 Acked Read 4 Sent Done Done false
8026 3183 Acked Write 2 Sent Done Done false
8027 3184 Acked Flush 0 Sent Done Done false
8028 3185 Acked Write 8 Sent Done Done false
8029 3186 Acked Write 1 Sent Done Done false
8030 3187 Acked Flush 0 Sent Done Done false
8031 3188 Acked Read 5 Sent Done Done false
8032 3189 Acked Write 10 Sent Done Done false
8033 3190 Acked Read 10 Sent Done Done false
8034 3191 Acked Write 1 Sent Done Done false
8035 3192 Acked Read 6 Sent Done Done false
8036 3193 Acked Flush 0 Sent Done Done false
8037 3194 Acked Write 6 Sent Done Done false
8038 3195 Acked Read 3 Sent Done Done false
8039 3196 Acked Read 5 Sent Done Done false
8040 3197 Acked Flush 0 Sent Done Done false
8041 3198 Acked Flush 0 Sent Done Done false
8042 3199 Acked Read 1 Sent Done Done false
8043 3200 Acked Read 8 Sent Done Done false
8044 3201 Acked Read 10 Sent Done Done false
8045 3202 Acked Read 5 Sent Done Done false
8046 3203 Acked Read 7 Sent Done Done false
8047 3204 Acked Read 5 Sent Done Done false
8048 3205 Acked Read 6 Sent Done Done false
8049 3206 Acked Read 6 Sent Done Done false
8050 3207 Acked Read 10 Sent Done Done false
8051 3208 Acked Read 8 Sent Done Done false
8052 3209 Acked Write 9 Sent Done Done false
8053 3210 Acked Write 9 Sent Done Done false
8054 3211 Acked Read 9 Sent Done Done false
8055 3212 Acked Write 10 Sent Done Done false
8056 3213 Acked Write 6 Sent Done Done false
8057 3214 Acked Read 5 Sent Done Done false
8058 3215 Acked Read 3 Sent Done Done false
8059 3216 Acked Read 4 Sent Done Done false
8060 3217 Acked Read 3 Sent Done Done false
8061 3218 Acked Write 3 Sent Done Done false
8062 3219 Acked Write 5 Sent Done Done false
8063 3220 Acked Read 1 Sent Done Done false
8064 3221 Acked Write 9 Sent Done Done false
8065 3222 Acked Read 8 Sent Done Done false
8066 3223 Acked Write 1 Sent Done Done false
8067 3224 Acked Write 9 Sent Done Done false
8068 3225 Acked Write 8 Sent Done Done false
8069 3226 Acked Write 4 Sent Done Done false
8070 3227 Acked Read 10 Sent Done Done false
8071 3228 Acked Read 8 Sent Done Done false
8072 3229 Acked Write 8 Sent Done Done false
8073 3230 Acked Write 9 Sent Done Done false
8074 3231 Acked Write 8 Sent Done Done false
8075 3232 Acked Flush 0 Sent Done Done false
8076 3233 Acked Read 8 Sent Done Done false
8077 3234 Acked Write 5 Sent Done Done false
8078 3235 Acked Write 5 Sent Done Done false
8079 3236 Acked Read 9 Sent Done Done false
8080 3237 Acked Flush 0 Sent Done Done false
8081 3238 Acked Read 8 Sent Done Done false
8082 3239 Acked Read 5 Sent Done Done false
8083 3240 Acked Write 6 Sent Done Done false
8084 3241 Acked Write 8 Sent Done Done false
8085 3242 Acked Read 10 Sent Done Done false
8086 3243 Acked Read 4 Sent Done Done false
8087 3244 Acked Read 4 Sent Done Done false
8088 3245 Acked Read 10 Sent Done Done false
8089 3246 Acked Read 1 Sent Done Done false
8090 3247 Acked Read 6 Sent Done Done false
8091 3248 Acked Read 2 Sent Done Done false
8092 3249 Acked Write 3 Sent Done Done false
8093 3250 Acked Read 7 Sent Done Done false
8094 3251 Acked Read 5 Sent Done Done false
8095 3252 Acked Write 2 Sent Done Done false
8096 3253 Acked Write 2 Sent Done Done false
8097 3254 Acked Read 3 Sent Done Done false
8098 3255 Acked Write 10 Sent Done Done false
8099 3256 Acked Read 5 Sent Done Done false
8100 3257 Acked Write 10 Sent Done Done false
8101 3258 Acked Read 1 Sent Done Done false
8102 3259 Acked Read 2 Sent Done Done false
8103 3260 Acked Read 8 Sent Done Done false
8104 3261 Acked Read 3 Sent Done Done false
8105 3262 Acked Read 7 Sent Done Done false
8106 3263 Acked Write 5 Sent Done Done false
8107 3264 Acked Read 10 Sent Done Done false
8108 3265 Acked Read 2 Sent Done Done false
8109 3266 Acked Read 3 Sent Done Done false
8110 3267 Acked Write 7 Sent Done Done false
8111 3268 Acked Read 7 Sent Done Done false
8112 3269 Acked Write 1 Sent Done Done false
8113 3270 Acked Write 2 Sent Done Done false
8114 3271 Acked Write 8 Sent Done Done false
8115 3272 Acked Read 10 Sent Done Done false
8116 3273 Acked Read 4 Sent Done Done false
8117 3274 Acked Read 7 Sent Done Done false
8118 3275 Acked Flush 0 Sent Done Done false
8119 3276 Acked Flush 0 Sent Done Done false
8120 3277 Acked Write 10 Sent Done Done false
8121 3278 Acked Read 4 Sent Done Done false
8122 3279 Acked Flush 0 Sent Done Done false
8123 3280 Acked Read 4 Sent Done Done false
8124 3281 Acked Read 2 Sent Done Done false
8125 3282 Acked Read 4 Sent Done Done false
8126 3283 Acked Write 10 Sent Done Done false
8127 3284 Acked Write 7 Sent Done Done false
8128 3285 Acked Write 5 Sent Done Done false
8129 3286 Acked Write 6 Sent Done Done false
8130 3287 Acked Read 8 Sent Done Done false
8131 3288 Acked Read 8 Sent Done Done false
8132 3289 Acked Read 3 Sent Done Done false
8133 3290 Acked Write 6 Sent Done Done false
8134 3291 Acked Write 1 Sent Done Done false
8135 3292 Acked Read 2 Sent Done Done false
8136 3293 Acked Write 2 Sent Done Done false
8137 3294 Acked Write 5 Sent Done Done false
8138 3295 Acked Write 6 Sent Done Done false
8139 3296 Acked Read 9 Sent Done Done false
8140 3297 Acked Read 7 Sent Done Done false
8141 3298 Acked Write 4 Sent Done Done false
8142 3299 Acked Write 10 Sent Done Done false
8143 3300 Acked Read 4 Sent Done Done false
8144 3301 Acked Write 5 Sent Done Done false
8145 3302 Acked Read 5 Sent Done Done false
8146 3303 Acked Read 10 Sent Done Done false
8147 3304 Acked Read 3 Sent Done Done false
8148 3305 Acked Read 4 Sent Done Done false
8149 3306 Acked Write 4 Sent Done Done false
8150 3307 Acked Write 7 Sent Done Done false
8151 3308 Acked Read 2 Sent Done Done false
8152 3309 Acked Write 5 Sent Done Done false
8153 3310 Acked Write 2 Sent Done Done false
8154 3311 Acked Read 5 Sent Done Done false
8155 3312 Acked Write 6 Sent Done Done false
8156 3313 Acked Write 5 Sent Done Done false
8157 3314 Acked Read 7 Sent Done Done false
8158 3315 Acked Write 10 Sent Done Done false
8159 3316 Acked Read 1 Sent Done Done false
8160 3317 Acked Read 6 Sent Done Done false
8161 3318 Acked Read 6 Sent Done Done false
8162 3319 Acked Read 10 Sent Done Done false
8163 3320 Acked Write 1 Sent Done Done false
8164 3321 Acked Read 10 Sent Done Done false
8165 3322 Acked Read 2 Sent Done Done false
8166 3323 Acked Write 1 Sent Done Done false
8167 3324 Acked Read 8 Sent Done Done false
8168 3325 Acked Read 2 Sent Done Done false
8169 3326 Acked Read 4 Sent Done Done false
8170 3327 Acked Flush 0 Sent Done Done false
8171 3328 Acked Flush 0 Sent Done Done false
8172 3329 Acked Write 2 Sent Done Done false
8173 3330 Acked Write 8 Sent Done Done false
8174 3331 Acked Read 6 Sent Done Done false
8175 3332 Acked Read 3 Sent Done Done false
8176 3333 Acked Flush 0 Sent Done Done false
8177 3334 Acked Read 7 Sent Done Done false
8178 3335 Acked Read 9 Sent Done Done false
8179 3336 Acked Write 5 Sent Done Done false
8180 3337 Acked Write 2 Sent Done Done false
8181 3338 Acked Read 8 Sent Done Done false
8182 3339 Acked Read 4 Sent Done Done false
8183 3340 Acked Write 6 Sent Done Done false
8184 3341 Acked Flush 0 Sent Done Done false
8185 3342 Acked Write 1 Sent Done Done false
8186 3343 Acked Read 4 Sent Done Done false
8187 3344 Acked Flush 0 Sent Done Done false
8188 3345 Acked Read 4 Sent Done Done false
8189 3346 Acked Write 2 Sent Done Done false
8190 3347 Acked Read 7 Sent Done Done false
8191 3348 Acked Write 5 Sent Done Done false
8192 3349 Acked Read 2 Sent Done Done false
8193 3350 Acked Read 4 Sent Done Done false
8194 3351 Acked Read 9 Sent Done Done false
8195 3352 Acked Read 9 Sent Done Done false
8196 3353 Acked Write 6 Sent Done Done false
8197 3354 Acked Read 10 Sent Done Done false
8198 3355 Acked Write 3 Sent Done Done false
8199 3356 Acked Flush 0 Sent Done Done false
8200 3357 Acked Write 2 Sent Done Done false
8201 3358 Acked Read 6 Sent Done Done false
8202 3359 Acked Read 5 Sent Done Done false
8203 3360 Acked Flush 0 Sent Done Done false
8204 3361 Acked Read 2 Sent Done Done false
8205 3362 Acked Read 10 Sent Done Done false
8206 3363 Acked Flush 0 Sent Done Done false
8207 3364 Acked Write 9 Sent Done Done false
8208 3365 Acked Read 7 Sent Done Done false
8209 3366 Acked Read 2 Sent Done Done false
8210 3367 Acked Read 10 Sent Done Done false
8211 3368 Acked Write 1 Sent Done Done false
8212 3369 Acked Read 4 Sent Done Done false
8213 3370 Acked Read 9 Sent Done Done false
8214 3371 Acked Read 1 Sent Done Done false
8215 3372 Acked Write 7 Sent Done Done false
8216 3373 Acked Write 6 Sent Done Done false
8217 3374 Acked Read 6 Sent Done Done false
8218 3375 Acked Read 5 Sent Done Done false
8219 3376 Acked Flush 0 Sent Done Done false
8220 3377 Acked Write 4 Sent Done Done false
8221 3378 Acked Read 6 Sent Done Done false
8222 3379 Acked Flush 0 Sent Done Done false
8223 3380 Acked Write 8 Sent Done Done false
8224 3381 Acked Write 2 Sent Done Done false
8225 3382 Acked Flush 0 Sent Done Done false
8226 3383 Acked Read 6 Sent Done Done false
8227 3384 Acked Write 9 Sent Done Done false
8228 3385 Acked Read 1 Sent Done Done false
8229 3386 Acked Write 10 Sent Done Done false
8230 3387 Acked Write 7 Sent Done Done false
8231 3388 Acked Read 1 Sent Done Done false
8232 3389 Acked Write 5 Sent Done Done false
8233 3390 Acked Write 4 Sent Done Done false
8234 3391 Acked Write 10 Sent Done Done false
8235 3392 Acked Read 9 Sent Done Done false
8236 3393 Acked Write 3 Sent Done Done false
8237 3394 Acked Flush 0 Sent Done Done false
8238 3395 Acked Read 3 Sent Done Done false
8239 3396 Acked Flush 0 Sent Done Done false
8240 3397 Acked Read 1 Sent Done Done false
8241 3398 Acked Write 2 Sent Done Done false
8242 3399 Acked Read 8 Sent Done Done false
8243 3400 Acked Read 6 Sent Done Done false
8244 3401 Acked Flush 0 Sent Done Done false
8245 3402 Acked Write 2 Sent Done Done false
8246 3403 Acked Flush 0 Sent Done Done false
8247 3404 Acked Write 7 Sent Done Done false
8248 3405 Acked Read 5 Sent Done Done false
8249 3406 Acked Write 10 Sent Done Done false
8250 3407 Acked Read 1 Sent Done Done false
8251 3408 Acked Read 5 Sent Done Done false
8252 3409 Acked Write 7 Sent Done Done false
8253 3410 Acked Read 10 Sent Done Done false
8254 3411 Acked Write 10 Sent Done Done false
8255 3412 Acked Read 4 Sent Done Done false
8256 3413 Acked Flush 0 Sent Done Done false
8257 STATES DS:0 DS:1 DS:2 TOTAL
8258 Sent 301 0 0 301
8259 Done 0 301 301 602
8260 Skipped 0 0 0 0
8261 Error 0 0 0 0
8262 Last Flush: 3112 3413 3413
8263 Downstairs last five completed: 3112 3111 3110 3109 3108
8264 Upstairs last five completed: 3413 3412 3411 3410 3409
8265 CLIENT: Up:0 ds:301 act:2
8266 [8810] got start action
8267 [8810] I am going to restart
8268 Starting downstairs at port 8810
8269 Make output file at "/tmp/dsc/downstairs-8810.txt"
8270 [8810][0] reports Starting
8271 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1208)
8272 [8810][0] reports Running
8273 ----------------------------------------------------------------
8274 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
8275 States: Offline Active Active
8276 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8277 3113 Acked Read 8 Sent Done Done false
8278 3114 Acked Read 3 Sent Done Done false
8279 3115 Acked Flush 0 Sent Done Done false
8280 3116 Acked Read 8 Sent Done Done false
8281 3117 Acked Read 4 Sent Done Done false
8282 3118 Acked Read 5 Sent Done Done false
8283 3119 Acked Flush 0 Sent Done Done false
8284 3120 Acked Write 8 Sent Done Done false
8285 3121 Acked Write 10 Sent Done Done false
8286 3122 Acked Write 5 Sent Done Done false
8287 3123 Acked Read 1 Sent Done Done false
8288 3124 Acked Read 8 Sent Done Done false
8289 3125 Acked Flush 0 Sent Done Done false
8290 3126 Acked Read 9 Sent Done Done false
8291 3127 Acked Read 8 Sent Done Done false
8292 3128 Acked Read 4 Sent Done Done false
8293 3129 Acked Write 9 Sent Done Done false
8294 3130 Acked Read 1 Sent Done Done false
8295 3131 Acked Write 3 Sent Done Done false
8296 3132 Acked Flush 0 Sent Done Done false
8297 3133 Acked Read 4 Sent Done Done false
8298 3134 Acked Read 1 Sent Done Done false
8299 3135 Acked Read 3 Sent Done Done false
8300 3136 Acked Flush 0 Sent Done Done false
8301 3137 Acked Read 1 Sent Done Done false
8302 3138 Acked Read 7 Sent Done Done false
8303 3139 Acked Write 8 Sent Done Done false
8304 3140 Acked Write 10 Sent Done Done false
8305 3141 Acked Flush 0 Sent Done Done false
8306 3142 Acked Flush 0 Sent Done Done false
8307 3143 Acked Read 3 Sent Done Done false
8308 3144 Acked Write 1 Sent Done Done false
8309 3145 Acked Read 1 Sent Done Done false
8310 3146 Acked Write 7 Sent Done Done false
8311 3147 Acked Write 5 Sent Done Done false
8312 3148 Acked Write 7 Sent Done Done false
8313 3149 Acked Read 6 Sent Done Done false
8314 3150 Acked Read 3 Sent Done Done false
8315 3151 Acked Read 8 Sent Done Done false
8316 3152 Acked Read 5 Sent Done Done false
8317 3153 Acked Read 4 Sent Done Done false
8318 3154 Acked Read 5 Sent Done Done false
8319 3155 Acked Read 6 Sent Done Done false
8320 3156 Acked Read 5 Sent Done Done false
8321 3157 Acked Write 8 Sent Done Done false
8322 3158 Acked Read 1 Sent Done Done false
8323 3159 Acked Write 5 Sent Done Done false
8324 3160 Acked Flush 0 Sent Done Done false
8325 3161 Acked Read 9 Sent Done Done false
8326 3162 Acked Flush 0 Sent Done Done false
8327 3163 Acked Write 4 Sent Done Done false
8328 3164 Acked Write 1 Sent Done Done false
8329 3165 Acked Write 7 Sent Done Done false
8330 3166 Acked Flush 0 Sent Done Done false
8331 3167 Acked Write 7 Sent Done Done false
8332 3168 Acked Write 2 Sent Done Done false
8333 3169 Acked Write 9 Sent Done Done false
8334 3170 Acked Flush 0 Sent Done Done false
8335 3171 Acked Flush 0 Sent Done Done false
8336 3172 Acked Write 7 Sent Done Done false
8337 3173 Acked Write 8 Sent Done Done false
8338 3174 Acked Read 9 Sent Done Done false
8339 3175 Acked Write 7 Sent Done Done false
8340 3176 Acked Read 4 Sent Done Done false
8341 3177 Acked Read 10 Sent Done Done false
8342 3178 Acked Write 9 Sent Done Done false
8343 3179 Acked Read 1 Sent Done Done false
8344 3180 Acked Read 5 Sent Done Done false
8345 3181 Acked Read 5 Sent Done Done false
8346 3182 Acked Read 4 Sent Done Done false
8347 3183 Acked Write 2 Sent Done Done false
8348 3184 Acked Flush 0 Sent Done Done false
8349 3185 Acked Write 8 Sent Done Done false
8350 3186 Acked Write 1 Sent Done Done false
8351 3187 Acked Flush 0 Sent Done Done false
8352 3188 Acked Read 5 Sent Done Done false
8353 3189 Acked Write 10 Sent Done Done false
8354 3190 Acked Read 10 Sent Done Done false
8355 3191 Acked Write 1 Sent Done Done false
8356 3192 Acked Read 6 Sent Done Done false
8357 3193 Acked Flush 0 Sent Done Done false
8358 3194 Acked Write 6 Sent Done Done false
8359 3195 Acked Read 3 Sent Done Done false
8360 3196 Acked Read 5 Sent Done Done false
8361 3197 Acked Flush 0 Sent Done Done false
8362 3198 Acked Flush 0 Sent Done Done false
8363 3199 Acked Read 1 Sent Done Done false
8364 3200 Acked Read 8 Sent Done Done false
8365 3201 Acked Read 10 Sent Done Done false
8366 3202 Acked Read 5 Sent Done Done false
8367 3203 Acked Read 7 Sent Done Done false
8368 3204 Acked Read 5 Sent Done Done false
8369 3205 Acked Read 6 Sent Done Done false
8370 3206 Acked Read 6 Sent Done Done false
8371 3207 Acked Read 10 Sent Done Done false
8372 3208 Acked Read 8 Sent Done Done false
8373 3209 Acked Write 9 Sent Done Done false
8374 3210 Acked Write 9 Sent Done Done false
8375 3211 Acked Read 9 Sent Done Done false
8376 3212 Acked Write 10 Sent Done Done false
8377 3213 Acked Write 6 Sent Done Done false
8378 3214 Acked Read 5 Sent Done Done false
8379 3215 Acked Read 3 Sent Done Done false
8380 3216 Acked Read 4 Sent Done Done false
8381 3217 Acked Read 3 Sent Done Done false
8382 3218 Acked Write 3 Sent Done Done false
8383 3219 Acked Write 5 Sent Done Done false
8384 3220 Acked Read 1 Sent Done Done false
8385 3221 Acked Write 9 Sent Done Done false
8386 3222 Acked Read 8 Sent Done Done false
8387 3223 Acked Write 1 Sent Done Done false
8388 3224 Acked Write 9 Sent Done Done false
8389 3225 Acked Write 8 Sent Done Done false
8390 3226 Acked Write 4 Sent Done Done false
8391 3227 Acked Read 10 Sent Done Done false
8392 3228 Acked Read 8 Sent Done Done false
8393 3229 Acked Write 8 Sent Done Done false
8394 3230 Acked Write 9 Sent Done Done false
8395 3231 Acked Write 8 Sent Done Done false
8396 3232 Acked Flush 0 Sent Done Done false
8397 3233 Acked Read 8 Sent Done Done false
8398 3234 Acked Write 5 Sent Done Done false
8399 3235 Acked Write 5 Sent Done Done false
8400 3236 Acked Read 9 Sent Done Done false
8401 3237 Acked Flush 0 Sent Done Done false
8402 3238 Acked Read 8 Sent Done Done false
8403 3239 Acked Read 5 Sent Done Done false
8404 3240 Acked Write 6 Sent Done Done false
8405 3241 Acked Write 8 Sent Done Done false
8406 3242 Acked Read 10 Sent Done Done false
8407 3243 Acked Read 4 Sent Done Done false
8408 3244 Acked Read 4 Sent Done Done false
8409 3245 Acked Read 10 Sent Done Done false
8410 3246 Acked Read 1 Sent Done Done false
8411 3247 Acked Read 6 Sent Done Done false
8412 3248 Acked Read 2 Sent Done Done false
8413 3249 Acked Write 3 Sent Done Done false
8414 3250 Acked Read 7 Sent Done Done false
8415 3251 Acked Read 5 Sent Done Done false
8416 3252 Acked Write 2 Sent Done Done false
8417 3253 Acked Write 2 Sent Done Done false
8418 3254 Acked Read 3 Sent Done Done false
8419 3255 Acked Write 10 Sent Done Done false
8420 3256 Acked Read 5 Sent Done Done false
8421 3257 Acked Write 10 Sent Done Done false
8422 3258 Acked Read 1 Sent Done Done false
8423 3259 Acked Read 2 Sent Done Done false
8424 3260 Acked Read 8 Sent Done Done false
8425 3261 Acked Read 3 Sent Done Done false
8426 3262 Acked Read 7 Sent Done Done false
8427 3263 Acked Write 5 Sent Done Done false
8428 3264 Acked Read 10 Sent Done Done false
8429 3265 Acked Read 2 Sent Done Done false
8430 3266 Acked Read 3 Sent Done Done false
8431 3267 Acked Write 7 Sent Done Done false
8432 3268 Acked Read 7 Sent Done Done false
8433 3269 Acked Write 1 Sent Done Done false
8434 3270 Acked Write 2 Sent Done Done false
8435 3271 Acked Write 8 Sent Done Done false
8436 3272 Acked Read 10 Sent Done Done false
8437 3273 Acked Read 4 Sent Done Done false
8438 3274 Acked Read 7 Sent Done Done false
8439 3275 Acked Flush 0 Sent Done Done false
8440 3276 Acked Flush 0 Sent Done Done false
8441 3277 Acked Write 10 Sent Done Done false
8442 3278 Acked Read 4 Sent Done Done false
8443 3279 Acked Flush 0 Sent Done Done false
8444 3280 Acked Read 4 Sent Done Done false
8445 3281 Acked Read 2 Sent Done Done false
8446 3282 Acked Read 4 Sent Done Done false
8447 3283 Acked Write 10 Sent Done Done false
8448 3284 Acked Write 7 Sent Done Done false
8449 3285 Acked Write 5 Sent Done Done false
8450 3286 Acked Write 6 Sent Done Done false
8451 3287 Acked Read 8 Sent Done Done false
8452 3288 Acked Read 8 Sent Done Done false
8453 3289 Acked Read 3 Sent Done Done false
8454 3290 Acked Write 6 Sent Done Done false
8455 3291 Acked Write 1 Sent Done Done false
8456 3292 Acked Read 2 Sent Done Done false
8457 3293 Acked Write 2 Sent Done Done false
8458 3294 Acked Write 5 Sent Done Done false
8459 3295 Acked Write 6 Sent Done Done false
8460 3296 Acked Read 9 Sent Done Done false
8461 3297 Acked Read 7 Sent Done Done false
8462 3298 Acked Write 4 Sent Done Done false
8463 3299 Acked Write 10 Sent Done Done false
8464 3300 Acked Read 4 Sent Done Done false
8465 3301 Acked Write 5 Sent Done Done false
8466 3302 Acked Read 5 Sent Done Done false
8467 3303 Acked Read 10 Sent Done Done false
8468 3304 Acked Read 3 Sent Done Done false
8469 3305 Acked Read 4 Sent Done Done false
8470 3306 Acked Write 4 Sent Done Done false
8471 3307 Acked Write 7 Sent Done Done false
8472 3308 Acked Read 2 Sent Done Done false
8473 3309 Acked Write 5 Sent Done Done false
8474 3310 Acked Write 2 Sent Done Done false
8475 3311 Acked Read 5 Sent Done Done false
8476 3312 Acked Write 6 Sent Done Done false
8477 3313 Acked Write 5 Sent Done Done false
8478 3314 Acked Read 7 Sent Done Done false
8479 3315 Acked Write 10 Sent Done Done false
8480 3316 Acked Read 1 Sent Done Done false
8481 3317 Acked Read 6 Sent Done Done false
8482 3318 Acked Read 6 Sent Done Done false
8483 3319 Acked Read 10 Sent Done Done false
8484 3320 Acked Write 1 Sent Done Done false
8485 3321 Acked Read 10 Sent Done Done false
8486 3322 Acked Read 2 Sent Done Done false
8487 3323 Acked Write 1 Sent Done Done false
8488 3324 Acked Read 8 Sent Done Done false
8489 3325 Acked Read 2 Sent Done Done false
8490 3326 Acked Read 4 Sent Done Done false
8491 3327 Acked Flush 0 Sent Done Done false
8492 3328 Acked Flush 0 Sent Done Done false
8493 3329 Acked Write 2 Sent Done Done false
8494 3330 Acked Write 8 Sent Done Done false
8495 3331 Acked Read 6 Sent Done Done false
8496 3332 Acked Read 3 Sent Done Done false
8497 3333 Acked Flush 0 Sent Done Done false
8498 3334 Acked Read 7 Sent Done Done false
8499 3335 Acked Read 9 Sent Done Done false
8500 3336 Acked Write 5 Sent Done Done false
8501 3337 Acked Write 2 Sent Done Done false
8502 3338 Acked Read 8 Sent Done Done false
8503 3339 Acked Read 4 Sent Done Done false
8504 3340 Acked Write 6 Sent Done Done false
8505 3341 Acked Flush 0 Sent Done Done false
8506 3342 Acked Write 1 Sent Done Done false
8507 3343 Acked Read 4 Sent Done Done false
8508 3344 Acked Flush 0 Sent Done Done false
8509 3345 Acked Read 4 Sent Done Done false
8510 3346 Acked Write 2 Sent Done Done false
8511 3347 Acked Read 7 Sent Done Done false
8512 3348 Acked Write 5 Sent Done Done false
8513 3349 Acked Read 2 Sent Done Done false
8514 3350 Acked Read 4 Sent Done Done false
8515 3351 Acked Read 9 Sent Done Done false
8516 3352 Acked Read 9 Sent Done Done false
8517 3353 Acked Write 6 Sent Done Done false
8518 3354 Acked Read 10 Sent Done Done false
8519 3355 Acked Write 3 Sent Done Done false
8520 3356 Acked Flush 0 Sent Done Done false
8521 3357 Acked Write 2 Sent Done Done false
8522 3358 Acked Read 6 Sent Done Done false
8523 3359 Acked Read 5 Sent Done Done false
8524 3360 Acked Flush 0 Sent Done Done false
8525 3361 Acked Read 2 Sent Done Done false
8526 3362 Acked Read 10 Sent Done Done false
8527 3363 Acked Flush 0 Sent Done Done false
8528 3364 Acked Write 9 Sent Done Done false
8529 3365 Acked Read 7 Sent Done Done false
8530 3366 Acked Read 2 Sent Done Done false
8531 3367 Acked Read 10 Sent Done Done false
8532 3368 Acked Write 1 Sent Done Done false
8533 3369 Acked Read 4 Sent Done Done false
8534 3370 Acked Read 9 Sent Done Done false
8535 3371 Acked Read 1 Sent Done Done false
8536 3372 Acked Write 7 Sent Done Done false
8537 3373 Acked Write 6 Sent Done Done false
8538 3374 Acked Read 6 Sent Done Done false
8539 3375 Acked Read 5 Sent Done Done false
8540 3376 Acked Flush 0 Sent Done Done false
8541 3377 Acked Write 4 Sent Done Done false
8542 3378 Acked Read 6 Sent Done Done false
8543 3379 Acked Flush 0 Sent Done Done false
8544 3380 Acked Write 8 Sent Done Done false
8545 3381 Acked Write 2 Sent Done Done false
8546 3382 Acked Flush 0 Sent Done Done false
8547 3383 Acked Read 6 Sent Done Done false
8548 3384 Acked Write 9 Sent Done Done false
8549 3385 Acked Read 1 Sent Done Done false
8550 3386 Acked Write 10 Sent Done Done false
8551 3387 Acked Write 7 Sent Done Done false
8552 3388 Acked Read 1 Sent Done Done false
8553 3389 Acked Write 5 Sent Done Done false
8554 3390 Acked Write 4 Sent Done Done false
8555 3391 Acked Write 10 Sent Done Done false
8556 3392 Acked Read 9 Sent Done Done false
8557 3393 Acked Write 3 Sent Done Done false
8558 3394 Acked Flush 0 Sent Done Done false
8559 3395 Acked Read 3 Sent Done Done false
8560 3396 Acked Flush 0 Sent Done Done false
8561 3397 Acked Read 1 Sent Done Done false
8562 3398 Acked Write 2 Sent Done Done false
8563 3399 Acked Read 8 Sent Done Done false
8564 3400 Acked Read 6 Sent Done Done false
8565 3401 Acked Flush 0 Sent Done Done false
8566 3402 Acked Write 2 Sent Done Done false
8567 3403 Acked Flush 0 Sent Done Done false
8568 3404 Acked Write 7 Sent Done Done false
8569 3405 Acked Read 5 Sent Done Done false
8570 3406 Acked Write 10 Sent Done Done false
8571 3407 Acked Read 1 Sent Done Done false
8572 3408 Acked Read 5 Sent Done Done false
8573 3409 Acked Write 7 Sent Done Done false
8574 3410 Acked Read 10 Sent Done Done false
8575 3411 Acked Write 10 Sent Done Done false
8576 3412 Acked Read 4 Sent Done Done false
8577 3413 Acked Flush 0 Sent Done Done false
8578 STATES DS:0 DS:1 DS:2 TOTAL
8579 Sent 301 0 0 301
8580 Done 0 301 301 602
8581 Skipped 0 0 0 0
8582 Error 0 0 0 0
8583 Last Flush: 3112 3413 3413
8584 Downstairs last five completed: 3112 3111 3110 3109 3108
8585 Upstairs last five completed: 3413 3412 3411 3410 3409
8586 CLIENT: Up:0 ds:301 act:2
85872025-01-17T21:12:54.101ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
85882025-01-17T21:12:54.101ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:45329) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
85892025-01-17T21:12:54.102ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
85902025-01-17T21:12:54.103ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
85912025-01-17T21:12:54.103ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
85922025-01-17T21:12:54.103ZINFOcrucible: send last flush ID to this DS: 3112 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
85932025-01-17T21:12:54.103ZINFOcrucible: Replied this last flush ID: 3112 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
85942025-01-17T21:12:54.103ZINFOcrucible: [0] client re-new 301 jobs since flush 3112 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
85952025-01-17T21:12:54.104ZINFOcrucible: [0] Replayed 301 jobs since flush: 3112 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
8596 ----------------------------------------------------------------
8597 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
8598 Downstairs last five completed: 3413 3412 3411 3410 3409
8599 Upstairs last five completed: 3413 3412 3411 3410 3409
8600 CLIENT: Up:0 ds:0 act:3
8601 Replay: All jobs finished
8602 Main task has work to do, go find it
8603 got dsc Stop(2)
8604 Jan 17 21:12:54.355 INFO stop 2
8605 request completed, latency_us: 313, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 1c3894a3-ef57-41c2-96f2-a72ff91460a7, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
8606 [8830] Got stop action so:false kr:false
8607 Jan 17 21:12:54.356 INFO request completed, latency_us: 140, response_code: 200, uri: /state/cid/2, method: GET, req_id: 03417ba9-0b59-490b-9d2e-dea54e86e865, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
86082025-01-17T21:12:54.356ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
86092025-01-17T21:12:54.356ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
86102025-01-17T21:12:54.356ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
86112025-01-17T21:12:54.356ZINFOcrucible: client task is exiting = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
8612 [8830] Exited with: ExitStatus(unix_wait_status(9))
8613 [8830][2] reports Exit
8614 Jan 17 21:12:58.359 INFO request completed, latency_us: 204, response_code: 200, uri: /state/cid/2, method: GET, req_id: c2b3502d-3e27-4373-84e7-00cc6a3a06d2, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
8615 001/300 Read block 4350 len 20480
8616 002/300 Read block 3004 len 4096
8617 003/300 Write block 200 len 12288 data: 4 4 3
8618 004/300 Write block 3449 len 24576 data: 5 5 4 3 3 2
8619 005/300 Flush
8620 006/300 Write block 1028 len 40960 data: 5 4 4 2 2 2 2 3 3 3
8621 007/300 Write block 426 len 36864 data: 4 4 4 2 2 2 2 2 2
8622 008/300 Write block 4242 len 32768 data: 2 2 2 2 2 2 2 2
8623 009/300 Read block 1456 len 8192
8624 010/300 Write block 1114 len 28672 data: 4 3 3 3 5 4 4
8625 011/300 Flush
8626 012/300 Read block 2519 len 12288
8627 013/300 Read block 510 len 8192
8628 014/300 Write block 3882 len 12288 data: 3 4 4
8629 015/300 Write block 2899 len 28672 data: 3 3 3 3 3 3 2
8630 016/300 Read block 266 len 24576
8631 017/300 Read block 2097 len 24576
8632 018/300 Write block 2152 len 4096 data: 4
8633 019/300 Read block 1483 len 32768
8634 020/300 Read block 2348 len 12288
8635 021/300 Flush
8636 022/300 Read block 2648 len 20480
8637 023/300 Write block 3294 len 36864 data: 4 3 4 4 3 3 3 3 3
8638 024/300 Write block 1775 len 12288 data: 2 2 2
8639 025/300 Write block 1347 len 8192 data: 3 3
8640 026/300 Flush
8641 027/300 Write block 3361 len 12288 data: 4 2 2
8642 028/300 Read block 1584 len 12288
8643 029/300 Write block 2484 len 4096 data: 5
8644 030/300 Read block 2853 len 16384
8645 031/300 Write block 4277 len 32768 data: 4 4 3 3 2 2 2 2
8646 032/300 Read block 2886 len 40960
8647 033/300 Flush
8648 034/300 Write block 4673 len 12288 data: 4 4 5
8649 035/300 Read block 1372 len 40960
8650 036/300 Read block 3221 len 32768
8651 037/300 Write block 4887 len 28672 data: 2 2 2 2 2 2 2
8652 038/300 Read block 1780 len 40960
8653 039/300 Read block 1821 len 36864
8654 040/300 Read block 2814 len 8192
8655 041/300 Flush
8656 042/300 Read block 2091 len 36864
8657 043/300 Read block 1675 len 16384
8658 044/300 Flush
8659 045/300 Write block 2682 len 32768 data: 4 4 3 3 3 2 2 2
8660 046/300 Write block 363 len 28672 data: 3 3 3 3 3 3 2
8661 047/300 Read block 4446 len 20480
8662 048/300 Read block 1923 len 40960
8663 049/300 Flush
8664 050/300 Write block 4021 len 32768 data: 2 3 2 2 3 3 2 2
8665 051/300 Read block 593 len 12288
8666 052/300 Write block 1075 len 28672 data: 2 2 2 2 2 2 2
8667 053/300 Read block 3020 len 8192
8668 054/300 Read block 2601 len 40960
8669 055/300 Write block 2942 len 40960 data: 2 2 3 3 3 3 4 3 2 2
8670 056/300 Read block 13 len 12288
8671 057/300 Write block 3042 len 32768 data: 3 4 2 2 2 2 2 2
8672 058/300 Write block 445 len 28672 data: 2 2 2 2 3 3 3
8673 059/300 Write block 1256 len 28672 data: 3 3 2 2 2 2 2
8674 060/300 Read block 2342 len 16384
8675 061/300 Write block 4310 len 12288 data: 2 2 2
8676 062/300 Read block 1800 len 32768
8677 063/300 Write block 2748 len 36864 data: 3 3 3 3 3 3 3 3 2
8678 064/300 Write block 2489 len 24576 data: 4 5 5 4 3 3
8679 065/300 Read block 4875 len 28672
8680 066/300 Write block 3297 len 32768 data: 5 4 4 4 4 4 3 2
8681 067/300 Read block 138 len 40960
8682 068/300 Write block 1917 len 32768 data: 4 3 2 3 3 3 3 3
8683 069/300 Write block 360 len 20480 data: 3 3 3 4 4
8684 070/300 Read block 3696 len 16384
8685 071/300 Read block 4670 len 32768
8686 072/300 Write block 1438 len 40960 data: 4 3 3 3 4 3 2 2 2 2
8687 073/300 Read block 3912 len 40960
8688 074/300 Read block 3591 len 32768
8689 075/300 Read block 1143 len 24576
8690 076/300 Flush
8691 077/300 Read block 4196 len 16384
8692 078/300 Read block 1431 len 8192
8693 079/300 Read block 659 len 40960
8694 080/300 Write block 4824 len 28672 data: 3 2 2 2 3 2 2
8695 081/300 Write block 3678 len 16384 data: 3 3 3 3
8696 082/300 Read block 2892 len 8192
8697 083/300 Write block 1564 len 16384 data: 3 3 3 3
8698 084/300 Flush
8699 085/300 Write block 2164 len 36864 data: 4 4 4 4 4 5 5 4 4
8700 086/300 Read block 2092 len 28672
8701 087/300 Write block 4841 len 24576 data: 2 2 4 4 4 4
8702 088/300 Read block 4863 len 32768
8703 089/300 Write block 3201 len 20480 data: 5 4 4 4 4
8704 090/300 Read block 3885 len 8192
8705 091/300 Read block 689 len 12288
8706 092/300 Read block 1879 len 40960
8707 093/300 Write block 913 len 16384 data: 3 3 3 3
8708 094/300 Read block 3973 len 32768
8709 095/300 Write block 3320 len 20480 data: 4 3 3 3 3
8710 096/300 Flush
8711 097/300 Flush
8712 098/300 Read block 640 len 24576
8713 099/300 Write block 1721 len 28672 data: 3 2 3 3 3 3 3
8714 100/300 Read block 3852 len 20480
8715 101/300 Read block 283 len 32768
8716 102/300 Write block 4898 len 4096 data: 2
8717 103/300 Write block 3161 len 12288 data: 3 3 3
8718 104/300 Read block 2906 len 32768
8719 105/300 Write block 4481 len 32768 data: 2 2 2 2 2 2 2 2
8720 106/300 Read block 4308 len 16384
8721 107/300 Write block 3063 len 40960 data: 3 3 3 2 2 2 2 2 2 2
8722 108/300 Read block 1695 len 36864
8723 109/300 Write block 274 len 40960 data: 3 3 2 2 2 2 3 3 3 3
8724 110/300 Read block 2398 len 36864
8725 111/300 Read block 4674 len 4096
8726 112/300 Write block 3064 len 4096 data: 4
8727 113/300 Read block 4088 len 16384
8728 114/300 Read block 4415 len 24576
8729 115/300 Write block 3574 len 16384 data: 2 2 2 2
8730 116/300 Read block 4795 len 20480
8731 117/300 Read block 3177 len 24576
8732 118/300 Write block 3707 len 20480 data: 4 4 4 3 4
8733 119/300 Write block 2296 len 12288 data: 3 3 3
8734 120/300 Read block 507 len 40960
8735 121/300 Write block 2353 len 32768 data: 3 3 3 3 3 3 2 3
8736 122/300 Read block 1753 len 36864
8737 123/300 Write block 590 len 40960 data: 3 3 3 2 2 2 2 2 2 2
8738 124/300 Write block 960 len 24576 data: 4 4 5 3 3 3
8739 125/300 Read block 365 len 24576
8740 126/300 Read block 1240 len 32768
8741 127/300 Write block 2142 len 12288 data: 2 2 2
8742 128/300 Read block 1488 len 8192
8743 129/300 Write block 2308 len 20480 data: 4 4 5 5 4
8744 130/300 Read block 4701 len 20480
8745 131/300 Write block 2182 len 16384 data: 2 3 3 3
8746 132/300 Read block 799 len 12288
8747 133/300 Read block 2896 len 20480
8748 134/300 Read block 2094 len 32768
8749 135/300 Read block 1507 len 24576
8750 136/300 Read block 2007 len 36864
8751 137/300 Write block 1355 len 4096 data: 5
8752 138/300 Read block 4293 len 32768
8753 139/300 Read block 1160 len 20480
8754 140/300 Flush
8755 141/300 Flush
8756 142/300 Write block 1081 len 28672 data: 3 3 3 4 4 4 4
8757 143/300 Write block 3000 len 28672 data: 2 2 2 3 3 4 4
8758 144/300 Write block 3328 len 12288 data: 2 2 2
8759 145/300 Write block 4063 len 4096 data: 4
8760 146/300 Read block 1508 len 28672
8761 147/300 Write block 2914 len 12288 data: 3 2 3
8762 148/300 Read block 2799 len 4096
8763 149/300 Read block 1020 len 28672
8764 150/300 Read block 3560 len 40960
8765 151/300 Write block 4065 len 12288 data: 4 4 4
8766 152/300 Read block 4682 len 36864
8767 153/300 Write block 2928 len 32768 data: 3 3 2 2 2 2 2 2
8768 154/300 Read block 2913 len 28672
8769 155/300 Read block 3509 len 20480
8770 156/300 Read block 966 len 20480
8771 157/300 Write block 4310 len 36864 data: 3 3 3 2 2 3 3 4 4
8772 158/300 Read block 1136 len 16384
8773 159/300 Write block 2230 len 40960 data: 5 6 4 3 4 3 3 2 3 3
8774 160/300 Read block 4224 len 4096
8775 161/300 Read block 2878 len 24576
8776 162/300 Write block 3609 len 20480 data: 4 4 5 6 4
8777 163/300 Read block 4989 len 20480
8778 164/300 Read block 1087 len 36864
8779 165/300 Write block 262 len 8192 data: 2 2
8780 166/300 Read block 3439 len 40960
8781 167/300 Write block 650 len 24576 data: 3 3 3 3 3 2
8782 168/300 Read block 3513 len 40960
8783 169/300 Write block 1677 len 16384 data: 3 3 4 4
8784 170/300 Flush
8785 171/300 Read block 2925 len 36864
8786 172/300 Write block 293 len 28672 data: 3 3 3 3 3 4 4
8787 173/300 Flush
8788 174/300 Flush
8789 175/300 Write block 3508 len 36864 data: 2 3 3 3 4 4 4 5 4
8790 176/300 Read block 3700 len 20480
8791 177/300 Write block 683 len 16384 data: 3 4 4 5
8792 178/300 Write block 4330 len 40960 data: 4 4 3 3 3 3 3 3 3 4
8793 179/300 Read block 4931 len 36864
8794 180/300 Write block 204 len 8192 data: 3 4
8795 181/300 Read block 1801 len 16384
8796 182/300 Read block 233 len 24576
8797 183/300 Write block 735 len 8192 data: 4 4
8798 184/300 Write block 3904 len 16384 data: 4 4 3 3
8799 185/300 Read block 2872 len 8192
8800 186/300 Read block 707 len 28672
8801 187/300 Read block 202 len 40960
8802 188/300 Write block 3774 len 36864 data: 3 3 3 2 3 3 3 3 3
8803 189/300 Read block 2406 len 40960
8804 190/300 Read block 2741 len 20480
8805 191/300 Write block 1068 len 16384 data: 3 2 2 2
8806 192/300 Read block 4652 len 12288
8807 193/300 Write block 4413 len 24576 data: 3 2 2 4 4 5
8808 194/300 Write block 3705 len 4096 data: 3
8809 195/300 Flush
8810 196/300 Write block 4844 len 24576 data: 5 5 5 3 4 4
8811 197/300 Read block 1279 len 20480
8812 198/300 Read block 2516 len 4096
8813 199/300 Read block 2179 len 32768
8814 200/300 Write block 2100 len 4096 data: 3
8815 201/300 Flush
8816 202/300 Flush
8817 203/300 Read block 3165 len 12288
8818 204/300 Write block 2058 len 20480 data: 2 2 2 2 3
8819 205/300 Write block 1294 len 28672 data: 2 3 3 3 3 2 2
8820 206/300 Read block 4540 len 28672
8821 207/300 Write block 1568 len 8192 data: 4 3
8822 208/300 Read block 4210 len 40960
8823 209/300 Read block 4170 len 36864
8824 210/300 Read block 873 len 32768
8825 211/300 Write block 138 len 4096 data: 2
8826 212/300 Read block 4584 len 12288
8827 213/300 Flush
8828 214/300 Read block 184 len 32768
8829 215/300 Read block 2198 len 28672
8830 216/300 Read block 1318 len 4096
8831 217/300 Read block 3389 len 12288
8832 218/300 Write block 3616 len 16384 data: 4 4 4 3
8833 219/300 Write block 2652 len 36864 data: 5 5 5 5 4 4 4 3 3
8834 220/300 Write block 180 len 16384 data: 4 3 3 2
8835 221/300 Write block 147 len 28672 data: 4 4 5 3 4 4 4
8836 222/300 Read block 1437 len 24576
8837 223/300 Flush
8838 224/300 Read block 4818 len 24576
8839 225/300 Write block 4607 len 24576 data: 3 3 3 3 3 3
8840 226/300 Write block 2123 len 8192 data: 3 3
8841 227/300 Flush
8842 228/300 Write block 3951 len 12288 data: 3 3 3
8843 229/300 Write block 2962 len 32768 data: 2 2 3 2 2 3 4 4
8844 230/300 Flush
8845 231/300 Read block 2765 len 12288
8846 232/300 Read block 3483 len 20480
8847 233/300 Write block 595 len 4096 data: 3
8848 234/300 Write block 4252 len 12288 data: 3 3 3
8849 235/300 Read block 2787 len 4096
8850 236/300 Read block 1840 len 32768
8851 237/300 Read block 973 len 4096
8852 238/300 Write block 3157 len 12288 data: 3 3 3
8853 239/300 Read block 2282 len 20480
8854 240/300 Read block 2578 len 36864
8855 241/300 Read block 4809 len 28672
8856 242/300 Write block 2104 len 32768 data: 2 2 4 4 4 4 2 2
8857 243/300 Flush
8858 244/300 Read block 3126 len 20480
8859 245/300 Read block 2930 len 40960
8860 246/300 Read block 2412 len 8192
8861 247/300 Write block 1440 len 40960 data: 4 4 5 4 3 3 3 3 2 2
8862 248/300 Write block 3086 len 32768 data: 5 5 3 3 2 2 2 2
8863 249/300 Write block 4778 len 8192 data: 4 6
8864 250/300 Write block 4826 len 32768 data: 3 3 4 3 3 2 2 2
8865 251/300 Read block 2747 len 12288
8866 252/300 Read block 1943 len 28672
8867 253/300 Read block 1873 len 20480
8868 254/300 Read block 3874 len 8192
8869 255/300 Read block 995 len 24576
8870 256/300 Read block 3469 len 24576
8871 257/300 Write block 3842 len 40960 data: 5 5 3 2 2 2 2 2 2 3
8872 258/300 Write block 3175 len 36864 data: 4 4 4 4 4 4 3 2 2
8873 259/300 Flush
8874 260/300 Write block 4644 len 4096 data: 2
8875 261/300 Flush
8876 262/300 Flush
8877 263/300 Read block 1684 len 16384
8878 264/300 Read block 20 len 16384
8879 265/300 Read block 3201 len 8192
8880 266/300 Write block 2328 len 36864 data: 3 4 4 4 4 4 2 2 2
8881 267/300 Read block 3587 len 20480
8882 268/300 Write block 4343 len 8192 data: 2 2
8883 269/300 Read block 1825 len 24576
8884 270/300 Write block 617 len 28672 data: 3 3 4 3 3 3 2
8885 271/300 Flush
8886 272/300 Flush
8887 273/300 Write block 4424 len 36864 data: 3 3 2 2 2 3 2 2 2
8888 274/300 Read block 2782 len 12288
8889 275/300 Write block 3527 len 8192 data: 4 3
8890 276/300 Read block 2861 len 28672
8891 277/300 Read block 4131 len 8192
8892 278/300 Read block 1360 len 20480
8893 279/300 Write block 2816 len 12288 data: 4 3 2
8894 280/300 Read block 2565 len 24576
8895 281/300 Write block 1102 len 36864 data: 3 3 3 2 2 2 4 4 3
8896 282/300 Write block 2935 len 8192 data: 3 2
8897 283/300 Flush
8898 284/300 Flush
8899 285/300 Write block 190 len 32768 data: 3 2 2 2 2 4 4 4
8900 286/300 Read block 2914 len 28672
8901 287/300 Read block 4810 len 4096
8902 288/300 Read block 4097 len 28672
8903 289/300 Read block 651 len 32768
8904 290/300 Read block 3159 len 20480
8905 291/300 Read block 3976 len 24576
8906 292/300 Write block 1096 len 36864 data: 4 4 4 4 3 3 4 4 4
8907 293/300 Read block 3990 len 24576
8908 294/300 Read block 3382 len 24576
8909 295/300 Write block 744 len 36864 data: 2 2 2 2 2 2 2 2 2
8910 296/300 Flush
8911 297/300 Write block 462 len 12288 data: 3 3 3
8912 298/300 Read block 807 len 36864
8913 299/300 Read block 4095 len 8192
8914 300/300 Write block 3682 len 36864 data: 3 2 2 2 2 2 2 2 2
8915 301/300 Write block 2938 len 32768 data: 2 2 2 2 3 3 4 4
8916 Main task has work to do, go find it
8917 got dsc Start(2)
8918 start 2
8919 Jan 17 21:12:58.609 INFO request completed, latency_us: 219, response_code: 204, uri: /start/cid/2, method: POST, req_id: 9d172699-47b8-4172-ae0d-57534e2d5120, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
8920 Replay: started 2, returned:Ok(())
8921 ----------------------------------------------------------------
8922 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
8923 States: Active Active Offline
8924 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8925 3414 Acked Read 5 Done Done Sent false
8926 3415 Acked Read 1 Done Done Sent false
8927 3416 Acked Write 3 Done Done Sent false
8928 3417 Acked Write 6 Done Done Sent false
8929 3418 Acked Flush 0 Done Done Sent false
8930 3419 Acked Write 10 Done Done Sent false
8931 3420 Acked Write 9 Done Done Sent false
8932 3421 Acked Write 8 Done Done Sent false
8933 3422 Acked Read 2 Done Done Sent false
8934 3423 Acked Write 7 Done Done Sent false
8935 3424 Acked Flush 0 Done Done Sent false
8936 3425 Acked Read 3 Done Done Sent false
8937 3426 Acked Read 2 Done Done Sent false
8938 3427 Acked Write 3 Done Done Sent false
8939 3428 Acked Write 7 Done Done Sent false
8940 3429 Acked Read 6 Done Done Sent false
8941 3430 Acked Read 6 Done Done Sent false
8942 3431 Acked Write 1 Done Done Sent false
8943 3432 Acked Read 8 Done Done Sent false
8944 3433 Acked Read 3 Done Done Sent false
8945 3434 Acked Flush 0 Done Done Sent false
8946 3435 Acked Read 5 Done Done Sent false
8947 3436 Acked Write 9 Done Done Sent false
8948 3437 Acked Write 3 Done Done Sent false
8949 3438 Acked Write 2 Done Done Sent false
8950 3439 Acked Flush 0 Done Done Sent false
8951 3440 Acked Write 3 Done Done Sent false
8952 3441 Acked Read 3 Done Done Sent false
8953 3442 Acked Write 1 Done Done Sent false
8954 3443 Acked Read 4 Done Done Sent false
8955 3444 Acked Write 8 Done Done Sent false
8956 3445 Acked Read 10 Done Done Sent false
8957 3446 Acked Flush 0 Done Done Sent false
8958 3447 Acked Write 3 Done Done Sent false
8959 3448 Acked Read 10 Done Done Sent false
8960 3449 Acked Read 8 Done Done Sent false
8961 3450 Acked Write 7 Done Done Sent false
8962 3451 Acked Read 10 Done Done Sent false
8963 3452 Acked Read 9 Done Done Sent false
8964 3453 Acked Read 2 Done Done Sent false
8965 3454 Acked Flush 0 Done Done Sent false
8966 3455 Acked Read 9 Done Done Sent false
8967 3456 Acked Read 4 Done Done Sent false
8968 3457 Acked Flush 0 Done Done Sent false
8969 3458 Acked Write 8 Done Done Sent false
8970 3459 Acked Write 7 Done Done Sent false
8971 3460 Acked Read 5 Done Done Sent false
8972 3461 Acked Read 10 Done Done Sent false
8973 3462 Acked Flush 0 Done Done Sent false
8974 3463 Acked Write 8 Done Done Sent false
8975 3464 Acked Read 3 Done Done Sent false
8976 3465 Acked Write 7 Done Done Sent false
8977 3466 Acked Read 2 Done Done Sent false
8978 3467 Acked Read 10 Done Done Sent false
8979 3468 Acked Write 10 Done Done Sent false
8980 3469 Acked Read 3 Done Done Sent false
8981 3470 Acked Write 8 Done Done Sent false
8982 3471 Acked Write 7 Done Done Sent false
8983 3472 Acked Write 7 Done Done Sent false
8984 3473 Acked Read 4 Done Done Sent false
8985 3474 Acked Write 3 Done Done Sent false
8986 3475 Acked Read 8 Done Done Sent false
8987 3476 Acked Write 9 Done Done Sent false
8988 3477 Acked Write 6 Done Done Sent false
8989 3478 Acked Read 7 Done Done Sent false
8990 3479 Acked Write 8 Done Done Sent false
8991 3480 Acked Read 10 Done Done Sent false
8992 3481 Acked Write 8 Done Done Sent false
8993 3482 Acked Write 5 Done Done Sent false
8994 3483 Acked Read 4 Done Done Sent false
8995 3484 Acked Read 8 Done Done Sent false
8996 3485 Acked Write 10 Done Done Sent false
8997 3486 Acked Read 10 Done Done Sent false
8998 3487 Acked Read 8 Done Done Sent false
8999 3488 Acked Read 6 Done Done Sent false
9000 3489 Acked Flush 0 Done Done Sent false
9001 3490 Acked Read 4 Done Done Sent false
9002 3491 Acked Read 2 Done Done Sent false
9003 3492 Acked Read 10 Done Done Sent false
9004 3493 Acked Write 7 Done Done Sent false
9005 3494 Acked Write 4 Done Done Sent false
9006 3495 Acked Read 2 Done Done Sent false
9007 3496 Acked Write 4 Done Done Sent false
9008 3497 Acked Flush 0 Done Done Sent false
9009 3498 Acked Write 9 Done Done Sent false
9010 3499 Acked Read 7 Done Done Sent false
9011 3500 Acked Write 6 Done Done Sent false
9012 3501 Acked Read 8 Done Done Sent false
9013 3502 Acked Write 5 Done Done Sent false
9014 3503 Acked Read 2 Done Done Sent false
9015 3504 Acked Read 3 Done Done Sent false
9016 3505 Acked Read 10 Done Done Sent false
9017 3506 Acked Write 4 Done Done Sent false
9018 3507 Acked Read 8 Done Done Sent false
9019 3508 Acked Write 5 Done Done Sent false
9020 3509 Acked Flush 0 Done Done Sent false
9021 3510 Acked Flush 0 Done Done Sent false
9022 3511 Acked Read 6 Done Done Sent false
9023 3512 Acked Write 7 Done Done Sent false
9024 3513 Acked Read 5 Done Done Sent false
9025 3514 Acked Read 8 Done Done Sent false
9026 3515 Acked Write 1 Done Done Sent false
9027 3516 Acked Write 3 Done Done Sent false
9028 3517 Acked Read 8 Done Done Sent false
9029 3518 Acked Write 8 Done Done Sent false
9030 3519 Acked Read 4 Done Done Sent false
9031 3520 Acked Write 10 Done Done Sent false
9032 3521 Acked Read 9 Done Done Sent false
9033 3522 Acked Write 10 Done Done Sent false
9034 3523 Acked Read 9 Done Done Sent false
9035 3524 Acked Read 1 Done Done Sent false
9036 3525 Acked Write 1 Done Done Sent false
9037 3526 Acked Read 4 Done Done Sent false
9038 3527 Acked Read 6 Done Done Sent false
9039 3528 Acked Write 4 Done Done Sent false
9040 3529 Acked Read 5 Done Done Sent false
9041 3530 Acked Read 6 Done Done Sent false
9042 3531 Acked Write 5 Done Done Sent false
9043 3532 Acked Write 3 Done Done Sent false
9044 3533 Acked Read 10 Done Done Sent false
9045 3534 Acked Write 8 Done Done Sent false
9046 3535 Acked Read 9 Done Done Sent false
9047 3536 Acked Write 10 Done Done Sent false
9048 3537 Acked Write 6 Done Done Sent false
9049 3538 Acked Read 6 Done Done Sent false
9050 3539 Acked Read 8 Done Done Sent false
9051 3540 Acked Write 3 Done Done Sent false
9052 3541 Acked Read 2 Done Done Sent false
9053 3542 Acked Write 5 Done Done Sent false
9054 3543 Acked Read 5 Done Done Sent false
9055 3544 Acked Write 4 Done Done Sent false
9056 3545 Acked Read 3 Done Done Sent false
9057 3546 Acked Read 5 Done Done Sent false
9058 3547 Acked Read 8 Done Done Sent false
9059 3548 Acked Read 6 Done Done Sent false
9060 3549 Acked Read 9 Done Done Sent false
9061 3550 Acked Write 1 Done Done Sent false
9062 3551 Acked Read 8 Done Done Sent false
9063 3552 Acked Read 5 Done Done Sent false
9064 3553 Acked Flush 0 Done Done Sent false
9065 3554 Acked Flush 0 Done Done Sent false
9066 3555 Acked Write 7 Done Done Sent false
9067 3556 Acked Write 7 Done Done Sent false
9068 3557 Acked Write 3 Done Done Sent false
9069 3558 Acked Write 1 Done Done Sent false
9070 3559 Acked Read 7 Done Done Sent false
9071 3560 Acked Write 3 Done Done Sent false
9072 3561 Acked Read 1 Done Done Sent false
9073 3562 Acked Read 7 Done Done Sent false
9074 3563 Acked Read 10 Done Done Sent false
9075 3564 Acked Write 3 Done Done Sent false
9076 3565 Acked Read 9 Done Done Sent false
9077 3566 Acked Write 8 Done Done Sent false
9078 3567 Acked Read 7 Done Done Sent false
9079 3568 Acked Read 5 Done Done Sent false
9080 3569 Acked Read 5 Done Done Sent false
9081 3570 Acked Write 9 Done Done Sent false
9082 3571 Acked Read 4 Done Done Sent false
9083 3572 Acked Write 10 Done Done Sent false
9084 3573 Acked Read 1 Done Done Sent false
9085 3574 Acked Read 6 Done Done Sent false
9086 3575 Acked Write 5 Done Done Sent false
9087 3576 Acked Read 5 Done Done Sent false
9088 3577 Acked Read 9 Done Done Sent false
9089 3578 Acked Write 2 Done Done Sent false
9090 3579 Acked Read 10 Done Done Sent false
9091 3580 Acked Write 6 Done Done Sent false
9092 3581 Acked Read 10 Done Done Sent false
9093 3582 Acked Write 4 Done Done Sent false
9094 3583 Acked Flush 0 Done Done Sent false
9095 3584 Acked Read 9 Done Done Sent false
9096 3585 Acked Write 7 Done Done Sent false
9097 3586 Acked Flush 0 Done Done Sent false
9098 3587 Acked Flush 0 Done Done Sent false
9099 3588 Acked Write 9 Done Done Sent false
9100 3589 Acked Read 5 Done Done Sent false
9101 3590 Acked Write 4 Done Done Sent false
9102 3591 Acked Write 10 Done Done Sent false
9103 3592 Acked Read 9 Done Done Sent false
9104 3593 Acked Write 2 Done Done Sent false
9105 3594 Acked Read 4 Done Done Sent false
9106 3595 Acked Read 6 Done Done Sent false
9107 3596 Acked Write 2 Done Done Sent false
9108 3597 Acked Write 4 Done Done Sent false
9109 3598 Acked Read 2 Done Done Sent false
9110 3599 Acked Read 7 Done Done Sent false
9111 3600 Acked Read 10 Done Done Sent false
9112 3601 Acked Write 9 Done Done Sent false
9113 3602 Acked Read 10 Done Done Sent false
9114 3603 Acked Read 5 Done Done Sent false
9115 3604 Acked Write 4 Done Done Sent false
9116 3605 Acked Read 3 Done Done Sent false
9117 3606 Acked Write 6 Done Done Sent false
9118 3607 Acked Write 1 Done Done Sent false
9119 3608 Acked Flush 0 Done Done Sent false
9120 3609 Acked Write 6 Done Done Sent false
9121 3610 Acked Read 5 Done Done Sent false
9122 3611 Acked Read 1 Done Done Sent false
9123 3612 Acked Read 8 Done Done Sent false
9124 3613 Acked Write 1 Done Done Sent false
9125 3614 Acked Flush 0 Done Done Sent false
9126 3615 Acked Flush 0 Done Done Sent false
9127 3616 Acked Read 3 Done Done Sent false
9128 3617 Acked Write 5 Done Done Sent false
9129 3618 Acked Write 7 Done Done Sent false
9130 3619 Acked Read 7 Done Done Sent false
9131 3620 Acked Write 2 Done Done Sent false
9132 3621 Acked Read 10 Done Done Sent false
9133 3622 Acked Read 9 Done Done Sent false
9134 3623 Acked Read 8 Done Done Sent false
9135 3624 Acked Write 1 Done Done Sent false
9136 3625 Acked Read 3 Done Done Sent false
9137 3626 Acked Flush 0 Done Done Sent false
9138 3627 Acked Read 8 Done Done Sent false
9139 3628 Acked Read 7 Done Done Sent false
9140 3629 Acked Read 1 Done Done Sent false
9141 3630 Acked Read 3 Done Done Sent false
9142 3631 Acked Write 4 Done Done Sent false
9143 3632 Acked Write 9 Done Done Sent false
9144 3633 Acked Write 4 Done Done Sent false
9145 3634 Acked Write 7 Done Done Sent false
9146 3635 Acked Read 6 Done Done Sent false
9147 3636 Acked Flush 0 Done Done Sent false
9148 3637 Acked Read 6 Done Done Sent false
9149 3638 Acked Write 6 Done Done Sent false
9150 3639 Acked Write 2 Done Done Sent false
9151 3640 Acked Flush 0 Done Done Sent false
9152 3641 Acked Write 3 Done Done Sent false
9153 3642 Acked Write 8 Done Done Sent false
9154 3643 Acked Flush 0 Done Done Sent false
9155 3644 Acked Read 3 Done Done Sent false
9156 3645 Acked Read 5 Done Done Sent false
9157 3646 Acked Write 1 Done Done Sent false
9158 3647 Acked Write 3 Done Done Sent false
9159 3648 Acked Read 1 Done Done Sent false
9160 3649 Acked Read 8 Done Done Sent false
9161 3650 Acked Read 1 Done Done Sent false
9162 3651 Acked Write 3 Done Done Sent false
9163 3652 Acked Read 5 Done Done Sent false
9164 3653 Acked Read 9 Done Done Sent false
9165 3654 Acked Read 7 Done Done Sent false
9166 3655 Acked Write 8 Done Done Sent false
9167 3656 Acked Flush 0 Done Done Sent false
9168 3657 Acked Read 5 Done Done Sent false
9169 3658 Acked Read 10 Done Done Sent false
9170 3659 Acked Read 2 Done Done Sent false
9171 3660 Acked Write 10 Done Done Sent false
9172 3661 Acked Write 8 Done Done Sent false
9173 3662 Acked Write 2 Done Done Sent false
9174 3663 Acked Write 8 Done Done Sent false
9175 3664 Acked Read 3 Done Done Sent false
9176 3665 Acked Read 7 Done Done Sent false
9177 3666 Acked Read 5 Done Done Sent false
9178 3667 Acked Read 2 Done Done Sent false
9179 3668 Acked Read 6 Done Done Sent false
9180 3669 Acked Read 6 Done Done Sent false
9181 3670 Acked Write 10 Done Done Sent false
9182 3671 Acked Write 9 Done Done Sent false
9183 3672 Acked Flush 0 Done Done Sent false
9184 3673 Acked Write 1 Done Done Sent false
9185 3674 Acked Flush 0 Done Done Sent false
9186 3675 Acked Flush 0 Done Done Sent false
9187 3676 Acked Read 4 Done Done Sent false
9188 3677 Acked Read 4 Done Done Sent false
9189 3678 Acked Read 2 Done Done Sent false
9190 3679 Acked Write 9 Done Done Sent false
9191 3680 Acked Read 5 Done Done Sent false
9192 3681 Acked Write 2 Done Done Sent false
9193 3682 Acked Read 6 Done Done Sent false
9194 3683 Acked Write 7 Done Done Sent false
9195 3684 Acked Flush 0 Done Done Sent false
9196 3685 Acked Flush 0 Done Done Sent false
9197 3686 Acked Write 9 Done Done Sent false
9198 3687 Acked Read 3 Done Done Sent false
9199 3688 Acked Write 2 Done Done Sent false
9200 3689 Acked Read 7 Done Done Sent false
9201 3690 Acked Read 2 Done Done Sent false
9202 3691 Acked Read 5 Done Done Sent false
9203 3692 Acked Write 3 Done Done Sent false
9204 3693 Acked Read 6 Done Done Sent false
9205 3694 Acked Write 9 Done Done Sent false
9206 3695 Acked Write 2 Done Done Sent false
9207 3696 Acked Flush 0 Done Done Sent false
9208 3697 Acked Flush 0 Done Done Sent false
9209 3698 Acked Write 8 Done Done Sent false
9210 3699 Acked Read 7 Done Done Sent false
9211 3700 Acked Read 1 Done Done Sent false
9212 3701 Acked Read 7 Done Done Sent false
9213 3702 Acked Read 8 Done Done Sent false
9214 3703 Acked Read 5 Done Done Sent false
9215 3704 Acked Read 6 Done Done Sent false
9216 3705 Acked Write 9 Done Done Sent false
9217 3706 Acked Read 6 Done Done Sent false
9218 3707 Acked Read 6 Done Done Sent false
9219 3708 Acked Write 9 Done Done Sent false
9220 3709 Acked Flush 0 Done Done Sent false
9221 3710 Acked Write 3 Done Done Sent false
9222 3711 Acked Read 9 Done Done Sent false
9223 3712 Acked Read 2 Done Done Sent false
9224 3713 Acked Write 9 Done Done Sent false
9225 3714 Acked Write 8 Done Done Sent false
9226 STATES DS:0 DS:1 DS:2 TOTAL
9227 Sent 0 0 301 301
9228 Done 301 301 0 602
9229 Skipped 0 0 0 0
9230 Error 0 0 0 0
9231 Last Flush: 3709 3709 3413
9232 Downstairs last five completed: 3413 3412 3411 3410 3409
9233 Upstairs last five completed: 3714 3713 3712 3711 3710
9234 CLIENT: Up:0 ds:301 act:2
9235 [8830] got start action
9236 [8830] I am going to restart
9237 Starting downstairs at port 8830
9238 Make output file at "/tmp/dsc/downstairs-8830.txt"
9239 [8830][2] reports Starting
9240 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1209)
9241 [8830][2] reports Running
9242 ----------------------------------------------------------------
9243 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
9244 States: Active Active Offline
9245 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9246 3414 Acked Read 5 Done Done Sent false
9247 3415 Acked Read 1 Done Done Sent false
9248 3416 Acked Write 3 Done Done Sent false
9249 3417 Acked Write 6 Done Done Sent false
9250 3418 Acked Flush 0 Done Done Sent false
9251 3419 Acked Write 10 Done Done Sent false
9252 3420 Acked Write 9 Done Done Sent false
9253 3421 Acked Write 8 Done Done Sent false
9254 3422 Acked Read 2 Done Done Sent false
9255 3423 Acked Write 7 Done Done Sent false
9256 3424 Acked Flush 0 Done Done Sent false
9257 3425 Acked Read 3 Done Done Sent false
9258 3426 Acked Read 2 Done Done Sent false
9259 3427 Acked Write 3 Done Done Sent false
9260 3428 Acked Write 7 Done Done Sent false
9261 3429 Acked Read 6 Done Done Sent false
9262 3430 Acked Read 6 Done Done Sent false
9263 3431 Acked Write 1 Done Done Sent false
9264 3432 Acked Read 8 Done Done Sent false
9265 3433 Acked Read 3 Done Done Sent false
9266 3434 Acked Flush 0 Done Done Sent false
9267 3435 Acked Read 5 Done Done Sent false
9268 3436 Acked Write 9 Done Done Sent false
9269 3437 Acked Write 3 Done Done Sent false
9270 3438 Acked Write 2 Done Done Sent false
9271 3439 Acked Flush 0 Done Done Sent false
9272 3440 Acked Write 3 Done Done Sent false
9273 3441 Acked Read 3 Done Done Sent false
9274 3442 Acked Write 1 Done Done Sent false
9275 3443 Acked Read 4 Done Done Sent false
9276 3444 Acked Write 8 Done Done Sent false
9277 3445 Acked Read 10 Done Done Sent false
9278 3446 Acked Flush 0 Done Done Sent false
9279 3447 Acked Write 3 Done Done Sent false
9280 3448 Acked Read 10 Done Done Sent false
9281 3449 Acked Read 8 Done Done Sent false
9282 3450 Acked Write 7 Done Done Sent false
9283 3451 Acked Read 10 Done Done Sent false
9284 3452 Acked Read 9 Done Done Sent false
9285 3453 Acked Read 2 Done Done Sent false
9286 3454 Acked Flush 0 Done Done Sent false
9287 3455 Acked Read 9 Done Done Sent false
9288 3456 Acked Read 4 Done Done Sent false
9289 3457 Acked Flush 0 Done Done Sent false
9290 3458 Acked Write 8 Done Done Sent false
9291 3459 Acked Write 7 Done Done Sent false
9292 3460 Acked Read 5 Done Done Sent false
9293 3461 Acked Read 10 Done Done Sent false
9294 3462 Acked Flush 0 Done Done Sent false
9295 3463 Acked Write 8 Done Done Sent false
9296 3464 Acked Read 3 Done Done Sent false
9297 3465 Acked Write 7 Done Done Sent false
9298 3466 Acked Read 2 Done Done Sent false
9299 3467 Acked Read 10 Done Done Sent false
9300 3468 Acked Write 10 Done Done Sent false
9301 3469 Acked Read 3 Done Done Sent false
9302 3470 Acked Write 8 Done Done Sent false
9303 3471 Acked Write 7 Done Done Sent false
9304 3472 Acked Write 7 Done Done Sent false
9305 3473 Acked Read 4 Done Done Sent false
9306 3474 Acked Write 3 Done Done Sent false
9307 3475 Acked Read 8 Done Done Sent false
9308 3476 Acked Write 9 Done Done Sent false
9309 3477 Acked Write 6 Done Done Sent false
9310 3478 Acked Read 7 Done Done Sent false
9311 3479 Acked Write 8 Done Done Sent false
9312 3480 Acked Read 10 Done Done Sent false
9313 3481 Acked Write 8 Done Done Sent false
9314 3482 Acked Write 5 Done Done Sent false
9315 3483 Acked Read 4 Done Done Sent false
9316 3484 Acked Read 8 Done Done Sent false
9317 3485 Acked Write 10 Done Done Sent false
9318 3486 Acked Read 10 Done Done Sent false
9319 3487 Acked Read 8 Done Done Sent false
9320 3488 Acked Read 6 Done Done Sent false
9321 3489 Acked Flush 0 Done Done Sent false
9322 3490 Acked Read 4 Done Done Sent false
9323 3491 Acked Read 2 Done Done Sent false
9324 3492 Acked Read 10 Done Done Sent false
9325 3493 Acked Write 7 Done Done Sent false
9326 3494 Acked Write 4 Done Done Sent false
9327 3495 Acked Read 2 Done Done Sent false
9328 3496 Acked Write 4 Done Done Sent false
9329 3497 Acked Flush 0 Done Done Sent false
9330 3498 Acked Write 9 Done Done Sent false
9331 3499 Acked Read 7 Done Done Sent false
9332 3500 Acked Write 6 Done Done Sent false
9333 3501 Acked Read 8 Done Done Sent false
9334 3502 Acked Write 5 Done Done Sent false
9335 3503 Acked Read 2 Done Done Sent false
9336 3504 Acked Read 3 Done Done Sent false
9337 3505 Acked Read 10 Done Done Sent false
9338 3506 Acked Write 4 Done Done Sent false
9339 3507 Acked Read 8 Done Done Sent false
9340 3508 Acked Write 5 Done Done Sent false
9341 3509 Acked Flush 0 Done Done Sent false
9342 3510 Acked Flush 0 Done Done Sent false
9343 3511 Acked Read 6 Done Done Sent false
9344 3512 Acked Write 7 Done Done Sent false
9345 3513 Acked Read 5 Done Done Sent false
9346 3514 Acked Read 8 Done Done Sent false
9347 3515 Acked Write 1 Done Done Sent false
9348 3516 Acked Write 3 Done Done Sent false
9349 3517 Acked Read 8 Done Done Sent false
9350 3518 Acked Write 8 Done Done Sent false
9351 3519 Acked Read 4 Done Done Sent false
9352 3520 Acked Write 10 Done Done Sent false
9353 3521 Acked Read 9 Done Done Sent false
9354 3522 Acked Write 10 Done Done Sent false
9355 3523 Acked Read 9 Done Done Sent false
9356 3524 Acked Read 1 Done Done Sent false
9357 3525 Acked Write 1 Done Done Sent false
9358 3526 Acked Read 4 Done Done Sent false
9359 3527 Acked Read 6 Done Done Sent false
9360 3528 Acked Write 4 Done Done Sent false
9361 3529 Acked Read 5 Done Done Sent false
9362 3530 Acked Read 6 Done Done Sent false
9363 3531 Acked Write 5 Done Done Sent false
9364 3532 Acked Write 3 Done Done Sent false
9365 3533 Acked Read 10 Done Done Sent false
9366 3534 Acked Write 8 Done Done Sent false
9367 3535 Acked Read 9 Done Done Sent false
9368 3536 Acked Write 10 Done Done Sent false
9369 3537 Acked Write 6 Done Done Sent false
9370 3538 Acked Read 6 Done Done Sent false
9371 3539 Acked Read 8 Done Done Sent false
9372 3540 Acked Write 3 Done Done Sent false
9373 3541 Acked Read 2 Done Done Sent false
9374 3542 Acked Write 5 Done Done Sent false
9375 3543 Acked Read 5 Done Done Sent false
9376 3544 Acked Write 4 Done Done Sent false
9377 3545 Acked Read 3 Done Done Sent false
9378 3546 Acked Read 5 Done Done Sent false
9379 3547 Acked Read 8 Done Done Sent false
9380 3548 Acked Read 6 Done Done Sent false
9381 3549 Acked Read 9 Done Done Sent false
9382 3550 Acked Write 1 Done Done Sent false
9383 3551 Acked Read 8 Done Done Sent false
9384 3552 Acked Read 5 Done Done Sent false
9385 3553 Acked Flush 0 Done Done Sent false
9386 3554 Acked Flush 0 Done Done Sent false
9387 3555 Acked Write 7 Done Done Sent false
9388 3556 Acked Write 7 Done Done Sent false
9389 3557 Acked Write 3 Done Done Sent false
9390 3558 Acked Write 1 Done Done Sent false
9391 3559 Acked Read 7 Done Done Sent false
9392 3560 Acked Write 3 Done Done Sent false
9393 3561 Acked Read 1 Done Done Sent false
9394 3562 Acked Read 7 Done Done Sent false
9395 3563 Acked Read 10 Done Done Sent false
9396 3564 Acked Write 3 Done Done Sent false
9397 3565 Acked Read 9 Done Done Sent false
9398 3566 Acked Write 8 Done Done Sent false
9399 3567 Acked Read 7 Done Done Sent false
9400 3568 Acked Read 5 Done Done Sent false
9401 3569 Acked Read 5 Done Done Sent false
9402 3570 Acked Write 9 Done Done Sent false
9403 3571 Acked Read 4 Done Done Sent false
9404 3572 Acked Write 10 Done Done Sent false
9405 3573 Acked Read 1 Done Done Sent false
9406 3574 Acked Read 6 Done Done Sent false
9407 3575 Acked Write 5 Done Done Sent false
9408 3576 Acked Read 5 Done Done Sent false
9409 3577 Acked Read 9 Done Done Sent false
9410 3578 Acked Write 2 Done Done Sent false
9411 3579 Acked Read 10 Done Done Sent false
9412 3580 Acked Write 6 Done Done Sent false
9413 3581 Acked Read 10 Done Done Sent false
9414 3582 Acked Write 4 Done Done Sent false
9415 3583 Acked Flush 0 Done Done Sent false
9416 3584 Acked Read 9 Done Done Sent false
9417 3585 Acked Write 7 Done Done Sent false
9418 3586 Acked Flush 0 Done Done Sent false
9419 3587 Acked Flush 0 Done Done Sent false
9420 3588 Acked Write 9 Done Done Sent false
9421 3589 Acked Read 5 Done Done Sent false
9422 3590 Acked Write 4 Done Done Sent false
9423 3591 Acked Write 10 Done Done Sent false
9424 3592 Acked Read 9 Done Done Sent false
9425 3593 Acked Write 2 Done Done Sent false
9426 3594 Acked Read 4 Done Done Sent false
9427 3595 Acked Read 6 Done Done Sent false
9428 3596 Acked Write 2 Done Done Sent false
9429 3597 Acked Write 4 Done Done Sent false
9430 3598 Acked Read 2 Done Done Sent false
9431 3599 Acked Read 7 Done Done Sent false
9432 3600 Acked Read 10 Done Done Sent false
9433 3601 Acked Write 9 Done Done Sent false
9434 3602 Acked Read 10 Done Done Sent false
9435 3603 Acked Read 5 Done Done Sent false
9436 3604 Acked Write 4 Done Done Sent false
9437 3605 Acked Read 3 Done Done Sent false
9438 3606 Acked Write 6 Done Done Sent false
9439 3607 Acked Write 1 Done Done Sent false
9440 3608 Acked Flush 0 Done Done Sent false
9441 3609 Acked Write 6 Done Done Sent false
9442 3610 Acked Read 5 Done Done Sent false
9443 3611 Acked Read 1 Done Done Sent false
9444 3612 Acked Read 8 Done Done Sent false
9445 3613 Acked Write 1 Done Done Sent false
9446 3614 Acked Flush 0 Done Done Sent false
9447 3615 Acked Flush 0 Done Done Sent false
9448 3616 Acked Read 3 Done Done Sent false
9449 3617 Acked Write 5 Done Done Sent false
9450 3618 Acked Write 7 Done Done Sent false
9451 3619 Acked Read 7 Done Done Sent false
9452 3620 Acked Write 2 Done Done Sent false
9453 3621 Acked Read 10 Done Done Sent false
9454 3622 Acked Read 9 Done Done Sent false
9455 3623 Acked Read 8 Done Done Sent false
9456 3624 Acked Write 1 Done Done Sent false
9457 3625 Acked Read 3 Done Done Sent false
9458 3626 Acked Flush 0 Done Done Sent false
9459 3627 Acked Read 8 Done Done Sent false
9460 3628 Acked Read 7 Done Done Sent false
9461 3629 Acked Read 1 Done Done Sent false
9462 3630 Acked Read 3 Done Done Sent false
9463 3631 Acked Write 4 Done Done Sent false
9464 3632 Acked Write 9 Done Done Sent false
9465 3633 Acked Write 4 Done Done Sent false
9466 3634 Acked Write 7 Done Done Sent false
9467 3635 Acked Read 6 Done Done Sent false
9468 3636 Acked Flush 0 Done Done Sent false
9469 3637 Acked Read 6 Done Done Sent false
9470 3638 Acked Write 6 Done Done Sent false
9471 3639 Acked Write 2 Done Done Sent false
9472 3640 Acked Flush 0 Done Done Sent false
9473 3641 Acked Write 3 Done Done Sent false
9474 3642 Acked Write 8 Done Done Sent false
9475 3643 Acked Flush 0 Done Done Sent false
9476 3644 Acked Read 3 Done Done Sent false
9477 3645 Acked Read 5 Done Done Sent false
9478 3646 Acked Write 1 Done Done Sent false
9479 3647 Acked Write 3 Done Done Sent false
9480 3648 Acked Read 1 Done Done Sent false
9481 3649 Acked Read 8 Done Done Sent false
9482 3650 Acked Read 1 Done Done Sent false
9483 3651 Acked Write 3 Done Done Sent false
9484 3652 Acked Read 5 Done Done Sent false
9485 3653 Acked Read 9 Done Done Sent false
9486 3654 Acked Read 7 Done Done Sent false
9487 3655 Acked Write 8 Done Done Sent false
9488 3656 Acked Flush 0 Done Done Sent false
9489 3657 Acked Read 5 Done Done Sent false
9490 3658 Acked Read 10 Done Done Sent false
9491 3659 Acked Read 2 Done Done Sent false
9492 3660 Acked Write 10 Done Done Sent false
9493 3661 Acked Write 8 Done Done Sent false
9494 3662 Acked Write 2 Done Done Sent false
9495 3663 Acked Write 8 Done Done Sent false
9496 3664 Acked Read 3 Done Done Sent false
9497 3665 Acked Read 7 Done Done Sent false
9498 3666 Acked Read 5 Done Done Sent false
9499 3667 Acked Read 2 Done Done Sent false
9500 3668 Acked Read 6 Done Done Sent false
9501 3669 Acked Read 6 Done Done Sent false
9502 3670 Acked Write 10 Done Done Sent false
9503 3671 Acked Write 9 Done Done Sent false
9504 3672 Acked Flush 0 Done Done Sent false
9505 3673 Acked Write 1 Done Done Sent false
9506 3674 Acked Flush 0 Done Done Sent false
9507 3675 Acked Flush 0 Done Done Sent false
9508 3676 Acked Read 4 Done Done Sent false
9509 3677 Acked Read 4 Done Done Sent false
9510 3678 Acked Read 2 Done Done Sent false
9511 3679 Acked Write 9 Done Done Sent false
9512 3680 Acked Read 5 Done Done Sent false
9513 3681 Acked Write 2 Done Done Sent false
9514 3682 Acked Read 6 Done Done Sent false
9515 3683 Acked Write 7 Done Done Sent false
9516 3684 Acked Flush 0 Done Done Sent false
9517 3685 Acked Flush 0 Done Done Sent false
9518 3686 Acked Write 9 Done Done Sent false
9519 3687 Acked Read 3 Done Done Sent false
9520 3688 Acked Write 2 Done Done Sent false
9521 3689 Acked Read 7 Done Done Sent false
9522 3690 Acked Read 2 Done Done Sent false
9523 3691 Acked Read 5 Done Done Sent false
9524 3692 Acked Write 3 Done Done Sent false
9525 3693 Acked Read 6 Done Done Sent false
9526 3694 Acked Write 9 Done Done Sent false
9527 3695 Acked Write 2 Done Done Sent false
9528 3696 Acked Flush 0 Done Done Sent false
9529 3697 Acked Flush 0 Done Done Sent false
9530 3698 Acked Write 8 Done Done Sent false
9531 3699 Acked Read 7 Done Done Sent false
9532 3700 Acked Read 1 Done Done Sent false
9533 3701 Acked Read 7 Done Done Sent false
9534 3702 Acked Read 8 Done Done Sent false
9535 3703 Acked Read 5 Done Done Sent false
9536 3704 Acked Read 6 Done Done Sent false
9537 3705 Acked Write 9 Done Done Sent false
9538 3706 Acked Read 6 Done Done Sent false
9539 3707 Acked Read 6 Done Done Sent false
9540 3708 Acked Write 9 Done Done Sent false
9541 3709 Acked Flush 0 Done Done Sent false
9542 3710 Acked Write 3 Done Done Sent false
9543 3711 Acked Read 9 Done Done Sent false
9544 3712 Acked Read 2 Done Done Sent false
9545 3713 Acked Write 9 Done Done Sent false
9546 3714 Acked Write 8 Done Done Sent false
9547 3715 Acked Flush 0 Done Done Sent false
9548 STATES DS:0 DS:1 DS:2 TOTAL
9549 Sent 0 0 302 302
9550 Done 302 302 0 604
9551 Skipped 0 0 0 0
9552 Error 0 0 0 0
9553 Last Flush: 3715 3715 3413
9554 Downstairs last five completed: 3413 3412 3411 3410 3409
9555 Upstairs last five completed: 3715 3714 3713 3712 3711
9556 CLIENT: Up:0 ds:302 act:2
95572025-01-17T21:13:04.358ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
95582025-01-17T21:13:04.358ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:60828) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
95592025-01-17T21:13:04.359ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
95602025-01-17T21:13:04.360ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
95612025-01-17T21:13:04.360ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
95622025-01-17T21:13:04.360ZINFOcrucible: send last flush ID to this DS: 3413 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
95632025-01-17T21:13:04.360ZINFOcrucible: Replied this last flush ID: 3413 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
95642025-01-17T21:13:04.360ZINFOcrucible: [2] client re-new 302 jobs since flush 3413 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
95652025-01-17T21:13:04.360ZINFOcrucible: [2] Replayed 302 jobs since flush: 3413 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
9566 ----------------------------------------------------------------
9567 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
9568 Downstairs last five completed: 3715 3714 3713 3712 3711
9569 Upstairs last five completed: 3715 3714 3713 3712 3711
9570 CLIENT: Up:0 ds:0 act:3
9571 Replay: All jobs finished
9572 Main task has work to do, go find it
9573 got dsc Stop(0)
9574 stop 0
9575 Jan 17 21:13:06.615 INFO request completed, latency_us: 312, response_code: 204, uri: /stop/cid/0, method: POST, req_id: e86cac85-5b7c-42ee-af72-a0881dd647c3, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
9576 [8810] Got stop action so:false kr:false
9577 Jan 17 21:13:06.615 INFO request completed, latency_us: 132, response_code: 200, uri: /state/cid/0, method: GET, req_id: 1e5c0914-b43b-48f0-8edb-35377a027e78, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
95782025-01-17T21:13:06.616ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
95792025-01-17T21:13:06.616ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
95802025-01-17T21:13:06.616ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
95812025-01-17T21:13:06.616ZINFOcrucible: client task is exiting = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
9582 [8810] Exited with: ExitStatus(unix_wait_status(9))
9583 [8810][0] reports Exit
9584 Jan 17 21:13:10.617 INFO request completed, latency_us: 216, response_code: 200, uri: /state/cid/0, method: GET, req_id: e4b7451b-50fd-4361-b1f3-f5d300a9323f, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
9585 001/300 Read block 4554 len 36864
9586 002/300 Read block 703 len 32768
9587 003/300 Write block 2682 len 16384 data: 5 5 4 4
9588 004/300 Read block 2865 len 8192
9589 005/300 Read block 4367 len 20480
9590 006/300 Write block 2537 len 24576 data: 4 4 4 4 4 4
9591 007/300 Write block 299 len 12288 data: 5 3 3
9592 008/300 Flush
9593 009/300 Read block 3499 len 24576
9594 010/300 Write block 3005 len 40960 data: 5 5 4 4 3 4 3 3 3 3
9595 011/300 Write block 3419 len 4096 data: 3
9596 012/300 Write block 101 len 4096 data: 5
9597 013/300 Read block 937 len 40960
9598 014/300 Write block 845 len 12288 data: 2 2 2
9599 015/300 Write block 2167 len 28672 data: 5 5 6 6 5 5 5
9600 016/300 Read block 2797 len 32768
9601 017/300 Flush
9602 018/300 Write block 1007 len 20480 data: 2 2 2 2 2
9603 019/300 Write block 2714 len 40960 data: 2 2 2 2 2 2 2 2 2 2
9604 020/300 Read block 2521 len 36864
9605 021/300 Read block 2815 len 16384
9606 022/300 Read block 3708 len 4096
9607 023/300 Flush
9608 024/300 Read block 1277 len 28672
9609 025/300 Read block 2655 len 8192
9610 026/300 Flush
9611 027/300 Read block 1764 len 20480
9612 028/300 Read block 2153 len 12288
9613 029/300 Write block 2695 len 8192 data: 2 2
9614 030/300 Read block 1352 len 12288
9615 031/300 Write block 171 len 36864 data: 3 4 5 5 5 5 4 4 4
9616 032/300 Read block 3391 len 36864
9617 033/300 Flush
9618 034/300 Flush
9619 035/300 Write block 1159 len 16384 data: 5 4 3 2
9620 036/300 Read block 672 len 16384
9621 037/300 Read block 1364 len 24576
9622 038/300 Read block 1755 len 8192
9623 039/300 Read block 3713 len 20480
9624 040/300 Read block 872 len 12288
9625 041/300 Flush
9626 042/300 Write block 585 len 36864 data: 4 5 4 3 3 4 4 4 3
9627 043/300 Read block 1977 len 8192
9628 044/300 Write block 2351 len 8192 data: 5 3
9629 045/300 Read block 981 len 24576
9630 046/300 Write block 440 len 20480 data: 3 3 3 2 2
9631 047/300 Write block 842 len 4096 data: 4
9632 048/300 Flush
9633 049/300 Write block 3346 len 4096 data: 2
9634 050/300 Read block 478 len 16384
9635 051/300 Read block 2187 len 20480
9636 052/300 Read block 2053 len 36864
9637 053/300 Read block 1211 len 16384
9638 054/300 Write block 1475 len 4096 data: 3
9639 055/300 Write block 3459 len 40960 data: 2 2 2 2 2 2 3 3 3 3
9640 056/300 Read block 4617 len 8192
9641 057/300 Read block 3922 len 36864
9642 058/300 Flush
9643 059/300 Write block 4259 len 28672 data: 2 2 4 4 4 2 3
9644 060/300 Write block 391 len 16384 data: 6 6 6 5
9645 061/300 Read block 1567 len 24576
9646 062/300 Read block 3360 len 32768
9647 063/300 Read block 4685 len 40960
9648 064/300 Read block 3554 len 32768
9649 065/300 Read block 3941 len 16384
9650 066/300 Read block 4776 len 24576
9651 067/300 Read block 1459 len 32768
9652 068/300 Read block 2988 len 16384
9653 069/300 Read block 1270 len 28672
9654 070/300 Write block 4709 len 4096 data: 2
9655 071/300 Write block 2370 len 8192 data: 2 3
9656 072/300 Write block 1942 len 16384 data: 3 3 3 2
9657 073/300 Read block 341 len 16384
9658 074/300 Write block 1136 len 4096 data: 4
9659 075/300 Write block 3295 len 40960 data: 4 5 6 5 5 5 5 5 4 3
9660 076/300 Flush
9661 077/300 Read block 282 len 12288
9662 078/300 Read block 4419 len 24576
9663 079/300 Write block 4033 len 4096 data: 3
9664 080/300 Write block 4472 len 20480 data: 3 3 3 3 3
9665 081/300 Write block 4127 len 36864 data: 2 2 2 2 2 2 2 2 2
9666 082/300 Read block 3938 len 28672
9667 083/300 Read block 4813 len 32768
9668 084/300 Read block 2451 len 8192
9669 085/300 Read block 3104 len 36864
9670 086/300 Read block 685 len 36864
9671 087/300 Write block 891 len 4096 data: 2
9672 088/300 Read block 3999 len 4096
9673 089/300 Write block 1149 len 40960 data: 4 3 3 3 3 3 4 5 6 5
9674 090/300 Read block 4518 len 12288
9675 091/300 Write block 612 len 4096 data: 3
9676 092/300 Write block 2603 len 16384 data: 3 3 3 3
9677 093/300 Write block 1394 len 32768 data: 2 3 3 4 4 4 3 3
9678 094/300 Read block 1266 len 4096
9679 095/300 Write block 2870 len 36864 data: 4 3 4 3 3 3 3 3 3
9680 096/300 Read block 2182 len 20480
9681 097/300 Read block 1318 len 36864
9682 098/300 Write block 3537 len 8192 data: 3 3
9683 099/300 Write block 2164 len 28672 data: 5 5 5 6 6 7 7
9684 100/300 Write block 1461 len 32768 data: 3 3 2 2 2 2 2 2
9685 101/300 Read block 1612 len 40960
9686 102/300 Read block 2123 len 20480
9687 103/300 Write block 915 len 24576 data: 4 4 3 4 5 5
9688 104/300 Write block 3313 len 28672 data: 2 3 3 4 4 4 5
9689 105/300 Read block 1253 len 20480
9690 106/300 Write block 469 len 36864 data: 3 3 3 3 4 4 3 2 2
9691 107/300 Write block 2037 len 8192 data: 3 3
9692 108/300 Read block 4236 len 28672
9693 109/300 Read block 2062 len 4096
9694 110/300 Write block 4759 len 32768 data: 3 3 3 5 4 4 4 4
9695 111/300 Read block 1238 len 8192
9696 112/300 Write block 3703 len 24576 data: 2 3 4 4 5 5
9697 113/300 Flush
9698 114/300 Write block 3486 len 40960 data: 2 2 2 2 2 2 2 2 2 3
9699 115/300 Read block 3401 len 12288
9700 116/300 Read block 759 len 16384
9701 117/300 Flush
9702 118/300 Write block 2415 len 12288 data: 2 2 2
9703 119/300 Read block 630 len 4096
9704 120/300 Read block 4064 len 20480
9705 121/300 Read block 3727 len 28672
9706 122/300 Read block 4338 len 8192
9707 123/300 Write block 405 len 28672 data: 4 3 3 3 3 3 3
9708 124/300 Read block 3271 len 8192
9709 125/300 Write block 1978 len 8192 data: 3 3
9710 126/300 Read block 2710 len 20480
9711 127/300 Read block 3600 len 36864
9712 128/300 Read block 3860 len 28672
9713 129/300 Flush
9714 130/300 Read block 718 len 12288
9715 131/300 Read block 2343 len 32768
9716 132/300 Write block 419 len 24576 data: 3 3 4 6 7 6
9717 133/300 Read block 4760 len 8192
9718 134/300 Read block 1466 len 24576
9719 135/300 Read block 2070 len 20480
9720 136/300 Write block 4921 len 40960 data: 2 3 3 3 3 3 2 2 2 2
9721 137/300 Write block 206 len 40960 data: 5 4 4 4 4 4 4 4 2 2
9722 138/300 Write block 134 len 28672 data: 3 2 2 2 3 2 2
9723 139/300 Read block 1871 len 8192
9724 140/300 Write block 1938 len 8192 data: 2 2
9725 141/300 Read block 1101 len 8192
9726 142/300 Read block 4922 len 40960
9727 143/300 Read block 3350 len 28672
9728 144/300 Read block 2461 len 16384
9729 145/300 Write block 1277 len 12288 data: 2 2 2
9730 146/300 Read block 2787 len 8192
9731 147/300 Read block 4984 len 4096
9732 148/300 Read block 3443 len 16384
9733 149/300 Write block 3192 len 24576 data: 5 4 4 4 4 4
9734 150/300 Flush
9735 151/300 Write block 1626 len 4096 data: 2
9736 152/300 Read block 2920 len 20480
9737 153/300 Read block 4088 len 40960
9738 154/300 Write block 2361 len 8192 data: 2 2
9739 155/300 Read block 3472 len 8192
9740 156/300 Read block 1227 len 32768
9741 157/300 Write block 3689 len 28672 data: 3 3 2 2 2 2 2
9742 158/300 Write block 1154 len 40960 data: 4 5 6 7 6 6 5 4 3 2
9743 159/300 Write block 2818 len 4096 data: 3
9744 160/300 Read block 3857 len 40960
9745 161/300 Write block 4921 len 40960 data: 3 4 4 4 4 4 3 3 3 3
9746 162/300 Read block 950 len 32768
9747 163/300 Flush
9748 164/300 Flush
9749 165/300 Write block 206 len 8192 data: 6 5
9750 166/300 Read block 501 len 32768
9751 167/300 Write block 804 len 12288 data: 3 3 3
9752 168/300 Flush
9753 169/300 Write block 943 len 36864 data: 4 4 4 4 3 3 3 3 4
9754 170/300 Flush
9755 171/300 Write block 1280 len 32768 data: 2 2 2 2 2 2 2 2
9756 172/300 Read block 347 len 20480
9757 173/300 Read block 4521 len 40960
9758 174/300 Read block 1733 len 12288
9759 175/300 Write block 2856 len 32768 data: 2 2 2 2 2 2 2 2
9760 176/300 Write block 1738 len 8192 data: 3 3
9761 177/300 Write block 3757 len 4096 data: 2
9762 178/300 Read block 500 len 36864
9763 179/300 Write block 3715 len 32768 data: 3 3 3 2 2 2 2 2
9764 180/300 Flush
9765 181/300 Read block 4640 len 12288
9766 182/300 Read block 2346 len 28672
9767 183/300 Write block 4553 len 36864 data: 3 3 2 3 3 2 2 2 2
9768 184/300 Read block 4859 len 8192
9769 185/300 Read block 3450 len 32768
9770 186/300 Read block 610 len 12288
9771 187/300 Write block 3203 len 4096 data: 5
9772 188/300 Flush
9773 189/300 Read block 1680 len 4096
9774 190/300 Flush
9775 191/300 Read block 4935 len 16384
9776 192/300 Read block 4751 len 12288
9777 193/300 Read block 2748 len 28672
9778 194/300 Read block 2249 len 12288
9779 195/300 Write block 4453 len 12288 data: 3 4 4
9780 196/300 Read block 3694 len 36864
9781 197/300 Read block 1450 len 24576
9782 198/300 Write block 4620 len 8192 data: 2 3
9783 199/300 Read block 1755 len 20480
9784 200/300 Read block 4006 len 12288
9785 201/300 Flush
9786 202/300 Write block 4233 len 28672 data: 2 2 4 4 4 3 3
9787 203/300 Write block 1380 len 20480 data: 4 4 4 5 4
9788 204/300 Read block 4542 len 8192
9789 205/300 Write block 2350 len 12288 data: 5 6 4
9790 206/300 Write block 195 len 12288 data: 5 5 5
9791 207/300 Read block 1649 len 20480
9792 208/300 Flush
9793 209/300 Write block 3392 len 4096 data: 3
9794 210/300 Read block 2466 len 28672
9795 211/300 Read block 4301 len 4096
9796 212/300 Flush
9797 213/300 Read block 4065 len 20480
9798 214/300 Write block 4701 len 16384 data: 3 4 4 5
9799 215/300 Write block 3250 len 12288 data: 2 2 2
9800 216/300 Read block 3532 len 24576
9801 217/300 Flush
9802 218/300 Read block 1241 len 28672
9803 219/300 Read block 1891 len 16384
9804 220/300 Write block 3724 len 16384 data: 2 2 3 3
9805 221/300 Read block 3215 len 16384
9806 222/300 Write block 1688 len 20480 data: 3 2 2 2 2
9807 223/300 Read block 460 len 32768
9808 224/300 Read block 1504 len 36864
9809 225/300 Read block 4909 len 16384
9810 226/300 Write block 3814 len 12288 data: 4 4 3
9811 227/300 Write block 436 len 12288 data: 3 3 3
9812 228/300 Read block 70 len 36864
9813 229/300 Write block 507 len 32768 data: 2 2 2 3 3 3 3 3
9814 230/300 Read block 3296 len 24576
9815 231/300 Read block 3659 len 12288
9816 232/300 Read block 4612 len 28672
9817 233/300 Read block 1521 len 36864
9818 234/300 Flush
9819 235/300 Flush
9820 236/300 Read block 4426 len 36864
9821 237/300 Read block 1840 len 12288
9822 238/300 Read block 2837 len 16384
9823 239/300 Write block 3251 len 32768 data: 3 3 2 3 3 3 3 3
9824 240/300 Read block 1189 len 32768
9825 241/300 Read block 79 len 28672
9826 242/300 Read block 3390 len 36864
9827 243/300 Write block 3876 len 24576 data: 5 4 4 2 2 3
9828 244/300 Read block 2734 len 40960
9829 245/300 Read block 2069 len 32768
9830 246/300 Write block 3842 len 12288 data: 6 6 4
9831 247/300 Flush
9832 248/300 Read block 1989 len 24576
9833 249/300 Read block 1936 len 36864
9834 250/300 Read block 3579 len 24576
9835 251/300 Read block 1146 len 40960
9836 252/300 Read block 2847 len 28672
9837 253/300 Flush
9838 254/300 Read block 2208 len 24576
9839 255/300 Read block 4071 len 20480
9840 256/300 Write block 733 len 8192 data: 3 4
9841 257/300 Write block 4397 len 24576 data: 5 4 4 3 3 3
9842 258/300 Read block 3813 len 4096
9843 259/300 Read block 474 len 36864
9844 260/300 Write block 2300 len 8192 data: 3 2
9845 261/300 Flush
9846 262/300 Flush
9847 263/300 Read block 1000 len 32768
9848 264/300 Flush
9849 265/300 Write block 2330 len 36864 data: 5 5 5 5 3 3 3 2 2
9850 266/300 Write block 1646 len 40960 data: 2 2 2 2 2 2 2 2 2 2
9851 267/300 Write block 170 len 4096 data: 2
9852 268/300 Read block 2791 len 36864
9853 269/300 Flush
9854 270/300 Flush
9855 271/300 Read block 2529 len 36864
9856 272/300 Read block 493 len 28672
9857 273/300 Read block 4954 len 36864
9858 274/300 Write block 2389 len 36864 data: 3 3 3 3 4 4 4 5 5
9859 275/300 Read block 2676 len 20480
9860 276/300 Read block 4330 len 24576
9861 277/300 Read block 870 len 40960
9862 278/300 Flush
9863 279/300 Write block 2889 len 12288 data: 4 4 4
9864 280/300 Read block 4679 len 28672
9865 281/300 Write block 2727 len 28672 data: 4 4 4 4 4 3 2
9866 282/300 Write block 3277 len 4096 data: 2
9867 283/300 Write block 4006 len 32768 data: 5 5 5 5 3 4 3 3
9868 284/300 Flush
9869 285/300 Read block 4826 len 24576
9870 286/300 Write block 2968 len 24576 data: 5 5 3 3 3 3
9871 287/300 Read block 1090 len 24576
9872 288/300 Read block 4932 len 28672
9873 289/300 Flush
9874 290/300 Write block 4637 len 16384 data: 4 4 4 3
9875 291/300 Write block 761 len 24576 data: 3 3 3 3 3 4
9876 292/300 Read block 3840 len 8192
9877 293/300 Write block 457 len 16384 data: 3 3 2 2
9878 294/300 Read block 1012 len 32768
9879 295/300 Write block 1976 len 12288 data: 3 3 4
9880 296/300 Read block 2289 len 24576
9881 297/300 Read block 1469 len 24576
9882 298/300 Write block 1992 len 4096 data: 3
9883 299/300 Read block 724 len 16384
9884 300/300 Read block 2590 len 12288
9885 301/300 Write block 2086 len 36864 data: 5 5 5 4 4 3 3 2 2
9886 Main task has work to do, go find it
9887 got dsc Start(0)
9888 start 0
9889 Jan 17 21:13:10.862 INFO request completed, latency_us: 257, response_code: 204, uri: /start/cid/0, method: POST, req_id: 2a8881cd-d731-4cd6-be70-de7cf545f9a8, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
9890 Replay: started 0, returned:Ok(())
9891 ----------------------------------------------------------------
9892 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
9893 States: Offline Active Active
9894 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9895 3716 Acked Read 9 Sent Done Done false
9896 3717 Acked Read 8 Sent Done Done false
9897 3718 Acked Write 4 Sent Done Done false
9898 3719 Acked Read 2 Sent Done Done false
9899 3720 Acked Read 5 Sent Done Done false
9900 3721 Acked Write 6 Sent Done Done false
9901 3722 Acked Write 3 Sent Done Done false
9902 3723 Acked Flush 0 Sent Done Done false
9903 3724 Acked Read 6 Sent Done Done false
9904 3725 Acked Write 10 Sent Done Done false
9905 3726 Acked Write 1 Sent Done Done false
9906 3727 Acked Write 1 Sent Done Done false
9907 3728 Acked Read 10 Sent Done Done false
9908 3729 Acked Write 3 Sent Done Done false
9909 3730 Acked Write 7 Sent Done Done false
9910 3731 Acked Read 8 Sent Done Done false
9911 3732 Acked Flush 0 Sent Done Done false
9912 3733 Acked Write 5 Sent Done Done false
9913 3734 Acked Write 10 Sent Done Done false
9914 3735 Acked Read 9 Sent Done Done false
9915 3736 Acked Read 4 Sent Done Done false
9916 3737 Acked Read 1 Sent Done Done false
9917 3738 Acked Flush 0 Sent Done Done false
9918 3739 Acked Read 7 Sent Done Done false
9919 3740 Acked Read 2 Sent Done Done false
9920 3741 Acked Flush 0 Sent Done Done false
9921 3742 Acked Read 5 Sent Done Done false
9922 3743 Acked Read 3 Sent Done Done false
9923 3744 Acked Write 2 Sent Done Done false
9924 3745 Acked Read 3 Sent Done Done false
9925 3746 Acked Write 9 Sent Done Done false
9926 3747 Acked Read 9 Sent Done Done false
9927 3748 Acked Flush 0 Sent Done Done false
9928 3749 Acked Flush 0 Sent Done Done false
9929 3750 Acked Write 4 Sent Done Done false
9930 3751 Acked Read 4 Sent Done Done false
9931 3752 Acked Read 6 Sent Done Done false
9932 3753 Acked Read 2 Sent Done Done false
9933 3754 Acked Read 5 Sent Done Done false
9934 3755 Acked Read 3 Sent Done Done false
9935 3756 Acked Flush 0 Sent Done Done false
9936 3757 Acked Write 9 Sent Done Done false
9937 3758 Acked Read 2 Sent Done Done false
9938 3759 Acked Write 2 Sent Done Done false
9939 3760 Acked Read 6 Sent Done Done false
9940 3761 Acked Write 5 Sent Done Done false
9941 3762 Acked Write 1 Sent Done Done false
9942 3763 Acked Flush 0 Sent Done Done false
9943 3764 Acked Write 1 Sent Done Done false
9944 3765 Acked Read 4 Sent Done Done false
9945 3766 Acked Read 5 Sent Done Done false
9946 3767 Acked Read 9 Sent Done Done false
9947 3768 Acked Read 4 Sent Done Done false
9948 3769 Acked Write 1 Sent Done Done false
9949 3770 Acked Write 10 Sent Done Done false
9950 3771 Acked Read 2 Sent Done Done false
9951 3772 Acked Read 9 Sent Done Done false
9952 3773 Acked Flush 0 Sent Done Done false
9953 3774 Acked Write 7 Sent Done Done false
9954 3775 Acked Write 4 Sent Done Done false
9955 3776 Acked Read 6 Sent Done Done false
9956 3777 Acked Read 8 Sent Done Done false
9957 3778 Acked Read 10 Sent Done Done false
9958 3779 Acked Read 8 Sent Done Done false
9959 3780 Acked Read 4 Sent Done Done false
9960 3781 Acked Read 6 Sent Done Done false
9961 3782 Acked Read 8 Sent Done Done false
9962 3783 Acked Read 4 Sent Done Done false
9963 3784 Acked Read 7 Sent Done Done false
9964 3785 Acked Write 1 Sent Done Done false
9965 3786 Acked Write 2 Sent Done Done false
9966 3787 Acked Write 4 Sent Done Done false
9967 3788 Acked Read 4 Sent Done Done false
9968 3789 Acked Write 1 Sent Done Done false
9969 3790 Acked Write 10 Sent Done Done false
9970 3791 Acked Flush 0 Sent Done Done false
9971 3792 Acked Read 3 Sent Done Done false
9972 3793 Acked Read 6 Sent Done Done false
9973 3794 Acked Write 1 Sent Done Done false
9974 3795 Acked Write 5 Sent Done Done false
9975 3796 Acked Write 9 Sent Done Done false
9976 3797 Acked Read 7 Sent Done Done false
9977 3798 Acked Read 8 Sent Done Done false
9978 3799 Acked Read 2 Sent Done Done false
9979 3800 Acked Read 9 Sent Done Done false
9980 3801 Acked Read 9 Sent Done Done false
9981 3802 Acked Write 1 Sent Done Done false
9982 3803 Acked Read 1 Sent Done Done false
9983 3804 Acked Write 10 Sent Done Done false
9984 3805 Acked Read 3 Sent Done Done false
9985 3806 Acked Write 1 Sent Done Done false
9986 3807 Acked Write 4 Sent Done Done false
9987 3808 Acked Write 8 Sent Done Done false
9988 3809 Acked Read 1 Sent Done Done false
9989 3810 Acked Write 9 Sent Done Done false
9990 3811 Acked Read 5 Sent Done Done false
9991 3812 Acked Read 9 Sent Done Done false
9992 3813 Acked Write 2 Sent Done Done false
9993 3814 Acked Write 7 Sent Done Done false
9994 3815 Acked Write 8 Sent Done Done false
9995 3816 Acked Read 10 Sent Done Done false
9996 3817 Acked Read 5 Sent Done Done false
9997 3818 Acked Write 6 Sent Done Done false
9998 3819 Acked Write 7 Sent Done Done false
9999 3820 Acked Read 5 Sent Done Done false
10000 3821 Acked Write 9 Sent Done Done false
10001 3822 Acked Write 2 Sent Done Done false
10002 3823 Acked Read 7 Sent Done Done false
10003 3824 Acked Read 1 Sent Done Done false
10004 3825 Acked Write 8 Sent Done Done false
10005 3826 Acked Read 2 Sent Done Done false
10006 3827 Acked Write 6 Sent Done Done false
10007 3828 Acked Flush 0 Sent Done Done false
10008 3829 Acked Write 10 Sent Done Done false
10009 3830 Acked Read 3 Sent Done Done false
10010 3831 Acked Read 4 Sent Done Done false
10011 3832 Acked Flush 0 Sent Done Done false
10012 3833 Acked Write 3 Sent Done Done false
10013 3834 Acked Read 1 Sent Done Done false
10014 3835 Acked Read 5 Sent Done Done false
10015 3836 Acked Read 7 Sent Done Done false
10016 3837 Acked Read 2 Sent Done Done false
10017 3838 Acked Write 7 Sent Done Done false
10018 3839 Acked Read 2 Sent Done Done false
10019 3840 Acked Write 2 Sent Done Done false
10020 3841 Acked Read 5 Sent Done Done false
10021 3842 Acked Read 9 Sent Done Done false
10022 3843 Acked Read 7 Sent Done Done false
10023 3844 Acked Flush 0 Sent Done Done false
10024 3845 Acked Read 3 Sent Done Done false
10025 3846 Acked Read 8 Sent Done Done false
10026 3847 Acked Write 6 Sent Done Done false
10027 3848 Acked Read 2 Sent Done Done false
10028 3849 Acked Read 6 Sent Done Done false
10029 3850 Acked Read 5 Sent Done Done false
10030 3851 Acked Write 10 Sent Done Done false
10031 3852 Acked Write 10 Sent Done Done false
10032 3853 Acked Write 7 Sent Done Done false
10033 3854 Acked Read 2 Sent Done Done false
10034 3855 Acked Write 2 Sent Done Done false
10035 3856 Acked Read 2 Sent Done Done false
10036 3857 Acked Read 10 Sent Done Done false
10037 3858 Acked Read 7 Sent Done Done false
10038 3859 Acked Read 4 Sent Done Done false
10039 3860 Acked Write 3 Sent Done Done false
10040 3861 Acked Read 2 Sent Done Done false
10041 3862 Acked Read 1 Sent Done Done false
10042 3863 Acked Read 4 Sent Done Done false
10043 3864 Acked Write 6 Sent Done Done false
10044 3865 Acked Flush 0 Sent Done Done false
10045 3866 Acked Write 1 Sent Done Done false
10046 3867 Acked Read 5 Sent Done Done false
10047 3868 Acked Read 10 Sent Done Done false
10048 3869 Acked Write 2 Sent Done Done false
10049 3870 Acked Read 2 Sent Done Done false
10050 3871 Acked Read 8 Sent Done Done false
10051 3872 Acked Write 7 Sent Done Done false
10052 3873 Acked Write 10 Sent Done Done false
10053 3874 Acked Write 1 Sent Done Done false
10054 3875 Acked Read 10 Sent Done Done false
10055 3876 Acked Write 10 Sent Done Done false
10056 3877 Acked Read 8 Sent Done Done false
10057 3878 Acked Flush 0 Sent Done Done false
10058 3879 Acked Flush 0 Sent Done Done false
10059 3880 Acked Write 2 Sent Done Done false
10060 3881 Acked Read 8 Sent Done Done false
10061 3882 Acked Write 3 Sent Done Done false
10062 3883 Acked Flush 0 Sent Done Done false
10063 3884 Acked Write 9 Sent Done Done false
10064 3885 Acked Flush 0 Sent Done Done false
10065 3886 Acked Write 8 Sent Done Done false
10066 3887 Acked Read 5 Sent Done Done false
10067 3888 Acked Read 10 Sent Done Done false
10068 3889 Acked Read 3 Sent Done Done false
10069 3890 Acked Write 8 Sent Done Done false
10070 3891 Acked Write 2 Sent Done Done false
10071 3892 Acked Write 1 Sent Done Done false
10072 3893 Acked Read 9 Sent Done Done false
10073 3894 Acked Write 8 Sent Done Done false
10074 3895 Acked Flush 0 Sent Done Done false
10075 3896 Acked Read 3 Sent Done Done false
10076 3897 Acked Read 7 Sent Done Done false
10077 3898 Acked Write 9 Sent Done Done false
10078 3899 Acked Read 2 Sent Done Done false
10079 3900 Acked Read 8 Sent Done Done false
10080 3901 Acked Read 3 Sent Done Done false
10081 3902 Acked Write 1 Sent Done Done false
10082 3903 Acked Flush 0 Sent Done Done false
10083 3904 Acked Read 1 Sent Done Done false
10084 3905 Acked Flush 0 Sent Done Done false
10085 3906 Acked Read 4 Sent Done Done false
10086 3907 Acked Read 3 Sent Done Done false
10087 3908 Acked Read 7 Sent Done Done false
10088 3909 Acked Read 3 Sent Done Done false
10089 3910 Acked Write 3 Sent Done Done false
10090 3911 Acked Read 9 Sent Done Done false
10091 3912 Acked Read 6 Sent Done Done false
10092 3913 Acked Write 2 Sent Done Done false
10093 3914 Acked Read 5 Sent Done Done false
10094 3915 Acked Read 3 Sent Done Done false
10095 3916 Acked Flush 0 Sent Done Done false
10096 3917 Acked Write 7 Sent Done Done false
10097 3918 Acked Write 5 Sent Done Done false
10098 3919 Acked Read 2 Sent Done Done false
10099 3920 Acked Write 3 Sent Done Done false
10100 3921 Acked Write 3 Sent Done Done false
10101 3922 Acked Read 5 Sent Done Done false
10102 3923 Acked Flush 0 Sent Done Done false
10103 3924 Acked Write 1 Sent Done Done false
10104 3925 Acked Read 7 Sent Done Done false
10105 3926 Acked Read 1 Sent Done Done false
10106 3927 Acked Flush 0 Sent Done Done false
10107 3928 Acked Read 5 Sent Done Done false
10108 3929 Acked Write 4 Sent Done Done false
10109 3930 Acked Write 3 Sent Done Done false
10110 3931 Acked Read 6 Sent Done Done false
10111 3932 Acked Flush 0 Sent Done Done false
10112 3933 Acked Read 7 Sent Done Done false
10113 3934 Acked Read 4 Sent Done Done false
10114 3935 Acked Write 4 Sent Done Done false
10115 3936 Acked Read 4 Sent Done Done false
10116 3937 Acked Write 5 Sent Done Done false
10117 3938 Acked Read 8 Sent Done Done false
10118 3939 Acked Read 9 Sent Done Done false
10119 3940 Acked Read 4 Sent Done Done false
10120 3941 Acked Write 3 Sent Done Done false
10121 3942 Acked Write 3 Sent Done Done false
10122 3943 Acked Read 9 Sent Done Done false
10123 3944 Acked Write 8 Sent Done Done false
10124 3945 Acked Read 6 Sent Done Done false
10125 3946 Acked Read 3 Sent Done Done false
10126 3947 Acked Read 7 Sent Done Done false
10127 3948 Acked Read 9 Sent Done Done false
10128 3949 Acked Flush 0 Sent Done Done false
10129 3950 Acked Flush 0 Sent Done Done false
10130 3951 Acked Read 9 Sent Done Done false
10131 3952 Acked Read 3 Sent Done Done false
10132 3953 Acked Read 4 Sent Done Done false
10133 3954 Acked Write 8 Sent Done Done false
10134 3955 Acked Read 8 Sent Done Done false
10135 3956 Acked Read 7 Sent Done Done false
10136 3957 Acked Read 9 Sent Done Done false
10137 3958 Acked Write 6 Sent Done Done false
10138 3959 Acked Read 10 Sent Done Done false
10139 3960 Acked Read 8 Sent Done Done false
10140 3961 Acked Write 3 Sent Done Done false
10141 3962 Acked Flush 0 Sent Done Done false
10142 3963 Acked Read 6 Sent Done Done false
10143 3964 Acked Read 9 Sent Done Done false
10144 3965 Acked Read 6 Sent Done Done false
10145 3966 Acked Read 10 Sent Done Done false
10146 3967 Acked Read 7 Sent Done Done false
10147 3968 Acked Flush 0 Sent Done Done false
10148 3969 Acked Read 6 Sent Done Done false
10149 3970 Acked Read 5 Sent Done Done false
10150 3971 Acked Write 2 Sent Done Done false
10151 3972 Acked Write 6 Sent Done Done false
10152 3973 Acked Read 1 Sent Done Done false
10153 3974 Acked Read 9 Sent Done Done false
10154 3975 Acked Write 2 Sent Done Done false
10155 3976 Acked Flush 0 Sent Done Done false
10156 3977 Acked Flush 0 Sent Done Done false
10157 3978 Acked Read 8 Sent Done Done false
10158 3979 Acked Flush 0 Sent Done Done false
10159 3980 Acked Write 9 Sent Done Done false
10160 3981 Acked Write 10 Sent Done Done false
10161 3982 Acked Write 1 Sent Done Done false
10162 3983 Acked Read 9 Sent Done Done false
10163 3984 Acked Flush 0 Sent Done Done false
10164 3985 Acked Flush 0 Sent Done Done false
10165 3986 Acked Read 9 Sent Done Done false
10166 3987 Acked Read 7 Sent Done Done false
10167 3988 Acked Read 9 Sent Done Done false
10168 3989 Acked Write 9 Sent Done Done false
10169 3990 Acked Read 5 Sent Done Done false
10170 3991 Acked Read 6 Sent Done Done false
10171 3992 Acked Read 10 Sent Done Done false
10172 3993 Acked Flush 0 Sent Done Done false
10173 3994 Acked Write 3 Sent Done Done false
10174 3995 Acked Read 7 Sent Done Done false
10175 3996 Acked Write 7 Sent Done Done false
10176 3997 Acked Write 1 Sent Done Done false
10177 3998 Acked Write 8 Sent Done Done false
10178 3999 Acked Flush 0 Sent Done Done false
10179 4000 Acked Read 6 Sent Done Done false
10180 4001 Acked Write 6 Sent Done Done false
10181 4002 Acked Read 6 Sent Done Done false
10182 4003 Acked Read 7 Sent Done Done false
10183 4004 Acked Flush 0 Sent Done Done false
10184 4005 Acked Write 4 Sent Done Done false
10185 4006 Acked Write 6 Sent Done Done false
10186 4007 Acked Read 2 Sent Done Done false
10187 4008 Acked Write 4 Sent Done Done false
10188 4009 Acked Read 8 Sent Done Done false
10189 4010 Acked Write 3 Sent Done Done false
10190 4011 Acked Read 6 Sent Done Done false
10191 4012 Acked Read 6 Sent Done Done false
10192 4013 Acked Write 1 Sent Done Done false
10193 4014 Acked Read 4 Sent Done Done false
10194 4015 Acked Read 3 Sent Done Done false
10195 4016 Acked Write 9 Sent Done Done false
10196 STATES DS:0 DS:1 DS:2 TOTAL
10197 Sent 301 0 0 301
10198 Done 0 301 301 602
10199 Skipped 0 0 0 0
10200 Error 0 0 0 0
10201 Last Flush: 3715 4004 4004
10202 Downstairs last five completed: 3715 3714 3713 3712 3711
10203 Upstairs last five completed: 4016 4015 4014 4013 4012
10204 CLIENT: Up:0 ds:301 act:2
10205 [8810] got start action
10206 [8810] I am going to restart
10207 Starting downstairs at port 8810
10208 Make output file at "/tmp/dsc/downstairs-8810.txt"
10209 [8810][0] reports Starting
10210 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1210)
10211 [8810][0] reports Running
10212 ----------------------------------------------------------------
10213 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
10214 States: Offline Active Active
10215 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10216 3716 Acked Read 9 Sent Done Done false
10217 3717 Acked Read 8 Sent Done Done false
10218 3718 Acked Write 4 Sent Done Done false
10219 3719 Acked Read 2 Sent Done Done false
10220 3720 Acked Read 5 Sent Done Done false
10221 3721 Acked Write 6 Sent Done Done false
10222 3722 Acked Write 3 Sent Done Done false
10223 3723 Acked Flush 0 Sent Done Done false
10224 3724 Acked Read 6 Sent Done Done false
10225 3725 Acked Write 10 Sent Done Done false
10226 3726 Acked Write 1 Sent Done Done false
10227 3727 Acked Write 1 Sent Done Done false
10228 3728 Acked Read 10 Sent Done Done false
10229 3729 Acked Write 3 Sent Done Done false
10230 3730 Acked Write 7 Sent Done Done false
10231 3731 Acked Read 8 Sent Done Done false
10232 3732 Acked Flush 0 Sent Done Done false
10233 3733 Acked Write 5 Sent Done Done false
10234 3734 Acked Write 10 Sent Done Done false
10235 3735 Acked Read 9 Sent Done Done false
10236 3736 Acked Read 4 Sent Done Done false
10237 3737 Acked Read 1 Sent Done Done false
10238 3738 Acked Flush 0 Sent Done Done false
10239 3739 Acked Read 7 Sent Done Done false
10240 3740 Acked Read 2 Sent Done Done false
10241 3741 Acked Flush 0 Sent Done Done false
10242 3742 Acked Read 5 Sent Done Done false
10243 3743 Acked Read 3 Sent Done Done false
10244 3744 Acked Write 2 Sent Done Done false
10245 3745 Acked Read 3 Sent Done Done false
10246 3746 Acked Write 9 Sent Done Done false
10247 3747 Acked Read 9 Sent Done Done false
10248 3748 Acked Flush 0 Sent Done Done false
10249 3749 Acked Flush 0 Sent Done Done false
10250 3750 Acked Write 4 Sent Done Done false
10251 3751 Acked Read 4 Sent Done Done false
10252 3752 Acked Read 6 Sent Done Done false
10253 3753 Acked Read 2 Sent Done Done false
10254 3754 Acked Read 5 Sent Done Done false
10255 3755 Acked Read 3 Sent Done Done false
10256 3756 Acked Flush 0 Sent Done Done false
10257 3757 Acked Write 9 Sent Done Done false
10258 3758 Acked Read 2 Sent Done Done false
10259 3759 Acked Write 2 Sent Done Done false
10260 3760 Acked Read 6 Sent Done Done false
10261 3761 Acked Write 5 Sent Done Done false
10262 3762 Acked Write 1 Sent Done Done false
10263 3763 Acked Flush 0 Sent Done Done false
10264 3764 Acked Write 1 Sent Done Done false
10265 3765 Acked Read 4 Sent Done Done false
10266 3766 Acked Read 5 Sent Done Done false
10267 3767 Acked Read 9 Sent Done Done false
10268 3768 Acked Read 4 Sent Done Done false
10269 3769 Acked Write 1 Sent Done Done false
10270 3770 Acked Write 10 Sent Done Done false
10271 3771 Acked Read 2 Sent Done Done false
10272 3772 Acked Read 9 Sent Done Done false
10273 3773 Acked Flush 0 Sent Done Done false
10274 3774 Acked Write 7 Sent Done Done false
10275 3775 Acked Write 4 Sent Done Done false
10276 3776 Acked Read 6 Sent Done Done false
10277 3777 Acked Read 8 Sent Done Done false
10278 3778 Acked Read 10 Sent Done Done false
10279 3779 Acked Read 8 Sent Done Done false
10280 3780 Acked Read 4 Sent Done Done false
10281 3781 Acked Read 6 Sent Done Done false
10282 3782 Acked Read 8 Sent Done Done false
10283 3783 Acked Read 4 Sent Done Done false
10284 3784 Acked Read 7 Sent Done Done false
10285 3785 Acked Write 1 Sent Done Done false
10286 3786 Acked Write 2 Sent Done Done false
10287 3787 Acked Write 4 Sent Done Done false
10288 3788 Acked Read 4 Sent Done Done false
10289 3789 Acked Write 1 Sent Done Done false
10290 3790 Acked Write 10 Sent Done Done false
10291 3791 Acked Flush 0 Sent Done Done false
10292 3792 Acked Read 3 Sent Done Done false
10293 3793 Acked Read 6 Sent Done Done false
10294 3794 Acked Write 1 Sent Done Done false
10295 3795 Acked Write 5 Sent Done Done false
10296 3796 Acked Write 9 Sent Done Done false
10297 3797 Acked Read 7 Sent Done Done false
10298 3798 Acked Read 8 Sent Done Done false
10299 3799 Acked Read 2 Sent Done Done false
10300 3800 Acked Read 9 Sent Done Done false
10301 3801 Acked Read 9 Sent Done Done false
10302 3802 Acked Write 1 Sent Done Done false
10303 3803 Acked Read 1 Sent Done Done false
10304 3804 Acked Write 10 Sent Done Done false
10305 3805 Acked Read 3 Sent Done Done false
10306 3806 Acked Write 1 Sent Done Done false
10307 3807 Acked Write 4 Sent Done Done false
10308 3808 Acked Write 8 Sent Done Done false
10309 3809 Acked Read 1 Sent Done Done false
10310 3810 Acked Write 9 Sent Done Done false
10311 3811 Acked Read 5 Sent Done Done false
10312 3812 Acked Read 9 Sent Done Done false
10313 3813 Acked Write 2 Sent Done Done false
10314 3814 Acked Write 7 Sent Done Done false
10315 3815 Acked Write 8 Sent Done Done false
10316 3816 Acked Read 10 Sent Done Done false
10317 3817 Acked Read 5 Sent Done Done false
10318 3818 Acked Write 6 Sent Done Done false
10319 3819 Acked Write 7 Sent Done Done false
10320 3820 Acked Read 5 Sent Done Done false
10321 3821 Acked Write 9 Sent Done Done false
10322 3822 Acked Write 2 Sent Done Done false
10323 3823 Acked Read 7 Sent Done Done false
10324 3824 Acked Read 1 Sent Done Done false
10325 3825 Acked Write 8 Sent Done Done false
10326 3826 Acked Read 2 Sent Done Done false
10327 3827 Acked Write 6 Sent Done Done false
10328 3828 Acked Flush 0 Sent Done Done false
10329 3829 Acked Write 10 Sent Done Done false
10330 3830 Acked Read 3 Sent Done Done false
10331 3831 Acked Read 4 Sent Done Done false
10332 3832 Acked Flush 0 Sent Done Done false
10333 3833 Acked Write 3 Sent Done Done false
10334 3834 Acked Read 1 Sent Done Done false
10335 3835 Acked Read 5 Sent Done Done false
10336 3836 Acked Read 7 Sent Done Done false
10337 3837 Acked Read 2 Sent Done Done false
10338 3838 Acked Write 7 Sent Done Done false
10339 3839 Acked Read 2 Sent Done Done false
10340 3840 Acked Write 2 Sent Done Done false
10341 3841 Acked Read 5 Sent Done Done false
10342 3842 Acked Read 9 Sent Done Done false
10343 3843 Acked Read 7 Sent Done Done false
10344 3844 Acked Flush 0 Sent Done Done false
10345 3845 Acked Read 3 Sent Done Done false
10346 3846 Acked Read 8 Sent Done Done false
10347 3847 Acked Write 6 Sent Done Done false
10348 3848 Acked Read 2 Sent Done Done false
10349 3849 Acked Read 6 Sent Done Done false
10350 3850 Acked Read 5 Sent Done Done false
10351 3851 Acked Write 10 Sent Done Done false
10352 3852 Acked Write 10 Sent Done Done false
10353 3853 Acked Write 7 Sent Done Done false
10354 3854 Acked Read 2 Sent Done Done false
10355 3855 Acked Write 2 Sent Done Done false
10356 3856 Acked Read 2 Sent Done Done false
10357 3857 Acked Read 10 Sent Done Done false
10358 3858 Acked Read 7 Sent Done Done false
10359 3859 Acked Read 4 Sent Done Done false
10360 3860 Acked Write 3 Sent Done Done false
10361 3861 Acked Read 2 Sent Done Done false
10362 3862 Acked Read 1 Sent Done Done false
10363 3863 Acked Read 4 Sent Done Done false
10364 3864 Acked Write 6 Sent Done Done false
10365 3865 Acked Flush 0 Sent Done Done false
10366 3866 Acked Write 1 Sent Done Done false
10367 3867 Acked Read 5 Sent Done Done false
10368 3868 Acked Read 10 Sent Done Done false
10369 3869 Acked Write 2 Sent Done Done false
10370 3870 Acked Read 2 Sent Done Done false
10371 3871 Acked Read 8 Sent Done Done false
10372 3872 Acked Write 7 Sent Done Done false
10373 3873 Acked Write 10 Sent Done Done false
10374 3874 Acked Write 1 Sent Done Done false
10375 3875 Acked Read 10 Sent Done Done false
10376 3876 Acked Write 10 Sent Done Done false
10377 3877 Acked Read 8 Sent Done Done false
10378 3878 Acked Flush 0 Sent Done Done false
10379 3879 Acked Flush 0 Sent Done Done false
10380 3880 Acked Write 2 Sent Done Done false
10381 3881 Acked Read 8 Sent Done Done false
10382 3882 Acked Write 3 Sent Done Done false
10383 3883 Acked Flush 0 Sent Done Done false
10384 3884 Acked Write 9 Sent Done Done false
10385 3885 Acked Flush 0 Sent Done Done false
10386 3886 Acked Write 8 Sent Done Done false
10387 3887 Acked Read 5 Sent Done Done false
10388 3888 Acked Read 10 Sent Done Done false
10389 3889 Acked Read 3 Sent Done Done false
10390 3890 Acked Write 8 Sent Done Done false
10391 3891 Acked Write 2 Sent Done Done false
10392 3892 Acked Write 1 Sent Done Done false
10393 3893 Acked Read 9 Sent Done Done false
10394 3894 Acked Write 8 Sent Done Done false
10395 3895 Acked Flush 0 Sent Done Done false
10396 3896 Acked Read 3 Sent Done Done false
10397 3897 Acked Read 7 Sent Done Done false
10398 3898 Acked Write 9 Sent Done Done false
10399 3899 Acked Read 2 Sent Done Done false
10400 3900 Acked Read 8 Sent Done Done false
10401 3901 Acked Read 3 Sent Done Done false
10402 3902 Acked Write 1 Sent Done Done false
10403 3903 Acked Flush 0 Sent Done Done false
10404 3904 Acked Read 1 Sent Done Done false
10405 3905 Acked Flush 0 Sent Done Done false
10406 3906 Acked Read 4 Sent Done Done false
10407 3907 Acked Read 3 Sent Done Done false
10408 3908 Acked Read 7 Sent Done Done false
10409 3909 Acked Read 3 Sent Done Done false
10410 3910 Acked Write 3 Sent Done Done false
10411 3911 Acked Read 9 Sent Done Done false
10412 3912 Acked Read 6 Sent Done Done false
10413 3913 Acked Write 2 Sent Done Done false
10414 3914 Acked Read 5 Sent Done Done false
10415 3915 Acked Read 3 Sent Done Done false
10416 3916 Acked Flush 0 Sent Done Done false
10417 3917 Acked Write 7 Sent Done Done false
10418 3918 Acked Write 5 Sent Done Done false
10419 3919 Acked Read 2 Sent Done Done false
10420 3920 Acked Write 3 Sent Done Done false
10421 3921 Acked Write 3 Sent Done Done false
10422 3922 Acked Read 5 Sent Done Done false
10423 3923 Acked Flush 0 Sent Done Done false
10424 3924 Acked Write 1 Sent Done Done false
10425 3925 Acked Read 7 Sent Done Done false
10426 3926 Acked Read 1 Sent Done Done false
10427 3927 Acked Flush 0 Sent Done Done false
10428 3928 Acked Read 5 Sent Done Done false
10429 3929 Acked Write 4 Sent Done Done false
10430 3930 Acked Write 3 Sent Done Done false
10431 3931 Acked Read 6 Sent Done Done false
10432 3932 Acked Flush 0 Sent Done Done false
10433 3933 Acked Read 7 Sent Done Done false
10434 3934 Acked Read 4 Sent Done Done false
10435 3935 Acked Write 4 Sent Done Done false
10436 3936 Acked Read 4 Sent Done Done false
10437 3937 Acked Write 5 Sent Done Done false
10438 3938 Acked Read 8 Sent Done Done false
10439 3939 Acked Read 9 Sent Done Done false
10440 3940 Acked Read 4 Sent Done Done false
10441 3941 Acked Write 3 Sent Done Done false
10442 3942 Acked Write 3 Sent Done Done false
10443 3943 Acked Read 9 Sent Done Done false
10444 3944 Acked Write 8 Sent Done Done false
10445 3945 Acked Read 6 Sent Done Done false
10446 3946 Acked Read 3 Sent Done Done false
10447 3947 Acked Read 7 Sent Done Done false
10448 3948 Acked Read 9 Sent Done Done false
10449 3949 Acked Flush 0 Sent Done Done false
10450 3950 Acked Flush 0 Sent Done Done false
10451 3951 Acked Read 9 Sent Done Done false
10452 3952 Acked Read 3 Sent Done Done false
10453 3953 Acked Read 4 Sent Done Done false
10454 3954 Acked Write 8 Sent Done Done false
10455 3955 Acked Read 8 Sent Done Done false
10456 3956 Acked Read 7 Sent Done Done false
10457 3957 Acked Read 9 Sent Done Done false
10458 3958 Acked Write 6 Sent Done Done false
10459 3959 Acked Read 10 Sent Done Done false
10460 3960 Acked Read 8 Sent Done Done false
10461 3961 Acked Write 3 Sent Done Done false
10462 3962 Acked Flush 0 Sent Done Done false
10463 3963 Acked Read 6 Sent Done Done false
10464 3964 Acked Read 9 Sent Done Done false
10465 3965 Acked Read 6 Sent Done Done false
10466 3966 Acked Read 10 Sent Done Done false
10467 3967 Acked Read 7 Sent Done Done false
10468 3968 Acked Flush 0 Sent Done Done false
10469 3969 Acked Read 6 Sent Done Done false
10470 3970 Acked Read 5 Sent Done Done false
10471 3971 Acked Write 2 Sent Done Done false
10472 3972 Acked Write 6 Sent Done Done false
10473 3973 Acked Read 1 Sent Done Done false
10474 3974 Acked Read 9 Sent Done Done false
10475 3975 Acked Write 2 Sent Done Done false
10476 3976 Acked Flush 0 Sent Done Done false
10477 3977 Acked Flush 0 Sent Done Done false
10478 3978 Acked Read 8 Sent Done Done false
10479 3979 Acked Flush 0 Sent Done Done false
10480 3980 Acked Write 9 Sent Done Done false
10481 3981 Acked Write 10 Sent Done Done false
10482 3982 Acked Write 1 Sent Done Done false
10483 3983 Acked Read 9 Sent Done Done false
10484 3984 Acked Flush 0 Sent Done Done false
10485 3985 Acked Flush 0 Sent Done Done false
10486 3986 Acked Read 9 Sent Done Done false
10487 3987 Acked Read 7 Sent Done Done false
10488 3988 Acked Read 9 Sent Done Done false
10489 3989 Acked Write 9 Sent Done Done false
10490 3990 Acked Read 5 Sent Done Done false
10491 3991 Acked Read 6 Sent Done Done false
10492 3992 Acked Read 10 Sent Done Done false
10493 3993 Acked Flush 0 Sent Done Done false
10494 3994 Acked Write 3 Sent Done Done false
10495 3995 Acked Read 7 Sent Done Done false
10496 3996 Acked Write 7 Sent Done Done false
10497 3997 Acked Write 1 Sent Done Done false
10498 3998 Acked Write 8 Sent Done Done false
10499 3999 Acked Flush 0 Sent Done Done false
10500 4000 Acked Read 6 Sent Done Done false
10501 4001 Acked Write 6 Sent Done Done false
10502 4002 Acked Read 6 Sent Done Done false
10503 4003 Acked Read 7 Sent Done Done false
10504 4004 Acked Flush 0 Sent Done Done false
10505 4005 Acked Write 4 Sent Done Done false
10506 4006 Acked Write 6 Sent Done Done false
10507 4007 Acked Read 2 Sent Done Done false
10508 4008 Acked Write 4 Sent Done Done false
10509 4009 Acked Read 8 Sent Done Done false
10510 4010 Acked Write 3 Sent Done Done false
10511 4011 Acked Read 6 Sent Done Done false
10512 4012 Acked Read 6 Sent Done Done false
10513 4013 Acked Write 1 Sent Done Done false
10514 4014 Acked Read 4 Sent Done Done false
10515 4015 Acked Read 3 Sent Done Done false
10516 4016 Acked Write 9 Sent Done Done false
10517 4017 Acked Flush 0 Sent Done Done false
10518 STATES DS:0 DS:1 DS:2 TOTAL
10519 Sent 302 0 0 302
10520 Done 0 302 302 604
10521 Skipped 0 0 0 0
10522 Error 0 0 0 0
10523 Last Flush: 3715 4017 4017
10524 Downstairs last five completed: 3715 3714 3713 3712 3711
10525 Upstairs last five completed: 4017 4016 4015 4014 4013
10526 CLIENT: Up:0 ds:302 act:2
105272025-01-17T21:13:16.617ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
105282025-01-17T21:13:16.617ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:53719) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
105292025-01-17T21:13:16.618ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
105302025-01-17T21:13:16.618ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
105312025-01-17T21:13:16.618ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
105322025-01-17T21:13:16.618ZINFOcrucible: send last flush ID to this DS: 3715 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
105332025-01-17T21:13:16.619ZINFOcrucible: Replied this last flush ID: 3715 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
105342025-01-17T21:13:16.619ZINFOcrucible: [0] client re-new 302 jobs since flush 3715 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
105352025-01-17T21:13:16.619ZINFOcrucible: [0] Replayed 302 jobs since flush: 3715 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
10536 ----------------------------------------------------------------
10537 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
10538 Downstairs last five completed: 4017 4016 4015 4014 4013
10539 Upstairs last five completed: 4017 4016 4015 4014 4013
10540 CLIENT: Up:0 ds:0 act:3
10541 Replay: All jobs finished
10542 Main task has work to do, go find it
10543 got dsc Stop(0)
10544 stop 0
10545 Jan 17 21:13:18.868 INFO request completed, latency_us: 365, response_code: 204, uri: /stop/cid/0, [8810] Got stop action so:false kr:false
10546 method: POST, req_id: 8aaf0df0-43bb-493e-98bb-89c4146526e0, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
105472025-01-17T21:13:18.869ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
10548 Jan 17 21:13:18.869{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:13:18.869312168Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
10549 INFO request completed, latency_us: 177, response_code: 200, uri: {"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:13:18.869385618Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
10550 /state/cid/0, method: GET, req_id: 6df1e39c-6e53-4e05-a0d0-64dd533be785, remote_addr: {"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-17T21:13:18.869421333Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
10551 127.0.0.1:57191, local_addr: 127.0.0.1:9998
10552 [8810] Exited with: ExitStatus(unix_wait_status(9))
10553 [8810][0] reports Exit
10554 Jan 17 21:13:22.872 INFO request completed, latency_us: 212, response_code: 200, uri: /state/cid/0, method: GET, req_id: 713052f3-e710-450b-a456-9cc229501efa, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
10555 001/300 Read block 3339 len 16384
10556 002/300 Flush
10557 003/300 Flush
10558 004/300 Read block 196 len 4096
10559 005/300 Write block 1589 len 16384 data: 4 3 3 4
10560 006/300 Write block 4750 len 12288 data: 3 4 4
10561 007/300 Read block 1519 len 36864
10562 008/300 Read block 2346 len 24576
10563 009/300 Read block 4765 len 28672
10564 010/300 Flush
10565 011/300 Flush
10566 012/300 Read block 1872 len 36864
10567 013/300 Flush
10568 014/300 Read block 3504 len 24576
10569 015/300 Read block 421 len 16384
10570 016/300 Write block 706 len 40960 data: 2 3 4 4 4 4 3 3 3 3
10571 017/300 Read block 4878 len 32768
10572 018/300 Write block 3588 len 16384 data: 3 3 3 2
10573 019/300 Read block 3576 len 4096
10574 020/300 Read block 1727 len 4096
10575 021/300 Read block 4112 len 20480
10576 022/300 Write block 116 len 12288 data: 5 5 4
10577 023/300 Write block 3948 len 8192 data: 3 3
10578 024/300 Write block 3354 len 28672 data: 3 3 3 3 3 2 3
10579 025/300 Read block 598 len 4096
10580 026/300 Flush
10581 027/300 Flush
10582 028/300 Write block 2922 len 20480 data: 2 2 2 2 3
10583 029/300 Read block 3879 len 20480
10584 030/300 Read block 4355 len 36864
10585 031/300 Write block 276 len 8192 data: 3 3
10586 032/300 Read block 4607 len 20480
10587 033/300 Flush
10588 034/300 Write block 175 len 8192 data: 6 6
10589 035/300 Read block 3683 len 16384
10590 036/300 Write block 4869 len 12288 data: 2 4 4
10591 037/300 Write block 4662 len 16384 data: 4 3 3 3
10592 038/300 Read block 2288 len 4096
10593 039/300 Flush
10594 040/300 Write block 1989 len 12288 data: 3 3 3
10595 041/300 Read block 4836 len 20480
10596 042/300 Read block 1114 len 40960
10597 043/300 Write block 4620 len 32768 data: 3 4 3 3 3 3 2 2
10598 044/300 Write block 4187 len 40960 data: 5 5 5 5 5 5 5 4 3 4
10599 045/300 Write block 1157 len 32768 data: 8 7 7 6 5 4 3 2
10600 046/300 Read block 709 len 16384
10601 047/300 Read block 2334 len 28672
10602 048/300 Write block 3893 len 16384 data: 4 4 4 4
10603 049/300 Write block 1580 len 32768 data: 4 5 6 7 8 6 5 4
10604 050/300 Read block 3311 len 24576
10605 051/300 Read block 2134 len 24576
10606 052/300 Write block 1152 len 8192 data: 4 4
10607 053/300 Read block 4316 len 12288
10608 054/300 Write block 2540 len 40960 data: 5 5 5 4 3 3 3 3 3 2
10609 055/300 Read block 3059 len 8192
10610 056/300 Read block 1649 len 12288
10611 057/300 Write block 1733 len 40960 data: 3 3 3 2 2 4 4 3 4 5
10612 058/300 Flush
10613 059/300 Write block 3700 len 20480 data: 2 2 2 3 4
10614 060/300 Read block 1029 len 12288
10615 061/300 Read block 2589 len 24576
10616 062/300 Read block 260 len 12288
10617 063/300 Read block 2570 len 40960
10618 064/300 Write block 665 len 12288 data: 2 3 3
10619 065/300 Read block 4233 len 28672
10620 066/300 Read block 116 len 16384
10621 067/300 Write block 4309 len 36864 data: 3 4 4 4 3 3 4 4 5
10622 068/300 Write block 547 len 16384 data: 2 2 3 3
10623 069/300 Write block 3991 len 40960 data: 4 4 4 3 3 4 3 3 2 3
10624 070/300 Write block 3445 len 32768 data: 7 8 8 7 6 6 5 4
10625 071/300 Write block 1392 len 32768 data: 4 3 3 4 4 5 5 5
10626 072/300 Write block 2021 len 20480 data: 2 3 3 4 4
10627 073/300 Read block 2890 len 12288
10628 074/300 Write block 703 len 4096 data: 3
10629 075/300 Write block 2704 len 40960 data: 2 2 4 5 5 3 3 3 2 2
10630 076/300 Read block 4455 len 36864
10631 077/300 Write block 505 len 28672 data: 2 2 3 3 3 4 4
10632 078/300 Flush
10633 079/300 Read block 4683 len 16384
10634 080/300 Flush
10635 081/300 Flush
10636 082/300 Flush
10637 083/300 Read block 3553 len 12288
10638 084/300 Write block 2539 len 4096 data: 5
10639 085/300 Read block 2087 len 8192
10640 086/300 Read block 3592 len 24576
10641 087/300 Flush
10642 088/300 Flush
10643 089/300 Write block 68 len 8192 data: 3 3
10644 090/300 Write block 1453 len 4096 data: 3
10645 091/300 Read block 3868 len 40960
10646 092/300 Flush
10647 093/300 Write block 398 len 20480 data: 5 5 5 5 5
10648 094/300 Read block 2438 len 28672
10649 095/300 Read block 99 len 40960
10650 096/300 Write block 362 len 12288 data: 4 5 5
10651 097/300 Read block 1956 len 16384
10652 098/300 Read block 4332 len 16384
10653 099/300 Read block 4817 len 28672
10654 100/300 Read block 1243 len 20480
10655 101/300 Write block 2098 len 32768 data: 3 3 4 3 2 2 3 3
10656 102/300 Read block 3973 len 40960
10657 103/300 Write block 1628 len 36864 data: 3 2 2 2 2 2 2 2 2
10658 104/300 Write block 663 len 20480 data: 2 2 3 4 4
10659 105/300 Read block 3012 len 4096
10660 106/300 Flush
10661 107/300 Read block 477 len 8192
10662 108/300 Flush
10663 109/300 Read block 3553 len 16384
10664 110/300 Write block 503 len 12288 data: 2 2 3
10665 111/300 Read block 3262 len 32768
10666 112/300 Write block 3581 len 8192 data: 2 2
10667 113/300 Read block 935 len 20480
10668 114/300 Flush
10669 115/300 Write block 3229 len 8192 data: 4 4
10670 116/300 Write block 4552 len 40960 data: 4 4 4 3 4 4 3 3 3 3
10671 117/300 Write block 1147 len 28672 data: 3 4 5 4 4 5 5
10672 118/300 Read block 2052 len 24576
10673 119/300 Read block 1084 len 16384
10674 120/300 Read block 274 len 8192
10675 121/300 Read block 4969 len 4096
10676 122/300 Write block 337 len 8192 data: 3 3
10677 123/300 Read block 1633 len 32768
10678 124/300 Read block 4961 len 24576
10679 125/300 Read block 3226 len 32768
10680 126/300 Write block 862 len 12288 data: 3 3 3
10681 127/300 Flush
10682 128/300 Read block 1862 len 28672
10683 129/300 Write block 220 len 40960 data: 2 2 2 2 2 2 2 2 2 3
10684 130/300 Write block 4900 len 20480 data: 2 2 2 2 2
10685 131/300 Flush
10686 132/300 Write block 4743 len 28672 data: 2 3 3 3 3 3 3
10687 133/300 Flush
10688 134/300 Write block 2274 len 28672 data: 2 3 3 3 4 4 3
10689 135/300 Read block 4512 len 36864
10690 136/300 Write block 2319 len 8192 data: 4 3
10691 137/300 Read block 1559 len 32768
10692 138/300 Read block 4734 len 16384
10693 139/300 Read block 2973 len 20480
10694 140/300 Write block 3995 len 24576 data: 4 5 4 4 3 4
10695 141/300 Read block 2029 len 24576
10696 142/300 Write block 1208 len 12288 data: 2 2 2
10697 143/300 Read block 2540 len 8192
10698 144/300 Read block 966 len 20480
10699 145/300 Write block 1623 len 8192 data: 2 2
10700 146/300 Read block 4047 len 40960
10701 147/300 Flush
10702 148/300 Write block 4619 len 24576 data: 2 4 5 4 4 4
10703 149/300 Write block 386 len 36864 data: 4 4 4 5 6 7 7 7 6
10704 150/300 Read block 3259 len 40960
10705 151/300 Read block 1079 len 28672
10706 152/300 Read block 3723 len 12288
10707 153/300 Read block 4847 len 28672
10708 154/300 Read block 1439 len 16384
10709 155/300 Read block 1495 len 28672
10710 156/300 Flush
10711 157/300 Write block 2897 len 8192 data: 3 3
10712 158/300 Write block 794 len 20480 data: 4 4 3 3 3
10713 159/300 Read block 4972 len 28672
10714 160/300 Read block 3713 len 36864
10715 161/300 Flush
10716 162/300 Write block 3068 len 12288 data: 3 3 3
10717 163/300 Read block 2882 len 28672
10718 164/300 Read block 4294 len 16384
10719 165/300 Write block 2531 len 28672 data: 2 2 2 2 2 2 5
10720 166/300 Write block 4408 len 4096 data: 2
10721 167/300 Write block 3737 len 32768 data: 3 3 3 3 2 2 3 3
10722 168/300 Write block 4505 len 12288 data: 2 2 3
10723 169/300 Write block 541 len 40960 data: 3 3 2 2 2 2 3 3 4 4
10724 170/300 Read block 4624 len 8192
10725 171/300 Read block 4968 len 8192
10726 172/300 Flush
10727 173/300 Read block 4304 len 24576
10728 174/300 Write block 4216 len 20480 data: 4 5 3 3 4
10729 175/300 Flush
10730 176/300 Read block 1840 len 16384
10731 177/300 Read block 4662 len 28672
10732 178/300 Read block 3031 len 8192
10733 179/300 Read block 3176 len 20480
10734 180/300 Flush
10735 181/300 Write block 2894 len 16384 data: 3 3 3 4
10736 182/300 Write block 1749 len 12288 data: 4 4 3
10737 183/300 Read block 1738 len 4096
10738 184/300 Write block 3679 len 24576 data: 4 4 4 4 3 3
10739 185/300 Read block 2935 len 16384
10740 186/300 Flush
10741 187/300 Write block 559 len 24576 data: 3 3 3 3 3 2
10742 188/300 Read block 868 len 32768
10743 189/300 Flush
10744 190/300 Read block 1283 len 36864
10745 191/300 Write block 2984 len 16384 data: 2 2 2 3
10746 192/300 Read block 2569 len 24576
10747 193/300 Flush
10748 194/300 Write block 3224 len 20480 data: 6 6 6 5 5
10749 195/300 Write block 3103 len 36864 data: 4 5 5 3 3 3 3 3 2
10750 196/300 Write block 4373 len 4096 data: 3
10751 197/300 Write block 1635 len 20480 data: 3 3 2 2 2
10752 198/300 Flush
10753 199/300 Write block 860 len 8192 data: 3 2
10754 200/300 Write block 2166 len 32768 data: 6 7 7 8 8 6 6 6
10755 201/300 Write block 36 len 40960 data: 2 2 2 2 2 2 2 2 2 2
10756 202/300 Read block 3571 len 8192
10757 203/300 Write block 239 len 32768 data: 2 2 2 2 2 2 3 4
10758 204/300 Read block 901 len 24576
10759 205/300 Read block 708 len 4096
10760 206/300 Flush
10761 207/300 Read block 2198 len 12288
10762 208/300 Read block 1458 len 32768
10763 209/300 Read block 2119 len 40960
10764 210/300 Read block 3385 len 24576
10765 211/300 Read block 1379 len 4096
10766 212/300 Read block 2425 len 24576
10767 213/300 Write block 2815 len 28672 data: 4 5 4 4 2 2 2
10768 214/300 Write block 2550 len 8192 data: 3 3
10769 215/300 Write block 867 len 12288 data: 2 3 2
10770 216/300 Read block 1956 len 36864
10771 217/300 Read block 4218 len 4096
10772 218/300 Flush
10773 219/300 Read block 336 len 4096
10774 220/300 Write block 1989 len 24576 data: 4 4 4 4 2 2
10775 221/300 Read block 2926 len 28672
10776 222/300 Read block 292 len 20480
10777 223/300 Read block 3131 len 40960
10778 224/300 Flush
10779 225/300 Write block 1517 len 40960 data: 3 4 4 4 4 4 5 5 5 4
10780 226/300 Write block 3699 len 24576 data: 2 3 3 3 4 5
10781 227/300 Write block 65 len 28672 data: 3 3 3 4 4 3 3
10782 228/300 Read block 149 len 4096
10783 229/300 Write block 1873 len 4096 data: 4
10784 230/300 Write block 3871 len 8192 data: 3 3
10785 231/300 Read block 738 len 28672
10786 232/300 Flush
10787 233/300 Read block 1323 len 12288
10788 234/300 Write block 2630 len 8192 data: 4 5
10789 235/300 Write block 4693 len 8192 data: 4 4
10790 236/300 Flush
10791 237/300 Flush
10792 238/300 Read block 173 len 12288
10793 239/300 Read block 1204 len 4096
10794 240/300 Write block 3002 len 12288 data: 3 4 4
10795 241/300 Write block 1659 len 36864 data: 3 3 3 3 3 4 5 5 3
10796 242/300 Read block 2978 len 36864
10797 243/300 Write block 1462 len 32768 data: 4 3 3 3 3 3 3 2
10798 244/300 Read block 4525 len 16384
10799 245/300 Write block 1358 len 16384 data: 5 4 4 4
10800 246/300 Write block 4254 len 4096 data: 4
10801 247/300 Write block 383 len 16384 data: 3 3 4 5
10802 248/300 Write block 4057 len 16384 data: 3 4 4 5
10803 249/300 Write block 966 len 28672 data: 3 3 3 4 4 4 4
10804 250/300 Write block 832 len 16384 data: 4 4 3 3
10805 251/300 Flush
10806 252/300 Read block 225 len 36864
10807 253/300 Write block 3375 len 16384 data: 2 2 2 2
10808 254/300 Flush
10809 255/300 Write block 3298 len 8192 data: 6 6
10810 256/300 Write block 2240 len 24576 data: 3 3 3 3 3 3
10811 257/300 Flush
10812 258/300 Write block 1925 len 4096 data: 3
10813 259/300 Write block 4055 len 16384 data: 3 3 4 5
10814 260/300 Read block 2738 len 24576
10815 261/300 Write block 2187 len 32768 data: 3 3 3 3 2 2 2 2
10816 262/300 Write block 332 len 32768 data: 2 2 3 3 3 4 4 3
10817 263/300 Read block 1494 len 40960
10818 264/300 Read block 1091 len 8192
10819 265/300 Read block 3813 len 12288
10820 266/300 Read block 3475 len 16384
10821 267/300 Read block 1084 len 36864
10822 268/300 Read block 2223 len 16384
10823 269/300 Write block 1885 len 12288 data: 5 5 5
10824 270/300 Write block 877 len 32768 data: 3 3 3 5 4 4 3 3
10825 271/300 Write block 1414 len 40960 data: 2 2 2 2 2 2 2 2 2 2
10826 272/300 Write block 4992 len 28672 data: 2 2 2 2 2 2 2
10827 273/300 Write block 1240 len 32768 data: 2 2 2 2 2 2 2 2
10828 274/300 Read block 3345 len 28672
10829 275/300 Write block 2881 len 40960 data: 4 3 3 3 3 4 5 5 5 5
10830 276/300 Read block 2816 len 28672
10831 277/300 Read block 4138 len 20480
10832 278/300 Read block 1467 len 20480
10833 279/300 Read block 4251 len 24576
10834 280/300 Read block 1742 len 24576
10835 281/300 Read block 4466 len 4096
10836 282/300 Read block 4442 len 8192
10837 283/300 Write block 3078 len 20480 data: 3 4 4 4 5
10838 284/300 Read block 1721 len 8192
10839 285/300 Flush
10840 286/300 Read block 1176 len 20480
10841 287/300 Write block 812 len 12288 data: 4 5 4
10842 288/300 Read block 3596 len 20480
10843 289/300 Read block 1098 len 40960
10844 290/300 Read block 4598 len 12288
10845 291/300 Write block 3000 len 28672 data: 3 3 4 5 5 6 6
10846 292/300 Write block 4757 len 32768 data: 4 3 4 4 4 6 5 5
10847 293/300 Read block 3920 len 8192
10848 294/300 Read block 1906 len 16384
10849 295/300 Read block 2869 len 28672
10850 296/300 Write block 3133 len 16384 data: 3 2 2 3
10851 297/300 Read block 2985 len 8192
10852 298/300 Read block 4553 len 32768
10853 299/300 Read block 4415 len 16384
10854 300/300 Write block 2516 len 8192 data: 5 4
10855 301/300 Read block 3930 len 4096
10856 Main task has work to do, go find it
10857 got dsc Start(0)
10858 start 0
10859 Jan 17 21:13:23.098 INFO request completed, latency_us: 214, response_code: 204, uri: /start/cid/0, method: POST, req_id: 523c224c-d545-49d2-855d-904d444dcfe0, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
10860 Replay: started 0, returned:Ok(())
10861 ----------------------------------------------------------------
10862 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
10863 States: Offline Active Active
10864 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10865 4018 Acked Read 4 Sent Done Done false
10866 4019 Acked Flush 0 Sent Done Done false
10867 4020 Acked Flush 0 Sent Done Done false
10868 4021 Acked Read 1 Sent Done Done false
10869 4022 Acked Write 4 Sent Done Done false
10870 4023 Acked Write 3 Sent Done Done false
10871 4024 Acked Read 9 Sent Done Done false
10872 4025 Acked Read 6 Sent Done Done false
10873 4026 Acked Read 7 Sent Done Done false
10874 4027 Acked Flush 0 Sent Done Done false
10875 4028 Acked Flush 0 Sent Done Done false
10876 4029 Acked Read 9 Sent Done Done false
10877 4030 Acked Flush 0 Sent Done Done false
10878 4031 Acked Read 6 Sent Done Done false
10879 4032 Acked Read 4 Sent Done Done false
10880 4033 Acked Write 10 Sent Done Done false
10881 4034 Acked Read 8 Sent Done Done false
10882 4035 Acked Write 4 Sent Done Done false
10883 4036 Acked Read 1 Sent Done Done false
10884 4037 Acked Read 1 Sent Done Done false
10885 4038 Acked Read 5 Sent Done Done false
10886 4039 Acked Write 3 Sent Done Done false
10887 4040 Acked Write 2 Sent Done Done false
10888 4041 Acked Write 7 Sent Done Done false
10889 4042 Acked Read 1 Sent Done Done false
10890 4043 Acked Flush 0 Sent Done Done false
10891 4044 Acked Flush 0 Sent Done Done false
10892 4045 Acked Write 5 Sent Done Done false
10893 4046 Acked Read 5 Sent Done Done false
10894 4047 Acked Read 9 Sent Done Done false
10895 4048 Acked Write 2 Sent Done Done false
10896 4049 Acked Read 5 Sent Done Done false
10897 4050 Acked Flush 0 Sent Done Done false
10898 4051 Acked Write 2 Sent Done Done false
10899 4052 Acked Read 4 Sent Done Done false
10900 4053 Acked Write 3 Sent Done Done false
10901 4054 Acked Write 4 Sent Done Done false
10902 4055 Acked Read 1 Sent Done Done false
10903 4056 Acked Flush 0 Sent Done Done false
10904 4057 Acked Write 3 Sent Done Done false
10905 4058 Acked Read 5 Sent Done Done false
10906 4059 Acked Read 10 Sent Done Done false
10907 4060 Acked Write 8 Sent Done Done false
10908 4061 Acked Write 10 Sent Done Done false
10909 4062 Acked Write 8 Sent Done Done false
10910 4063 Acked Read 4 Sent Done Done false
10911 4064 Acked Read 7 Sent Done Done false
10912 4065 Acked Write 4 Sent Done Done false
10913 4066 Acked Write 8 Sent Done Done false
10914 4067 Acked Read 6 Sent Done Done false
10915 4068 Acked Read 6 Sent Done Done false
10916 4069 Acked Write 2 Sent Done Done false
10917 4070 Acked Read 3 Sent Done Done false
10918 4071 Acked Write 10 Sent Done Done false
10919 4072 Acked Read 2 Sent Done Done false
10920 4073 Acked Read 3 Sent Done Done false
10921 4074 Acked Write 10 Sent Done Done false
10922 4075 Acked Flush 0 Sent Done Done false
10923 4076 Acked Write 5 Sent Done Done false
10924 4077 Acked Read 3 Sent Done Done false
10925 4078 Acked Read 6 Sent Done Done false
10926 4079 Acked Read 3 Sent Done Done false
10927 4080 Acked Read 10 Sent Done Done false
10928 4081 Acked Write 3 Sent Done Done false
10929 4082 Acked Read 7 Sent Done Done false
10930 4083 Acked Read 4 Sent Done Done false
10931 4084 Acked Write 9 Sent Done Done false
10932 4085 Acked Write 4 Sent Done Done false
10933 4086 Acked Write 10 Sent Done Done false
10934 4087 Acked Write 8 Sent Done Done false
10935 4088 Acked Write 8 Sent Done Done false
10936 4089 Acked Write 5 Sent Done Done false
10937 4090 Acked Read 3 Sent Done Done false
10938 4091 Acked Write 1 Sent Done Done false
10939 4092 Acked Write 10 Sent Done Done false
10940 4093 Acked Read 9 Sent Done Done false
10941 4094 Acked Write 7 Sent Done Done false
10942 4095 Acked Flush 0 Sent Done Done false
10943 4096 Acked Read 4 Sent Done Done false
10944 4097 Acked Flush 0 Sent Done Done false
10945 4098 Acked Flush 0 Sent Done Done false
10946 4099 Acked Flush 0 Sent Done Done false
10947 4100 Acked Read 3 Sent Done Done false
10948 4101 Acked Write 1 Sent Done Done false
10949 4102 Acked Read 2 Sent Done Done false
10950 4103 Acked Read 6 Sent Done Done false
10951 4104 Acked Flush 0 Sent Done Done false
10952 4105 Acked Flush 0 Sent Done Done false
10953 4106 Acked Write 2 Sent Done Done false
10954 4107 Acked Write 1 Sent Done Done false
10955 4108 Acked Read 10 Sent Done Done false
10956 4109 Acked Flush 0 Sent Done Done false
10957 4110 Acked Write 5 Sent Done Done false
10958 4111 Acked Read 7 Sent Done Done false
10959 4112 Acked Read 10 Sent Done Done false
10960 4113 Acked Write 3 Sent Done Done false
10961 4114 Acked Read 4 Sent Done Done false
10962 4115 Acked Read 4 Sent Done Done false
10963 4116 Acked Read 7 Sent Done Done false
10964 4117 Acked Read 5 Sent Done Done false
10965 4118 Acked Write 8 Sent Done Done false
10966 4119 Acked Read 10 Sent Done Done false
10967 4120 Acked Write 9 Sent Done Done false
10968 4121 Acked Write 5 Sent Done Done false
10969 4122 Acked Read 1 Sent Done Done false
10970 4123 Acked Flush 0 Sent Done Done false
10971 4124 Acked Read 2 Sent Done Done false
10972 4125 Acked Flush 0 Sent Done Done false
10973 4126 Acked Read 4 Sent Done Done false
10974 4127 Acked Write 3 Sent Done Done false
10975 4128 Acked Read 8 Sent Done Done false
10976 4129 Acked Write 2 Sent Done Done false
10977 4130 Acked Read 5 Sent Done Done false
10978 4131 Acked Flush 0 Sent Done Done false
10979 4132 Acked Write 2 Sent Done Done false
10980 4133 Acked Write 10 Sent Done Done false
10981 4134 Acked Write 7 Sent Done Done false
10982 4135 Acked Read 6 Sent Done Done false
10983 4136 Acked Read 4 Sent Done Done false
10984 4137 Acked Read 2 Sent Done Done false
10985 4138 Acked Read 1 Sent Done Done false
10986 4139 Acked Write 2 Sent Done Done false
10987 4140 Acked Read 8 Sent Done Done false
10988 4141 Acked Read 6 Sent Done Done false
10989 4142 Acked Read 8 Sent Done Done false
10990 4143 Acked Write 3 Sent Done Done false
10991 4144 Acked Flush 0 Sent Done Done false
10992 4145 Acked Read 7 Sent Done Done false
10993 4146 Acked Write 10 Sent Done Done false
10994 4147 Acked Write 5 Sent Done Done false
10995 4148 Acked Flush 0 Sent Done Done false
10996 4149 Acked Write 7 Sent Done Done false
10997 4150 Acked Flush 0 Sent Done Done false
10998 4151 Acked Write 7 Sent Done Done false
10999 4152 Acked Read 9 Sent Done Done false
11000 4153 Acked Write 2 Sent Done Done false
11001 4154 Acked Read 8 Sent Done Done false
11002 4155 Acked Read 4 Sent Done Done false
11003 4156 Acked Read 5 Sent Done Done false
11004 4157 Acked Write 6 Sent Done Done false
11005 4158 Acked Read 6 Sent Done Done false
11006 4159 Acked Write 3 Sent Done Done false
11007 4160 Acked Read 2 Sent Done Done false
11008 4161 Acked Read 5 Sent Done Done false
11009 4162 Acked Write 2 Sent Done Done false
11010 4163 Acked Read 10 Sent Done Done false
11011 4164 Acked Flush 0 Sent Done Done false
11012 4165 Acked Write 6 Sent Done Done false
11013 4166 Acked Write 9 Sent Done Done false
11014 4167 Acked Read 10 Sent Done Done false
11015 4168 Acked Read 7 Sent Done Done false
11016 4169 Acked Read 3 Sent Done Done false
11017 4170 Acked Read 7 Sent Done Done false
11018 4171 Acked Read 4 Sent Done Done false
11019 4172 Acked Read 7 Sent Done Done false
11020 4173 Acked Flush 0 Sent Done Done false
11021 4174 Acked Write 2 Sent Done Done false
11022 4175 Acked Write 5 Sent Done Done false
11023 4176 Acked Read 7 Sent Done Done false
11024 4177 Acked Read 9 Sent Done Done false
11025 4178 Acked Flush 0 Sent Done Done false
11026 4179 Acked Write 3 Sent Done Done false
11027 4180 Acked Read 7 Sent Done Done false
11028 4181 Acked Read 4 Sent Done Done false
11029 4182 Acked Write 7 Sent Done Done false
11030 4183 Acked Write 1 Sent Done Done false
11031 4184 Acked Write 8 Sent Done Done false
11032 4185 Acked Write 3 Sent Done Done false
11033 4186 Acked Write 10 Sent Done Done false
11034 4187 Acked Read 2 Sent Done Done false
11035 4188 Acked Read 2 Sent Done Done false
11036 4189 Acked Flush 0 Sent Done Done false
11037 4190 Acked Read 6 Sent Done Done false
11038 4191 Acked Write 5 Sent Done Done false
11039 4192 Acked Flush 0 Sent Done Done false
11040 4193 Acked Read 4 Sent Done Done false
11041 4194 Acked Read 7 Sent Done Done false
11042 4195 Acked Read 2 Sent Done Done false
11043 4196 Acked Read 5 Sent Done Done false
11044 4197 Acked Flush 0 Sent Done Done false
11045 4198 Acked Write 4 Sent Done Done false
11046 4199 Acked Write 3 Sent Done Done false
11047 4200 Acked Read 1 Sent Done Done false
11048 4201 Acked Write 6 Sent Done Done false
11049 4202 Acked Read 4 Sent Done Done false
11050 4203 Acked Flush 0 Sent Done Done false
11051 4204 Acked Write 6 Sent Done Done false
11052 4205 Acked Read 8 Sent Done Done false
11053 4206 Acked Flush 0 Sent Done Done false
11054 4207 Acked Read 9 Sent Done Done false
11055 4208 Acked Write 4 Sent Done Done false
11056 4209 Acked Read 6 Sent Done Done false
11057 4210 Acked Flush 0 Sent Done Done false
11058 4211 Acked Write 5 Sent Done Done false
11059 4212 Acked Write 9 Sent Done Done false
11060 4213 Acked Write 1 Sent Done Done false
11061 4214 Acked Write 5 Sent Done Done false
11062 4215 Acked Flush 0 Sent Done Done false
11063 4216 Acked Write 2 Sent Done Done false
11064 4217 Acked Write 8 Sent Done Done false
11065 4218 Acked Write 10 Sent Done Done false
11066 4219 Acked Read 2 Sent Done Done false
11067 4220 Acked Write 8 Sent Done Done false
11068 4221 Acked Read 6 Sent Done Done false
11069 4222 Acked Read 1 Sent Done Done false
11070 4223 Acked Flush 0 Sent Done Done false
11071 4224 Acked Read 3 Sent Done Done false
11072 4225 Acked Read 8 Sent Done Done false
11073 4226 Acked Read 10 Sent Done Done false
11074 4227 Acked Read 6 Sent Done Done false
11075 4228 Acked Read 1 Sent Done Done false
11076 4229 Acked Read 6 Sent Done Done false
11077 4230 Acked Write 7 Sent Done Done false
11078 4231 Acked Write 2 Sent Done Done false
11079 4232 Acked Write 3 Sent Done Done false
11080 4233 Acked Read 9 Sent Done Done false
11081 4234 Acked Read 1 Sent Done Done false
11082 4235 Acked Flush 0 Sent Done Done false
11083 4236 Acked Read 1 Sent Done Done false
11084 4237 Acked Write 6 Sent Done Done false
11085 4238 Acked Read 7 Sent Done Done false
11086 4239 Acked Read 5 Sent Done Done false
11087 4240 Acked Read 10 Sent Done Done false
11088 4241 Acked Flush 0 Sent Done Done false
11089 4242 Acked Write 10 Sent Done Done false
11090 4243 Acked Write 6 Sent Done Done false
11091 4244 Acked Write 7 Sent Done Done false
11092 4245 Acked Read 1 Sent Done Done false
11093 4246 Acked Write 1 Sent Done Done false
11094 4247 Acked Write 2 Sent Done Done false
11095 4248 Acked Read 7 Sent Done Done false
11096 4249 Acked Flush 0 Sent Done Done false
11097 4250 Acked Read 3 Sent Done Done false
11098 4251 Acked Write 2 Sent Done Done false
11099 4252 Acked Write 2 Sent Done Done false
11100 4253 Acked Flush 0 Sent Done Done false
11101 4254 Acked Flush 0 Sent Done Done false
11102 4255 Acked Read 3 Sent Done Done false
11103 4256 Acked Read 1 Sent Done Done false
11104 4257 Acked Write 3 Sent Done Done false
11105 4258 Acked Write 9 Sent Done Done false
11106 4259 Acked Read 9 Sent Done Done false
11107 4260 Acked Write 8 Sent Done Done false
11108 4261 Acked Read 4 Sent Done Done false
11109 4262 Acked Write 4 Sent Done Done false
11110 4263 Acked Write 1 Sent Done Done false
11111 4264 Acked Write 4 Sent Done Done false
11112 4265 Acked Write 4 Sent Done Done false
11113 4266 Acked Write 7 Sent Done Done false
11114 4267 Acked Write 4 Sent Done Done false
11115 4268 Acked Flush 0 Sent Done Done false
11116 4269 Acked Read 9 Sent Done Done false
11117 4270 Acked Write 4 Sent Done Done false
11118 4271 Acked Flush 0 Sent Done Done false
11119 4272 Acked Write 2 Sent Done Done false
11120 4273 Acked Write 6 Sent Done Done false
11121 4274 Acked Flush 0 Sent Done Done false
11122 4275 Acked Write 1 Sent Done Done false
11123 4276 Acked Write 4 Sent Done Done false
11124 4277 Acked Read 6 Sent Done Done false
11125 4278 Acked Write 8 Sent Done Done false
11126 4279 Acked Write 8 Sent Done Done false
11127 4280 Acked Read 10 Sent Done Done false
11128 4281 Acked Read 2 Sent Done Done false
11129 4282 Acked Read 3 Sent Done Done false
11130 4283 Acked Read 4 Sent Done Done false
11131 4284 Acked Read 9 Sent Done Done false
11132 4285 Acked Read 4 Sent Done Done false
11133 4286 Acked Write 3 Sent Done Done false
11134 4287 Acked Write 8 Sent Done Done false
11135 4288 Acked Write 10 Sent Done Done false
11136 4289 Acked Write 7 Sent Done Done false
11137 4290 Acked Write 8 Sent Done Done false
11138 4291 Acked Read 7 Sent Done Done false
11139 4292 Acked Write 10 Sent Done Done false
11140 4293 Acked Read 7 Sent Done Done false
11141 4294 Acked Read 5 Sent Done Done false
11142 4295 Acked Read 5 Sent Done Done false
11143 4296 Acked Read 6 Sent Done Done false
11144 4297 Acked Read 6 Sent Done Done false
11145 4298 Acked Read 1 Sent Done Done false
11146 4299 Acked Read 2 Sent Done Done false
11147 4300 Acked Write 5 Sent Done Done false
11148 4301 Acked Read 2 Sent Done Done false
11149 4302 Acked Flush 0 Sent Done Done false
11150 4303 Acked Read 5 Sent Done Done false
11151 4304 Acked Write 3 Sent Done Done false
11152 4305 Acked Read 5 Sent Done Done false
11153 4306 Acked Read 10 Sent Done Done false
11154 4307 Acked Read 3 Sent Done Done false
11155 4308 Acked Write 7 Sent Done Done false
11156 4309 Acked Write 8 Sent Done Done false
11157 4310 Acked Read 2 Sent Done Done false
11158 4311 Acked Read 4 Sent Done Done false
11159 4312 Acked Read 7 Sent Done Done false
11160 4313 Acked Write 4 Sent Done Done false
11161 4314 Acked Read 2 Sent Done Done false
11162 4315 Acked Read 8 Sent Done Done false
11163 4316 Acked Read 4 Sent Done Done false
11164 4317 Acked Write 2 Sent Done Done false
11165 4318 Acked Read 1 Sent Done Done false
11166 STATES DS:0 DS:1 DS:2 TOTAL
11167 Sent 301 0 0 301
11168 Done 0 301 301 602
11169 Skipped 0 0 0 0
11170 Error 0 0 0 0
11171 Last Flush: 4017 4302 4302
11172 Downstairs last five completed: 4017 4016 4015 4014 4013
11173 Upstairs last five completed: 4318 4317 4316 4315 4314
11174 CLIENT: Up:0 ds:301 act:2
11175 ----------------------------------------------------------------
11176 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11177 States: Offline Active Active
11178 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11179 4018 Acked Read 4 Sent Done Done false
11180 4019 Acked Flush 0 Sent Done Done false
11181 4020 Acked Flush 0 Sent Done Done false
11182 4021 Acked Read 1 Sent Done Done false
11183 4022 Acked Write 4 Sent Done Done false
11184 4023 Acked Write 3 Sent Done Done false
11185 4024 Acked Read 9 Sent Done Done false
11186 4025 Acked Read 6 Sent Done Done false
11187 4026 Acked Read 7 Sent Done Done false
11188 4027 Acked Flush 0 Sent Done Done false
11189 4028 Acked Flush 0 Sent Done Done false
11190 4029 Acked Read 9 Sent Done Done false
11191 4030 Acked Flush 0 Sent Done Done false
11192 4031 Acked Read 6 Sent Done Done false
11193 4032 Acked Read 4 Sent Done Done false
11194 4033 Acked Write 10 Sent Done Done false
11195 4034 Acked Read 8 Sent Done Done false
11196 4035 Acked Write 4 Sent Done Done false
11197 4036 Acked Read 1 Sent Done Done false
11198 4037 Acked Read 1 Sent Done Done false
11199 4038 Acked Read 5 Sent Done Done false
11200 4039 Acked Write 3 Sent Done Done false
11201 4040 Acked Write 2 Sent Done Done false
11202 4041 Acked Write 7 Sent Done Done false
11203 4042 Acked Read 1 Sent Done Done false
11204 4043 Acked Flush 0 Sent Done Done false
11205 4044 Acked Flush 0 Sent Done Done false
11206 4045 Acked Write 5 Sent Done Done false
11207 4046 Acked Read 5 Sent Done Done false
11208 4047 Acked Read 9 Sent Done Done false
11209 4048 Acked Write 2 Sent Done Done false
11210 4049 Acked Read 5 Sent Done Done false
11211 4050 Acked Flush 0 Sent Done Done false
11212 4051 Acked Write 2 Sent Done Done false
11213 4052 Acked Read 4 Sent Done Done false
11214 4053 Acked Write 3 Sent Done Done false
11215 4054 Acked Write 4 Sent Done Done false
11216 4055 Acked Read 1 Sent Done Done false
11217 4056 Acked Flush 0 Sent Done Done false
11218 4057 Acked Write 3 Sent Done Done false
11219 4058 Acked Read 5 Sent Done Done false
11220 4059 Acked Read 10 Sent Done Done false
11221 4060 Acked Write 8 Sent Done Done false
11222 4061 Acked Write 10 Sent Done Done false
11223 4062 Acked Write 8 Sent Done Done false
11224 4063 Acked Read 4 Sent Done Done false
11225 4064 Acked Read 7 Sent Done Done false
11226 4065 Acked Write 4 Sent Done Done false
11227 4066 Acked Write 8 Sent Done Done false
11228 4067 Acked Read 6 Sent Done Done false
11229 4068 Acked Read 6 Sent Done Done false
11230 4069 Acked Write 2 Sent Done Done false
11231 4070 Acked Read 3 Sent Done Done false
11232 4071 Acked Write 10 Sent Done Done false
11233 4072 Acked Read 2 Sent Done Done false
11234 4073 Acked Read 3 Sent Done Done false
11235 4074 Acked Write 10 Sent Done Done false
11236 4075 Acked Flush 0 Sent Done Done false
11237 4076 Acked Write 5 Sent Done Done false
11238 4077 Acked Read 3 Sent Done Done false
11239 4078 Acked Read 6 Sent Done Done false
11240 4079 Acked Read 3 Sent Done Done false
11241 4080 Acked Read 10 Sent Done Done false
11242 4081 Acked Write 3 Sent Done Done false
11243 4082 Acked Read 7 Sent Done Done false
11244 4083 Acked Read 4 Sent Done Done false
11245 4084 Acked Write 9 Sent Done Done false
11246 4085 Acked Write 4 Sent Done Done false
11247 4086 Acked Write 10 Sent Done Done false
11248 4087 Acked Write 8 Sent Done Done false
11249 4088 Acked Write 8 Sent Done Done false
11250 4089 Acked Write 5 Sent Done Done false
11251 4090 Acked Read 3 Sent Done Done false
11252 4091 Acked Write 1 Sent Done Done false
11253 4092 Acked Write 10 Sent Done Done false
11254 4093 Acked Read 9 Sent Done Done false
11255 4094 Acked Write 7 Sent Done Done false
11256 4095 Acked Flush 0 Sent Done Done false
11257 4096 Acked Read 4 Sent Done Done false
11258 4097 Acked Flush 0 Sent Done Done false
11259 4098 Acked Flush 0 Sent Done Done false
11260 4099 Acked Flush 0 Sent Done Done false
11261 4100 Acked Read 3 Sent Done Done false
11262 4101 Acked Write 1 Sent Done Done false
11263 4102 Acked Read 2 Sent Done Done false
11264 4103 Acked Read 6 Sent Done Done false
11265 4104 Acked Flush 0 Sent Done Done false
11266 4105 Acked Flush 0 Sent Done Done false
11267 4106 Acked Write 2 Sent Done Done false
11268 4107 Acked Write 1 Sent Done Done false
11269 4108 Acked Read 10 Sent Done Done false
11270 4109 Acked Flush 0 Sent Done Done false
11271 4110 Acked Write 5 Sent Done Done false
11272 4111 Acked Read 7 Sent Done Done false
11273 4112 Acked Read 10 Sent Done Done false
11274 4113 Acked Write 3 Sent Done Done false
11275 4114 Acked Read 4 Sent Done Done false
11276 4115 Acked Read 4 Sent Done Done false
11277 4116 Acked Read 7 Sent Done Done false
11278 4117 Acked Read 5 Sent Done Done false
11279 4118 Acked Write 8 Sent Done Done false
11280 4119 Acked Read 10 Sent Done Done false
11281 4120 Acked Write 9 Sent Done Done false
11282 4121 Acked Write 5 Sent Done Done false
11283 4122 Acked Read 1 Sent Done Done false
11284 4123 Acked Flush 0 Sent Done Done false
11285 4124 Acked Read 2 Sent Done Done false
11286 4125 Acked Flush 0 Sent Done Done false
11287 4126 Acked Read 4 Sent Done Done false
11288 4127 Acked Write 3 Sent Done Done false
11289 4128 Acked Read 8 Sent Done Done false
11290 4129 Acked Write 2 Sent Done Done false
11291 4130 Acked Read 5 Sent Done Done false
11292 4131 Acked Flush 0 Sent Done Done false
11293 4132 Acked Write 2 Sent Done Done false
11294 4133 Acked Write 10 Sent Done Done false
11295 4134 Acked Write 7 Sent Done Done false
11296 4135 Acked Read 6 Sent Done Done false
11297 4136 Acked Read 4 Sent Done Done false
11298 4137 Acked Read 2 Sent Done Done false
11299 4138 Acked Read 1 Sent Done Done false
11300 4139 Acked Write 2 Sent Done Done false
11301 4140 Acked Read 8 Sent Done Done false
11302 4141 Acked Read 6 Sent Done Done false
11303 4142 Acked Read 8 Sent Done Done false
11304 4143 Acked Write 3 Sent Done Done false
11305 4144 Acked Flush 0 Sent Done Done false
11306 4145 Acked Read 7 Sent Done Done false
11307 4146 Acked Write 10 Sent Done Done false
11308 4147 Acked Write 5 Sent Done Done false
11309 4148 Acked Flush 0 Sent Done Done false
11310 4149 Acked Write 7 Sent Done Done false
11311 4150 Acked Flush 0 Sent Done Done false
11312 4151 Acked Write 7 Sent Done Done false
11313 4152 Acked Read 9 Sent Done Done false
11314 4153 Acked Write 2 Sent Done Done false
11315 4154 Acked Read 8 Sent Done Done false
11316 4155 Acked Read 4 Sent Done Done false
11317 4156 Acked Read 5 Sent Done Done false
11318 4157 Acked Write 6 Sent Done Done false
11319 4158 Acked Read 6 Sent Done Done false
11320 4159 Acked Write 3 Sent Done Done false
11321 4160 Acked Read 2 Sent Done Done false
11322 4161 Acked Read 5 Sent Done Done false
11323 4162 Acked Write 2 Sent Done Done false
11324 4163 Acked Read 10 Sent Done Done false
11325 4164 Acked Flush 0 Sent Done Done false
11326 4165 Acked Write 6 Sent Done Done false
11327 4166 Acked Write 9 Sent Done Done false
11328 4167 Acked Read 10 Sent Done Done false
11329 4168 Acked Read 7 Sent Done Done false
11330 4169 Acked Read 3 Sent Done Done false
11331 4170 Acked Read 7 Sent Done Done false
11332 4171 Acked Read 4 Sent Done Done false
11333 4172 Acked Read 7 Sent Done Done false
11334 4173 Acked Flush 0 Sent Done Done false
11335 4174 Acked Write 2 Sent Done Done false
11336 4175 Acked Write 5 Sent Done Done false
11337 4176 Acked Read 7 Sent Done Done false
11338 4177 Acked Read 9 Sent Done Done false
11339 4178 Acked Flush 0 Sent Done Done false
11340 4179 Acked Write 3 Sent Done Done false
11341 4180 Acked Read 7 Sent Done Done false
11342 4181 Acked Read 4 Sent Done Done false
11343 4182 Acked Write 7 Sent Done Done false
11344 4183 Acked Write 1 Sent Done Done false
11345 4184 Acked Write 8 Sent Done Done false
11346 4185 Acked Write 3 Sent Done Done false
11347 4186 Acked Write 10 Sent Done Done false
11348 4187 Acked Read 2 Sent Done Done false
11349 4188 Acked Read 2 Sent Done Done false
11350 4189 Acked Flush 0 Sent Done Done false
11351 4190 Acked Read 6 Sent Done Done false
11352 4191 Acked Write 5 Sent Done Done false
11353 4192 Acked Flush 0 Sent Done Done false
11354 4193 Acked Read 4 Sent Done Done false
11355 4194 Acked Read 7 Sent Done Done false
11356 4195 Acked Read 2 Sent Done Done false
11357 4196 Acked Read 5 Sent Done Done false
11358 4197 Acked Flush 0 Sent Done Done false
11359 4198 Acked Write 4 Sent Done Done false
11360 4199 Acked Write 3 Sent Done Done false
11361 4200 Acked Read 1 Sent Done Done false
11362 4201 Acked Write 6 Sent Done Done false
11363 4202 Acked Read 4 Sent Done Done false
11364 4203 Acked Flush 0 Sent Done Done false
11365 4204 Acked Write 6 Sent Done Done false
11366 4205 Acked Read 8 Sent Done Done false
11367 4206 Acked Flush 0 Sent Done Done false
11368 4207 Acked Read 9 Sent Done Done false
11369 4208 Acked Write 4 Sent Done Done false
11370 4209 Acked Read 6 Sent Done Done false
11371 4210 Acked Flush 0 Sent Done Done false
11372 4211 Acked Write 5 Sent Done Done false
11373 4212 Acked Write 9 Sent Done Done false
11374 4213 Acked Write 1 Sent Done Done false
11375 4214 Acked Write 5 Sent Done Done false
11376 4215 Acked Flush 0 Sent Done Done false
11377 4216 Acked Write 2 Sent Done Done false
11378 4217 Acked Write 8 Sent Done Done false
11379 4218 Acked Write 10 Sent Done Done false
11380 4219 Acked Read 2 Sent Done Done false
11381 4220 Acked Write 8 Sent Done Done false
11382 4221 Acked Read 6 Sent Done Done false
11383 4222 Acked Read 1 Sent Done Done false
11384 4223 Acked Flush 0 Sent Done Done false
11385 4224 Acked Read 3 Sent Done Done false
11386 4225 Acked Read 8 Sent Done Done false
11387 4226 Acked Read 10 Sent Done Done false
11388 4227 Acked Read 6 Sent Done Done false
11389 4228 Acked Read 1 Sent Done Done false
11390 4229 Acked Read 6 Sent Done Done false
11391 4230 Acked Write 7 Sent Done Done false
11392 4231 Acked Write 2 Sent Done Done false
11393 4232 Acked Write 3 Sent Done Done false
11394 4233 Acked Read 9 Sent Done Done false
11395 4234 Acked Read 1 Sent Done Done false
11396 4235 Acked Flush 0 Sent Done Done false
11397 4236 Acked Read 1 Sent Done Done false
11398 4237 Acked Write 6 Sent Done Done false
11399 4238 Acked Read 7 Sent Done Done false
11400 4239 Acked Read 5 Sent Done Done false
11401 4240 Acked Read 10 Sent Done Done false
11402 4241 Acked Flush 0 Sent Done Done false
11403 4242 Acked Write 10 Sent Done Done false
11404 4243 Acked Write 6 Sent Done Done false
11405 4244 Acked Write 7 Sent Done Done false
11406 4245 Acked Read 1 Sent Done Done false
11407 4246 Acked Write 1 Sent Done Done false
11408 4247 Acked Write 2 Sent Done Done false
11409 4248 Acked Read 7 Sent Done Done false
11410 4249 Acked Flush 0 Sent Done Done false
11411 4250 Acked Read 3 Sent Done Done false
11412 4251 Acked Write 2 Sent Done Done false
11413 4252 Acked Write 2 Sent Done Done false
11414 4253 Acked Flush 0 Sent Done Done false
11415 4254 Acked Flush 0 Sent Done Done false
11416 4255 Acked Read 3 Sent Done Done false
11417 4256 Acked Read 1 Sent Done Done false
11418 4257 Acked Write 3 Sent Done Done false
11419 4258 Acked Write 9 Sent Done Done false
11420 4259 Acked Read 9 Sent Done Done false
11421 4260 Acked Write 8 Sent Done Done false
11422 4261 Acked Read 4 Sent Done Done false
11423 4262 Acked Write 4 Sent Done Done false
11424 4263 Acked Write 1 Sent Done Done false
11425 4264 Acked Write 4 Sent Done Done false
11426 4265 Acked Write 4 Sent Done Done false
11427 4266 Acked Write 7 Sent Done Done false
11428 4267 Acked Write 4 Sent Done Done false
11429 4268 Acked Flush 0 Sent Done Done false
11430 4269 Acked Read 9 Sent Done Done false
11431 4270 Acked Write 4 Sent Done Done false
11432 4271 Acked Flush 0 Sent Done Done false
11433 4272 Acked Write 2 Sent Done Done false
11434 4273 Acked Write 6 Sent Done Done false
11435 4274 Acked Flush 0 Sent Done Done false
11436 4275 Acked Write 1 Sent Done Done false
11437 4276 Acked Write 4 Sent Done Done false
11438 4277 Acked Read 6 Sent Done Done false
11439 4278 Acked Write 8 Sent Done Done false
11440 4279 Acked Write 8 Sent Done Done false
11441 4280 Acked Read 10 Sent Done Done false
11442 4281 Acked Read 2 Sent Done Done false
11443 4282 Acked Read 3 Sent Done Done false
11444 4283 Acked Read 4 Sent Done Done false
11445 4284 Acked Read 9 Sent Done Done false
11446 4285 Acked Read 4 Sent Done Done false
11447 4286 Acked Write 3 Sent Done Done false
11448 4287 Acked Write 8 Sent Done Done false
11449 4288 Acked Write 10 Sent Done Done false
11450 4289 Acked Write 7 Sent Done Done false
11451 4290 Acked Write 8 Sent Done Done false
11452 4291 Acked Read 7 Sent Done Done false
11453 4292 Acked Write 10 Sent Done Done false
11454 4293 Acked Read 7 Sent Done Done false
11455 4294 Acked Read 5 Sent Done Done false
11456 4295 Acked Read 5 Sent Done Done false
11457 4296 Acked Read 6 Sent Done Done false
11458 4297 Acked Read 6 Sent Done Done false
11459 4298 Acked Read 1 Sent Done Done false
11460 4299 Acked Read 2 Sent Done Done false
11461 4300 Acked Write 5 Sent Done Done false
11462 4301 Acked Read 2 Sent Done Done false
11463 4302 Acked Flush 0 Sent Done Done false
11464 4303 Acked Read 5 Sent Done Done false
11465 4304 Acked Write 3 Sent Done Done false
11466 4305 Acked Read 5 Sent Done Done false
11467 4306 Acked Read 10 Sent Done Done false
11468 4307 Acked Read 3 Sent Done Done false
11469 4308 Acked Write 7 Sent Done Done false
11470 4309 Acked Write 8 Sent Done Done false
11471 4310 Acked Read 2 Sent Done Done false
11472 4311 Acked Read 4 Sent Done Done false
11473 4312 Acked Read 7 Sent Done Done false
11474 4313 Acked Write 4 Sent Done Done false
11475 4314 Acked Read 2 Sent Done Done false
11476 4315 Acked Read 8 Sent Done Done false
11477 4316 Acked Read 4 Sent Done Done false
11478 4317 Acked Write 2 Sent Done Done false
11479 4318 Acked Read 1 Sent Done Done false
11480 4319 Acked Flush 0 Sent Done Done false
11481 STATES DS:0 DS:1 DS:2 TOTAL
11482 Sent 302 0 0 302
11483 Done 0 302 302 604
11484 Skipped 0 0 0 0
11485 Error 0 0 0 0
11486 Last Flush: 4017 4319 4319
11487 Downstairs last five completed: 4017 4016 4015 4014 4013
11488 Upstairs last five completed: 4319 4318 4317 4316 4315
11489 CLIENT: Up:0 ds:302 act:2
114902025-01-17T21:13:28.871ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
114912025-01-17T21:13:28.871ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
114922025-01-17T21:13:28.871ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
114932025-01-17T21:13:28.871ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
114942025-01-17T21:13:28.871ZINFOcrucible: client task is exiting = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
11495 [8810] got start action
11496 [8810] I am going to restart
11497 Starting downstairs at port 8810
11498 Make output file at "/tmp/dsc/downstairs-8810.txt"
11499 [8810][0] reports Starting
11500 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1211)
11501 [8810][0] reports Running
11502 ----------------------------------------------------------------
11503 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11504 States: Offline Active Active
11505 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11506 4018 Acked Read 4 Sent Done Done false
11507 4019 Acked Flush 0 Sent Done Done false
11508 4020 Acked Flush 0 Sent Done Done false
11509 4021 Acked Read 1 Sent Done Done false
11510 4022 Acked Write 4 Sent Done Done false
11511 4023 Acked Write 3 Sent Done Done false
11512 4024 Acked Read 9 Sent Done Done false
11513 4025 Acked Read 6 Sent Done Done false
11514 4026 Acked Read 7 Sent Done Done false
11515 4027 Acked Flush 0 Sent Done Done false
11516 4028 Acked Flush 0 Sent Done Done false
11517 4029 Acked Read 9 Sent Done Done false
11518 4030 Acked Flush 0 Sent Done Done false
11519 4031 Acked Read 6 Sent Done Done false
11520 4032 Acked Read 4 Sent Done Done false
11521 4033 Acked Write 10 Sent Done Done false
11522 4034 Acked Read 8 Sent Done Done false
11523 4035 Acked Write 4 Sent Done Done false
11524 4036 Acked Read 1 Sent Done Done false
11525 4037 Acked Read 1 Sent Done Done false
11526 4038 Acked Read 5 Sent Done Done false
11527 4039 Acked Write 3 Sent Done Done false
11528 4040 Acked Write 2 Sent Done Done false
11529 4041 Acked Write 7 Sent Done Done false
11530 4042 Acked Read 1 Sent Done Done false
11531 4043 Acked Flush 0 Sent Done Done false
11532 4044 Acked Flush 0 Sent Done Done false
11533 4045 Acked Write 5 Sent Done Done false
11534 4046 Acked Read 5 Sent Done Done false
11535 4047 Acked Read 9 Sent Done Done false
11536 4048 Acked Write 2 Sent Done Done false
11537 4049 Acked Read 5 Sent Done Done false
11538 4050 Acked Flush 0 Sent Done Done false
11539 4051 Acked Write 2 Sent Done Done false
11540 4052 Acked Read 4 Sent Done Done false
11541 4053 Acked Write 3 Sent Done Done false
11542 4054 Acked Write 4 Sent Done Done false
11543 4055 Acked Read 1 Sent Done Done false
11544 4056 Acked Flush 0 Sent Done Done false
11545 4057 Acked Write 3 Sent Done Done false
11546 4058 Acked Read 5 Sent Done Done false
11547 4059 Acked Read 10 Sent Done Done false
11548 4060 Acked Write 8 Sent Done Done false
11549 4061 Acked Write 10 Sent Done Done false
11550 4062 Acked Write 8 Sent Done Done false
11551 4063 Acked Read 4 Sent Done Done false
11552 4064 Acked Read 7 Sent Done Done false
11553 4065 Acked Write 4 Sent Done Done false
11554 4066 Acked Write 8 Sent Done Done false
11555 4067 Acked Read 6 Sent Done Done false
11556 4068 Acked Read 6 Sent Done Done false
11557 4069 Acked Write 2 Sent Done Done false
11558 4070 Acked Read 3 Sent Done Done false
11559 4071 Acked Write 10 Sent Done Done false
11560 4072 Acked Read 2 Sent Done Done false
11561 4073 Acked Read 3 Sent Done Done false
11562 4074 Acked Write 10 Sent Done Done false
11563 4075 Acked Flush 0 Sent Done Done false
11564 4076 Acked Write 5 Sent Done Done false
11565 4077 Acked Read 3 Sent Done Done false
11566 4078 Acked Read 6 Sent Done Done false
11567 4079 Acked Read 3 Sent Done Done false
11568 4080 Acked Read 10 Sent Done Done false
11569 4081 Acked Write 3 Sent Done Done false
11570 4082 Acked Read 7 Sent Done Done false
11571 4083 Acked Read 4 Sent Done Done false
11572 4084 Acked Write 9 Sent Done Done false
11573 4085 Acked Write 4 Sent Done Done false
11574 4086 Acked Write 10 Sent Done Done false
11575 4087 Acked Write 8 Sent Done Done false
11576 4088 Acked Write 8 Sent Done Done false
11577 4089 Acked Write 5 Sent Done Done false
11578 4090 Acked Read 3 Sent Done Done false
11579 4091 Acked Write 1 Sent Done Done false
11580 4092 Acked Write 10 Sent Done Done false
11581 4093 Acked Read 9 Sent Done Done false
11582 4094 Acked Write 7 Sent Done Done false
11583 4095 Acked Flush 0 Sent Done Done false
11584 4096 Acked Read 4 Sent Done Done false
11585 4097 Acked Flush 0 Sent Done Done false
11586 4098 Acked Flush 0 Sent Done Done false
11587 4099 Acked Flush 0 Sent Done Done false
11588 4100 Acked Read 3 Sent Done Done false
11589 4101 Acked Write 1 Sent Done Done false
11590 4102 Acked Read 2 Sent Done Done false
11591 4103 Acked Read 6 Sent Done Done false
11592 4104 Acked Flush 0 Sent Done Done false
11593 4105 Acked Flush 0 Sent Done Done false
11594 4106 Acked Write 2 Sent Done Done false
11595 4107 Acked Write 1 Sent Done Done false
11596 4108 Acked Read 10 Sent Done Done false
11597 4109 Acked Flush 0 Sent Done Done false
11598 4110 Acked Write 5 Sent Done Done false
11599 4111 Acked Read 7 Sent Done Done false
11600 4112 Acked Read 10 Sent Done Done false
11601 4113 Acked Write 3 Sent Done Done false
11602 4114 Acked Read 4 Sent Done Done false
11603 4115 Acked Read 4 Sent Done Done false
11604 4116 Acked Read 7 Sent Done Done false
11605 4117 Acked Read 5 Sent Done Done false
11606 4118 Acked Write 8 Sent Done Done false
11607 4119 Acked Read 10 Sent Done Done false
11608 4120 Acked Write 9 Sent Done Done false
11609 4121 Acked Write 5 Sent Done Done false
11610 4122 Acked Read 1 Sent Done Done false
11611 4123 Acked Flush 0 Sent Done Done false
11612 4124 Acked Read 2 Sent Done Done false
11613 4125 Acked Flush 0 Sent Done Done false
11614 4126 Acked Read 4 Sent Done Done false
11615 4127 Acked Write 3 Sent Done Done false
11616 4128 Acked Read 8 Sent Done Done false
11617 4129 Acked Write 2 Sent Done Done false
11618 4130 Acked Read 5 Sent Done Done false
11619 4131 Acked Flush 0 Sent Done Done false
11620 4132 Acked Write 2 Sent Done Done false
11621 4133 Acked Write 10 Sent Done Done false
11622 4134 Acked Write 7 Sent Done Done false
11623 4135 Acked Read 6 Sent Done Done false
11624 4136 Acked Read 4 Sent Done Done false
11625 4137 Acked Read 2 Sent Done Done false
11626 4138 Acked Read 1 Sent Done Done false
11627 4139 Acked Write 2 Sent Done Done false
11628 4140 Acked Read 8 Sent Done Done false
11629 4141 Acked Read 6 Sent Done Done false
11630 4142 Acked Read 8 Sent Done Done false
11631 4143 Acked Write 3 Sent Done Done false
11632 4144 Acked Flush 0 Sent Done Done false
11633 4145 Acked Read 7 Sent Done Done false
11634 4146 Acked Write 10 Sent Done Done false
11635 4147 Acked Write 5 Sent Done Done false
11636 4148 Acked Flush 0 Sent Done Done false
11637 4149 Acked Write 7 Sent Done Done false
11638 4150 Acked Flush 0 Sent Done Done false
11639 4151 Acked Write 7 Sent Done Done false
11640 4152 Acked Read 9 Sent Done Done false
11641 4153 Acked Write 2 Sent Done Done false
11642 4154 Acked Read 8 Sent Done Done false
11643 4155 Acked Read 4 Sent Done Done false
11644 4156 Acked Read 5 Sent Done Done false
11645 4157 Acked Write 6 Sent Done Done false
11646 4158 Acked Read 6 Sent Done Done false
11647 4159 Acked Write 3 Sent Done Done false
11648 4160 Acked Read 2 Sent Done Done false
11649 4161 Acked Read 5 Sent Done Done false
11650 4162 Acked Write 2 Sent Done Done false
11651 4163 Acked Read 10 Sent Done Done false
11652 4164 Acked Flush 0 Sent Done Done false
11653 4165 Acked Write 6 Sent Done Done false
11654 4166 Acked Write 9 Sent Done Done false
11655 4167 Acked Read 10 Sent Done Done false
11656 4168 Acked Read 7 Sent Done Done false
11657 4169 Acked Read 3 Sent Done Done false
11658 4170 Acked Read 7 Sent Done Done false
11659 4171 Acked Read 4 Sent Done Done false
11660 4172 Acked Read 7 Sent Done Done false
11661 4173 Acked Flush 0 Sent Done Done false
11662 4174 Acked Write 2 Sent Done Done false
11663 4175 Acked Write 5 Sent Done Done false
11664 4176 Acked Read 7 Sent Done Done false
11665 4177 Acked Read 9 Sent Done Done false
11666 4178 Acked Flush 0 Sent Done Done false
11667 4179 Acked Write 3 Sent Done Done false
11668 4180 Acked Read 7 Sent Done Done false
11669 4181 Acked Read 4 Sent Done Done false
11670 4182 Acked Write 7 Sent Done Done false
11671 4183 Acked Write 1 Sent Done Done false
11672 4184 Acked Write 8 Sent Done Done false
11673 4185 Acked Write 3 Sent Done Done false
11674 4186 Acked Write 10 Sent Done Done false
11675 4187 Acked Read 2 Sent Done Done false
11676 4188 Acked Read 2 Sent Done Done false
11677 4189 Acked Flush 0 Sent Done Done false
11678 4190 Acked Read 6 Sent Done Done false
11679 4191 Acked Write 5 Sent Done Done false
11680 4192 Acked Flush 0 Sent Done Done false
11681 4193 Acked Read 4 Sent Done Done false
11682 4194 Acked Read 7 Sent Done Done false
11683 4195 Acked Read 2 Sent Done Done false
11684 4196 Acked Read 5 Sent Done Done false
11685 4197 Acked Flush 0 Sent Done Done false
11686 4198 Acked Write 4 Sent Done Done false
11687 4199 Acked Write 3 Sent Done Done false
11688 4200 Acked Read 1 Sent Done Done false
11689 4201 Acked Write 6 Sent Done Done false
11690 4202 Acked Read 4 Sent Done Done false
11691 4203 Acked Flush 0 Sent Done Done false
11692 4204 Acked Write 6 Sent Done Done false
11693 4205 Acked Read 8 Sent Done Done false
11694 4206 Acked Flush 0 Sent Done Done false
11695 4207 Acked Read 9 Sent Done Done false
11696 4208 Acked Write 4 Sent Done Done false
11697 4209 Acked Read 6 Sent Done Done false
11698 4210 Acked Flush 0 Sent Done Done false
11699 4211 Acked Write 5 Sent Done Done false
11700 4212 Acked Write 9 Sent Done Done false
11701 4213 Acked Write 1 Sent Done Done false
11702 4214 Acked Write 5 Sent Done Done false
11703 4215 Acked Flush 0 Sent Done Done false
11704 4216 Acked Write 2 Sent Done Done false
11705 4217 Acked Write 8 Sent Done Done false
11706 4218 Acked Write 10 Sent Done Done false
11707 4219 Acked Read 2 Sent Done Done false
11708 4220 Acked Write 8 Sent Done Done false
11709 4221 Acked Read 6 Sent Done Done false
11710 4222 Acked Read 1 Sent Done Done false
11711 4223 Acked Flush 0 Sent Done Done false
11712 4224 Acked Read 3 Sent Done Done false
11713 4225 Acked Read 8 Sent Done Done false
11714 4226 Acked Read 10 Sent Done Done false
11715 4227 Acked Read 6 Sent Done Done false
11716 4228 Acked Read 1 Sent Done Done false
11717 4229 Acked Read 6 Sent Done Done false
11718 4230 Acked Write 7 Sent Done Done false
11719 4231 Acked Write 2 Sent Done Done false
11720 4232 Acked Write 3 Sent Done Done false
11721 4233 Acked Read 9 Sent Done Done false
11722 4234 Acked Read 1 Sent Done Done false
11723 4235 Acked Flush 0 Sent Done Done false
11724 4236 Acked Read 1 Sent Done Done false
11725 4237 Acked Write 6 Sent Done Done false
11726 4238 Acked Read 7 Sent Done Done false
11727 4239 Acked Read 5 Sent Done Done false
11728 4240 Acked Read 10 Sent Done Done false
11729 4241 Acked Flush 0 Sent Done Done false
11730 4242 Acked Write 10 Sent Done Done false
11731 4243 Acked Write 6 Sent Done Done false
11732 4244 Acked Write 7 Sent Done Done false
11733 4245 Acked Read 1 Sent Done Done false
11734 4246 Acked Write 1 Sent Done Done false
11735 4247 Acked Write 2 Sent Done Done false
11736 4248 Acked Read 7 Sent Done Done false
11737 4249 Acked Flush 0 Sent Done Done false
11738 4250 Acked Read 3 Sent Done Done false
11739 4251 Acked Write 2 Sent Done Done false
11740 4252 Acked Write 2 Sent Done Done false
11741 4253 Acked Flush 0 Sent Done Done false
11742 4254 Acked Flush 0 Sent Done Done false
11743 4255 Acked Read 3 Sent Done Done false
11744 4256 Acked Read 1 Sent Done Done false
11745 4257 Acked Write 3 Sent Done Done false
11746 4258 Acked Write 9 Sent Done Done false
11747 4259 Acked Read 9 Sent Done Done false
11748 4260 Acked Write 8 Sent Done Done false
11749 4261 Acked Read 4 Sent Done Done false
11750 4262 Acked Write 4 Sent Done Done false
11751 4263 Acked Write 1 Sent Done Done false
11752 4264 Acked Write 4 Sent Done Done false
11753 4265 Acked Write 4 Sent Done Done false
11754 4266 Acked Write 7 Sent Done Done false
11755 4267 Acked Write 4 Sent Done Done false
11756 4268 Acked Flush 0 Sent Done Done false
11757 4269 Acked Read 9 Sent Done Done false
11758 4270 Acked Write 4 Sent Done Done false
11759 4271 Acked Flush 0 Sent Done Done false
11760 4272 Acked Write 2 Sent Done Done false
11761 4273 Acked Write 6 Sent Done Done false
11762 4274 Acked Flush 0 Sent Done Done false
11763 4275 Acked Write 1 Sent Done Done false
11764 4276 Acked Write 4 Sent Done Done false
11765 4277 Acked Read 6 Sent Done Done false
11766 4278 Acked Write 8 Sent Done Done false
11767 4279 Acked Write 8 Sent Done Done false
11768 4280 Acked Read 10 Sent Done Done false
11769 4281 Acked Read 2 Sent Done Done false
11770 4282 Acked Read 3 Sent Done Done false
11771 4283 Acked Read 4 Sent Done Done false
11772 4284 Acked Read 9 Sent Done Done false
11773 4285 Acked Read 4 Sent Done Done false
11774 4286 Acked Write 3 Sent Done Done false
11775 4287 Acked Write 8 Sent Done Done false
11776 4288 Acked Write 10 Sent Done Done false
11777 4289 Acked Write 7 Sent Done Done false
11778 4290 Acked Write 8 Sent Done Done false
11779 4291 Acked Read 7 Sent Done Done false
11780 4292 Acked Write 10 Sent Done Done false
11781 4293 Acked Read 7 Sent Done Done false
11782 4294 Acked Read 5 Sent Done Done false
11783 4295 Acked Read 5 Sent Done Done false
11784 4296 Acked Read 6 Sent Done Done false
11785 4297 Acked Read 6 Sent Done Done false
11786 4298 Acked Read 1 Sent Done Done false
11787 4299 Acked Read 2 Sent Done Done false
11788 4300 Acked Write 5 Sent Done Done false
11789 4301 Acked Read 2 Sent Done Done false
11790 4302 Acked Flush 0 Sent Done Done false
11791 4303 Acked Read 5 Sent Done Done false
11792 4304 Acked Write 3 Sent Done Done false
11793 4305 Acked Read 5 Sent Done Done false
11794 4306 Acked Read 10 Sent Done Done false
11795 4307 Acked Read 3 Sent Done Done false
11796 4308 Acked Write 7 Sent Done Done false
11797 4309 Acked Write 8 Sent Done Done false
11798 4310 Acked Read 2 Sent Done Done false
11799 4311 Acked Read 4 Sent Done Done false
11800 4312 Acked Read 7 Sent Done Done false
11801 4313 Acked Write 4 Sent Done Done false
11802 4314 Acked Read 2 Sent Done Done false
11803 4315 Acked Read 8 Sent Done Done false
11804 4316 Acked Read 4 Sent Done Done false
11805 4317 Acked Write 2 Sent Done Done false
11806 4318 Acked Read 1 Sent Done Done false
11807 4319 Acked Flush 0 Sent Done Done false
11808 STATES DS:0 DS:1 DS:2 TOTAL
11809 Sent 302 0 0 302
11810 Done 0 302 302 604
11811 Skipped 0 0 0 0
11812 Error 0 0 0 0
11813 Last Flush: 4017 4319 4319
11814 Downstairs last five completed: 4017 4016 4015 4014 4013
11815 Upstairs last five completed: 4319 4318 4317 4316 4315
11816 CLIENT: Up:0 ds:302 act:2
11817 ----------------------------------------------------------------
11818 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11819 States: Offline Active Active
11820 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11821 4018 Acked Read 4 Sent Done Done false
11822 4019 Acked Flush 0 Sent Done Done false
11823 4020 Acked Flush 0 Sent Done Done false
11824 4021 Acked Read 1 Sent Done Done false
11825 4022 Acked Write 4 Sent Done Done false
11826 4023 Acked Write 3 Sent Done Done false
11827 4024 Acked Read 9 Sent Done Done false
11828 4025 Acked Read 6 Sent Done Done false
11829 4026 Acked Read 7 Sent Done Done false
11830 4027 Acked Flush 0 Sent Done Done false
11831 4028 Acked Flush 0 Sent Done Done false
11832 4029 Acked Read 9 Sent Done Done false
11833 4030 Acked Flush 0 Sent Done Done false
11834 4031 Acked Read 6 Sent Done Done false
11835 4032 Acked Read 4 Sent Done Done false
11836 4033 Acked Write 10 Sent Done Done false
11837 4034 Acked Read 8 Sent Done Done false
11838 4035 Acked Write 4 Sent Done Done false
11839 4036 Acked Read 1 Sent Done Done false
11840 4037 Acked Read 1 Sent Done Done false
11841 4038 Acked Read 5 Sent Done Done false
11842 4039 Acked Write 3 Sent Done Done false
11843 4040 Acked Write 2 Sent Done Done false
11844 4041 Acked Write 7 Sent Done Done false
11845 4042 Acked Read 1 Sent Done Done false
11846 4043 Acked Flush 0 Sent Done Done false
11847 4044 Acked Flush 0 Sent Done Done false
11848 4045 Acked Write 5 Sent Done Done false
11849 4046 Acked Read 5 Sent Done Done false
11850 4047 Acked Read 9 Sent Done Done false
11851 4048 Acked Write 2 Sent Done Done false
11852 4049 Acked Read 5 Sent Done Done false
11853 4050 Acked Flush 0 Sent Done Done false
11854 4051 Acked Write 2 Sent Done Done false
11855 4052 Acked Read 4 Sent Done Done false
11856 4053 Acked Write 3 Sent Done Done false
11857 4054 Acked Write 4 Sent Done Done false
11858 4055 Acked Read 1 Sent Done Done false
11859 4056 Acked Flush 0 Sent Done Done false
11860 4057 Acked Write 3 Sent Done Done false
11861 4058 Acked Read 5 Sent Done Done false
11862 4059 Acked Read 10 Sent Done Done false
11863 4060 Acked Write 8 Sent Done Done false
11864 4061 Acked Write 10 Sent Done Done false
11865 4062 Acked Write 8 Sent Done Done false
11866 4063 Acked Read 4 Sent Done Done false
11867 4064 Acked Read 7 Sent Done Done false
11868 4065 Acked Write 4 Sent Done Done false
11869 4066 Acked Write 8 Sent Done Done false
11870 4067 Acked Read 6 Sent Done Done false
11871 4068 Acked Read 6 Sent Done Done false
11872 4069 Acked Write 2 Sent Done Done false
11873 4070 Acked Read 3 Sent Done Done false
11874 4071 Acked Write 10 Sent Done Done false
11875 4072 Acked Read 2 Sent Done Done false
11876 4073 Acked Read 3 Sent Done Done false
11877 4074 Acked Write 10 Sent Done Done false
11878 4075 Acked Flush 0 Sent Done Done false
11879 4076 Acked Write 5 Sent Done Done false
11880 4077 Acked Read 3 Sent Done Done false
11881 4078 Acked Read 6 Sent Done Done false
11882 4079 Acked Read 3 Sent Done Done false
11883 4080 Acked Read 10 Sent Done Done false
11884 4081 Acked Write 3 Sent Done Done false
11885 4082 Acked Read 7 Sent Done Done false
11886 4083 Acked Read 4 Sent Done Done false
11887 4084 Acked Write 9 Sent Done Done false
11888 4085 Acked Write 4 Sent Done Done false
11889 4086 Acked Write 10 Sent Done Done false
11890 4087 Acked Write 8 Sent Done Done false
11891 4088 Acked Write 8 Sent Done Done false
11892 4089 Acked Write 5 Sent Done Done false
11893 4090 Acked Read 3 Sent Done Done false
11894 4091 Acked Write 1 Sent Done Done false
11895 4092 Acked Write 10 Sent Done Done false
11896 4093 Acked Read 9 Sent Done Done false
11897 4094 Acked Write 7 Sent Done Done false
11898 4095 Acked Flush 0 Sent Done Done false
11899 4096 Acked Read 4 Sent Done Done false
11900 4097 Acked Flush 0 Sent Done Done false
11901 4098 Acked Flush 0 Sent Done Done false
11902 4099 Acked Flush 0 Sent Done Done false
11903 4100 Acked Read 3 Sent Done Done false
11904 4101 Acked Write 1 Sent Done Done false
11905 4102 Acked Read 2 Sent Done Done false
11906 4103 Acked Read 6 Sent Done Done false
11907 4104 Acked Flush 0 Sent Done Done false
11908 4105 Acked Flush 0 Sent Done Done false
11909 4106 Acked Write 2 Sent Done Done false
11910 4107 Acked Write 1 Sent Done Done false
11911 4108 Acked Read 10 Sent Done Done false
11912 4109 Acked Flush 0 Sent Done Done false
11913 4110 Acked Write 5 Sent Done Done false
11914 4111 Acked Read 7 Sent Done Done false
11915 4112 Acked Read 10 Sent Done Done false
11916 4113 Acked Write 3 Sent Done Done false
11917 4114 Acked Read 4 Sent Done Done false
11918 4115 Acked Read 4 Sent Done Done false
11919 4116 Acked Read 7 Sent Done Done false
11920 4117 Acked Read 5 Sent Done Done false
11921 4118 Acked Write 8 Sent Done Done false
11922 4119 Acked Read 10 Sent Done Done false
11923 4120 Acked Write 9 Sent Done Done false
11924 4121 Acked Write 5 Sent Done Done false
11925 4122 Acked Read 1 Sent Done Done false
11926 4123 Acked Flush 0 Sent Done Done false
11927 4124 Acked Read 2 Sent Done Done false
11928 4125 Acked Flush 0 Sent Done Done false
11929 4126 Acked Read 4 Sent Done Done false
11930 4127 Acked Write 3 Sent Done Done false
11931 4128 Acked Read 8 Sent Done Done false
11932 4129 Acked Write 2 Sent Done Done false
11933 4130 Acked Read 5 Sent Done Done false
11934 4131 Acked Flush 0 Sent Done Done false
11935 4132 Acked Write 2 Sent Done Done false
11936 4133 Acked Write 10 Sent Done Done false
11937 4134 Acked Write 7 Sent Done Done false
11938 4135 Acked Read 6 Sent Done Done false
11939 4136 Acked Read 4 Sent Done Done false
11940 4137 Acked Read 2 Sent Done Done false
11941 4138 Acked Read 1 Sent Done Done false
11942 4139 Acked Write 2 Sent Done Done false
11943 4140 Acked Read 8 Sent Done Done false
11944 4141 Acked Read 6 Sent Done Done false
11945 4142 Acked Read 8 Sent Done Done false
11946 4143 Acked Write 3 Sent Done Done false
11947 4144 Acked Flush 0 Sent Done Done false
11948 4145 Acked Read 7 Sent Done Done false
11949 4146 Acked Write 10 Sent Done Done false
11950 4147 Acked Write 5 Sent Done Done false
11951 4148 Acked Flush 0 Sent Done Done false
11952 4149 Acked Write 7 Sent Done Done false
11953 4150 Acked Flush 0 Sent Done Done false
11954 4151 Acked Write 7 Sent Done Done false
11955 4152 Acked Read 9 Sent Done Done false
11956 4153 Acked Write 2 Sent Done Done false
11957 4154 Acked Read 8 Sent Done Done false
11958 4155 Acked Read 4 Sent Done Done false
11959 4156 Acked Read 5 Sent Done Done false
11960 4157 Acked Write 6 Sent Done Done false
11961 4158 Acked Read 6 Sent Done Done false
11962 4159 Acked Write 3 Sent Done Done false
11963 4160 Acked Read 2 Sent Done Done false
11964 4161 Acked Read 5 Sent Done Done false
11965 4162 Acked Write 2 Sent Done Done false
11966 4163 Acked Read 10 Sent Done Done false
11967 4164 Acked Flush 0 Sent Done Done false
11968 4165 Acked Write 6 Sent Done Done false
11969 4166 Acked Write 9 Sent Done Done false
11970 4167 Acked Read 10 Sent Done Done false
11971 4168 Acked Read 7 Sent Done Done false
11972 4169 Acked Read 3 Sent Done Done false
11973 4170 Acked Read 7 Sent Done Done false
11974 4171 Acked Read 4 Sent Done Done false
11975 4172 Acked Read 7 Sent Done Done false
11976 4173 Acked Flush 0 Sent Done Done false
11977 4174 Acked Write 2 Sent Done Done false
11978 4175 Acked Write 5 Sent Done Done false
11979 4176 Acked Read 7 Sent Done Done false
11980 4177 Acked Read 9 Sent Done Done false
11981 4178 Acked Flush 0 Sent Done Done false
11982 4179 Acked Write 3 Sent Done Done false
11983 4180 Acked Read 7 Sent Done Done false
11984 4181 Acked Read 4 Sent Done Done false
11985 4182 Acked Write 7 Sent Done Done false
11986 4183 Acked Write 1 Sent Done Done false
11987 4184 Acked Write 8 Sent Done Done false
11988 4185 Acked Write 3 Sent Done Done false
11989 4186 Acked Write 10 Sent Done Done false
11990 4187 Acked Read 2 Sent Done Done false
11991 4188 Acked Read 2 Sent Done Done false
11992 4189 Acked Flush 0 Sent Done Done false
11993 4190 Acked Read 6 Sent Done Done false
11994 4191 Acked Write 5 Sent Done Done false
11995 4192 Acked Flush 0 Sent Done Done false
11996 4193 Acked Read 4 Sent Done Done false
11997 4194 Acked Read 7 Sent Done Done false
11998 4195 Acked Read 2 Sent Done Done false
11999 4196 Acked Read 5 Sent Done Done false
12000 4197 Acked Flush 0 Sent Done Done false
12001 4198 Acked Write 4 Sent Done Done false
12002 4199 Acked Write 3 Sent Done Done false
12003 4200 Acked Read 1 Sent Done Done false
12004 4201 Acked Write 6 Sent Done Done false
12005 4202 Acked Read 4 Sent Done Done false
12006 4203 Acked Flush 0 Sent Done Done false
12007 4204 Acked Write 6 Sent Done Done false
12008 4205 Acked Read 8 Sent Done Done false
12009 4206 Acked Flush 0 Sent Done Done false
12010 4207 Acked Read 9 Sent Done Done false
12011 4208 Acked Write 4 Sent Done Done false
12012 4209 Acked Read 6 Sent Done Done false
12013 4210 Acked Flush 0 Sent Done Done false
12014 4211 Acked Write 5 Sent Done Done false
12015 4212 Acked Write 9 Sent Done Done false
12016 4213 Acked Write 1 Sent Done Done false
12017 4214 Acked Write 5 Sent Done Done false
12018 4215 Acked Flush 0 Sent Done Done false
12019 4216 Acked Write 2 Sent Done Done false
12020 4217 Acked Write 8 Sent Done Done false
12021 4218 Acked Write 10 Sent Done Done false
12022 4219 Acked Read 2 Sent Done Done false
12023 4220 Acked Write 8 Sent Done Done false
12024 4221 Acked Read 6 Sent Done Done false
12025 4222 Acked Read 1 Sent Done Done false
12026 4223 Acked Flush 0 Sent Done Done false
12027 4224 Acked Read 3 Sent Done Done false
12028 4225 Acked Read 8 Sent Done Done false
12029 4226 Acked Read 10 Sent Done Done false
12030 4227 Acked Read 6 Sent Done Done false
12031 4228 Acked Read 1 Sent Done Done false
12032 4229 Acked Read 6 Sent Done Done false
12033 4230 Acked Write 7 Sent Done Done false
12034 4231 Acked Write 2 Sent Done Done false
12035 4232 Acked Write 3 Sent Done Done false
12036 4233 Acked Read 9 Sent Done Done false
12037 4234 Acked Read 1 Sent Done Done false
12038 4235 Acked Flush 0 Sent Done Done false
12039 4236 Acked Read 1 Sent Done Done false
12040 4237 Acked Write 6 Sent Done Done false
12041 4238 Acked Read 7 Sent Done Done false
12042 4239 Acked Read 5 Sent Done Done false
12043 4240 Acked Read 10 Sent Done Done false
12044 4241 Acked Flush 0 Sent Done Done false
12045 4242 Acked Write 10 Sent Done Done false
12046 4243 Acked Write 6 Sent Done Done false
12047 4244 Acked Write 7 Sent Done Done false
12048 4245 Acked Read 1 Sent Done Done false
12049 4246 Acked Write 1 Sent Done Done false
12050 4247 Acked Write 2 Sent Done Done false
12051 4248 Acked Read 7 Sent Done Done false
12052 4249 Acked Flush 0 Sent Done Done false
12053 4250 Acked Read 3 Sent Done Done false
12054 4251 Acked Write 2 Sent Done Done false
12055 4252 Acked Write 2 Sent Done Done false
12056 4253 Acked Flush 0 Sent Done Done false
12057 4254 Acked Flush 0 Sent Done Done false
12058 4255 Acked Read 3 Sent Done Done false
12059 4256 Acked Read 1 Sent Done Done false
12060 4257 Acked Write 3 Sent Done Done false
12061 4258 Acked Write 9 Sent Done Done false
12062 4259 Acked Read 9 Sent Done Done false
12063 4260 Acked Write 8 Sent Done Done false
12064 4261 Acked Read 4 Sent Done Done false
12065 4262 Acked Write 4 Sent Done Done false
12066 4263 Acked Write 1 Sent Done Done false
12067 4264 Acked Write 4 Sent Done Done false
12068 4265 Acked Write 4 Sent Done Done false
12069 4266 Acked Write 7 Sent Done Done false
12070 4267 Acked Write 4 Sent Done Done false
12071 4268 Acked Flush 0 Sent Done Done false
12072 4269 Acked Read 9 Sent Done Done false
12073 4270 Acked Write 4 Sent Done Done false
12074 4271 Acked Flush 0 Sent Done Done false
12075 4272 Acked Write 2 Sent Done Done false
12076 4273 Acked Write 6 Sent Done Done false
12077 4274 Acked Flush 0 Sent Done Done false
12078 4275 Acked Write 1 Sent Done Done false
12079 4276 Acked Write 4 Sent Done Done false
12080 4277 Acked Read 6 Sent Done Done false
12081 4278 Acked Write 8 Sent Done Done false
12082 4279 Acked Write 8 Sent Done Done false
12083 4280 Acked Read 10 Sent Done Done false
12084 4281 Acked Read 2 Sent Done Done false
12085 4282 Acked Read 3 Sent Done Done false
12086 4283 Acked Read 4 Sent Done Done false
12087 4284 Acked Read 9 Sent Done Done false
12088 4285 Acked Read 4 Sent Done Done false
12089 4286 Acked Write 3 Sent Done Done false
12090 4287 Acked Write 8 Sent Done Done false
12091 4288 Acked Write 10 Sent Done Done false
12092 4289 Acked Write 7 Sent Done Done false
12093 4290 Acked Write 8 Sent Done Done false
12094 4291 Acked Read 7 Sent Done Done false
12095 4292 Acked Write 10 Sent Done Done false
12096 4293 Acked Read 7 Sent Done Done false
12097 4294 Acked Read 5 Sent Done Done false
12098 4295 Acked Read 5 Sent Done Done false
12099 4296 Acked Read 6 Sent Done Done false
12100 4297 Acked Read 6 Sent Done Done false
12101 4298 Acked Read 1 Sent Done Done false
12102 4299 Acked Read 2 Sent Done Done false
12103 4300 Acked Write 5 Sent Done Done false
12104 4301 Acked Read 2 Sent Done Done false
12105 4302 Acked Flush 0 Sent Done Done false
12106 4303 Acked Read 5 Sent Done Done false
12107 4304 Acked Write 3 Sent Done Done false
12108 4305 Acked Read 5 Sent Done Done false
12109 4306 Acked Read 10 Sent Done Done false
12110 4307 Acked Read 3 Sent Done Done false
12111 4308 Acked Write 7 Sent Done Done false
12112 4309 Acked Write 8 Sent Done Done false
12113 4310 Acked Read 2 Sent Done Done false
12114 4311 Acked Read 4 Sent Done Done false
12115 4312 Acked Read 7 Sent Done Done false
12116 4313 Acked Write 4 Sent Done Done false
12117 4314 Acked Read 2 Sent Done Done false
12118 4315 Acked Read 8 Sent Done Done false
12119 4316 Acked Read 4 Sent Done Done false
12120 4317 Acked Write 2 Sent Done Done false
12121 4318 Acked Read 1 Sent Done Done false
12122 4319 Acked Flush 0 Sent Done Done false
12123 STATES DS:0 DS:1 DS:2 TOTAL
12124 Sent 302 0 0 302
12125 Done 0 302 302 604
12126 Skipped 0 0 0 0
12127 Error 0 0 0 0
12128 Last Flush: 4017 4319 4319
12129 Downstairs last five completed: 4017 4016 4015 4014 4013
12130 Upstairs last five completed: 4319 4318 4317 4316 4315
12131 CLIENT: Up:0 ds:302 act:2
121322025-01-17T21:13:38.872ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
121332025-01-17T21:13:38.872ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40109) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
121342025-01-17T21:13:38.873ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
121352025-01-17T21:13:38.874ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
121362025-01-17T21:13:38.874ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
121372025-01-17T21:13:38.874ZINFOcrucible: send last flush ID to this DS: 4017 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
121382025-01-17T21:13:38.874ZINFOcrucible: Replied this last flush ID: 4017 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
121392025-01-17T21:13:38.874ZINFOcrucible: [0] client re-new 302 jobs since flush 4017 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
121402025-01-17T21:13:38.874ZINFOcrucible: [0] Replayed 302 jobs since flush: 4017 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
12141 ----------------------------------------------------------------
12142 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
12143 Downstairs last five completed: 4319 4318 4317 4316 4315
12144 Upstairs last five completed: 4319 4318 4317 4316 4315
12145 CLIENT: Up:0 ds:0 act:3
12146 Replay: All jobs finished
12147 Main task has work to do, go find it
12148 got dsc Stop(2)
12149 stop 2
12150 Jan 17 21:13:39.107 INFO[8830] Got stop action so:false kr:false
12151 request completed, latency_us: 307, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 0360d810-d7b5-4eeb-b47b-7b039ea1dd69, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
121522025-01-17T21:13:39.108ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
121532025-01-17T21:13:39.108ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
121542025-01-17T21:13:39.108ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
121552025-01-17T21:13:39.108ZINFOcrucible: client task is exiting = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
12156 Jan 17 21:13:39.108 INFO request completed, latency_us: 148, response_code: 200, uri: /state/cid/2, method: GET, req_id: 7cbcc3ed-bc9a-45b8-91f5-4f2a274129ec, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
12157 [8830] Exited with: ExitStatus(unix_wait_status(9))
12158 [8830][2] reports Exit
12159 Jan 17 21:13:43.110 INFO request completed, latency_us: 186, response_code: 200, uri: /state/cid/2, method: GET, req_id: 5317c775-bc8e-4be4-bbd4-4c135003fbc7, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
12160 001/300 Read block 1852 len 16384
12161 002/300 Write block 996 len 24576 data: 2 2 2 2 2 2
12162 003/300 Read block 3797 len 12288
12163 004/300 Write block 4839 len 4096 data: 2
12164 005/300 Read block 715 len 12288
12165 006/300 Read block 1510 len 28672
12166 007/300 Read block 2422 len 16384
12167 008/300 Write block 1655 len 4096 data: 3
12168 009/300 Read block 2308 len 4096
12169 010/300 Read block 4099 len 16384
12170 011/300 Flush
12171 012/300 Write block 28 len 4096 data: 6
12172 013/300 Read block 2793 len 36864
12173 014/300 Write block 4385 len 16384 data: 2 2 2 2
12174 015/300 Write block 2332 len 16384 data: 6 6 4 4
12175 016/300 Read block 1109 len 12288
12176 017/300 Read block 3486 len 16384
12177 018/300 Write block 4421 len 16384 data: 3 3 3 4
12178 019/300 Write block 1958 len 32768 data: 3 3 3 2 2 2 4 4
12179 020/300 Read block 1952 len 8192
12180 021/300 Read block 221 len 24576
12181 022/300 Write block 4424 len 24576 data: 5 4 3 3 3 4
12182 023/300 Write block 1736 len 12288 data: 3 3 5
12183 024/300 Write block 424 len 12288 data: 7 4 5
12184 025/300 Read block 3505 len 4096
12185 026/300 Write block 530 len 36864 data: 4 5 4 3 3 3 4 3 3
12186 027/300 Read block 72 len 4096
12187 028/300 Read block 1863 len 32768
12188 029/300 Write block 4728 len 12288 data: 3 3 4
12189 030/300 Write block 2183 len 16384 data: 4 4 4 3
12190 031/300 Write block 2774 len 4096 data: 4
12191 032/300 Write block 2943 len 8192 data: 4 5
12192 033/300 Write block 39 len 20480 data: 3 3 3 3 3
12193 034/300 Write block 2097 len 40960 data: 3 4 4 5 4 3 3 4 4 5
12194 035/300 Read block 369 len 24576
12195 036/300 Read block 221 len 40960
12196 037/300 Flush
12197 038/300 Write block 4131 len 16384 data: 3 3 3 3
12198 039/300 Read block 3254 len 16384
12199 040/300 Read block 4893 len 12288
12200 041/300 Write block 2548 len 24576 data: 4 3 4 4 3 4
12201 042/300 Flush
12202 043/300 Flush
12203 044/300 Write block 2461 len 24576 data: 3 3 3 3 4 5
12204 045/300 Write block 2299 len 4096 data: 3
12205 046/300 Write block 4263 len 16384 data: 5 3 4 4
12206 047/300 Write block 4901 len 12288 data: 3 3 3
12207 048/300 Write block 2288 len 36864 data: 3 3 3 3 3 3 4 4 4
12208 049/300 Write block 3163 len 4096 data: 4
12209 050/300 Write block 2826 len 8192 data: 4 4
12210 051/300 Write block 2925 len 28672 data: 3 4 3 4 4 3 3
12211 052/300 Read block 1813 len 16384
12212 053/300 Write block 636 len 4096 data: 2
12213 054/300 Read block 1847 len 24576
12214 055/300 Flush
12215 056/300 Read block 299 len 12288
12216 057/300 Write block 1155 len 24576 data: 6 7 9 8 8 7
12217 058/300 Read block 234 len 32768
12218 059/300 Read block 4084 len 40960
12219 060/300 Read block 1134 len 4096
12220 061/300 Read block 2259 len 36864
12221 062/300 Read block 3910 len 28672
12222 063/300 Write block 3603 len 28672 data: 3 3 3 2 2 3 5
12223 064/300 Read block 3922 len 32768
12224 065/300 Read block 2769 len 16384
12225 066/300 Read block 4118 len 28672
12226 067/300 Read block 532 len 36864
12227 068/300 Read block 2149 len 28672
12228 069/300 Flush
12229 070/300 Write block 2345 len 20480 data: 3 5 4 4 5
12230 071/300 Write block 4360 len 24576 data: 2 2 3 4 4 4
12231 072/300 Read block 3383 len 24576
12232 073/300 Write block 816 len 36864 data: 4 3 3 3 3 3 3 3 4
12233 074/300 Write block 3270 len 16384 data: 2 2 2 2
12234 075/300 Flush
12235 076/300 Flush
12236 077/300 Write block 4470 len 36864 data: 3 3 4 4 4 4 4 3 2
12237 078/300 Flush
12238 079/300 Read block 4658 len 36864
12239 080/300 Read block 1756 len 8192
12240 081/300 Write block 3293 len 8192 data: 3 5
12241 082/300 Read block 442 len 40960
12242 083/300 Read block 125 len 28672
12243 084/300 Write block 4564 len 4096 data: 2
12244 085/300 Write block 1008 len 20480 data: 3 3 3 3 2
12245 086/300 Write block 2027 len 8192 data: 2 2
12246 087/300 Read block 3377 len 4096
12247 088/300 Write block 596 len 28672 data: 3 3 3 3 2 2 2
12248 089/300 Read block 901 len 4096
12249 090/300 Read block 4059 len 32768
12250 091/300 Flush
12251 092/300 Write block 975 len 8192 data: 3 3
12252 093/300 Read block 1618 len 20480
12253 094/300 Read block 633 len 16384
12254 095/300 Read block 1383 len 36864
12255 096/300 Write block 3231 len 8192 data: 3 3
12256 097/300 Read block 4885 len 32768
12257 098/300 Write block 2790 len 28672 data: 2 2 2 2 2 2 2
12258 099/300 Read block 1248 len 36864
12259 100/300 Write block 4715 len 20480 data: 3 3 3 3 3
12260 101/300 Read block 4879 len 12288
12261 102/300 Write block 1273 len 20480 data: 2 2 2 2 3
12262 103/300 Read block 894 len 28672
12263 104/300 Read block 3438 len 4096
12264 105/300 Read block 1964 len 24576
12265 106/300 Read block 3637 len 24576
12266 107/300 Write block 3636 len 4096 data: 2
12267 108/300 Write block 929 len 36864 data: 4 4 4 4 4 5 5 5 5
12268 109/300 Write block 1069 len 12288 data: 3 3 3
12269 110/300 Write block 4279 len 28672 data: 4 4 3 3 3 3 2
12270 111/300 Read block 3956 len 20480
12271 112/300 Read block 4239 len 24576
12272 113/300 Write block 3852 len 40960 data: 3 3 3 4 3 3 3 3 2 2
12273 114/300 Write block 215 len 12288 data: 3 2 2
12274 115/300 Read block 4841 len 20480
12275 116/300 Flush
12276 117/300 Read block 3716 len 36864
12277 118/300 Read block 4755 len 24576
12278 119/300 Read block 4014 len 16384
12279 120/300 Read block 2824 len 32768
12280 121/300 Write block 3829 len 36864 data: 4 4 3 4 3 3 3 3 3
12281 122/300 Write block 1252 len 4096 data: 3
12282 123/300 Read block 4693 len 32768
12283 124/300 Write block 4728 len 4096 data: 4
12284 125/300 Flush
12285 126/300 Read block 682 len 16384
12286 127/300 Write block 3335 len 12288 data: 4 4 4
12287 128/300 Read block 3813 len 40960
12288 129/300 Read block 519 len 36864
12289 130/300 Read block 1637 len 20480
12290 131/300 Read block 4381 len 28672
12291 132/300 Read block 4459 len 4096
12292 133/300 Read block 2622 len 28672
12293 134/300 Write block 2704 len 40960 data: 3 3 5 6 6 4 4 4 3 3
12294 135/300 Read block 898 len 24576
12295 136/300 Read block 1097 len 32768
12296 137/300 Write block 1887 len 28672 data: 6 5 3 3 3 3 2
12297 138/300 Read block 378 len 12288
12298 139/300 Write block 2470 len 28672 data: 3 3 4 4 4 3 3
12299 140/300 Read block 862 len 32768
12300 141/300 Read block 1177 len 8192
12301 142/300 Read block 190 len 16384
12302 143/300 Read block 1779 len 24576
12303 144/300 Write block 1470 len 20480 data: 2 2 3 3 3
12304 145/300 Flush
12305 146/300 Read block 2296 len 8192
12306 147/300 Read block 4377 len 40960
12307 148/300 Read block 1810 len 24576
12308 149/300 Read block 550 len 4096
12309 150/300 Write block 1083 len 24576 data: 4 5 5 5 5 4
12310 151/300 Write block 2614 len 12288 data: 2 2 2
12311 152/300 Read block 1825 len 24576
12312 153/300 Write block 1757 len 40960 data: 3 3 2 2 2 2 2 2 3 3
12313 154/300 Read block 3541 len 36864
12314 155/300 Read block 3925 len 40960
12315 156/300 Flush
12316 157/300 Read block 4912 len 32768
12317 158/300 Read block 2923 len 16384
12318 159/300 Write block 4942 len 4096 data: 4
12319 160/300 Read block 2045 len 20480
12320 161/300 Flush
12321 162/300 Write block 44 len 4096 data: 3
12322 163/300 Write block 4815 len 16384 data: 3 2 2 2
12323 164/300 Write block 4013 len 36864 data: 4 3 3 3 2 2 2 2 3
12324 165/300 Read block 2724 len 40960
12325 166/300 Write block 2230 len 4096 data: 6
12326 167/300 Read block 2220 len 40960
12327 168/300 Read block 174 len 8192
12328 169/300 Write block 4400 len 40960 data: 4 4 4 3 3 2 2 2 3 2
12329 170/300 Write block 2372 len 8192 data: 4 5
12330 171/300 Write block 4028 len 16384 data: 3 3 3 3
12331 172/300 Write block 1947 len 36864 data: 2 2 3 4 5 5 5 5 5
12332 173/300 Write block 1030 len 40960 data: 5 3 3 3 3 4 4 4 2 2
12333 174/300 Read block 1628 len 4096
12334 175/300 Read block 673 len 20480
12335 176/300 Write block 755 len 32768 data: 2 3 2 2 2 3 4 4
12336 177/300 Read block 990 len 12288
12337 178/300 Read block 1402 len 8192
12338 179/300 Write block 3729 len 16384 data: 2 2 2 3
12339 180/300 Read block 3685 len 32768
12340 181/300 Read block 1120 len 36864
12341 182/300 Read block 1507 len 12288
12342 183/300 Read block 4903 len 16384
12343 184/300 Write block 4349 len 8192 data: 3 4
12344 185/300 Read block 1196 len 16384
12345 186/300 Write block 3746 len 16384 data: 3 4 3 3
12346 187/300 Write block 1203 len 36864 data: 4 4 3 3 3 3 3 3 2
12347 188/300 Read block 2851 len 4096
12348 189/300 Write block 520 len 4096 data: 2
12349 190/300 Write block 1913 len 40960 data: 4 4 4 4 5 4 3 4 4 4
12350 191/300 Write block 2791 len 28672 data: 3 3 3 3 3 3 3
12351 192/300 Write block 3244 len 20480 data: 3 3 3 2 2
12352 193/300 Write block 1599 len 24576 data: 3 3 3 3 3 3
12353 194/300 Read block 3192 len 12288
12354 195/300 Write block 2604 len 16384 data: 4 4 4 3
12355 196/300 Read block 2450 len 4096
12356 197/300 Read block 4092 len 20480
12357 198/300 Write block 1788 len 36864 data: 4 4 5 5 5 5 4 5 4
12358 199/300 Write block 2473 len 36864 data: 5 5 4 4 4 5 4 4 4
12359 200/300 Read block 436 len 4096
12360 201/300 Read block 4827 len 24576
12361 202/300 Write block 120 len 4096 data: 4
12362 203/300 Read block 3300 len 16384
12363 204/300 Read block 2287 len 40960
12364 205/300 Read block 2612 len 32768
12365 206/300 Read block 4130 len 20480
12366 207/300 Read block 2214 len 20480
12367 208/300 Read block 1879 len 24576
12368 209/300 Read block 4762 len 16384
12369 210/300 Write block 2218 len 28672 data: 2 2 2 3 3 3 3
12370 211/300 Read block 1816 len 40960
12371 212/300 Read block 88 len 36864
12372 213/300 Read block 398 len 28672
12373 214/300 Write block 4192 len 24576 data: 6 6 5 4 5 3
12374 215/300 Flush
12375 216/300 Write block 1655 len 40960 data: 4 2 2 3 4 4 4 4 4 5
12376 217/300 Read block 4369 len 20480
12377 218/300 Read block 1855 len 24576
12378 219/300 Read block 2350 len 40960
12379 220/300 Read block 2652 len 28672
12380 221/300 Read block 4641 len 12288
12381 222/300 Write block 1963 len 28672 data: 3 5 5 4 4 4 5
12382 223/300 Write block 3 len 8192 data: 2 2
12383 224/300 Read block 4159 len 20480
12384 225/300 Read block 1145 len 28672
12385 226/300 Read block 1714 len 12288
12386 227/300 Read block 1364 len 40960
12387 228/300 Read block 4776 len 40960
12388 229/300 Read block 3763 len 36864
12389 230/300 Read block 1487 len 24576
12390 231/300 Read block 621 len 12288
12391 232/300 Read block 2010 len 16384
12392 233/300 Write block 842 len 24576 data: 5 4 2 3 3 3
12393 234/300 Read block 4565 len 40960
12394 235/300 Read block 2581 len 28672
12395 236/300 Write block 313 len 20480 data: 3 3 3 3 3
12396 237/300 Write block 2265 len 12288 data: 5 5 4
12397 238/300 Write block 3570 len 40960 data: 2 2 2 2 3 3 3 3 2 2
12398 239/300 Read block 255 len 32768
12399 240/300 Write block 3561 len 12288 data: 2 2 2
12400 241/300 Write block 2342 len 12288 data: 2 2 3
12401 242/300 Write block 3430 len 36864 data: 2 2 2 2 2 2 2 2 2
12402 243/300 Read block 3013 len 20480
12403 244/300 Read block 2210 len 28672
12404 245/300 Write block 3204 len 40960 data: 5 5 3 3 2 2 2 3 3 3
12405 246/300 Write block 4632 len 24576 data: 2 2 2 3 4 5
12406 247/300 Read block 323 len 24576
12407 248/300 Write block 878 len 16384 data: 4 4 6 5
12408 249/300 Write block 2260 len 28672 data: 3 3 3 3 5 6 6
12409 250/300 Write block 3798 len 40960 data: 2 2 2 2 2 2 2 2 2 4
12410 251/300 Flush
12411 252/300 Write block 3938 len 12288 data: 4 3 3
12412 253/300 Flush
12413 254/300 Read block 2317 len 32768
12414 255/300 Write block 2859 len 36864 data: 3 3 3 3 3 4 5 5 5
12415 256/300 Read block 1548 len 4096
12416 257/300 Write block 2917 len 20480 data: 3 3 2 2 2
12417 258/300 Flush
12418 259/300 Write block 1660 len 28672 data: 5 5 5 5 6 6 6
12419 260/300 Read block 4363 len 12288
12420 261/300 Write block 1810 len 28672 data: 3 3 3 3 3 3 3
12421 262/300 Read block 2525 len 40960
12422 263/300 Flush
12423 264/300 Read block 2916 len 24576
12424 265/300 Write block 1176 len 8192 data: 3 3
12425 266/300 Read block 1231 len 24576
12426 267/300 Write block 1908 len 32768 data: 2 3 3 3 5 5 5 5
12427 268/300 Read block 1039 len 28672
12428 269/300 Read block 933 len 12288
12429 270/300 Write block 2136 len 4096 data: 2
12430 271/300 Read block 2565 len 40960
12431 272/300 Write block 4856 len 16384 data: 2 2 2 2
12432 273/300 Read block 3053 len 32768
12433 274/300 Write block 877 len 20480 data: 4 5 5 7 6
12434 275/300 Write block 1985 len 16384 data: 2 2 2 2
12435 276/300 Write block 1331 len 28672 data: 4 3 3 2 2 3 3
12436 277/300 Read block 1454 len 8192
12437 278/300 Read block 4054 len 4096
12438 279/300 Write block 3186 len 12288 data: 4 4 4
12439 280/300 Flush
12440 281/300 Write block 3074 len 28672 data: 2 3 3 3 4 5 5
12441 282/300 Read block 4839 len 12288
12442 283/300 Write block 1983 len 28672 data: 2 2 3 3 3 3 5
12443 284/300 Read block 3530 len 4096
12444 285/300 Read block 2832 len 36864
12445 286/300 Read block 1278 len 36864
12446 287/300 Read block 291 len 24576
12447 288/300 Write block 4337 len 32768 data: 4 4 5 3 3 2 3 3
12448 289/300 Read block 4554 len 20480
12449 290/300 Write block 378 len 28672 data: 2 2 2 3 4 4 4
12450 291/300 Write block 208 len 16384 data: 5 5 5 5
12451 292/300 Read block 3318 len 28672
12452 293/300 Read block 4182 len 28672
12453 294/300 Write block 2916 len 36864 data: 4 4 4 3 3 3 3 3 3
12454 295/300 Flush
12455 296/300 Flush
12456 297/300 Read block 246 len 20480
12457 298/300 Read block 450 len 8192
12458 299/300 Read block 3537 len 40960
12459 300/300 Write block 3330 len 20480 data: 3 2 2 3 3
12460 301/300 Read block 820 len 40960
12461 Main task has work to do, go find it
12462 got dsc Start(2)
12463 start 2
12464 Jan 17 21:13:43.354 INFO request completed, latency_us: 202, response_code: 204, uri: /start/cid/2, method: POST, req_id: 21d51e16-196b-4448-9570-81c401a81936, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
12465 Replay: started 2, returned:Ok(())
12466 ----------------------------------------------------------------
12467 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
12468 States: Active Active Offline
12469 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12470 4320 Acked Read 4 Done Done Sent false
12471 4321 Acked Write 6 Done Done Sent false
12472 4322 Acked Read 3 Done Done Sent false
12473 4323 Acked Write 1 Done Done Sent false
12474 4324 Acked Read 3 Done Done Sent false
12475 4325 Acked Read 7 Done Done Sent false
12476 4326 Acked Read 4 Done Done Sent false
12477 4327 Acked Write 1 Done Done Sent false
12478 4328 Acked Read 1 Done Done Sent false
12479 4329 Acked Read 4 Done Done Sent false
12480 4330 Acked Flush 0 Done Done Sent false
12481 4331 Acked Write 1 Done Done Sent false
12482 4332 Acked Read 9 Done Done Sent false
12483 4333 Acked Write 4 Done Done Sent false
12484 4334 Acked Write 4 Done Done Sent false
12485 4335 Acked Read 3 Done Done Sent false
12486 4336 Acked Read 4 Done Done Sent false
12487 4337 Acked Write 4 Done Done Sent false
12488 4338 Acked Write 8 Done Done Sent false
12489 4339 Acked Read 2 Done Done Sent false
12490 4340 Acked Read 6 Done Done Sent false
12491 4341 Acked Write 6 Done Done Sent false
12492 4342 Acked Write 3 Done Done Sent false
12493 4343 Acked Write 3 Done Done Sent false
12494 4344 Acked Read 1 Done Done Sent false
12495 4345 Acked Write 9 Done Done Sent false
12496 4346 Acked Read 1 Done Done Sent false
12497 4347 Acked Read 8 Done Done Sent false
12498 4348 Acked Write 3 Done Done Sent false
12499 4349 Acked Write 4 Done Done Sent false
12500 4350 Acked Write 1 Done Done Sent false
12501 4351 Acked Write 2 Done Done Sent false
12502 4352 Acked Write 5 Done Done Sent false
12503 4353 Acked Write 10 Done Done Sent false
12504 4354 Acked Read 6 Done Done Sent false
12505 4355 Acked Read 10 Done Done Sent false
12506 4356 Acked Flush 0 Done Done Sent false
12507 4357 Acked Write 4 Done Done Sent false
12508 4358 Acked Read 4 Done Done Sent false
12509 4359 Acked Read 3 Done Done Sent false
12510 4360 Acked Write 6 Done Done Sent false
12511 4361 Acked Flush 0 Done Done Sent false
12512 4362 Acked Flush 0 Done Done Sent false
12513 4363 Acked Write 6 Done Done Sent false
12514 4364 Acked Write 1 Done Done Sent false
12515 4365 Acked Write 4 Done Done Sent false
12516 4366 Acked Write 3 Done Done Sent false
12517 4367 Acked Write 9 Done Done Sent false
12518 4368 Acked Write 1 Done Done Sent false
12519 4369 Acked Write 2 Done Done Sent false
12520 4370 Acked Write 7 Done Done Sent false
12521 4371 Acked Read 4 Done Done Sent false
12522 4372 Acked Write 1 Done Done Sent false
12523 4373 Acked Read 6 Done Done Sent false
12524 4374 Acked Flush 0 Done Done Sent false
12525 4375 Acked Read 3 Done Done Sent false
12526 4376 Acked Write 6 Done Done Sent false
12527 4377 Acked Read 8 Done Done Sent false
12528 4378 Acked Read 10 Done Done Sent false
12529 4379 Acked Read 1 Done Done Sent false
12530 4380 Acked Read 9 Done Done Sent false
12531 4381 Acked Read 7 Done Done Sent false
12532 4382 Acked Write 7 Done Done Sent false
12533 4383 Acked Read 8 Done Done Sent false
12534 4384 Acked Read 4 Done Done Sent false
12535 4385 Acked Read 7 Done Done Sent false
12536 4386 Acked Read 9 Done Done Sent false
12537 4387 Acked Read 7 Done Done Sent false
12538 4388 Acked Flush 0 Done Done Sent false
12539 4389 Acked Write 5 Done Done Sent false
12540 4390 Acked Write 6 Done Done Sent false
12541 4391 Acked Read 6 Done Done Sent false
12542 4392 Acked Write 9 Done Done Sent false
12543 4393 Acked Write 4 Done Done Sent false
12544 4394 Acked Flush 0 Done Done Sent false
12545 4395 Acked Flush 0 Done Done Sent false
12546 4396 Acked Write 9 Done Done Sent false
12547 4397 Acked Flush 0 Done Done Sent false
12548 4398 Acked Read 9 Done Done Sent false
12549 4399 Acked Read 2 Done Done Sent false
12550 4400 Acked Write 2 Done Done Sent false
12551 4401 Acked Read 10 Done Done Sent false
12552 4402 Acked Read 7 Done Done Sent false
12553 4403 Acked Write 1 Done Done Sent false
12554 4404 Acked Write 5 Done Done Sent false
12555 4405 Acked Write 2 Done Done Sent false
12556 4406 Acked Read 1 Done Done Sent false
12557 4407 Acked Write 7 Done Done Sent false
12558 4408 Acked Read 1 Done Done Sent false
12559 4409 Acked Read 8 Done Done Sent false
12560 4410 Acked Flush 0 Done Done Sent false
12561 4411 Acked Write 2 Done Done Sent false
12562 4412 Acked Read 5 Done Done Sent false
12563 4413 Acked Read 4 Done Done Sent false
12564 4414 Acked Read 9 Done Done Sent false
12565 4415 Acked Write 2 Done Done Sent false
12566 4416 Acked Read 8 Done Done Sent false
12567 4417 Acked Write 7 Done Done Sent false
12568 4418 Acked Read 9 Done Done Sent false
12569 4419 Acked Write 5 Done Done Sent false
12570 4420 Acked Read 3 Done Done Sent false
12571 4421 Acked Write 5 Done Done Sent false
12572 4422 Acked Read 7 Done Done Sent false
12573 4423 Acked Read 1 Done Done Sent false
12574 4424 Acked Read 6 Done Done Sent false
12575 4425 Acked Read 6 Done Done Sent false
12576 4426 Acked Write 1 Done Done Sent false
12577 4427 Acked Write 9 Done Done Sent false
12578 4428 Acked Write 3 Done Done Sent false
12579 4429 Acked Write 7 Done Done Sent false
12580 4430 Acked Read 5 Done Done Sent false
12581 4431 Acked Read 6 Done Done Sent false
12582 4432 Acked Write 10 Done Done Sent false
12583 4433 Acked Write 3 Done Done Sent false
12584 4434 Acked Read 5 Done Done Sent false
12585 4435 Acked Flush 0 Done Done Sent false
12586 4436 Acked Read 9 Done Done Sent false
12587 4437 Acked Read 6 Done Done Sent false
12588 4438 Acked Read 4 Done Done Sent false
12589 4439 Acked Read 8 Done Done Sent false
12590 4440 Acked Write 9 Done Done Sent false
12591 4441 Acked Write 1 Done Done Sent false
12592 4442 Acked Read 8 Done Done Sent false
12593 4443 Acked Write 1 Done Done Sent false
12594 4444 Acked Flush 0 Done Done Sent false
12595 4445 Acked Read 4 Done Done Sent false
12596 4446 Acked Write 3 Done Done Sent false
12597 4447 Acked Read 10 Done Done Sent false
12598 4448 Acked Read 9 Done Done Sent false
12599 4449 Acked Read 5 Done Done Sent false
12600 4450 Acked Read 7 Done Done Sent false
12601 4451 Acked Read 1 Done Done Sent false
12602 4452 Acked Read 7 Done Done Sent false
12603 4453 Acked Write 10 Done Done Sent false
12604 4454 Acked Read 6 Done Done Sent false
12605 4455 Acked Read 8 Done Done Sent false
12606 4456 Acked Write 7 Done Done Sent false
12607 4457 Acked Read 3 Done Done Sent false
12608 4458 Acked Write 7 Done Done Sent false
12609 4459 Acked Read 8 Done Done Sent false
12610 4460 Acked Read 2 Done Done Sent false
12611 4461 Acked Read 4 Done Done Sent false
12612 4462 Acked Read 6 Done Done Sent false
12613 4463 Acked Write 5 Done Done Sent false
12614 4464 Acked Flush 0 Done Done Sent false
12615 4465 Acked Read 2 Done Done Sent false
12616 4466 Acked Read 10 Done Done Sent false
12617 4467 Acked Read 6 Done Done Sent false
12618 4468 Acked Read 1 Done Done Sent false
12619 4469 Acked Write 6 Done Done Sent false
12620 4470 Acked Write 3 Done Done Sent false
12621 4471 Acked Read 6 Done Done Sent false
12622 4472 Acked Write 10 Done Done Sent false
12623 4473 Acked Read 9 Done Done Sent false
12624 4474 Acked Read 10 Done Done Sent false
12625 4475 Acked Flush 0 Done Done Sent false
12626 4476 Acked Read 8 Done Done Sent false
12627 4477 Acked Read 4 Done Done Sent false
12628 4478 Acked Write 1 Done Done Sent false
12629 4479 Acked Read 5 Done Done Sent false
12630 4480 Acked Flush 0 Done Done Sent false
12631 4481 Acked Write 1 Done Done Sent false
12632 4482 Acked Write 4 Done Done Sent false
12633 4483 Acked Write 9 Done Done Sent false
12634 4484 Acked Read 10 Done Done Sent false
12635 4485 Acked Write 1 Done Done Sent false
12636 4486 Acked Read 10 Done Done Sent false
12637 4487 Acked Read 2 Done Done Sent false
12638 4488 Acked Write 10 Done Done Sent false
12639 4489 Acked Write 2 Done Done Sent false
12640 4490 Acked Write 4 Done Done Sent false
12641 4491 Acked Write 9 Done Done Sent false
12642 4492 Acked Write 10 Done Done Sent false
12643 4493 Acked Read 1 Done Done Sent false
12644 4494 Acked Read 5 Done Done Sent false
12645 4495 Acked Write 8 Done Done Sent false
12646 4496 Acked Read 3 Done Done Sent false
12647 4497 Acked Read 2 Done Done Sent false
12648 4498 Acked Write 4 Done Done Sent false
12649 4499 Acked Read 8 Done Done Sent false
12650 4500 Acked Read 9 Done Done Sent false
12651 4501 Acked Read 3 Done Done Sent false
12652 4502 Acked Read 4 Done Done Sent false
12653 4503 Acked Write 2 Done Done Sent false
12654 4504 Acked Read 4 Done Done Sent false
12655 4505 Acked Write 4 Done Done Sent false
12656 4506 Acked Write 9 Done Done Sent false
12657 4507 Acked Read 1 Done Done Sent false
12658 4508 Acked Write 1 Done Done Sent false
12659 4509 Acked Write 10 Done Done Sent false
12660 4510 Acked Write 7 Done Done Sent false
12661 4511 Acked Write 5 Done Done Sent false
12662 4512 Acked Write 6 Done Done Sent false
12663 4513 Acked Read 3 Done Done Sent false
12664 4514 Acked Write 4 Done Done Sent false
12665 4515 Acked Read 1 Done Done Sent false
12666 4516 Acked Read 5 Done Done Sent false
12667 4517 Acked Write 9 Done Done Sent false
12668 4518 Acked Write 9 Done Done Sent false
12669 4519 Acked Read 1 Done Done Sent false
12670 4520 Acked Read 6 Done Done Sent false
12671 4521 Acked Write 1 Done Done Sent false
12672 4522 Acked Read 4 Done Done Sent false
12673 4523 Acked Read 10 Done Done Sent false
12674 4524 Acked Read 8 Done Done Sent false
12675 4525 Acked Read 5 Done Done Sent false
12676 4526 Acked Read 5 Done Done Sent false
12677 4527 Acked Read 6 Done Done Sent false
12678 4528 Acked Read 4 Done Done Sent false
12679 4529 Acked Write 7 Done Done Sent false
12680 4530 Acked Read 10 Done Done Sent false
12681 4531 Acked Read 9 Done Done Sent false
12682 4532 Acked Read 7 Done Done Sent false
12683 4533 Acked Write 6 Done Done Sent false
12684 4534 Acked Flush 0 Done Done Sent false
12685 4535 Acked Write 10 Done Done Sent false
12686 4536 Acked Read 5 Done Done Sent false
12687 4537 Acked Read 6 Done Done Sent false
12688 4538 Acked Read 10 Done Done Sent false
12689 4539 Acked Read 7 Done Done Sent false
12690 4540 Acked Read 3 Done Done Sent false
12691 4541 Acked Write 7 Done Done Sent false
12692 4542 Acked Write 2 Done Done Sent false
12693 4543 Acked Read 5 Done Done Sent false
12694 4544 Acked Read 7 Done Done Sent false
12695 4545 Acked Read 3 Done Done Sent false
12696 4546 Acked Read 10 Done Done Sent false
12697 4547 Acked Read 10 Done Done Sent false
12698 4548 Acked Read 9 Done Done Sent false
12699 4549 Acked Read 6 Done Done Sent false
12700 4550 Acked Read 3 Done Done Sent false
12701 4551 Acked Read 4 Done Done Sent false
12702 4552 Acked Write 6 Done Done Sent false
12703 4553 Acked Read 10 Done Done Sent false
12704 4554 Acked Read 7 Done Done Sent false
12705 4555 Acked Write 5 Done Done Sent false
12706 4556 Acked Write 3 Done Done Sent false
12707 4557 Acked Write 10 Done Done Sent false
12708 4558 Acked Read 8 Done Done Sent false
12709 4559 Acked Write 3 Done Done Sent false
12710 4560 Acked Write 3 Done Done Sent false
12711 4561 Acked Write 9 Done Done Sent false
12712 4562 Acked Read 5 Done Done Sent false
12713 4563 Acked Read 7 Done Done Sent false
12714 4564 Acked Write 10 Done Done Sent false
12715 4565 Acked Write 6 Done Done Sent false
12716 4566 Acked Read 6 Done Done Sent false
12717 4567 Acked Write 4 Done Done Sent false
12718 4568 Acked Write 7 Done Done Sent false
12719 4569 Acked Write 10 Done Done Sent false
12720 4570 Acked Flush 0 Done Done Sent false
12721 4571 Acked Write 3 Done Done Sent false
12722 4572 Acked Flush 0 Done Done Sent false
12723 4573 Acked Read 8 Done Done Sent false
12724 4574 Acked Write 9 Done Done Sent false
12725 4575 Acked Read 1 Done Done Sent false
12726 4576 Acked Write 5 Done Done Sent false
12727 4577 Acked Flush 0 Done Done Sent false
12728 4578 Acked Write 7 Done Done Sent false
12729 4579 Acked Read 3 Done Done Sent false
12730 4580 Acked Write 7 Done Done Sent false
12731 4581 Acked Read 10 Done Done Sent false
12732 4582 Acked Flush 0 Done Done Sent false
12733 4583 Acked Read 6 Done Done Sent false
12734 4584 Acked Write 2 Done Done Sent false
12735 4585 Acked Read 6 Done Done Sent false
12736 4586 Acked Write 8 Done Done Sent false
12737 4587 Acked Read 7 Done Done Sent false
12738 4588 Acked Read 3 Done Done Sent false
12739 4589 Acked Write 1 Done Done Sent false
12740 4590 Acked Read 10 Done Done Sent false
12741 4591 Acked Write 4 Done Done Sent false
12742 4592 Acked Read 8 Done Done Sent false
12743 4593 Acked Write 5 Done Done Sent false
12744 4594 Acked Write 4 Done Done Sent false
12745 4595 Acked Write 7 Done Done Sent false
12746 4596 Acked Read 2 Done Done Sent false
12747 4597 Acked Read 1 Done Done Sent false
12748 4598 Acked Write 3 Done Done Sent false
12749 4599 Acked Flush 0 Done Done Sent false
12750 4600 Acked Write 7 Done Done Sent false
12751 4601 Acked Read 3 Done Done Sent false
12752 4602 Acked Write 7 Done Done Sent false
12753 4603 Acked Read 1 Done Done Sent false
12754 4604 Acked Read 9 Done Done Sent false
12755 4605 Acked Read 9 Done Done Sent false
12756 4606 Acked Read 6 Done Done Sent false
12757 4607 Acked Write 8 Done Done Sent false
12758 4608 Acked Read 5 Done Done Sent false
12759 4609 Acked Write 7 Done Done Sent false
12760 4610 Acked Write 4 Done Done Sent false
12761 4611 Acked Read 7 Done Done Sent false
12762 4612 Acked Read 7 Done Done Sent false
12763 4613 Acked Write 9 Done Done Sent false
12764 4614 Acked Flush 0 Done Done Sent false
12765 4615 Acked Flush 0 Done Done Sent false
12766 4616 Acked Read 5 Done Done Sent false
12767 4617 Acked Read 2 Done Done Sent false
12768 4618 Acked Read 10 Done Done Sent false
12769 4619 Acked Write 5 Done Done Sent false
12770 4620 Acked Read 10 Done Done Sent false
12771 STATES DS:0 DS:1 DS:2 TOTAL
12772 Sent 0 0 301 301
12773 Done 301 301 0 602
12774 Skipped 0 0 0 0
12775 Error 0 0 0 0
12776 Last Flush: 4615 4615 4319
12777 Downstairs last five completed: 4319 4318 4317 4316 4315
12778 Upstairs last five completed: 4620 4619 4618 4617 4616
12779 CLIENT: Up:0 ds:301 act:2
12780 ----------------------------------------------------------------
12781 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
12782 States: Active Active Offline
12783 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12784 4320 Acked Read 4 Done Done Sent false
12785 4321 Acked Write 6 Done Done Sent false
12786 4322 Acked Read 3 Done Done Sent false
12787 4323 Acked Write 1 Done Done Sent false
12788 4324 Acked Read 3 Done Done Sent false
12789 4325 Acked Read 7 Done Done Sent false
12790 4326 Acked Read 4 Done Done Sent false
12791 4327 Acked Write 1 Done Done Sent false
12792 4328 Acked Read 1 Done Done Sent false
12793 4329 Acked Read 4 Done Done Sent false
12794 4330 Acked Flush 0 Done Done Sent false
12795 4331 Acked Write 1 Done Done Sent false
12796 4332 Acked Read 9 Done Done Sent false
12797 4333 Acked Write 4 Done Done Sent false
12798 4334 Acked Write 4 Done Done Sent false
12799 4335 Acked Read 3 Done Done Sent false
12800 4336 Acked Read 4 Done Done Sent false
12801 4337 Acked Write 4 Done Done Sent false
12802 4338 Acked Write 8 Done Done Sent false
12803 4339 Acked Read 2 Done Done Sent false
12804 4340 Acked Read 6 Done Done Sent false
12805 4341 Acked Write 6 Done Done Sent false
12806 4342 Acked Write 3 Done Done Sent false
12807 4343 Acked Write 3 Done Done Sent false
12808 4344 Acked Read 1 Done Done Sent false
12809 4345 Acked Write 9 Done Done Sent false
12810 4346 Acked Read 1 Done Done Sent false
12811 4347 Acked Read 8 Done Done Sent false
12812 4348 Acked Write 3 Done Done Sent false
12813 4349 Acked Write 4 Done Done Sent false
12814 4350 Acked Write 1 Done Done Sent false
12815 4351 Acked Write 2 Done Done Sent false
12816 4352 Acked Write 5 Done Done Sent false
12817 4353 Acked Write 10 Done Done Sent false
12818 4354 Acked Read 6 Done Done Sent false
12819 4355 Acked Read 10 Done Done Sent false
12820 4356 Acked Flush 0 Done Done Sent false
12821 4357 Acked Write 4 Done Done Sent false
12822 4358 Acked Read 4 Done Done Sent false
12823 4359 Acked Read 3 Done Done Sent false
12824 4360 Acked Write 6 Done Done Sent false
12825 4361 Acked Flush 0 Done Done Sent false
12826 4362 Acked Flush 0 Done Done Sent false
12827 4363 Acked Write 6 Done Done Sent false
12828 4364 Acked Write 1 Done Done Sent false
12829 4365 Acked Write 4 Done Done Sent false
12830 4366 Acked Write 3 Done Done Sent false
12831 4367 Acked Write 9 Done Done Sent false
12832 4368 Acked Write 1 Done Done Sent false
12833 4369 Acked Write 2 Done Done Sent false
12834 4370 Acked Write 7 Done Done Sent false
12835 4371 Acked Read 4 Done Done Sent false
12836 4372 Acked Write 1 Done Done Sent false
12837 4373 Acked Read 6 Done Done Sent false
12838 4374 Acked Flush 0 Done Done Sent false
12839 4375 Acked Read 3 Done Done Sent false
12840 4376 Acked Write 6 Done Done Sent false
12841 4377 Acked Read 8 Done Done Sent false
12842 4378 Acked Read 10 Done Done Sent false
12843 4379 Acked Read 1 Done Done Sent false
12844 4380 Acked Read 9 Done Done Sent false
12845 4381 Acked Read 7 Done Done Sent false
12846 4382 Acked Write 7 Done Done Sent false
12847 4383 Acked Read 8 Done Done Sent false
12848 4384 Acked Read 4 Done Done Sent false
12849 4385 Acked Read 7 Done Done Sent false
12850 4386 Acked Read 9 Done Done Sent false
12851 4387 Acked Read 7 Done Done Sent false
12852 4388 Acked Flush 0 Done Done Sent false
12853 4389 Acked Write 5 Done Done Sent false
12854 4390 Acked Write 6 Done Done Sent false
12855 4391 Acked Read 6 Done Done Sent false
12856 4392 Acked Write 9 Done Done Sent false
12857 4393 Acked Write 4 Done Done Sent false
12858 4394 Acked Flush 0 Done Done Sent false
12859 4395 Acked Flush 0 Done Done Sent false
12860 4396 Acked Write 9 Done Done Sent false
12861 4397 Acked Flush 0 Done Done Sent false
12862 4398 Acked Read 9 Done Done Sent false
12863 4399 Acked Read 2 Done Done Sent false
12864 4400 Acked Write 2 Done Done Sent false
12865 4401 Acked Read 10 Done Done Sent false
12866 4402 Acked Read 7 Done Done Sent false
12867 4403 Acked Write 1 Done Done Sent false
12868 4404 Acked Write 5 Done Done Sent false
12869 4405 Acked Write 2 Done Done Sent false
12870 4406 Acked Read 1 Done Done Sent false
12871 4407 Acked Write 7 Done Done Sent false
12872 4408 Acked Read 1 Done Done Sent false
12873 4409 Acked Read 8 Done Done Sent false
12874 4410 Acked Flush 0 Done Done Sent false
12875 4411 Acked Write 2 Done Done Sent false
12876 4412 Acked Read 5 Done Done Sent false
12877 4413 Acked Read 4 Done Done Sent false
12878 4414 Acked Read 9 Done Done Sent false
12879 4415 Acked Write 2 Done Done Sent false
12880 4416 Acked Read 8 Done Done Sent false
12881 4417 Acked Write 7 Done Done Sent false
12882 4418 Acked Read 9 Done Done Sent false
12883 4419 Acked Write 5 Done Done Sent false
12884 4420 Acked Read 3 Done Done Sent false
12885 4421 Acked Write 5 Done Done Sent false
12886 4422 Acked Read 7 Done Done Sent false
12887 4423 Acked Read 1 Done Done Sent false
12888 4424 Acked Read 6 Done Done Sent false
12889 4425 Acked Read 6 Done Done Sent false
12890 4426 Acked Write 1 Done Done Sent false
12891 4427 Acked Write 9 Done Done Sent false
12892 4428 Acked Write 3 Done Done Sent false
12893 4429 Acked Write 7 Done Done Sent false
12894 4430 Acked Read 5 Done Done Sent false
12895 4431 Acked Read 6 Done Done Sent false
12896 4432 Acked Write 10 Done Done Sent false
12897 4433 Acked Write 3 Done Done Sent false
12898 4434 Acked Read 5 Done Done Sent false
12899 4435 Acked Flush 0 Done Done Sent false
12900 4436 Acked Read 9 Done Done Sent false
12901 4437 Acked Read 6 Done Done Sent false
12902 4438 Acked Read 4 Done Done Sent false
12903 4439 Acked Read 8 Done Done Sent false
12904 4440 Acked Write 9 Done Done Sent false
12905 4441 Acked Write 1 Done Done Sent false
12906 4442 Acked Read 8 Done Done Sent false
12907 4443 Acked Write 1 Done Done Sent false
12908 4444 Acked Flush 0 Done Done Sent false
12909 4445 Acked Read 4 Done Done Sent false
12910 4446 Acked Write 3 Done Done Sent false
12911 4447 Acked Read 10 Done Done Sent false
12912 4448 Acked Read 9 Done Done Sent false
12913 4449 Acked Read 5 Done Done Sent false
12914 4450 Acked Read 7 Done Done Sent false
12915 4451 Acked Read 1 Done Done Sent false
12916 4452 Acked Read 7 Done Done Sent false
12917 4453 Acked Write 10 Done Done Sent false
12918 4454 Acked Read 6 Done Done Sent false
12919 4455 Acked Read 8 Done Done Sent false
12920 4456 Acked Write 7 Done Done Sent false
12921 4457 Acked Read 3 Done Done Sent false
12922 4458 Acked Write 7 Done Done Sent false
12923 4459 Acked Read 8 Done Done Sent false
12924 4460 Acked Read 2 Done Done Sent false
12925 4461 Acked Read 4 Done Done Sent false
12926 4462 Acked Read 6 Done Done Sent false
12927 4463 Acked Write 5 Done Done Sent false
12928 4464 Acked Flush 0 Done Done Sent false
12929 4465 Acked Read 2 Done Done Sent false
12930 4466 Acked Read 10 Done Done Sent false
12931 4467 Acked Read 6 Done Done Sent false
12932 4468 Acked Read 1 Done Done Sent false
12933 4469 Acked Write 6 Done Done Sent false
12934 4470 Acked Write 3 Done Done Sent false
12935 4471 Acked Read 6 Done Done Sent false
12936 4472 Acked Write 10 Done Done Sent false
12937 4473 Acked Read 9 Done Done Sent false
12938 4474 Acked Read 10 Done Done Sent false
12939 4475 Acked Flush 0 Done Done Sent false
12940 4476 Acked Read 8 Done Done Sent false
12941 4477 Acked Read 4 Done Done Sent false
12942 4478 Acked Write 1 Done Done Sent false
12943 4479 Acked Read 5 Done Done Sent false
12944 4480 Acked Flush 0 Done Done Sent false
12945 4481 Acked Write 1 Done Done Sent false
12946 4482 Acked Write 4 Done Done Sent false
12947 4483 Acked Write 9 Done Done Sent false
12948 4484 Acked Read 10 Done Done Sent false
12949 4485 Acked Write 1 Done Done Sent false
12950 4486 Acked Read 10 Done Done Sent false
12951 4487 Acked Read 2 Done Done Sent false
12952 4488 Acked Write 10 Done Done Sent false
12953 4489 Acked Write 2 Done Done Sent false
12954 4490 Acked Write 4 Done Done Sent false
12955 4491 Acked Write 9 Done Done Sent false
12956 4492 Acked Write 10 Done Done Sent false
12957 4493 Acked Read 1 Done Done Sent false
12958 4494 Acked Read 5 Done Done Sent false
12959 4495 Acked Write 8 Done Done Sent false
12960 4496 Acked Read 3 Done Done Sent false
12961 4497 Acked Read 2 Done Done Sent false
12962 4498 Acked Write 4 Done Done Sent false
12963 4499 Acked Read 8 Done Done Sent false
12964 4500 Acked Read 9 Done Done Sent false
12965 4501 Acked Read 3 Done Done Sent false
12966 4502 Acked Read 4 Done Done Sent false
12967 4503 Acked Write 2 Done Done Sent false
12968 4504 Acked Read 4 Done Done Sent false
12969 4505 Acked Write 4 Done Done Sent false
12970 4506 Acked Write 9 Done Done Sent false
12971 4507 Acked Read 1 Done Done Sent false
12972 4508 Acked Write 1 Done Done Sent false
12973 4509 Acked Write 10 Done Done Sent false
12974 4510 Acked Write 7 Done Done Sent false
12975 4511 Acked Write 5 Done Done Sent false
12976 4512 Acked Write 6 Done Done Sent false
12977 4513 Acked Read 3 Done Done Sent false
12978 4514 Acked Write 4 Done Done Sent false
12979 4515 Acked Read 1 Done Done Sent false
12980 4516 Acked Read 5 Done Done Sent false
12981 4517 Acked Write 9 Done Done Sent false
12982 4518 Acked Write 9 Done Done Sent false
12983 4519 Acked Read 1 Done Done Sent false
12984 4520 Acked Read 6 Done Done Sent false
12985 4521 Acked Write 1 Done Done Sent false
12986 4522 Acked Read 4 Done Done Sent false
12987 4523 Acked Read 10 Done Done Sent false
12988 4524 Acked Read 8 Done Done Sent false
12989 4525 Acked Read 5 Done Done Sent false
12990 4526 Acked Read 5 Done Done Sent false
12991 4527 Acked Read 6 Done Done Sent false
12992 4528 Acked Read 4 Done Done Sent false
12993 4529 Acked Write 7 Done Done Sent false
12994 4530 Acked Read 10 Done Done Sent false
12995 4531 Acked Read 9 Done Done Sent false
12996 4532 Acked Read 7 Done Done Sent false
12997 4533 Acked Write 6 Done Done Sent false
12998 4534 Acked Flush 0 Done Done Sent false
12999 4535 Acked Write 10 Done Done Sent false
13000 4536 Acked Read 5 Done Done Sent false
13001 4537 Acked Read 6 Done Done Sent false
13002 4538 Acked Read 10 Done Done Sent false
13003 4539 Acked Read 7 Done Done Sent false
13004 4540 Acked Read 3 Done Done Sent false
13005 4541 Acked Write 7 Done Done Sent false
13006 4542 Acked Write 2 Done Done Sent false
13007 4543 Acked Read 5 Done Done Sent false
13008 4544 Acked Read 7 Done Done Sent false
13009 4545 Acked Read 3 Done Done Sent false
13010 4546 Acked Read 10 Done Done Sent false
13011 4547 Acked Read 10 Done Done Sent false
13012 4548 Acked Read 9 Done Done Sent false
13013 4549 Acked Read 6 Done Done Sent false
13014 4550 Acked Read 3 Done Done Sent false
13015 4551 Acked Read 4 Done Done Sent false
13016 4552 Acked Write 6 Done Done Sent false
13017 4553 Acked Read 10 Done Done Sent false
13018 4554 Acked Read 7 Done Done Sent false
13019 4555 Acked Write 5 Done Done Sent false
13020 4556 Acked Write 3 Done Done Sent false
13021 4557 Acked Write 10 Done Done Sent false
13022 4558 Acked Read 8 Done Done Sent false
13023 4559 Acked Write 3 Done Done Sent false
13024 4560 Acked Write 3 Done Done Sent false
13025 4561 Acked Write 9 Done Done Sent false
13026 4562 Acked Read 5 Done Done Sent false
13027 4563 Acked Read 7 Done Done Sent false
13028 4564 Acked Write 10 Done Done Sent false
13029 4565 Acked Write 6 Done Done Sent false
13030 4566 Acked Read 6 Done Done Sent false
13031 4567 Acked Write 4 Done Done Sent false
13032 4568 Acked Write 7 Done Done Sent false
13033 4569 Acked Write 10 Done Done Sent false
13034 4570 Acked Flush 0 Done Done Sent false
13035 4571 Acked Write 3 Done Done Sent false
13036 4572 Acked Flush 0 Done Done Sent false
13037 4573 Acked Read 8 Done Done Sent false
13038 4574 Acked Write 9 Done Done Sent false
13039 4575 Acked Read 1 Done Done Sent false
13040 4576 Acked Write 5 Done Done Sent false
13041 4577 Acked Flush 0 Done Done Sent false
13042 4578 Acked Write 7 Done Done Sent false
13043 4579 Acked Read 3 Done Done Sent false
13044 4580 Acked Write 7 Done Done Sent false
13045 4581 Acked Read 10 Done Done Sent false
13046 4582 Acked Flush 0 Done Done Sent false
13047 4583 Acked Read 6 Done Done Sent false
13048 4584 Acked Write 2 Done Done Sent false
13049 4585 Acked Read 6 Done Done Sent false
13050 4586 Acked Write 8 Done Done Sent false
13051 4587 Acked Read 7 Done Done Sent false
13052 4588 Acked Read 3 Done Done Sent false
13053 4589 Acked Write 1 Done Done Sent false
13054 4590 Acked Read 10 Done Done Sent false
13055 4591 Acked Write 4 Done Done Sent false
13056 4592 Acked Read 8 Done Done Sent false
13057 4593 Acked Write 5 Done Done Sent false
13058 4594 Acked Write 4 Done Done Sent false
13059 4595 Acked Write 7 Done Done Sent false
13060 4596 Acked Read 2 Done Done Sent false
13061 4597 Acked Read 1 Done Done Sent false
13062 4598 Acked Write 3 Done Done Sent false
13063 4599 Acked Flush 0 Done Done Sent false
13064 4600 Acked Write 7 Done Done Sent false
13065 4601 Acked Read 3 Done Done Sent false
13066 4602 Acked Write 7 Done Done Sent false
13067 4603 Acked Read 1 Done Done Sent false
13068 4604 Acked Read 9 Done Done Sent false
13069 4605 Acked Read 9 Done Done Sent false
13070 4606 Acked Read 6 Done Done Sent false
13071 4607 Acked Write 8 Done Done Sent false
13072 4608 Acked Read 5 Done Done Sent false
13073 4609 Acked Write 7 Done Done Sent false
13074 4610 Acked Write 4 Done Done Sent false
13075 4611 Acked Read 7 Done Done Sent false
13076 4612 Acked Read 7 Done Done Sent false
13077 4613 Acked Write 9 Done Done Sent false
13078 4614 Acked Flush 0 Done Done Sent false
13079 4615 Acked Flush 0 Done Done Sent false
13080 4616 Acked Read 5 Done Done Sent false
13081 4617 Acked Read 2 Done Done Sent false
13082 4618 Acked Read 10 Done Done Sent false
13083 4619 Acked Write 5 Done Done Sent false
13084 4620 Acked Read 10 Done Done Sent false
13085 4621 Acked Flush 0 Done Done Sent false
13086 STATES DS:0 DS:1 DS:2 TOTAL
13087 Sent 0 0 302 302
13088 Done 302 302 0 604
13089 Skipped 0 0 0 0
13090 Error 0 0 0 0
13091 Last Flush: 4621 4621 4319
13092 Downstairs last five completed: 4319 4318 4317 4316 4315
13093 Upstairs last five completed: 4621 4620 4619 4618 4617
13094 CLIENT: Up:0 ds:302 act:2
130952025-01-17T21:13:49.110ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
130962025-01-17T21:13:49.110ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
130972025-01-17T21:13:49.110ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
130982025-01-17T21:13:49.110ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
130992025-01-17T21:13:49.110ZINFOcrucible: client task is exiting = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
13100 ----------------------------------------------------------------
13101 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13102 States: Active Active Offline
13103 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13104 4320 Acked Read 4 Done Done Sent false
13105 4321 Acked Write 6 Done Done Sent false
13106 4322 Acked Read 3 Done Done Sent false
13107 4323 Acked Write 1 Done Done Sent false
13108 4324 Acked Read 3 Done Done Sent false
13109 4325 Acked Read 7 Done Done Sent false
13110 4326 Acked Read 4 Done Done Sent false
13111 4327 Acked Write 1 Done Done Sent false
13112 4328 Acked Read 1 Done Done Sent false
13113 4329 Acked Read 4 Done Done Sent false
13114 4330 Acked Flush 0 Done Done Sent false
13115 4331 Acked Write 1 Done Done Sent false
13116 4332 Acked Read 9 Done Done Sent false
13117 4333 Acked Write 4 Done Done Sent false
13118 4334 Acked Write 4 Done Done Sent false
13119 4335 Acked Read 3 Done Done Sent false
13120 4336 Acked Read 4 Done Done Sent false
13121 4337 Acked Write 4 Done Done Sent false
13122 4338 Acked Write 8 Done Done Sent false
13123 4339 Acked Read 2 Done Done Sent false
13124 4340 Acked Read 6 Done Done Sent false
13125 4341 Acked Write 6 Done Done Sent false
13126 4342 Acked Write 3 Done Done Sent false
13127 4343 Acked Write 3 Done Done Sent false
13128 4344 Acked Read 1 Done Done Sent false
13129 4345 Acked Write 9 Done Done Sent false
13130 4346 Acked Read 1 Done Done Sent false
13131 4347 Acked Read 8 Done Done Sent false
13132 4348 Acked Write 3 Done Done Sent false
13133 4349 Acked Write 4 Done Done Sent false
13134 4350 Acked Write 1 Done Done Sent false
13135 4351 Acked Write 2 Done Done Sent false
13136 4352 Acked Write 5 Done Done Sent false
13137 4353 Acked Write 10 Done Done Sent false
13138 4354 Acked Read 6 Done Done Sent false
13139 4355 Acked Read 10 Done Done Sent false
13140 4356 Acked Flush 0 Done Done Sent false
13141 4357 Acked Write 4 Done Done Sent false
13142 4358 Acked Read 4 Done Done Sent false
13143 4359 Acked Read 3 Done Done Sent false
13144 4360 Acked Write 6 Done Done Sent false
13145 4361 Acked Flush 0 Done Done Sent false
13146 4362 Acked Flush 0 Done Done Sent false
13147 4363 Acked Write 6 Done Done Sent false
13148 4364 Acked Write 1 Done Done Sent false
13149 4365 Acked Write 4 Done Done Sent false
13150 4366 Acked Write 3 Done Done Sent false
13151 4367 Acked Write 9 Done Done Sent false
13152 4368 Acked Write 1 Done Done Sent false
13153 4369 Acked Write 2 Done Done Sent false
13154 4370 Acked Write 7 Done Done Sent false
13155 4371 Acked Read 4 Done Done Sent false
13156 4372 Acked Write 1 Done Done Sent false
13157 4373 Acked Read 6 Done Done Sent false
13158 4374 Acked Flush 0 Done Done Sent false
13159 4375 Acked Read 3 Done Done Sent false
13160 4376 Acked Write 6 Done Done Sent false
13161 4377 Acked Read 8 Done Done Sent false
13162 4378 Acked Read 10 Done Done Sent false
13163 4379 Acked Read 1 Done Done Sent false
13164 4380 Acked Read 9 Done Done Sent false
13165 4381 Acked Read 7 Done Done Sent false
13166 4382 Acked Write 7 Done Done Sent false
13167 4383 Acked Read 8 Done Done Sent false
13168 4384 Acked Read 4 Done Done Sent false
13169 4385 Acked Read 7 Done Done Sent false
13170 4386 Acked Read 9 Done Done Sent false
13171 4387 Acked Read 7 Done Done Sent false
13172 4388 Acked Flush 0 Done Done Sent false
13173 4389 Acked Write 5 Done Done Sent false
13174 4390 Acked Write 6 Done Done Sent false
13175 4391 Acked Read 6 Done Done Sent false
13176 4392 Acked Write 9 Done Done Sent false
13177 4393 Acked Write 4 Done Done Sent false
13178 4394 Acked Flush 0 Done Done Sent false
13179 4395 Acked Flush 0 Done Done Sent false
13180 4396 Acked Write 9 Done Done Sent false
13181 4397 Acked Flush 0 Done Done Sent false
13182 4398 Acked Read 9 Done Done Sent false
13183 4399 Acked Read 2 Done Done Sent false
13184 4400 Acked Write 2 Done Done Sent false
13185 4401 Acked Read 10 Done Done Sent false
13186 4402 Acked Read 7 Done Done Sent false
13187 4403 Acked Write 1 Done Done Sent false
13188 4404 Acked Write 5 Done Done Sent false
13189 4405 Acked Write 2 Done Done Sent false
13190 4406 Acked Read 1 Done Done Sent false
13191 4407 Acked Write 7 Done Done Sent false
13192 4408 Acked Read 1 Done Done Sent false
13193 4409 Acked Read 8 Done Done Sent false
13194 4410 Acked Flush 0 Done Done Sent false
13195 4411 Acked Write 2 Done Done Sent false
13196 4412 Acked Read 5 Done Done Sent false
13197 4413 Acked Read 4 Done Done Sent false
13198 4414 Acked Read 9 Done Done Sent false
13199 4415 Acked Write 2 Done Done Sent false
13200 4416 Acked Read 8 Done Done Sent false
13201 4417 Acked Write 7 Done Done Sent false
13202 4418 Acked Read 9 Done Done Sent false
13203 4419 Acked Write 5 Done Done Sent false
13204 4420 Acked Read 3 Done Done Sent false
13205 4421 Acked Write 5 Done Done Sent false
13206 4422 Acked Read 7 Done Done Sent false
13207 4423 Acked Read 1 Done Done Sent false
13208 4424 Acked Read 6 Done Done Sent false
13209 4425 Acked Read 6 Done Done Sent false
13210 4426 Acked Write 1 Done Done Sent false
13211 4427 Acked Write 9 Done Done Sent false
13212 4428 Acked Write 3 Done Done Sent false
13213 4429 Acked Write 7 Done Done Sent false
13214 4430 Acked Read 5 Done Done Sent false
13215 4431 Acked Read 6 Done Done Sent false
13216 4432 Acked Write 10 Done Done Sent false
13217 4433 Acked Write 3 Done Done Sent false
13218 4434 Acked Read 5 Done Done Sent false
13219 4435 Acked Flush 0 Done Done Sent false
13220 4436 Acked Read 9 Done Done Sent false
13221 4437 Acked Read 6 Done Done Sent false
13222 4438 Acked Read 4 Done Done Sent false
13223 4439 Acked Read 8 Done Done Sent false
13224 4440 Acked Write 9 Done Done Sent false
13225 4441 Acked Write 1 Done Done Sent false
13226 4442 Acked Read 8 Done Done Sent false
13227 4443 Acked Write 1 Done Done Sent false
13228 4444 Acked Flush 0 Done Done Sent false
13229 4445 Acked Read 4 Done Done Sent false
13230 4446 Acked Write 3 Done Done Sent false
13231 4447 Acked Read 10 Done Done Sent false
13232 4448 Acked Read 9 Done Done Sent false
13233 4449 Acked Read 5 Done Done Sent false
13234 4450 Acked Read 7 Done Done Sent false
13235 4451 Acked Read 1 Done Done Sent false
13236 4452 Acked Read 7 Done Done Sent false
13237 4453 Acked Write 10 Done Done Sent false
13238 4454 Acked Read 6 Done Done Sent false
13239 4455 Acked Read 8 Done Done Sent false
13240 4456 Acked Write 7 Done Done Sent false
13241 4457 Acked Read 3 Done Done Sent false
13242 4458 Acked Write 7 Done Done Sent false
13243 4459 Acked Read 8 Done Done Sent false
13244 4460 Acked Read 2 Done Done Sent false
13245 4461 Acked Read 4 Done Done Sent false
13246 4462 Acked Read 6 Done Done Sent false
13247 4463 Acked Write 5 Done Done Sent false
13248 4464 Acked Flush 0 Done Done Sent false
13249 4465 Acked Read 2 Done Done Sent false
13250 4466 Acked Read 10 Done Done Sent false
13251 4467 Acked Read 6 Done Done Sent false
13252 4468 Acked Read 1 Done Done Sent false
13253 4469 Acked Write 6 Done Done Sent false
13254 4470 Acked Write 3 Done Done Sent false
13255 4471 Acked Read 6 Done Done Sent false
13256 4472 Acked Write 10 Done Done Sent false
13257 4473 Acked Read 9 Done Done Sent false
13258 4474 Acked Read 10 Done Done Sent false
13259 4475 Acked Flush 0 Done Done Sent false
13260 4476 Acked Read 8 Done Done Sent false
13261 4477 Acked Read 4 Done Done Sent false
13262 4478 Acked Write 1 Done Done Sent false
13263 4479 Acked Read 5 Done Done Sent false
13264 4480 Acked Flush 0 Done Done Sent false
13265 4481 Acked Write 1 Done Done Sent false
13266 4482 Acked Write 4 Done Done Sent false
13267 4483 Acked Write 9 Done Done Sent false
13268 4484 Acked Read 10 Done Done Sent false
13269 4485 Acked Write 1 Done Done Sent false
13270 4486 Acked Read 10 Done Done Sent false
13271 4487 Acked Read 2 Done Done Sent false
13272 4488 Acked Write 10 Done Done Sent false
13273 4489 Acked Write 2 Done Done Sent false
13274 4490 Acked Write 4 Done Done Sent false
13275 4491 Acked Write 9 Done Done Sent false
13276 4492 Acked Write 10 Done Done Sent false
13277 4493 Acked Read 1 Done Done Sent false
13278 4494 Acked Read 5 Done Done Sent false
13279 4495 Acked Write 8 Done Done Sent false
13280 4496 Acked Read 3 Done Done Sent false
13281 4497 Acked Read 2 Done Done Sent false
13282 4498 Acked Write 4 Done Done Sent false
13283 4499 Acked Read 8 Done Done Sent false
13284 4500 Acked Read 9 Done Done Sent false
13285 4501 Acked Read 3 Done Done Sent false
13286 4502 Acked Read 4 Done Done Sent false
13287 4503 Acked Write 2 Done Done Sent false
13288 4504 Acked Read 4 Done Done Sent false
13289 4505 Acked Write 4 Done Done Sent false
13290 4506 Acked Write 9 Done Done Sent false
13291 4507 Acked Read 1 Done Done Sent false
13292 4508 Acked Write 1 Done Done Sent false
13293 4509 Acked Write 10 Done Done Sent false
13294 4510 Acked Write 7 Done Done Sent false
13295 4511 Acked Write 5 Done Done Sent false
13296 4512 Acked Write 6 Done Done Sent false
13297 4513 Acked Read 3 Done Done Sent false
13298 4514 Acked Write 4 Done Done Sent false
13299 4515 Acked Read 1 Done Done Sent false
13300 4516 Acked Read 5 Done Done Sent false
13301 4517 Acked Write 9 Done Done Sent false
13302 4518 Acked Write 9 Done Done Sent false
13303 4519 Acked Read 1 Done Done Sent false
13304 4520 Acked Read 6 Done Done Sent false
13305 4521 Acked Write 1 Done Done Sent false
13306 4522 Acked Read 4 Done Done Sent false
13307 4523 Acked Read 10 Done Done Sent false
13308 4524 Acked Read 8 Done Done Sent false
13309 4525 Acked Read 5 Done Done Sent false
13310 4526 Acked Read 5 Done Done Sent false
13311 4527 Acked Read 6 Done Done Sent false
13312 4528 Acked Read 4 Done Done Sent false
13313 4529 Acked Write 7 Done Done Sent false
13314 4530 Acked Read 10 Done Done Sent false
13315 4531 Acked Read 9 Done Done Sent false
13316 4532 Acked Read 7 Done Done Sent false
13317 4533 Acked Write 6 Done Done Sent false
13318 4534 Acked Flush 0 Done Done Sent false
13319 4535 Acked Write 10 Done Done Sent false
13320 4536 Acked Read 5 Done Done Sent false
13321 4537 Acked Read 6 Done Done Sent false
13322 4538 Acked Read 10 Done Done Sent false
13323 4539 Acked Read 7 Done Done Sent false
13324 4540 Acked Read 3 Done Done Sent false
13325 4541 Acked Write 7 Done Done Sent false
13326 4542 Acked Write 2 Done Done Sent false
13327 4543 Acked Read 5 Done Done Sent false
13328 4544 Acked Read 7 Done Done Sent false
13329 4545 Acked Read 3 Done Done Sent false
13330 4546 Acked Read 10 Done Done Sent false
13331 4547 Acked Read 10 Done Done Sent false
13332 4548 Acked Read 9 Done Done Sent false
13333 4549 Acked Read 6 Done Done Sent false
13334 4550 Acked Read 3 Done Done Sent false
13335 4551 Acked Read 4 Done Done Sent false
13336 4552 Acked Write 6 Done Done Sent false
13337 4553 Acked Read 10 Done Done Sent false
13338 4554 Acked Read 7 Done Done Sent false
13339 4555 Acked Write 5 Done Done Sent false
13340 4556 Acked Write 3 Done Done Sent false
13341 4557 Acked Write 10 Done Done Sent false
13342 4558 Acked Read 8 Done Done Sent false
13343 4559 Acked Write 3 Done Done Sent false
13344 4560 Acked Write 3 Done Done Sent false
13345 4561 Acked Write 9 Done Done Sent false
13346 4562 Acked Read 5 Done Done Sent false
13347 4563 Acked Read 7 Done Done Sent false
13348 4564 Acked Write 10 Done Done Sent false
13349 4565 Acked Write 6 Done Done Sent false
13350 4566 Acked Read 6 Done Done Sent false
13351 4567 Acked Write 4 Done Done Sent false
13352 4568 Acked Write 7 Done Done Sent false
13353 4569 Acked Write 10 Done Done Sent false
13354 4570 Acked Flush 0 Done Done Sent false
13355 4571 Acked Write 3 Done Done Sent false
13356 4572 Acked Flush 0 Done Done Sent false
13357 4573 Acked Read 8 Done Done Sent false
13358 4574 Acked Write 9 Done Done Sent false
13359 4575 Acked Read 1 Done Done Sent false
13360 4576 Acked Write 5 Done Done Sent false
13361 4577 Acked Flush 0 Done Done Sent false
13362 4578 Acked Write 7 Done Done Sent false
13363 4579 Acked Read 3 Done Done Sent false
13364 4580 Acked Write 7 Done Done Sent false
13365 4581 Acked Read 10 Done Done Sent false
13366 4582 Acked Flush 0 Done Done Sent false
13367 4583 Acked Read 6 Done Done Sent false
13368 4584 Acked Write 2 Done Done Sent false
13369 4585 Acked Read 6 Done Done Sent false
13370 4586 Acked Write 8 Done Done Sent false
13371 4587 Acked Read 7 Done Done Sent false
13372 4588 Acked Read 3 Done Done Sent false
13373 4589 Acked Write 1 Done Done Sent false
13374 4590 Acked Read 10 Done Done Sent false
13375 4591 Acked Write 4 Done Done Sent false
13376 4592 Acked Read 8 Done Done Sent false
13377 4593 Acked Write 5 Done Done Sent false
13378 4594 Acked Write 4 Done Done Sent false
13379 4595 Acked Write 7 Done Done Sent false
13380 4596 Acked Read 2 Done Done Sent false
13381 4597 Acked Read 1 Done Done Sent false
13382 4598 Acked Write 3 Done Done Sent false
13383 4599 Acked Flush 0 Done Done Sent false
13384 4600 Acked Write 7 Done Done Sent false
13385 4601 Acked Read 3 Done Done Sent false
13386 4602 Acked Write 7 Done Done Sent false
13387 4603 Acked Read 1 Done Done Sent false
13388 4604 Acked Read 9 Done Done Sent false
13389 4605 Acked Read 9 Done Done Sent false
13390 4606 Acked Read 6 Done Done Sent false
13391 4607 Acked Write 8 Done Done Sent false
13392 4608 Acked Read 5 Done Done Sent false
13393 4609 Acked Write 7 Done Done Sent false
13394 4610 Acked Write 4 Done Done Sent false
13395 4611 Acked Read 7 Done Done Sent false
13396 4612 Acked Read 7 Done Done Sent false
13397 4613 Acked Write 9 Done Done Sent false
13398 4614 Acked Flush 0 Done Done Sent false
13399 4615 Acked Flush 0 Done Done Sent false
13400 4616 Acked Read 5 Done Done Sent false
13401 4617 Acked Read 2 Done Done Sent false
13402 4618 Acked Read 10 Done Done Sent false
13403 4619 Acked Write 5 Done Done Sent false
13404 4620 Acked Read 10 Done Done Sent false
13405 4621 Acked Flush 0 Done Done Sent false
13406 STATES DS:0 DS:1 DS:2 TOTAL
13407 Sent 0 0 302 302
13408 Done 302 302 0 604
13409 Skipped 0 0 0 0
13410 Error 0 0 0 0
13411 Last Flush: 4621 4621 4319
13412 Downstairs last five completed: 4319 4318 4317 4316 4315
13413 Upstairs last five completed: 4621 4620 4619 4618 4617
13414 CLIENT: Up:0 ds:302 act:2
13415 [8830] got start action
13416 [8830] I am going to restart
13417 Starting downstairs at port 8830
13418 Make output file at "/tmp/dsc/downstairs-8830.txt"
13419 [8830][2] reports Starting
13420 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1212)
13421 [8830][2] reports Running
13422 ----------------------------------------------------------------
13423 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13424 States: Active Active Offline
13425 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13426 4320 Acked Read 4 Done Done Sent false
13427 4321 Acked Write 6 Done Done Sent false
13428 4322 Acked Read 3 Done Done Sent false
13429 4323 Acked Write 1 Done Done Sent false
13430 4324 Acked Read 3 Done Done Sent false
13431 4325 Acked Read 7 Done Done Sent false
13432 4326 Acked Read 4 Done Done Sent false
13433 4327 Acked Write 1 Done Done Sent false
13434 4328 Acked Read 1 Done Done Sent false
13435 4329 Acked Read 4 Done Done Sent false
13436 4330 Acked Flush 0 Done Done Sent false
13437 4331 Acked Write 1 Done Done Sent false
13438 4332 Acked Read 9 Done Done Sent false
13439 4333 Acked Write 4 Done Done Sent false
13440 4334 Acked Write 4 Done Done Sent false
13441 4335 Acked Read 3 Done Done Sent false
13442 4336 Acked Read 4 Done Done Sent false
13443 4337 Acked Write 4 Done Done Sent false
13444 4338 Acked Write 8 Done Done Sent false
13445 4339 Acked Read 2 Done Done Sent false
13446 4340 Acked Read 6 Done Done Sent false
13447 4341 Acked Write 6 Done Done Sent false
13448 4342 Acked Write 3 Done Done Sent false
13449 4343 Acked Write 3 Done Done Sent false
13450 4344 Acked Read 1 Done Done Sent false
13451 4345 Acked Write 9 Done Done Sent false
13452 4346 Acked Read 1 Done Done Sent false
13453 4347 Acked Read 8 Done Done Sent false
13454 4348 Acked Write 3 Done Done Sent false
13455 4349 Acked Write 4 Done Done Sent false
13456 4350 Acked Write 1 Done Done Sent false
13457 4351 Acked Write 2 Done Done Sent false
13458 4352 Acked Write 5 Done Done Sent false
13459 4353 Acked Write 10 Done Done Sent false
13460 4354 Acked Read 6 Done Done Sent false
13461 4355 Acked Read 10 Done Done Sent false
13462 4356 Acked Flush 0 Done Done Sent false
13463 4357 Acked Write 4 Done Done Sent false
13464 4358 Acked Read 4 Done Done Sent false
13465 4359 Acked Read 3 Done Done Sent false
13466 4360 Acked Write 6 Done Done Sent false
13467 4361 Acked Flush 0 Done Done Sent false
13468 4362 Acked Flush 0 Done Done Sent false
13469 4363 Acked Write 6 Done Done Sent false
13470 4364 Acked Write 1 Done Done Sent false
13471 4365 Acked Write 4 Done Done Sent false
13472 4366 Acked Write 3 Done Done Sent false
13473 4367 Acked Write 9 Done Done Sent false
13474 4368 Acked Write 1 Done Done Sent false
13475 4369 Acked Write 2 Done Done Sent false
13476 4370 Acked Write 7 Done Done Sent false
13477 4371 Acked Read 4 Done Done Sent false
13478 4372 Acked Write 1 Done Done Sent false
13479 4373 Acked Read 6 Done Done Sent false
13480 4374 Acked Flush 0 Done Done Sent false
13481 4375 Acked Read 3 Done Done Sent false
13482 4376 Acked Write 6 Done Done Sent false
13483 4377 Acked Read 8 Done Done Sent false
13484 4378 Acked Read 10 Done Done Sent false
13485 4379 Acked Read 1 Done Done Sent false
13486 4380 Acked Read 9 Done Done Sent false
13487 4381 Acked Read 7 Done Done Sent false
13488 4382 Acked Write 7 Done Done Sent false
13489 4383 Acked Read 8 Done Done Sent false
13490 4384 Acked Read 4 Done Done Sent false
13491 4385 Acked Read 7 Done Done Sent false
13492 4386 Acked Read 9 Done Done Sent false
13493 4387 Acked Read 7 Done Done Sent false
13494 4388 Acked Flush 0 Done Done Sent false
13495 4389 Acked Write 5 Done Done Sent false
13496 4390 Acked Write 6 Done Done Sent false
13497 4391 Acked Read 6 Done Done Sent false
13498 4392 Acked Write 9 Done Done Sent false
13499 4393 Acked Write 4 Done Done Sent false
13500 4394 Acked Flush 0 Done Done Sent false
13501 4395 Acked Flush 0 Done Done Sent false
13502 4396 Acked Write 9 Done Done Sent false
13503 4397 Acked Flush 0 Done Done Sent false
13504 4398 Acked Read 9 Done Done Sent false
13505 4399 Acked Read 2 Done Done Sent false
13506 4400 Acked Write 2 Done Done Sent false
13507 4401 Acked Read 10 Done Done Sent false
13508 4402 Acked Read 7 Done Done Sent false
13509 4403 Acked Write 1 Done Done Sent false
13510 4404 Acked Write 5 Done Done Sent false
13511 4405 Acked Write 2 Done Done Sent false
13512 4406 Acked Read 1 Done Done Sent false
13513 4407 Acked Write 7 Done Done Sent false
13514 4408 Acked Read 1 Done Done Sent false
13515 4409 Acked Read 8 Done Done Sent false
13516 4410 Acked Flush 0 Done Done Sent false
13517 4411 Acked Write 2 Done Done Sent false
13518 4412 Acked Read 5 Done Done Sent false
13519 4413 Acked Read 4 Done Done Sent false
13520 4414 Acked Read 9 Done Done Sent false
13521 4415 Acked Write 2 Done Done Sent false
13522 4416 Acked Read 8 Done Done Sent false
13523 4417 Acked Write 7 Done Done Sent false
13524 4418 Acked Read 9 Done Done Sent false
13525 4419 Acked Write 5 Done Done Sent false
13526 4420 Acked Read 3 Done Done Sent false
13527 4421 Acked Write 5 Done Done Sent false
13528 4422 Acked Read 7 Done Done Sent false
13529 4423 Acked Read 1 Done Done Sent false
13530 4424 Acked Read 6 Done Done Sent false
13531 4425 Acked Read 6 Done Done Sent false
13532 4426 Acked Write 1 Done Done Sent false
13533 4427 Acked Write 9 Done Done Sent false
13534 4428 Acked Write 3 Done Done Sent false
13535 4429 Acked Write 7 Done Done Sent false
13536 4430 Acked Read 5 Done Done Sent false
13537 4431 Acked Read 6 Done Done Sent false
13538 4432 Acked Write 10 Done Done Sent false
13539 4433 Acked Write 3 Done Done Sent false
13540 4434 Acked Read 5 Done Done Sent false
13541 4435 Acked Flush 0 Done Done Sent false
13542 4436 Acked Read 9 Done Done Sent false
13543 4437 Acked Read 6 Done Done Sent false
13544 4438 Acked Read 4 Done Done Sent false
13545 4439 Acked Read 8 Done Done Sent false
13546 4440 Acked Write 9 Done Done Sent false
13547 4441 Acked Write 1 Done Done Sent false
13548 4442 Acked Read 8 Done Done Sent false
13549 4443 Acked Write 1 Done Done Sent false
13550 4444 Acked Flush 0 Done Done Sent false
13551 4445 Acked Read 4 Done Done Sent false
13552 4446 Acked Write 3 Done Done Sent false
13553 4447 Acked Read 10 Done Done Sent false
13554 4448 Acked Read 9 Done Done Sent false
13555 4449 Acked Read 5 Done Done Sent false
13556 4450 Acked Read 7 Done Done Sent false
13557 4451 Acked Read 1 Done Done Sent false
13558 4452 Acked Read 7 Done Done Sent false
13559 4453 Acked Write 10 Done Done Sent false
13560 4454 Acked Read 6 Done Done Sent false
13561 4455 Acked Read 8 Done Done Sent false
13562 4456 Acked Write 7 Done Done Sent false
13563 4457 Acked Read 3 Done Done Sent false
13564 4458 Acked Write 7 Done Done Sent false
13565 4459 Acked Read 8 Done Done Sent false
13566 4460 Acked Read 2 Done Done Sent false
13567 4461 Acked Read 4 Done Done Sent false
13568 4462 Acked Read 6 Done Done Sent false
13569 4463 Acked Write 5 Done Done Sent false
13570 4464 Acked Flush 0 Done Done Sent false
13571 4465 Acked Read 2 Done Done Sent false
13572 4466 Acked Read 10 Done Done Sent false
13573 4467 Acked Read 6 Done Done Sent false
13574 4468 Acked Read 1 Done Done Sent false
13575 4469 Acked Write 6 Done Done Sent false
13576 4470 Acked Write 3 Done Done Sent false
13577 4471 Acked Read 6 Done Done Sent false
13578 4472 Acked Write 10 Done Done Sent false
13579 4473 Acked Read 9 Done Done Sent false
13580 4474 Acked Read 10 Done Done Sent false
13581 4475 Acked Flush 0 Done Done Sent false
13582 4476 Acked Read 8 Done Done Sent false
13583 4477 Acked Read 4 Done Done Sent false
13584 4478 Acked Write 1 Done Done Sent false
13585 4479 Acked Read 5 Done Done Sent false
13586 4480 Acked Flush 0 Done Done Sent false
13587 4481 Acked Write 1 Done Done Sent false
13588 4482 Acked Write 4 Done Done Sent false
13589 4483 Acked Write 9 Done Done Sent false
13590 4484 Acked Read 10 Done Done Sent false
13591 4485 Acked Write 1 Done Done Sent false
13592 4486 Acked Read 10 Done Done Sent false
13593 4487 Acked Read 2 Done Done Sent false
13594 4488 Acked Write 10 Done Done Sent false
13595 4489 Acked Write 2 Done Done Sent false
13596 4490 Acked Write 4 Done Done Sent false
13597 4491 Acked Write 9 Done Done Sent false
13598 4492 Acked Write 10 Done Done Sent false
13599 4493 Acked Read 1 Done Done Sent false
13600 4494 Acked Read 5 Done Done Sent false
13601 4495 Acked Write 8 Done Done Sent false
13602 4496 Acked Read 3 Done Done Sent false
13603 4497 Acked Read 2 Done Done Sent false
13604 4498 Acked Write 4 Done Done Sent false
13605 4499 Acked Read 8 Done Done Sent false
13606 4500 Acked Read 9 Done Done Sent false
13607 4501 Acked Read 3 Done Done Sent false
13608 4502 Acked Read 4 Done Done Sent false
13609 4503 Acked Write 2 Done Done Sent false
13610 4504 Acked Read 4 Done Done Sent false
13611 4505 Acked Write 4 Done Done Sent false
13612 4506 Acked Write 9 Done Done Sent false
13613 4507 Acked Read 1 Done Done Sent false
13614 4508 Acked Write 1 Done Done Sent false
13615 4509 Acked Write 10 Done Done Sent false
13616 4510 Acked Write 7 Done Done Sent false
13617 4511 Acked Write 5 Done Done Sent false
13618 4512 Acked Write 6 Done Done Sent false
13619 4513 Acked Read 3 Done Done Sent false
13620 4514 Acked Write 4 Done Done Sent false
13621 4515 Acked Read 1 Done Done Sent false
13622 4516 Acked Read 5 Done Done Sent false
13623 4517 Acked Write 9 Done Done Sent false
13624 4518 Acked Write 9 Done Done Sent false
13625 4519 Acked Read 1 Done Done Sent false
13626 4520 Acked Read 6 Done Done Sent false
13627 4521 Acked Write 1 Done Done Sent false
13628 4522 Acked Read 4 Done Done Sent false
13629 4523 Acked Read 10 Done Done Sent false
13630 4524 Acked Read 8 Done Done Sent false
13631 4525 Acked Read 5 Done Done Sent false
13632 4526 Acked Read 5 Done Done Sent false
13633 4527 Acked Read 6 Done Done Sent false
13634 4528 Acked Read 4 Done Done Sent false
13635 4529 Acked Write 7 Done Done Sent false
13636 4530 Acked Read 10 Done Done Sent false
13637 4531 Acked Read 9 Done Done Sent false
13638 4532 Acked Read 7 Done Done Sent false
13639 4533 Acked Write 6 Done Done Sent false
13640 4534 Acked Flush 0 Done Done Sent false
13641 4535 Acked Write 10 Done Done Sent false
13642 4536 Acked Read 5 Done Done Sent false
13643 4537 Acked Read 6 Done Done Sent false
13644 4538 Acked Read 10 Done Done Sent false
13645 4539 Acked Read 7 Done Done Sent false
13646 4540 Acked Read 3 Done Done Sent false
13647 4541 Acked Write 7 Done Done Sent false
13648 4542 Acked Write 2 Done Done Sent false
13649 4543 Acked Read 5 Done Done Sent false
13650 4544 Acked Read 7 Done Done Sent false
13651 4545 Acked Read 3 Done Done Sent false
13652 4546 Acked Read 10 Done Done Sent false
13653 4547 Acked Read 10 Done Done Sent false
13654 4548 Acked Read 9 Done Done Sent false
13655 4549 Acked Read 6 Done Done Sent false
13656 4550 Acked Read 3 Done Done Sent false
13657 4551 Acked Read 4 Done Done Sent false
13658 4552 Acked Write 6 Done Done Sent false
13659 4553 Acked Read 10 Done Done Sent false
13660 4554 Acked Read 7 Done Done Sent false
13661 4555 Acked Write 5 Done Done Sent false
13662 4556 Acked Write 3 Done Done Sent false
13663 4557 Acked Write 10 Done Done Sent false
13664 4558 Acked Read 8 Done Done Sent false
13665 4559 Acked Write 3 Done Done Sent false
13666 4560 Acked Write 3 Done Done Sent false
13667 4561 Acked Write 9 Done Done Sent false
13668 4562 Acked Read 5 Done Done Sent false
13669 4563 Acked Read 7 Done Done Sent false
13670 4564 Acked Write 10 Done Done Sent false
13671 4565 Acked Write 6 Done Done Sent false
13672 4566 Acked Read 6 Done Done Sent false
13673 4567 Acked Write 4 Done Done Sent false
13674 4568 Acked Write 7 Done Done Sent false
13675 4569 Acked Write 10 Done Done Sent false
13676 4570 Acked Flush 0 Done Done Sent false
13677 4571 Acked Write 3 Done Done Sent false
13678 4572 Acked Flush 0 Done Done Sent false
13679 4573 Acked Read 8 Done Done Sent false
13680 4574 Acked Write 9 Done Done Sent false
13681 4575 Acked Read 1 Done Done Sent false
13682 4576 Acked Write 5 Done Done Sent false
13683 4577 Acked Flush 0 Done Done Sent false
13684 4578 Acked Write 7 Done Done Sent false
13685 4579 Acked Read 3 Done Done Sent false
13686 4580 Acked Write 7 Done Done Sent false
13687 4581 Acked Read 10 Done Done Sent false
13688 4582 Acked Flush 0 Done Done Sent false
13689 4583 Acked Read 6 Done Done Sent false
13690 4584 Acked Write 2 Done Done Sent false
13691 4585 Acked Read 6 Done Done Sent false
13692 4586 Acked Write 8 Done Done Sent false
13693 4587 Acked Read 7 Done Done Sent false
13694 4588 Acked Read 3 Done Done Sent false
13695 4589 Acked Write 1 Done Done Sent false
13696 4590 Acked Read 10 Done Done Sent false
13697 4591 Acked Write 4 Done Done Sent false
13698 4592 Acked Read 8 Done Done Sent false
13699 4593 Acked Write 5 Done Done Sent false
13700 4594 Acked Write 4 Done Done Sent false
13701 4595 Acked Write 7 Done Done Sent false
13702 4596 Acked Read 2 Done Done Sent false
13703 4597 Acked Read 1 Done Done Sent false
13704 4598 Acked Write 3 Done Done Sent false
13705 4599 Acked Flush 0 Done Done Sent false
13706 4600 Acked Write 7 Done Done Sent false
13707 4601 Acked Read 3 Done Done Sent false
13708 4602 Acked Write 7 Done Done Sent false
13709 4603 Acked Read 1 Done Done Sent false
13710 4604 Acked Read 9 Done Done Sent false
13711 4605 Acked Read 9 Done Done Sent false
13712 4606 Acked Read 6 Done Done Sent false
13713 4607 Acked Write 8 Done Done Sent false
13714 4608 Acked Read 5 Done Done Sent false
13715 4609 Acked Write 7 Done Done Sent false
13716 4610 Acked Write 4 Done Done Sent false
13717 4611 Acked Read 7 Done Done Sent false
13718 4612 Acked Read 7 Done Done Sent false
13719 4613 Acked Write 9 Done Done Sent false
13720 4614 Acked Flush 0 Done Done Sent false
13721 4615 Acked Flush 0 Done Done Sent false
13722 4616 Acked Read 5 Done Done Sent false
13723 4617 Acked Read 2 Done Done Sent false
13724 4618 Acked Read 10 Done Done Sent false
13725 4619 Acked Write 5 Done Done Sent false
13726 4620 Acked Read 10 Done Done Sent false
13727 4621 Acked Flush 0 Done Done Sent false
13728 STATES DS:0 DS:1 DS:2 TOTAL
13729 Sent 0 0 302 302
13730 Done 302 302 0 604
13731 Skipped 0 0 0 0
13732 Error 0 0 0 0
13733 Last Flush: 4621 4621 4319
13734 Downstairs last five completed: 4319 4318 4317 4316 4315
13735 Upstairs last five completed: 4621 4620 4619 4618 4617
13736 CLIENT: Up:0 ds:302 act:2
137372025-01-17T21:13:59.110ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
137382025-01-17T21:13:59.110ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49180) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
137392025-01-17T21:13:59.112ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
137402025-01-17T21:13:59.112ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
137412025-01-17T21:13:59.112ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
137422025-01-17T21:13:59.112ZINFOcrucible: send last flush ID to this DS: 4319 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
137432025-01-17T21:13:59.112ZINFOcrucible: Replied this last flush ID: 4319 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
137442025-01-17T21:13:59.112ZINFOcrucible: [2] client re-new 302 jobs since flush 4319 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
137452025-01-17T21:13:59.113ZINFOcrucible: [2] Replayed 302 jobs since flush: 4319 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
13746 ----------------------------------------------------------------
13747 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
13748 Downstairs last five completed: 4621 4620 4619 4618 4617
13749 Upstairs last five completed: 4621 4620 4619 4618 4617
13750 CLIENT: Up:0 ds:0 act:3
13751 Replay: All jobs finished
13752 Main task has work to do, go find it
13753 got dsc Stop(1)
13754 stop 1
13755 [8820] Got stop action so:false kr:false
13756 Jan 17 21:13:59.363 INFO request completed, latency_us: 382, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 90d543ef-7d14-43a8-9432-a9c5b5bf26c0, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
137572025-01-17T21:13:59.363ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
13758 Jan 17 21:13:59.364{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:13:59.364056978Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
13759 INFO request completed, latency_us: 131, response_code: 200, uri: /state/cid/1, method: GET, {"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:13:59.364108811Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
13760 req_id: f64bd56f-2f4c-4525-880a-24c316302dff, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
137612025-01-17T21:13:59.364ZINFOcrucible: client task is exiting = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
13762 [8820] Exited with: ExitStatus(unix_wait_status(9))
13763 [8820][1] reports Exit
13764 Jan 17 21:14:03.367 INFO request completed, latency_us: 205, response_code: 200, uri: /state/cid/1, method: GET, req_id: 551946de-5549-4532-994b-ac0405645ca8, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
13765 001/300 Read block 460 len 24576
13766 002/300 Read block 804 len 28672
13767 003/300 Write block 1848 len 20480 data: 2 2 2 2 2
13768 004/300 Read block 4842 len 24576
13769 005/300 Read block 3086 len 16384
13770 006/300 Read block 2150 len 8192
13771 007/300 Read block 1833 len 32768
13772 008/300 Read block 2909 len 4096
13773 009/300 Read block 1856 len 12288
13774 010/300 Write block 3754 len 20480 data: 3 3 3 3 2
13775 011/300 Read block 692 len 24576
13776 012/300 Read block 4652 len 32768
13777 013/300 Read block 718 len 40960
13778 014/300 Write block 4867 len 4096 data: 2
13779 015/300 Read block 4075 len 24576
13780 016/300 Write block 1173 len 20480 data: 3 2 2 4 4
13781 017/300 Write block 4116 len 4096 data: 2
13782 018/300 Read block 915 len 20480
13783 019/300 Read block 1439 len 8192
13784 020/300 Read block 4958 len 32768
13785 021/300 Read block 2264 len 4096
13786 022/300 Read block 3416 len 24576
13787 023/300 Read block 3275 len 8192
13788 024/300 Write block 3605 len 32768 data: 4 3 3 4 6 5 6 7
13789 025/300 Write block 1056 len 20480 data: 6 7 7 7 6
13790 026/300 Read block 406 len 28672
13791 027/300 Write block 3991 len 36864 data: 5 5 5 4 5 6 5 5 4
13792 028/300 Read block 1782 len 20480
13793 029/300 Read block 219 len 24576
13794 030/300 Read block 2363 len 28672
13795 031/300 Flush
13796 032/300 Write block 1692 len 32768 data: 3 2 3 3 3 3 3 3
13797 033/300 Read block 3690 len 8192
13798 034/300 Read block 1629 len 20480
13799 035/300 Write block 1959 len 36864 data: 4 4 3 3 4 6 6 5 5
13800 036/300 Read block 4158 len 8192
13801 037/300 Read block 1559 len 16384
13802 038/300 Read block 520 len 16384
13803 039/300 Read block 116 len 16384
13804 040/300 Write block 2053 len 16384 data: 4 4 4 4
13805 041/300 Read block 3320 len 20480
13806 042/300 Read block 2914 len 4096
13807 043/300 Read block 4311 len 8192
13808 044/300 Write block 1923 len 8192 data: 4 4
13809 045/300 Read block 4764 len 24576
13810 046/300 Read block 3357 len 20480
13811 047/300 Write block 1244 len 40960 data: 3 3 3 3 2 2 3 3 4 3
13812 048/300 Flush
13813 049/300 Read block 4123 len 28672
13814 050/300 Read block 3160 len 16384
13815 051/300 Read block 1740 len 24576
13816 052/300 Read block 550 len 12288
13817 053/300 Read block 3032 len 20480
13818 054/300 Write block 2109 len 40960 data: 5 3 3 2 2 2 2 2 2 2
13819 055/300 Write block 4754 len 28672 data: 4 4 5 5 4 5 5
13820 056/300 Read block 2163 len 12288
13821 057/300 Write block 2060 len 32768 data: 3 3 4 4 4 5 5 5
13822 058/300 Read block 3657 len 16384
13823 059/300 Flush
13824 060/300 Write block 492 len 28672 data: 2 2 3 2 3 3 3
13825 061/300 Read block 1402 len 24576
13826 062/300 Read block 1612 len 36864
13827 063/300 Read block 2664 len 24576
13828 064/300 Read block 4728 len 24576
13829 065/300 Read block 4634 len 28672
13830 066/300 Read block 3563 len 20480
13831 067/300 Read block 4472 len 8192
13832 068/300 Write block 798 len 28672 data: 4 2 2 2 3 3 4
13833 069/300 Write block 3008 len 32768 data: 5 4 5 4 4 4 4 3
13834 070/300 Read block 4441 len 36864
13835 071/300 Read block 3029 len 24576
13836 072/300 Read block 1884 len 28672
13837 073/300 Read block 2154 len 16384
13838 074/300 Write block 3524 len 40960 data: 4 3 4 5 4 2 3 2 2 2
13839 075/300 Read block 3487 len 12288
13840 076/300 Flush
13841 077/300 Read block 4065 len 32768
13842 078/300 Flush
13843 079/300 Flush
13844 080/300 Write block 2648 len 8192 data: 3 4
13845 081/300 Flush
13846 082/300 Write block 206 len 32768 data: 7 6 6 6 6 6 5 5
13847 083/300 Read block 3233 len 32768
13848 084/300 Write block 2175 len 16384 data: 4 3 4 3
13849 085/300 Read block 403 len 24576
13850 086/300 Write block 3216 len 12288 data: 3 3 4
13851 087/300 Write block 2427 len 12288 data: 3 3 3
13852 088/300 Read block 472 len 4096
13853 089/300 Read block 4673 len 24576
13854 090/300 Write block 3119 len 8192 data: 3 3
13855 091/300 Read block 1672 len 16384
13856 092/300 Write block 2113 len 8192 data: 3 3
13857 093/300 Read block 2698 len 12288
13858 094/300 Write block 989 len 8192 data: 5 4
13859 095/300 Read block 1545 len 20480
13860 096/300 Read block 2076 len 32768
13861 097/300 Write block 3707 len 12288 data: 6 6 5
13862 098/300 Write block 2242 len 12288 data: 4 4 4
13863 099/300 Read block 10 len 12288
13864 100/300 Write block 1859 len 4096 data: 3
13865 101/300 Write block 243 len 40960 data: 3 3 4 5 3 3 2 4 4 4
13866 102/300 Read block 428 len 36864
13867 103/300 Read block 2307 len 8192
13868 104/300 Read block 864 len 20480
13869 105/300 Read block 3751 len 4096
13870 106/300 Write block 2105 len 36864 data: 5 6 5 5 6 4 4 3 4
13871 107/300 Read block 647 len 16384
13872 108/300 Read block 3711 len 8192
13873 109/300 Write block 1674 len 24576 data: 2 3 3 4 4 5
13874 110/300 Read block 2005 len 24576
13875 111/300 Write block 3487 len 12288 data: 3 3 3
13876 112/300 Read block 2966 len 32768
13877 113/300 Read block 182 len 40960
13878 114/300 Read block 4790 len 4096
13879 115/300 Write block 2390 len 32768 data: 4 4 4 5 5 5 6 6
13880 116/300 Read block 336 len 32768
13881 117/300 Read block 1485 len 32768
13882 118/300 Write block 2544 len 40960 data: 4 4 4 4 5 4 5 5 4 5
13883 119/300 Read block 4886 len 16384
13884 120/300 Write block 4177 len 16384 data: 6 6 4 2
13885 121/300 Write block 4308 len 16384 data: 3 4 5 5
13886 122/300 Flush
13887 123/300 Read block 4451 len 12288
13888 124/300 Write block 302 len 20480 data: 3 2 2 2 2
13889 125/300 Read block 1091 len 20480
13890 126/300 Read block 336 len 32768
13891 127/300 Flush
13892 128/300 Read block 1642 len 20480
13893 129/300 Write block 3971 len 28672 data: 2 2 2 2 2 2 2
13894 130/300 Flush
13895 131/300 Flush
13896 132/300 Write block 24 len 32768 data: 5 5 5 6 7 6 5 5
13897 133/300 Write block 2060 len 36864 data: 4 4 5 5 5 6 6 6 6
13898 134/300 Read block 2024 len 8192
13899 135/300 Read block 1345 len 28672
13900 136/300 Read block 3355 len 40960
13901 137/300 Read block 3990 len 36864
13902 138/300 Read block 1804 len 12288
13903 139/300 Write block 870 len 36864 data: 2 3 3 3 3 3 3 5 6
13904 140/300 Write block 807 len 8192 data: 3 3
13905 141/300 Read block 2213 len 16384
13906 142/300 Write block 2859 len 12288 data: 4 4 4
13907 143/300 Read block 2390 len 36864
13908 144/300 Read block 1766 len 16384
13909 145/300 Write block 4993 len 20480 data: 3 3 3 3 3
13910 146/300 Write block 3445 len 16384 data: 8 9 9 8
13911 147/300 Write block 4764 len 40960 data: 6 5 5 4 4 3 3 3 2 3
13912 148/300 Write block 3233 len 40960 data: 2 3 4 4 5 5 3 3 3 4
13913 149/300 Write block 3860 len 4096 data: 3
13914 150/300 Read block 1392 len 24576
13915 151/300 Read block 1392 len 24576
13916 152/300 Write block 4297 len 4096 data: 5
13917 153/300 Read block 2972 len 36864
13918 154/300 Write block 1432 len 32768 data: 2 2 3 3 3 4 5 4
13919 155/300 Read block 2369 len 24576
13920 156/300 Read block 2602 len 16384
13921 157/300 Write block 3340 len 36864 data: 4 3 2 2 2 2 3 2 2
13922 158/300 Write block 2913 len 32768 data: 3 4 3 5 5 5 4 4
13923 159/300 Write block 884 len 20480 data: 4 3 4 4 5
13924 160/300 Write block 3118 len 40960 data: 3 4 4 3 3 3 3 3 2 3
13925 161/300 Write block 919 len 20480 data: 6 6 4 5 5
13926 162/300 Read block 2408 len 36864
13927 163/300 Read block 1387 len 24576
13928 164/300 Flush
13929 165/300 Read block 961 len 4096
13930 166/300 Read block 4810 len 28672
13931 167/300 Read block 2966 len 16384
13932 168/300 Write block 2713 len 20480 data: 4 3 3 3 3
13933 169/300 Read block 206 len 28672
13934 170/300 Write block 1028 len 8192 data: 6 5
13935 171/300 Read block 1473 len 16384
13936 172/300 Read block 3600 len 4096
13937 173/300 Write block 1832 len 32768 data: 4 2 3 4 3 3 3 3
13938 174/300 Read block 1958 len 20480
13939 175/300 Write block 2882 len 4096 data: 4
13940 176/300 Read block 2948 len 16384
13941 177/300 Write block 2685 len 40960 data: 5 4 3 3 3 2 2 2 3 3
13942 178/300 Write block 2291 len 8192 data: 4 4
13943 179/300 Read block 3135 len 16384
13944 180/300 Read block 2622 len 4096
13945 181/300 Read block 580 len 32768
13946 182/300 Read block 68 len 28672
13947 183/300 Write block 4460 len 28672 data: 4 4 4 3 3 3 3
13948 184/300 Read block 1289 len 8192
13949 185/300 Write block 270 len 16384 data: 4 4 4 4
13950 186/300 Read block 4444 len 8192
13951 187/300 Read block 2351 len 4096
13952 188/300 Read block 2431 len 20480
13953 189/300 Read block 2243 len 4096
13954 190/300 Write block 1524 len 8192 data: 6 6
13955 191/300 Read block 1172 len 36864
13956 192/300 Read block 4507 len 20480
13957 193/300 Write block 358 len 28672 data: 2 3 4 4 5 6 6
13958 194/300 Read block 1849 len 16384
13959 195/300 Read block 4936 len 4096
13960 196/300 Write block 1730 len 24576 data: 3 4 3 4 4 4
13961 197/300 Read block 3442 len 16384
13962 198/300 Write block 1079 len 12288 data: 3 3 4
13963 199/300 Read block 1664 len 24576
13964 200/300 Write block 562 len 40960 data: 4 4 3 2 2 2 2 2 2 2
13965 201/300 Read block 684 len 40960
13966 202/300 Write block 1719 len 12288 data: 3 3 4
13967 203/300 Read block 4985 len 8192
13968 204/300 Read block 1870 len 28672
13969 205/300 Read block 2107 len 36864
13970 206/300 Write block 3574 len 16384 data: 4 4 4 4
13971 207/300 Write block 4553 len 4096 data: 5
13972 208/300 Write block 902 len 28672 data: 5 4 3 3 3 3 2
13973 209/300 Write block 4139 len 12288 data: 5 5 4
13974 210/300 Read block 395 len 8192
13975 211/300 Write block 1148 len 40960 data: 5 6 5 5 6 6 5 7 8 10
13976 212/300 Write block 3795 len 36864 data: 2 2 2 3 3 3 3 3 3
13977 213/300 Read block 788 len 36864
13978 214/300 Read block 3404 len 20480
13979 215/300 Read block 677 len 4096
13980 216/300 Write block 683 len 24576 data: 4 5 5 6 6 6
13981 217/300 Read block 2936 len 20480
13982 218/300 Read block 574 len 40960
13983 219/300 Read block 455 len 28672
13984 220/300 Read block 4114 len 16384
13985 221/300 Flush
13986 222/300 Write block 2280 len 36864 data: 4 3 3 3 3 2 2 2 4
13987 223/300 Read block 1483 len 16384
13988 224/300 Read block 3504 len 4096
13989 225/300 Write block 91 len 16384 data: 3 3 3 3
13990 226/300 Write block 2466 len 8192 data: 6 4
13991 227/300 Read block 1101 len 8192
13992 228/300 Write block 3133 len 24576 data: 4 3 3 4 3 3
13993 229/300 Write block 2662 len 36864 data: 3 4 4 4 3 2 2 2 2
13994 230/300 Read block 4898 len 16384
13995 231/300 Write block 4416 len 20480 data: 5 5 6 3 3
13996 232/300 Write block 1582 len 4096 data: 7
13997 233/300 Write block 2015 len 36864 data: 3 3 3 3 3 3 3 4 4
13998 234/300 Read block 2770 len 16384
13999 235/300 Read block 1232 len 16384
14000 236/300 Read block 4626 len 24576
14001 237/300 Read block 1723 len 16384
14002 238/300 Read block 4057 len 16384
14003 239/300 Write block 498 len 32768 data: 4 3 2 2 3 3 3 4
14004 240/300 Read block 2991 len 8192
14005 241/300 Flush
14006 242/300 Read block 45 len 24576
14007 243/300 Write block 1711 len 20480 data: 3 3 2 3 3
14008 244/300 Write block 2337 len 36864 data: 3 3 2 2 2 3 3 4 4
14009 245/300 Read block 3477 len 4096
14010 246/300 Write block 4576 len 32768 data: 3 2 2 2 2 3 2 2
14011 247/300 Read block 1201 len 32768
14012 248/300 Read block 3619 len 12288
14013 249/300 Read block 2019 len 36864
14014 250/300 Write block 3595 len 40960 data: 3 2 3 2 2 2 2 2 4 4
14015 251/300 Read block 1758 len 36864
14016 252/300 Write block 86 len 20480 data: 2 3 3 3 3
14017 253/300 Read block 214 len 4096
14018 254/300 Read block 519 len 8192
14019 255/300 Write block 1653 len 36864 data: 3 3 5 3 3 4 5 6 6
14020 256/300 Read block 3286 len 28672
14021 257/300 Write block 3251 len 36864 data: 4 4 3 4 4 4 4 4 2
14022 258/300 Read block 267 len 16384
14023 259/300 Flush
14024 260/300 Write block 950 len 32768 data: 4 5 3 3 3 5 5 5
14025 261/300 Write block 4000 len 40960 data: 5 3 3 3 4 4 6 6 6 6
14026 262/300 Read block 4936 len 20480
14027 263/300 Read block 3812 len 24576
14028 264/300 Read block 4767 len 12288
14029 265/300 Write block 4007 len 20480 data: 7 7 7 4 5
14030 266/300 Read block 1807 len 4096
14031 267/300 Read block 4427 len 12288
14032 268/300 Read block 2968 len 12288
14033 269/300 Read block 3623 len 4096
14034 270/300 Read block 1122 len 32768
14035 271/300 Read block 1885 len 40960
14036 272/300 Read block 3749 len 16384
14037 273/300 Write block 3 len 24576 data: 3 3 2 2 2 2
14038 274/300 Write block 2784 len 36864 data: 2 2 3 3 2 2 3 4 4
14039 275/300 Flush
14040 276/300 Write block 2334 len 16384 data: 5 5 4 4
14041 277/300 Write block 2017 len 8192 data: 4 4
14042 278/300 Read block 4338 len 40960
14043 279/300 Flush
14044 280/300 Write block 655 len 28672 data: 3 2 2 2 2 2 2
14045 281/300 Read block 153 len 4096
14046 282/300 Flush
14047 283/300 Flush
14048 284/300 Write block 483 len 16384 data: 2 2 2 2
14049 285/300 Flush
14050 286/300 Write block 3471 len 28672 data: 4 4 4 4 3 2 2
14051 287/300 Flush
14052 288/300 Read block 4049 len 20480
14053 289/300 Read block 3272 len 24576
14054 290/300 Write block 1099 len 28672 data: 5 4 4 5 5 5 3
14055 291/300 Read block 1058 len 40960
14056 292/300 Read block 4966 len 28672
14057 293/300 Write block 2112 len 8192 data: 4 5
14058 294/300 Read block 2377 len 28672
14059 295/300 Write block 3999 len 4096 data: 5
14060 296/300 Read block 4715 len 8192
14061 297/300 Flush
14062 298/300 Write block 2107 len 32768 data: 6 6 7 5 5 5 6 4
14063 299/300 Flush
14064 300/300 Write block 2755 len 28672 data: 4 3 3 2 2 2 2
14065 301/300 Write block 2630 len 4096 data: 5
14066 Main task has work to do, go find it
14067 got dsc Start(1)
14068 start 1
14069 Jan 17 21:14:03.616 INFO request completed, latency_us: 222, response_code: 204, uri: /start/cid/1, method: POST, req_id: 6039c11a-ca09-415a-b208-d496e71aa00d, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
14070 Replay: started 1, returned:Ok(())
14071 ----------------------------------------------------------------
14072 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
14073 States: Active Offline Active
14074 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14075 4622 Acked Read 6 Done Sent Done false
14076 4623 Acked Read 7 Done Sent Done false
14077 4624 Acked Write 5 Done Sent Done false
14078 4625 Acked Read 6 Done Sent Done false
14079 4626 Acked Read 4 Done Sent Done false
14080 4627 Acked Read 2 Done Sent Done false
14081 4628 Acked Read 8 Done Sent Done false
14082 4629 Acked Read 1 Done Sent Done false
14083 4630 Acked Read 3 Done Sent Done false
14084 4631 Acked Write 5 Done Sent Done false
14085 4632 Acked Read 6 Done Sent Done false
14086 4633 Acked Read 8 Done Sent Done false
14087 4634 Acked Read 10 Done Sent Done false
14088 4635 Acked Write 1 Done Sent Done false
14089 4636 Acked Read 6 Done Sent Done false
14090 4637 Acked Write 5 Done Sent Done false
14091 4638 Acked Write 1 Done Sent Done false
14092 4639 Acked Read 5 Done Sent Done false
14093 4640 Acked Read 2 Done Sent Done false
14094 4641 Acked Read 8 Done Sent Done false
14095 4642 Acked Read 1 Done Sent Done false
14096 4643 Acked Read 6 Done Sent Done false
14097 4644 Acked Read 2 Done Sent Done false
14098 4645 Acked Write 8 Done Sent Done false
14099 4646 Acked Write 5 Done Sent Done false
14100 4647 Acked Read 7 Done Sent Done false
14101 4648 Acked Write 9 Done Sent Done false
14102 4649 Acked Read 5 Done Sent Done false
14103 4650 Acked Read 6 Done Sent Done false
14104 4651 Acked Read 7 Done Sent Done false
14105 4652 Acked Flush 0 Done Sent Done false
14106 4653 Acked Write 8 Done Sent Done false
14107 4654 Acked Read 2 Done Sent Done false
14108 4655 Acked Read 5 Done Sent Done false
14109 4656 Acked Write 9 Done Sent Done false
14110 4657 Acked Read 2 Done Sent Done false
14111 4658 Acked Read 4 Done Sent Done false
14112 4659 Acked Read 4 Done Sent Done false
14113 4660 Acked Read 4 Done Sent Done false
14114 4661 Acked Write 4 Done Sent Done false
14115 4662 Acked Read 5 Done Sent Done false
14116 4663 Acked Read 1 Done Sent Done false
14117 4664 Acked Read 2 Done Sent Done false
14118 4665 Acked Write 2 Done Sent Done false
14119 4666 Acked Read 6 Done Sent Done false
14120 4667 Acked Read 5 Done Sent Done false
14121 4668 Acked Write 10 Done Sent Done false
14122 4669 Acked Flush 0 Done Sent Done false
14123 4670 Acked Read 7 Done Sent Done false
14124 4671 Acked Read 4 Done Sent Done false
14125 4672 Acked Read 6 Done Sent Done false
14126 4673 Acked Read 3 Done Sent Done false
14127 4674 Acked Read 5 Done Sent Done false
14128 4675 Acked Write 10 Done Sent Done false
14129 4676 Acked Write 7 Done Sent Done false
14130 4677 Acked Read 3 Done Sent Done false
14131 4678 Acked Write 8 Done Sent Done false
14132 4679 Acked Read 4 Done Sent Done false
14133 4680 Acked Flush 0 Done Sent Done false
14134 4681 Acked Write 7 Done Sent Done false
14135 4682 Acked Read 6 Done Sent Done false
14136 4683 Acked Read 9 Done Sent Done false
14137 4684 Acked Read 6 Done Sent Done false
14138 4685 Acked Read 6 Done Sent Done false
14139 4686 Acked Read 7 Done Sent Done false
14140 4687 Acked Read 5 Done Sent Done false
14141 4688 Acked Read 2 Done Sent Done false
14142 4689 Acked Write 7 Done Sent Done false
14143 4690 Acked Write 8 Done Sent Done false
14144 4691 Acked Read 9 Done Sent Done false
14145 4692 Acked Read 6 Done Sent Done false
14146 4693 Acked Read 7 Done Sent Done false
14147 4694 Acked Read 4 Done Sent Done false
14148 4695 Acked Write 10 Done Sent Done false
14149 4696 Acked Read 3 Done Sent Done false
14150 4697 Acked Flush 0 Done Sent Done false
14151 4698 Acked Read 8 Done Sent Done false
14152 4699 Acked Flush 0 Done Sent Done false
14153 4700 Acked Flush 0 Done Sent Done false
14154 4701 Acked Write 2 Done Sent Done false
14155 4702 Acked Flush 0 Done Sent Done false
14156 4703 Acked Write 8 Done Sent Done false
14157 4704 Acked Read 8 Done Sent Done false
14158 4705 Acked Write 4 Done Sent Done false
14159 4706 Acked Read 6 Done Sent Done false
14160 4707 Acked Write 3 Done Sent Done false
14161 4708 Acked Write 3 Done Sent Done false
14162 4709 Acked Read 1 Done Sent Done false
14163 4710 Acked Read 6 Done Sent Done false
14164 4711 Acked Write 2 Done Sent Done false
14165 4712 Acked Read 4 Done Sent Done false
14166 4713 Acked Write 2 Done Sent Done false
14167 4714 Acked Read 3 Done Sent Done false
14168 4715 Acked Write 2 Done Sent Done false
14169 4716 Acked Read 5 Done Sent Done false
14170 4717 Acked Read 8 Done Sent Done false
14171 4718 Acked Write 3 Done Sent Done false
14172 4719 Acked Write 3 Done Sent Done false
14173 4720 Acked Read 3 Done Sent Done false
14174 4721 Acked Write 1 Done Sent Done false
14175 4722 Acked Write 10 Done Sent Done false
14176 4723 Acked Read 9 Done Sent Done false
14177 4724 Acked Read 2 Done Sent Done false
14178 4725 Acked Read 5 Done Sent Done false
14179 4726 Acked Read 1 Done Sent Done false
14180 4727 Acked Write 9 Done Sent Done false
14181 4728 Acked Read 4 Done Sent Done false
14182 4729 Acked Read 2 Done Sent Done false
14183 4730 Acked Write 6 Done Sent Done false
14184 4731 Acked Read 6 Done Sent Done false
14185 4732 Acked Write 3 Done Sent Done false
14186 4733 Acked Read 8 Done Sent Done false
14187 4734 Acked Read 10 Done Sent Done false
14188 4735 Acked Read 1 Done Sent Done false
14189 4736 Acked Write 8 Done Sent Done false
14190 4737 Acked Read 8 Done Sent Done false
14191 4738 Acked Read 8 Done Sent Done false
14192 4739 Acked Write 10 Done Sent Done false
14193 4740 Acked Read 4 Done Sent Done false
14194 4741 Acked Write 4 Done Sent Done false
14195 4742 Acked Write 4 Done Sent Done false
14196 4743 Acked Flush 0 Done Sent Done false
14197 4744 Acked Read 3 Done Sent Done false
14198 4745 Acked Write 5 Done Sent Done false
14199 4746 Acked Read 5 Done Sent Done false
14200 4747 Acked Read 8 Done Sent Done false
14201 4748 Acked Flush 0 Done Sent Done false
14202 4749 Acked Read 5 Done Sent Done false
14203 4750 Acked Write 7 Done Sent Done false
14204 4751 Acked Flush 0 Done Sent Done false
14205 4752 Acked Flush 0 Done Sent Done false
14206 4753 Acked Write 8 Done Sent Done false
14207 4754 Acked Write 9 Done Sent Done false
14208 4755 Acked Read 2 Done Sent Done false
14209 4756 Acked Read 7 Done Sent Done false
14210 4757 Acked Read 10 Done Sent Done false
14211 4758 Acked Read 9 Done Sent Done false
14212 4759 Acked Read 3 Done Sent Done false
14213 4760 Acked Write 9 Done Sent Done false
14214 4761 Acked Write 2 Done Sent Done false
14215 4762 Acked Read 4 Done Sent Done false
14216 4763 Acked Write 3 Done Sent Done false
14217 4764 Acked Read 9 Done Sent Done false
14218 4765 Acked Read 4 Done Sent Done false
14219 4766 Acked Write 5 Done Sent Done false
14220 4767 Acked Write 4 Done Sent Done false
14221 4768 Acked Write 10 Done Sent Done false
14222 4769 Acked Write 10 Done Sent Done false
14223 4770 Acked Write 1 Done Sent Done false
14224 4771 Acked Read 6 Done Sent Done false
14225 4772 Acked Read 6 Done Sent Done false
14226 4773 Acked Write 1 Done Sent Done false
14227 4774 Acked Read 9 Done Sent Done false
14228 4775 Acked Write 8 Done Sent Done false
14229 4776 Acked Read 6 Done Sent Done false
14230 4777 Acked Read 4 Done Sent Done false
14231 4778 Acked Write 9 Done Sent Done false
14232 4779 Acked Write 8 Done Sent Done false
14233 4780 Acked Write 5 Done Sent Done false
14234 4781 Acked Write 10 Done Sent Done false
14235 4782 Acked Write 5 Done Sent Done false
14236 4783 Acked Read 9 Done Sent Done false
14237 4784 Acked Read 6 Done Sent Done false
14238 4785 Acked Flush 0 Done Sent Done false
14239 4786 Acked Read 1 Done Sent Done false
14240 4787 Acked Read 7 Done Sent Done false
14241 4788 Acked Read 4 Done Sent Done false
14242 4789 Acked Write 5 Done Sent Done false
14243 4790 Acked Read 7 Done Sent Done false
14244 4791 Acked Write 2 Done Sent Done false
14245 4792 Acked Read 4 Done Sent Done false
14246 4793 Acked Read 1 Done Sent Done false
14247 4794 Acked Write 8 Done Sent Done false
14248 4795 Acked Read 5 Done Sent Done false
14249 4796 Acked Write 1 Done Sent Done false
14250 4797 Acked Read 4 Done Sent Done false
14251 4798 Acked Write 10 Done Sent Done false
14252 4799 Acked Write 2 Done Sent Done false
14253 4800 Acked Read 4 Done Sent Done false
14254 4801 Acked Read 1 Done Sent Done false
14255 4802 Acked Read 8 Done Sent Done false
14256 4803 Acked Read 7 Done Sent Done false
14257 4804 Acked Write 7 Done Sent Done false
14258 4805 Acked Read 2 Done Sent Done false
14259 4806 Acked Write 4 Done Sent Done false
14260 4807 Acked Read 2 Done Sent Done false
14261 4808 Acked Read 1 Done Sent Done false
14262 4809 Acked Read 5 Done Sent Done false
14263 4810 Acked Read 1 Done Sent Done false
14264 4811 Acked Write 2 Done Sent Done false
14265 4812 Acked Read 9 Done Sent Done false
14266 4813 Acked Read 5 Done Sent Done false
14267 4814 Acked Write 7 Done Sent Done false
14268 4815 Acked Read 4 Done Sent Done false
14269 4816 Acked Read 1 Done Sent Done false
14270 4817 Acked Write 6 Done Sent Done false
14271 4818 Acked Read 4 Done Sent Done false
14272 4819 Acked Write 3 Done Sent Done false
14273 4820 Acked Read 6 Done Sent Done false
14274 4821 Acked Write 10 Done Sent Done false
14275 4822 Acked Read 10 Done Sent Done false
14276 4823 Acked Write 3 Done Sent Done false
14277 4824 Acked Read 2 Done Sent Done false
14278 4825 Acked Read 7 Done Sent Done false
14279 4826 Acked Read 9 Done Sent Done false
14280 4827 Acked Write 4 Done Sent Done false
14281 4828 Acked Write 1 Done Sent Done false
14282 4829 Acked Write 7 Done Sent Done false
14283 4830 Acked Write 3 Done Sent Done false
14284 4831 Acked Read 2 Done Sent Done false
14285 4832 Acked Write 10 Done Sent Done false
14286 4833 Acked Write 9 Done Sent Done false
14287 4834 Acked Read 9 Done Sent Done false
14288 4835 Acked Read 5 Done Sent Done false
14289 4836 Acked Read 1 Done Sent Done false
14290 4837 Acked Write 6 Done Sent Done false
14291 4838 Acked Read 5 Done Sent Done false
14292 4839 Acked Read 10 Done Sent Done false
14293 4840 Acked Read 7 Done Sent Done false
14294 4841 Acked Read 4 Done Sent Done false
14295 4842 Acked Flush 0 Done Sent Done false
14296 4843 Acked Write 9 Done Sent Done false
14297 4844 Acked Read 4 Done Sent Done false
14298 4845 Acked Read 1 Done Sent Done false
14299 4846 Acked Write 4 Done Sent Done false
14300 4847 Acked Write 2 Done Sent Done false
14301 4848 Acked Read 2 Done Sent Done false
14302 4849 Acked Write 6 Done Sent Done false
14303 4850 Acked Write 9 Done Sent Done false
14304 4851 Acked Read 4 Done Sent Done false
14305 4852 Acked Write 5 Done Sent Done false
14306 4853 Acked Write 1 Done Sent Done false
14307 4854 Acked Write 9 Done Sent Done false
14308 4855 Acked Read 4 Done Sent Done false
14309 4856 Acked Read 4 Done Sent Done false
14310 4857 Acked Read 6 Done Sent Done false
14311 4858 Acked Read 4 Done Sent Done false
14312 4859 Acked Read 4 Done Sent Done false
14313 4860 Acked Write 8 Done Sent Done false
14314 4861 Acked Read 2 Done Sent Done false
14315 4862 Acked Flush 0 Done Sent Done false
14316 4863 Acked Read 6 Done Sent Done false
14317 4864 Acked Write 5 Done Sent Done false
14318 4865 Acked Write 9 Done Sent Done false
14319 4866 Acked Read 1 Done Sent Done false
14320 4867 Acked Write 8 Done Sent Done false
14321 4868 Acked Read 8 Done Sent Done false
14322 4869 Acked Read 3 Done Sent Done false
14323 4870 Acked Read 9 Done Sent Done false
14324 4871 Acked Write 10 Done Sent Done false
14325 4872 Acked Read 9 Done Sent Done false
14326 4873 Acked Write 5 Done Sent Done false
14327 4874 Acked Read 1 Done Sent Done false
14328 4875 Acked Read 2 Done Sent Done false
14329 4876 Acked Write 9 Done Sent Done false
14330 4877 Acked Read 7 Done Sent Done false
14331 4878 Acked Write 9 Done Sent Done false
14332 4879 Acked Read 4 Done Sent Done false
14333 4880 Acked Flush 0 Done Sent Done false
14334 4881 Acked Write 8 Done Sent Done false
14335 4882 Acked Write 10 Done Sent Done false
14336 4883 Acked Read 5 Done Sent Done false
14337 4884 Acked Read 6 Done Sent Done false
14338 4885 Acked Read 3 Done Sent Done false
14339 4886 Acked Write 5 Done Sent Done false
14340 4887 Acked Read 1 Done Sent Done false
14341 4888 Acked Read 3 Done Sent Done false
14342 4889 Acked Read 3 Done Sent Done false
14343 4890 Acked Read 1 Done Sent Done false
14344 4891 Acked Read 8 Done Sent Done false
14345 4892 Acked Read 10 Done Sent Done false
14346 4893 Acked Read 4 Done Sent Done false
14347 4894 Acked Write 6 Done Sent Done false
14348 4895 Acked Write 9 Done Sent Done false
14349 4896 Acked Flush 0 Done Sent Done false
14350 4897 Acked Write 4 Done Sent Done false
14351 4898 Acked Write 2 Done Sent Done false
14352 4899 Acked Read 10 Done Sent Done false
14353 4900 Acked Flush 0 Done Sent Done false
14354 4901 Acked Write 7 Done Sent Done false
14355 4902 Acked Read 1 Done Sent Done false
14356 4903 Acked Flush 0 Done Sent Done false
14357 4904 Acked Flush 0 Done Sent Done false
14358 4905 Acked Write 4 Done Sent Done false
14359 4906 Acked Flush 0 Done Sent Done false
14360 4907 Acked Write 7 Done Sent Done false
14361 4908 Acked Flush 0 Done Sent Done false
14362 4909 Acked Read 5 Done Sent Done false
14363 4910 Acked Read 6 Done Sent Done false
14364 4911 Acked Write 7 Done Sent Done false
14365 4912 Acked Read 10 Done Sent Done false
14366 4913 Acked Read 7 Done Sent Done false
14367 4914 Acked Write 2 Done Sent Done false
14368 4915 Acked Read 7 Done Sent Done false
14369 4916 Acked Write 1 Done Sent Done false
14370 4917 Acked Read 2 Done Sent Done false
14371 4918 Acked Flush 0 Done Sent Done false
14372 4919 Acked Write 8 Done Sent Done false
14373 4920 Acked Flush 0 Done Sent Done false
14374 4921 Acked Write 7 Done Sent Done false
14375 4922 Acked Write 1 Done Sent Done false
14376 STATES DS:0 DS:1 DS:2 TOTAL
14377 Sent 0 301 0 301
14378 Done 301 0 301 602
14379 Skipped 0 0 0 0
14380 Error 0 0 0 0
14381 Last Flush: 4920 4621 4920
14382 Downstairs last five completed: 4621 4620 4619 4618 4617
14383 Upstairs last five completed: 4922 4921 4920 4919 4918
14384 CLIENT: Up:0 ds:301 act:2
14385 [8820] got start action
14386 [8820] I am going to restart
14387 Starting downstairs at port 8820
14388 Make output file at "/tmp/dsc/downstairs-8820.txt"
14389 [8820][1] reports Starting
14390 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1213)
14391 [8820][1] reports Running
14392 ----------------------------------------------------------------
14393 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14394 States: Active Offline Active
14395 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14396 4622 Acked Read 6 Done Sent Done false
14397 4623 Acked Read 7 Done Sent Done false
14398 4624 Acked Write 5 Done Sent Done false
14399 4625 Acked Read 6 Done Sent Done false
14400 4626 Acked Read 4 Done Sent Done false
14401 4627 Acked Read 2 Done Sent Done false
14402 4628 Acked Read 8 Done Sent Done false
14403 4629 Acked Read 1 Done Sent Done false
14404 4630 Acked Read 3 Done Sent Done false
14405 4631 Acked Write 5 Done Sent Done false
14406 4632 Acked Read 6 Done Sent Done false
14407 4633 Acked Read 8 Done Sent Done false
14408 4634 Acked Read 10 Done Sent Done false
14409 4635 Acked Write 1 Done Sent Done false
14410 4636 Acked Read 6 Done Sent Done false
14411 4637 Acked Write 5 Done Sent Done false
14412 4638 Acked Write 1 Done Sent Done false
14413 4639 Acked Read 5 Done Sent Done false
14414 4640 Acked Read 2 Done Sent Done false
14415 4641 Acked Read 8 Done Sent Done false
14416 4642 Acked Read 1 Done Sent Done false
14417 4643 Acked Read 6 Done Sent Done false
14418 4644 Acked Read 2 Done Sent Done false
14419 4645 Acked Write 8 Done Sent Done false
14420 4646 Acked Write 5 Done Sent Done false
14421 4647 Acked Read 7 Done Sent Done false
14422 4648 Acked Write 9 Done Sent Done false
14423 4649 Acked Read 5 Done Sent Done false
14424 4650 Acked Read 6 Done Sent Done false
14425 4651 Acked Read 7 Done Sent Done false
14426 4652 Acked Flush 0 Done Sent Done false
14427 4653 Acked Write 8 Done Sent Done false
14428 4654 Acked Read 2 Done Sent Done false
14429 4655 Acked Read 5 Done Sent Done false
14430 4656 Acked Write 9 Done Sent Done false
14431 4657 Acked Read 2 Done Sent Done false
14432 4658 Acked Read 4 Done Sent Done false
14433 4659 Acked Read 4 Done Sent Done false
14434 4660 Acked Read 4 Done Sent Done false
14435 4661 Acked Write 4 Done Sent Done false
14436 4662 Acked Read 5 Done Sent Done false
14437 4663 Acked Read 1 Done Sent Done false
14438 4664 Acked Read 2 Done Sent Done false
14439 4665 Acked Write 2 Done Sent Done false
14440 4666 Acked Read 6 Done Sent Done false
14441 4667 Acked Read 5 Done Sent Done false
14442 4668 Acked Write 10 Done Sent Done false
14443 4669 Acked Flush 0 Done Sent Done false
14444 4670 Acked Read 7 Done Sent Done false
14445 4671 Acked Read 4 Done Sent Done false
14446 4672 Acked Read 6 Done Sent Done false
14447 4673 Acked Read 3 Done Sent Done false
14448 4674 Acked Read 5 Done Sent Done false
14449 4675 Acked Write 10 Done Sent Done false
14450 4676 Acked Write 7 Done Sent Done false
14451 4677 Acked Read 3 Done Sent Done false
14452 4678 Acked Write 8 Done Sent Done false
14453 4679 Acked Read 4 Done Sent Done false
14454 4680 Acked Flush 0 Done Sent Done false
14455 4681 Acked Write 7 Done Sent Done false
14456 4682 Acked Read 6 Done Sent Done false
14457 4683 Acked Read 9 Done Sent Done false
14458 4684 Acked Read 6 Done Sent Done false
14459 4685 Acked Read 6 Done Sent Done false
14460 4686 Acked Read 7 Done Sent Done false
14461 4687 Acked Read 5 Done Sent Done false
14462 4688 Acked Read 2 Done Sent Done false
14463 4689 Acked Write 7 Done Sent Done false
14464 4690 Acked Write 8 Done Sent Done false
14465 4691 Acked Read 9 Done Sent Done false
14466 4692 Acked Read 6 Done Sent Done false
14467 4693 Acked Read 7 Done Sent Done false
14468 4694 Acked Read 4 Done Sent Done false
14469 4695 Acked Write 10 Done Sent Done false
14470 4696 Acked Read 3 Done Sent Done false
14471 4697 Acked Flush 0 Done Sent Done false
14472 4698 Acked Read 8 Done Sent Done false
14473 4699 Acked Flush 0 Done Sent Done false
14474 4700 Acked Flush 0 Done Sent Done false
14475 4701 Acked Write 2 Done Sent Done false
14476 4702 Acked Flush 0 Done Sent Done false
14477 4703 Acked Write 8 Done Sent Done false
14478 4704 Acked Read 8 Done Sent Done false
14479 4705 Acked Write 4 Done Sent Done false
14480 4706 Acked Read 6 Done Sent Done false
14481 4707 Acked Write 3 Done Sent Done false
14482 4708 Acked Write 3 Done Sent Done false
14483 4709 Acked Read 1 Done Sent Done false
14484 4710 Acked Read 6 Done Sent Done false
14485 4711 Acked Write 2 Done Sent Done false
14486 4712 Acked Read 4 Done Sent Done false
14487 4713 Acked Write 2 Done Sent Done false
14488 4714 Acked Read 3 Done Sent Done false
14489 4715 Acked Write 2 Done Sent Done false
14490 4716 Acked Read 5 Done Sent Done false
14491 4717 Acked Read 8 Done Sent Done false
14492 4718 Acked Write 3 Done Sent Done false
14493 4719 Acked Write 3 Done Sent Done false
14494 4720 Acked Read 3 Done Sent Done false
14495 4721 Acked Write 1 Done Sent Done false
14496 4722 Acked Write 10 Done Sent Done false
14497 4723 Acked Read 9 Done Sent Done false
14498 4724 Acked Read 2 Done Sent Done false
14499 4725 Acked Read 5 Done Sent Done false
14500 4726 Acked Read 1 Done Sent Done false
14501 4727 Acked Write 9 Done Sent Done false
14502 4728 Acked Read 4 Done Sent Done false
14503 4729 Acked Read 2 Done Sent Done false
14504 4730 Acked Write 6 Done Sent Done false
14505 4731 Acked Read 6 Done Sent Done false
14506 4732 Acked Write 3 Done Sent Done false
14507 4733 Acked Read 8 Done Sent Done false
14508 4734 Acked Read 10 Done Sent Done false
14509 4735 Acked Read 1 Done Sent Done false
14510 4736 Acked Write 8 Done Sent Done false
14511 4737 Acked Read 8 Done Sent Done false
14512 4738 Acked Read 8 Done Sent Done false
14513 4739 Acked Write 10 Done Sent Done false
14514 4740 Acked Read 4 Done Sent Done false
14515 4741 Acked Write 4 Done Sent Done false
14516 4742 Acked Write 4 Done Sent Done false
14517 4743 Acked Flush 0 Done Sent Done false
14518 4744 Acked Read 3 Done Sent Done false
14519 4745 Acked Write 5 Done Sent Done false
14520 4746 Acked Read 5 Done Sent Done false
14521 4747 Acked Read 8 Done Sent Done false
14522 4748 Acked Flush 0 Done Sent Done false
14523 4749 Acked Read 5 Done Sent Done false
14524 4750 Acked Write 7 Done Sent Done false
14525 4751 Acked Flush 0 Done Sent Done false
14526 4752 Acked Flush 0 Done Sent Done false
14527 4753 Acked Write 8 Done Sent Done false
14528 4754 Acked Write 9 Done Sent Done false
14529 4755 Acked Read 2 Done Sent Done false
14530 4756 Acked Read 7 Done Sent Done false
14531 4757 Acked Read 10 Done Sent Done false
14532 4758 Acked Read 9 Done Sent Done false
14533 4759 Acked Read 3 Done Sent Done false
14534 4760 Acked Write 9 Done Sent Done false
14535 4761 Acked Write 2 Done Sent Done false
14536 4762 Acked Read 4 Done Sent Done false
14537 4763 Acked Write 3 Done Sent Done false
14538 4764 Acked Read 9 Done Sent Done false
14539 4765 Acked Read 4 Done Sent Done false
14540 4766 Acked Write 5 Done Sent Done false
14541 4767 Acked Write 4 Done Sent Done false
14542 4768 Acked Write 10 Done Sent Done false
14543 4769 Acked Write 10 Done Sent Done false
14544 4770 Acked Write 1 Done Sent Done false
14545 4771 Acked Read 6 Done Sent Done false
14546 4772 Acked Read 6 Done Sent Done false
14547 4773 Acked Write 1 Done Sent Done false
14548 4774 Acked Read 9 Done Sent Done false
14549 4775 Acked Write 8 Done Sent Done false
14550 4776 Acked Read 6 Done Sent Done false
14551 4777 Acked Read 4 Done Sent Done false
14552 4778 Acked Write 9 Done Sent Done false
14553 4779 Acked Write 8 Done Sent Done false
14554 4780 Acked Write 5 Done Sent Done false
14555 4781 Acked Write 10 Done Sent Done false
14556 4782 Acked Write 5 Done Sent Done false
14557 4783 Acked Read 9 Done Sent Done false
14558 4784 Acked Read 6 Done Sent Done false
14559 4785 Acked Flush 0 Done Sent Done false
14560 4786 Acked Read 1 Done Sent Done false
14561 4787 Acked Read 7 Done Sent Done false
14562 4788 Acked Read 4 Done Sent Done false
14563 4789 Acked Write 5 Done Sent Done false
14564 4790 Acked Read 7 Done Sent Done false
14565 4791 Acked Write 2 Done Sent Done false
14566 4792 Acked Read 4 Done Sent Done false
14567 4793 Acked Read 1 Done Sent Done false
14568 4794 Acked Write 8 Done Sent Done false
14569 4795 Acked Read 5 Done Sent Done false
14570 4796 Acked Write 1 Done Sent Done false
14571 4797 Acked Read 4 Done Sent Done false
14572 4798 Acked Write 10 Done Sent Done false
14573 4799 Acked Write 2 Done Sent Done false
14574 4800 Acked Read 4 Done Sent Done false
14575 4801 Acked Read 1 Done Sent Done false
14576 4802 Acked Read 8 Done Sent Done false
14577 4803 Acked Read 7 Done Sent Done false
14578 4804 Acked Write 7 Done Sent Done false
14579 4805 Acked Read 2 Done Sent Done false
14580 4806 Acked Write 4 Done Sent Done false
14581 4807 Acked Read 2 Done Sent Done false
14582 4808 Acked Read 1 Done Sent Done false
14583 4809 Acked Read 5 Done Sent Done false
14584 4810 Acked Read 1 Done Sent Done false
14585 4811 Acked Write 2 Done Sent Done false
14586 4812 Acked Read 9 Done Sent Done false
14587 4813 Acked Read 5 Done Sent Done false
14588 4814 Acked Write 7 Done Sent Done false
14589 4815 Acked Read 4 Done Sent Done false
14590 4816 Acked Read 1 Done Sent Done false
14591 4817 Acked Write 6 Done Sent Done false
14592 4818 Acked Read 4 Done Sent Done false
14593 4819 Acked Write 3 Done Sent Done false
14594 4820 Acked Read 6 Done Sent Done false
14595 4821 Acked Write 10 Done Sent Done false
14596 4822 Acked Read 10 Done Sent Done false
14597 4823 Acked Write 3 Done Sent Done false
14598 4824 Acked Read 2 Done Sent Done false
14599 4825 Acked Read 7 Done Sent Done false
14600 4826 Acked Read 9 Done Sent Done false
14601 4827 Acked Write 4 Done Sent Done false
14602 4828 Acked Write 1 Done Sent Done false
14603 4829 Acked Write 7 Done Sent Done false
14604 4830 Acked Write 3 Done Sent Done false
14605 4831 Acked Read 2 Done Sent Done false
14606 4832 Acked Write 10 Done Sent Done false
14607 4833 Acked Write 9 Done Sent Done false
14608 4834 Acked Read 9 Done Sent Done false
14609 4835 Acked Read 5 Done Sent Done false
14610 4836 Acked Read 1 Done Sent Done false
14611 4837 Acked Write 6 Done Sent Done false
14612 4838 Acked Read 5 Done Sent Done false
14613 4839 Acked Read 10 Done Sent Done false
14614 4840 Acked Read 7 Done Sent Done false
14615 4841 Acked Read 4 Done Sent Done false
14616 4842 Acked Flush 0 Done Sent Done false
14617 4843 Acked Write 9 Done Sent Done false
14618 4844 Acked Read 4 Done Sent Done false
14619 4845 Acked Read 1 Done Sent Done false
14620 4846 Acked Write 4 Done Sent Done false
14621 4847 Acked Write 2 Done Sent Done false
14622 4848 Acked Read 2 Done Sent Done false
14623 4849 Acked Write 6 Done Sent Done false
14624 4850 Acked Write 9 Done Sent Done false
14625 4851 Acked Read 4 Done Sent Done false
14626 4852 Acked Write 5 Done Sent Done false
14627 4853 Acked Write 1 Done Sent Done false
14628 4854 Acked Write 9 Done Sent Done false
14629 4855 Acked Read 4 Done Sent Done false
14630 4856 Acked Read 4 Done Sent Done false
14631 4857 Acked Read 6 Done Sent Done false
14632 4858 Acked Read 4 Done Sent Done false
14633 4859 Acked Read 4 Done Sent Done false
14634 4860 Acked Write 8 Done Sent Done false
14635 4861 Acked Read 2 Done Sent Done false
14636 4862 Acked Flush 0 Done Sent Done false
14637 4863 Acked Read 6 Done Sent Done false
14638 4864 Acked Write 5 Done Sent Done false
14639 4865 Acked Write 9 Done Sent Done false
14640 4866 Acked Read 1 Done Sent Done false
14641 4867 Acked Write 8 Done Sent Done false
14642 4868 Acked Read 8 Done Sent Done false
14643 4869 Acked Read 3 Done Sent Done false
14644 4870 Acked Read 9 Done Sent Done false
14645 4871 Acked Write 10 Done Sent Done false
14646 4872 Acked Read 9 Done Sent Done false
14647 4873 Acked Write 5 Done Sent Done false
14648 4874 Acked Read 1 Done Sent Done false
14649 4875 Acked Read 2 Done Sent Done false
14650 4876 Acked Write 9 Done Sent Done false
14651 4877 Acked Read 7 Done Sent Done false
14652 4878 Acked Write 9 Done Sent Done false
14653 4879 Acked Read 4 Done Sent Done false
14654 4880 Acked Flush 0 Done Sent Done false
14655 4881 Acked Write 8 Done Sent Done false
14656 4882 Acked Write 10 Done Sent Done false
14657 4883 Acked Read 5 Done Sent Done false
14658 4884 Acked Read 6 Done Sent Done false
14659 4885 Acked Read 3 Done Sent Done false
14660 4886 Acked Write 5 Done Sent Done false
14661 4887 Acked Read 1 Done Sent Done false
14662 4888 Acked Read 3 Done Sent Done false
14663 4889 Acked Read 3 Done Sent Done false
14664 4890 Acked Read 1 Done Sent Done false
14665 4891 Acked Read 8 Done Sent Done false
14666 4892 Acked Read 10 Done Sent Done false
14667 4893 Acked Read 4 Done Sent Done false
14668 4894 Acked Write 6 Done Sent Done false
14669 4895 Acked Write 9 Done Sent Done false
14670 4896 Acked Flush 0 Done Sent Done false
14671 4897 Acked Write 4 Done Sent Done false
14672 4898 Acked Write 2 Done Sent Done false
14673 4899 Acked Read 10 Done Sent Done false
14674 4900 Acked Flush 0 Done Sent Done false
14675 4901 Acked Write 7 Done Sent Done false
14676 4902 Acked Read 1 Done Sent Done false
14677 4903 Acked Flush 0 Done Sent Done false
14678 4904 Acked Flush 0 Done Sent Done false
14679 4905 Acked Write 4 Done Sent Done false
14680 4906 Acked Flush 0 Done Sent Done false
14681 4907 Acked Write 7 Done Sent Done false
14682 4908 Acked Flush 0 Done Sent Done false
14683 4909 Acked Read 5 Done Sent Done false
14684 4910 Acked Read 6 Done Sent Done false
14685 4911 Acked Write 7 Done Sent Done false
14686 4912 Acked Read 10 Done Sent Done false
14687 4913 Acked Read 7 Done Sent Done false
14688 4914 Acked Write 2 Done Sent Done false
14689 4915 Acked Read 7 Done Sent Done false
14690 4916 Acked Write 1 Done Sent Done false
14691 4917 Acked Read 2 Done Sent Done false
14692 4918 Acked Flush 0 Done Sent Done false
14693 4919 Acked Write 8 Done Sent Done false
14694 4920 Acked Flush 0 Done Sent Done false
14695 4921 Acked Write 7 Done Sent Done false
14696 4922 Acked Write 1 Done Sent Done false
14697 4923 Acked Flush 0 Done Sent Done false
14698 STATES DS:0 DS:1 DS:2 TOTAL
14699 Sent 0 302 0 302
14700 Done 302 0 302 604
14701 Skipped 0 0 0 0
14702 Error 0 0 0 0
14703 Last Flush: 4923 4621 4923
14704 Downstairs last five completed: 4621 4620 4619 4618 4617
14705 Upstairs last five completed: 4923 4922 4921 4920 4919
14706 CLIENT: Up:0 ds:302 act:2
147072025-01-17T21:14:09.365ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
147082025-01-17T21:14:09.365ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:47772) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
147092025-01-17T21:14:09.366ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
147102025-01-17T21:14:09.367ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
147112025-01-17T21:14:09.367ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
147122025-01-17T21:14:09.367ZINFOcrucible: send last flush ID to this DS: 4621 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
147132025-01-17T21:14:09.367ZINFOcrucible: Replied this last flush ID: 4621 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
147142025-01-17T21:14:09.367ZINFOcrucible: [1] client re-new 302 jobs since flush 4621 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
147152025-01-17T21:14:09.367ZINFOcrucible: [1] Replayed 302 jobs since flush: 4621 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
14716 ----------------------------------------------------------------
14717 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
14718 Downstairs last five completed: 4923 4922 4921 4920 4919
14719 Upstairs last five completed: 4923 4922 4921 4920 4919
14720 CLIENT: Up:0 ds:0 act:3
14721 Replay: All jobs finished
14722 Main task has work to do, go find it
14723 got dsc Stop(2)
14724 stop 2
14725 [8830] Got stop action so:false kr:false
14726 Jan 17 21:14:11.620 INFO request completed, latency_us: 341, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 8461d2bc-462a-4f7d-9f85-fd594ee3bf50, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
147272025-01-17T21:14:11.621ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
147282025-01-17T21:14:11.621ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
147292025-01-17T21:14:11.621ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
147302025-01-17T21:14:11.621ZINFOcrucible: client task is exiting = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
14731 Jan 17 21:14:11.621 INFO request completed, latency_us: 150, response_code: 200, uri: /state/cid/2, method: GET, req_id: 69b9fd97-e0cf-4418-8f85-e8b5a26f447a, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
14732 [8830] Exited with: ExitStatus(unix_wait_status(9))
14733 [8830][2] reports Exit
14734 Jan 17 21:14:15.623 INFO request completed, latency_us: 190, response_code: 200, uri: /state/cid/2, method: GET, req_id: 11cdf30d-395a-413b-97b4-bbd33e9034ee, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
14735 001/300 Flush
14736 002/300 Write block 3508 len 32768 data: 3 4 4 4 5 5 5 6
14737 003/300 Read block 1250 len 12288
14738 004/300 Read block 2577 len 36864
14739 005/300 Write block 747 len 16384 data: 3 3 3 3
14740 006/300 Flush
14741 007/300 Write block 1486 len 28672 data: 5 4 3 3 3 3 2
14742 008/300 Write block 562 len 4096 data: 5
14743 009/300 Write block 4483 len 16384 data: 3 3 3 3
14744 010/300 Write block 2210 len 12288 data: 2 2 2
14745 011/300 Read block 3494 len 12288
14746 012/300 Flush
14747 013/300 Write block 1703 len 36864 data: 3 3 3 4 4 5 7 6 4
14748 014/300 Flush
14749 015/300 Flush
14750 016/300 Read block 3042 len 36864
14751 017/300 Flush
14752 018/300 Read block 4846 len 36864
14753 019/300 Read block 4835 len 28672
14754 020/300 Read block 482 len 28672
14755 021/300 Write block 1557 len 28672 data: 3 3 3 3 2 3 3
14756 022/300 Write block 2266 len 28672 data: 7 5 3 3 3 3 3
14757 023/300 Write block 1622 len 20480 data: 2 3 3 2 3
14758 024/300 Write block 3036 len 4096 data: 3
14759 025/300 Write block 2792 len 32768 data: 5 4 4 4 4 4 3 3
14760 026/300 Write block 2790 len 24576 data: 4 5 6 5 5 5
14761 027/300 Write block 826 len 24576 data: 4 4 4 4 4 4
14762 028/300 Read block 4758 len 28672
14763 029/300 Flush
14764 030/300 Read block 1524 len 32768
14765 031/300 Write block 2723 len 36864 data: 3 2 2 2 5 5 5 5 5
14766 032/300 Read block 173 len 28672
14767 033/300 Write block 665 len 36864 data: 4 5 5 3 3 3 3 3 2
14768 034/300 Read block 2588 len 20480
14769 035/300 Flush
14770 036/300 Read block 3569 len 24576
14771 037/300 Flush
14772 038/300 Flush
14773 039/300 Read block 903 len 12288
14774 040/300 Write block 4287 len 40960 data: 3 3 3 3 3 3 3 4 4 4
14775 041/300 Write block 3449 len 36864 data: 7 7 6 5 4 3 2 2 2
14776 042/300 Read block 4613 len 20480
14777 043/300 Read block 3082 len 32768
14778 044/300 Read block 2111 len 20480
14779 045/300 Read block 1417 len 24576
14780 046/300 Write block 2778 len 32768 data: 4 3 3 3 2 2 3 3
14781 047/300 Read block 2439 len 4096
14782 048/300 Read block 3705 len 36864
14783 049/300 Read block 3262 len 36864
14784 050/300 Flush
14785 051/300 Read block 3946 len 36864
14786 052/300 Read block 1849 len 16384
14787 053/300 Write block 4576 len 24576 data: 4 3 3 3 3 4
14788 054/300 Write block 3530 len 24576 data: 4 3 3 3 2 2
14789 055/300 Read block 4976 len 32768
14790 056/300 Read block 2236 len 40960
14791 057/300 Read block 575 len 24576
14792 058/300 Read block 3297 len 32768
14793 059/300 Read block 3876 len 32768
14794 060/300 Read block 3548 len 36864
14795 061/300 Flush
14796 062/300 Flush
14797 063/300 Write block 1062 len 16384 data: 7 6 7 6
14798 064/300 Read block 4447 len 20480
14799 065/300 Read block 3488 len 16384
14800 066/300 Flush
14801 067/300 Write block 4984 len 8192 data: 2 2
14802 068/300 Read block 4101 len 16384
14803 069/300 Read block 2868 len 32768
14804 070/300 Write block 1649 len 12288 data: 3 3 3
14805 071/300 Write block 2687 len 8192 data: 4 4
14806 072/300 Read block 3161 len 8192
14807 073/300 Read block 3765 len 28672
14808 074/300 Write block 2190 len 36864 data: 4 3 3 3 3 2 2 2 2
14809 075/300 Write block 223 len 40960 data: 3 3 3 3 3 3 4 4 4 4
14810 076/300 Read block 1607 len 28672
14811 077/300 Read block 1522 len 28672
14812 078/300 Read block 4471 len 40960
14813 079/300 Write block 4412 len 24576 data: 3 4 3 3 6 6
14814 080/300 Write block 2904 len 36864 data: 4 3 2 2 2 2 2 3 3
14815 081/300 Read block 1282 len 36864
14816 082/300 Write block 3761 len 40960 data: 3 3 3 3 3 4 4 4 4 3
14817 083/300 Read block 550 len 28672
14818 084/300 Read block 4246 len 36864
14819 085/300 Flush
14820 086/300 Write block 2943 len 16384 data: 5 6 5 4
14821 087/300 Write block 4666 len 40960 data: 3 3 3 3 3 4 4 5 5 6
14822 088/300 Write block 3633 len 20480 data: 2 2 3 3 2
14823 089/300 Flush
14824 090/300 Write block 4883 len 12288 data: 2 2 2
14825 091/300 Read block 2369 len 4096
14826 092/300 Write block 1981 len 40960 data: 4 3 3 3 4 4 4 4 6 5
14827 093/300 Write block 237 len 12288 data: 2 2 3
14828 094/300 Write block 1584 len 32768 data: 9 7 6 5 4 5 4 4
14829 095/300 Write block 3313 len 36864 data: 3 4 4 5 5 5 6 5 4
14830 096/300 Flush
14831 097/300 Write block 100 len 24576 data: 4 6 5 6 6 5
14832 098/300 Flush
14833 099/300 Read block 1245 len 8192
14834 100/300 Flush
14835 101/300 Read block 4073 len 36864
14836 102/300 Read block 1772 len 16384
14837 103/300 Read block 3932 len 20480
14838 104/300 Read block 2897 len 32768
14839 105/300 Read block 2366 len 8192
14840 106/300 Flush
14841 107/300 Read block 2990 len 8192
14842 108/300 Read block 4218 len 8192
14843 109/300 Read block 384 len 32768
14844 110/300 Write block 2478 len 12288 data: 6 5 5
14845 111/300 Read block 1377 len 32768
14846 112/300 Write block 3353 len 4096 data: 3
14847 113/300 Read block 1411 len 8192
14848 114/300 Flush
14849 115/300 Read block 727 len 4096
14850 116/300 Read block 2538 len 40960
14851 117/300 Write block 3782 len 40960 data: 4 3 3 3 3 3 4 3 3 2
14852 118/300 Write block 4156 len 4096 data: 5
14853 119/300 Read block 2897 len 32768
14854 120/300 Flush
14855 121/300 Write block 1448 len 20480 data: 3 3 2 2 2
14856 122/300 Read block 398 len 8192
14857 123/300 Read block 4711 len 28672
14858 124/300 Write block 3100 len 36864 data: 3 4 4 5 6 6 4 4 4
14859 125/300 Write block 2491 len 20480 data: 6 5 4 4 3
14860 126/300 Write block 649 len 20480 data: 3 4 4 4 4
14861 127/300 Read block 756 len 24576
14862 128/300 Read block 3888 len 4096
14863 129/300 Write block 75 len 32768 data: 5 5 5 4 3 3 3 2
14864 130/300 Write block 1633 len 4096 data: 3
14865 131/300 Write block 1322 len 40960 data: 3 3 3 3 4 4 5 5 5 5
14866 132/300 Write block 874 len 24576 data: 4 4 4 6 7 6
14867 133/300 Read block 532 len 16384
14868 134/300 Read block 4009 len 32768
14869 135/300 Flush
14870 136/300 Write block 1864 len 24576 data: 4 3 2 3 2 2
14871 137/300 Read block 1210 len 16384
14872 138/300 Read block 1128 len 12288
14873 139/300 Write block 3664 len 16384 data: 3 3 3 3
14874 140/300 Write block 2585 len 20480 data: 4 6 6 6 6
14875 141/300 Read block 1205 len 8192
14876 142/300 Flush
14877 143/300 Read block 1558 len 8192
14878 144/300 Read block 1668 len 16384
14879 145/300 Write block 2643 len 16384 data: 2 3 5 4
14880 146/300 Read block 4058 len 12288
14881 147/300 Read block 151 len 4096
14882 148/300 Flush
14883 149/300 Read block 2223 len 16384
14884 150/300 Write block 3616 len 16384 data: 5 5 5 4
14885 151/300 Read block 353 len 36864
14886 152/300 Read block 2976 len 20480
14887 153/300 Flush
14888 154/300 Read block 1709 len 24576
14889 155/300 Read block 3214 len 12288
14890 156/300 Read block 1779 len 36864
14891 157/300 Read block 4990 len 32768
14892 158/300 Read block 2992 len 8192
14893 159/300 Read block 2491 len 24576
14894 160/300 Write block 4812 len 36864 data: 3 3 3 4 3 3 3 2 3
14895 161/300 Read block 3862 len 28672
14896 162/300 Read block 2512 len 4096
14897 163/300 Write block 2517 len 12288 data: 5 4 4
14898 164/300 Write block 1574 len 24576 data: 3 3 3 3 3 4
14899 165/300 Write block 2976 len 20480 data: 4 2 2 2 3
14900 166/300 Read block 3900 len 16384
14901 167/300 Read block 1920 len 4096
14902 168/300 Write block 3589 len 8192 data: 4 4
14903 169/300 Read block 643 len 12288
14904 170/300 Write block 1462 len 36864 data: 5 4 4 4 4 4 4 3 3
14905 171/300 Write block 4471 len 20480 data: 4 5 5 5 5
14906 172/300 Read block 3979 len 36864
14907 173/300 Flush
14908 174/300 Write block 540 len 16384 data: 3 4 4 3
14909 175/300 Read block 1900 len 28672
14910 176/300 Read block 94 len 8192
14911 177/300 Flush
14912 178/300 Write block 686 len 36864 data: 7 7 7 4 5 4 4 4 4
14913 179/300 Write block 3990 len 32768 data: 4 6 6 6 5 6 7 6
14914 180/300 Write block 4038 len 24576 data: 2 3 3 3 3 3
14915 181/300 Write block 1582 len 16384 data: 8 8 10 8
14916 182/300 Write block 612 len 16384 data: 4 3 3 3
14917 183/300 Write block 301 len 28672 data: 4 4 3 3 3 3 3
14918 184/300 Read block 3815 len 32768
14919 185/300 Read block 554 len 40960
14920 186/300 Flush
14921 187/300 Flush
14922 188/300 Write block 3418 len 20480 data: 3 4 2 2 2
14923 189/300 Write block 1695 len 16384 data: 4 4 4 4
14924 190/300 Write block 658 len 32768 data: 3 3 3 3 2 3 3 5
14925 191/300 Write block 575 len 28672 data: 3 2 2 2 2 3 3
14926 192/300 Write block 2057 len 16384 data: 3 3 3 5
14927 193/300 Write block 1467 len 24576 data: 5 5 4 4 3 4
14928 194/300 Write block 1894 len 20480 data: 2 2 2 2 2
14929 195/300 Write block 4003 len 16384 data: 4 5 5 7
14930 196/300 Read block 3096 len 4096
14931 197/300 Read block 4276 len 4096
14932 198/300 Read block 1435 len 40960
14933 199/300 Write block 3648 len 24576 data: 4 3 3 3 2 2
14934 200/300 Read block 2635 len 36864
14935 201/300 Read block 3669 len 20480
14936 202/300 Write block 4687 len 36864 data: 3 4 4 4 4 4 5 5 3
14937 203/300 Write block 4045 len 12288 data: 3 3 3
14938 204/300 Read block 3003 len 16384
14939 205/300 Read block 736 len 16384
14940 206/300 Read block 4795 len 16384
14941 207/300 Write block 2838 len 8192 data: 2 2
14942 208/300 Write block 3421 len 28672 data: 3 3 3 3 2 2 2
14943 209/300 Write block 2256 len 12288 data: 3 3 4
14944 210/300 Read block 3651 len 36864
14945 211/300 Read block 2013 len 32768
14946 212/300 Write block 3617 len 36864 data: 6 6 5 5 5 4 4 3 3
14947 213/300 Read block 1201 len 4096
14948 214/300 Read block 4386 len 32768
14949 215/300 Read block 3207 len 28672
14950 216/300 Flush
14951 217/300 Read block 4608 len 4096
14952 218/300 Read block 1628 len 36864
14953 219/300 Read block 4461 len 36864
14954 220/300 Write block 629 len 16384 data: 2 3 3 3
14955 221/300 Flush
14956 222/300 Write block 4392 len 8192 data: 2 3
14957 223/300 Read block 813 len 8192
14958 224/300 Read block 2554 len 12288
14959 225/300 Write block 4414 len 24576 data: 4 4 7 7 7 4
14960 226/300 Write block 280 len 40960 data: 4 4 4 4 3 3 3 3 3 3
14961 227/300 Read block 2935 len 20480
14962 228/300 Write block 1569 len 28672 data: 4 3 3 3 3 4 4
14963 229/300 Read block 1064 len 32768
14964 230/300 Write block 4845 len 24576 data: 6 6 4 5 5 4
14965 231/300 Read block 2064 len 16384
14966 232/300 Write block 1824 len 12288 data: 4 5 5
14967 233/300 Read block 287 len 40960
14968 234/300 Read block 1767 len 20480
14969 235/300 Read block 1000 len 24576
14970 236/300 Write block 1787 len 4096 data: 4
14971 237/300 Read block 2304 len 20480
14972 238/300 Flush
14973 239/300 Flush
14974 240/300 Read block 418 len 4096
14975 241/300 Read block 1412 len 32768
14976 242/300 Write block 4073 len 8192 data: 2 2
14977 243/300 Read block 970 len 20480
14978 244/300 Read block 1294 len 40960
14979 245/300 Read block 3437 len 36864
14980 246/300 Read block 1768 len 20480
14981 247/300 Write block 504 len 32768 data: 4 5 3 4 4 4 5 5
14982 248/300 Read block 340 len 20480
14983 249/300 Flush
14984 250/300 Read block 3640 len 4096
14985 251/300 Read block 2203 len 12288
14986 252/300 Flush
14987 253/300 Read block 561 len 32768
14988 254/300 Read block 4275 len 4096
14989 255/300 Read block 2608 len 20480
14990 256/300 Write block 964 len 8192 data: 4 4
14991 257/300 Read block 4573 len 36864
14992 258/300 Write block 1931 len 16384 data: 2 2 2 2
14993 259/300 Write block 11 len 28672 data: 2 3 3 3 3 3 3
14994 260/300 Read block 3200 len 32768
14995 261/300 Read block 982 len 8192
14996 262/300 Write block 4175 len 36864 data: 7 5 7 7 5 3 3 3 3
14997 263/300 Read block 4345 len 40960
14998 264/300 Write block 2140 len 40960 data: 2 2 3 3 3 3 3 3 4 4
14999 265/300 Write block 2488 len 20480 data: 6 5 6 7 6
15000 266/300 Flush
15001 267/300 Write block 3722 len 28672 data: 3 2 3 3 4 4 2
15002 268/300 Read block 2 len 20480
15003 269/300 Flush
15004 270/300 Write block 3518 len 8192 data: 3 3
15005 271/300 Write block 955 len 12288 data: 6 6 6
15006 272/300 Read block 2252 len 36864
15007 273/300 Read block 1728 len 36864
15008 274/300 Write block 4956 len 28672 data: 3 3 3 3 3 3 3
15009 275/300 Write block 1404 len 4096 data: 2
15010 276/300 Write block 4759 len 32768 data: 6 6 5 7 6 7 6 6
15011 277/300 Write block 3267 len 16384 data: 2 2 2 3
15012 278/300 Read block 570 len 8192
15013 279/300 Flush
15014 280/300 Read block 4530 len 20480
15015 281/300 Read block 3972 len 40960
15016 282/300 Read block 2649 len 8192
15017 283/300 Read block 2124 len 16384
15018 284/300 Write block 2900 len 20480 data: 4 4 4 4 5
15019 285/300 Read block 4840 len 16384
15020 286/300 Read block 2509 len 8192
15021 287/300 Read block 3224 len 12288
15022 288/300 Flush
15023 289/300 Read block 4011 len 28672
15024 290/300 Write block 4076 len 20480 data: 2 3 3 3 3
15025 291/300 Read block 555 len 36864
15026 292/300 Read block 103 len 36864
15027 293/300 Read block 281 len 8192
15028 294/300 Write block 861 len 8192 data: 3 4
15029 295/300 Write block 4028 len 8192 data: 4 4
15030 296/300 Read block 512 len 40960
15031 297/300 Flush
15032 298/300 Write block 1181 len 28672 data: 3 3 3 5 5 4 3
15033 299/300 Read block 193 len 32768
15034 300/300 Write block 4889 len 40960 data: 3 3 3 3 3 2 2 2 2 3
15035 301/300 Read block 4026 len 16384
15036 Main task has work to do, go find it
15037 got dsc Start(2)
15038 start 2
15039 Jan 17 21:14:15.865 INFO request completed, latency_us: 204, response_code: 204, uri: /start/cid/2, method: POST, req_id: 5952391e-bc71-436b-a068-e7ec1dea6985, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
15040 Replay: started 2, returned:Ok(())
15041 ----------------------------------------------------------------
15042 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
15043 States: Active Active Offline
15044 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15045 4924 Acked Flush 0 Done Done Sent false
15046 4925 Acked Write 8 Done Done Sent false
15047 4926 Acked Read 3 Done Done Sent false
15048 4927 Acked Read 9 Done Done Sent false
15049 4928 Acked Write 4 Done Done Sent false
15050 4929 Acked Flush 0 Done Done Sent false
15051 4930 Acked Write 7 Done Done Sent false
15052 4931 Acked Write 1 Done Done Sent false
15053 4932 Acked Write 4 Done Done Sent false
15054 4933 Acked Write 3 Done Done Sent false
15055 4934 Acked Read 3 Done Done Sent false
15056 4935 Acked Flush 0 Done Done Sent false
15057 4936 Acked Write 9 Done Done Sent false
15058 4937 Acked Flush 0 Done Done Sent false
15059 4938 Acked Flush 0 Done Done Sent false
15060 4939 Acked Read 9 Done Done Sent false
15061 4940 Acked Flush 0 Done Done Sent false
15062 4941 Acked Read 9 Done Done Sent false
15063 4942 Acked Read 7 Done Done Sent false
15064 4943 Acked Read 7 Done Done Sent false
15065 4944 Acked Write 7 Done Done Sent false
15066 4945 Acked Write 7 Done Done Sent false
15067 4946 Acked Write 5 Done Done Sent false
15068 4947 Acked Write 1 Done Done Sent false
15069 4948 Acked Write 8 Done Done Sent false
15070 4949 Acked Write 6 Done Done Sent false
15071 4950 Acked Write 6 Done Done Sent false
15072 4951 Acked Read 7 Done Done Sent false
15073 4952 Acked Flush 0 Done Done Sent false
15074 4953 Acked Read 8 Done Done Sent false
15075 4954 Acked Write 9 Done Done Sent false
15076 4955 Acked Read 7 Done Done Sent false
15077 4956 Acked Write 9 Done Done Sent false
15078 4957 Acked Read 5 Done Done Sent false
15079 4958 Acked Flush 0 Done Done Sent false
15080 4959 Acked Read 6 Done Done Sent false
15081 4960 Acked Flush 0 Done Done Sent false
15082 4961 Acked Flush 0 Done Done Sent false
15083 4962 Acked Read 3 Done Done Sent false
15084 4963 Acked Write 10 Done Done Sent false
15085 4964 Acked Write 9 Done Done Sent false
15086 4965 Acked Read 5 Done Done Sent false
15087 4966 Acked Read 8 Done Done Sent false
15088 4967 Acked Read 5 Done Done Sent false
15089 4968 Acked Read 6 Done Done Sent false
15090 4969 Acked Write 8 Done Done Sent false
15091 4970 Acked Read 1 Done Done Sent false
15092 4971 Acked Read 9 Done Done Sent false
15093 4972 Acked Read 9 Done Done Sent false
15094 4973 Acked Flush 0 Done Done Sent false
15095 4974 Acked Read 9 Done Done Sent false
15096 4975 Acked Read 4 Done Done Sent false
15097 4976 Acked Write 6 Done Done Sent false
15098 4977 Acked Write 6 Done Done Sent false
15099 4978 Acked Read 8 Done Done Sent false
15100 4979 Acked Read 10 Done Done Sent false
15101 4980 Acked Read 6 Done Done Sent false
15102 4981 Acked Read 8 Done Done Sent false
15103 4982 Acked Read 8 Done Done Sent false
15104 4983 Acked Read 9 Done Done Sent false
15105 4984 Acked Flush 0 Done Done Sent false
15106 4985 Acked Flush 0 Done Done Sent false
15107 4986 Acked Write 4 Done Done Sent false
15108 4987 Acked Read 5 Done Done Sent false
15109 4988 Acked Read 4 Done Done Sent false
15110 4989 Acked Flush 0 Done Done Sent false
15111 4990 Acked Write 2 Done Done Sent false
15112 4991 Acked Read 4 Done Done Sent false
15113 4992 Acked Read 8 Done Done Sent false
15114 4993 Acked Write 3 Done Done Sent false
15115 4994 Acked Write 2 Done Done Sent false
15116 4995 Acked Read 2 Done Done Sent false
15117 4996 Acked Read 7 Done Done Sent false
15118 4997 Acked Write 9 Done Done Sent false
15119 4998 Acked Write 10 Done Done Sent false
15120 4999 Acked Read 7 Done Done Sent false
15121 5000 Acked Read 7 Done Done Sent false
15122 5001 Acked Read 10 Done Done Sent false
15123 5002 Acked Write 6 Done Done Sent false
15124 5003 Acked Write 9 Done Done Sent false
15125 5004 Acked Read 9 Done Done Sent false
15126 5005 Acked Write 10 Done Done Sent false
15127 5006 Acked Read 7 Done Done Sent false
15128 5007 Acked Read 9 Done Done Sent false
15129 5008 Acked Flush 0 Done Done Sent false
15130 5009 Acked Write 4 Done Done Sent false
15131 5010 Acked Write 10 Done Done Sent false
15132 5011 Acked Write 5 Done Done Sent false
15133 5012 Acked Flush 0 Done Done Sent false
15134 5013 Acked Write 3 Done Done Sent false
15135 5014 Acked Read 1 Done Done Sent false
15136 5015 Acked Write 10 Done Done Sent false
15137 5016 Acked Write 3 Done Done Sent false
15138 5017 Acked Write 8 Done Done Sent false
15139 5018 Acked Write 9 Done Done Sent false
15140 5019 Acked Flush 0 Done Done Sent false
15141 5020 Acked Write 6 Done Done Sent false
15142 5021 Acked Flush 0 Done Done Sent false
15143 5022 Acked Read 2 Done Done Sent false
15144 5023 Acked Flush 0 Done Done Sent false
15145 5024 Acked Read 9 Done Done Sent false
15146 5025 Acked Read 4 Done Done Sent false
15147 5026 Acked Read 5 Done Done Sent false
15148 5027 Acked Read 8 Done Done Sent false
15149 5028 Acked Read 2 Done Done Sent false
15150 5029 Acked Flush 0 Done Done Sent false
15151 5030 Acked Read 2 Done Done Sent false
15152 5031 Acked Read 2 Done Done Sent false
15153 5032 Acked Read 8 Done Done Sent false
15154 5033 Acked Write 3 Done Done Sent false
15155 5034 Acked Read 8 Done Done Sent false
15156 5035 Acked Write 1 Done Done Sent false
15157 5036 Acked Read 2 Done Done Sent false
15158 5037 Acked Flush 0 Done Done Sent false
15159 5038 Acked Read 1 Done Done Sent false
15160 5039 Acked Read 10 Done Done Sent false
15161 5040 Acked Write 10 Done Done Sent false
15162 5041 Acked Write 1 Done Done Sent false
15163 5042 Acked Read 8 Done Done Sent false
15164 5043 Acked Flush 0 Done Done Sent false
15165 5044 Acked Write 5 Done Done Sent false
15166 5045 Acked Read 2 Done Done Sent false
15167 5046 Acked Read 7 Done Done Sent false
15168 5047 Acked Write 9 Done Done Sent false
15169 5048 Acked Write 5 Done Done Sent false
15170 5049 Acked Write 5 Done Done Sent false
15171 5050 Acked Read 6 Done Done Sent false
15172 5051 Acked Read 1 Done Done Sent false
15173 5052 Acked Write 8 Done Done Sent false
15174 5053 Acked Write 1 Done Done Sent false
15175 5054 Acked Write 10 Done Done Sent false
15176 5055 Acked Write 6 Done Done Sent false
15177 5056 Acked Read 4 Done Done Sent false
15178 5057 Acked Read 8 Done Done Sent false
15179 5058 Acked Flush 0 Done Done Sent false
15180 5059 Acked Write 6 Done Done Sent false
15181 5060 Acked Read 4 Done Done Sent false
15182 5061 Acked Read 3 Done Done Sent false
15183 5062 Acked Write 4 Done Done Sent false
15184 5063 Acked Write 5 Done Done Sent false
15185 5064 Acked Read 2 Done Done Sent false
15186 5065 Acked Flush 0 Done Done Sent false
15187 5066 Acked Read 2 Done Done Sent false
15188 5067 Acked Read 4 Done Done Sent false
15189 5068 Acked Write 4 Done Done Sent false
15190 5069 Acked Read 3 Done Done Sent false
15191 5070 Acked Read 1 Done Done Sent false
15192 5071 Acked Flush 0 Done Done Sent false
15193 5072 Acked Read 4 Done Done Sent false
15194 5073 Acked Write 4 Done Done Sent false
15195 5074 Acked Read 9 Done Done Sent false
15196 5075 Acked Read 5 Done Done Sent false
15197 5076 Acked Flush 0 Done Done Sent false
15198 5077 Acked Read 6 Done Done Sent false
15199 5078 Acked Read 3 Done Done Sent false
15200 5079 Acked Read 9 Done Done Sent false
15201 5080 Acked Read 8 Done Done Sent false
15202 5081 Acked Read 2 Done Done Sent false
15203 5082 Acked Read 6 Done Done Sent false
15204 5083 Acked Write 9 Done Done Sent false
15205 5084 Acked Read 7 Done Done Sent false
15206 5085 Acked Read 1 Done Done Sent false
15207 5086 Acked Write 3 Done Done Sent false
15208 5087 Acked Write 6 Done Done Sent false
15209 5088 Acked Write 5 Done Done Sent false
15210 5089 Acked Read 4 Done Done Sent false
15211 5090 Acked Read 1 Done Done Sent false
15212 5091 Acked Write 2 Done Done Sent false
15213 5092 Acked Read 3 Done Done Sent false
15214 5093 Acked Write 9 Done Done Sent false
15215 5094 Acked Write 5 Done Done Sent false
15216 5095 Acked Read 9 Done Done Sent false
15217 5096 Acked Flush 0 Done Done Sent false
15218 5097 Acked Write 4 Done Done Sent false
15219 5098 Acked Read 7 Done Done Sent false
15220 5099 Acked Read 2 Done Done Sent false
15221 5100 Acked Flush 0 Done Done Sent false
15222 5101 Acked Write 9 Done Done Sent false
15223 5102 Acked Write 8 Done Done Sent false
15224 5103 Acked Write 6 Done Done Sent false
15225 5104 Acked Write 4 Done Done Sent false
15226 5105 Acked Write 4 Done Done Sent false
15227 5106 Acked Write 7 Done Done Sent false
15228 5107 Acked Read 8 Done Done Sent false
15229 5108 Acked Read 10 Done Done Sent false
15230 5109 Acked Flush 0 Done Done Sent false
15231 5110 Acked Flush 0 Done Done Sent false
15232 5111 Acked Write 5 Done Done Sent false
15233 5112 Acked Write 4 Done Done Sent false
15234 5113 Acked Write 8 Done Done Sent false
15235 5114 Acked Write 7 Done Done Sent false
15236 5115 Acked Write 4 Done Done Sent false
15237 5116 Acked Write 6 Done Done Sent false
15238 5117 Acked Write 5 Done Done Sent false
15239 5118 Acked Write 4 Done Done Sent false
15240 5119 Acked Read 1 Done Done Sent false
15241 5120 Acked Read 1 Done Done Sent false
15242 5121 Acked Read 10 Done Done Sent false
15243 5122 Acked Write 6 Done Done Sent false
15244 5123 Acked Read 9 Done Done Sent false
15245 5124 Acked Read 5 Done Done Sent false
15246 5125 Acked Write 9 Done Done Sent false
15247 5126 Acked Write 3 Done Done Sent false
15248 5127 Acked Read 4 Done Done Sent false
15249 5128 Acked Read 4 Done Done Sent false
15250 5129 Acked Read 4 Done Done Sent false
15251 5130 Acked Write 2 Done Done Sent false
15252 5131 Acked Write 7 Done Done Sent false
15253 5132 Acked Write 3 Done Done Sent false
15254 5133 Acked Read 9 Done Done Sent false
15255 5134 Acked Read 8 Done Done Sent false
15256 5135 Acked Write 9 Done Done Sent false
15257 5136 Acked Read 1 Done Done Sent false
15258 5137 Acked Read 8 Done Done Sent false
15259 5138 Acked Read 7 Done Done Sent false
15260 5139 Acked Flush 0 Done Done Sent false
15261 5140 Acked Read 1 Done Done Sent false
15262 5141 Acked Read 9 Done Done Sent false
15263 5142 Acked Read 9 Done Done Sent false
15264 5143 Acked Write 4 Done Done Sent false
15265 5144 Acked Flush 0 Done Done Sent false
15266 5145 Acked Write 2 Done Done Sent false
15267 5146 Acked Read 2 Done Done Sent false
15268 5147 Acked Read 3 Done Done Sent false
15269 5148 Acked Write 6 Done Done Sent false
15270 5149 Acked Write 10 Done Done Sent false
15271 5150 Acked Read 5 Done Done Sent false
15272 5151 Acked Write 7 Done Done Sent false
15273 5152 Acked Read 8 Done Done Sent false
15274 5153 Acked Write 6 Done Done Sent false
15275 5154 Acked Read 4 Done Done Sent false
15276 5155 Acked Write 3 Done Done Sent false
15277 5156 Acked Read 10 Done Done Sent false
15278 5157 Acked Read 5 Done Done Sent false
15279 5158 Acked Read 6 Done Done Sent false
15280 5159 Acked Write 1 Done Done Sent false
15281 5160 Acked Read 5 Done Done Sent false
15282 5161 Acked Flush 0 Done Done Sent false
15283 5162 Acked Flush 0 Done Done Sent false
15284 5163 Acked Read 1 Done Done Sent false
15285 5164 Acked Read 8 Done Done Sent false
15286 5165 Acked Write 2 Done Done Sent false
15287 5166 Acked Read 5 Done Done Sent false
15288 5167 Acked Read 10 Done Done Sent false
15289 5168 Acked Read 9 Done Done Sent false
15290 5169 Acked Read 5 Done Done Sent false
15291 5170 Acked Write 8 Done Done Sent false
15292 5171 Acked Read 5 Done Done Sent false
15293 5172 Acked Flush 0 Done Done Sent false
15294 5173 Acked Read 1 Done Done Sent false
15295 5174 Acked Read 3 Done Done Sent false
15296 5175 Acked Flush 0 Done Done Sent false
15297 5176 Acked Read 8 Done Done Sent false
15298 5177 Acked Read 1 Done Done Sent false
15299 5178 Acked Read 5 Done Done Sent false
15300 5179 Acked Write 2 Done Done Sent false
15301 5180 Acked Read 9 Done Done Sent false
15302 5181 Acked Write 4 Done Done Sent false
15303 5182 Acked Write 7 Done Done Sent false
15304 5183 Acked Read 8 Done Done Sent false
15305 5184 Acked Read 2 Done Done Sent false
15306 5185 Acked Write 9 Done Done Sent false
15307 5186 Acked Read 10 Done Done Sent false
15308 5187 Acked Write 10 Done Done Sent false
15309 5188 Acked Write 5 Done Done Sent false
15310 5189 Acked Flush 0 Done Done Sent false
15311 5190 Acked Write 7 Done Done Sent false
15312 5191 Acked Read 5 Done Done Sent false
15313 5192 Acked Flush 0 Done Done Sent false
15314 5193 Acked Write 2 Done Done Sent false
15315 5194 Acked Write 3 Done Done Sent false
15316 5195 Acked Read 9 Done Done Sent false
15317 5196 Acked Read 9 Done Done Sent false
15318 5197 Acked Write 7 Done Done Sent false
15319 5198 Acked Write 1 Done Done Sent false
15320 5199 Acked Write 8 Done Done Sent false
15321 5200 Acked Write 4 Done Done Sent false
15322 5201 Acked Read 2 Done Done Sent false
15323 5202 Acked Flush 0 Done Done Sent false
15324 5203 Acked Read 5 Done Done Sent false
15325 5204 Acked Read 10 Done Done Sent false
15326 5205 Acked Read 2 Done Done Sent false
15327 5206 Acked Read 4 Done Done Sent false
15328 5207 Acked Write 5 Done Done Sent false
15329 5208 Acked Read 4 Done Done Sent false
15330 5209 Acked Read 2 Done Done Sent false
15331 5210 Acked Read 3 Done Done Sent false
15332 5211 Acked Flush 0 Done Done Sent false
15333 5212 Acked Read 7 Done Done Sent false
15334 5213 Acked Write 5 Done Done Sent false
15335 5214 Acked Read 9 Done Done Sent false
15336 5215 Acked Read 9 Done Done Sent false
15337 5216 Acked Read 2 Done Done Sent false
15338 5217 Acked Write 2 Done Done Sent false
15339 5218 Acked Write 2 Done Done Sent false
15340 5219 Acked Read 10 Done Done Sent false
15341 5220 Acked Flush 0 Done Done Sent false
15342 5221 Acked Write 7 Done Done Sent false
15343 5222 Acked Read 8 Done Done Sent false
15344 5223 Acked Write 10 Done Done Sent false
15345 5224 Acked Read 4 Done Done Sent false
15346 STATES DS:0 DS:1 DS:2 TOTAL
15347 Sent 0 0 301 301
15348 Done 301 301 0 602
15349 Skipped 0 0 0 0
15350 Error 0 0 0 0
15351 Last Flush: 5220 5220 4923
15352 Downstairs last five completed: 4923 4922 4921 4920 4919
15353 Upstairs last five completed: 5224 5223 5222 5221 5220
15354 CLIENT: Up:0 ds:301 act:2
15355 ----------------------------------------------------------------
15356 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15357 States: Active Active Offline
15358 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15359 4924 Acked Flush 0 Done Done Sent false
15360 4925 Acked Write 8 Done Done Sent false
15361 4926 Acked Read 3 Done Done Sent false
15362 4927 Acked Read 9 Done Done Sent false
15363 4928 Acked Write 4 Done Done Sent false
15364 4929 Acked Flush 0 Done Done Sent false
15365 4930 Acked Write 7 Done Done Sent false
15366 4931 Acked Write 1 Done Done Sent false
15367 4932 Acked Write 4 Done Done Sent false
15368 4933 Acked Write 3 Done Done Sent false
15369 4934 Acked Read 3 Done Done Sent false
15370 4935 Acked Flush 0 Done Done Sent false
15371 4936 Acked Write 9 Done Done Sent false
15372 4937 Acked Flush 0 Done Done Sent false
15373 4938 Acked Flush 0 Done Done Sent false
15374 4939 Acked Read 9 Done Done Sent false
15375 4940 Acked Flush 0 Done Done Sent false
15376 4941 Acked Read 9 Done Done Sent false
15377 4942 Acked Read 7 Done Done Sent false
15378 4943 Acked Read 7 Done Done Sent false
15379 4944 Acked Write 7 Done Done Sent false
15380 4945 Acked Write 7 Done Done Sent false
15381 4946 Acked Write 5 Done Done Sent false
15382 4947 Acked Write 1 Done Done Sent false
15383 4948 Acked Write 8 Done Done Sent false
15384 4949 Acked Write 6 Done Done Sent false
15385 4950 Acked Write 6 Done Done Sent false
15386 4951 Acked Read 7 Done Done Sent false
15387 4952 Acked Flush 0 Done Done Sent false
15388 4953 Acked Read 8 Done Done Sent false
15389 4954 Acked Write 9 Done Done Sent false
15390 4955 Acked Read 7 Done Done Sent false
15391 4956 Acked Write 9 Done Done Sent false
15392 4957 Acked Read 5 Done Done Sent false
15393 4958 Acked Flush 0 Done Done Sent false
15394 4959 Acked Read 6 Done Done Sent false
15395 4960 Acked Flush 0 Done Done Sent false
15396 4961 Acked Flush 0 Done Done Sent false
15397 4962 Acked Read 3 Done Done Sent false
15398 4963 Acked Write 10 Done Done Sent false
15399 4964 Acked Write 9 Done Done Sent false
15400 4965 Acked Read 5 Done Done Sent false
15401 4966 Acked Read 8 Done Done Sent false
15402 4967 Acked Read 5 Done Done Sent false
15403 4968 Acked Read 6 Done Done Sent false
15404 4969 Acked Write 8 Done Done Sent false
15405 4970 Acked Read 1 Done Done Sent false
15406 4971 Acked Read 9 Done Done Sent false
15407 4972 Acked Read 9 Done Done Sent false
15408 4973 Acked Flush 0 Done Done Sent false
15409 4974 Acked Read 9 Done Done Sent false
15410 4975 Acked Read 4 Done Done Sent false
15411 4976 Acked Write 6 Done Done Sent false
15412 4977 Acked Write 6 Done Done Sent false
15413 4978 Acked Read 8 Done Done Sent false
15414 4979 Acked Read 10 Done Done Sent false
15415 4980 Acked Read 6 Done Done Sent false
15416 4981 Acked Read 8 Done Done Sent false
15417 4982 Acked Read 8 Done Done Sent false
15418 4983 Acked Read 9 Done Done Sent false
15419 4984 Acked Flush 0 Done Done Sent false
15420 4985 Acked Flush 0 Done Done Sent false
15421 4986 Acked Write 4 Done Done Sent false
15422 4987 Acked Read 5 Done Done Sent false
15423 4988 Acked Read 4 Done Done Sent false
15424 4989 Acked Flush 0 Done Done Sent false
15425 4990 Acked Write 2 Done Done Sent false
15426 4991 Acked Read 4 Done Done Sent false
15427 4992 Acked Read 8 Done Done Sent false
15428 4993 Acked Write 3 Done Done Sent false
15429 4994 Acked Write 2 Done Done Sent false
15430 4995 Acked Read 2 Done Done Sent false
15431 4996 Acked Read 7 Done Done Sent false
15432 4997 Acked Write 9 Done Done Sent false
15433 4998 Acked Write 10 Done Done Sent false
15434 4999 Acked Read 7 Done Done Sent false
15435 5000 Acked Read 7 Done Done Sent false
15436 5001 Acked Read 10 Done Done Sent false
15437 5002 Acked Write 6 Done Done Sent false
15438 5003 Acked Write 9 Done Done Sent false
15439 5004 Acked Read 9 Done Done Sent false
15440 5005 Acked Write 10 Done Done Sent false
15441 5006 Acked Read 7 Done Done Sent false
15442 5007 Acked Read 9 Done Done Sent false
15443 5008 Acked Flush 0 Done Done Sent false
15444 5009 Acked Write 4 Done Done Sent false
15445 5010 Acked Write 10 Done Done Sent false
15446 5011 Acked Write 5 Done Done Sent false
15447 5012 Acked Flush 0 Done Done Sent false
15448 5013 Acked Write 3 Done Done Sent false
15449 5014 Acked Read 1 Done Done Sent false
15450 5015 Acked Write 10 Done Done Sent false
15451 5016 Acked Write 3 Done Done Sent false
15452 5017 Acked Write 8 Done Done Sent false
15453 5018 Acked Write 9 Done Done Sent false
15454 5019 Acked Flush 0 Done Done Sent false
15455 5020 Acked Write 6 Done Done Sent false
15456 5021 Acked Flush 0 Done Done Sent false
15457 5022 Acked Read 2 Done Done Sent false
15458 5023 Acked Flush 0 Done Done Sent false
15459 5024 Acked Read 9 Done Done Sent false
15460 5025 Acked Read 4 Done Done Sent false
15461 5026 Acked Read 5 Done Done Sent false
15462 5027 Acked Read 8 Done Done Sent false
15463 5028 Acked Read 2 Done Done Sent false
15464 5029 Acked Flush 0 Done Done Sent false
15465 5030 Acked Read 2 Done Done Sent false
15466 5031 Acked Read 2 Done Done Sent false
15467 5032 Acked Read 8 Done Done Sent false
15468 5033 Acked Write 3 Done Done Sent false
15469 5034 Acked Read 8 Done Done Sent false
15470 5035 Acked Write 1 Done Done Sent false
15471 5036 Acked Read 2 Done Done Sent false
15472 5037 Acked Flush 0 Done Done Sent false
15473 5038 Acked Read 1 Done Done Sent false
15474 5039 Acked Read 10 Done Done Sent false
15475 5040 Acked Write 10 Done Done Sent false
15476 5041 Acked Write 1 Done Done Sent false
15477 5042 Acked Read 8 Done Done Sent false
15478 5043 Acked Flush 0 Done Done Sent false
15479 5044 Acked Write 5 Done Done Sent false
15480 5045 Acked Read 2 Done Done Sent false
15481 5046 Acked Read 7 Done Done Sent false
15482 5047 Acked Write 9 Done Done Sent false
15483 5048 Acked Write 5 Done Done Sent false
15484 5049 Acked Write 5 Done Done Sent false
15485 5050 Acked Read 6 Done Done Sent false
15486 5051 Acked Read 1 Done Done Sent false
15487 5052 Acked Write 8 Done Done Sent false
15488 5053 Acked Write 1 Done Done Sent false
15489 5054 Acked Write 10 Done Done Sent false
15490 5055 Acked Write 6 Done Done Sent false
15491 5056 Acked Read 4 Done Done Sent false
15492 5057 Acked Read 8 Done Done Sent false
15493 5058 Acked Flush 0 Done Done Sent false
15494 5059 Acked Write 6 Done Done Sent false
15495 5060 Acked Read 4 Done Done Sent false
15496 5061 Acked Read 3 Done Done Sent false
15497 5062 Acked Write 4 Done Done Sent false
15498 5063 Acked Write 5 Done Done Sent false
15499 5064 Acked Read 2 Done Done Sent false
15500 5065 Acked Flush 0 Done Done Sent false
15501 5066 Acked Read 2 Done Done Sent false
15502 5067 Acked Read 4 Done Done Sent false
15503 5068 Acked Write 4 Done Done Sent false
15504 5069 Acked Read 3 Done Done Sent false
15505 5070 Acked Read 1 Done Done Sent false
15506 5071 Acked Flush 0 Done Done Sent false
15507 5072 Acked Read 4 Done Done Sent false
15508 5073 Acked Write 4 Done Done Sent false
15509 5074 Acked Read 9 Done Done Sent false
15510 5075 Acked Read 5 Done Done Sent false
15511 5076 Acked Flush 0 Done Done Sent false
15512 5077 Acked Read 6 Done Done Sent false
15513 5078 Acked Read 3 Done Done Sent false
15514 5079 Acked Read 9 Done Done Sent false
15515 5080 Acked Read 8 Done Done Sent false
15516 5081 Acked Read 2 Done Done Sent false
15517 5082 Acked Read 6 Done Done Sent false
15518 5083 Acked Write 9 Done Done Sent false
15519 5084 Acked Read 7 Done Done Sent false
15520 5085 Acked Read 1 Done Done Sent false
15521 5086 Acked Write 3 Done Done Sent false
15522 5087 Acked Write 6 Done Done Sent false
15523 5088 Acked Write 5 Done Done Sent false
15524 5089 Acked Read 4 Done Done Sent false
15525 5090 Acked Read 1 Done Done Sent false
15526 5091 Acked Write 2 Done Done Sent false
15527 5092 Acked Read 3 Done Done Sent false
15528 5093 Acked Write 9 Done Done Sent false
15529 5094 Acked Write 5 Done Done Sent false
15530 5095 Acked Read 9 Done Done Sent false
15531 5096 Acked Flush 0 Done Done Sent false
15532 5097 Acked Write 4 Done Done Sent false
15533 5098 Acked Read 7 Done Done Sent false
15534 5099 Acked Read 2 Done Done Sent false
15535 5100 Acked Flush 0 Done Done Sent false
15536 5101 Acked Write 9 Done Done Sent false
15537 5102 Acked Write 8 Done Done Sent false
15538 5103 Acked Write 6 Done Done Sent false
15539 5104 Acked Write 4 Done Done Sent false
15540 5105 Acked Write 4 Done Done Sent false
15541 5106 Acked Write 7 Done Done Sent false
15542 5107 Acked Read 8 Done Done Sent false
15543 5108 Acked Read 10 Done Done Sent false
15544 5109 Acked Flush 0 Done Done Sent false
15545 5110 Acked Flush 0 Done Done Sent false
15546 5111 Acked Write 5 Done Done Sent false
15547 5112 Acked Write 4 Done Done Sent false
15548 5113 Acked Write 8 Done Done Sent false
15549 5114 Acked Write 7 Done Done Sent false
15550 5115 Acked Write 4 Done Done Sent false
15551 5116 Acked Write 6 Done Done Sent false
15552 5117 Acked Write 5 Done Done Sent false
15553 5118 Acked Write 4 Done Done Sent false
15554 5119 Acked Read 1 Done Done Sent false
15555 5120 Acked Read 1 Done Done Sent false
15556 5121 Acked Read 10 Done Done Sent false
15557 5122 Acked Write 6 Done Done Sent false
15558 5123 Acked Read 9 Done Done Sent false
15559 5124 Acked Read 5 Done Done Sent false
15560 5125 Acked Write 9 Done Done Sent false
15561 5126 Acked Write 3 Done Done Sent false
15562 5127 Acked Read 4 Done Done Sent false
15563 5128 Acked Read 4 Done Done Sent false
15564 5129 Acked Read 4 Done Done Sent false
15565 5130 Acked Write 2 Done Done Sent false
15566 5131 Acked Write 7 Done Done Sent false
15567 5132 Acked Write 3 Done Done Sent false
15568 5133 Acked Read 9 Done Done Sent false
15569 5134 Acked Read 8 Done Done Sent false
15570 5135 Acked Write 9 Done Done Sent false
15571 5136 Acked Read 1 Done Done Sent false
15572 5137 Acked Read 8 Done Done Sent false
15573 5138 Acked Read 7 Done Done Sent false
15574 5139 Acked Flush 0 Done Done Sent false
15575 5140 Acked Read 1 Done Done Sent false
15576 5141 Acked Read 9 Done Done Sent false
15577 5142 Acked Read 9 Done Done Sent false
15578 5143 Acked Write 4 Done Done Sent false
15579 5144 Acked Flush 0 Done Done Sent false
15580 5145 Acked Write 2 Done Done Sent false
15581 5146 Acked Read 2 Done Done Sent false
15582 5147 Acked Read 3 Done Done Sent false
15583 5148 Acked Write 6 Done Done Sent false
15584 5149 Acked Write 10 Done Done Sent false
15585 5150 Acked Read 5 Done Done Sent false
15586 5151 Acked Write 7 Done Done Sent false
15587 5152 Acked Read 8 Done Done Sent false
15588 5153 Acked Write 6 Done Done Sent false
15589 5154 Acked Read 4 Done Done Sent false
15590 5155 Acked Write 3 Done Done Sent false
15591 5156 Acked Read 10 Done Done Sent false
15592 5157 Acked Read 5 Done Done Sent false
15593 5158 Acked Read 6 Done Done Sent false
15594 5159 Acked Write 1 Done Done Sent false
15595 5160 Acked Read 5 Done Done Sent false
15596 5161 Acked Flush 0 Done Done Sent false
15597 5162 Acked Flush 0 Done Done Sent false
15598 5163 Acked Read 1 Done Done Sent false
15599 5164 Acked Read 8 Done Done Sent false
15600 5165 Acked Write 2 Done Done Sent false
15601 5166 Acked Read 5 Done Done Sent false
15602 5167 Acked Read 10 Done Done Sent false
15603 5168 Acked Read 9 Done Done Sent false
15604 5169 Acked Read 5 Done Done Sent false
15605 5170 Acked Write 8 Done Done Sent false
15606 5171 Acked Read 5 Done Done Sent false
15607 5172 Acked Flush 0 Done Done Sent false
15608 5173 Acked Read 1 Done Done Sent false
15609 5174 Acked Read 3 Done Done Sent false
15610 5175 Acked Flush 0 Done Done Sent false
15611 5176 Acked Read 8 Done Done Sent false
15612 5177 Acked Read 1 Done Done Sent false
15613 5178 Acked Read 5 Done Done Sent false
15614 5179 Acked Write 2 Done Done Sent false
15615 5180 Acked Read 9 Done Done Sent false
15616 5181 Acked Write 4 Done Done Sent false
15617 5182 Acked Write 7 Done Done Sent false
15618 5183 Acked Read 8 Done Done Sent false
15619 5184 Acked Read 2 Done Done Sent false
15620 5185 Acked Write 9 Done Done Sent false
15621 5186 Acked Read 10 Done Done Sent false
15622 5187 Acked Write 10 Done Done Sent false
15623 5188 Acked Write 5 Done Done Sent false
15624 5189 Acked Flush 0 Done Done Sent false
15625 5190 Acked Write 7 Done Done Sent false
15626 5191 Acked Read 5 Done Done Sent false
15627 5192 Acked Flush 0 Done Done Sent false
15628 5193 Acked Write 2 Done Done Sent false
15629 5194 Acked Write 3 Done Done Sent false
15630 5195 Acked Read 9 Done Done Sent false
15631 5196 Acked Read 9 Done Done Sent false
15632 5197 Acked Write 7 Done Done Sent false
15633 5198 Acked Write 1 Done Done Sent false
15634 5199 Acked Write 8 Done Done Sent false
15635 5200 Acked Write 4 Done Done Sent false
15636 5201 Acked Read 2 Done Done Sent false
15637 5202 Acked Flush 0 Done Done Sent false
15638 5203 Acked Read 5 Done Done Sent false
15639 5204 Acked Read 10 Done Done Sent false
15640 5205 Acked Read 2 Done Done Sent false
15641 5206 Acked Read 4 Done Done Sent false
15642 5207 Acked Write 5 Done Done Sent false
15643 5208 Acked Read 4 Done Done Sent false
15644 5209 Acked Read 2 Done Done Sent false
15645 5210 Acked Read 3 Done Done Sent false
15646 5211 Acked Flush 0 Done Done Sent false
15647 5212 Acked Read 7 Done Done Sent false
15648 5213 Acked Write 5 Done Done Sent false
15649 5214 Acked Read 9 Done Done Sent false
15650 5215 Acked Read 9 Done Done Sent false
15651 5216 Acked Read 2 Done Done Sent false
15652 5217 Acked Write 2 Done Done Sent false
15653 5218 Acked Write 2 Done Done Sent false
15654 5219 Acked Read 10 Done Done Sent false
15655 5220 Acked Flush 0 Done Done Sent false
15656 5221 Acked Write 7 Done Done Sent false
15657 5222 Acked Read 8 Done Done Sent false
15658 5223 Acked Write 10 Done Done Sent false
15659 5224 Acked Read 4 Done Done Sent false
15660 5225 Acked Flush 0 Done Done Sent false
15661 STATES DS:0 DS:1 DS:2 TOTAL
15662 Sent 0 0 302 302
15663 Done 302 302 0 604
15664 Skipped 0 0 0 0
15665 Error 0 0 0 0
15666 Last Flush: 5225 5225 4923
15667 Downstairs last five completed: 4923 4922 4921 4920 4919
15668 Upstairs last five completed: 5225 5224 5223 5222 5221
15669 CLIENT: Up:0 ds:302 act:2
156702025-01-17T21:14:21.622ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
156712025-01-17T21:14:21.623ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
156722025-01-17T21:14:21.623ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
156732025-01-17T21:14:21.623ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
156742025-01-17T21:14:21.623ZINFOcrucible: client task is exiting = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
15675 [8830] got start action
15676 [8830] I am going to restart
15677 Starting downstairs at port 8830
15678 Make output file at "/tmp/dsc/downstairs-8830.txt"
15679 [8830][2] reports Starting
15680 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1214)
15681 [8830][2] reports Running
15682 ----------------------------------------------------------------
15683 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15684 States: Active Active Offline
15685 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15686 4924 Acked Flush 0 Done Done Sent false
15687 4925 Acked Write 8 Done Done Sent false
15688 4926 Acked Read 3 Done Done Sent false
15689 4927 Acked Read 9 Done Done Sent false
15690 4928 Acked Write 4 Done Done Sent false
15691 4929 Acked Flush 0 Done Done Sent false
15692 4930 Acked Write 7 Done Done Sent false
15693 4931 Acked Write 1 Done Done Sent false
15694 4932 Acked Write 4 Done Done Sent false
15695 4933 Acked Write 3 Done Done Sent false
15696 4934 Acked Read 3 Done Done Sent false
15697 4935 Acked Flush 0 Done Done Sent false
15698 4936 Acked Write 9 Done Done Sent false
15699 4937 Acked Flush 0 Done Done Sent false
15700 4938 Acked Flush 0 Done Done Sent false
15701 4939 Acked Read 9 Done Done Sent false
15702 4940 Acked Flush 0 Done Done Sent false
15703 4941 Acked Read 9 Done Done Sent false
15704 4942 Acked Read 7 Done Done Sent false
15705 4943 Acked Read 7 Done Done Sent false
15706 4944 Acked Write 7 Done Done Sent false
15707 4945 Acked Write 7 Done Done Sent false
15708 4946 Acked Write 5 Done Done Sent false
15709 4947 Acked Write 1 Done Done Sent false
15710 4948 Acked Write 8 Done Done Sent false
15711 4949 Acked Write 6 Done Done Sent false
15712 4950 Acked Write 6 Done Done Sent false
15713 4951 Acked Read 7 Done Done Sent false
15714 4952 Acked Flush 0 Done Done Sent false
15715 4953 Acked Read 8 Done Done Sent false
15716 4954 Acked Write 9 Done Done Sent false
15717 4955 Acked Read 7 Done Done Sent false
15718 4956 Acked Write 9 Done Done Sent false
15719 4957 Acked Read 5 Done Done Sent false
15720 4958 Acked Flush 0 Done Done Sent false
15721 4959 Acked Read 6 Done Done Sent false
15722 4960 Acked Flush 0 Done Done Sent false
15723 4961 Acked Flush 0 Done Done Sent false
15724 4962 Acked Read 3 Done Done Sent false
15725 4963 Acked Write 10 Done Done Sent false
15726 4964 Acked Write 9 Done Done Sent false
15727 4965 Acked Read 5 Done Done Sent false
15728 4966 Acked Read 8 Done Done Sent false
15729 4967 Acked Read 5 Done Done Sent false
15730 4968 Acked Read 6 Done Done Sent false
15731 4969 Acked Write 8 Done Done Sent false
15732 4970 Acked Read 1 Done Done Sent false
15733 4971 Acked Read 9 Done Done Sent false
15734 4972 Acked Read 9 Done Done Sent false
15735 4973 Acked Flush 0 Done Done Sent false
15736 4974 Acked Read 9 Done Done Sent false
15737 4975 Acked Read 4 Done Done Sent false
15738 4976 Acked Write 6 Done Done Sent false
15739 4977 Acked Write 6 Done Done Sent false
15740 4978 Acked Read 8 Done Done Sent false
15741 4979 Acked Read 10 Done Done Sent false
15742 4980 Acked Read 6 Done Done Sent false
15743 4981 Acked Read 8 Done Done Sent false
15744 4982 Acked Read 8 Done Done Sent false
15745 4983 Acked Read 9 Done Done Sent false
15746 4984 Acked Flush 0 Done Done Sent false
15747 4985 Acked Flush 0 Done Done Sent false
15748 4986 Acked Write 4 Done Done Sent false
15749 4987 Acked Read 5 Done Done Sent false
15750 4988 Acked Read 4 Done Done Sent false
15751 4989 Acked Flush 0 Done Done Sent false
15752 4990 Acked Write 2 Done Done Sent false
15753 4991 Acked Read 4 Done Done Sent false
15754 4992 Acked Read 8 Done Done Sent false
15755 4993 Acked Write 3 Done Done Sent false
15756 4994 Acked Write 2 Done Done Sent false
15757 4995 Acked Read 2 Done Done Sent false
15758 4996 Acked Read 7 Done Done Sent false
15759 4997 Acked Write 9 Done Done Sent false
15760 4998 Acked Write 10 Done Done Sent false
15761 4999 Acked Read 7 Done Done Sent false
15762 5000 Acked Read 7 Done Done Sent false
15763 5001 Acked Read 10 Done Done Sent false
15764 5002 Acked Write 6 Done Done Sent false
15765 5003 Acked Write 9 Done Done Sent false
15766 5004 Acked Read 9 Done Done Sent false
15767 5005 Acked Write 10 Done Done Sent false
15768 5006 Acked Read 7 Done Done Sent false
15769 5007 Acked Read 9 Done Done Sent false
15770 5008 Acked Flush 0 Done Done Sent false
15771 5009 Acked Write 4 Done Done Sent false
15772 5010 Acked Write 10 Done Done Sent false
15773 5011 Acked Write 5 Done Done Sent false
15774 5012 Acked Flush 0 Done Done Sent false
15775 5013 Acked Write 3 Done Done Sent false
15776 5014 Acked Read 1 Done Done Sent false
15777 5015 Acked Write 10 Done Done Sent false
15778 5016 Acked Write 3 Done Done Sent false
15779 5017 Acked Write 8 Done Done Sent false
15780 5018 Acked Write 9 Done Done Sent false
15781 5019 Acked Flush 0 Done Done Sent false
15782 5020 Acked Write 6 Done Done Sent false
15783 5021 Acked Flush 0 Done Done Sent false
15784 5022 Acked Read 2 Done Done Sent false
15785 5023 Acked Flush 0 Done Done Sent false
15786 5024 Acked Read 9 Done Done Sent false
15787 5025 Acked Read 4 Done Done Sent false
15788 5026 Acked Read 5 Done Done Sent false
15789 5027 Acked Read 8 Done Done Sent false
15790 5028 Acked Read 2 Done Done Sent false
15791 5029 Acked Flush 0 Done Done Sent false
15792 5030 Acked Read 2 Done Done Sent false
15793 5031 Acked Read 2 Done Done Sent false
15794 5032 Acked Read 8 Done Done Sent false
15795 5033 Acked Write 3 Done Done Sent false
15796 5034 Acked Read 8 Done Done Sent false
15797 5035 Acked Write 1 Done Done Sent false
15798 5036 Acked Read 2 Done Done Sent false
15799 5037 Acked Flush 0 Done Done Sent false
15800 5038 Acked Read 1 Done Done Sent false
15801 5039 Acked Read 10 Done Done Sent false
15802 5040 Acked Write 10 Done Done Sent false
15803 5041 Acked Write 1 Done Done Sent false
15804 5042 Acked Read 8 Done Done Sent false
15805 5043 Acked Flush 0 Done Done Sent false
15806 5044 Acked Write 5 Done Done Sent false
15807 5045 Acked Read 2 Done Done Sent false
15808 5046 Acked Read 7 Done Done Sent false
15809 5047 Acked Write 9 Done Done Sent false
15810 5048 Acked Write 5 Done Done Sent false
15811 5049 Acked Write 5 Done Done Sent false
15812 5050 Acked Read 6 Done Done Sent false
15813 5051 Acked Read 1 Done Done Sent false
15814 5052 Acked Write 8 Done Done Sent false
15815 5053 Acked Write 1 Done Done Sent false
15816 5054 Acked Write 10 Done Done Sent false
15817 5055 Acked Write 6 Done Done Sent false
15818 5056 Acked Read 4 Done Done Sent false
15819 5057 Acked Read 8 Done Done Sent false
15820 5058 Acked Flush 0 Done Done Sent false
15821 5059 Acked Write 6 Done Done Sent false
15822 5060 Acked Read 4 Done Done Sent false
15823 5061 Acked Read 3 Done Done Sent false
15824 5062 Acked Write 4 Done Done Sent false
15825 5063 Acked Write 5 Done Done Sent false
15826 5064 Acked Read 2 Done Done Sent false
15827 5065 Acked Flush 0 Done Done Sent false
15828 5066 Acked Read 2 Done Done Sent false
15829 5067 Acked Read 4 Done Done Sent false
15830 5068 Acked Write 4 Done Done Sent false
15831 5069 Acked Read 3 Done Done Sent false
15832 5070 Acked Read 1 Done Done Sent false
15833 5071 Acked Flush 0 Done Done Sent false
15834 5072 Acked Read 4 Done Done Sent false
15835 5073 Acked Write 4 Done Done Sent false
15836 5074 Acked Read 9 Done Done Sent false
15837 5075 Acked Read 5 Done Done Sent false
15838 5076 Acked Flush 0 Done Done Sent false
15839 5077 Acked Read 6 Done Done Sent false
15840 5078 Acked Read 3 Done Done Sent false
15841 5079 Acked Read 9 Done Done Sent false
15842 5080 Acked Read 8 Done Done Sent false
15843 5081 Acked Read 2 Done Done Sent false
15844 5082 Acked Read 6 Done Done Sent false
15845 5083 Acked Write 9 Done Done Sent false
15846 5084 Acked Read 7 Done Done Sent false
15847 5085 Acked Read 1 Done Done Sent false
15848 5086 Acked Write 3 Done Done Sent false
15849 5087 Acked Write 6 Done Done Sent false
15850 5088 Acked Write 5 Done Done Sent false
15851 5089 Acked Read 4 Done Done Sent false
15852 5090 Acked Read 1 Done Done Sent false
15853 5091 Acked Write 2 Done Done Sent false
15854 5092 Acked Read 3 Done Done Sent false
15855 5093 Acked Write 9 Done Done Sent false
15856 5094 Acked Write 5 Done Done Sent false
15857 5095 Acked Read 9 Done Done Sent false
15858 5096 Acked Flush 0 Done Done Sent false
15859 5097 Acked Write 4 Done Done Sent false
15860 5098 Acked Read 7 Done Done Sent false
15861 5099 Acked Read 2 Done Done Sent false
15862 5100 Acked Flush 0 Done Done Sent false
15863 5101 Acked Write 9 Done Done Sent false
15864 5102 Acked Write 8 Done Done Sent false
15865 5103 Acked Write 6 Done Done Sent false
15866 5104 Acked Write 4 Done Done Sent false
15867 5105 Acked Write 4 Done Done Sent false
15868 5106 Acked Write 7 Done Done Sent false
15869 5107 Acked Read 8 Done Done Sent false
15870 5108 Acked Read 10 Done Done Sent false
15871 5109 Acked Flush 0 Done Done Sent false
15872 5110 Acked Flush 0 Done Done Sent false
15873 5111 Acked Write 5 Done Done Sent false
15874 5112 Acked Write 4 Done Done Sent false
15875 5113 Acked Write 8 Done Done Sent false
15876 5114 Acked Write 7 Done Done Sent false
15877 5115 Acked Write 4 Done Done Sent false
15878 5116 Acked Write 6 Done Done Sent false
15879 5117 Acked Write 5 Done Done Sent false
15880 5118 Acked Write 4 Done Done Sent false
15881 5119 Acked Read 1 Done Done Sent false
15882 5120 Acked Read 1 Done Done Sent false
15883 5121 Acked Read 10 Done Done Sent false
15884 5122 Acked Write 6 Done Done Sent false
15885 5123 Acked Read 9 Done Done Sent false
15886 5124 Acked Read 5 Done Done Sent false
15887 5125 Acked Write 9 Done Done Sent false
15888 5126 Acked Write 3 Done Done Sent false
15889 5127 Acked Read 4 Done Done Sent false
15890 5128 Acked Read 4 Done Done Sent false
15891 5129 Acked Read 4 Done Done Sent false
15892 5130 Acked Write 2 Done Done Sent false
15893 5131 Acked Write 7 Done Done Sent false
15894 5132 Acked Write 3 Done Done Sent false
15895 5133 Acked Read 9 Done Done Sent false
15896 5134 Acked Read 8 Done Done Sent false
15897 5135 Acked Write 9 Done Done Sent false
15898 5136 Acked Read 1 Done Done Sent false
15899 5137 Acked Read 8 Done Done Sent false
15900 5138 Acked Read 7 Done Done Sent false
15901 5139 Acked Flush 0 Done Done Sent false
15902 5140 Acked Read 1 Done Done Sent false
15903 5141 Acked Read 9 Done Done Sent false
15904 5142 Acked Read 9 Done Done Sent false
15905 5143 Acked Write 4 Done Done Sent false
15906 5144 Acked Flush 0 Done Done Sent false
15907 5145 Acked Write 2 Done Done Sent false
15908 5146 Acked Read 2 Done Done Sent false
15909 5147 Acked Read 3 Done Done Sent false
15910 5148 Acked Write 6 Done Done Sent false
15911 5149 Acked Write 10 Done Done Sent false
15912 5150 Acked Read 5 Done Done Sent false
15913 5151 Acked Write 7 Done Done Sent false
15914 5152 Acked Read 8 Done Done Sent false
15915 5153 Acked Write 6 Done Done Sent false
15916 5154 Acked Read 4 Done Done Sent false
15917 5155 Acked Write 3 Done Done Sent false
15918 5156 Acked Read 10 Done Done Sent false
15919 5157 Acked Read 5 Done Done Sent false
15920 5158 Acked Read 6 Done Done Sent false
15921 5159 Acked Write 1 Done Done Sent false
15922 5160 Acked Read 5 Done Done Sent false
15923 5161 Acked Flush 0 Done Done Sent false
15924 5162 Acked Flush 0 Done Done Sent false
15925 5163 Acked Read 1 Done Done Sent false
15926 5164 Acked Read 8 Done Done Sent false
15927 5165 Acked Write 2 Done Done Sent false
15928 5166 Acked Read 5 Done Done Sent false
15929 5167 Acked Read 10 Done Done Sent false
15930 5168 Acked Read 9 Done Done Sent false
15931 5169 Acked Read 5 Done Done Sent false
15932 5170 Acked Write 8 Done Done Sent false
15933 5171 Acked Read 5 Done Done Sent false
15934 5172 Acked Flush 0 Done Done Sent false
15935 5173 Acked Read 1 Done Done Sent false
15936 5174 Acked Read 3 Done Done Sent false
15937 5175 Acked Flush 0 Done Done Sent false
15938 5176 Acked Read 8 Done Done Sent false
15939 5177 Acked Read 1 Done Done Sent false
15940 5178 Acked Read 5 Done Done Sent false
15941 5179 Acked Write 2 Done Done Sent false
15942 5180 Acked Read 9 Done Done Sent false
15943 5181 Acked Write 4 Done Done Sent false
15944 5182 Acked Write 7 Done Done Sent false
15945 5183 Acked Read 8 Done Done Sent false
15946 5184 Acked Read 2 Done Done Sent false
15947 5185 Acked Write 9 Done Done Sent false
15948 5186 Acked Read 10 Done Done Sent false
15949 5187 Acked Write 10 Done Done Sent false
15950 5188 Acked Write 5 Done Done Sent false
15951 5189 Acked Flush 0 Done Done Sent false
15952 5190 Acked Write 7 Done Done Sent false
15953 5191 Acked Read 5 Done Done Sent false
15954 5192 Acked Flush 0 Done Done Sent false
15955 5193 Acked Write 2 Done Done Sent false
15956 5194 Acked Write 3 Done Done Sent false
15957 5195 Acked Read 9 Done Done Sent false
15958 5196 Acked Read 9 Done Done Sent false
15959 5197 Acked Write 7 Done Done Sent false
15960 5198 Acked Write 1 Done Done Sent false
15961 5199 Acked Write 8 Done Done Sent false
15962 5200 Acked Write 4 Done Done Sent false
15963 5201 Acked Read 2 Done Done Sent false
15964 5202 Acked Flush 0 Done Done Sent false
15965 5203 Acked Read 5 Done Done Sent false
15966 5204 Acked Read 10 Done Done Sent false
15967 5205 Acked Read 2 Done Done Sent false
15968 5206 Acked Read 4 Done Done Sent false
15969 5207 Acked Write 5 Done Done Sent false
15970 5208 Acked Read 4 Done Done Sent false
15971 5209 Acked Read 2 Done Done Sent false
15972 5210 Acked Read 3 Done Done Sent false
15973 5211 Acked Flush 0 Done Done Sent false
15974 5212 Acked Read 7 Done Done Sent false
15975 5213 Acked Write 5 Done Done Sent false
15976 5214 Acked Read 9 Done Done Sent false
15977 5215 Acked Read 9 Done Done Sent false
15978 5216 Acked Read 2 Done Done Sent false
15979 5217 Acked Write 2 Done Done Sent false
15980 5218 Acked Write 2 Done Done Sent false
15981 5219 Acked Read 10 Done Done Sent false
15982 5220 Acked Flush 0 Done Done Sent false
15983 5221 Acked Write 7 Done Done Sent false
15984 5222 Acked Read 8 Done Done Sent false
15985 5223 Acked Write 10 Done Done Sent false
15986 5224 Acked Read 4 Done Done Sent false
15987 5225 Acked Flush 0 Done Done Sent false
15988 STATES DS:0 DS:1 DS:2 TOTAL
15989 Sent 0 0 302 302
15990 Done 302 302 0 604
15991 Skipped 0 0 0 0
15992 Error 0 0 0 0
15993 Last Flush: 5225 5225 4923
15994 Downstairs last five completed: 4923 4922 4921 4920 4919
15995 Upstairs last five completed: 5225 5224 5223 5222 5221
15996 CLIENT: Up:0 ds:302 act:2
15997 ----------------------------------------------------------------
15998 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15999 States: Active Active Offline
16000 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16001 4924 Acked Flush 0 Done Done Sent false
16002 4925 Acked Write 8 Done Done Sent false
16003 4926 Acked Read 3 Done Done Sent false
16004 4927 Acked Read 9 Done Done Sent false
16005 4928 Acked Write 4 Done Done Sent false
16006 4929 Acked Flush 0 Done Done Sent false
16007 4930 Acked Write 7 Done Done Sent false
16008 4931 Acked Write 1 Done Done Sent false
16009 4932 Acked Write 4 Done Done Sent false
16010 4933 Acked Write 3 Done Done Sent false
16011 4934 Acked Read 3 Done Done Sent false
16012 4935 Acked Flush 0 Done Done Sent false
16013 4936 Acked Write 9 Done Done Sent false
16014 4937 Acked Flush 0 Done Done Sent false
16015 4938 Acked Flush 0 Done Done Sent false
16016 4939 Acked Read 9 Done Done Sent false
16017 4940 Acked Flush 0 Done Done Sent false
16018 4941 Acked Read 9 Done Done Sent false
16019 4942 Acked Read 7 Done Done Sent false
16020 4943 Acked Read 7 Done Done Sent false
16021 4944 Acked Write 7 Done Done Sent false
16022 4945 Acked Write 7 Done Done Sent false
16023 4946 Acked Write 5 Done Done Sent false
16024 4947 Acked Write 1 Done Done Sent false
16025 4948 Acked Write 8 Done Done Sent false
16026 4949 Acked Write 6 Done Done Sent false
16027 4950 Acked Write 6 Done Done Sent false
16028 4951 Acked Read 7 Done Done Sent false
16029 4952 Acked Flush 0 Done Done Sent false
16030 4953 Acked Read 8 Done Done Sent false
16031 4954 Acked Write 9 Done Done Sent false
16032 4955 Acked Read 7 Done Done Sent false
16033 4956 Acked Write 9 Done Done Sent false
16034 4957 Acked Read 5 Done Done Sent false
16035 4958 Acked Flush 0 Done Done Sent false
16036 4959 Acked Read 6 Done Done Sent false
16037 4960 Acked Flush 0 Done Done Sent false
16038 4961 Acked Flush 0 Done Done Sent false
16039 4962 Acked Read 3 Done Done Sent false
16040 4963 Acked Write 10 Done Done Sent false
16041 4964 Acked Write 9 Done Done Sent false
16042 4965 Acked Read 5 Done Done Sent false
16043 4966 Acked Read 8 Done Done Sent false
16044 4967 Acked Read 5 Done Done Sent false
16045 4968 Acked Read 6 Done Done Sent false
16046 4969 Acked Write 8 Done Done Sent false
16047 4970 Acked Read 1 Done Done Sent false
16048 4971 Acked Read 9 Done Done Sent false
16049 4972 Acked Read 9 Done Done Sent false
16050 4973 Acked Flush 0 Done Done Sent false
16051 4974 Acked Read 9 Done Done Sent false
16052 4975 Acked Read 4 Done Done Sent false
16053 4976 Acked Write 6 Done Done Sent false
16054 4977 Acked Write 6 Done Done Sent false
16055 4978 Acked Read 8 Done Done Sent false
16056 4979 Acked Read 10 Done Done Sent false
16057 4980 Acked Read 6 Done Done Sent false
16058 4981 Acked Read 8 Done Done Sent false
16059 4982 Acked Read 8 Done Done Sent false
16060 4983 Acked Read 9 Done Done Sent false
16061 4984 Acked Flush 0 Done Done Sent false
16062 4985 Acked Flush 0 Done Done Sent false
16063 4986 Acked Write 4 Done Done Sent false
16064 4987 Acked Read 5 Done Done Sent false
16065 4988 Acked Read 4 Done Done Sent false
16066 4989 Acked Flush 0 Done Done Sent false
16067 4990 Acked Write 2 Done Done Sent false
16068 4991 Acked Read 4 Done Done Sent false
16069 4992 Acked Read 8 Done Done Sent false
16070 4993 Acked Write 3 Done Done Sent false
16071 4994 Acked Write 2 Done Done Sent false
16072 4995 Acked Read 2 Done Done Sent false
16073 4996 Acked Read 7 Done Done Sent false
16074 4997 Acked Write 9 Done Done Sent false
16075 4998 Acked Write 10 Done Done Sent false
16076 4999 Acked Read 7 Done Done Sent false
16077 5000 Acked Read 7 Done Done Sent false
16078 5001 Acked Read 10 Done Done Sent false
16079 5002 Acked Write 6 Done Done Sent false
16080 5003 Acked Write 9 Done Done Sent false
16081 5004 Acked Read 9 Done Done Sent false
16082 5005 Acked Write 10 Done Done Sent false
16083 5006 Acked Read 7 Done Done Sent false
16084 5007 Acked Read 9 Done Done Sent false
16085 5008 Acked Flush 0 Done Done Sent false
16086 5009 Acked Write 4 Done Done Sent false
16087 5010 Acked Write 10 Done Done Sent false
16088 5011 Acked Write 5 Done Done Sent false
16089 5012 Acked Flush 0 Done Done Sent false
16090 5013 Acked Write 3 Done Done Sent false
16091 5014 Acked Read 1 Done Done Sent false
16092 5015 Acked Write 10 Done Done Sent false
16093 5016 Acked Write 3 Done Done Sent false
16094 5017 Acked Write 8 Done Done Sent false
16095 5018 Acked Write 9 Done Done Sent false
16096 5019 Acked Flush 0 Done Done Sent false
16097 5020 Acked Write 6 Done Done Sent false
16098 5021 Acked Flush 0 Done Done Sent false
16099 5022 Acked Read 2 Done Done Sent false
16100 5023 Acked Flush 0 Done Done Sent false
16101 5024 Acked Read 9 Done Done Sent false
16102 5025 Acked Read 4 Done Done Sent false
16103 5026 Acked Read 5 Done Done Sent false
16104 5027 Acked Read 8 Done Done Sent false
16105 5028 Acked Read 2 Done Done Sent false
16106 5029 Acked Flush 0 Done Done Sent false
16107 5030 Acked Read 2 Done Done Sent false
16108 5031 Acked Read 2 Done Done Sent false
16109 5032 Acked Read 8 Done Done Sent false
16110 5033 Acked Write 3 Done Done Sent false
16111 5034 Acked Read 8 Done Done Sent false
16112 5035 Acked Write 1 Done Done Sent false
16113 5036 Acked Read 2 Done Done Sent false
16114 5037 Acked Flush 0 Done Done Sent false
16115 5038 Acked Read 1 Done Done Sent false
16116 5039 Acked Read 10 Done Done Sent false
16117 5040 Acked Write 10 Done Done Sent false
16118 5041 Acked Write 1 Done Done Sent false
16119 5042 Acked Read 8 Done Done Sent false
16120 5043 Acked Flush 0 Done Done Sent false
16121 5044 Acked Write 5 Done Done Sent false
16122 5045 Acked Read 2 Done Done Sent false
16123 5046 Acked Read 7 Done Done Sent false
16124 5047 Acked Write 9 Done Done Sent false
16125 5048 Acked Write 5 Done Done Sent false
16126 5049 Acked Write 5 Done Done Sent false
16127 5050 Acked Read 6 Done Done Sent false
16128 5051 Acked Read 1 Done Done Sent false
16129 5052 Acked Write 8 Done Done Sent false
16130 5053 Acked Write 1 Done Done Sent false
16131 5054 Acked Write 10 Done Done Sent false
16132 5055 Acked Write 6 Done Done Sent false
16133 5056 Acked Read 4 Done Done Sent false
16134 5057 Acked Read 8 Done Done Sent false
16135 5058 Acked Flush 0 Done Done Sent false
16136 5059 Acked Write 6 Done Done Sent false
16137 5060 Acked Read 4 Done Done Sent false
16138 5061 Acked Read 3 Done Done Sent false
16139 5062 Acked Write 4 Done Done Sent false
16140 5063 Acked Write 5 Done Done Sent false
16141 5064 Acked Read 2 Done Done Sent false
16142 5065 Acked Flush 0 Done Done Sent false
16143 5066 Acked Read 2 Done Done Sent false
16144 5067 Acked Read 4 Done Done Sent false
16145 5068 Acked Write 4 Done Done Sent false
16146 5069 Acked Read 3 Done Done Sent false
16147 5070 Acked Read 1 Done Done Sent false
16148 5071 Acked Flush 0 Done Done Sent false
16149 5072 Acked Read 4 Done Done Sent false
16150 5073 Acked Write 4 Done Done Sent false
16151 5074 Acked Read 9 Done Done Sent false
16152 5075 Acked Read 5 Done Done Sent false
16153 5076 Acked Flush 0 Done Done Sent false
16154 5077 Acked Read 6 Done Done Sent false
16155 5078 Acked Read 3 Done Done Sent false
16156 5079 Acked Read 9 Done Done Sent false
16157 5080 Acked Read 8 Done Done Sent false
16158 5081 Acked Read 2 Done Done Sent false
16159 5082 Acked Read 6 Done Done Sent false
16160 5083 Acked Write 9 Done Done Sent false
16161 5084 Acked Read 7 Done Done Sent false
16162 5085 Acked Read 1 Done Done Sent false
16163 5086 Acked Write 3 Done Done Sent false
16164 5087 Acked Write 6 Done Done Sent false
16165 5088 Acked Write 5 Done Done Sent false
16166 5089 Acked Read 4 Done Done Sent false
16167 5090 Acked Read 1 Done Done Sent false
16168 5091 Acked Write 2 Done Done Sent false
16169 5092 Acked Read 3 Done Done Sent false
16170 5093 Acked Write 9 Done Done Sent false
16171 5094 Acked Write 5 Done Done Sent false
16172 5095 Acked Read 9 Done Done Sent false
16173 5096 Acked Flush 0 Done Done Sent false
16174 5097 Acked Write 4 Done Done Sent false
16175 5098 Acked Read 7 Done Done Sent false
16176 5099 Acked Read 2 Done Done Sent false
16177 5100 Acked Flush 0 Done Done Sent false
16178 5101 Acked Write 9 Done Done Sent false
16179 5102 Acked Write 8 Done Done Sent false
16180 5103 Acked Write 6 Done Done Sent false
16181 5104 Acked Write 4 Done Done Sent false
16182 5105 Acked Write 4 Done Done Sent false
16183 5106 Acked Write 7 Done Done Sent false
16184 5107 Acked Read 8 Done Done Sent false
16185 5108 Acked Read 10 Done Done Sent false
16186 5109 Acked Flush 0 Done Done Sent false
16187 5110 Acked Flush 0 Done Done Sent false
16188 5111 Acked Write 5 Done Done Sent false
16189 5112 Acked Write 4 Done Done Sent false
16190 5113 Acked Write 8 Done Done Sent false
16191 5114 Acked Write 7 Done Done Sent false
16192 5115 Acked Write 4 Done Done Sent false
16193 5116 Acked Write 6 Done Done Sent false
16194 5117 Acked Write 5 Done Done Sent false
16195 5118 Acked Write 4 Done Done Sent false
16196 5119 Acked Read 1 Done Done Sent false
16197 5120 Acked Read 1 Done Done Sent false
16198 5121 Acked Read 10 Done Done Sent false
16199 5122 Acked Write 6 Done Done Sent false
16200 5123 Acked Read 9 Done Done Sent false
16201 5124 Acked Read 5 Done Done Sent false
16202 5125 Acked Write 9 Done Done Sent false
16203 5126 Acked Write 3 Done Done Sent false
16204 5127 Acked Read 4 Done Done Sent false
16205 5128 Acked Read 4 Done Done Sent false
16206 5129 Acked Read 4 Done Done Sent false
16207 5130 Acked Write 2 Done Done Sent false
16208 5131 Acked Write 7 Done Done Sent false
16209 5132 Acked Write 3 Done Done Sent false
16210 5133 Acked Read 9 Done Done Sent false
16211 5134 Acked Read 8 Done Done Sent false
16212 5135 Acked Write 9 Done Done Sent false
16213 5136 Acked Read 1 Done Done Sent false
16214 5137 Acked Read 8 Done Done Sent false
16215 5138 Acked Read 7 Done Done Sent false
16216 5139 Acked Flush 0 Done Done Sent false
16217 5140 Acked Read 1 Done Done Sent false
16218 5141 Acked Read 9 Done Done Sent false
16219 5142 Acked Read 9 Done Done Sent false
16220 5143 Acked Write 4 Done Done Sent false
16221 5144 Acked Flush 0 Done Done Sent false
16222 5145 Acked Write 2 Done Done Sent false
16223 5146 Acked Read 2 Done Done Sent false
16224 5147 Acked Read 3 Done Done Sent false
16225 5148 Acked Write 6 Done Done Sent false
16226 5149 Acked Write 10 Done Done Sent false
16227 5150 Acked Read 5 Done Done Sent false
16228 5151 Acked Write 7 Done Done Sent false
16229 5152 Acked Read 8 Done Done Sent false
16230 5153 Acked Write 6 Done Done Sent false
16231 5154 Acked Read 4 Done Done Sent false
16232 5155 Acked Write 3 Done Done Sent false
16233 5156 Acked Read 10 Done Done Sent false
16234 5157 Acked Read 5 Done Done Sent false
16235 5158 Acked Read 6 Done Done Sent false
16236 5159 Acked Write 1 Done Done Sent false
16237 5160 Acked Read 5 Done Done Sent false
16238 5161 Acked Flush 0 Done Done Sent false
16239 5162 Acked Flush 0 Done Done Sent false
16240 5163 Acked Read 1 Done Done Sent false
16241 5164 Acked Read 8 Done Done Sent false
16242 5165 Acked Write 2 Done Done Sent false
16243 5166 Acked Read 5 Done Done Sent false
16244 5167 Acked Read 10 Done Done Sent false
16245 5168 Acked Read 9 Done Done Sent false
16246 5169 Acked Read 5 Done Done Sent false
16247 5170 Acked Write 8 Done Done Sent false
16248 5171 Acked Read 5 Done Done Sent false
16249 5172 Acked Flush 0 Done Done Sent false
16250 5173 Acked Read 1 Done Done Sent false
16251 5174 Acked Read 3 Done Done Sent false
16252 5175 Acked Flush 0 Done Done Sent false
16253 5176 Acked Read 8 Done Done Sent false
16254 5177 Acked Read 1 Done Done Sent false
16255 5178 Acked Read 5 Done Done Sent false
16256 5179 Acked Write 2 Done Done Sent false
16257 5180 Acked Read 9 Done Done Sent false
16258 5181 Acked Write 4 Done Done Sent false
16259 5182 Acked Write 7 Done Done Sent false
16260 5183 Acked Read 8 Done Done Sent false
16261 5184 Acked Read 2 Done Done Sent false
16262 5185 Acked Write 9 Done Done Sent false
16263 5186 Acked Read 10 Done Done Sent false
16264 5187 Acked Write 10 Done Done Sent false
16265 5188 Acked Write 5 Done Done Sent false
16266 5189 Acked Flush 0 Done Done Sent false
16267 5190 Acked Write 7 Done Done Sent false
16268 5191 Acked Read 5 Done Done Sent false
16269 5192 Acked Flush 0 Done Done Sent false
16270 5193 Acked Write 2 Done Done Sent false
16271 5194 Acked Write 3 Done Done Sent false
16272 5195 Acked Read 9 Done Done Sent false
16273 5196 Acked Read 9 Done Done Sent false
16274 5197 Acked Write 7 Done Done Sent false
16275 5198 Acked Write 1 Done Done Sent false
16276 5199 Acked Write 8 Done Done Sent false
16277 5200 Acked Write 4 Done Done Sent false
16278 5201 Acked Read 2 Done Done Sent false
16279 5202 Acked Flush 0 Done Done Sent false
16280 5203 Acked Read 5 Done Done Sent false
16281 5204 Acked Read 10 Done Done Sent false
16282 5205 Acked Read 2 Done Done Sent false
16283 5206 Acked Read 4 Done Done Sent false
16284 5207 Acked Write 5 Done Done Sent false
16285 5208 Acked Read 4 Done Done Sent false
16286 5209 Acked Read 2 Done Done Sent false
16287 5210 Acked Read 3 Done Done Sent false
16288 5211 Acked Flush 0 Done Done Sent false
16289 5212 Acked Read 7 Done Done Sent false
16290 5213 Acked Write 5 Done Done Sent false
16291 5214 Acked Read 9 Done Done Sent false
16292 5215 Acked Read 9 Done Done Sent false
16293 5216 Acked Read 2 Done Done Sent false
16294 5217 Acked Write 2 Done Done Sent false
16295 5218 Acked Write 2 Done Done Sent false
16296 5219 Acked Read 10 Done Done Sent false
16297 5220 Acked Flush 0 Done Done Sent false
16298 5221 Acked Write 7 Done Done Sent false
16299 5222 Acked Read 8 Done Done Sent false
16300 5223 Acked Write 10 Done Done Sent false
16301 5224 Acked Read 4 Done Done Sent false
16302 5225 Acked Flush 0 Done Done Sent false
16303 STATES DS:0 DS:1 DS:2 TOTAL
16304 Sent 0 0 302 302
16305 Done 302 302 0 604
16306 Skipped 0 0 0 0
16307 Error 0 0 0 0
16308 Last Flush: 5225 5225 4923
16309 Downstairs last five completed: 4923 4922 4921 4920 4919
16310 Upstairs last five completed: 5225 5224 5223 5222 5221
16311 CLIENT: Up:0 ds:302 act:2
163122025-01-17T21:14:31.624ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
163132025-01-17T21:14:31.624ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:43591) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
163142025-01-17T21:14:31.625ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
163152025-01-17T21:14:31.626ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
163162025-01-17T21:14:31.626ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
163172025-01-17T21:14:31.626ZINFOcrucible: send last flush ID to this DS: 4923 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
163182025-01-17T21:14:31.626ZINFOcrucible: Replied this last flush ID: 4923 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
163192025-01-17T21:14:31.626ZINFOcrucible: [2] client re-new 302 jobs since flush 4923 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
163202025-01-17T21:14:31.626ZINFOcrucible: [2] Replayed 302 jobs since flush: 4923 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
16321 ----------------------------------------------------------------
16322 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
16323 Downstairs last five completed: 5225 5224 5223 5222 5221
16324 Upstairs last five completed: 5225 5224 5223 5222 5221
16325 CLIENT: Up:0 ds:0 act:3
16326 Replay: All jobs finished
16327 Main task has work to do, go find it
16328 got dsc Stop(0)
16329 stop 0
16330 [8810] Got stop action so:false kr:false
16331 Jan 17 21:14:31.874 INFO request completed, latency_us: 305, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 41320544-f993-4dd6-8eba-21ed873cb682, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
163322025-01-17T21:14:31.875ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
16333 Jan 17 21:14:31.875 INFO request completed, latency_us: 126, response_code: 200, uri: /state/cid/0, method:{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:14:31.875655018Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
16334 GET, req_id: 0e54653a-973b-4007-afe6-394de54e3b88, remote_addr: {"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:14:31.875677065Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
16335 127.0.0.1:57191, local_addr: 127.0.0.1:9998
163362025-01-17T21:14:31.875ZINFOcrucible: client task is exiting = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
16337 [8810] Exited with: ExitStatus(unix_wait_status(9))
16338 [8810][0] reports Exit
16339 Jan 17 21:14:35.877 INFO request completed, latency_us: 204, response_code: 200, uri: /state/cid/0, method: GET, req_id: 790f4d15-3540-4e4f-a27c-664ff3943c02, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
16340 001/300 Write block 4297 len 32768 data: 6 5 5 5 5 4 4 2
16341 002/300 Write block 2619 len 40960 data: 2 2 2 3 3 3 3 3 4 5
16342 003/300 Read block 757 len 8192
16343 004/300 Write block 1117 len 4096 data: 4
16344 005/300 Read block 1945 len 40960
16345 006/300 Write block 4755 len 40960 data: 5 6 6 5 7 7 6 8 7 8
16346 007/300 Read block 1943 len 28672
16347 008/300 Read block 4673 len 8192
16348 009/300 Read block 4413 len 16384
16349 010/300 Write block 4221 len 28672 data: 4 3 3 3 3 2 2
16350 011/300 Read block 4790 len 32768
16351 012/300 Read block 383 len 36864
16352 013/300 Write block 4969 len 36864 data: 2 2 2 2 2 2 3 3 4
16353 014/300 Read block 1525 len 28672
16354 015/300 Read block 4181 len 36864
16355 016/300 Read block 2410 len 12288
16356 017/300 Read block 3215 len 24576
16357 018/300 Read block 3289 len 8192
16358 019/300 Read block 2482 len 16384
16359 020/300 Flush
16360 021/300 Read block 311 len 36864
16361 022/300 Write block 860 len 12288 data: 4 4 5
16362 023/300 Read block 957 len 40960
16363 024/300 Read block 271 len 36864
16364 025/300 Write block 3104 len 20480 data: 7 7 5 5 5
16365 026/300 Write block 152 len 12288 data: 5 5 4
16366 027/300 Write block 1707 len 24576 data: 5 6 8 7 5 4
16367 028/300 Write block 1392 len 32768 data: 5 4 4 5 5 6 6 6
16368 029/300 Write block 1604 len 20480 data: 4 3 3 3 2
16369 030/300 Write block 508 len 24576 data: 5 5 6 6 4 4
16370 031/300 Write block 3212 len 8192 data: 4 4
16371 032/300 Read block 384 len 12288
16372 033/300 Read block 805 len 8192
16373 034/300 Flush
16374 035/300 Flush
16375 036/300 Write block 150 len 24576 data: 4 5 6 6 5 5
16376 037/300 Read block 3838 len 40960
16377 038/300 Write block 4229 len 24576 data: 2 2 2 2 3 3
16378 039/300 Write block 1917 len 24576 data: 6 5 4 5 5 5
16379 040/300 Read block 2518 len 24576
16380 041/300 Flush
16381 042/300 Write block 3207 len 32768 data: 4 3 3 3 4 5 5 3
16382 043/300 Read block 1256 len 20480
16383 044/300 Read block 1089 len 20480
16384 045/300 Read block 3418 len 36864
16385 046/300 Read block 695 len 28672
16386 047/300 Write block 4423 len 32768 data: 4 6 5 4 4 4 5 3
16387 048/300 Write block 4470 len 12288 data: 4 5 6
16388 049/300 Read block 3959 len 32768
16389 050/300 Write block 445 len 4096 data: 3
16390 051/300 Write block 3324 len 4096 data: 4
16391 052/300 Write block 1675 len 40960 data: 4 4 5 5 6 5 2 2 3 3
16392 053/300 Flush
16393 054/300 Read block 3880 len 4096
16394 055/300 Write block 2239 len 8192 data: 4 4
16395 056/300 Write block 416 len 20480 data: 2 2 3 4 4
16396 057/300 Read block 1394 len 12288
16397 058/300 Read block 3830 len 8192
16398 059/300 Write block 3440 len 28672 data: 3 3 3 4 5 9 10
16399 060/300 Read block 1831 len 36864
16400 061/300 Read block 833 len 28672
16401 062/300 Write block 4675 len 24576 data: 7 5 6 6 6 5
16402 063/300 Write block 2509 len 36864 data: 2 3 3 4 4 5 5 6 6
16403 064/300 Write block 3109 len 40960 data: 4 4 3 2 2 2 3 3 3 4
16404 065/300 Read block 3808 len 8192
16405 066/300 Flush
16406 067/300 Read block 4776 len 32768
16407 068/300 Read block 3214 len 36864
16408 069/300 Read block 3502 len 28672
16409 070/300 Read block 1556 len 12288
16410 071/300 Read block 2103 len 8192
16411 072/300 Write block 740 len 20480 data: 3 3 3 2 3
16412 073/300 Read block 4016 len 8192
16413 074/300 Read block 4284 len 40960
16414 075/300 Write block 456 len 8192 data: 3 4
16415 076/300 Write block 275 len 32768 data: 4 4 4 3 3 5 5 5
16416 077/300 Read block 1579 len 24576
16417 078/300 Read block 832 len 36864
16418 079/300 Read block 1129 len 28672
16419 080/300 Write block 4024 len 8192 data: 3 4
16420 081/300 Write block 707 len 28672 data: 4 5 5 5 5 4 4
16421 082/300 Write block 575 len 28672 data: 4 3 3 3 3 4 4
16422 083/300 Flush
16423 084/300 Read block 2100 len 16384
16424 085/300 Read block 1725 len 20480
16425 086/300 Read block 3146 len 20480
16426 087/300 Read block 4355 len 24576
16427 088/300 Flush
16428 089/300 Write block 3783 len 4096 data: 4
16429 090/300 Write block 4820 len 4096 data: 4
16430 091/300 Flush
16431 092/300 Read block 4273 len 32768
16432 093/300 Read block 1622 len 24576
16433 094/300 Write block 3594 len 12288 data: 2 4 3
16434 095/300 Flush
16435 096/300 Write block 672 len 8192 data: 4 3
16436 097/300 Read block 580 len 4096
16437 098/300 Write block 3931 len 40960 data: 6 6 7 7 6 6 4 5 4 4
16438 099/300 Read block 4463 len 16384
16439 100/300 Write block 3411 len 32768 data: 2 3 3 3 3 3 3 4
16440 101/300 Write block 2345 len 32768 data: 5 6 5 5 6 6 7 5
16441 102/300 Write block 2160 len 28672 data: 3 5 5 5 6 6 7
16442 103/300 Write block 4364 len 24576 data: 5 5 4 4 4 4
16443 104/300 Read block 613 len 32768
16444 105/300 Flush
16445 106/300 Flush
16446 107/300 Write block 194 len 24576 data: 3 6 6 6 4 4
16447 108/300 Read block 273 len 4096
16448 109/300 Write block 4418 len 16384 data: 8 5 4 4
16449 110/300 Read block 4875 len 12288
16450 111/300 Read block 1614 len 36864
16451 112/300 Read block 4122 len 40960
16452 113/300 Write block 4774 len 4096 data: 3
16453 114/300 Read block 4451 len 24576
16454 115/300 Write block 3331 len 28672 data: 3 3 4 4 5 5 5
16455 116/300 Write block 4549 len 36864 data: 4 4 4 5 6 5 4 5 5
16456 117/300 Read block 3918 len 24576
16457 118/300 Flush
16458 119/300 Read block 4172 len 28672
16459 120/300 Read block 4152 len 24576
16460 121/300 Write block 1737 len 36864 data: 4 6 5 4 5 6 5 5 5
16461 122/300 Read block 1057 len 32768
16462 123/300 Read block 3126 len 20480
16463 124/300 Read block 2519 len 40960
16464 125/300 Read block 3503 len 24576
16465 126/300 Write block 3437 len 4096 data: 3
16466 127/300 Read block 369 len 36864
16467 128/300 Read block 1423 len 40960
16468 129/300 Flush
16469 130/300 Write block 4224 len 40960 data: 4 4 3 3 2 3 3 3 3 4
16470 131/300 Read block 3737 len 12288
16471 132/300 Read block 4433 len 28672
16472 133/300 Read block 2651 len 20480
16473 134/300 Read block 1018 len 28672
16474 135/300 Write block 2322 len 24576 data: 4 3 3 4 4 3
16475 136/300 Write block 1477 len 40960 data: 2 2 2 2 3 3 3 4 4 6
16476 137/300 Flush
16477 138/300 Read block 2526 len 4096
16478 139/300 Read block 4669 len 8192
16479 140/300 Write block 4261 len 12288 data: 5 5 6
16480 141/300 Read block 3390 len 20480
16481 142/300 Read block 1833 len 16384
16482 143/300 Write block 4720 len 28672 data: 3 3 3 2 2 2 2
16483 144/300 Read block 2411 len 24576
16484 145/300 Read block 4686 len 4096
16485 146/300 Read block 4503 len 24576
16486 147/300 Write block 2264 len 32768 data: 6 7 8 6 4 4 4 4
16487 148/300 Write block 1326 len 28672 data: 5 5 6 6 6 6 4
16488 149/300 Write block 3779 len 36864 data: 4 4 4 5 5 4 4 4 4
16489 150/300 Flush
16490 151/300 Flush
16491 152/300 Write block 139 len 36864 data: 3 3 2 3 3 3 4 4 5
16492 153/300 Write block 1753 len 8192 data: 2 2
16493 154/300 Read block 1088 len 20480
16494 155/300 Write block 3071 len 16384 data: 3 3 2 3
16495 156/300 Read block 3657 len 24576
16496 157/300 Read block 1643 len 36864
16497 158/300 Write block 45 len 36864 data: 3 2 2 2 2 3 3 3 4
16498 159/300 Read block 580 len 8192
16499 160/300 Write block 3407 len 12288 data: 3 3 3
16500 161/300 Read block 3204 len 12288
16501 162/300 Write block 1261 len 8192 data: 3 3
16502 163/300 Write block 3081 len 12288 data: 5 6 5
16503 164/300 Read block 1526 len 12288
16504 165/300 Read block 1575 len 28672
16505 166/300 Read block 2036 len 28672
16506 167/300 Write block 1084 len 8192 data: 6 6
16507 168/300 Write block 153 len 12288 data: 7 6 6
16508 169/300 Write block 3692 len 16384 data: 3 3 3 3
16509 170/300 Read block 3408 len 28672
16510 171/300 Write block 4004 len 28672 data: 6 6 8 8 8 8 5
16511 172/300 Write block 2683 len 40960 data: 6 5 6 5 5 5 4 3 3 3
16512 173/300 Flush
16513 174/300 Write block 4474 len 40960 data: 6 6 5 4 3 2 2 3 3 4
16514 175/300 Read block 3611 len 8192
16515 176/300 Write block 269 len 40960 data: 3 5 5 5 5 4 5 5 5 4
16516 177/300 Flush
16517 178/300 Write block 284 len 8192 data: 4 4
16518 179/300 Read block 2186 len 12288
16519 180/300 Write block 633 len 20480 data: 3 3 2 3 3
16520 181/300 Write block 2149 len 32768 data: 5 4 4 5 4 4 3 3
16521 182/300 Read block 4799 len 20480
16522 183/300 Flush
16523 184/300 Flush
16524 185/300 Read block 4632 len 8192
16525 186/300 Write block 543 len 40960 data: 4 3 3 3 4 4 5 5 3 3
16526 187/300 Write block 4885 len 32768 data: 3 2 3 3 4 4 4 4
16527 188/300 Write block 1047 len 4096 data: 3
16528 189/300 Flush
16529 190/300 Write block 3172 len 28672 data: 3 4 4 5 5 5 5
16530 191/300 Read block 729 len 36864
16531 192/300 Flush
16532 193/300 Read block 3372 len 28672
16533 194/300 Read block 362 len 8192
16534 195/300 Write block 1498 len 28672 data: 3 2 4 4 4 3 4
16535 196/300 Read block 3531 len 28672
16536 197/300 Read block 3558 len 28672
16537 198/300 Read block 280 len 12288
16538 199/300 Read block 1982 len 12288
16539 200/300 Write block 664 len 20480 data: 4 6 6 6 4
16540 201/300 Write block 3071 len 28672 data: 4 4 3 4 4 4 4
16541 202/300 Flush
16542 203/300 Flush
16543 204/300 Read block 52 len 36864
16544 205/300 Read block 2566 len 12288
16545 206/300 Flush
16546 207/300 Read block 4525 len 16384
16547 208/300 Write block 2389 len 40960 data: 4 5 5 5 6 6 6 7 7 4
16548 209/300 Write block 1110 len 4096 data: 4
16549 210/300 Write block 3148 len 32768 data: 3 3 3 3 3 3 3 3
16550 211/300 Write block 3374 len 16384 data: 2 3 3 3
16551 212/300 Read block 4679 len 32768
16552 213/300 Write block 2155 len 32768 data: 4 4 3 2 3 4 6 6
16553 214/300 Read block 2409 len 24576
16554 215/300 Read block 4409 len 40960
16555 216/300 Read block 12 len 20480
16556 217/300 Write block 1035 len 12288 data: 5 5 5
16557 218/300 Read block 401 len 16384
16558 219/300 Write block 1098 len 28672 data: 5 6 5 5 6 6 6
16559 220/300 Write block 70 len 28672 data: 4 4 5 5 5 6 6
16560 221/300 Write block 4605 len 28672 data: 2 2 4 4 4 4 4
16561 222/300 Write block 2067 len 36864 data: 7 7 6 4 3 4 3 3 3
16562 223/300 Write block 4606 len 40960 data: 3 5 5 5 5 5 4 3 3 2
16563 224/300 Read block 531 len 28672
16564 225/300 Read block 268 len 36864
16565 226/300 Read block 1035 len 36864
16566 227/300 Write block 4593 len 16384 data: 2 2 2 2
16567 228/300 Flush
16568 229/300 Flush
16569 230/300 Write block 2394 len 16384 data: 7 7 8 8
16570 231/300 Read block 4222 len 8192
16571 232/300 Read block 1216 len 20480
16572 233/300 Write block 87 len 20480 data: 4 4 4 4 4
16573 234/300 Read block 2006 len 36864
16574 235/300 Write block 1332 len 24576 data: 5 4 3 3 4 4
16575 236/300 Write block 2672 len 24576 data: 2 2 3 3 3 3
16576 237/300 Flush
16577 238/300 Read block 3400 len 12288
16578 239/300 Read block 4018 len 24576
16579 240/300 Write block 3916 len 40960 data: 3 4 4 3 2 2 2 2 2 2
16580 241/300 Flush
16581 242/300 Read block 3588 len 32768
16582 243/300 Read block 1071 len 8192
16583 244/300 Write block 2499 len 40960 data: 2 2 2 2 2 2 2 2 2 2
16584 245/300 Read block 2980 len 4096
16585 246/300 Write block 2407 len 32768 data: 5 5 4 4 4 3 2 2
16586 247/300 Read block 3712 len 20480
16587 248/300 Read block 3547 len 20480
16588 249/300 Write block 4808 len 4096 data: 4
16589 250/300 Read block 379 len 40960
16590 251/300 Write block 3467 len 40960 data: 4 4 3 3 5 5 5 5 4 3
16591 252/300 Flush
16592 253/300 Write block 4010 len 16384 data: 6 6 4 5
16593 254/300 Read block 4572 len 20480
16594 255/300 Read block 4952 len 36864
16595 256/300 Write block 693 len 12288 data: 5 5 4
16596 257/300 Read block 3758 len 8192
16597 258/300 Write block 1212 len 8192 data: 2 2
16598 259/300 Read block 2708 len 12288
16599 260/300 Read block 1640 len 36864
16600 261/300 Write block 888 len 40960 data: 6 4 3 3 2 2 2 3 3 3
16601 262/300 Read block 222 len 32768
16602 263/300 Read block 223 len 4096
16603 264/300 Read block 3277 len 4096
16604 265/300 Write block 3890 len 4096 data: 3
16605 266/300 Read block 4633 len 8192
16606 267/300 Write block 3809 len 20480 data: 3 4 4 4 4
16607 268/300 Write block 1137 len 40960 data: 3 4 3 2 2 2 2 2 2 2
16608 269/300 Write block 712 len 32768 data: 5 5 4 4 4 2 3 3
16609 270/300 Write block 4984 len 12288 data: 3 3 2
16610 271/300 Read block 1307 len 12288
16611 272/300 Write block 1245 len 16384 data: 4 4 4 3
16612 273/300 Read block 3905 len 20480
16613 274/300 Write block 4426 len 20480 data: 5 5 5 6 4
16614 275/300 Write block 622 len 40960 data: 4 3 3 3 3 3 2 3 4 4
16615 276/300 Write block 3629 len 8192 data: 2 2
16616 277/300 Read block 771 len 8192
16617 278/300 Read block 374 len 40960
16618 279/300 Flush
16619 280/300 Write block 1594 len 32768 data: 3 3 3 2 2 4 4 4
16620 281/300 Read block 2480 len 28672
16621 282/300 Read block 3156 len 40960
16622 283/300 Write block 3966 len 12288 data: 3 3 2
16623 284/300 Read block 1244 len 20480
16624 285/300 Read block 1180 len 32768
16625 286/300 Write block 2638 len 28672 data: 4 4 2 2 2 3 4
16626 287/300 Read block 3831 len 32768
16627 288/300 Write block 1582 len 4096 data: 9
16628 289/300 Write block 2428 len 36864 data: 4 4 3 3 3 2 2 2 2
16629 290/300 Read block 1948 len 12288
16630 291/300 Read block 3760 len 4096
16631 292/300 Flush
16632 293/300 Flush
16633 294/300 Read block 577 len 8192
16634 295/300 Write block 282 len 8192 data: 6 5
16635 296/300 Read block 4031 len 36864
16636 297/300 Write block 1447 len 36864 data: 4 4 4 3 3 3 4 3 3
16637 298/300 Write block 4717 len 4096 data: 4
16638 299/300 Write block 3157 len 32768 data: 4 4 4 3 4 4 5 3
16639 300/300 Write block 2316 len 40960 data: 3 3 5 5 4 3 5 4 4 5
16640 301/300 Write block 2548 len 4096 data: 6
16641 Main task has work to do, go find it
16642 got dsc Start(0)
16643 start 0
16644 Jan 17 21:14:36.119 INFO request completed, latency_us: 212, response_code: 204, uri: /start/cid/0, method: POST, req_id: a1a64449-7147-4d56-8dad-5d3130c40ad0, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
16645 Replay: started 0, returned:Ok(())
16646 ----------------------------------------------------------------
16647 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16648 States: Offline Active Active
16649 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16650 5226 Acked Write 8 Sent Done Done false
16651 5227 Acked Write 10 Sent Done Done false
16652 5228 Acked Read 2 Sent Done Done false
16653 5229 Acked Write 1 Sent Done Done false
16654 5230 Acked Read 10 Sent Done Done false
16655 5231 Acked Write 10 Sent Done Done false
16656 5232 Acked Read 7 Sent Done Done false
16657 5233 Acked Read 2 Sent Done Done false
16658 5234 Acked Read 4 Sent Done Done false
16659 5235 Acked Write 7 Sent Done Done false
16660 5236 Acked Read 8 Sent Done Done false
16661 5237 Acked Read 9 Sent Done Done false
16662 5238 Acked Write 9 Sent Done Done false
16663 5239 Acked Read 7 Sent Done Done false
16664 5240 Acked Read 9 Sent Done Done false
16665 5241 Acked Read 3 Sent Done Done false
16666 5242 Acked Read 6 Sent Done Done false
16667 5243 Acked Read 2 Sent Done Done false
16668 5244 Acked Read 4 Sent Done Done false
16669 5245 Acked Flush 0 Sent Done Done false
16670 5246 Acked Read 9 Sent Done Done false
16671 5247 Acked Write 3 Sent Done Done false
16672 5248 Acked Read 10 Sent Done Done false
16673 5249 Acked Read 9 Sent Done Done false
16674 5250 Acked Write 5 Sent Done Done false
16675 5251 Acked Write 3 Sent Done Done false
16676 5252 Acked Write 6 Sent Done Done false
16677 5253 Acked Write 8 Sent Done Done false
16678 5254 Acked Write 5 Sent Done Done false
16679 5255 Acked Write 6 Sent Done Done false
16680 5256 Acked Write 2 Sent Done Done false
16681 5257 Acked Read 3 Sent Done Done false
16682 5258 Acked Read 2 Sent Done Done false
16683 5259 Acked Flush 0 Sent Done Done false
16684 5260 Acked Flush 0 Sent Done Done false
16685 5261 Acked Write 6 Sent Done Done false
16686 5262 Acked Read 10 Sent Done Done false
16687 5263 Acked Write 6 Sent Done Done false
16688 5264 Acked Write 6 Sent Done Done false
16689 5265 Acked Read 6 Sent Done Done false
16690 5266 Acked Flush 0 Sent Done Done false
16691 5267 Acked Write 8 Sent Done Done false
16692 5268 Acked Read 5 Sent Done Done false
16693 5269 Acked Read 5 Sent Done Done false
16694 5270 Acked Read 9 Sent Done Done false
16695 5271 Acked Read 7 Sent Done Done false
16696 5272 Acked Write 8 Sent Done Done false
16697 5273 Acked Write 3 Sent Done Done false
16698 5274 Acked Read 8 Sent Done Done false
16699 5275 Acked Write 1 Sent Done Done false
16700 5276 Acked Write 1 Sent Done Done false
16701 5277 Acked Write 10 Sent Done Done false
16702 5278 Acked Flush 0 Sent Done Done false
16703 5279 Acked Read 1 Sent Done Done false
16704 5280 Acked Write 2 Sent Done Done false
16705 5281 Acked Write 5 Sent Done Done false
16706 5282 Acked Read 3 Sent Done Done false
16707 5283 Acked Read 2 Sent Done Done false
16708 5284 Acked Write 7 Sent Done Done false
16709 5285 Acked Read 9 Sent Done Done false
16710 5286 Acked Read 7 Sent Done Done false
16711 5287 Acked Write 6 Sent Done Done false
16712 5288 Acked Write 9 Sent Done Done false
16713 5289 Acked Write 10 Sent Done Done false
16714 5290 Acked Read 2 Sent Done Done false
16715 5291 Acked Flush 0 Sent Done Done false
16716 5292 Acked Read 8 Sent Done Done false
16717 5293 Acked Read 9 Sent Done Done false
16718 5294 Acked Read 7 Sent Done Done false
16719 5295 Acked Read 3 Sent Done Done false
16720 5296 Acked Read 2 Sent Done Done false
16721 5297 Acked Write 5 Sent Done Done false
16722 5298 Acked Read 2 Sent Done Done false
16723 5299 Acked Read 10 Sent Done Done false
16724 5300 Acked Write 2 Sent Done Done false
16725 5301 Acked Write 8 Sent Done Done false
16726 5302 Acked Read 6 Sent Done Done false
16727 5303 Acked Read 9 Sent Done Done false
16728 5304 Acked Read 7 Sent Done Done false
16729 5305 Acked Write 2 Sent Done Done false
16730 5306 Acked Write 7 Sent Done Done false
16731 5307 Acked Write 7 Sent Done Done false
16732 5308 Acked Flush 0 Sent Done Done false
16733 5309 Acked Read 4 Sent Done Done false
16734 5310 Acked Read 5 Sent Done Done false
16735 5311 Acked Read 5 Sent Done Done false
16736 5312 Acked Read 6 Sent Done Done false
16737 5313 Acked Flush 0 Sent Done Done false
16738 5314 Acked Write 1 Sent Done Done false
16739 5315 Acked Write 1 Sent Done Done false
16740 5316 Acked Flush 0 Sent Done Done false
16741 5317 Acked Read 8 Sent Done Done false
16742 5318 Acked Read 6 Sent Done Done false
16743 5319 Acked Write 3 Sent Done Done false
16744 5320 Acked Flush 0 Sent Done Done false
16745 5321 Acked Write 2 Sent Done Done false
16746 5322 Acked Read 1 Sent Done Done false
16747 5323 Acked Write 10 Sent Done Done false
16748 5324 Acked Read 4 Sent Done Done false
16749 5325 Acked Write 8 Sent Done Done false
16750 5326 Acked Write 8 Sent Done Done false
16751 5327 Acked Write 7 Sent Done Done false
16752 5328 Acked Write 6 Sent Done Done false
16753 5329 Acked Read 8 Sent Done Done false
16754 5330 Acked Flush 0 Sent Done Done false
16755 5331 Acked Flush 0 Sent Done Done false
16756 5332 Acked Write 6 Sent Done Done false
16757 5333 Acked Read 1 Sent Done Done false
16758 5334 Acked Write 4 Sent Done Done false
16759 5335 Acked Read 3 Sent Done Done false
16760 5336 Acked Read 9 Sent Done Done false
16761 5337 Acked Read 10 Sent Done Done false
16762 5338 Acked Write 1 Sent Done Done false
16763 5339 Acked Read 6 Sent Done Done false
16764 5340 Acked Write 7 Sent Done Done false
16765 5341 Acked Write 9 Sent Done Done false
16766 5342 Acked Read 6 Sent Done Done false
16767 5343 Acked Flush 0 Sent Done Done false
16768 5344 Acked Read 7 Sent Done Done false
16769 5345 Acked Read 6 Sent Done Done false
16770 5346 Acked Write 9 Sent Done Done false
16771 5347 Acked Read 8 Sent Done Done false
16772 5348 Acked Read 5 Sent Done Done false
16773 5349 Acked Read 10 Sent Done Done false
16774 5350 Acked Read 6 Sent Done Done false
16775 5351 Acked Write 1 Sent Done Done false
16776 5352 Acked Read 9 Sent Done Done false
16777 5353 Acked Read 10 Sent Done Done false
16778 5354 Acked Flush 0 Sent Done Done false
16779 5355 Acked Write 10 Sent Done Done false
16780 5356 Acked Read 3 Sent Done Done false
16781 5357 Acked Read 7 Sent Done Done false
16782 5358 Acked Read 5 Sent Done Done false
16783 5359 Acked Read 7 Sent Done Done false
16784 5360 Acked Write 6 Sent Done Done false
16785 5361 Acked Write 10 Sent Done Done false
16786 5362 Acked Flush 0 Sent Done Done false
16787 5363 Acked Read 1 Sent Done Done false
16788 5364 Acked Read 2 Sent Done Done false
16789 5365 Acked Write 3 Sent Done Done false
16790 5366 Acked Read 5 Sent Done Done false
16791 5367 Acked Read 4 Sent Done Done false
16792 5368 Acked Write 7 Sent Done Done false
16793 5369 Acked Read 6 Sent Done Done false
16794 5370 Acked Read 1 Sent Done Done false
16795 5371 Acked Read 6 Sent Done Done false
16796 5372 Acked Write 8 Sent Done Done false
16797 5373 Acked Write 7 Sent Done Done false
16798 5374 Acked Write 9 Sent Done Done false
16799 5375 Acked Flush 0 Sent Done Done false
16800 5376 Acked Flush 0 Sent Done Done false
16801 5377 Acked Write 9 Sent Done Done false
16802 5378 Acked Write 2 Sent Done Done false
16803 5379 Acked Read 5 Sent Done Done false
16804 5380 Acked Write 4 Sent Done Done false
16805 5381 Acked Read 6 Sent Done Done false
16806 5382 Acked Read 9 Sent Done Done false
16807 5383 Acked Write 9 Sent Done Done false
16808 5384 Acked Read 2 Sent Done Done false
16809 5385 Acked Write 3 Sent Done Done false
16810 5386 Acked Read 3 Sent Done Done false
16811 5387 Acked Write 2 Sent Done Done false
16812 5388 Acked Write 3 Sent Done Done false
16813 5389 Acked Read 3 Sent Done Done false
16814 5390 Acked Read 7 Sent Done Done false
16815 5391 Acked Read 7 Sent Done Done false
16816 5392 Acked Write 2 Sent Done Done false
16817 5393 Acked Write 3 Sent Done Done false
16818 5394 Acked Write 4 Sent Done Done false
16819 5395 Acked Read 7 Sent Done Done false
16820 5396 Acked Write 7 Sent Done Done false
16821 5397 Acked Write 10 Sent Done Done false
16822 5398 Acked Flush 0 Sent Done Done false
16823 5399 Acked Write 10 Sent Done Done false
16824 5400 Acked Read 2 Sent Done Done false
16825 5401 Acked Write 10 Sent Done Done false
16826 5402 Acked Flush 0 Sent Done Done false
16827 5403 Acked Write 2 Sent Done Done false
16828 5404 Acked Read 3 Sent Done Done false
16829 5405 Acked Write 5 Sent Done Done false
16830 5406 Acked Write 8 Sent Done Done false
16831 5407 Acked Read 5 Sent Done Done false
16832 5408 Acked Flush 0 Sent Done Done false
16833 5409 Acked Flush 0 Sent Done Done false
16834 5410 Acked Read 2 Sent Done Done false
16835 5411 Acked Write 10 Sent Done Done false
16836 5412 Acked Write 8 Sent Done Done false
16837 5413 Acked Write 1 Sent Done Done false
16838 5414 Acked Flush 0 Sent Done Done false
16839 5415 Acked Write 7 Sent Done Done false
16840 5416 Acked Read 9 Sent Done Done false
16841 5417 Acked Flush 0 Sent Done Done false
16842 5418 Acked Read 7 Sent Done Done false
16843 5419 Acked Read 2 Sent Done Done false
16844 5420 Acked Write 7 Sent Done Done false
16845 5421 Acked Read 7 Sent Done Done false
16846 5422 Acked Read 7 Sent Done Done false
16847 5423 Acked Read 3 Sent Done Done false
16848 5424 Acked Read 3 Sent Done Done false
16849 5425 Acked Write 5 Sent Done Done false
16850 5426 Acked Write 7 Sent Done Done false
16851 5427 Acked Flush 0 Sent Done Done false
16852 5428 Acked Flush 0 Sent Done Done false
16853 5429 Acked Read 9 Sent Done Done false
16854 5430 Acked Read 3 Sent Done Done false
16855 5431 Acked Flush 0 Sent Done Done false
16856 5432 Acked Read 4 Sent Done Done false
16857 5433 Acked Write 10 Sent Done Done false
16858 5434 Acked Write 1 Sent Done Done false
16859 5435 Acked Write 8 Sent Done Done false
16860 5436 Acked Write 4 Sent Done Done false
16861 5437 Acked Read 8 Sent Done Done false
16862 5438 Acked Write 8 Sent Done Done false
16863 5439 Acked Read 6 Sent Done Done false
16864 5440 Acked Read 10 Sent Done Done false
16865 5441 Acked Read 5 Sent Done Done false
16866 5442 Acked Write 3 Sent Done Done false
16867 5443 Acked Read 4 Sent Done Done false
16868 5444 Acked Write 7 Sent Done Done false
16869 5445 Acked Write 7 Sent Done Done false
16870 5446 Acked Write 7 Sent Done Done false
16871 5447 Acked Write 9 Sent Done Done false
16872 5448 Acked Write 10 Sent Done Done false
16873 5449 Acked Read 7 Sent Done Done false
16874 5450 Acked Read 9 Sent Done Done false
16875 5451 Acked Read 9 Sent Done Done false
16876 5452 Acked Write 4 Sent Done Done false
16877 5453 Acked Flush 0 Sent Done Done false
16878 5454 Acked Flush 0 Sent Done Done false
16879 5455 Acked Write 4 Sent Done Done false
16880 5456 Acked Read 2 Sent Done Done false
16881 5457 Acked Read 5 Sent Done Done false
16882 5458 Acked Write 5 Sent Done Done false
16883 5459 Acked Read 9 Sent Done Done false
16884 5460 Acked Write 6 Sent Done Done false
16885 5461 Acked Write 6 Sent Done Done false
16886 5462 Acked Flush 0 Sent Done Done false
16887 5463 Acked Read 3 Sent Done Done false
16888 5464 Acked Read 6 Sent Done Done false
16889 5465 Acked Write 10 Sent Done Done false
16890 5466 Acked Flush 0 Sent Done Done false
16891 5467 Acked Read 8 Sent Done Done false
16892 5468 Acked Read 2 Sent Done Done false
16893 5469 Acked Write 10 Sent Done Done false
16894 5470 Acked Read 1 Sent Done Done false
16895 5471 Acked Write 8 Sent Done Done false
16896 5472 Acked Read 5 Sent Done Done false
16897 5473 Acked Read 5 Sent Done Done false
16898 5474 Acked Write 1 Sent Done Done false
16899 5475 Acked Read 10 Sent Done Done false
16900 5476 Acked Write 10 Sent Done Done false
16901 5477 Acked Flush 0 Sent Done Done false
16902 5478 Acked Write 4 Sent Done Done false
16903 5479 Acked Read 5 Sent Done Done false
16904 5480 Acked Read 9 Sent Done Done false
16905 5481 Acked Write 3 Sent Done Done false
16906 5482 Acked Read 2 Sent Done Done false
16907 5483 Acked Write 2 Sent Done Done false
16908 5484 Acked Read 3 Sent Done Done false
16909 5485 Acked Read 9 Sent Done Done false
16910 5486 Acked Write 10 Sent Done Done false
16911 5487 Acked Read 8 Sent Done Done false
16912 5488 Acked Read 1 Sent Done Done false
16913 5489 Acked Read 1 Sent Done Done false
16914 5490 Acked Write 1 Sent Done Done false
16915 5491 Acked Read 2 Sent Done Done false
16916 5492 Acked Write 5 Sent Done Done false
16917 5493 Acked Write 10 Sent Done Done false
16918 5494 Acked Write 8 Sent Done Done false
16919 5495 Acked Write 3 Sent Done Done false
16920 5496 Acked Read 3 Sent Done Done false
16921 5497 Acked Write 4 Sent Done Done false
16922 5498 Acked Read 5 Sent Done Done false
16923 5499 Acked Write 5 Sent Done Done false
16924 5500 Acked Write 10 Sent Done Done false
16925 5501 Acked Write 2 Sent Done Done false
16926 5502 Acked Read 2 Sent Done Done false
16927 5503 Acked Read 10 Sent Done Done false
16928 5504 Acked Flush 0 Sent Done Done false
16929 5505 Acked Write 8 Sent Done Done false
16930 5506 Acked Read 7 Sent Done Done false
16931 5507 Acked Read 10 Sent Done Done false
16932 5508 Acked Write 3 Sent Done Done false
16933 5509 Acked Read 5 Sent Done Done false
16934 5510 Acked Read 8 Sent Done Done false
16935 5511 Acked Write 7 Sent Done Done false
16936 5512 Acked Read 8 Sent Done Done false
16937 5513 Acked Write 1 Sent Done Done false
16938 5514 Acked Write 9 Sent Done Done false
16939 5515 Acked Read 3 Sent Done Done false
16940 5516 Acked Read 1 Sent Done Done false
16941 5517 Acked Flush 0 Sent Done Done false
16942 5518 Acked Flush 0 Sent Done Done false
16943 5519 Acked Read 2 Sent Done Done false
16944 5520 Acked Write 2 Sent Done Done false
16945 5521 Acked Read 9 Sent Done Done false
16946 5522 Acked Write 9 Sent Done Done false
16947 5523 Acked Write 1 Sent Done Done false
16948 5524 Acked Write 8 Sent Done Done false
16949 5525 Acked Write 10 Sent Done Done false
16950 5526 Acked Write 1 Sent Done Done false
16951 STATES DS:0 DS:1 DS:2 TOTAL
16952 Sent 301 0 0 301
16953 Done 0 301 301 602
16954 Skipped 0 0 0 0
16955 Error 0 0 0 0
16956 Last Flush: 5225 5518 5518
16957 Downstairs last five completed: 5225 5224 5223 5222 5221
16958 Upstairs last five completed: 5526 5525 5524 5523 5522
16959 CLIENT: Up:0 ds:301 act:2
16960 [8810] got start action
16961 [8810] I am going to restart
16962 Starting downstairs at port 8810
16963 Make output file at "/tmp/dsc/downstairs-8810.txt"
16964 [8810][0] reports Starting
16965 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1215)
16966 [8810][0] reports Running
16967 ----------------------------------------------------------------
16968 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
16969 States: Offline Active Active
16970 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16971 5226 Acked Write 8 Sent Done Done false
16972 5227 Acked Write 10 Sent Done Done false
16973 5228 Acked Read 2 Sent Done Done false
16974 5229 Acked Write 1 Sent Done Done false
16975 5230 Acked Read 10 Sent Done Done false
16976 5231 Acked Write 10 Sent Done Done false
16977 5232 Acked Read 7 Sent Done Done false
16978 5233 Acked Read 2 Sent Done Done false
16979 5234 Acked Read 4 Sent Done Done false
16980 5235 Acked Write 7 Sent Done Done false
16981 5236 Acked Read 8 Sent Done Done false
16982 5237 Acked Read 9 Sent Done Done false
16983 5238 Acked Write 9 Sent Done Done false
16984 5239 Acked Read 7 Sent Done Done false
16985 5240 Acked Read 9 Sent Done Done false
16986 5241 Acked Read 3 Sent Done Done false
16987 5242 Acked Read 6 Sent Done Done false
16988 5243 Acked Read 2 Sent Done Done false
16989 5244 Acked Read 4 Sent Done Done false
16990 5245 Acked Flush 0 Sent Done Done false
16991 5246 Acked Read 9 Sent Done Done false
16992 5247 Acked Write 3 Sent Done Done false
16993 5248 Acked Read 10 Sent Done Done false
16994 5249 Acked Read 9 Sent Done Done false
16995 5250 Acked Write 5 Sent Done Done false
16996 5251 Acked Write 3 Sent Done Done false
16997 5252 Acked Write 6 Sent Done Done false
16998 5253 Acked Write 8 Sent Done Done false
16999 5254 Acked Write 5 Sent Done Done false
17000 5255 Acked Write 6 Sent Done Done false
17001 5256 Acked Write 2 Sent Done Done false
17002 5257 Acked Read 3 Sent Done Done false
17003 5258 Acked Read 2 Sent Done Done false
17004 5259 Acked Flush 0 Sent Done Done false
17005 5260 Acked Flush 0 Sent Done Done false
17006 5261 Acked Write 6 Sent Done Done false
17007 5262 Acked Read 10 Sent Done Done false
17008 5263 Acked Write 6 Sent Done Done false
17009 5264 Acked Write 6 Sent Done Done false
17010 5265 Acked Read 6 Sent Done Done false
17011 5266 Acked Flush 0 Sent Done Done false
17012 5267 Acked Write 8 Sent Done Done false
17013 5268 Acked Read 5 Sent Done Done false
17014 5269 Acked Read 5 Sent Done Done false
17015 5270 Acked Read 9 Sent Done Done false
17016 5271 Acked Read 7 Sent Done Done false
17017 5272 Acked Write 8 Sent Done Done false
17018 5273 Acked Write 3 Sent Done Done false
17019 5274 Acked Read 8 Sent Done Done false
17020 5275 Acked Write 1 Sent Done Done false
17021 5276 Acked Write 1 Sent Done Done false
17022 5277 Acked Write 10 Sent Done Done false
17023 5278 Acked Flush 0 Sent Done Done false
17024 5279 Acked Read 1 Sent Done Done false
17025 5280 Acked Write 2 Sent Done Done false
17026 5281 Acked Write 5 Sent Done Done false
17027 5282 Acked Read 3 Sent Done Done false
17028 5283 Acked Read 2 Sent Done Done false
17029 5284 Acked Write 7 Sent Done Done false
17030 5285 Acked Read 9 Sent Done Done false
17031 5286 Acked Read 7 Sent Done Done false
17032 5287 Acked Write 6 Sent Done Done false
17033 5288 Acked Write 9 Sent Done Done false
17034 5289 Acked Write 10 Sent Done Done false
17035 5290 Acked Read 2 Sent Done Done false
17036 5291 Acked Flush 0 Sent Done Done false
17037 5292 Acked Read 8 Sent Done Done false
17038 5293 Acked Read 9 Sent Done Done false
17039 5294 Acked Read 7 Sent Done Done false
17040 5295 Acked Read 3 Sent Done Done false
17041 5296 Acked Read 2 Sent Done Done false
17042 5297 Acked Write 5 Sent Done Done false
17043 5298 Acked Read 2 Sent Done Done false
17044 5299 Acked Read 10 Sent Done Done false
17045 5300 Acked Write 2 Sent Done Done false
17046 5301 Acked Write 8 Sent Done Done false
17047 5302 Acked Read 6 Sent Done Done false
17048 5303 Acked Read 9 Sent Done Done false
17049 5304 Acked Read 7 Sent Done Done false
17050 5305 Acked Write 2 Sent Done Done false
17051 5306 Acked Write 7 Sent Done Done false
17052 5307 Acked Write 7 Sent Done Done false
17053 5308 Acked Flush 0 Sent Done Done false
17054 5309 Acked Read 4 Sent Done Done false
17055 5310 Acked Read 5 Sent Done Done false
17056 5311 Acked Read 5 Sent Done Done false
17057 5312 Acked Read 6 Sent Done Done false
17058 5313 Acked Flush 0 Sent Done Done false
17059 5314 Acked Write 1 Sent Done Done false
17060 5315 Acked Write 1 Sent Done Done false
17061 5316 Acked Flush 0 Sent Done Done false
17062 5317 Acked Read 8 Sent Done Done false
17063 5318 Acked Read 6 Sent Done Done false
17064 5319 Acked Write 3 Sent Done Done false
17065 5320 Acked Flush 0 Sent Done Done false
17066 5321 Acked Write 2 Sent Done Done false
17067 5322 Acked Read 1 Sent Done Done false
17068 5323 Acked Write 10 Sent Done Done false
17069 5324 Acked Read 4 Sent Done Done false
17070 5325 Acked Write 8 Sent Done Done false
17071 5326 Acked Write 8 Sent Done Done false
17072 5327 Acked Write 7 Sent Done Done false
17073 5328 Acked Write 6 Sent Done Done false
17074 5329 Acked Read 8 Sent Done Done false
17075 5330 Acked Flush 0 Sent Done Done false
17076 5331 Acked Flush 0 Sent Done Done false
17077 5332 Acked Write 6 Sent Done Done false
17078 5333 Acked Read 1 Sent Done Done false
17079 5334 Acked Write 4 Sent Done Done false
17080 5335 Acked Read 3 Sent Done Done false
17081 5336 Acked Read 9 Sent Done Done false
17082 5337 Acked Read 10 Sent Done Done false
17083 5338 Acked Write 1 Sent Done Done false
17084 5339 Acked Read 6 Sent Done Done false
17085 5340 Acked Write 7 Sent Done Done false
17086 5341 Acked Write 9 Sent Done Done false
17087 5342 Acked Read 6 Sent Done Done false
17088 5343 Acked Flush 0 Sent Done Done false
17089 5344 Acked Read 7 Sent Done Done false
17090 5345 Acked Read 6 Sent Done Done false
17091 5346 Acked Write 9 Sent Done Done false
17092 5347 Acked Read 8 Sent Done Done false
17093 5348 Acked Read 5 Sent Done Done false
17094 5349 Acked Read 10 Sent Done Done false
17095 5350 Acked Read 6 Sent Done Done false
17096 5351 Acked Write 1 Sent Done Done false
17097 5352 Acked Read 9 Sent Done Done false
17098 5353 Acked Read 10 Sent Done Done false
17099 5354 Acked Flush 0 Sent Done Done false
17100 5355 Acked Write 10 Sent Done Done false
17101 5356 Acked Read 3 Sent Done Done false
17102 5357 Acked Read 7 Sent Done Done false
17103 5358 Acked Read 5 Sent Done Done false
17104 5359 Acked Read 7 Sent Done Done false
17105 5360 Acked Write 6 Sent Done Done false
17106 5361 Acked Write 10 Sent Done Done false
17107 5362 Acked Flush 0 Sent Done Done false
17108 5363 Acked Read 1 Sent Done Done false
17109 5364 Acked Read 2 Sent Done Done false
17110 5365 Acked Write 3 Sent Done Done false
17111 5366 Acked Read 5 Sent Done Done false
17112 5367 Acked Read 4 Sent Done Done false
17113 5368 Acked Write 7 Sent Done Done false
17114 5369 Acked Read 6 Sent Done Done false
17115 5370 Acked Read 1 Sent Done Done false
17116 5371 Acked Read 6 Sent Done Done false
17117 5372 Acked Write 8 Sent Done Done false
17118 5373 Acked Write 7 Sent Done Done false
17119 5374 Acked Write 9 Sent Done Done false
17120 5375 Acked Flush 0 Sent Done Done false
17121 5376 Acked Flush 0 Sent Done Done false
17122 5377 Acked Write 9 Sent Done Done false
17123 5378 Acked Write 2 Sent Done Done false
17124 5379 Acked Read 5 Sent Done Done false
17125 5380 Acked Write 4 Sent Done Done false
17126 5381 Acked Read 6 Sent Done Done false
17127 5382 Acked Read 9 Sent Done Done false
17128 5383 Acked Write 9 Sent Done Done false
17129 5384 Acked Read 2 Sent Done Done false
17130 5385 Acked Write 3 Sent Done Done false
17131 5386 Acked Read 3 Sent Done Done false
17132 5387 Acked Write 2 Sent Done Done false
17133 5388 Acked Write 3 Sent Done Done false
17134 5389 Acked Read 3 Sent Done Done false
17135 5390 Acked Read 7 Sent Done Done false
17136 5391 Acked Read 7 Sent Done Done false
17137 5392 Acked Write 2 Sent Done Done false
17138 5393 Acked Write 3 Sent Done Done false
17139 5394 Acked Write 4 Sent Done Done false
17140 5395 Acked Read 7 Sent Done Done false
17141 5396 Acked Write 7 Sent Done Done false
17142 5397 Acked Write 10 Sent Done Done false
17143 5398 Acked Flush 0 Sent Done Done false
17144 5399 Acked Write 10 Sent Done Done false
17145 5400 Acked Read 2 Sent Done Done false
17146 5401 Acked Write 10 Sent Done Done false
17147 5402 Acked Flush 0 Sent Done Done false
17148 5403 Acked Write 2 Sent Done Done false
17149 5404 Acked Read 3 Sent Done Done false
17150 5405 Acked Write 5 Sent Done Done false
17151 5406 Acked Write 8 Sent Done Done false
17152 5407 Acked Read 5 Sent Done Done false
17153 5408 Acked Flush 0 Sent Done Done false
17154 5409 Acked Flush 0 Sent Done Done false
17155 5410 Acked Read 2 Sent Done Done false
17156 5411 Acked Write 10 Sent Done Done false
17157 5412 Acked Write 8 Sent Done Done false
17158 5413 Acked Write 1 Sent Done Done false
17159 5414 Acked Flush 0 Sent Done Done false
17160 5415 Acked Write 7 Sent Done Done false
17161 5416 Acked Read 9 Sent Done Done false
17162 5417 Acked Flush 0 Sent Done Done false
17163 5418 Acked Read 7 Sent Done Done false
17164 5419 Acked Read 2 Sent Done Done false
17165 5420 Acked Write 7 Sent Done Done false
17166 5421 Acked Read 7 Sent Done Done false
17167 5422 Acked Read 7 Sent Done Done false
17168 5423 Acked Read 3 Sent Done Done false
17169 5424 Acked Read 3 Sent Done Done false
17170 5425 Acked Write 5 Sent Done Done false
17171 5426 Acked Write 7 Sent Done Done false
17172 5427 Acked Flush 0 Sent Done Done false
17173 5428 Acked Flush 0 Sent Done Done false
17174 5429 Acked Read 9 Sent Done Done false
17175 5430 Acked Read 3 Sent Done Done false
17176 5431 Acked Flush 0 Sent Done Done false
17177 5432 Acked Read 4 Sent Done Done false
17178 5433 Acked Write 10 Sent Done Done false
17179 5434 Acked Write 1 Sent Done Done false
17180 5435 Acked Write 8 Sent Done Done false
17181 5436 Acked Write 4 Sent Done Done false
17182 5437 Acked Read 8 Sent Done Done false
17183 5438 Acked Write 8 Sent Done Done false
17184 5439 Acked Read 6 Sent Done Done false
17185 5440 Acked Read 10 Sent Done Done false
17186 5441 Acked Read 5 Sent Done Done false
17187 5442 Acked Write 3 Sent Done Done false
17188 5443 Acked Read 4 Sent Done Done false
17189 5444 Acked Write 7 Sent Done Done false
17190 5445 Acked Write 7 Sent Done Done false
17191 5446 Acked Write 7 Sent Done Done false
17192 5447 Acked Write 9 Sent Done Done false
17193 5448 Acked Write 10 Sent Done Done false
17194 5449 Acked Read 7 Sent Done Done false
17195 5450 Acked Read 9 Sent Done Done false
17196 5451 Acked Read 9 Sent Done Done false
17197 5452 Acked Write 4 Sent Done Done false
17198 5453 Acked Flush 0 Sent Done Done false
17199 5454 Acked Flush 0 Sent Done Done false
17200 5455 Acked Write 4 Sent Done Done false
17201 5456 Acked Read 2 Sent Done Done false
17202 5457 Acked Read 5 Sent Done Done false
17203 5458 Acked Write 5 Sent Done Done false
17204 5459 Acked Read 9 Sent Done Done false
17205 5460 Acked Write 6 Sent Done Done false
17206 5461 Acked Write 6 Sent Done Done false
17207 5462 Acked Flush 0 Sent Done Done false
17208 5463 Acked Read 3 Sent Done Done false
17209 5464 Acked Read 6 Sent Done Done false
17210 5465 Acked Write 10 Sent Done Done false
17211 5466 Acked Flush 0 Sent Done Done false
17212 5467 Acked Read 8 Sent Done Done false
17213 5468 Acked Read 2 Sent Done Done false
17214 5469 Acked Write 10 Sent Done Done false
17215 5470 Acked Read 1 Sent Done Done false
17216 5471 Acked Write 8 Sent Done Done false
17217 5472 Acked Read 5 Sent Done Done false
17218 5473 Acked Read 5 Sent Done Done false
17219 5474 Acked Write 1 Sent Done Done false
17220 5475 Acked Read 10 Sent Done Done false
17221 5476 Acked Write 10 Sent Done Done false
17222 5477 Acked Flush 0 Sent Done Done false
17223 5478 Acked Write 4 Sent Done Done false
17224 5479 Acked Read 5 Sent Done Done false
17225 5480 Acked Read 9 Sent Done Done false
17226 5481 Acked Write 3 Sent Done Done false
17227 5482 Acked Read 2 Sent Done Done false
17228 5483 Acked Write 2 Sent Done Done false
17229 5484 Acked Read 3 Sent Done Done false
17230 5485 Acked Read 9 Sent Done Done false
17231 5486 Acked Write 10 Sent Done Done false
17232 5487 Acked Read 8 Sent Done Done false
17233 5488 Acked Read 1 Sent Done Done false
17234 5489 Acked Read 1 Sent Done Done false
17235 5490 Acked Write 1 Sent Done Done false
17236 5491 Acked Read 2 Sent Done Done false
17237 5492 Acked Write 5 Sent Done Done false
17238 5493 Acked Write 10 Sent Done Done false
17239 5494 Acked Write 8 Sent Done Done false
17240 5495 Acked Write 3 Sent Done Done false
17241 5496 Acked Read 3 Sent Done Done false
17242 5497 Acked Write 4 Sent Done Done false
17243 5498 Acked Read 5 Sent Done Done false
17244 5499 Acked Write 5 Sent Done Done false
17245 5500 Acked Write 10 Sent Done Done false
17246 5501 Acked Write 2 Sent Done Done false
17247 5502 Acked Read 2 Sent Done Done false
17248 5503 Acked Read 10 Sent Done Done false
17249 5504 Acked Flush 0 Sent Done Done false
17250 5505 Acked Write 8 Sent Done Done false
17251 5506 Acked Read 7 Sent Done Done false
17252 5507 Acked Read 10 Sent Done Done false
17253 5508 Acked Write 3 Sent Done Done false
17254 5509 Acked Read 5 Sent Done Done false
17255 5510 Acked Read 8 Sent Done Done false
17256 5511 Acked Write 7 Sent Done Done false
17257 5512 Acked Read 8 Sent Done Done false
17258 5513 Acked Write 1 Sent Done Done false
17259 5514 Acked Write 9 Sent Done Done false
17260 5515 Acked Read 3 Sent Done Done false
17261 5516 Acked Read 1 Sent Done Done false
17262 5517 Acked Flush 0 Sent Done Done false
17263 5518 Acked Flush 0 Sent Done Done false
17264 5519 Acked Read 2 Sent Done Done false
17265 5520 Acked Write 2 Sent Done Done false
17266 5521 Acked Read 9 Sent Done Done false
17267 5522 Acked Write 9 Sent Done Done false
17268 5523 Acked Write 1 Sent Done Done false
17269 5524 Acked Write 8 Sent Done Done false
17270 5525 Acked Write 10 Sent Done Done false
17271 5526 Acked Write 1 Sent Done Done false
17272 5527 Acked Flush 0 Sent Done Done false
17273 STATES DS:0 DS:1 DS:2 TOTAL
17274 Sent 302 0 0 302
17275 Done 0 302 302 604
17276 Skipped 0 0 0 0
17277 Error 0 0 0 0
17278 Last Flush: 5225 5527 5527
17279 Downstairs last five completed: 5225 5224 5223 5222 5221
17280 Upstairs last five completed: 5527 5526 5525 5524 5523
17281 CLIENT: Up:0 ds:302 act:2
172822025-01-17T21:14:41.877ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
172832025-01-17T21:14:41.877ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39190) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
172842025-01-17T21:14:41.878ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
172852025-01-17T21:14:41.879ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
172862025-01-17T21:14:41.879ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
172872025-01-17T21:14:41.879ZINFOcrucible: send last flush ID to this DS: 5225 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
172882025-01-17T21:14:41.879ZINFOcrucible: Replied this last flush ID: 5225 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
172892025-01-17T21:14:41.879ZINFOcrucible: [0] client re-new 302 jobs since flush 5225 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
172902025-01-17T21:14:41.879ZINFOcrucible: [0] Replayed 302 jobs since flush: 5225 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
17291 ----------------------------------------------------------------
17292 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
17293 Downstairs last five completed: 5527 5526 5525 5524 5523
17294 Upstairs last five completed: 5527 5526 5525 5524 5523
17295 CLIENT: Up:0 ds:0 act:3
17296 Replay: All jobs finished
17297 Main task has work to do, go find it
17298 got dsc Stop(2)
17299 stop 2
17300 Jan 17 21:14:44.123 INFO request completed[8830] Got stop action so:false kr:false
17301 , latency_us: 360, response_code: 204, uri: /stop/cid/2, method: POST, req_id: c4166ef8-8883-4921-8cae-91746ad9b852, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
173022025-01-17T21:14:44.124ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
17303 Jan 17 21:14:44.124 INFO request completed, latency_us: 99, response_code: 200, uri: /state/cid/2, method: GET, req_id: dca509bf-727a-4985-a588-a203398fca99, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
173042025-01-17T21:14:44.124ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
173052025-01-17T21:14:44.124ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
173062025-01-17T21:14:44.124ZINFOcrucible: client task is exiting = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
17307 [8830] Exited with: ExitStatus(unix_wait_status(9))
17308 [8830][2] reports Exit
17309 Jan 17 21:14:48.126 INFO request completed, latency_us: 193, response_code: 200, uri: /state/cid/2, method: GET, req_id: d4926f77-e922-486c-bc05-7b7d7bb30382, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
17310 001/300 Read block 4615 len 8192
17311 002/300 Write block 4677 len 32768 data: 7 7 7 6 4 3 3 3
17312 003/300 Write block 1477 len 40960 data: 3 3 3 3 4 4 4 5 5 7
17313 004/300 Write block 4572 len 32768 data: 2 2 2 3 5 4 4 4
17314 005/300 Read block 3319 len 40960
17315 006/300 Read block 4907 len 4096
17316 007/300 Read block 4646 len 8192
17317 008/300 Write block 4579 len 20480 data: 5 4 5 3 3
17318 009/300 Write block 728 len 28672 data: 3 4 4 3 3 4 5
17319 010/300 Read block 2601 len 24576
17320 011/300 Read block 2220 len 24576
17321 012/300 Read block 885 len 20480
17322 013/300 Flush
17323 014/300 Write block 1466 len 36864 data: 5 6 6 5 5 4 5 4 4
17324 015/300 Write block 2219 len 24576 data: 3 3 4 4 4 4
17325 016/300 Read block 4855 len 24576
17326 017/300 Flush
17327 018/300 Flush
17328 019/300 Read block 414 len 20480
17329 020/300 Read block 4720 len 40960
17330 021/300 Write block 4119 len 20480 data: 5 4 5 5 4
17331 022/300 Read block 1374 len 12288
17332 023/300 Write block 4298 len 16384 data: 6 6 6 6
17333 024/300 Read block 2738 len 28672
17334 025/300 Write block 2851 len 4096 data: 2
17335 026/300 Write block 2847 len 20480 data: 2 2 2 2 3
17336 027/300 Read block 136 len 28672
17337 028/300 Read block 2246 len 16384
17338 029/300 Read block 777 len 4096
17339 030/300 Read block 710 len 20480
17340 031/300 Read block 2368 len 4096
17341 032/300 Read block 4090 len 28672
17342 033/300 Write block 4531 len 40960 data: 5 6 5 5 5 5 4 4 3 2
17343 034/300 Read block 2360 len 8192
17344 035/300 Read block 1186 len 40960
17345 036/300 Read block 3726 len 20480
17346 037/300 Write block 3054 len 12288 data: 3 3 3
17347 038/300 Flush
17348 039/300 Read block 2062 len 24576
17349 040/300 Flush
17350 041/300 Read block 1391 len 8192
17351 042/300 Read block 4304 len 24576
17352 043/300 Read block 3545 len 32768
17353 044/300 Read block 4280 len 20480
17354 045/300 Write block 4168 len 16384 data: 2 2 5 5
17355 046/300 Write block 803 len 40960 data: 4 5 4 4 4 4 3 3 3 5
17356 047/300 Read block 660 len 16384
17357 048/300 Read block 3940 len 20480
17358 049/300 Write block 284 len 32768 data: 5 5 4 4 4 4 2 2
17359 050/300 Read block 3558 len 8192
17360 051/300 Write block 4178 len 32768 data: 8 6 4 4 4 4 4 6
17361 052/300 Write block 4421 len 16384 data: 5 4 5 7
17362 053/300 Write block 2594 len 32768 data: 3 3 3 3 3 4 4 3
17363 054/300 Write block 182 len 4096 data: 4
17364 055/300 Read block 4626 len 8192
17365 056/300 Write block 1472 len 20480 data: 6 5 5 4 2
17366 057/300 Read block 2795 len 12288
17367 058/300 Read block 2177 len 16384
17368 059/300 Read block 2390 len 20480
17369 060/300 Read block 4906 len 28672
17370 061/300 Write block 2109 len 16384 data: 8 6 6 6
17371 062/300 Write block 4711 len 12288 data: 3 3 2
17372 063/300 Read block 3095 len 32768
17373 064/300 Write block 261 len 40960 data: 3 3 3 2 3 3 2 2 4 6
17374 065/300 Read block 1203 len 20480
17375 066/300 Read block 3958 len 4096
17376 067/300 Read block 2801 len 28672
17377 068/300 Read block 4817 len 16384
17378 069/300 Read block 412 len 4096
17379 070/300 Read block 1070 len 40960
17380 071/300 Read block 4005 len 40960
17381 072/300 Read block 788 len 28672
17382 073/300 Flush
17383 074/300 Read block 4660 len 12288
17384 075/300 Write block 2330 len 40960 data: 6 6 7 7 6 6 5 5 4 3
17385 076/300 Flush
17386 077/300 Write block 2195 len 36864 data: 3 3 3 3 2 2 2 2 2
17387 078/300 Write block 4815 len 20480 data: 5 4 4 4 3
17388 079/300 Read block 1342 len 32768
17389 080/300 Read block 1602 len 36864
17390 081/300 Read block 4526 len 16384
17391 082/300 Write block 2090 len 36864 data: 5 4 4 3 3 2 3 4 5
17392 083/300 Read block 2647 len 28672
17393 084/300 Read block 190 len 8192
17394 085/300 Write block 254 len 24576 data: 3 3 2 2 2 2
17395 086/300 Read block 1952 len 8192
17396 087/300 Read block 483 len 8192
17397 088/300 Read block 1016 len 16384
17398 089/300 Read block 315 len 40960
17399 090/300 Write block 3633 len 32768 data: 3 3 4 4 3 2 2 2
17400 091/300 Read block 72 len 40960
17401 092/300 Write block 4952 len 8192 data: 4 3
17402 093/300 Write block 3303 len 32768 data: 5 4 2 2 2 2 2 2
17403 094/300 Flush
17404 095/300 Read block 426 len 36864
17405 096/300 Flush
17406 097/300 Write block 1855 len 40960 data: 3 3 3 3 4 3 5 4 4 5
17407 098/300 Read block 50 len 20480
17408 099/300 Read block 4615 len 8192
17409 100/300 Read block 3481 len 24576
17410 101/300 Read block 2940 len 16384
17411 102/300 Read block 4409 len 8192
17412 103/300 Read block 841 len 40960
17413 104/300 Write block 3610 len 16384 data: 6 7 8 5
17414 105/300 Write block 4588 len 20480 data: 3 3 2 2 2
17415 106/300 Flush
17416 107/300 Write block 1977 len 12288 data: 4 5 4
17417 108/300 Write block 4806 len 16384 data: 4 4 5 4
17418 109/300 Read block 525 len 28672
17419 110/300 Read block 745 len 20480
17420 111/300 Write block 4054 len 16384 data: 3 4 4 5
17421 112/300 Write block 973 len 28672 data: 3 3 4 4 3 3 2
17422 113/300 Read block 466 len 40960
17423 114/300 Write block 2952 len 36864 data: 2 2 2 2 2 3 3 2 2
17424 115/300 Read block 3275 len 40960
17425 116/300 Write block 1118 len 32768 data: 6 5 5 4 4 3 3 3
17426 117/300 Read block 4489 len 24576
17427 118/300 Read block 2221 len 20480
17428 119/300 Write block 2289 len 40960 data: 4 4 5 5 4 5 5 5 4 4
17429 120/300 Read block 3949 len 32768
17430 121/300 Flush
17431 122/300 Write block 3824 len 4096 data: 3
17432 123/300 Read block 4004 len 4096
17433 124/300 Write block 3805 len 8192 data: 3 3
17434 125/300 Write block 4754 len 32768 data: 5 6 7 7 6 8 8 7
17435 126/300 Read block 2397 len 12288
17436 127/300 Write block 4215 len 16384 data: 4 5 6 4
17437 128/300 Write block 873 len 4096 data: 4
17438 129/300 Write block 3736 len 4096 data: 3
17439 130/300 Flush
17440 131/300 Write block 4990 len 16384 data: 2 2 3 4
17441 132/300 Flush
17442 133/300 Write block 2255 len 40960 data: 3 4 4 5 3 4 4 4 4 7
17443 134/300 Read block 1062 len 20480
17444 135/300 Read block 1688 len 8192
17445 136/300 Write block 4027 len 12288 data: 3 5 5
17446 137/300 Read block 138 len 40960
17447 138/300 Read block 338 len 16384
17448 139/300 Read block 3802 len 12288
17449 140/300 Read block 3084 len 20480
17450 141/300 Flush
17451 142/300 Read block 2168 len 16384
17452 143/300 Read block 874 len 4096
17453 144/300 Flush
17454 145/300 Write block 1871 len 8192 data: 2 2
17455 146/300 Read block 1902 len 36864
17456 147/300 Write block 628 len 16384 data: 3 4 5 5
17457 148/300 Flush
17458 149/300 Write block 4102 len 12288 data: 4 4 4
17459 150/300 Read block 4293 len 24576
17460 151/300 Read block 1470 len 20480
17461 152/300 Read block 3269 len 40960
17462 153/300 Read block 739 len 4096
17463 154/300 Read block 253 len 40960
17464 155/300 Flush
17465 156/300 Write block 1312 len 12288 data: 3 4 4
17466 157/300 Read block 4854 len 40960
17467 158/300 Read block 1852 len 28672
17468 159/300 Write block 2763 len 36864 data: 2 2 2 2 3 3 3 3 3
17469 160/300 Flush
17470 161/300 Read block 968 len 8192
17471 162/300 Write block 3989 len 8192 data: 3 5
17472 163/300 Read block 2310 len 32768
17473 164/300 Write block 1029 len 32768 data: 6 6 4 4 4 4 6 6
17474 165/300 Read block 2945 len 40960
17475 166/300 Write block 32 len 20480 data: 5 3 4 2 3
17476 167/300 Write block 3426 len 16384 data: 3 3 2 2
17477 168/300 Write block 1007 len 32768 data: 3 4 4 4 4 3 3 3
17478 169/300 Read block 4821 len 36864
17479 170/300 Write block 4185 len 28672 data: 7 6 6 6 6 6 6
17480 171/300 Write block 4579 len 16384 data: 6 5 6 4
17481 172/300 Write block 2670 len 36864 data: 3 3 3 3 4 4 4 4 2
17482 173/300 Write block 396 len 32768 data: 3 4 6 6 6 6 6 5
17483 174/300 Write block 31 len 20480 data: 6 6 4 5 3
17484 175/300 Write block 1048 len 36864 data: 3 3 3 4 4 4 5 6 7
17485 176/300 Read block 4899 len 24576
17486 177/300 Write block 2699 len 4096 data: 2
17487 178/300 Read block 4883 len 16384
17488 179/300 Read block 4685 len 28672
17489 180/300 Write block 256 len 20480 data: 3 3 3 3 3
17490 181/300 Write block 4927 len 4096 data: 4
17491 182/300 Read block 4486 len 40960
17492 183/300 Flush
17493 184/300 Read block 563 len 20480
17494 185/300 Write block 972 len 8192 data: 5 4
17495 186/300 Read block 2599 len 16384
17496 187/300 Read block 287 len 8192
17497 188/300 Read block 3467 len 4096
17498 189/300 Read block 62 len 36864
17499 190/300 Write block 2075 len 20480 data: 4 3 3 3 3
17500 191/300 Write block 525 len 40960 data: 4 3 3 3 4 5 6 5 4 4
17501 192/300 Read block 3126 len 36864
17502 193/300 Write block 4701 len 4096 data: 4
17503 194/300 Read block 831 len 12288
17504 195/300 Read block 375 len 20480
17505 196/300 Read block 2149 len 40960
17506 197/300 Write block 4468 len 28672 data: 4 3 5 6 7 6 7
17507 198/300 Read block 3791 len 4096
17508 199/300 Flush
17509 200/300 Write block 190 len 36864 data: 4 3 3 3 4 7 7 7 5
17510 201/300 Write block 603 len 40960 data: 3 3 3 3 3 4 4 2 3 5
17511 202/300 Read block 3251 len 4096
17512 203/300 Write block 3310 len 24576 data: 3 2 2 4 5 5
17513 204/300 Write block 4883 len 24576 data: 3 3 4 3 4 4
17514 205/300 Write block 2339 len 32768 data: 4 3 3 4 4 5 6 7
17515 206/300 Flush
17516 207/300 Read block 3497 len 40960
17517 208/300 Read block 3329 len 20480
17518 209/300 Read block 4797 len 40960
17519 210/300 Write block 2661 len 16384 data: 3 4 5 5
17520 211/300 Read block 1240 len 28672
17521 212/300 Read block 3452 len 32768
17522 213/300 Read block 3752 len 8192
17523 214/300 Read block 3856 len 12288
17524 215/300 Read block 4561 len 8192
17525 216/300 Write block 3701 len 32768 data: 4 4 5 6 5 5 7 7
17526 217/300 Read block 4405 len 4096
17527 218/300 Read block 473 len 32768
17528 219/300 Read block 2583 len 20480
17529 220/300 Read block 3789 len 4096
17530 221/300 Read block 3136 len 12288
17531 222/300 Write block 2122 len 36864 data: 3 4 4 3 3 3 2 2 3
17532 223/300 Read block 1063 len 32768
17533 224/300 Read block 2661 len 16384
17534 225/300 Write block 1487 len 8192 data: 5 4
17535 226/300 Read block 3416 len 40960
17536 227/300 Read block 849 len 20480
17537 228/300 Read block 2243 len 40960
17538 229/300 Read block 4398 len 24576
17539 230/300 Write block 3149 len 12288 data: 4 4 4
17540 231/300 Read block 2524 len 36864
17541 232/300 Flush
17542 233/300 Write block 1725 len 28672 data: 4 4 4 3 3 4 5
17543 234/300 Read block 3090 len 32768
17544 235/300 Write block 3078 len 40960 data: 5 6 6 6 7 6 4 5 6 6
17545 236/300 Read block 2701 len 40960
17546 237/300 Write block 2040 len 16384 data: 3 3 2 2
17547 238/300 Write block 2547 len 24576 data: 5 7 5 6 6 5
17548 239/300 Read block 3807 len 24576
17549 240/300 Write block 2510 len 32768 data: 4 4 5 5 6 6 7 7
17550 241/300 Read block 125 len 28672
17551 242/300 Write block 4123 len 40960 data: 5 3 2 2 3 3 3 3 4 4
17552 243/300 Flush
17553 244/300 Read block 1458 len 12288
17554 245/300 Read block 3371 len 8192
17555 246/300 Read block 2048 len 32768
17556 247/300 Read block 4449 len 8192
17557 248/300 Write block 4712 len 24576 data: 4 3 4 4 4 5
17558 249/300 Read block 946 len 16384
17559 250/300 Read block 4282 len 24576
17560 251/300 Write block 12 len 8192 data: 4 4
17561 252/300 Read block 550 len 28672
17562 253/300 Read block 4369 len 20480
17563 254/300 Read block 643 len 12288
17564 255/300 Read block 2152 len 32768
17565 256/300 Flush
17566 257/300 Read block 897 len 40960
17567 258/300 Read block 2694 len 32768
17568 259/300 Read block 4322 len 12288
17569 260/300 Read block 2521 len 4096
17570 261/300 Read block 3302 len 20480
17571 262/300 Write block 67 len 40960 data: 4 5 5 5 5 6 6 6 7 7
17572 263/300 Write block 4703 len 24576 data: 5 6 5 5 2 2
17573 264/300 Write block 4369 len 36864 data: 5 4 5 3 4 3 3 4 4
17574 265/300 Read block 1309 len 36864
17575 266/300 Flush
17576 267/300 Read block 2256 len 20480
17577 268/300 Read block 1788 len 12288
17578 269/300 Read block 1165 len 28672
17579 270/300 Read block 2769 len 4096
17580 271/300 Read block 1347 len 28672
17581 272/300 Write block 3795 len 40960 data: 3 3 3 4 4 4 4 4 4 3
17582 273/300 Write block 4475 len 40960 data: 7 6 5 4 3 3 4 4 5 4
17583 274/300 Write block 4344 len 24576 data: 4 3 3 4 4 4
17584 275/300 Write block 3565 len 8192 data: 2 2
17585 276/300 Read block 4062 len 20480
17586 277/300 Read block 1266 len 24576
17587 278/300 Read block 2050 len 28672
17588 279/300 Read block 261 len 12288
17589 280/300 Write block 3102 len 20480 data: 5 6 8 8 6
17590 281/300 Read block 356 len 8192
17591 282/300 Write block 4746 len 8192 data: 4 4
17592 283/300 Flush
17593 284/300 Write block 4465 len 40960 data: 4 4 3 5 4 6 7 8 7 8
17594 285/300 Read block 4139 len 12288
17595 286/300 Write block 2743 len 16384 data: 3 3 3 3
17596 287/300 Write block 2915 len 20480 data: 4 6 6 6 5
17597 288/300 Flush
17598 289/300 Flush
17599 290/300 Write block 214 len 8192 data: 3 4
17600 291/300 Flush
17601 292/300 Write block 2581 len 12288 data: 2 2 4
17602 293/300 Read block 240 len 32768
17603 294/300 Write block 3681 len 4096 data: 5
17604 295/300 Write block 2069 len 32768 data: 7 5 4 5 4 4 5 4
17605 296/300 Flush
17606 297/300 Write block 1361 len 4096 data: 5
17607 298/300 Write block 4078 len 20480 data: 4 4 4 3 3
17608 299/300 Flush
17609 300/300 Read block 782 len 24576
17610 301/300 Write block 3785 len 32768 data: 5 5 5 5 4 4 3 3
17611 Main task has work to do, go find it
17612 got dsc Start(2)
17613 start 2
17614 Jan 17 21:14:48.383 INFO request completed, latency_us: 272, response_code: 204, uri: /start/cid/2, method: POST, req_id: 83568cf7-6649-4a66-96a4-8dbd61276686, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
17615 Replay: started 2, returned:Ok(())
17616 ----------------------------------------------------------------
17617 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
17618 States: Active Active Offline
17619 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17620 5528 Acked Read 2 Done Done Sent false
17621 5529 Acked Write 8 Done Done Sent false
17622 5530 Acked Write 10 Done Done Sent false
17623 5531 Acked Write 8 Done Done Sent false
17624 5532 Acked Read 10 Done Done Sent false
17625 5533 Acked Read 1 Done Done Sent false
17626 5534 Acked Read 2 Done Done Sent false
17627 5535 Acked Write 5 Done Done Sent false
17628 5536 Acked Write 7 Done Done Sent false
17629 5537 Acked Read 6 Done Done Sent false
17630 5538 Acked Read 6 Done Done Sent false
17631 5539 Acked Read 5 Done Done Sent false
17632 5540 Acked Flush 0 Done Done Sent false
17633 5541 Acked Write 9 Done Done Sent false
17634 5542 Acked Write 6 Done Done Sent false
17635 5543 Acked Read 6 Done Done Sent false
17636 5544 Acked Flush 0 Done Done Sent false
17637 5545 Acked Flush 0 Done Done Sent false
17638 5546 Acked Read 5 Done Done Sent false
17639 5547 Acked Read 10 Done Done Sent false
17640 5548 Acked Write 5 Done Done Sent false
17641 5549 Acked Read 3 Done Done Sent false
17642 5550 Acked Write 4 Done Done Sent false
17643 5551 Acked Read 7 Done Done Sent false
17644 5552 Acked Write 1 Done Done Sent false
17645 5553 Acked Write 5 Done Done Sent false
17646 5554 Acked Read 7 Done Done Sent false
17647 5555 Acked Read 4 Done Done Sent false
17648 5556 Acked Read 1 Done Done Sent false
17649 5557 Acked Read 5 Done Done Sent false
17650 5558 Acked Read 1 Done Done Sent false
17651 5559 Acked Read 7 Done Done Sent false
17652 5560 Acked Write 10 Done Done Sent false
17653 5561 Acked Read 2 Done Done Sent false
17654 5562 Acked Read 10 Done Done Sent false
17655 5563 Acked Read 5 Done Done Sent false
17656 5564 Acked Write 3 Done Done Sent false
17657 5565 Acked Flush 0 Done Done Sent false
17658 5566 Acked Read 6 Done Done Sent false
17659 5567 Acked Flush 0 Done Done Sent false
17660 5568 Acked Read 2 Done Done Sent false
17661 5569 Acked Read 6 Done Done Sent false
17662 5570 Acked Read 8 Done Done Sent false
17663 5571 Acked Read 5 Done Done Sent false
17664 5572 Acked Write 4 Done Done Sent false
17665 5573 Acked Write 10 Done Done Sent false
17666 5574 Acked Read 4 Done Done Sent false
17667 5575 Acked Read 5 Done Done Sent false
17668 5576 Acked Write 8 Done Done Sent false
17669 5577 Acked Read 2 Done Done Sent false
17670 5578 Acked Write 8 Done Done Sent false
17671 5579 Acked Write 4 Done Done Sent false
17672 5580 Acked Write 8 Done Done Sent false
17673 5581 Acked Write 1 Done Done Sent false
17674 5582 Acked Read 2 Done Done Sent false
17675 5583 Acked Write 5 Done Done Sent false
17676 5584 Acked Read 3 Done Done Sent false
17677 5585 Acked Read 4 Done Done Sent false
17678 5586 Acked Read 5 Done Done Sent false
17679 5587 Acked Read 7 Done Done Sent false
17680 5588 Acked Write 4 Done Done Sent false
17681 5589 Acked Write 3 Done Done Sent false
17682 5590 Acked Read 8 Done Done Sent false
17683 5591 Acked Write 10 Done Done Sent false
17684 5592 Acked Read 5 Done Done Sent false
17685 5593 Acked Read 1 Done Done Sent false
17686 5594 Acked Read 7 Done Done Sent false
17687 5595 Acked Read 4 Done Done Sent false
17688 5596 Acked Read 1 Done Done Sent false
17689 5597 Acked Read 10 Done Done Sent false
17690 5598 Acked Read 10 Done Done Sent false
17691 5599 Acked Read 7 Done Done Sent false
17692 5600 Acked Flush 0 Done Done Sent false
17693 5601 Acked Read 3 Done Done Sent false
17694 5602 Acked Write 10 Done Done Sent false
17695 5603 Acked Flush 0 Done Done Sent false
17696 5604 Acked Write 9 Done Done Sent false
17697 5605 Acked Write 5 Done Done Sent false
17698 5606 Acked Read 8 Done Done Sent false
17699 5607 Acked Read 9 Done Done Sent false
17700 5608 Acked Read 4 Done Done Sent false
17701 5609 Acked Write 9 Done Done Sent false
17702 5610 Acked Read 7 Done Done Sent false
17703 5611 Acked Read 2 Done Done Sent false
17704 5612 Acked Write 6 Done Done Sent false
17705 5613 Acked Read 2 Done Done Sent false
17706 5614 Acked Read 2 Done Done Sent false
17707 5615 Acked Read 4 Done Done Sent false
17708 5616 Acked Read 10 Done Done Sent false
17709 5617 Acked Write 8 Done Done Sent false
17710 5618 Acked Read 10 Done Done Sent false
17711 5619 Acked Write 2 Done Done Sent false
17712 5620 Acked Write 8 Done Done Sent false
17713 5621 Acked Flush 0 Done Done Sent false
17714 5622 Acked Read 9 Done Done Sent false
17715 5623 Acked Flush 0 Done Done Sent false
17716 5624 Acked Write 10 Done Done Sent false
17717 5625 Acked Read 5 Done Done Sent false
17718 5626 Acked Read 2 Done Done Sent false
17719 5627 Acked Read 6 Done Done Sent false
17720 5628 Acked Read 4 Done Done Sent false
17721 5629 Acked Read 2 Done Done Sent false
17722 5630 Acked Read 10 Done Done Sent false
17723 5631 Acked Write 4 Done Done Sent false
17724 5632 Acked Write 5 Done Done Sent false
17725 5633 Acked Flush 0 Done Done Sent false
17726 5634 Acked Write 3 Done Done Sent false
17727 5635 Acked Write 4 Done Done Sent false
17728 5636 Acked Read 7 Done Done Sent false
17729 5637 Acked Read 5 Done Done Sent false
17730 5638 Acked Write 4 Done Done Sent false
17731 5639 Acked Write 7 Done Done Sent false
17732 5640 Acked Read 10 Done Done Sent false
17733 5641 Acked Write 9 Done Done Sent false
17734 5642 Acked Read 10 Done Done Sent false
17735 5643 Acked Write 8 Done Done Sent false
17736 5644 Acked Read 6 Done Done Sent false
17737 5645 Acked Read 5 Done Done Sent false
17738 5646 Acked Write 10 Done Done Sent false
17739 5647 Acked Read 8 Done Done Sent false
17740 5648 Acked Flush 0 Done Done Sent false
17741 5649 Acked Write 1 Done Done Sent false
17742 5650 Acked Read 1 Done Done Sent false
17743 5651 Acked Write 2 Done Done Sent false
17744 5652 Acked Write 8 Done Done Sent false
17745 5653 Acked Read 3 Done Done Sent false
17746 5654 Acked Write 4 Done Done Sent false
17747 5655 Acked Write 1 Done Done Sent false
17748 5656 Acked Write 1 Done Done Sent false
17749 5657 Acked Flush 0 Done Done Sent false
17750 5658 Acked Write 4 Done Done Sent false
17751 5659 Acked Flush 0 Done Done Sent false
17752 5660 Acked Write 10 Done Done Sent false
17753 5661 Acked Read 5 Done Done Sent false
17754 5662 Acked Read 2 Done Done Sent false
17755 5663 Acked Write 3 Done Done Sent false
17756 5664 Acked Read 10 Done Done Sent false
17757 5665 Acked Read 4 Done Done Sent false
17758 5666 Acked Read 3 Done Done Sent false
17759 5667 Acked Read 5 Done Done Sent false
17760 5668 Acked Flush 0 Done Done Sent false
17761 5669 Acked Read 4 Done Done Sent false
17762 5670 Acked Read 1 Done Done Sent false
17763 5671 Acked Flush 0 Done Done Sent false
17764 5672 Acked Write 2 Done Done Sent false
17765 5673 Acked Read 9 Done Done Sent false
17766 5674 Acked Write 4 Done Done Sent false
17767 5675 Acked Flush 0 Done Done Sent false
17768 5676 Acked Write 3 Done Done Sent false
17769 5677 Acked Read 6 Done Done Sent false
17770 5678 Acked Read 5 Done Done Sent false
17771 5679 Acked Read 10 Done Done Sent false
17772 5680 Acked Read 1 Done Done Sent false
17773 5681 Acked Read 10 Done Done Sent false
17774 5682 Acked Flush 0 Done Done Sent false
17775 5683 Acked Write 3 Done Done Sent false
17776 5684 Acked Read 10 Done Done Sent false
17777 5685 Acked Read 7 Done Done Sent false
17778 5686 Acked Write 9 Done Done Sent false
17779 5687 Acked Flush 0 Done Done Sent false
17780 5688 Acked Read 2 Done Done Sent false
17781 5689 Acked Write 2 Done Done Sent false
17782 5690 Acked Read 8 Done Done Sent false
17783 5691 Acked Write 8 Done Done Sent false
17784 5692 Acked Read 10 Done Done Sent false
17785 5693 Acked Write 5 Done Done Sent false
17786 5694 Acked Write 4 Done Done Sent false
17787 5695 Acked Write 8 Done Done Sent false
17788 5696 Acked Read 9 Done Done Sent false
17789 5697 Acked Write 7 Done Done Sent false
17790 5698 Acked Write 4 Done Done Sent false
17791 5699 Acked Write 9 Done Done Sent false
17792 5700 Acked Write 8 Done Done Sent false
17793 5701 Acked Write 5 Done Done Sent false
17794 5702 Acked Write 9 Done Done Sent false
17795 5703 Acked Read 6 Done Done Sent false
17796 5704 Acked Write 1 Done Done Sent false
17797 5705 Acked Read 4 Done Done Sent false
17798 5706 Acked Read 7 Done Done Sent false
17799 5707 Acked Write 5 Done Done Sent false
17800 5708 Acked Write 1 Done Done Sent false
17801 5709 Acked Read 10 Done Done Sent false
17802 5710 Acked Flush 0 Done Done Sent false
17803 5711 Acked Read 5 Done Done Sent false
17804 5712 Acked Write 2 Done Done Sent false
17805 5713 Acked Read 4 Done Done Sent false
17806 5714 Acked Read 2 Done Done Sent false
17807 5715 Acked Read 1 Done Done Sent false
17808 5716 Acked Read 9 Done Done Sent false
17809 5717 Acked Write 5 Done Done Sent false
17810 5718 Acked Write 10 Done Done Sent false
17811 5719 Acked Read 9 Done Done Sent false
17812 5720 Acked Write 1 Done Done Sent false
17813 5721 Acked Read 3 Done Done Sent false
17814 5722 Acked Read 5 Done Done Sent false
17815 5723 Acked Read 10 Done Done Sent false
17816 5724 Acked Write 7 Done Done Sent false
17817 5725 Acked Read 1 Done Done Sent false
17818 5726 Acked Flush 0 Done Done Sent false
17819 5727 Acked Write 9 Done Done Sent false
17820 5728 Acked Write 10 Done Done Sent false
17821 5729 Acked Read 1 Done Done Sent false
17822 5730 Acked Write 6 Done Done Sent false
17823 5731 Acked Write 6 Done Done Sent false
17824 5732 Acked Write 8 Done Done Sent false
17825 5733 Acked Flush 0 Done Done Sent false
17826 5734 Acked Read 10 Done Done Sent false
17827 5735 Acked Read 5 Done Done Sent false
17828 5736 Acked Read 10 Done Done Sent false
17829 5737 Acked Write 4 Done Done Sent false
17830 5738 Acked Read 7 Done Done Sent false
17831 5739 Acked Read 8 Done Done Sent false
17832 5740 Acked Read 2 Done Done Sent false
17833 5741 Acked Read 3 Done Done Sent false
17834 5742 Acked Read 2 Done Done Sent false
17835 5743 Acked Write 8 Done Done Sent false
17836 5744 Acked Read 1 Done Done Sent false
17837 5745 Acked Read 8 Done Done Sent false
17838 5746 Acked Read 5 Done Done Sent false
17839 5747 Acked Read 1 Done Done Sent false
17840 5748 Acked Read 3 Done Done Sent false
17841 5749 Acked Write 9 Done Done Sent false
17842 5750 Acked Read 8 Done Done Sent false
17843 5751 Acked Read 4 Done Done Sent false
17844 5752 Acked Write 2 Done Done Sent false
17845 5753 Acked Read 10 Done Done Sent false
17846 5754 Acked Read 5 Done Done Sent false
17847 5755 Acked Read 10 Done Done Sent false
17848 5756 Acked Read 6 Done Done Sent false
17849 5757 Acked Write 3 Done Done Sent false
17850 5758 Acked Read 9 Done Done Sent false
17851 5759 Acked Flush 0 Done Done Sent false
17852 5760 Acked Write 7 Done Done Sent false
17853 5761 Acked Read 8 Done Done Sent false
17854 5762 Acked Write 10 Done Done Sent false
17855 5763 Acked Read 10 Done Done Sent false
17856 5764 Acked Write 4 Done Done Sent false
17857 5765 Acked Write 6 Done Done Sent false
17858 5766 Acked Read 6 Done Done Sent false
17859 5767 Acked Write 8 Done Done Sent false
17860 5768 Acked Read 7 Done Done Sent false
17861 5769 Acked Write 10 Done Done Sent false
17862 5770 Acked Flush 0 Done Done Sent false
17863 5771 Acked Read 3 Done Done Sent false
17864 5772 Acked Read 2 Done Done Sent false
17865 5773 Acked Read 8 Done Done Sent false
17866 5774 Acked Read 2 Done Done Sent false
17867 5775 Acked Write 6 Done Done Sent false
17868 5776 Acked Read 4 Done Done Sent false
17869 5777 Acked Read 6 Done Done Sent false
17870 5778 Acked Write 2 Done Done Sent false
17871 5779 Acked Read 7 Done Done Sent false
17872 5780 Acked Read 5 Done Done Sent false
17873 5781 Acked Read 3 Done Done Sent false
17874 5782 Acked Read 8 Done Done Sent false
17875 5783 Acked Flush 0 Done Done Sent false
17876 5784 Acked Read 10 Done Done Sent false
17877 5785 Acked Read 8 Done Done Sent false
17878 5786 Acked Read 3 Done Done Sent false
17879 5787 Acked Read 1 Done Done Sent false
17880 5788 Acked Read 5 Done Done Sent false
17881 5789 Acked Write 10 Done Done Sent false
17882 5790 Acked Write 6 Done Done Sent false
17883 5791 Acked Write 9 Done Done Sent false
17884 5792 Acked Read 9 Done Done Sent false
17885 5793 Acked Flush 0 Done Done Sent false
17886 5794 Acked Read 5 Done Done Sent false
17887 5795 Acked Read 3 Done Done Sent false
17888 5796 Acked Read 7 Done Done Sent false
17889 5797 Acked Read 1 Done Done Sent false
17890 5798 Acked Read 7 Done Done Sent false
17891 5799 Acked Write 10 Done Done Sent false
17892 5800 Acked Write 10 Done Done Sent false
17893 5801 Acked Write 6 Done Done Sent false
17894 5802 Acked Write 2 Done Done Sent false
17895 5803 Acked Read 5 Done Done Sent false
17896 5804 Acked Read 6 Done Done Sent false
17897 5805 Acked Read 7 Done Done Sent false
17898 5806 Acked Read 3 Done Done Sent false
17899 5807 Acked Write 5 Done Done Sent false
17900 5808 Acked Read 2 Done Done Sent false
17901 5809 Acked Write 2 Done Done Sent false
17902 5810 Acked Flush 0 Done Done Sent false
17903 5811 Acked Write 10 Done Done Sent false
17904 5812 Acked Read 3 Done Done Sent false
17905 5813 Acked Write 4 Done Done Sent false
17906 5814 Acked Write 5 Done Done Sent false
17907 5815 Acked Flush 0 Done Done Sent false
17908 5816 Acked Flush 0 Done Done Sent false
17909 5817 Acked Write 2 Done Done Sent false
17910 5818 Acked Flush 0 Done Done Sent false
17911 5819 Acked Write 3 Done Done Sent false
17912 5820 Acked Read 8 Done Done Sent false
17913 5821 Acked Write 1 Done Done Sent false
17914 5822 Acked Write 8 Done Done Sent false
17915 5823 Acked Flush 0 Done Done Sent false
17916 5824 Acked Write 1 Done Done Sent false
17917 5825 Acked Write 5 Done Done Sent false
17918 5826 Acked Flush 0 Done Done Sent false
17919 5827 Acked Read 6 Done Done Sent false
17920 5828 Acked Write 8 Done Done Sent false
17921 STATES DS:0 DS:1 DS:2 TOTAL
17922 Sent 0 0 301 301
17923 Done 301 301 0 602
17924 Skipped 0 0 0 0
17925 Error 0 0 0 0
17926 Last Flush: 5826 5826 5527
17927 Downstairs last five completed: 5527 5526 5525 5524 5523
17928 Upstairs last five completed: 5828 5827 5826 5825 5824
17929 CLIENT: Up:0 ds:301 act:2
17930 [8830] got start action
17931 [8830] I am going to restart
17932 Starting downstairs at port 8830
17933 Make output file at "/tmp/dsc/downstairs-8830.txt"
17934 [8830][2] reports Starting
17935 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1216)
17936 [8830][2] reports Running
17937 ----------------------------------------------------------------
17938 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17939 States: Active Active Offline
17940 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17941 5528 Acked Read 2 Done Done Sent false
17942 5529 Acked Write 8 Done Done Sent false
17943 5530 Acked Write 10 Done Done Sent false
17944 5531 Acked Write 8 Done Done Sent false
17945 5532 Acked Read 10 Done Done Sent false
17946 5533 Acked Read 1 Done Done Sent false
17947 5534 Acked Read 2 Done Done Sent false
17948 5535 Acked Write 5 Done Done Sent false
17949 5536 Acked Write 7 Done Done Sent false
17950 5537 Acked Read 6 Done Done Sent false
17951 5538 Acked Read 6 Done Done Sent false
17952 5539 Acked Read 5 Done Done Sent false
17953 5540 Acked Flush 0 Done Done Sent false
17954 5541 Acked Write 9 Done Done Sent false
17955 5542 Acked Write 6 Done Done Sent false
17956 5543 Acked Read 6 Done Done Sent false
17957 5544 Acked Flush 0 Done Done Sent false
17958 5545 Acked Flush 0 Done Done Sent false
17959 5546 Acked Read 5 Done Done Sent false
17960 5547 Acked Read 10 Done Done Sent false
17961 5548 Acked Write 5 Done Done Sent false
17962 5549 Acked Read 3 Done Done Sent false
17963 5550 Acked Write 4 Done Done Sent false
17964 5551 Acked Read 7 Done Done Sent false
17965 5552 Acked Write 1 Done Done Sent false
17966 5553 Acked Write 5 Done Done Sent false
17967 5554 Acked Read 7 Done Done Sent false
17968 5555 Acked Read 4 Done Done Sent false
17969 5556 Acked Read 1 Done Done Sent false
17970 5557 Acked Read 5 Done Done Sent false
17971 5558 Acked Read 1 Done Done Sent false
17972 5559 Acked Read 7 Done Done Sent false
17973 5560 Acked Write 10 Done Done Sent false
17974 5561 Acked Read 2 Done Done Sent false
17975 5562 Acked Read 10 Done Done Sent false
17976 5563 Acked Read 5 Done Done Sent false
17977 5564 Acked Write 3 Done Done Sent false
17978 5565 Acked Flush 0 Done Done Sent false
17979 5566 Acked Read 6 Done Done Sent false
17980 5567 Acked Flush 0 Done Done Sent false
17981 5568 Acked Read 2 Done Done Sent false
17982 5569 Acked Read 6 Done Done Sent false
17983 5570 Acked Read 8 Done Done Sent false
17984 5571 Acked Read 5 Done Done Sent false
17985 5572 Acked Write 4 Done Done Sent false
17986 5573 Acked Write 10 Done Done Sent false
17987 5574 Acked Read 4 Done Done Sent false
17988 5575 Acked Read 5 Done Done Sent false
17989 5576 Acked Write 8 Done Done Sent false
17990 5577 Acked Read 2 Done Done Sent false
17991 5578 Acked Write 8 Done Done Sent false
17992 5579 Acked Write 4 Done Done Sent false
17993 5580 Acked Write 8 Done Done Sent false
17994 5581 Acked Write 1 Done Done Sent false
17995 5582 Acked Read 2 Done Done Sent false
17996 5583 Acked Write 5 Done Done Sent false
17997 5584 Acked Read 3 Done Done Sent false
17998 5585 Acked Read 4 Done Done Sent false
17999 5586 Acked Read 5 Done Done Sent false
18000 5587 Acked Read 7 Done Done Sent false
18001 5588 Acked Write 4 Done Done Sent false
18002 5589 Acked Write 3 Done Done Sent false
18003 5590 Acked Read 8 Done Done Sent false
18004 5591 Acked Write 10 Done Done Sent false
18005 5592 Acked Read 5 Done Done Sent false
18006 5593 Acked Read 1 Done Done Sent false
18007 5594 Acked Read 7 Done Done Sent false
18008 5595 Acked Read 4 Done Done Sent false
18009 5596 Acked Read 1 Done Done Sent false
18010 5597 Acked Read 10 Done Done Sent false
18011 5598 Acked Read 10 Done Done Sent false
18012 5599 Acked Read 7 Done Done Sent false
18013 5600 Acked Flush 0 Done Done Sent false
18014 5601 Acked Read 3 Done Done Sent false
18015 5602 Acked Write 10 Done Done Sent false
18016 5603 Acked Flush 0 Done Done Sent false
18017 5604 Acked Write 9 Done Done Sent false
18018 5605 Acked Write 5 Done Done Sent false
18019 5606 Acked Read 8 Done Done Sent false
18020 5607 Acked Read 9 Done Done Sent false
18021 5608 Acked Read 4 Done Done Sent false
18022 5609 Acked Write 9 Done Done Sent false
18023 5610 Acked Read 7 Done Done Sent false
18024 5611 Acked Read 2 Done Done Sent false
18025 5612 Acked Write 6 Done Done Sent false
18026 5613 Acked Read 2 Done Done Sent false
18027 5614 Acked Read 2 Done Done Sent false
18028 5615 Acked Read 4 Done Done Sent false
18029 5616 Acked Read 10 Done Done Sent false
18030 5617 Acked Write 8 Done Done Sent false
18031 5618 Acked Read 10 Done Done Sent false
18032 5619 Acked Write 2 Done Done Sent false
18033 5620 Acked Write 8 Done Done Sent false
18034 5621 Acked Flush 0 Done Done Sent false
18035 5622 Acked Read 9 Done Done Sent false
18036 5623 Acked Flush 0 Done Done Sent false
18037 5624 Acked Write 10 Done Done Sent false
18038 5625 Acked Read 5 Done Done Sent false
18039 5626 Acked Read 2 Done Done Sent false
18040 5627 Acked Read 6 Done Done Sent false
18041 5628 Acked Read 4 Done Done Sent false
18042 5629 Acked Read 2 Done Done Sent false
18043 5630 Acked Read 10 Done Done Sent false
18044 5631 Acked Write 4 Done Done Sent false
18045 5632 Acked Write 5 Done Done Sent false
18046 5633 Acked Flush 0 Done Done Sent false
18047 5634 Acked Write 3 Done Done Sent false
18048 5635 Acked Write 4 Done Done Sent false
18049 5636 Acked Read 7 Done Done Sent false
18050 5637 Acked Read 5 Done Done Sent false
18051 5638 Acked Write 4 Done Done Sent false
18052 5639 Acked Write 7 Done Done Sent false
18053 5640 Acked Read 10 Done Done Sent false
18054 5641 Acked Write 9 Done Done Sent false
18055 5642 Acked Read 10 Done Done Sent false
18056 5643 Acked Write 8 Done Done Sent false
18057 5644 Acked Read 6 Done Done Sent false
18058 5645 Acked Read 5 Done Done Sent false
18059 5646 Acked Write 10 Done Done Sent false
18060 5647 Acked Read 8 Done Done Sent false
18061 5648 Acked Flush 0 Done Done Sent false
18062 5649 Acked Write 1 Done Done Sent false
18063 5650 Acked Read 1 Done Done Sent false
18064 5651 Acked Write 2 Done Done Sent false
18065 5652 Acked Write 8 Done Done Sent false
18066 5653 Acked Read 3 Done Done Sent false
18067 5654 Acked Write 4 Done Done Sent false
18068 5655 Acked Write 1 Done Done Sent false
18069 5656 Acked Write 1 Done Done Sent false
18070 5657 Acked Flush 0 Done Done Sent false
18071 5658 Acked Write 4 Done Done Sent false
18072 5659 Acked Flush 0 Done Done Sent false
18073 5660 Acked Write 10 Done Done Sent false
18074 5661 Acked Read 5 Done Done Sent false
18075 5662 Acked Read 2 Done Done Sent false
18076 5663 Acked Write 3 Done Done Sent false
18077 5664 Acked Read 10 Done Done Sent false
18078 5665 Acked Read 4 Done Done Sent false
18079 5666 Acked Read 3 Done Done Sent false
18080 5667 Acked Read 5 Done Done Sent false
18081 5668 Acked Flush 0 Done Done Sent false
18082 5669 Acked Read 4 Done Done Sent false
18083 5670 Acked Read 1 Done Done Sent false
18084 5671 Acked Flush 0 Done Done Sent false
18085 5672 Acked Write 2 Done Done Sent false
18086 5673 Acked Read 9 Done Done Sent false
18087 5674 Acked Write 4 Done Done Sent false
18088 5675 Acked Flush 0 Done Done Sent false
18089 5676 Acked Write 3 Done Done Sent false
18090 5677 Acked Read 6 Done Done Sent false
18091 5678 Acked Read 5 Done Done Sent false
18092 5679 Acked Read 10 Done Done Sent false
18093 5680 Acked Read 1 Done Done Sent false
18094 5681 Acked Read 10 Done Done Sent false
18095 5682 Acked Flush 0 Done Done Sent false
18096 5683 Acked Write 3 Done Done Sent false
18097 5684 Acked Read 10 Done Done Sent false
18098 5685 Acked Read 7 Done Done Sent false
18099 5686 Acked Write 9 Done Done Sent false
18100 5687 Acked Flush 0 Done Done Sent false
18101 5688 Acked Read 2 Done Done Sent false
18102 5689 Acked Write 2 Done Done Sent false
18103 5690 Acked Read 8 Done Done Sent false
18104 5691 Acked Write 8 Done Done Sent false
18105 5692 Acked Read 10 Done Done Sent false
18106 5693 Acked Write 5 Done Done Sent false
18107 5694 Acked Write 4 Done Done Sent false
18108 5695 Acked Write 8 Done Done Sent false
18109 5696 Acked Read 9 Done Done Sent false
18110 5697 Acked Write 7 Done Done Sent false
18111 5698 Acked Write 4 Done Done Sent false
18112 5699 Acked Write 9 Done Done Sent false
18113 5700 Acked Write 8 Done Done Sent false
18114 5701 Acked Write 5 Done Done Sent false
18115 5702 Acked Write 9 Done Done Sent false
18116 5703 Acked Read 6 Done Done Sent false
18117 5704 Acked Write 1 Done Done Sent false
18118 5705 Acked Read 4 Done Done Sent false
18119 5706 Acked Read 7 Done Done Sent false
18120 5707 Acked Write 5 Done Done Sent false
18121 5708 Acked Write 1 Done Done Sent false
18122 5709 Acked Read 10 Done Done Sent false
18123 5710 Acked Flush 0 Done Done Sent false
18124 5711 Acked Read 5 Done Done Sent false
18125 5712 Acked Write 2 Done Done Sent false
18126 5713 Acked Read 4 Done Done Sent false
18127 5714 Acked Read 2 Done Done Sent false
18128 5715 Acked Read 1 Done Done Sent false
18129 5716 Acked Read 9 Done Done Sent false
18130 5717 Acked Write 5 Done Done Sent false
18131 5718 Acked Write 10 Done Done Sent false
18132 5719 Acked Read 9 Done Done Sent false
18133 5720 Acked Write 1 Done Done Sent false
18134 5721 Acked Read 3 Done Done Sent false
18135 5722 Acked Read 5 Done Done Sent false
18136 5723 Acked Read 10 Done Done Sent false
18137 5724 Acked Write 7 Done Done Sent false
18138 5725 Acked Read 1 Done Done Sent false
18139 5726 Acked Flush 0 Done Done Sent false
18140 5727 Acked Write 9 Done Done Sent false
18141 5728 Acked Write 10 Done Done Sent false
18142 5729 Acked Read 1 Done Done Sent false
18143 5730 Acked Write 6 Done Done Sent false
18144 5731 Acked Write 6 Done Done Sent false
18145 5732 Acked Write 8 Done Done Sent false
18146 5733 Acked Flush 0 Done Done Sent false
18147 5734 Acked Read 10 Done Done Sent false
18148 5735 Acked Read 5 Done Done Sent false
18149 5736 Acked Read 10 Done Done Sent false
18150 5737 Acked Write 4 Done Done Sent false
18151 5738 Acked Read 7 Done Done Sent false
18152 5739 Acked Read 8 Done Done Sent false
18153 5740 Acked Read 2 Done Done Sent false
18154 5741 Acked Read 3 Done Done Sent false
18155 5742 Acked Read 2 Done Done Sent false
18156 5743 Acked Write 8 Done Done Sent false
18157 5744 Acked Read 1 Done Done Sent false
18158 5745 Acked Read 8 Done Done Sent false
18159 5746 Acked Read 5 Done Done Sent false
18160 5747 Acked Read 1 Done Done Sent false
18161 5748 Acked Read 3 Done Done Sent false
18162 5749 Acked Write 9 Done Done Sent false
18163 5750 Acked Read 8 Done Done Sent false
18164 5751 Acked Read 4 Done Done Sent false
18165 5752 Acked Write 2 Done Done Sent false
18166 5753 Acked Read 10 Done Done Sent false
18167 5754 Acked Read 5 Done Done Sent false
18168 5755 Acked Read 10 Done Done Sent false
18169 5756 Acked Read 6 Done Done Sent false
18170 5757 Acked Write 3 Done Done Sent false
18171 5758 Acked Read 9 Done Done Sent false
18172 5759 Acked Flush 0 Done Done Sent false
18173 5760 Acked Write 7 Done Done Sent false
18174 5761 Acked Read 8 Done Done Sent false
18175 5762 Acked Write 10 Done Done Sent false
18176 5763 Acked Read 10 Done Done Sent false
18177 5764 Acked Write 4 Done Done Sent false
18178 5765 Acked Write 6 Done Done Sent false
18179 5766 Acked Read 6 Done Done Sent false
18180 5767 Acked Write 8 Done Done Sent false
18181 5768 Acked Read 7 Done Done Sent false
18182 5769 Acked Write 10 Done Done Sent false
18183 5770 Acked Flush 0 Done Done Sent false
18184 5771 Acked Read 3 Done Done Sent false
18185 5772 Acked Read 2 Done Done Sent false
18186 5773 Acked Read 8 Done Done Sent false
18187 5774 Acked Read 2 Done Done Sent false
18188 5775 Acked Write 6 Done Done Sent false
18189 5776 Acked Read 4 Done Done Sent false
18190 5777 Acked Read 6 Done Done Sent false
18191 5778 Acked Write 2 Done Done Sent false
18192 5779 Acked Read 7 Done Done Sent false
18193 5780 Acked Read 5 Done Done Sent false
18194 5781 Acked Read 3 Done Done Sent false
18195 5782 Acked Read 8 Done Done Sent false
18196 5783 Acked Flush 0 Done Done Sent false
18197 5784 Acked Read 10 Done Done Sent false
18198 5785 Acked Read 8 Done Done Sent false
18199 5786 Acked Read 3 Done Done Sent false
18200 5787 Acked Read 1 Done Done Sent false
18201 5788 Acked Read 5 Done Done Sent false
18202 5789 Acked Write 10 Done Done Sent false
18203 5790 Acked Write 6 Done Done Sent false
18204 5791 Acked Write 9 Done Done Sent false
18205 5792 Acked Read 9 Done Done Sent false
18206 5793 Acked Flush 0 Done Done Sent false
18207 5794 Acked Read 5 Done Done Sent false
18208 5795 Acked Read 3 Done Done Sent false
18209 5796 Acked Read 7 Done Done Sent false
18210 5797 Acked Read 1 Done Done Sent false
18211 5798 Acked Read 7 Done Done Sent false
18212 5799 Acked Write 10 Done Done Sent false
18213 5800 Acked Write 10 Done Done Sent false
18214 5801 Acked Write 6 Done Done Sent false
18215 5802 Acked Write 2 Done Done Sent false
18216 5803 Acked Read 5 Done Done Sent false
18217 5804 Acked Read 6 Done Done Sent false
18218 5805 Acked Read 7 Done Done Sent false
18219 5806 Acked Read 3 Done Done Sent false
18220 5807 Acked Write 5 Done Done Sent false
18221 5808 Acked Read 2 Done Done Sent false
18222 5809 Acked Write 2 Done Done Sent false
18223 5810 Acked Flush 0 Done Done Sent false
18224 5811 Acked Write 10 Done Done Sent false
18225 5812 Acked Read 3 Done Done Sent false
18226 5813 Acked Write 4 Done Done Sent false
18227 5814 Acked Write 5 Done Done Sent false
18228 5815 Acked Flush 0 Done Done Sent false
18229 5816 Acked Flush 0 Done Done Sent false
18230 5817 Acked Write 2 Done Done Sent false
18231 5818 Acked Flush 0 Done Done Sent false
18232 5819 Acked Write 3 Done Done Sent false
18233 5820 Acked Read 8 Done Done Sent false
18234 5821 Acked Write 1 Done Done Sent false
18235 5822 Acked Write 8 Done Done Sent false
18236 5823 Acked Flush 0 Done Done Sent false
18237 5824 Acked Write 1 Done Done Sent false
18238 5825 Acked Write 5 Done Done Sent false
18239 5826 Acked Flush 0 Done Done Sent false
18240 5827 Acked Read 6 Done Done Sent false
18241 5828 Acked Write 8 Done Done Sent false
18242 5829 Acked Flush 0 Done Done Sent false
18243 STATES DS:0 DS:1 DS:2 TOTAL
18244 Sent 0 0 302 302
18245 Done 302 302 0 604
18246 Skipped 0 0 0 0
18247 Error 0 0 0 0
18248 Last Flush: 5829 5829 5527
18249 Downstairs last five completed: 5527 5526 5525 5524 5523
18250 Upstairs last five completed: 5829 5828 5827 5826 5825
18251 CLIENT: Up:0 ds:302 act:2
182522025-01-17T21:14:54.126ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
182532025-01-17T21:14:54.126ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:63897) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
182542025-01-17T21:14:54.127ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
182552025-01-17T21:14:54.128ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
182562025-01-17T21:14:54.128ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
182572025-01-17T21:14:54.128ZINFOcrucible: send last flush ID to this DS: 5527 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
182582025-01-17T21:14:54.128ZINFOcrucible: Replied this last flush ID: 5527 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
182592025-01-17T21:14:54.128ZINFOcrucible: [2] client re-new 302 jobs since flush 5527 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
182602025-01-17T21:14:54.128ZINFOcrucible: [2] Replayed 302 jobs since flush: 5527 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
18261 ----------------------------------------------------------------
18262 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
18263 Downstairs last five completed: 5829 5828 5827 5826 5825
18264 Upstairs last five completed: 5829 5828 5827 5826 5825
18265 CLIENT: Up:0 ds:0 act:3
18266 Replay: All jobs finished
18267 Main task has work to do, go find it
18268 got dsc Stop(1)
18269 stop 1
18270 Jan 17 21:14:56.387 INFO request completed, latency_us: 320, response_code: 204, uri[8820] Got stop action so:false kr:false
18271 : /stop/cid/1, method: POST, req_id: eca007c0-0c52-49b2-89ba-6ab7c2ee96a3, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
18272 Jan 17 21:14:56.388 INFO request completed, latency_us: 103, response_code: 200, uri{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:14:56.388402628Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
18273 : /state/cid/1, method: GET, req_id: a8f9301a-f34d-4ce9-982f-3eef060f8e29, remote_addr{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:14:56.388453061Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
18274 : 127.0.0.1:57191{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:14:56.388470368Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
18275 , local_addr: 127.0.0.1:9998
182762025-01-17T21:14:56.388ZINFOcrucible: client task is exiting = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
18277 [8820] Exited with: ExitStatus(unix_wait_status(9))
18278 [8820][1] reports Exit
18279 Jan 17 21:15:00.390 INFO request completed, latency_us: 233, response_code: 200, uri: /state/cid/1, method: GET, req_id: 7ec15975-0f53-438c-9052-77c3aae264ab, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
18280 001/300 Flush
18281 002/300 Read block 4504 len 4096
18282 003/300 Read block 2844 len 8192
18283 004/300 Write block 1794 len 8192 data: 5 6
18284 005/300 Write block 3137 len 32768 data: 4 4 3 5 4 4 4 4
18285 006/300 Flush
18286 007/300 Write block 3450 len 20480 data: 8 7 6 5 4
18287 008/300 Read block 812 len 20480
18288 009/300 Flush
18289 010/300 Write block 1875 len 4096 data: 3
18290 011/300 Read block 1467 len 36864
18291 012/300 Write block 3624 len 4096 data: 4
18292 013/300 Flush
18293 014/300 Read block 982 len 36864
18294 015/300 Write block 304 len 32768 data: 4 4 4 4 3 2 2 2
18295 016/300 Read block 1802 len 40960
18296 017/300 Read block 936 len 16384
18297 018/300 Write block 4468 len 28672 data: 6 5 7 8 9 8 9
18298 019/300 Write block 1888 len 20480 data: 6 4 4 4 4
18299 020/300 Read block 2848 len 40960
18300 021/300 Read block 775 len 40960
18301 022/300 Write block 2974 len 36864 data: 3 3 5 3 3 3 4 3 3
18302 023/300 Flush
18303 024/300 Write block 3836 len 28672 data: 4 4 4 5 5 5 7
18304 025/300 Write block 4293 len 16384 data: 4 5 5 5
18305 026/300 Read block 1668 len 8192
18306 027/300 Read block 4524 len 16384
18307 028/300 Flush
18308 029/300 Write block 2622 len 4096 data: 4
18309 030/300 Read block 1659 len 16384
18310 031/300 Write block 905 len 4096 data: 4
18311 032/300 Read block 2433 len 36864
18312 033/300 Write block 1326 len 16384 data: 6 6 7 7
18313 034/300 Write block 3924 len 40960 data: 3 3 2 2 2 3 4 7 7 8
18314 035/300 Read block 18 len 28672
18315 036/300 Read block 3760 len 12288
18316 037/300 Write block 1858 len 4096 data: 4
18317 038/300 Read block 3692 len 36864
18318 039/300 Write block 2586 len 8192 data: 7 7
18319 040/300 Read block 3595 len 8192
18320 041/300 Read block 4761 len 16384
18321 042/300 Write block 1564 len 20480 data: 4 4 4 4 5
18322 043/300 Read block 4144 len 32768
18323 044/300 Write block 3993 len 12288 data: 7 6 7
18324 045/300 Write block 3795 len 8192 data: 4 4
18325 046/300 Write block 2585 len 36864 data: 5 8 8 7 7 5 5 5 3
18326 047/300 Read block 3642 len 24576
18327 048/300 Read block 3307 len 16384
18328 049/300 Read block 4456 len 40960
18329 050/300 Flush
18330 051/300 Write block 2604 len 4096 data: 5
18331 052/300 Read block 4456 len 12288
18332 053/300 Read block 4188 len 4096
18333 054/300 Write block 3873 len 32768 data: 3 4 5 6 5 5 3 3
18334 055/300 Flush
18335 056/300 Read block 2116 len 16384
18336 057/300 Read block 4917 len 8192
18337 058/300 Flush
18338 059/300 Read block 1384 len 20480
18339 060/300 Read block 1029 len 16384
18340 061/300 Read block 1275 len 24576
18341 062/300 Read block 2757 len 16384
18342 063/300 Write block 3036 len 20480 data: 4 3 3 3 3
18343 064/300 Write block 8 len 4096 data: 3
18344 065/300 Write block 3712 len 32768 data: 3 3 3 4 4 4 3 3
18345 066/300 Read block 172 len 4096
18346 067/300 Write block 3821 len 24576 data: 2 2 3 4 5 4
18347 068/300 Read block 3239 len 16384
18348 069/300 Read block 4020 len 28672
18349 070/300 Flush
18350 071/300 Write block 2041 len 4096 data: 4
18351 072/300 Read block 1437 len 32768
18352 073/300 Read block 3558 len 20480
18353 074/300 Write block 1363 len 36864 data: 3 2 2 2 3 3 3 2 2
18354 075/300 Read block 759 len 36864
18355 076/300 Read block 4400 len 32768
18356 077/300 Flush
18357 078/300 Write block 1557 len 16384 data: 4 4 4 4
18358 079/300 Write block 4373 len 16384 data: 5 4 4 5
18359 080/300 Write block 2155 len 16384 data: 5 5 4 3
18360 081/300 Read block 1951 len 40960
18361 082/300 Write block 2531 len 24576 data: 3 3 3 3 3 3
18362 083/300 Read block 4135 len 4096
18363 084/300 Flush
18364 085/300 Read block 1328 len 4096
18365 086/300 Write block 1678 len 20480 data: 6 7 6 3 3
18366 087/300 Write block 3627 len 12288 data: 2 2 3
18367 088/300 Read block 1918 len 28672
18368 089/300 Write block 4199 len 20480 data: 3 3 3 4 4
18369 090/300 Flush
18370 091/300 Write block 1744 len 20480 data: 6 6 4 4 4
18371 092/300 Read block 2313 len 4096
18372 093/300 Read block 1561 len 12288
18373 094/300 Read block 3261 len 24576
18374 095/300 Write block 2379 len 32768 data: 5 4 4 4 4 4 4 4
18375 096/300 Read block 4810 len 36864
18376 097/300 Read block 3722 len 4096
18377 098/300 Read block 1432 len 16384
18378 099/300 Read block 1531 len 12288
18379 100/300 Write block 4666 len 16384 data: 4 4 4 4
18380 101/300 Write block 3320 len 8192 data: 6 5
18381 102/300 Read block 4447 len 32768
18382 103/300 Write block 1844 len 28672 data: 3 3 3 3 3 3 3
18383 104/300 Write block 3228 len 36864 data: 6 5 5 4 4 3 4 5 5
18384 105/300 Write block 2445 len 24576 data: 3 3 3 3 3 2
18385 106/300 Read block 30 len 36864
18386 107/300 Flush
18387 108/300 Read block 4944 len 36864
18388 109/300 Write block 4172 len 24576 data: 5 6 7 8 6 8
18389 110/300 Read block 4704 len 20480
18390 111/300 Write block 586 len 28672 data: 6 5 4 4 5 5 5
18391 112/300 Write block 2190 len 20480 data: 5 4 4 4 4
18392 113/300 Read block 296 len 4096
18393 114/300 Read block 3341 len 12288
18394 115/300 Read block 4192 len 16384
18395 116/300 Write block 285 len 40960 data: 6 5 5 5 5 3 3 3 4 4
18396 117/300 Write block 4652 len 36864 data: 4 4 4 4 2 2 3 3 3
18397 118/300 Read block 2302 len 12288
18398 119/300 Write block 2513 len 32768 data: 6 7 7 8 8 5 5 3
18399 120/300 Flush
18400 121/300 Write block 1195 len 20480 data: 4 4 3 3 3
18401 122/300 Flush
18402 123/300 Read block 4519 len 28672
18403 124/300 Read block 3803 len 28672
18404 125/300 Flush
18405 126/300 Write block 1217 len 28672 data: 2 2 2 2 2 2 2
18406 127/300 Flush
18407 128/300 Write block 2729 len 12288 data: 6 6 6
18408 129/300 Write block 3528 len 32768 data: 5 3 5 4 4 4 3 3
18409 130/300 Read block 3614 len 20480
18410 131/300 Read block 4432 len 12288
18411 132/300 Write block 1318 len 28672 data: 2 2 2 3 4 4 4
18412 133/300 Read block 3896 len 12288
18413 134/300 Read block 1375 len 4096
18414 135/300 Read block 543 len 36864
18415 136/300 Write block 4617 len 28672 data: 2 2 3 5 6 5 5
18416 137/300 Read block 4549 len 16384
18417 138/300 Read block 4126 len 12288
18418 139/300 Write block 190 len 36864 data: 5 4 4 4 5 8 8 8 6
18419 140/300 Write block 755 len 16384 data: 3 4 3 3
18420 141/300 Write block 4156 len 20480 data: 6 5 5 4 4
18421 142/300 Read block 2699 len 32768
18422 143/300 Write block 3494 len 28672 data: 3 4 5 4 4 4 4
18423 144/300 Write block 3203 len 4096 data: 6
18424 145/300 Write block 3992 len 28672 data: 7 8 7 8 8 7 6
18425 146/300 Write block 3830 len 28672 data: 5 4 5 4 4 4 5
18426 147/300 Read block 437 len 40960
18427 148/300 Read block 1381 len 32768
18428 149/300 Read block 2749 len 24576
18429 150/300 Write block 3782 len 28672 data: 6 6 5 6 6 6 6
18430 151/300 Read block 2213 len 32768
18431 152/300 Read block 234 len 28672
18432 153/300 Read block 331 len 36864
18433 154/300 Write block 4596 len 40960 data: 3 2 2 3 3 3 3 3 2 3
18434 155/300 Read block 4711 len 28672
18435 156/300 Write block 4235 len 12288 data: 5 5 5
18436 157/300 Read block 463 len 32768
18437 158/300 Read block 4362 len 40960
18438 159/300 Write block 4692 len 24576 data: 5 6 6 4 3 3
18439 160/300 Read block 201 len 36864
18440 161/300 Write block 870 len 20480 data: 3 4 4 5 5
18441 162/300 Read block 4158 len 40960
18442 163/300 Read block 746 len 40960
18443 164/300 Read block 3484 len 16384
18444 165/300 Write block 515 len 24576 data: 3 3 4 3 2 3
18445 166/300 Write block 1620 len 12288 data: 3 2 3
18446 167/300 Write block 3640 len 4096 data: 3
18447 168/300 Read block 1394 len 36864
18448 169/300 Flush
18449 170/300 Read block 1674 len 4096
18450 171/300 Write block 853 len 36864 data: 2 2 2 3 3 3 3 5 5
18451 172/300 Read block 4171 len 28672
18452 173/300 Read block 3277 len 16384
18453 174/300 Write block 1921 len 16384 data: 6 6 5 5
18454 175/300 Read block 600 len 32768
18455 176/300 Read block 835 len 36864
18456 177/300 Write block 1463 len 4096 data: 5
18457 178/300 Write block 4684 len 32768 data: 4 3 3 4 5 5 5 5
18458 179/300 Read block 2825 len 36864
18459 180/300 Read block 502 len 36864
18460 181/300 Read block 483 len 36864
18461 182/300 Read block 2725 len 12288
18462 183/300 Read block 1671 len 28672
18463 184/300 Flush
18464 185/300 Write block 4692 len 8192 data: 6 7
18465 186/300 Write block 3756 len 36864 data: 4 4 3 2 2 4 4 4 4
18466 187/300 Read block 2412 len 8192
18467 188/300 Read block 4371 len 32768
18468 189/300 Write block 4818 len 36864 data: 5 4 5 3 3 3 4 3 4
18469 190/300 Write block 2787 len 28672 data: 4 3 3 5 6 7 6
18470 191/300 Read block 3566 len 24576
18471 192/300 Read block 1930 len 8192
18472 193/300 Write block 1743 len 36864 data: 6 7 7 5 5 5 5 5 4
18473 194/300 Write block 3541 len 16384 data: 4 4 4 4
18474 195/300 Write block 847 len 8192 data: 4 2
18475 196/300 Read block 4263 len 4096
18476 197/300 Write block 1332 len 4096 data: 6
18477 198/300 Read block 4452 len 20480
18478 199/300 Write block 1585 len 28672 data: 9 7 6 5 6 5 5
18479 200/300 Read block 788 len 16384
18480 201/300 Write block 3871 len 12288 data: 4 4 4
18481 202/300 Read block 2939 len 12288
18482 203/300 Read block 3881 len 20480
18483 204/300 Read block 3956 len 4096
18484 205/300 Write block 3203 len 40960 data: 7 6 6 4 5 4 4 4 5 6
18485 206/300 Read block 1641 len 40960
18486 207/300 Write block 4685 len 16384 data: 4 4 5 6
18487 208/300 Write block 4638 len 24576 data: 5 5 4 3 3 2
18488 209/300 Read block 367 len 32768
18489 210/300 Read block 641 len 4096
18490 211/300 Write block 3495 len 8192 data: 5 6
18491 212/300 Read block 3683 len 32768
18492 213/300 Flush
18493 214/300 Write block 3210 len 36864 data: 5 6 7 6 4 3 4 4 5
18494 215/300 Write block 1692 len 4096 data: 4
18495 216/300 Read block 1055 len 32768
18496 217/300 Read block 2774 len 4096
18497 218/300 Flush
18498 219/300 Read block 1678 len 36864
18499 220/300 Read block 1056 len 8192
18500 221/300 Flush
18501 222/300 Read block 365 len 36864
18502 223/300 Flush
18503 224/300 Flush
18504 225/300 Write block 4261 len 12288 data: 6 6 7
18505 226/300 Write block 3921 len 20480 data: 3 3 3 4 4
18506 227/300 Write block 3969 len 4096 data: 2
18507 228/300 Write block 1009 len 24576 data: 5 5 5 4 4 4
18508 229/300 Read block 2664 len 4096
18509 230/300 Write block 1783 len 8192 data: 2 3
18510 231/300 Read block 4948 len 8192
18511 232/300 Write block 2790 len 12288 data: 6 7 8
18512 233/300 Write block 4934 len 8192 data: 3 4
18513 234/300 Read block 2842 len 32768
18514 235/300 Flush
18515 236/300 Write block 659 len 4096 data: 4
18516 237/300 Flush
18517 238/300 Flush
18518 239/300 Flush
18519 240/300 Read block 3108 len 24576
18520 241/300 Read block 4287 len 32768
18521 242/300 Read block 4884 len 16384
18522 243/300 Write block 532 len 20480 data: 6 5 5 4 5
18523 244/300 Flush
18524 245/300 Read block 3090 len 8192
18525 246/300 Read block 476 len 28672
18526 247/300 Write block 2043 len 20480 data: 3 3 3 3 4
18527 248/300 Write block 789 len 8192 data: 4 5
18528 249/300 Read block 143 len 4096
18529 250/300 Read block 3322 len 28672
18530 251/300 Flush
18531 252/300 Read block 3157 len 40960
18532 253/300 Read block 4531 len 8192
18533 254/300 Write block 2728 len 12288 data: 6 7 7
18534 255/300 Write block 4371 len 40960 data: 6 4 6 5 5 6 5 4 3 3
18535 256/300 Read block 4212 len 8192
18536 257/300 Read block 2798 len 28672
18537 258/300 Write block 3924 len 8192 data: 5 5
18538 259/300 Write block 1256 len 8192 data: 4 4
18539 260/300 Read block 730 len 36864
18540 261/300 Write block 1585 len 40960 data: 10 8 7 6 7 6 6 5 3 4
18541 262/300 Write block 2055 len 40960 data: 5 5 4 4 4 6 5 6 6 6
18542 263/300 Write block 3276 len 36864 data: 2 3 3 3 3 3 3 3 3
18543 264/300 Write block 1193 len 32768 data: 4 4 5 5 4 4 4 2
18544 265/300 Read block 1229 len 8192
18545 266/300 Read block 2322 len 12288
18546 267/300 Read block 4104 len 16384
18547 268/300 Write block 1004 len 28672 data: 2 2 2 4 5 6 6
18548 269/300 Write block 1234 len 12288 data: 3 3 3
18549 270/300 Write block 3423 len 16384 data: 4 4 3 4
18550 271/300 Read block 1380 len 36864
18551 272/300 Flush
18552 273/300 Write block 2427 len 36864 data: 4 5 5 4 4 4 3 3 3
18553 274/300 Write block 3120 len 16384 data: 5 4 4 4
18554 275/300 Read block 88 len 24576
18555 276/300 Write block 2903 len 40960 data: 5 6 4 3 3 3 3 3 4 4
18556 277/300 Read block 12 len 36864
18557 278/300 Flush
18558 279/300 Read block 4900 len 28672
18559 280/300 Read block 3306 len 4096
18560 281/300 Write block 2844 len 20480 data: 3 3 4 3 3
18561 282/300 Flush
18562 283/300 Write block 1506 len 16384 data: 5 4 4 3
18563 284/300 Write block 2503 len 32768 data: 3 3 3 3 3 3 3 5
18564 285/300 Write block 1231 len 32768 data: 5 4 3 4 4 4 3 2
18565 286/300 Write block 2524 len 12288 data: 2 2 3
18566 287/300 Flush
18567 288/300 Write block 1901 len 20480 data: 5 5 4 5 3
18568 289/300 Write block 1439 len 20480 data: 5 5 5 6 5
18569 290/300 Write block 2567 len 28672 data: 3 3 2 2 2 2 2
18570 291/300 Read block 1172 len 40960
18571 292/300 Write block 2374 len 16384 data: 4 4 5 5
18572 293/300 Flush
18573 294/300 Flush
18574 295/300 Read block 4294 len 20480
18575 296/300 Read block 1740 len 40960
18576 297/300 Write block 180 len 12288 data: 5 4 5
18577 298/300 Read block 2857 len 36864
18578 299/300 Read block 1582 len 40960
18579 300/300 Read block 3750 len 40960
18580 301/300 Write block 3283 len 12288 data: 4 4 3
18581 Main task has work to do, go find it
18582 got dsc Start(1)
18583 start 1
18584 Jan 17 21:15:00.625 INFO request completed, latency_us: 251, response_code: 204, uri: /start/cid/1, method: POST, req_id: d1547479-65b0-4504-82d8-5c67afdeba89, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
18585 Replay: started 1, returned:Ok(())
18586 ----------------------------------------------------------------
18587 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
18588 States: Active Offline Active
18589 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18590 5830 Acked Flush 0 Done Sent Done false
18591 5831 Acked Read 1 Done Sent Done false
18592 5832 Acked Read 2 Done Sent Done false
18593 5833 Acked Write 2 Done Sent Done false
18594 5834 Acked Write 8 Done Sent Done false
18595 5835 Acked Flush 0 Done Sent Done false
18596 5836 Acked Write 5 Done Sent Done false
18597 5837 Acked Read 5 Done Sent Done false
18598 5838 Acked Flush 0 Done Sent Done false
18599 5839 Acked Write 1 Done Sent Done false
18600 5840 Acked Read 9 Done Sent Done false
18601 5841 Acked Write 1 Done Sent Done false
18602 5842 Acked Flush 0 Done Sent Done false
18603 5843 Acked Read 9 Done Sent Done false
18604 5844 Acked Write 8 Done Sent Done false
18605 5845 Acked Read 10 Done Sent Done false
18606 5846 Acked Read 4 Done Sent Done false
18607 5847 Acked Write 7 Done Sent Done false
18608 5848 Acked Write 5 Done Sent Done false
18609 5849 Acked Read 10 Done Sent Done false
18610 5850 Acked Read 10 Done Sent Done false
18611 5851 Acked Write 9 Done Sent Done false
18612 5852 Acked Flush 0 Done Sent Done false
18613 5853 Acked Write 7 Done Sent Done false
18614 5854 Acked Write 4 Done Sent Done false
18615 5855 Acked Read 2 Done Sent Done false
18616 5856 Acked Read 4 Done Sent Done false
18617 5857 Acked Flush 0 Done Sent Done false
18618 5858 Acked Write 1 Done Sent Done false
18619 5859 Acked Read 4 Done Sent Done false
18620 5860 Acked Write 1 Done Sent Done false
18621 5861 Acked Read 9 Done Sent Done false
18622 5862 Acked Write 4 Done Sent Done false
18623 5863 Acked Write 10 Done Sent Done false
18624 5864 Acked Read 7 Done Sent Done false
18625 5865 Acked Read 3 Done Sent Done false
18626 5866 Acked Write 1 Done Sent Done false
18627 5867 Acked Read 9 Done Sent Done false
18628 5868 Acked Write 2 Done Sent Done false
18629 5869 Acked Read 2 Done Sent Done false
18630 5870 Acked Read 4 Done Sent Done false
18631 5871 Acked Write 5 Done Sent Done false
18632 5872 Acked Read 8 Done Sent Done false
18633 5873 Acked Write 3 Done Sent Done false
18634 5874 Acked Write 2 Done Sent Done false
18635 5875 Acked Write 9 Done Sent Done false
18636 5876 Acked Read 6 Done Sent Done false
18637 5877 Acked Read 4 Done Sent Done false
18638 5878 Acked Read 10 Done Sent Done false
18639 5879 Acked Flush 0 Done Sent Done false
18640 5880 Acked Write 1 Done Sent Done false
18641 5881 Acked Read 3 Done Sent Done false
18642 5882 Acked Read 1 Done Sent Done false
18643 5883 Acked Write 8 Done Sent Done false
18644 5884 Acked Flush 0 Done Sent Done false
18645 5885 Acked Read 4 Done Sent Done false
18646 5886 Acked Read 2 Done Sent Done false
18647 5887 Acked Flush 0 Done Sent Done false
18648 5888 Acked Read 5 Done Sent Done false
18649 5889 Acked Read 4 Done Sent Done false
18650 5890 Acked Read 6 Done Sent Done false
18651 5891 Acked Read 4 Done Sent Done false
18652 5892 Acked Write 5 Done Sent Done false
18653 5893 Acked Write 1 Done Sent Done false
18654 5894 Acked Write 8 Done Sent Done false
18655 5895 Acked Read 1 Done Sent Done false
18656 5896 Acked Write 6 Done Sent Done false
18657 5897 Acked Read 4 Done Sent Done false
18658 5898 Acked Read 7 Done Sent Done false
18659 5899 Acked Flush 0 Done Sent Done false
18660 5900 Acked Write 1 Done Sent Done false
18661 5901 Acked Read 8 Done Sent Done false
18662 5902 Acked Read 5 Done Sent Done false
18663 5903 Acked Write 9 Done Sent Done false
18664 5904 Acked Read 9 Done Sent Done false
18665 5905 Acked Read 8 Done Sent Done false
18666 5906 Acked Flush 0 Done Sent Done false
18667 5907 Acked Write 4 Done Sent Done false
18668 5908 Acked Write 4 Done Sent Done false
18669 5909 Acked Write 4 Done Sent Done false
18670 5910 Acked Read 10 Done Sent Done false
18671 5911 Acked Write 6 Done Sent Done false
18672 5912 Acked Read 1 Done Sent Done false
18673 5913 Acked Flush 0 Done Sent Done false
18674 5914 Acked Read 1 Done Sent Done false
18675 5915 Acked Write 5 Done Sent Done false
18676 5916 Acked Write 3 Done Sent Done false
18677 5917 Acked Read 7 Done Sent Done false
18678 5918 Acked Write 5 Done Sent Done false
18679 5919 Acked Flush 0 Done Sent Done false
18680 5920 Acked Write 5 Done Sent Done false
18681 5921 Acked Read 1 Done Sent Done false
18682 5922 Acked Read 3 Done Sent Done false
18683 5923 Acked Read 6 Done Sent Done false
18684 5924 Acked Write 8 Done Sent Done false
18685 5925 Acked Read 9 Done Sent Done false
18686 5926 Acked Read 1 Done Sent Done false
18687 5927 Acked Read 4 Done Sent Done false
18688 5928 Acked Read 3 Done Sent Done false
18689 5929 Acked Write 4 Done Sent Done false
18690 5930 Acked Write 2 Done Sent Done false
18691 5931 Acked Read 8 Done Sent Done false
18692 5932 Acked Write 7 Done Sent Done false
18693 5933 Acked Write 9 Done Sent Done false
18694 5934 Acked Write 6 Done Sent Done false
18695 5935 Acked Read 9 Done Sent Done false
18696 5936 Acked Flush 0 Done Sent Done false
18697 5937 Acked Read 9 Done Sent Done false
18698 5938 Acked Write 6 Done Sent Done false
18699 5939 Acked Read 5 Done Sent Done false
18700 5940 Acked Write 7 Done Sent Done false
18701 5941 Acked Write 5 Done Sent Done false
18702 5942 Acked Read 1 Done Sent Done false
18703 5943 Acked Read 3 Done Sent Done false
18704 5944 Acked Read 4 Done Sent Done false
18705 5945 Acked Write 10 Done Sent Done false
18706 5946 Acked Write 9 Done Sent Done false
18707 5947 Acked Read 3 Done Sent Done false
18708 5948 Acked Write 8 Done Sent Done false
18709 5949 Acked Flush 0 Done Sent Done false
18710 5950 Acked Write 5 Done Sent Done false
18711 5951 Acked Flush 0 Done Sent Done false
18712 5952 Acked Read 7 Done Sent Done false
18713 5953 Acked Read 7 Done Sent Done false
18714 5954 Acked Flush 0 Done Sent Done false
18715 5955 Acked Write 7 Done Sent Done false
18716 5956 Acked Flush 0 Done Sent Done false
18717 5957 Acked Write 3 Done Sent Done false
18718 5958 Acked Write 8 Done Sent Done false
18719 5959 Acked Read 5 Done Sent Done false
18720 5960 Acked Read 3 Done Sent Done false
18721 5961 Acked Write 7 Done Sent Done false
18722 5962 Acked Read 3 Done Sent Done false
18723 5963 Acked Read 1 Done Sent Done false
18724 5964 Acked Read 9 Done Sent Done false
18725 5965 Acked Write 7 Done Sent Done false
18726 5966 Acked Read 4 Done Sent Done false
18727 5967 Acked Read 3 Done Sent Done false
18728 5968 Acked Write 9 Done Sent Done false
18729 5969 Acked Write 4 Done Sent Done false
18730 5970 Acked Write 5 Done Sent Done false
18731 5971 Acked Read 8 Done Sent Done false
18732 5972 Acked Write 7 Done Sent Done false
18733 5973 Acked Write 1 Done Sent Done false
18734 5974 Acked Write 7 Done Sent Done false
18735 5975 Acked Write 7 Done Sent Done false
18736 5976 Acked Read 10 Done Sent Done false
18737 5977 Acked Read 8 Done Sent Done false
18738 5978 Acked Read 6 Done Sent Done false
18739 5979 Acked Write 7 Done Sent Done false
18740 5980 Acked Read 8 Done Sent Done false
18741 5981 Acked Read 7 Done Sent Done false
18742 5982 Acked Read 9 Done Sent Done false
18743 5983 Acked Write 10 Done Sent Done false
18744 5984 Acked Read 7 Done Sent Done false
18745 5985 Acked Write 3 Done Sent Done false
18746 5986 Acked Read 8 Done Sent Done false
18747 5987 Acked Read 10 Done Sent Done false
18748 5988 Acked Write 6 Done Sent Done false
18749 5989 Acked Read 9 Done Sent Done false
18750 5990 Acked Write 5 Done Sent Done false
18751 5991 Acked Read 10 Done Sent Done false
18752 5992 Acked Read 10 Done Sent Done false
18753 5993 Acked Read 4 Done Sent Done false
18754 5994 Acked Write 6 Done Sent Done false
18755 5995 Acked Write 3 Done Sent Done false
18756 5996 Acked Write 1 Done Sent Done false
18757 5997 Acked Read 9 Done Sent Done false
18758 5998 Acked Flush 0 Done Sent Done false
18759 5999 Acked Read 1 Done Sent Done false
18760 6000 Acked Write 9 Done Sent Done false
18761 6001 Acked Read 7 Done Sent Done false
18762 6002 Acked Read 4 Done Sent Done false
18763 6003 Acked Write 4 Done Sent Done false
18764 6004 Acked Read 8 Done Sent Done false
18765 6005 Acked Read 9 Done Sent Done false
18766 6006 Acked Write 1 Done Sent Done false
18767 6007 Acked Write 8 Done Sent Done false
18768 6008 Acked Read 9 Done Sent Done false
18769 6009 Acked Read 9 Done Sent Done false
18770 6010 Acked Read 9 Done Sent Done false
18771 6011 Acked Read 3 Done Sent Done false
18772 6012 Acked Read 7 Done Sent Done false
18773 6013 Acked Flush 0 Done Sent Done false
18774 6014 Acked Write 2 Done Sent Done false
18775 6015 Acked Write 9 Done Sent Done false
18776 6016 Acked Read 2 Done Sent Done false
18777 6017 Acked Read 8 Done Sent Done false
18778 6018 Acked Write 9 Done Sent Done false
18779 6019 Acked Write 7 Done Sent Done false
18780 6020 Acked Read 6 Done Sent Done false
18781 6021 Acked Read 2 Done Sent Done false
18782 6022 Acked Write 9 Done Sent Done false
18783 6023 Acked Write 4 Done Sent Done false
18784 6024 Acked Write 2 Done Sent Done false
18785 6025 Acked Read 1 Done Sent Done false
18786 6026 Acked Write 1 Done Sent Done false
18787 6027 Acked Read 5 Done Sent Done false
18788 6028 Acked Write 7 Done Sent Done false
18789 6029 Acked Read 4 Done Sent Done false
18790 6030 Acked Write 3 Done Sent Done false
18791 6031 Acked Read 3 Done Sent Done false
18792 6032 Acked Read 5 Done Sent Done false
18793 6033 Acked Read 1 Done Sent Done false
18794 6034 Acked Write 10 Done Sent Done false
18795 6035 Acked Read 10 Done Sent Done false
18796 6036 Acked Write 4 Done Sent Done false
18797 6037 Acked Write 6 Done Sent Done false
18798 6038 Acked Read 8 Done Sent Done false
18799 6039 Acked Read 1 Done Sent Done false
18800 6040 Acked Write 2 Done Sent Done false
18801 6041 Acked Read 8 Done Sent Done false
18802 6042 Acked Flush 0 Done Sent Done false
18803 6043 Acked Write 9 Done Sent Done false
18804 6044 Acked Write 1 Done Sent Done false
18805 6045 Acked Read 8 Done Sent Done false
18806 6046 Acked Read 1 Done Sent Done false
18807 6047 Acked Flush 0 Done Sent Done false
18808 6048 Acked Read 9 Done Sent Done false
18809 6049 Acked Read 2 Done Sent Done false
18810 6050 Acked Flush 0 Done Sent Done false
18811 6051 Acked Read 9 Done Sent Done false
18812 6052 Acked Flush 0 Done Sent Done false
18813 6053 Acked Flush 0 Done Sent Done false
18814 6054 Acked Write 3 Done Sent Done false
18815 6055 Acked Write 5 Done Sent Done false
18816 6056 Acked Write 1 Done Sent Done false
18817 6057 Acked Write 6 Done Sent Done false
18818 6058 Acked Read 1 Done Sent Done false
18819 6059 Acked Write 2 Done Sent Done false
18820 6060 Acked Read 2 Done Sent Done false
18821 6061 Acked Write 3 Done Sent Done false
18822 6062 Acked Write 2 Done Sent Done false
18823 6063 Acked Read 8 Done Sent Done false
18824 6064 Acked Flush 0 Done Sent Done false
18825 6065 Acked Write 1 Done Sent Done false
18826 6066 Acked Flush 0 Done Sent Done false
18827 6067 Acked Flush 0 Done Sent Done false
18828 6068 Acked Flush 0 Done Sent Done false
18829 6069 Acked Read 6 Done Sent Done false
18830 6070 Acked Read 8 Done Sent Done false
18831 6071 Acked Read 4 Done Sent Done false
18832 6072 Acked Write 5 Done Sent Done false
18833 6073 Acked Flush 0 Done Sent Done false
18834 6074 Acked Read 2 Done Sent Done false
18835 6075 Acked Read 7 Done Sent Done false
18836 6076 Acked Write 5 Done Sent Done false
18837 6077 Acked Write 2 Done Sent Done false
18838 6078 Acked Read 1 Done Sent Done false
18839 6079 Acked Read 7 Done Sent Done false
18840 6080 Acked Flush 0 Done Sent Done false
18841 6081 Acked Read 10 Done Sent Done false
18842 6082 Acked Read 2 Done Sent Done false
18843 6083 Acked Write 3 Done Sent Done false
18844 6084 Acked Write 10 Done Sent Done false
18845 6085 Acked Read 2 Done Sent Done false
18846 6086 Acked Read 7 Done Sent Done false
18847 6087 Acked Write 2 Done Sent Done false
18848 6088 Acked Write 2 Done Sent Done false
18849 6089 Acked Read 9 Done Sent Done false
18850 6090 Acked Write 10 Done Sent Done false
18851 6091 Acked Write 10 Done Sent Done false
18852 6092 Acked Write 9 Done Sent Done false
18853 6093 Acked Write 8 Done Sent Done false
18854 6094 Acked Read 2 Done Sent Done false
18855 6095 Acked Read 3 Done Sent Done false
18856 6096 Acked Read 4 Done Sent Done false
18857 6097 Acked Write 7 Done Sent Done false
18858 6098 Acked Write 3 Done Sent Done false
18859 6099 Acked Write 4 Done Sent Done false
18860 6100 Acked Read 9 Done Sent Done false
18861 6101 Acked Flush 0 Done Sent Done false
18862 6102 Acked Write 9 Done Sent Done false
18863 6103 Acked Write 4 Done Sent Done false
18864 6104 Acked Read 6 Done Sent Done false
18865 6105 Acked Write 10 Done Sent Done false
18866 6106 Acked Read 9 Done Sent Done false
18867 6107 Acked Flush 0 Done Sent Done false
18868 6108 Acked Read 7 Done Sent Done false
18869 6109 Acked Read 1 Done Sent Done false
18870 6110 Acked Write 5 Done Sent Done false
18871 6111 Acked Flush 0 Done Sent Done false
18872 6112 Acked Write 4 Done Sent Done false
18873 6113 Acked Write 8 Done Sent Done false
18874 6114 Acked Write 8 Done Sent Done false
18875 6115 Acked Write 3 Done Sent Done false
18876 6116 Acked Flush 0 Done Sent Done false
18877 6117 Acked Write 5 Done Sent Done false
18878 6118 Acked Write 5 Done Sent Done false
18879 6119 Acked Write 7 Done Sent Done false
18880 6120 Acked Read 10 Done Sent Done false
18881 6121 Acked Write 4 Done Sent Done false
18882 6122 Acked Flush 0 Done Sent Done false
18883 6123 Acked Flush 0 Done Sent Done false
18884 6124 Acked Read 5 Done Sent Done false
18885 6125 Acked Read 10 Done Sent Done false
18886 6126 Acked Write 3 Done Sent Done false
18887 6127 Acked Read 9 Done Sent Done false
18888 6128 Acked Read 10 Done Sent Done false
18889 6129 Acked Read 10 Done Sent Done false
18890 6130 Acked Write 3 Done Sent Done false
18891 STATES DS:0 DS:1 DS:2 TOTAL
18892 Sent 0 301 0 301
18893 Done 301 0 301 602
18894 Skipped 0 0 0 0
18895 Error 0 0 0 0
18896 Last Flush: 6123 5829 6123
18897 Downstairs last five completed: 5829 5828 5827 5826 5825
18898 Upstairs last five completed: 6130 6129 6128 6127 6126
18899 CLIENT: Up:0 ds:301 act:2
18900 [8820] got start action
18901 [8820] I am going to restart
18902 Starting downstairs at port 8820
18903 Make output file at "/tmp/dsc/downstairs-8820.txt"
18904 [8820][1] reports Starting
18905 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1217)
18906 [8820][1] reports Running
18907 ----------------------------------------------------------------
18908 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18909 States: Active Offline Active
18910 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18911 5830 Acked Flush 0 Done Sent Done false
18912 5831 Acked Read 1 Done Sent Done false
18913 5832 Acked Read 2 Done Sent Done false
18914 5833 Acked Write 2 Done Sent Done false
18915 5834 Acked Write 8 Done Sent Done false
18916 5835 Acked Flush 0 Done Sent Done false
18917 5836 Acked Write 5 Done Sent Done false
18918 5837 Acked Read 5 Done Sent Done false
18919 5838 Acked Flush 0 Done Sent Done false
18920 5839 Acked Write 1 Done Sent Done false
18921 5840 Acked Read 9 Done Sent Done false
18922 5841 Acked Write 1 Done Sent Done false
18923 5842 Acked Flush 0 Done Sent Done false
18924 5843 Acked Read 9 Done Sent Done false
18925 5844 Acked Write 8 Done Sent Done false
18926 5845 Acked Read 10 Done Sent Done false
18927 5846 Acked Read 4 Done Sent Done false
18928 5847 Acked Write 7 Done Sent Done false
18929 5848 Acked Write 5 Done Sent Done false
18930 5849 Acked Read 10 Done Sent Done false
18931 5850 Acked Read 10 Done Sent Done false
18932 5851 Acked Write 9 Done Sent Done false
18933 5852 Acked Flush 0 Done Sent Done false
18934 5853 Acked Write 7 Done Sent Done false
18935 5854 Acked Write 4 Done Sent Done false
18936 5855 Acked Read 2 Done Sent Done false
18937 5856 Acked Read 4 Done Sent Done false
18938 5857 Acked Flush 0 Done Sent Done false
18939 5858 Acked Write 1 Done Sent Done false
18940 5859 Acked Read 4 Done Sent Done false
18941 5860 Acked Write 1 Done Sent Done false
18942 5861 Acked Read 9 Done Sent Done false
18943 5862 Acked Write 4 Done Sent Done false
18944 5863 Acked Write 10 Done Sent Done false
18945 5864 Acked Read 7 Done Sent Done false
18946 5865 Acked Read 3 Done Sent Done false
18947 5866 Acked Write 1 Done Sent Done false
18948 5867 Acked Read 9 Done Sent Done false
18949 5868 Acked Write 2 Done Sent Done false
18950 5869 Acked Read 2 Done Sent Done false
18951 5870 Acked Read 4 Done Sent Done false
18952 5871 Acked Write 5 Done Sent Done false
18953 5872 Acked Read 8 Done Sent Done false
18954 5873 Acked Write 3 Done Sent Done false
18955 5874 Acked Write 2 Done Sent Done false
18956 5875 Acked Write 9 Done Sent Done false
18957 5876 Acked Read 6 Done Sent Done false
18958 5877 Acked Read 4 Done Sent Done false
18959 5878 Acked Read 10 Done Sent Done false
18960 5879 Acked Flush 0 Done Sent Done false
18961 5880 Acked Write 1 Done Sent Done false
18962 5881 Acked Read 3 Done Sent Done false
18963 5882 Acked Read 1 Done Sent Done false
18964 5883 Acked Write 8 Done Sent Done false
18965 5884 Acked Flush 0 Done Sent Done false
18966 5885 Acked Read 4 Done Sent Done false
18967 5886 Acked Read 2 Done Sent Done false
18968 5887 Acked Flush 0 Done Sent Done false
18969 5888 Acked Read 5 Done Sent Done false
18970 5889 Acked Read 4 Done Sent Done false
18971 5890 Acked Read 6 Done Sent Done false
18972 5891 Acked Read 4 Done Sent Done false
18973 5892 Acked Write 5 Done Sent Done false
18974 5893 Acked Write 1 Done Sent Done false
18975 5894 Acked Write 8 Done Sent Done false
18976 5895 Acked Read 1 Done Sent Done false
18977 5896 Acked Write 6 Done Sent Done false
18978 5897 Acked Read 4 Done Sent Done false
18979 5898 Acked Read 7 Done Sent Done false
18980 5899 Acked Flush 0 Done Sent Done false
18981 5900 Acked Write 1 Done Sent Done false
18982 5901 Acked Read 8 Done Sent Done false
18983 5902 Acked Read 5 Done Sent Done false
18984 5903 Acked Write 9 Done Sent Done false
18985 5904 Acked Read 9 Done Sent Done false
18986 5905 Acked Read 8 Done Sent Done false
18987 5906 Acked Flush 0 Done Sent Done false
18988 5907 Acked Write 4 Done Sent Done false
18989 5908 Acked Write 4 Done Sent Done false
18990 5909 Acked Write 4 Done Sent Done false
18991 5910 Acked Read 10 Done Sent Done false
18992 5911 Acked Write 6 Done Sent Done false
18993 5912 Acked Read 1 Done Sent Done false
18994 5913 Acked Flush 0 Done Sent Done false
18995 5914 Acked Read 1 Done Sent Done false
18996 5915 Acked Write 5 Done Sent Done false
18997 5916 Acked Write 3 Done Sent Done false
18998 5917 Acked Read 7 Done Sent Done false
18999 5918 Acked Write 5 Done Sent Done false
19000 5919 Acked Flush 0 Done Sent Done false
19001 5920 Acked Write 5 Done Sent Done false
19002 5921 Acked Read 1 Done Sent Done false
19003 5922 Acked Read 3 Done Sent Done false
19004 5923 Acked Read 6 Done Sent Done false
19005 5924 Acked Write 8 Done Sent Done false
19006 5925 Acked Read 9 Done Sent Done false
19007 5926 Acked Read 1 Done Sent Done false
19008 5927 Acked Read 4 Done Sent Done false
19009 5928 Acked Read 3 Done Sent Done false
19010 5929 Acked Write 4 Done Sent Done false
19011 5930 Acked Write 2 Done Sent Done false
19012 5931 Acked Read 8 Done Sent Done false
19013 5932 Acked Write 7 Done Sent Done false
19014 5933 Acked Write 9 Done Sent Done false
19015 5934 Acked Write 6 Done Sent Done false
19016 5935 Acked Read 9 Done Sent Done false
19017 5936 Acked Flush 0 Done Sent Done false
19018 5937 Acked Read 9 Done Sent Done false
19019 5938 Acked Write 6 Done Sent Done false
19020 5939 Acked Read 5 Done Sent Done false
19021 5940 Acked Write 7 Done Sent Done false
19022 5941 Acked Write 5 Done Sent Done false
19023 5942 Acked Read 1 Done Sent Done false
19024 5943 Acked Read 3 Done Sent Done false
19025 5944 Acked Read 4 Done Sent Done false
19026 5945 Acked Write 10 Done Sent Done false
19027 5946 Acked Write 9 Done Sent Done false
19028 5947 Acked Read 3 Done Sent Done false
19029 5948 Acked Write 8 Done Sent Done false
19030 5949 Acked Flush 0 Done Sent Done false
19031 5950 Acked Write 5 Done Sent Done false
19032 5951 Acked Flush 0 Done Sent Done false
19033 5952 Acked Read 7 Done Sent Done false
19034 5953 Acked Read 7 Done Sent Done false
19035 5954 Acked Flush 0 Done Sent Done false
19036 5955 Acked Write 7 Done Sent Done false
19037 5956 Acked Flush 0 Done Sent Done false
19038 5957 Acked Write 3 Done Sent Done false
19039 5958 Acked Write 8 Done Sent Done false
19040 5959 Acked Read 5 Done Sent Done false
19041 5960 Acked Read 3 Done Sent Done false
19042 5961 Acked Write 7 Done Sent Done false
19043 5962 Acked Read 3 Done Sent Done false
19044 5963 Acked Read 1 Done Sent Done false
19045 5964 Acked Read 9 Done Sent Done false
19046 5965 Acked Write 7 Done Sent Done false
19047 5966 Acked Read 4 Done Sent Done false
19048 5967 Acked Read 3 Done Sent Done false
19049 5968 Acked Write 9 Done Sent Done false
19050 5969 Acked Write 4 Done Sent Done false
19051 5970 Acked Write 5 Done Sent Done false
19052 5971 Acked Read 8 Done Sent Done false
19053 5972 Acked Write 7 Done Sent Done false
19054 5973 Acked Write 1 Done Sent Done false
19055 5974 Acked Write 7 Done Sent Done false
19056 5975 Acked Write 7 Done Sent Done false
19057 5976 Acked Read 10 Done Sent Done false
19058 5977 Acked Read 8 Done Sent Done false
19059 5978 Acked Read 6 Done Sent Done false
19060 5979 Acked Write 7 Done Sent Done false
19061 5980 Acked Read 8 Done Sent Done false
19062 5981 Acked Read 7 Done Sent Done false
19063 5982 Acked Read 9 Done Sent Done false
19064 5983 Acked Write 10 Done Sent Done false
19065 5984 Acked Read 7 Done Sent Done false
19066 5985 Acked Write 3 Done Sent Done false
19067 5986 Acked Read 8 Done Sent Done false
19068 5987 Acked Read 10 Done Sent Done false
19069 5988 Acked Write 6 Done Sent Done false
19070 5989 Acked Read 9 Done Sent Done false
19071 5990 Acked Write 5 Done Sent Done false
19072 5991 Acked Read 10 Done Sent Done false
19073 5992 Acked Read 10 Done Sent Done false
19074 5993 Acked Read 4 Done Sent Done false
19075 5994 Acked Write 6 Done Sent Done false
19076 5995 Acked Write 3 Done Sent Done false
19077 5996 Acked Write 1 Done Sent Done false
19078 5997 Acked Read 9 Done Sent Done false
19079 5998 Acked Flush 0 Done Sent Done false
19080 5999 Acked Read 1 Done Sent Done false
19081 6000 Acked Write 9 Done Sent Done false
19082 6001 Acked Read 7 Done Sent Done false
19083 6002 Acked Read 4 Done Sent Done false
19084 6003 Acked Write 4 Done Sent Done false
19085 6004 Acked Read 8 Done Sent Done false
19086 6005 Acked Read 9 Done Sent Done false
19087 6006 Acked Write 1 Done Sent Done false
19088 6007 Acked Write 8 Done Sent Done false
19089 6008 Acked Read 9 Done Sent Done false
19090 6009 Acked Read 9 Done Sent Done false
19091 6010 Acked Read 9 Done Sent Done false
19092 6011 Acked Read 3 Done Sent Done false
19093 6012 Acked Read 7 Done Sent Done false
19094 6013 Acked Flush 0 Done Sent Done false
19095 6014 Acked Write 2 Done Sent Done false
19096 6015 Acked Write 9 Done Sent Done false
19097 6016 Acked Read 2 Done Sent Done false
19098 6017 Acked Read 8 Done Sent Done false
19099 6018 Acked Write 9 Done Sent Done false
19100 6019 Acked Write 7 Done Sent Done false
19101 6020 Acked Read 6 Done Sent Done false
19102 6021 Acked Read 2 Done Sent Done false
19103 6022 Acked Write 9 Done Sent Done false
19104 6023 Acked Write 4 Done Sent Done false
19105 6024 Acked Write 2 Done Sent Done false
19106 6025 Acked Read 1 Done Sent Done false
19107 6026 Acked Write 1 Done Sent Done false
19108 6027 Acked Read 5 Done Sent Done false
19109 6028 Acked Write 7 Done Sent Done false
19110 6029 Acked Read 4 Done Sent Done false
19111 6030 Acked Write 3 Done Sent Done false
19112 6031 Acked Read 3 Done Sent Done false
19113 6032 Acked Read 5 Done Sent Done false
19114 6033 Acked Read 1 Done Sent Done false
19115 6034 Acked Write 10 Done Sent Done false
19116 6035 Acked Read 10 Done Sent Done false
19117 6036 Acked Write 4 Done Sent Done false
19118 6037 Acked Write 6 Done Sent Done false
19119 6038 Acked Read 8 Done Sent Done false
19120 6039 Acked Read 1 Done Sent Done false
19121 6040 Acked Write 2 Done Sent Done false
19122 6041 Acked Read 8 Done Sent Done false
19123 6042 Acked Flush 0 Done Sent Done false
19124 6043 Acked Write 9 Done Sent Done false
19125 6044 Acked Write 1 Done Sent Done false
19126 6045 Acked Read 8 Done Sent Done false
19127 6046 Acked Read 1 Done Sent Done false
19128 6047 Acked Flush 0 Done Sent Done false
19129 6048 Acked Read 9 Done Sent Done false
19130 6049 Acked Read 2 Done Sent Done false
19131 6050 Acked Flush 0 Done Sent Done false
19132 6051 Acked Read 9 Done Sent Done false
19133 6052 Acked Flush 0 Done Sent Done false
19134 6053 Acked Flush 0 Done Sent Done false
19135 6054 Acked Write 3 Done Sent Done false
19136 6055 Acked Write 5 Done Sent Done false
19137 6056 Acked Write 1 Done Sent Done false
19138 6057 Acked Write 6 Done Sent Done false
19139 6058 Acked Read 1 Done Sent Done false
19140 6059 Acked Write 2 Done Sent Done false
19141 6060 Acked Read 2 Done Sent Done false
19142 6061 Acked Write 3 Done Sent Done false
19143 6062 Acked Write 2 Done Sent Done false
19144 6063 Acked Read 8 Done Sent Done false
19145 6064 Acked Flush 0 Done Sent Done false
19146 6065 Acked Write 1 Done Sent Done false
19147 6066 Acked Flush 0 Done Sent Done false
19148 6067 Acked Flush 0 Done Sent Done false
19149 6068 Acked Flush 0 Done Sent Done false
19150 6069 Acked Read 6 Done Sent Done false
19151 6070 Acked Read 8 Done Sent Done false
19152 6071 Acked Read 4 Done Sent Done false
19153 6072 Acked Write 5 Done Sent Done false
19154 6073 Acked Flush 0 Done Sent Done false
19155 6074 Acked Read 2 Done Sent Done false
19156 6075 Acked Read 7 Done Sent Done false
19157 6076 Acked Write 5 Done Sent Done false
19158 6077 Acked Write 2 Done Sent Done false
19159 6078 Acked Read 1 Done Sent Done false
19160 6079 Acked Read 7 Done Sent Done false
19161 6080 Acked Flush 0 Done Sent Done false
19162 6081 Acked Read 10 Done Sent Done false
19163 6082 Acked Read 2 Done Sent Done false
19164 6083 Acked Write 3 Done Sent Done false
19165 6084 Acked Write 10 Done Sent Done false
19166 6085 Acked Read 2 Done Sent Done false
19167 6086 Acked Read 7 Done Sent Done false
19168 6087 Acked Write 2 Done Sent Done false
19169 6088 Acked Write 2 Done Sent Done false
19170 6089 Acked Read 9 Done Sent Done false
19171 6090 Acked Write 10 Done Sent Done false
19172 6091 Acked Write 10 Done Sent Done false
19173 6092 Acked Write 9 Done Sent Done false
19174 6093 Acked Write 8 Done Sent Done false
19175 6094 Acked Read 2 Done Sent Done false
19176 6095 Acked Read 3 Done Sent Done false
19177 6096 Acked Read 4 Done Sent Done false
19178 6097 Acked Write 7 Done Sent Done false
19179 6098 Acked Write 3 Done Sent Done false
19180 6099 Acked Write 4 Done Sent Done false
19181 6100 Acked Read 9 Done Sent Done false
19182 6101 Acked Flush 0 Done Sent Done false
19183 6102 Acked Write 9 Done Sent Done false
19184 6103 Acked Write 4 Done Sent Done false
19185 6104 Acked Read 6 Done Sent Done false
19186 6105 Acked Write 10 Done Sent Done false
19187 6106 Acked Read 9 Done Sent Done false
19188 6107 Acked Flush 0 Done Sent Done false
19189 6108 Acked Read 7 Done Sent Done false
19190 6109 Acked Read 1 Done Sent Done false
19191 6110 Acked Write 5 Done Sent Done false
19192 6111 Acked Flush 0 Done Sent Done false
19193 6112 Acked Write 4 Done Sent Done false
19194 6113 Acked Write 8 Done Sent Done false
19195 6114 Acked Write 8 Done Sent Done false
19196 6115 Acked Write 3 Done Sent Done false
19197 6116 Acked Flush 0 Done Sent Done false
19198 6117 Acked Write 5 Done Sent Done false
19199 6118 Acked Write 5 Done Sent Done false
19200 6119 Acked Write 7 Done Sent Done false
19201 6120 Acked Read 10 Done Sent Done false
19202 6121 Acked Write 4 Done Sent Done false
19203 6122 Acked Flush 0 Done Sent Done false
19204 6123 Acked Flush 0 Done Sent Done false
19205 6124 Acked Read 5 Done Sent Done false
19206 6125 Acked Read 10 Done Sent Done false
19207 6126 Acked Write 3 Done Sent Done false
19208 6127 Acked Read 9 Done Sent Done false
19209 6128 Acked Read 10 Done Sent Done false
19210 6129 Acked Read 10 Done Sent Done false
19211 6130 Acked Write 3 Done Sent Done false
19212 6131 Acked Flush 0 Done Sent Done false
19213 STATES DS:0 DS:1 DS:2 TOTAL
19214 Sent 0 302 0 302
19215 Done 302 0 302 604
19216 Skipped 0 0 0 0
19217 Error 0 0 0 0
19218 Last Flush: 6131 5829 6131
19219 Downstairs last five completed: 5829 5828 5827 5826 5825
19220 Upstairs last five completed: 6131 6130 6129 6128 6127
19221 CLIENT: Up:0 ds:302 act:2
192222025-01-17T21:15:06.389ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
192232025-01-17T21:15:06.389ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:35144) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
192242025-01-17T21:15:06.390ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
192252025-01-17T21:15:06.391ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
192262025-01-17T21:15:06.391ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
192272025-01-17T21:15:06.391ZINFOcrucible: send last flush ID to this DS: 5829 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
192282025-01-17T21:15:06.391ZINFOcrucible: Replied this last flush ID: 5829 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
192292025-01-17T21:15:06.391ZINFOcrucible: [1] client re-new 302 jobs since flush 5829 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
192302025-01-17T21:15:06.392ZINFOcrucible: [1] Replayed 302 jobs since flush: 5829 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
19231 ----------------------------------------------------------------
19232 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
19233 Downstairs last five completed: 6131 6130 6129 6128 6127
19234 Upstairs last five completed: 6131 6130 6129 6128 6127
19235 CLIENT: Up:0 ds:0 act:3
19236 Replay: All jobs finished
19237 Main task has work to do, go find it
19238 got dsc Stop(2)
19239 stop 2
19240 Jan 17 21:15:08.630[8830] Got stop action so:false kr:false
19241 INFO request completed, latency_us: 343, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 01672b1d-a4aa-4ef4-b4f5-0e9518e1bd22, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
192422025-01-17T21:15:08.631ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
192432025-01-17T21:15:08.631ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
192442025-01-17T21:15:08.631ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
19245 Jan 17 21:15:08.631 INFO request completed, latency_us: 141, response_code: 200, uri: /state/cid/2, method: GET, req_id: 2bd87f80-f629-413c-ab29-34f88b9c3d33, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
192462025-01-17T21:15:08.631ZINFOcrucible: client task is exiting = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
19247 [8830] Exited with: ExitStatus(unix_wait_status(9))
19248 [8830][2] reports Exit
19249 Jan 17 21:15:12.633 INFO request completed, latency_us: 182, response_code: 200, uri: /state/cid/2, method: GET, req_id: 508b2737-ad9a-435d-b3c5-84e6a50b782f, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
19250 001/300 Read block 2420 len 32768
19251 002/300 Write block 2530 len 20480 data: 2 4 4 4 4
19252 003/300 Read block 4991 len 4096
19253 004/300 Write block 1880 len 36864 data: 4 4 5 5 5 6 6 7 7
19254 005/300 Write block 764 len 12288 data: 4 4 5
19255 006/300 Write block 965 len 36864 data: 5 4 4 4 5 5 5 6 5
19256 007/300 Flush
19257 008/300 Read block 1804 len 12288
19258 009/300 Write block 562 len 20480 data: 6 5 4 3 3
19259 010/300 Flush
19260 011/300 Read block 2869 len 24576
19261 012/300 Flush
19262 013/300 Read block 2676 len 4096
19263 014/300 Write block 4131 len 28672 data: 5 5 4 4 3 2 4
19264 015/300 Read block 4359 len 8192
19265 016/300 Read block 957 len 28672
19266 017/300 Flush
19267 018/300 Write block 1253 len 28672 data: 4 3 3 5 5 3 3
19268 019/300 Read block 3468 len 12288
19269 020/300 Read block 32 len 36864
19270 021/300 Write block 738 len 32768 data: 4 3 4 4 4 3 4 3
19271 022/300 Read block 1714 len 32768
19272 023/300 Read block 656 len 32768
19273 024/300 Write block 4555 len 16384 data: 5 6 6 4
19274 025/300 Read block 241 len 24576
19275 026/300 Read block 3187 len 20480
19276 027/300 Read block 2404 len 12288
19277 028/300 Write block 4624 len 32768 data: 5 4 3 3 2 2 2 2
19278 029/300 Read block 3900 len 12288
19279 030/300 Read block 282 len 4096
19280 031/300 Write block 2081 len 12288 data: 2 3 3
19281 032/300 Write block 4216 len 28672 data: 6 7 5 4 5 5 4
19282 033/300 Read block 1245 len 40960
19283 034/300 Write block 1368 len 20480 data: 4 4 3 3 2
19284 035/300 Read block 505 len 12288
19285 036/300 Read block 2465 len 40960
19286 037/300 Write block 4356 len 36864 data: 3 3 2 2 3 3 4 5 6
19287 038/300 Read block 1753 len 20480
19288 039/300 Flush
19289 040/300 Read block 1793 len 28672
19290 041/300 Read block 31 len 16384
19291 042/300 Read block 3619 len 36864
19292 043/300 Write block 4553 len 32768 data: 7 6 6 7 7 5 4 4
19293 044/300 Write block 3836 len 20480 data: 6 5 5 6 6
19294 045/300 Read block 4031 len 28672
19295 046/300 Write block 2320 len 20480 data: 5 4 6 5 5
19296 047/300 Flush
19297 048/300 Write block 2221 len 12288 data: 5 5 5
19298 049/300 Read block 3457 len 36864
19299 050/300 Write block 371 len 8192 data: 2 2
19300 051/300 Read block 2066 len 32768
19301 052/300 Write block 3114 len 8192 data: 3 4
19302 053/300 Read block 1080 len 16384
19303 054/300 Flush
19304 055/300 Read block 2225 len 12288
19305 056/300 Flush
19306 057/300 Read block 1629 len 16384
19307 058/300 Read block 2808 len 12288
19308 059/300 Write block 2346 len 32768 data: 8 6 6 7 7 8 6 4
19309 060/300 Read block 2572 len 20480
19310 061/300 Read block 1143 len 20480
19311 062/300 Read block 2968 len 12288
19312 063/300 Read block 3178 len 40960
19313 064/300 Flush
19314 065/300 Write block 3908 len 32768 data: 4 4 4 4 4 3 3 3
19315 066/300 Write block 94 len 24576 data: 4 3 4 3 3 5
19316 067/300 Read block 2275 len 36864
19317 068/300 Read block 1816 len 20480
19318 069/300 Write block 2202 len 28672 data: 3 3 3 3 3 3 2
19319 070/300 Write block 3748 len 40960 data: 4 4 3 2 2 2 4 4 5 5
19320 071/300 Read block 3898 len 24576
19321 072/300 Write block 4526 len 8192 data: 3 3
19322 073/300 Write block 4557 len 36864 data: 8 6 5 5 4 2 2 3 2
19323 074/300 Write block 1169 len 12288 data: 4 4 3
19324 075/300 Read block 2876 len 8192
19325 076/300 Write block 413 len 36864 data: 2 2 2 3 3 4 5 5 5
19326 077/300 Flush
19327 078/300 Write block 3346 len 36864 data: 4 3 3 2 2 3 3 4 4
19328 079/300 Write block 4002 len 12288 data: 4 5 7
19329 080/300 Flush
19330 081/300 Read block 1906 len 12288
19331 082/300 Write block 3690 len 36864 data: 4 3 4 4 4 4 2 2 2
19332 083/300 Read block 2413 len 40960
19333 084/300 Write block 4015 len 4096 data: 4
19334 085/300 Read block 3261 len 36864
19335 086/300 Read block 3694 len 32768
19336 087/300 Read block 4862 len 40960
19337 088/300 Write block 2699 len 12288 data: 3 2 2
19338 089/300 Write block 4268 len 8192 data: 5 5
19339 090/300 Write block 360 len 28672 data: 5 5 6 7 7 4 4
19340 091/300 Read block 3064 len 12288
19341 092/300 Write block 809 len 36864 data: 4 4 4 6 6 5 4 5 4
19342 093/300 Write block 2431 len 40960 data: 5 5 4 4 4 3 2 2 2 3
19343 094/300 Write block 1952 len 40960 data: 6 6 6 6 4 4 4 5 5 4
19344 095/300 Write block 2783 len 32768 data: 3 4 4 4 5 4 4 7
19345 096/300 Read block 1825 len 24576
19346 097/300 Read block 694 len 4096
19347 098/300 Write block 3381 len 12288 data: 3 3 3
19348 099/300 Flush
19349 100/300 Write block 2869 len 8192 data: 4 5
19350 101/300 Read block 1039 len 20480
19351 102/300 Write block 3661 len 20480 data: 2 2 2 4 4
19352 103/300 Read block 3158 len 24576
19353 104/300 Read block 1601 len 16384
19354 105/300 Write block 862 len 16384 data: 6 4 4 2
19355 106/300 Read block 2759 len 4096
19356 107/300 Write block 3592 len 20480 data: 2 2 3 5 4
19357 108/300 Read block 911 len 12288
19358 109/300 Flush
19359 110/300 Read block 1943 len 4096
19360 111/300 Write block 4608 len 32768 data: 6 6 6 6 5 4 4 3
19361 112/300 Write block 4521 len 20480 data: 2 2 2 3 3
19362 113/300 Read block 2231 len 28672
19363 114/300 Read block 519 len 12288
19364 115/300 Read block 4086 len 20480
19365 116/300 Write block 1559 len 28672 data: 5 5 3 4 4 5 5
19366 117/300 Write block 3732 len 8192 data: 4 3
19367 118/300 Read block 710 len 16384
19368 119/300 Flush
19369 120/300 Read block 2167 len 8192
19370 121/300 Write block 1261 len 36864 data: 4 4 2 2 2 2 3 4 4
19371 122/300 Read block 3173 len 28672
19372 123/300 Write block 4454 len 16384 data: 5 5 4 3
19373 124/300 Read block 4875 len 36864
19374 125/300 Write block 1787 len 16384 data: 5 5 5 6
19375 126/300 Write block 1980 len 8192 data: 3 5
19376 127/300 Write block 4 len 12288 data: 4 3 3
19377 128/300 Write block 2474 len 12288 data: 6 5 5
19378 129/300 Write block 1884 len 36864 data: 6 7 7 8 8 5 5 5 5
19379 130/300 Read block 2190 len 24576
19380 131/300 Read block 1637 len 36864
19381 132/300 Flush
19382 133/300 Write block 3695 len 32768 data: 5 3 3 3 3 4 5 5
19383 134/300 Read block 3139 len 16384
19384 135/300 Write block 4008 len 36864 data: 9 9 7 7 5 6 4 5 4
19385 136/300 Write block 2458 len 24576 data: 2 2 3 4 4 4
19386 137/300 Read block 1827 len 4096
19387 138/300 Flush
19388 139/300 Write block 2125 len 8192 data: 4 4
19389 140/300 Read block 4865 len 12288
19390 141/300 Write block 3933 len 32768 data: 9 8 7 7 5 6 5 5
19391 142/300 Read block 4517 len 36864
19392 143/300 Write block 3618 len 8192 data: 7 6
19393 144/300 Write block 3748 len 28672 data: 5 5 4 3 3 3 5
19394 145/300 Write block 2010 len 8192 data: 3 4
19395 146/300 Read block 3934 len 20480
19396 147/300 Flush
19397 148/300 Read block 2268 len 20480
19398 149/300 Write block 3088 len 28672 data: 4 4 3 3 3 3 3
19399 150/300 Write block 4033 len 4096 data: 4
19400 151/300 Write block 4214 len 16384 data: 4 5 7 8
19401 152/300 Write block 3362 len 12288 data: 3 3 2
19402 153/300 Flush
19403 154/300 Read block 2379 len 8192
19404 155/300 Read block 2346 len 20480
19405 156/300 Read block 3596 len 24576
19406 157/300 Flush
19407 158/300 Write block 4412 len 8192 data: 4 5
19408 159/300 Read block 114 len 24576
19409 160/300 Read block 2917 len 4096
19410 161/300 Read block 2293 len 20480
19411 162/300 Write block 958 len 28672 data: 5 4 5 5 6 4 5
19412 163/300 Read block 3175 len 8192
19413 164/300 Write block 3630 len 16384 data: 3 2 2 4
19414 165/300 Read block 4523 len 20480
19415 166/300 Read block 781 len 12288
19416 167/300 Read block 74 len 8192
19417 168/300 Read block 4623 len 24576
19418 169/300 Flush
19419 170/300 Write block 3089 len 8192 data: 5 4
19420 171/300 Read block 3425 len 12288
19421 172/300 Write block 3702 len 36864 data: 6 6 7 6 6 8 8 6 4
19422 173/300 Read block 1850 len 40960
19423 174/300 Read block 1511 len 24576
19424 175/300 Read block 814 len 8192
19425 176/300 Read block 4633 len 32768
19426 177/300 Flush
19427 178/300 Write block 1992 len 4096 data: 5
19428 179/300 Read block 4223 len 20480
19429 180/300 Read block 1114 len 4096
19430 181/300 Write block 697 len 40960 data: 3 4 4 3 3 3 4 2 2 3
19431 182/300 Write block 1032 len 40960 data: 5 5 5 7 7 6 3 3 2 2
19432 183/300 Read block 4893 len 12288
19433 184/300 Read block 3198 len 8192
19434 185/300 Read block 3311 len 4096
19435 186/300 Flush
19436 187/300 Write block 997 len 36864 data: 3 3 3 3 3 3 2 3 3
19437 188/300 Read block 1761 len 12288
19438 189/300 Write block 992 len 32768 data: 4 4 3 3 3 4 4 4
19439 190/300 Read block 2789 len 36864
19440 191/300 Write block 4105 len 4096 data: 3
19441 192/300 Read block 2061 len 16384
19442 193/300 Write block 2044 len 28672 data: 4 4 4 5 4 4 6
19443 194/300 Flush
19444 195/300 Write block 2441 len 36864 data: 3 3 3 3 4 4 4 4 4
19445 196/300 Read block 2469 len 20480
19446 197/300 Read block 4916 len 32768
19447 198/300 Write block 2760 len 40960 data: 3 3 2 3 3 3 3 4 4 4
19448 199/300 Write block 360 len 20480 data: 6 6 7 8 8
19449 200/300 Read block 1029 len 40960
19450 201/300 Write block 2039 len 4096 data: 3
19451 202/300 Write block 1918 len 12288 data: 6 5 6
19452 203/300 Write block 3038 len 4096 data: 4
19453 204/300 Read block 4671 len 20480
19454 205/300 Write block 4250 len 36864 data: 2 2 4 4 5 2 2 2 2
19455 206/300 Write block 4647 len 12288 data: 2 2 2
19456 207/300 Write block 4585 len 8192 data: 2 2
19457 208/300 Write block 4294 len 8192 data: 6 6
19458 209/300 Flush
19459 210/300 Read block 558 len 32768
19460 211/300 Write block 827 len 28672 data: 5 5 5 5 5 5 5
19461 212/300 Flush
19462 213/300 Read block 3455 len 16384
19463 214/300 Write block 216 len 40960 data: 3 3 2 2 3 3 3 4 4 4
19464 215/300 Read block 3395 len 36864
19465 216/300 Flush
19466 217/300 Read block 1371 len 28672
19467 218/300 Read block 135 len 24576
19468 219/300 Flush
19469 220/300 Write block 4400 len 40960 data: 5 5 5 4 4 3 3 3 4 3
19470 221/300 Flush
19471 222/300 Read block 3998 len 32768
19472 223/300 Write block 3716 len 40960 data: 5 5 4 4 3 3 4 3 4 4
19473 224/300 Read block 2989 len 12288
19474 225/300 Write block 1434 len 36864 data: 4 4 4 5 6 6 6 6 7
19475 226/300 Read block 2198 len 20480
19476 227/300 Read block 804 len 20480
19477 228/300 Read block 2970 len 36864
19478 229/300 Read block 338 len 8192
19479 230/300 Read block 1294 len 16384
19480 231/300 Read block 3626 len 4096
19481 232/300 Read block 4177 len 12288
19482 233/300 Flush
19483 234/300 Write block 1237 len 4096 data: 4
19484 235/300 Read block 2250 len 28672
19485 236/300 Write block 2787 len 32768 data: 6 5 5 8 8 9 7 6
19486 237/300 Write block 1989 len 32768 data: 7 6 5 6 3 3 2 2
19487 238/300 Read block 2734 len 32768
19488 239/300 Read block 4518 len 36864
19489 240/300 Write block 2737 len 36864 data: 2 2 3 3 3 3 4 4 4
19490 241/300 Write block 4086 len 16384 data: 3 3 3 3
19491 242/300 Flush
19492 243/300 Read block 3673 len 36864
19493 244/300 Flush
19494 245/300 Read block 3191 len 36864
19495 246/300 Flush
19496 247/300 Read block 4978 len 40960
19497 248/300 Read block 414 len 32768
19498 249/300 Read block 3876 len 28672
19499 250/300 Write block 3615 len 8192 data: 3 6
19500 251/300 Write block 1070 len 40960 data: 4 4 3 3 3 3 3 3 3 4
19501 252/300 Read block 4187 len 20480
19502 253/300 Write block 518 len 12288 data: 4 3 4
19503 254/300 Read block 2204 len 20480
19504 255/300 Read block 2802 len 32768
19505 256/300 Write block 2287 len 40960 data: 3 5 5 5 6 6 5 6 6 6
19506 257/300 Read block 2337 len 24576
19507 258/300 Write block 4879 len 16384 data: 4 4 3 4
19508 259/300 Write block 286 len 40960 data: 6 6 6 6 4 4 4 5 5 4
19509 260/300 Read block 738 len 20480
19510 261/300 Read block 1792 len 40960
19511 262/300 Write block 2544 len 28672 data: 5 5 5 6 8 6 7
19512 263/300 Write block 1223 len 40960 data: 3 2 3 3 3 3 4 5 6 5
19513 264/300 Read block 1399 len 40960
19514 265/300 Read block 274 len 28672
19515 266/300 Read block 687 len 16384
19516 267/300 Flush
19517 268/300 Flush
19518 269/300 Read block 1505 len 4096
19519 270/300 Write block 4542 len 16384 data: 3 3 4 5
19520 271/300 Write block 3138 len 4096 data: 5
19521 272/300 Write block 699 len 4096 data: 5
19522 273/300 Read block 3764 len 16384
19523 274/300 Write block 4134 len 28672 data: 5 4 3 5 5 6 6
19524 275/300 Write block 2872 len 12288 data: 5 4 4
19525 276/300 Flush
19526 277/300 Read block 850 len 8192
19527 278/300 Read block 183 len 36864
19528 279/300 Write block 828 len 4096 data: 6
19529 280/300 Read block 4140 len 16384
19530 281/300 Read block 4861 len 28672
19531 282/300 Read block 1167 len 8192
19532 283/300 Read block 4335 len 40960
19533 284/300 Flush
19534 285/300 Write block 4882 len 16384 data: 5 4 4 5
19535 286/300 Read block 3661 len 4096
19536 287/300 Write block 964 len 32768 data: 6 6 5 5 5 6 6 6
19537 288/300 Read block 1621 len 16384
19538 289/300 Read block 2696 len 32768
19539 290/300 Read block 1042 len 12288
19540 291/300 Write block 3681 len 36864 data: 6 5 4 4 3 3 3 3 4
19541 292/300 Read block 2326 len 36864
19542 293/300 Read block 3366 len 36864
19543 294/300 Read block 3876 len 24576
19544 295/300 Flush
19545 296/300 Read block 4018 len 4096
19546 297/300 Read block 2769 len 4096
19547 298/300 Read block 2858 len 28672
19548 299/300 Write block 3107 len 40960 data: 6 6 5 5 4 3 3 4 5 4
19549 300/300 Read block 787 len 4096
19550 301/300 Flush
19551 Main task has work to do, go find it
19552 got dsc Start(2)
19553 start 2
19554 Jan 17 21:15:12.875 INFO request completed, latency_us: 205, response_code: 204, uri: /start/cid/2, method: POST, req_id: 75fb2560-35d3-4cea-a29c-30a62b30447c, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
19555 Replay: started 2, returned:Ok(())
19556 ----------------------------------------------------------------
19557 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
19558 States: Active Active Offline
19559 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19560 6132 Acked Read 8 Done Done Sent false
19561 6133 Acked Write 5 Done Done Sent false
19562 6134 Acked Read 1 Done Done Sent false
19563 6135 Acked Write 9 Done Done Sent false
19564 6136 Acked Write 3 Done Done Sent false
19565 6137 Acked Write 9 Done Done Sent false
19566 6138 Acked Flush 0 Done Done Sent false
19567 6139 Acked Read 3 Done Done Sent false
19568 6140 Acked Write 5 Done Done Sent false
19569 6141 Acked Flush 0 Done Done Sent false
19570 6142 Acked Read 6 Done Done Sent false
19571 6143 Acked Flush 0 Done Done Sent false
19572 6144 Acked Read 1 Done Done Sent false
19573 6145 Acked Write 7 Done Done Sent false
19574 6146 Acked Read 2 Done Done Sent false
19575 6147 Acked Read 7 Done Done Sent false
19576 6148 Acked Flush 0 Done Done Sent false
19577 6149 Acked Write 7 Done Done Sent false
19578 6150 Acked Read 3 Done Done Sent false
19579 6151 Acked Read 9 Done Done Sent false
19580 6152 Acked Write 8 Done Done Sent false
19581 6153 Acked Read 8 Done Done Sent false
19582 6154 Acked Read 8 Done Done Sent false
19583 6155 Acked Write 4 Done Done Sent false
19584 6156 Acked Read 6 Done Done Sent false
19585 6157 Acked Read 5 Done Done Sent false
19586 6158 Acked Read 3 Done Done Sent false
19587 6159 Acked Write 8 Done Done Sent false
19588 6160 Acked Read 3 Done Done Sent false
19589 6161 Acked Read 1 Done Done Sent false
19590 6162 Acked Write 3 Done Done Sent false
19591 6163 Acked Write 7 Done Done Sent false
19592 6164 Acked Read 10 Done Done Sent false
19593 6165 Acked Write 5 Done Done Sent false
19594 6166 Acked Read 3 Done Done Sent false
19595 6167 Acked Read 10 Done Done Sent false
19596 6168 Acked Write 9 Done Done Sent false
19597 6169 Acked Read 5 Done Done Sent false
19598 6170 Acked Flush 0 Done Done Sent false
19599 6171 Acked Read 7 Done Done Sent false
19600 6172 Acked Read 4 Done Done Sent false
19601 6173 Acked Read 9 Done Done Sent false
19602 6174 Acked Write 8 Done Done Sent false
19603 6175 Acked Write 5 Done Done Sent false
19604 6176 Acked Read 7 Done Done Sent false
19605 6177 Acked Write 5 Done Done Sent false
19606 6178 Acked Flush 0 Done Done Sent false
19607 6179 Acked Write 3 Done Done Sent false
19608 6180 Acked Read 9 Done Done Sent false
19609 6181 Acked Write 2 Done Done Sent false
19610 6182 Acked Read 8 Done Done Sent false
19611 6183 Acked Write 2 Done Done Sent false
19612 6184 Acked Read 4 Done Done Sent false
19613 6185 Acked Flush 0 Done Done Sent false
19614 6186 Acked Read 3 Done Done Sent false
19615 6187 Acked Flush 0 Done Done Sent false
19616 6188 Acked Read 4 Done Done Sent false
19617 6189 Acked Read 3 Done Done Sent false
19618 6190 Acked Write 8 Done Done Sent false
19619 6191 Acked Read 5 Done Done Sent false
19620 6192 Acked Read 5 Done Done Sent false
19621 6193 Acked Read 3 Done Done Sent false
19622 6194 Acked Read 10 Done Done Sent false
19623 6195 Acked Flush 0 Done Done Sent false
19624 6196 Acked Write 8 Done Done Sent false
19625 6197 Acked Write 6 Done Done Sent false
19626 6198 Acked Read 9 Done Done Sent false
19627 6199 Acked Read 5 Done Done Sent false
19628 6200 Acked Write 7 Done Done Sent false
19629 6201 Acked Write 10 Done Done Sent false
19630 6202 Acked Read 6 Done Done Sent false
19631 6203 Acked Write 2 Done Done Sent false
19632 6204 Acked Write 9 Done Done Sent false
19633 6205 Acked Write 3 Done Done Sent false
19634 6206 Acked Read 2 Done Done Sent false
19635 6207 Acked Write 9 Done Done Sent false
19636 6208 Acked Flush 0 Done Done Sent false
19637 6209 Acked Write 9 Done Done Sent false
19638 6210 Acked Write 3 Done Done Sent false
19639 6211 Acked Flush 0 Done Done Sent false
19640 6212 Acked Read 3 Done Done Sent false
19641 6213 Acked Write 9 Done Done Sent false
19642 6214 Acked Read 10 Done Done Sent false
19643 6215 Acked Write 1 Done Done Sent false
19644 6216 Acked Read 9 Done Done Sent false
19645 6217 Acked Read 8 Done Done Sent false
19646 6218 Acked Read 10 Done Done Sent false
19647 6219 Acked Write 3 Done Done Sent false
19648 6220 Acked Write 2 Done Done Sent false
19649 6221 Acked Write 7 Done Done Sent false
19650 6222 Acked Read 3 Done Done Sent false
19651 6223 Acked Write 9 Done Done Sent false
19652 6224 Acked Write 10 Done Done Sent false
19653 6225 Acked Write 10 Done Done Sent false
19654 6226 Acked Write 8 Done Done Sent false
19655 6227 Acked Read 6 Done Done Sent false
19656 6228 Acked Read 1 Done Done Sent false
19657 6229 Acked Write 3 Done Done Sent false
19658 6230 Acked Flush 0 Done Done Sent false
19659 6231 Acked Write 2 Done Done Sent false
19660 6232 Acked Read 5 Done Done Sent false
19661 6233 Acked Write 5 Done Done Sent false
19662 6234 Acked Read 6 Done Done Sent false
19663 6235 Acked Read 4 Done Done Sent false
19664 6236 Acked Write 4 Done Done Sent false
19665 6237 Acked Read 1 Done Done Sent false
19666 6238 Acked Write 5 Done Done Sent false
19667 6239 Acked Read 3 Done Done Sent false
19668 6240 Acked Flush 0 Done Done Sent false
19669 6241 Acked Read 1 Done Done Sent false
19670 6242 Acked Write 8 Done Done Sent false
19671 6243 Acked Write 5 Done Done Sent false
19672 6244 Acked Read 7 Done Done Sent false
19673 6245 Acked Read 3 Done Done Sent false
19674 6246 Acked Read 5 Done Done Sent false
19675 6247 Acked Write 7 Done Done Sent false
19676 6248 Acked Write 2 Done Done Sent false
19677 6249 Acked Read 4 Done Done Sent false
19678 6250 Acked Flush 0 Done Done Sent false
19679 6251 Acked Read 2 Done Done Sent false
19680 6252 Acked Write 9 Done Done Sent false
19681 6253 Acked Read 7 Done Done Sent false
19682 6254 Acked Write 4 Done Done Sent false
19683 6255 Acked Read 9 Done Done Sent false
19684 6256 Acked Write 4 Done Done Sent false
19685 6257 Acked Write 2 Done Done Sent false
19686 6258 Acked Write 3 Done Done Sent false
19687 6259 Acked Write 3 Done Done Sent false
19688 6260 Acked Write 9 Done Done Sent false
19689 6261 Acked Read 6 Done Done Sent false
19690 6262 Acked Read 9 Done Done Sent false
19691 6263 Acked Flush 0 Done Done Sent false
19692 6264 Acked Write 8 Done Done Sent false
19693 6265 Acked Read 4 Done Done Sent false
19694 6266 Acked Write 9 Done Done Sent false
19695 6267 Acked Write 6 Done Done Sent false
19696 6268 Acked Read 1 Done Done Sent false
19697 6269 Acked Flush 0 Done Done Sent false
19698 6270 Acked Write 2 Done Done Sent false
19699 6271 Acked Read 3 Done Done Sent false
19700 6272 Acked Write 8 Done Done Sent false
19701 6273 Acked Read 9 Done Done Sent false
19702 6274 Acked Write 2 Done Done Sent false
19703 6275 Acked Write 7 Done Done Sent false
19704 6276 Acked Write 2 Done Done Sent false
19705 6277 Acked Read 5 Done Done Sent false
19706 6278 Acked Flush 0 Done Done Sent false
19707 6279 Acked Read 5 Done Done Sent false
19708 6280 Acked Write 7 Done Done Sent false
19709 6281 Acked Write 1 Done Done Sent false
19710 6282 Acked Write 4 Done Done Sent false
19711 6283 Acked Write 3 Done Done Sent false
19712 6284 Acked Flush 0 Done Done Sent false
19713 6285 Acked Read 2 Done Done Sent false
19714 6286 Acked Read 5 Done Done Sent false
19715 6287 Acked Read 6 Done Done Sent false
19716 6288 Acked Flush 0 Done Done Sent false
19717 6289 Acked Write 2 Done Done Sent false
19718 6290 Acked Read 6 Done Done Sent false
19719 6291 Acked Read 1 Done Done Sent false
19720 6292 Acked Read 5 Done Done Sent false
19721 6293 Acked Write 7 Done Done Sent false
19722 6294 Acked Read 2 Done Done Sent false
19723 6295 Acked Write 4 Done Done Sent false
19724 6296 Acked Read 5 Done Done Sent false
19725 6297 Acked Read 3 Done Done Sent false
19726 6298 Acked Read 2 Done Done Sent false
19727 6299 Acked Read 6 Done Done Sent false
19728 6300 Acked Flush 0 Done Done Sent false
19729 6301 Acked Write 2 Done Done Sent false
19730 6302 Acked Read 3 Done Done Sent false
19731 6303 Acked Write 9 Done Done Sent false
19732 6304 Acked Read 10 Done Done Sent false
19733 6305 Acked Read 6 Done Done Sent false
19734 6306 Acked Read 2 Done Done Sent false
19735 6307 Acked Read 8 Done Done Sent false
19736 6308 Acked Flush 0 Done Done Sent false
19737 6309 Acked Write 1 Done Done Sent false
19738 6310 Acked Read 5 Done Done Sent false
19739 6311 Acked Read 1 Done Done Sent false
19740 6312 Acked Write 10 Done Done Sent false
19741 6313 Acked Write 10 Done Done Sent false
19742 6314 Acked Read 3 Done Done Sent false
19743 6315 Acked Read 2 Done Done Sent false
19744 6316 Acked Read 1 Done Done Sent false
19745 6317 Acked Flush 0 Done Done Sent false
19746 6318 Acked Write 9 Done Done Sent false
19747 6319 Acked Read 3 Done Done Sent false
19748 6320 Acked Write 8 Done Done Sent false
19749 6321 Acked Read 9 Done Done Sent false
19750 6322 Acked Write 1 Done Done Sent false
19751 6323 Acked Read 4 Done Done Sent false
19752 6324 Acked Write 7 Done Done Sent false
19753 6325 Acked Flush 0 Done Done Sent false
19754 6326 Acked Write 9 Done Done Sent false
19755 6327 Acked Read 5 Done Done Sent false
19756 6328 Acked Read 8 Done Done Sent false
19757 6329 Acked Write 10 Done Done Sent false
19758 6330 Acked Write 5 Done Done Sent false
19759 6331 Acked Read 10 Done Done Sent false
19760 6332 Acked Write 1 Done Done Sent false
19761 6333 Acked Write 3 Done Done Sent false
19762 6334 Acked Write 1 Done Done Sent false
19763 6335 Acked Read 5 Done Done Sent false
19764 6336 Acked Write 9 Done Done Sent false
19765 6337 Acked Write 3 Done Done Sent false
19766 6338 Acked Write 2 Done Done Sent false
19767 6339 Acked Write 2 Done Done Sent false
19768 6340 Acked Flush 0 Done Done Sent false
19769 6341 Acked Read 8 Done Done Sent false
19770 6342 Acked Write 7 Done Done Sent false
19771 6343 Acked Flush 0 Done Done Sent false
19772 6344 Acked Read 4 Done Done Sent false
19773 6345 Acked Write 10 Done Done Sent false
19774 6346 Acked Read 9 Done Done Sent false
19775 6347 Acked Flush 0 Done Done Sent false
19776 6348 Acked Read 7 Done Done Sent false
19777 6349 Acked Read 6 Done Done Sent false
19778 6350 Acked Flush 0 Done Done Sent false
19779 6351 Acked Write 10 Done Done Sent false
19780 6352 Acked Flush 0 Done Done Sent false
19781 6353 Acked Read 8 Done Done Sent false
19782 6354 Acked Write 10 Done Done Sent false
19783 6355 Acked Read 3 Done Done Sent false
19784 6356 Acked Write 9 Done Done Sent false
19785 6357 Acked Read 5 Done Done Sent false
19786 6358 Acked Read 5 Done Done Sent false
19787 6359 Acked Read 9 Done Done Sent false
19788 6360 Acked Read 2 Done Done Sent false
19789 6361 Acked Read 4 Done Done Sent false
19790 6362 Acked Read 1 Done Done Sent false
19791 6363 Acked Read 3 Done Done Sent false
19792 6364 Acked Flush 0 Done Done Sent false
19793 6365 Acked Write 1 Done Done Sent false
19794 6366 Acked Read 7 Done Done Sent false
19795 6367 Acked Write 8 Done Done Sent false
19796 6368 Acked Write 8 Done Done Sent false
19797 6369 Acked Read 8 Done Done Sent false
19798 6370 Acked Read 9 Done Done Sent false
19799 6371 Acked Write 9 Done Done Sent false
19800 6372 Acked Write 4 Done Done Sent false
19801 6373 Acked Flush 0 Done Done Sent false
19802 6374 Acked Read 9 Done Done Sent false
19803 6375 Acked Flush 0 Done Done Sent false
19804 6376 Acked Read 9 Done Done Sent false
19805 6377 Acked Flush 0 Done Done Sent false
19806 6378 Acked Read 10 Done Done Sent false
19807 6379 Acked Read 8 Done Done Sent false
19808 6380 Acked Read 7 Done Done Sent false
19809 6381 Acked Write 2 Done Done Sent false
19810 6382 Acked Write 10 Done Done Sent false
19811 6383 Acked Read 5 Done Done Sent false
19812 6384 Acked Write 3 Done Done Sent false
19813 6385 Acked Read 5 Done Done Sent false
19814 6386 Acked Read 8 Done Done Sent false
19815 6387 Acked Write 10 Done Done Sent false
19816 6388 Acked Read 6 Done Done Sent false
19817 6389 Acked Write 4 Done Done Sent false
19818 6390 Acked Write 10 Done Done Sent false
19819 6391 Acked Read 5 Done Done Sent false
19820 6392 Acked Read 10 Done Done Sent false
19821 6393 Acked Write 7 Done Done Sent false
19822 6394 Acked Write 10 Done Done Sent false
19823 6395 Acked Read 10 Done Done Sent false
19824 6396 Acked Read 7 Done Done Sent false
19825 6397 Acked Read 4 Done Done Sent false
19826 6398 Acked Flush 0 Done Done Sent false
19827 6399 Acked Flush 0 Done Done Sent false
19828 6400 Acked Read 1 Done Done Sent false
19829 6401 Acked Write 4 Done Done Sent false
19830 6402 Acked Write 1 Done Done Sent false
19831 6403 Acked Write 1 Done Done Sent false
19832 6404 Acked Read 4 Done Done Sent false
19833 6405 Acked Write 7 Done Done Sent false
19834 6406 Acked Write 3 Done Done Sent false
19835 6407 Acked Flush 0 Done Done Sent false
19836 6408 Acked Read 2 Done Done Sent false
19837 6409 Acked Read 9 Done Done Sent false
19838 6410 Acked Write 1 Done Done Sent false
19839 6411 Acked Read 4 Done Done Sent false
19840 6412 Acked Read 7 Done Done Sent false
19841 6413 Acked Read 2 Done Done Sent false
19842 6414 Acked Read 10 Done Done Sent false
19843 6415 Acked Flush 0 Done Done Sent false
19844 6416 Acked Write 4 Done Done Sent false
19845 6417 Acked Read 1 Done Done Sent false
19846 6418 Acked Write 8 Done Done Sent false
19847 6419 Acked Read 4 Done Done Sent false
19848 6420 Acked Read 8 Done Done Sent false
19849 6421 Acked Read 3 Done Done Sent false
19850 6422 Acked Write 9 Done Done Sent false
19851 6423 Acked Read 9 Done Done Sent false
19852 6424 Acked Read 9 Done Done Sent false
19853 6425 Acked Read 6 Done Done Sent false
19854 6426 Acked Flush 0 Done Done Sent false
19855 6427 Acked Read 1 Done Done Sent false
19856 6428 Acked Read 1 Done Done Sent false
19857 6429 Acked Read 7 Done Done Sent false
19858 6430 Acked Write 10 Done Done Sent false
19859 6431 Acked Read 1 Done Done Sent false
19860 6432 Acked Flush 0 Done Done Sent false
19861 STATES DS:0 DS:1 DS:2 TOTAL
19862 Sent 0 0 301 301
19863 Done 301 301 0 602
19864 Skipped 0 0 0 0
19865 Error 0 0 0 0
19866 Last Flush: 6432 6432 6131
19867 Downstairs last five completed: 6131 6130 6129 6128 6127
19868 Upstairs last five completed: 6432 6431 6430 6429 6428
19869 CLIENT: Up:0 ds:301 act:2
19870 [8830] got start action
19871 [8830] I am going to restart
19872 Starting downstairs at port 8830
19873 Make output file at "/tmp/dsc/downstairs-8830.txt"
19874 [8830][2] reports Starting
19875 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1218)
19876 [8830][2] reports Running
19877 ----------------------------------------------------------------
19878 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
19879 States: Active Active Offline
19880 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19881 6132 Acked Read 8 Done Done Sent false
19882 6133 Acked Write 5 Done Done Sent false
19883 6134 Acked Read 1 Done Done Sent false
19884 6135 Acked Write 9 Done Done Sent false
19885 6136 Acked Write 3 Done Done Sent false
19886 6137 Acked Write 9 Done Done Sent false
19887 6138 Acked Flush 0 Done Done Sent false
19888 6139 Acked Read 3 Done Done Sent false
19889 6140 Acked Write 5 Done Done Sent false
19890 6141 Acked Flush 0 Done Done Sent false
19891 6142 Acked Read 6 Done Done Sent false
19892 6143 Acked Flush 0 Done Done Sent false
19893 6144 Acked Read 1 Done Done Sent false
19894 6145 Acked Write 7 Done Done Sent false
19895 6146 Acked Read 2 Done Done Sent false
19896 6147 Acked Read 7 Done Done Sent false
19897 6148 Acked Flush 0 Done Done Sent false
19898 6149 Acked Write 7 Done Done Sent false
19899 6150 Acked Read 3 Done Done Sent false
19900 6151 Acked Read 9 Done Done Sent false
19901 6152 Acked Write 8 Done Done Sent false
19902 6153 Acked Read 8 Done Done Sent false
19903 6154 Acked Read 8 Done Done Sent false
19904 6155 Acked Write 4 Done Done Sent false
19905 6156 Acked Read 6 Done Done Sent false
19906 6157 Acked Read 5 Done Done Sent false
19907 6158 Acked Read 3 Done Done Sent false
19908 6159 Acked Write 8 Done Done Sent false
19909 6160 Acked Read 3 Done Done Sent false
19910 6161 Acked Read 1 Done Done Sent false
19911 6162 Acked Write 3 Done Done Sent false
19912 6163 Acked Write 7 Done Done Sent false
19913 6164 Acked Read 10 Done Done Sent false
19914 6165 Acked Write 5 Done Done Sent false
19915 6166 Acked Read 3 Done Done Sent false
19916 6167 Acked Read 10 Done Done Sent false
19917 6168 Acked Write 9 Done Done Sent false
19918 6169 Acked Read 5 Done Done Sent false
19919 6170 Acked Flush 0 Done Done Sent false
19920 6171 Acked Read 7 Done Done Sent false
19921 6172 Acked Read 4 Done Done Sent false
19922 6173 Acked Read 9 Done Done Sent false
19923 6174 Acked Write 8 Done Done Sent false
19924 6175 Acked Write 5 Done Done Sent false
19925 6176 Acked Read 7 Done Done Sent false
19926 6177 Acked Write 5 Done Done Sent false
19927 6178 Acked Flush 0 Done Done Sent false
19928 6179 Acked Write 3 Done Done Sent false
19929 6180 Acked Read 9 Done Done Sent false
19930 6181 Acked Write 2 Done Done Sent false
19931 6182 Acked Read 8 Done Done Sent false
19932 6183 Acked Write 2 Done Done Sent false
19933 6184 Acked Read 4 Done Done Sent false
19934 6185 Acked Flush 0 Done Done Sent false
19935 6186 Acked Read 3 Done Done Sent false
19936 6187 Acked Flush 0 Done Done Sent false
19937 6188 Acked Read 4 Done Done Sent false
19938 6189 Acked Read 3 Done Done Sent false
19939 6190 Acked Write 8 Done Done Sent false
19940 6191 Acked Read 5 Done Done Sent false
19941 6192 Acked Read 5 Done Done Sent false
19942 6193 Acked Read 3 Done Done Sent false
19943 6194 Acked Read 10 Done Done Sent false
19944 6195 Acked Flush 0 Done Done Sent false
19945 6196 Acked Write 8 Done Done Sent false
19946 6197 Acked Write 6 Done Done Sent false
19947 6198 Acked Read 9 Done Done Sent false
19948 6199 Acked Read 5 Done Done Sent false
19949 6200 Acked Write 7 Done Done Sent false
19950 6201 Acked Write 10 Done Done Sent false
19951 6202 Acked Read 6 Done Done Sent false
19952 6203 Acked Write 2 Done Done Sent false
19953 6204 Acked Write 9 Done Done Sent false
19954 6205 Acked Write 3 Done Done Sent false
19955 6206 Acked Read 2 Done Done Sent false
19956 6207 Acked Write 9 Done Done Sent false
19957 6208 Acked Flush 0 Done Done Sent false
19958 6209 Acked Write 9 Done Done Sent false
19959 6210 Acked Write 3 Done Done Sent false
19960 6211 Acked Flush 0 Done Done Sent false
19961 6212 Acked Read 3 Done Done Sent false
19962 6213 Acked Write 9 Done Done Sent false
19963 6214 Acked Read 10 Done Done Sent false
19964 6215 Acked Write 1 Done Done Sent false
19965 6216 Acked Read 9 Done Done Sent false
19966 6217 Acked Read 8 Done Done Sent false
19967 6218 Acked Read 10 Done Done Sent false
19968 6219 Acked Write 3 Done Done Sent false
19969 6220 Acked Write 2 Done Done Sent false
19970 6221 Acked Write 7 Done Done Sent false
19971 6222 Acked Read 3 Done Done Sent false
19972 6223 Acked Write 9 Done Done Sent false
19973 6224 Acked Write 10 Done Done Sent false
19974 6225 Acked Write 10 Done Done Sent false
19975 6226 Acked Write 8 Done Done Sent false
19976 6227 Acked Read 6 Done Done Sent false
19977 6228 Acked Read 1 Done Done Sent false
19978 6229 Acked Write 3 Done Done Sent false
19979 6230 Acked Flush 0 Done Done Sent false
19980 6231 Acked Write 2 Done Done Sent false
19981 6232 Acked Read 5 Done Done Sent false
19982 6233 Acked Write 5 Done Done Sent false
19983 6234 Acked Read 6 Done Done Sent false
19984 6235 Acked Read 4 Done Done Sent false
19985 6236 Acked Write 4 Done Done Sent false
19986 6237 Acked Read 1 Done Done Sent false
19987 6238 Acked Write 5 Done Done Sent false
19988 6239 Acked Read 3 Done Done Sent false
19989 6240 Acked Flush 0 Done Done Sent false
19990 6241 Acked Read 1 Done Done Sent false
19991 6242 Acked Write 8 Done Done Sent false
19992 6243 Acked Write 5 Done Done Sent false
19993 6244 Acked Read 7 Done Done Sent false
19994 6245 Acked Read 3 Done Done Sent false
19995 6246 Acked Read 5 Done Done Sent false
19996 6247 Acked Write 7 Done Done Sent false
19997 6248 Acked Write 2 Done Done Sent false
19998 6249 Acked Read 4 Done Done Sent false
19999 6250 Acked Flush 0 Done Done Sent false
20000 6251 Acked Read 2 Done Done Sent false
20001 6252 Acked Write 9 Done Done Sent false
20002 6253 Acked Read 7 Done Done Sent false
20003 6254 Acked Write 4 Done Done Sent false
20004 6255 Acked Read 9 Done Done Sent false
20005 6256 Acked Write 4 Done Done Sent false
20006 6257 Acked Write 2 Done Done Sent false
20007 6258 Acked Write 3 Done Done Sent false
20008 6259 Acked Write 3 Done Done Sent false
20009 6260 Acked Write 9 Done Done Sent false
20010 6261 Acked Read 6 Done Done Sent false
20011 6262 Acked Read 9 Done Done Sent false
20012 6263 Acked Flush 0 Done Done Sent false
20013 6264 Acked Write 8 Done Done Sent false
20014 6265 Acked Read 4 Done Done Sent false
20015 6266 Acked Write 9 Done Done Sent false
20016 6267 Acked Write 6 Done Done Sent false
20017 6268 Acked Read 1 Done Done Sent false
20018 6269 Acked Flush 0 Done Done Sent false
20019 6270 Acked Write 2 Done Done Sent false
20020 6271 Acked Read 3 Done Done Sent false
20021 6272 Acked Write 8 Done Done Sent false
20022 6273 Acked Read 9 Done Done Sent false
20023 6274 Acked Write 2 Done Done Sent false
20024 6275 Acked Write 7 Done Done Sent false
20025 6276 Acked Write 2 Done Done Sent false
20026 6277 Acked Read 5 Done Done Sent false
20027 6278 Acked Flush 0 Done Done Sent false
20028 6279 Acked Read 5 Done Done Sent false
20029 6280 Acked Write 7 Done Done Sent false
20030 6281 Acked Write 1 Done Done Sent false
20031 6282 Acked Write 4 Done Done Sent false
20032 6283 Acked Write 3 Done Done Sent false
20033 6284 Acked Flush 0 Done Done Sent false
20034 6285 Acked Read 2 Done Done Sent false
20035 6286 Acked Read 5 Done Done Sent false
20036 6287 Acked Read 6 Done Done Sent false
20037 6288 Acked Flush 0 Done Done Sent false
20038 6289 Acked Write 2 Done Done Sent false
20039 6290 Acked Read 6 Done Done Sent false
20040 6291 Acked Read 1 Done Done Sent false
20041 6292 Acked Read 5 Done Done Sent false
20042 6293 Acked Write 7 Done Done Sent false
20043 6294 Acked Read 2 Done Done Sent false
20044 6295 Acked Write 4 Done Done Sent false
20045 6296 Acked Read 5 Done Done Sent false
20046 6297 Acked Read 3 Done Done Sent false
20047 6298 Acked Read 2 Done Done Sent false
20048 6299 Acked Read 6 Done Done Sent false
20049 6300 Acked Flush 0 Done Done Sent false
20050 6301 Acked Write 2 Done Done Sent false
20051 6302 Acked Read 3 Done Done Sent false
20052 6303 Acked Write 9 Done Done Sent false
20053 6304 Acked Read 10 Done Done Sent false
20054 6305 Acked Read 6 Done Done Sent false
20055 6306 Acked Read 2 Done Done Sent false
20056 6307 Acked Read 8 Done Done Sent false
20057 6308 Acked Flush 0 Done Done Sent false
20058 6309 Acked Write 1 Done Done Sent false
20059 6310 Acked Read 5 Done Done Sent false
20060 6311 Acked Read 1 Done Done Sent false
20061 6312 Acked Write 10 Done Done Sent false
20062 6313 Acked Write 10 Done Done Sent false
20063 6314 Acked Read 3 Done Done Sent false
20064 6315 Acked Read 2 Done Done Sent false
20065 6316 Acked Read 1 Done Done Sent false
20066 6317 Acked Flush 0 Done Done Sent false
20067 6318 Acked Write 9 Done Done Sent false
20068 6319 Acked Read 3 Done Done Sent false
20069 6320 Acked Write 8 Done Done Sent false
20070 6321 Acked Read 9 Done Done Sent false
20071 6322 Acked Write 1 Done Done Sent false
20072 6323 Acked Read 4 Done Done Sent false
20073 6324 Acked Write 7 Done Done Sent false
20074 6325 Acked Flush 0 Done Done Sent false
20075 6326 Acked Write 9 Done Done Sent false
20076 6327 Acked Read 5 Done Done Sent false
20077 6328 Acked Read 8 Done Done Sent false
20078 6329 Acked Write 10 Done Done Sent false
20079 6330 Acked Write 5 Done Done Sent false
20080 6331 Acked Read 10 Done Done Sent false
20081 6332 Acked Write 1 Done Done Sent false
20082 6333 Acked Write 3 Done Done Sent false
20083 6334 Acked Write 1 Done Done Sent false
20084 6335 Acked Read 5 Done Done Sent false
20085 6336 Acked Write 9 Done Done Sent false
20086 6337 Acked Write 3 Done Done Sent false
20087 6338 Acked Write 2 Done Done Sent false
20088 6339 Acked Write 2 Done Done Sent false
20089 6340 Acked Flush 0 Done Done Sent false
20090 6341 Acked Read 8 Done Done Sent false
20091 6342 Acked Write 7 Done Done Sent false
20092 6343 Acked Flush 0 Done Done Sent false
20093 6344 Acked Read 4 Done Done Sent false
20094 6345 Acked Write 10 Done Done Sent false
20095 6346 Acked Read 9 Done Done Sent false
20096 6347 Acked Flush 0 Done Done Sent false
20097 6348 Acked Read 7 Done Done Sent false
20098 6349 Acked Read 6 Done Done Sent false
20099 6350 Acked Flush 0 Done Done Sent false
20100 6351 Acked Write 10 Done Done Sent false
20101 6352 Acked Flush 0 Done Done Sent false
20102 6353 Acked Read 8 Done Done Sent false
20103 6354 Acked Write 10 Done Done Sent false
20104 6355 Acked Read 3 Done Done Sent false
20105 6356 Acked Write 9 Done Done Sent false
20106 6357 Acked Read 5 Done Done Sent false
20107 6358 Acked Read 5 Done Done Sent false
20108 6359 Acked Read 9 Done Done Sent false
20109 6360 Acked Read 2 Done Done Sent false
20110 6361 Acked Read 4 Done Done Sent false
20111 6362 Acked Read 1 Done Done Sent false
20112 6363 Acked Read 3 Done Done Sent false
20113 6364 Acked Flush 0 Done Done Sent false
20114 6365 Acked Write 1 Done Done Sent false
20115 6366 Acked Read 7 Done Done Sent false
20116 6367 Acked Write 8 Done Done Sent false
20117 6368 Acked Write 8 Done Done Sent false
20118 6369 Acked Read 8 Done Done Sent false
20119 6370 Acked Read 9 Done Done Sent false
20120 6371 Acked Write 9 Done Done Sent false
20121 6372 Acked Write 4 Done Done Sent false
20122 6373 Acked Flush 0 Done Done Sent false
20123 6374 Acked Read 9 Done Done Sent false
20124 6375 Acked Flush 0 Done Done Sent false
20125 6376 Acked Read 9 Done Done Sent false
20126 6377 Acked Flush 0 Done Done Sent false
20127 6378 Acked Read 10 Done Done Sent false
20128 6379 Acked Read 8 Done Done Sent false
20129 6380 Acked Read 7 Done Done Sent false
20130 6381 Acked Write 2 Done Done Sent false
20131 6382 Acked Write 10 Done Done Sent false
20132 6383 Acked Read 5 Done Done Sent false
20133 6384 Acked Write 3 Done Done Sent false
20134 6385 Acked Read 5 Done Done Sent false
20135 6386 Acked Read 8 Done Done Sent false
20136 6387 Acked Write 10 Done Done Sent false
20137 6388 Acked Read 6 Done Done Sent false
20138 6389 Acked Write 4 Done Done Sent false
20139 6390 Acked Write 10 Done Done Sent false
20140 6391 Acked Read 5 Done Done Sent false
20141 6392 Acked Read 10 Done Done Sent false
20142 6393 Acked Write 7 Done Done Sent false
20143 6394 Acked Write 10 Done Done Sent false
20144 6395 Acked Read 10 Done Done Sent false
20145 6396 Acked Read 7 Done Done Sent false
20146 6397 Acked Read 4 Done Done Sent false
20147 6398 Acked Flush 0 Done Done Sent false
20148 6399 Acked Flush 0 Done Done Sent false
20149 6400 Acked Read 1 Done Done Sent false
20150 6401 Acked Write 4 Done Done Sent false
20151 6402 Acked Write 1 Done Done Sent false
20152 6403 Acked Write 1 Done Done Sent false
20153 6404 Acked Read 4 Done Done Sent false
20154 6405 Acked Write 7 Done Done Sent false
20155 6406 Acked Write 3 Done Done Sent false
20156 6407 Acked Flush 0 Done Done Sent false
20157 6408 Acked Read 2 Done Done Sent false
20158 6409 Acked Read 9 Done Done Sent false
20159 6410 Acked Write 1 Done Done Sent false
20160 6411 Acked Read 4 Done Done Sent false
20161 6412 Acked Read 7 Done Done Sent false
20162 6413 Acked Read 2 Done Done Sent false
20163 6414 Acked Read 10 Done Done Sent false
20164 6415 Acked Flush 0 Done Done Sent false
20165 6416 Acked Write 4 Done Done Sent false
20166 6417 Acked Read 1 Done Done Sent false
20167 6418 Acked Write 8 Done Done Sent false
20168 6419 Acked Read 4 Done Done Sent false
20169 6420 Acked Read 8 Done Done Sent false
20170 6421 Acked Read 3 Done Done Sent false
20171 6422 Acked Write 9 Done Done Sent false
20172 6423 Acked Read 9 Done Done Sent false
20173 6424 Acked Read 9 Done Done Sent false
20174 6425 Acked Read 6 Done Done Sent false
20175 6426 Acked Flush 0 Done Done Sent false
20176 6427 Acked Read 1 Done Done Sent false
20177 6428 Acked Read 1 Done Done Sent false
20178 6429 Acked Read 7 Done Done Sent false
20179 6430 Acked Write 10 Done Done Sent false
20180 6431 Acked Read 1 Done Done Sent false
20181 6432 Acked Flush 0 Done Done Sent false
20182 STATES DS:0 DS:1 DS:2 TOTAL
20183 Sent 0 0 301 301
20184 Done 301 301 0 602
20185 Skipped 0 0 0 0
20186 Error 0 0 0 0
20187 Last Flush: 6432 6432 6131
20188 Downstairs last five completed: 6131 6130 6129 6128 6127
20189 Upstairs last five completed: 6432 6431 6430 6429 6428
20190 CLIENT: Up:0 ds:301 act:2
201912025-01-17T21:15:18.633ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
201922025-01-17T21:15:18.633ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57821) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
201932025-01-17T21:15:18.634ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
201942025-01-17T21:15:18.635ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
201952025-01-17T21:15:18.635ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
201962025-01-17T21:15:18.635ZINFOcrucible: send last flush ID to this DS: 6131 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
201972025-01-17T21:15:18.635ZINFOcrucible: Replied this last flush ID: 6131 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
201982025-01-17T21:15:18.635ZINFOcrucible: [2] client re-new 301 jobs since flush 6131 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
201992025-01-17T21:15:18.635ZINFOcrucible: [2] Replayed 301 jobs since flush: 6131 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
20200 ----------------------------------------------------------------
20201 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
20202 Downstairs last five completed: 6432 6431 6430 6429 6428
20203 Upstairs last five completed: 6432 6431 6430 6429 6428
20204 CLIENT: Up:0 ds:0 act:3
20205 Replay: All jobs finished
20206 Main task has work to do, go find it
20207 got dsc Stop(2)
20208 stop 2
20209 Jan 17 21:15:20.880 [8830] Got stop action so:false kr:false
20210 INFO request completed, latency_us: 312, response_code: 204, uri: /stop/cid/2, method: POST, req_id: f53bc631-9ead-48f4-a1dc-00ef481af5bf, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
202112025-01-17T21:15:20.881ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
202122025-01-17T21:15:20.881ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
202132025-01-17T21:15:20.881ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
202142025-01-17T21:15:20.881ZINFOcrucible: client task is exiting = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
20215 Jan 17 21:15:20.881 INFO request completed, latency_us: 192, response_code: 200, uri: /state/cid/2, method: GET, req_id: f07fcdf3-d6ef-4b0f-b790-0cd640015b34, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
20216 [8830] Exited with: ExitStatus(unix_wait_status(9))
20217 [8830][2] reports Exit
20218 Jan 17 21:15:24.883 INFO request completed, latency_us: 186, response_code: 200, uri: /state/cid/2, method: GET, req_id: 8175a3db-e697-4b42-b94b-6a860edee37e, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
20219 001/300 Write block 4808 len 20480 data: 6 5 4 3 4
20220 002/300 Read block 2141 len 12288
20221 003/300 Read block 4938 len 8192
20222 004/300 Read block 3551 len 20480
20223 005/300 Read block 2540 len 20480
20224 006/300 Write block 1348 len 8192 data: 4 3
20225 007/300 Write block 3378 len 12288 data: 3 3 3
20226 008/300 Read block 220 len 32768
20227 009/300 Write block 3473 len 16384 data: 6 6 5 4
20228 010/300 Write block 3436 len 28672 data: 3 4 3 3 4 4 4
20229 011/300 Read block 2577 len 36864
20230 012/300 Read block 1164 len 16384
20231 013/300 Read block 299 len 24576
20232 014/300 Write block 383 len 40960 data: 5 5 5 6 5 5 6 7 8 8
20233 015/300 Write block 1609 len 4096 data: 2
20234 016/300 Write block 506 len 28672 data: 4 5 6 6 7 7 5
20235 017/300 Read block 4112 len 24576
20236 018/300 Write block 2916 len 24576 data: 7 7 7 6 5 4
20237 019/300 Write block 365 len 32768 data: 5 5 4 4 3 2 3 3
20238 020/300 Read block 3882 len 36864
20239 021/300 Read block 168 len 12288
20240 022/300 Read block 4355 len 12288
20241 023/300 Read block 3725 len 28672
20242 024/300 Read block 2784 len 8192
20243 025/300 Read block 3111 len 20480
20244 026/300 Read block 1987 len 4096
20245 027/300 Write block 4658 len 40960 data: 4 4 4 3 5 4 4 4 5 5
20246 028/300 Read block 3351 len 16384
20247 029/300 Write block 2367 len 4096 data: 2
20248 030/300 Read block 2527 len 28672
20249 031/300 Write block 1793 len 8192 data: 6 6
20250 032/300 Read block 3087 len 4096
20251 033/300 Read block 15 len 20480
20252 034/300 Flush
20253 035/300 Read block 186 len 32768
20254 036/300 Read block 2594 len 12288
20255 037/300 Read block 3713 len 40960
20256 038/300 Write block 4904 len 4096 data: 3
20257 039/300 Read block 1234 len 8192
20258 040/300 Write block 1259 len 28672 data: 4 3 5 5 3 3 3
20259 041/300 Write block 3122 len 24576 data: 5 5 4 4 3 4
20260 042/300 Read block 4643 len 8192
20261 043/300 Write block 1325 len 4096 data: 4
20262 044/300 Write block 3621 len 36864 data: 6 5 5 5 4 2 3 3 4
20263 045/300 Write block 3990 len 32768 data: 6 7 8 9 8 9 9 8
20264 046/300 Write block 3889 len 20480 data: 4 4 3 4 5
20265 047/300 Read block 4894 len 32768
20266 048/300 Read block 4861 len 12288
20267 049/300 Write block 2572 len 4096 data: 3
20268 050/300 Write block 1553 len 20480 data: 2 3 3 3 5
20269 051/300 Write block 555 len 28672 data: 2 2 2 2 4 4 4
20270 052/300 Read block 2789 len 12288
20271 053/300 Read block 3244 len 40960
20272 054/300 Write block 2204 len 32768 data: 4 4 4 4 3 2 3 3
20273 055/300 Read block 1774 len 28672
20274 056/300 Read block 554 len 24576
20275 057/300 Read block 4262 len 40960
20276 058/300 Read block 513 len 36864
20277 059/300 Read block 3130 len 40960
20278 060/300 Write block 3890 len 28672 data: 5 4 5 6 5 5 5
20279 061/300 Read block 3546 len 36864
20280 062/300 Write block 3277 len 36864 data: 4 4 4 4 4 4 5 5 4
20281 063/300 Read block 3463 len 8192
20282 064/300 Read block 1991 len 28672
20283 065/300 Read block 4413 len 24576
20284 066/300 Read block 1888 len 32768
20285 067/300 Write block 3962 len 20480 data: 3 3 3 4 4
20286 068/300 Write block 216 len 16384 data: 4 4 3 3
20287 069/300 Write block 3363 len 8192 data: 4 3
20288 070/300 Write block 21 len 36864 data: 4 4 3 6 6 6 7 8 7
20289 071/300 Read block 3355 len 16384
20290 072/300 Read block 518 len 20480
20291 073/300 Read block 1939 len 16384
20292 074/300 Read block 4322 len 36864
20293 075/300 Read block 532 len 40960
20294 076/300 Write block 3835 len 32768 data: 5 7 6 6 7 7 6 8
20295 077/300 Read block 2884 len 16384
20296 078/300 Read block 1461 len 20480
20297 079/300 Read block 3509 len 24576
20298 080/300 Read block 55 len 12288
20299 081/300 Read block 2887 len 32768
20300 082/300 Write block 2516 len 4096 data: 9
20301 083/300 Write block 3516 len 12288 data: 5 3 4
20302 084/300 Write block 3292 len 40960 data: 3 4 6 5 6 7 7 7 6 6
20303 085/300 Write block 3591 len 8192 data: 3 3
20304 086/300 Read block 4057 len 16384
20305 087/300 Read block 4051 len 40960
20306 088/300 Write block 4501 len 8192 data: 2 2
20307 089/300 Read block 2012 len 12288
20308 090/300 Write block 2617 len 36864 data: 2 2 3 3 3 5 4 4 4
20309 091/300 Read block 2335 len 24576
20310 092/300 Write block 3371 len 28672 data: 2 2 2 3 4 4 4
20311 093/300 Write block 3249 len 40960 data: 2 3 5 5 4 5 5 5 5 5
20312 094/300 Read block 2677 len 32768
20313 095/300 Read block 1071 len 32768
20314 096/300 Write block 863 len 4096 data: 5
20315 097/300 Write block 2114 len 32768 data: 5 3 3 3 3 2 3 3
20316 098/300 Read block 3836 len 40960
20317 099/300 Read block 3585 len 32768
20318 100/300 Read block 2185 len 24576
20319 101/300 Write block 1030 len 24576 data: 7 5 6 6 6 8
20320 102/300 Read block 701 len 20480
20321 103/300 Read block 4064 len 4096
20322 104/300 Read block 1747 len 16384
20323 105/300 Read block 2518 len 28672
20324 106/300 Read block 4996 len 16384
20325 107/300 Write block 846 len 20480 data: 4 5 3 2 2
20326 108/300 Read block 3216 len 40960
20327 109/300 Read block 553 len 24576
20328 110/300 Read block 3806 len 36864
20329 111/300 Read block 2299 len 36864
20330 112/300 Read block 4633 len 24576
20331 113/300 Write block 1637 len 24576 data: 3 3 3 2 2 2
20332 114/300 Write block 1528 len 32768 data: 3 3 4 5 5 3 3 3
20333 115/300 Read block 856 len 8192
20334 116/300 Write block 2760 len 40960 data: 4 4 3 4 4 4 4 5 5 5
20335 117/300 Read block 1849 len 12288
20336 118/300 Write block 3608 len 24576 data: 5 7 7 8 9 6
20337 119/300 Read block 4711 len 28672
20338 120/300 Read block 4234 len 40960
20339 121/300 Flush
20340 122/300 Read block 1921 len 28672
20341 123/300 Write block 4986 len 20480 data: 3 2 2 2 3
20342 124/300 Read block 3810 len 8192
20343 125/300 Read block 935 len 16384
20344 126/300 Write block 4787 len 40960 data: 4 4 3 2 2 2 2 3 3 3
20345 127/300 Read block 3024 len 36864
20346 128/300 Write block 761 len 20480 data: 5 5 4 5 5
20347 129/300 Read block 2759 len 36864
20348 130/300 Read block 1450 len 16384
20349 131/300 Read block 4736 len 4096
20350 132/300 Read block 3582 len 16384
20351 133/300 Read block 1076 len 24576
20352 134/300 Read block 2322 len 4096
20353 135/300 Write block 1243 len 20480 data: 3 4 5 5 5
20354 136/300 Flush
20355 137/300 Read block 2116 len 16384
20356 138/300 Read block 3003 len 28672
20357 139/300 Read block 3581 len 24576
20358 140/300 Read block 487 len 20480
20359 141/300 Write block 2866 len 36864 data: 6 6 5 5 6 4 6 5 5
20360 142/300 Write block 2552 len 12288 data: 6 6 4
20361 143/300 Read block 4724 len 4096
20362 144/300 Read block 3616 len 12288
20363 145/300 Write block 71 len 32768 data: 6 7 7 7 8 8 6 5
20364 146/300 Read block 4060 len 24576
20365 147/300 Write block 2199 len 16384 data: 3 3 3 4
20366 148/300 Write block 2983 len 8192 data: 3 3
20367 149/300 Write block 4759 len 40960 data: 9 9 8 9 8 9 7 7 5 5
20368 150/300 Read block 1312 len 4096
20369 151/300 Read block 4199 len 16384
20370 152/300 Write block 3725 len 40960 data: 5 5 5 3 3 3 3 5 4 3
20371 153/300 Read block 3283 len 36864
20372 154/300 Read block 4283 len 4096
20373 155/300 Read block 2151 len 28672
20374 156/300 Write block 2423 len 32768 data: 3 2 2 3 5 6 6 5
20375 157/300 Read block 4219 len 20480
20376 158/300 Read block 3675 len 16384
20377 159/300 Read block 3636 len 28672
20378 160/300 Read block 4715 len 8192
20379 161/300 Write block 2245 len 8192 data: 4 3
20380 162/300 Read block 2201 len 16384
20381 163/300 Read block 4603 len 24576
20382 164/300 Write block 3570 len 24576 data: 3 3 3 3 5 5
20383 165/300 Write block 4039 len 36864 data: 4 4 4 4 4 3 4 4 4
20384 166/300 Read block 3285 len 32768
20385 167/300 Read block 2000 len 40960
20386 168/300 Flush
20387 169/300 Write block 2141 len 28672 data: 3 4 4 4 4 4 4
20388 170/300 Write block 3583 len 4096 data: 2
20389 171/300 Flush
20390 172/300 Write block 4783 len 16384 data: 5 5 6 5
20391 173/300 Read block 1156 len 24576
20392 174/300 Write block 4412 len 20480 data: 5 6 5 5 8
20393 175/300 Read block 200 len 16384
20394 176/300 Flush
20395 177/300 Read block 1198 len 8192
20396 178/300 Write block 3406 len 36864 data: 3 4 4 4 3 3 4 4 4
20397 179/300 Read block 2619 len 4096
20398 180/300 Write block 3121 len 28672 data: 5 6 6 5 5 4 5
20399 181/300 Read block 3638 len 28672
20400 182/300 Read block 1078 len 32768
20401 183/300 Write block 1820 len 36864 data: 2 3 3 4 5 6 6 5 4
20402 184/300 Read block 4862 len 16384
20403 185/300 Read block 2484 len 4096
20404 186/300 Write block 2925 len 20480 data: 4 5 4 5 5
20405 187/300 Write block 3275 len 24576 data: 2 3 5 5 5 5
20406 188/300 Read block 1300 len 4096
20407 189/300 Read block 2106 len 36864
20408 190/300 Write block 1591 len 28672 data: 7 6 4 5 4 4 3
20409 191/300 Read block 344 len 12288
20410 192/300 Write block 1416 len 24576 data: 3 3 3 3 3 3
20411 193/300 Write block 291 len 20480 data: 5 5 6 6 5
20412 194/300 Read block 814 len 36864
20413 195/300 Read block 4792 len 16384
20414 196/300 Write block 2678 len 20480 data: 3 2 3 4 6
20415 197/300 Write block 80 len 28672 data: 4 4 3 2 2 2 3
20416 198/300 Read block 4531 len 28672
20417 199/300 Read block 3356 len 40960
20418 200/300 Read block 766 len 16384
20419 201/300 Read block 2393 len 32768
20420 202/300 Write block 1193 len 36864 data: 5 5 6 6 5 5 5 3 3
20421 203/300 Write block 192 len 32768 data: 5 5 6 9 9 9 7 5
20422 204/300 Write block 1268 len 28672 data: 5 5 4 4 4 3 3
20423 205/300 Write block 1382 len 36864 data: 5 6 5 4 3 3 3 3 3
20424 206/300 Write block 2928 len 12288 data: 6 6 4
20425 207/300 Write block 1624 len 40960 data: 4 3 4 2 4 3 3 3 3 4
20426 208/300 Flush
20427 209/300 Write block 3320 len 12288 data: 7 6 4
20428 210/300 Read block 3757 len 16384
20429 211/300 Read block 1080 len 36864
20430 212/300 Write block 2349 len 24576 data: 8 8 9 7 5 4
20431 213/300 Read block 4397 len 24576
20432 214/300 Read block 2629 len 20480
20433 215/300 Read block 485 len 28672
20434 216/300 Write block 2742 len 36864 data: 4 5 5 5 4 3 4 4 4
20435 217/300 Read block 2426 len 32768
20436 218/300 Write block 2447 len 36864 data: 5 5 5 3 2 2 2 2 2
20437 219/300 Flush
20438 220/300 Read block 1534 len 16384
20439 221/300 Write block 1674 len 12288 data: 3 5 5
20440 222/300 Write block 4497 len 8192 data: 3 3
20441 223/300 Write block 2278 len 24576 data: 5 5 5 4 4 4
20442 224/300 Write block 3695 len 40960 data: 6 4 4 4 4 5 6 7 7 8
20443 225/300 Write block 1977 len 36864 data: 5 6 5 4 6 4 4 4 5
20444 226/300 Read block 3060 len 28672
20445 227/300 Write block 4157 len 4096 data: 6
20446 228/300 Write block 573 len 16384 data: 3 3 5 4
20447 229/300 Read block 2709 len 20480
20448 230/300 Read block 2505 len 20480
20449 231/300 Read block 1477 len 16384
20450 232/300 Write block 2764 len 20480 data: 5 5 5 6 6
20451 233/300 Read block 943 len 4096
20452 234/300 Write block 2735 len 20480 data: 2 2 3 3 4
20453 235/300 Read block 2464 len 4096
20454 236/300 Write block 1326 len 16384 data: 7 7 8 8
20455 237/300 Read block 35 len 28672
20456 238/300 Flush
20457 239/300 Read block 4599 len 36864
20458 240/300 Read block 3063 len 24576
20459 241/300 Read block 1262 len 36864
20460 242/300 Read block 2610 len 28672
20461 243/300 Read block 2852 len 20480
20462 244/300 Read block 1465 len 20480
20463 245/300 Write block 4847 len 32768 data: 5 6 6 5 3 3 2 2
20464 246/300 Flush
20465 247/300 Write block 4431 len 40960 data: 3 3 2 3 4 4 4 4 4 4
20466 248/300 Write block 4426 len 20480 data: 6 6 6 7 5
20467 249/300 Read block 279 len 12288
20468 250/300 Read block 812 len 32768
20469 251/300 Read block 4656 len 12288
20470 252/300 Read block 2440 len 28672
20471 253/300 Write block 4479 len 24576 data: 4 4 5 5 6 5
20472 254/300 Read block 3611 len 24576
20473 255/300 Read block 3752 len 36864
20474 256/300 Write block 480 len 24576 data: 2 2 2 3 3 3
20475 257/300 Read block 2363 len 4096
20476 258/300 Flush
20477 259/300 Read block 582 len 36864
20478 260/300 Write block 2566 len 20480 data: 3 4 4 3 3
20479 261/300 Flush
20480 262/300 Write block 695 len 12288 data: 5 4 4
20481 263/300 Read block 4637 len 16384
20482 264/300 Read block 1284 len 4096
20483 265/300 Read block 562 len 20480
20484 266/300 Read block 2328 len 40960
20485 267/300 Write block 3782 len 36864 data: 7 7 6 7 7 7 7 5 5
20486 268/300 Write block 86 len 24576 data: 4 5 5 5 5 5
20487 269/300 Read block 2682 len 8192
20488 270/300 Read block 2401 len 16384
20489 271/300 Write block 4958 len 32768 data: 4 4 4 4 4 4 3 3
20490 272/300 Write block 3012 len 20480 data: 5 5 5 4 2
20491 273/300 Write block 2196 len 24576 data: 4 4 4 4 4 4
20492 274/300 Read block 169 len 40960
20493 275/300 Read block 2558 len 24576
20494 276/300 Read block 4295 len 16384
20495 277/300 Write block 4333 len 40960 data: 4 4 4 4 5 5 6 4 4 3
20496 278/300 Write block 81 len 28672 data: 5 4 3 3 3 5 6
20497 279/300 Write block 2737 len 28672 data: 4 4 5 4 4 5 6
20498 280/300 Read block 1743 len 12288
20499 281/300 Write block 4825 len 24576 data: 4 5 4 5 4 4
20500 282/300 Read block 591 len 40960
20501 283/300 Read block 4385 len 4096
20502 284/300 Read block 952 len 32768
20503 285/300 Write block 3384 len 24576 data: 3 3 2 2 2 2
20504 286/300 Write block 3891 len 36864 data: 5 6 7 6 6 6 4 3 3
20505 287/300 Read block 1040 len 20480
20506 288/300 Read block 4232 len 40960
20507 289/300 Write block 3120 len 12288 data: 6 6 7
20508 290/300 Read block 2220 len 8192
20509 291/300 Write block 1739 len 40960 data: 6 5 6 7 7 8 8 6 6 6
20510 292/300 Read block 3515 len 40960
20511 293/300 Write block 1952 len 36864 data: 7 7 7 7 5 5 5 6 6
20512 294/300 Write block 4614 len 40960 data: 5 4 2 3 3 4 6 7 6 6
20513 295/300 Read block 2419 len 36864
20514 296/300 Read block 2629 len 20480
20515 297/300 Read block 3505 len 20480
20516 298/300 Read block 837 len 28672
20517 299/300 Write block 1342 len 32768 data: 4 4 3 3 3 4 5 4
20518 300/300 Read block 4047 len 36864
20519 301/300 Read block 510 len 4096
20520 Main task has work to do, go find it
20521 got dsc Start(2)
20522 start 2
20523 Jan 17 21:15:25.142 INFO request completed, latency_us: 215, response_code: 204, uri: /start/cid/2, method: POST, req_id: daa2bade-2f47-4460-a5e6-46e92d7f8d51, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
20524 Replay: started 2, returned:Ok(())
20525 ----------------------------------------------------------------
20526 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
20527 States: Active Active Offline
20528 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20529 6433 Acked Write 5 Done Done Sent false
20530 6434 Acked Read 3 Done Done Sent false
20531 6435 Acked Read 2 Done Done Sent false
20532 6436 Acked Read 5 Done Done Sent false
20533 6437 Acked Read 5 Done Done Sent false
20534 6438 Acked Write 2 Done Done Sent false
20535 6439 Acked Write 3 Done Done Sent false
20536 6440 Acked Read 8 Done Done Sent false
20537 6441 Acked Write 4 Done Done Sent false
20538 6442 Acked Write 7 Done Done Sent false
20539 6443 Acked Read 9 Done Done Sent false
20540 6444 Acked Read 4 Done Done Sent false
20541 6445 Acked Read 6 Done Done Sent false
20542 6446 Acked Write 10 Done Done Sent false
20543 6447 Acked Write 1 Done Done Sent false
20544 6448 Acked Write 7 Done Done Sent false
20545 6449 Acked Read 6 Done Done Sent false
20546 6450 Acked Write 6 Done Done Sent false
20547 6451 Acked Write 8 Done Done Sent false
20548 6452 Acked Read 9 Done Done Sent false
20549 6453 Acked Read 3 Done Done Sent false
20550 6454 Acked Read 3 Done Done Sent false
20551 6455 Acked Read 7 Done Done Sent false
20552 6456 Acked Read 2 Done Done Sent false
20553 6457 Acked Read 5 Done Done Sent false
20554 6458 Acked Read 1 Done Done Sent false
20555 6459 Acked Write 10 Done Done Sent false
20556 6460 Acked Read 4 Done Done Sent false
20557 6461 Acked Write 1 Done Done Sent false
20558 6462 Acked Read 7 Done Done Sent false
20559 6463 Acked Write 2 Done Done Sent false
20560 6464 Acked Read 1 Done Done Sent false
20561 6465 Acked Read 5 Done Done Sent false
20562 6466 Acked Flush 0 Done Done Sent false
20563 6467 Acked Read 8 Done Done Sent false
20564 6468 Acked Read 3 Done Done Sent false
20565 6469 Acked Read 10 Done Done Sent false
20566 6470 Acked Write 1 Done Done Sent false
20567 6471 Acked Read 2 Done Done Sent false
20568 6472 Acked Write 7 Done Done Sent false
20569 6473 Acked Write 6 Done Done Sent false
20570 6474 Acked Read 2 Done Done Sent false
20571 6475 Acked Write 1 Done Done Sent false
20572 6476 Acked Write 9 Done Done Sent false
20573 6477 Acked Write 8 Done Done Sent false
20574 6478 Acked Write 5 Done Done Sent false
20575 6479 Acked Read 8 Done Done Sent false
20576 6480 Acked Read 3 Done Done Sent false
20577 6481 Acked Write 1 Done Done Sent false
20578 6482 Acked Write 5 Done Done Sent false
20579 6483 Acked Write 7 Done Done Sent false
20580 6484 Acked Read 3 Done Done Sent false
20581 6485 Acked Read 10 Done Done Sent false
20582 6486 Acked Write 8 Done Done Sent false
20583 6487 Acked Read 7 Done Done Sent false
20584 6488 Acked Read 6 Done Done Sent false
20585 6489 Acked Read 10 Done Done Sent false
20586 6490 Acked Read 9 Done Done Sent false
20587 6491 Acked Read 10 Done Done Sent false
20588 6492 Acked Write 7 Done Done Sent false
20589 6493 Acked Read 9 Done Done Sent false
20590 6494 Acked Write 9 Done Done Sent false
20591 6495 Acked Read 2 Done Done Sent false
20592 6496 Acked Read 7 Done Done Sent false
20593 6497 Acked Read 6 Done Done Sent false
20594 6498 Acked Read 8 Done Done Sent false
20595 6499 Acked Write 5 Done Done Sent false
20596 6500 Acked Write 4 Done Done Sent false
20597 6501 Acked Write 2 Done Done Sent false
20598 6502 Acked Write 9 Done Done Sent false
20599 6503 Acked Read 4 Done Done Sent false
20600 6504 Acked Read 5 Done Done Sent false
20601 6505 Acked Read 4 Done Done Sent false
20602 6506 Acked Read 9 Done Done Sent false
20603 6507 Acked Read 10 Done Done Sent false
20604 6508 Acked Write 8 Done Done Sent false
20605 6509 Acked Read 4 Done Done Sent false
20606 6510 Acked Read 5 Done Done Sent false
20607 6511 Acked Read 6 Done Done Sent false
20608 6512 Acked Read 3 Done Done Sent false
20609 6513 Acked Read 8 Done Done Sent false
20610 6514 Acked Write 1 Done Done Sent false
20611 6515 Acked Write 3 Done Done Sent false
20612 6516 Acked Write 10 Done Done Sent false
20613 6517 Acked Write 2 Done Done Sent false
20614 6518 Acked Read 4 Done Done Sent false
20615 6519 Acked Read 10 Done Done Sent false
20616 6520 Acked Write 2 Done Done Sent false
20617 6521 Acked Read 3 Done Done Sent false
20618 6522 Acked Write 9 Done Done Sent false
20619 6523 Acked Read 6 Done Done Sent false
20620 6524 Acked Write 7 Done Done Sent false
20621 6525 Acked Write 10 Done Done Sent false
20622 6526 Acked Read 8 Done Done Sent false
20623 6527 Acked Read 8 Done Done Sent false
20624 6528 Acked Write 1 Done Done Sent false
20625 6529 Acked Write 8 Done Done Sent false
20626 6530 Acked Read 10 Done Done Sent false
20627 6531 Acked Read 8 Done Done Sent false
20628 6532 Acked Read 6 Done Done Sent false
20629 6533 Acked Write 6 Done Done Sent false
20630 6534 Acked Read 5 Done Done Sent false
20631 6535 Acked Read 1 Done Done Sent false
20632 6536 Acked Read 4 Done Done Sent false
20633 6537 Acked Read 7 Done Done Sent false
20634 6538 Acked Read 4 Done Done Sent false
20635 6539 Acked Write 5 Done Done Sent false
20636 6540 Acked Read 10 Done Done Sent false
20637 6541 Acked Read 6 Done Done Sent false
20638 6542 Acked Read 9 Done Done Sent false
20639 6543 Acked Read 9 Done Done Sent false
20640 6544 Acked Read 6 Done Done Sent false
20641 6545 Acked Write 6 Done Done Sent false
20642 6546 Acked Write 8 Done Done Sent false
20643 6547 Acked Read 2 Done Done Sent false
20644 6548 Acked Write 10 Done Done Sent false
20645 6549 Acked Read 3 Done Done Sent false
20646 6550 Acked Write 6 Done Done Sent false
20647 6551 Acked Read 7 Done Done Sent false
20648 6552 Acked Read 10 Done Done Sent false
20649 6553 Acked Flush 0 Done Done Sent false
20650 6554 Acked Read 7 Done Done Sent false
20651 6555 Acked Write 5 Done Done Sent false
20652 6556 Acked Read 2 Done Done Sent false
20653 6557 Acked Read 4 Done Done Sent false
20654 6558 Acked Write 10 Done Done Sent false
20655 6559 Acked Read 9 Done Done Sent false
20656 6560 Acked Write 5 Done Done Sent false
20657 6561 Acked Read 9 Done Done Sent false
20658 6562 Acked Read 4 Done Done Sent false
20659 6563 Acked Read 1 Done Done Sent false
20660 6564 Acked Read 4 Done Done Sent false
20661 6565 Acked Read 6 Done Done Sent false
20662 6566 Acked Read 1 Done Done Sent false
20663 6567 Acked Write 5 Done Done Sent false
20664 6568 Acked Flush 0 Done Done Sent false
20665 6569 Acked Read 4 Done Done Sent false
20666 6570 Acked Read 7 Done Done Sent false
20667 6571 Acked Read 6 Done Done Sent false
20668 6572 Acked Read 5 Done Done Sent false
20669 6573 Acked Write 9 Done Done Sent false
20670 6574 Acked Write 3 Done Done Sent false
20671 6575 Acked Read 1 Done Done Sent false
20672 6576 Acked Read 3 Done Done Sent false
20673 6577 Acked Write 8 Done Done Sent false
20674 6578 Acked Read 6 Done Done Sent false
20675 6579 Acked Write 4 Done Done Sent false
20676 6580 Acked Write 2 Done Done Sent false
20677 6581 Acked Write 10 Done Done Sent false
20678 6582 Acked Read 1 Done Done Sent false
20679 6583 Acked Read 4 Done Done Sent false
20680 6584 Acked Write 10 Done Done Sent false
20681 6585 Acked Read 9 Done Done Sent false
20682 6586 Acked Read 1 Done Done Sent false
20683 6587 Acked Read 7 Done Done Sent false
20684 6588 Acked Write 8 Done Done Sent false
20685 6589 Acked Read 5 Done Done Sent false
20686 6590 Acked Read 4 Done Done Sent false
20687 6591 Acked Read 7 Done Done Sent false
20688 6592 Acked Read 2 Done Done Sent false
20689 6593 Acked Write 2 Done Done Sent false
20690 6594 Acked Read 4 Done Done Sent false
20691 6595 Acked Read 6 Done Done Sent false
20692 6596 Acked Write 6 Done Done Sent false
20693 6597 Acked Write 9 Done Done Sent false
20694 6598 Acked Read 8 Done Done Sent false
20695 6599 Acked Read 10 Done Done Sent false
20696 6600 Acked Flush 0 Done Done Sent false
20697 6601 Acked Write 7 Done Done Sent false
20698 6602 Acked Write 1 Done Done Sent false
20699 6603 Acked Flush 0 Done Done Sent false
20700 6604 Acked Write 4 Done Done Sent false
20701 6605 Acked Read 6 Done Done Sent false
20702 6606 Acked Write 5 Done Done Sent false
20703 6607 Acked Read 4 Done Done Sent false
20704 6608 Acked Flush 0 Done Done Sent false
20705 6609 Acked Read 2 Done Done Sent false
20706 6610 Acked Write 9 Done Done Sent false
20707 6611 Acked Read 1 Done Done Sent false
20708 6612 Acked Write 7 Done Done Sent false
20709 6613 Acked Read 7 Done Done Sent false
20710 6614 Acked Read 8 Done Done Sent false
20711 6615 Acked Write 9 Done Done Sent false
20712 6616 Acked Read 4 Done Done Sent false
20713 6617 Acked Read 1 Done Done Sent false
20714 6618 Acked Write 5 Done Done Sent false
20715 6619 Acked Write 6 Done Done Sent false
20716 6620 Acked Read 1 Done Done Sent false
20717 6621 Acked Read 9 Done Done Sent false
20718 6622 Acked Write 7 Done Done Sent false
20719 6623 Acked Read 3 Done Done Sent false
20720 6624 Acked Write 6 Done Done Sent false
20721 6625 Acked Write 5 Done Done Sent false
20722 6626 Acked Read 9 Done Done Sent false
20723 6627 Acked Read 4 Done Done Sent false
20724 6628 Acked Write 5 Done Done Sent false
20725 6629 Acked Write 7 Done Done Sent false
20726 6630 Acked Read 7 Done Done Sent false
20727 6631 Acked Read 10 Done Done Sent false
20728 6632 Acked Read 4 Done Done Sent false
20729 6633 Acked Read 8 Done Done Sent false
20730 6634 Acked Write 9 Done Done Sent false
20731 6635 Acked Write 8 Done Done Sent false
20732 6636 Acked Write 7 Done Done Sent false
20733 6637 Acked Write 9 Done Done Sent false
20734 6638 Acked Write 3 Done Done Sent false
20735 6639 Acked Write 10 Done Done Sent false
20736 6640 Acked Flush 0 Done Done Sent false
20737 6641 Acked Write 3 Done Done Sent false
20738 6642 Acked Read 4 Done Done Sent false
20739 6643 Acked Read 9 Done Done Sent false
20740 6644 Acked Write 6 Done Done Sent false
20741 6645 Acked Read 6 Done Done Sent false
20742 6646 Acked Read 5 Done Done Sent false
20743 6647 Acked Read 7 Done Done Sent false
20744 6648 Acked Write 9 Done Done Sent false
20745 6649 Acked Read 8 Done Done Sent false
20746 6650 Acked Write 9 Done Done Sent false
20747 6651 Acked Flush 0 Done Done Sent false
20748 6652 Acked Read 4 Done Done Sent false
20749 6653 Acked Write 3 Done Done Sent false
20750 6654 Acked Write 2 Done Done Sent false
20751 6655 Acked Write 6 Done Done Sent false
20752 6656 Acked Write 10 Done Done Sent false
20753 6657 Acked Write 9 Done Done Sent false
20754 6658 Acked Read 7 Done Done Sent false
20755 6659 Acked Write 1 Done Done Sent false
20756 6660 Acked Write 4 Done Done Sent false
20757 6661 Acked Read 5 Done Done Sent false
20758 6662 Acked Read 5 Done Done Sent false
20759 6663 Acked Read 4 Done Done Sent false
20760 6664 Acked Write 5 Done Done Sent false
20761 6665 Acked Read 1 Done Done Sent false
20762 6666 Acked Write 5 Done Done Sent false
20763 6667 Acked Read 1 Done Done Sent false
20764 6668 Acked Write 4 Done Done Sent false
20765 6669 Acked Read 7 Done Done Sent false
20766 6670 Acked Flush 0 Done Done Sent false
20767 6671 Acked Read 9 Done Done Sent false
20768 6672 Acked Read 6 Done Done Sent false
20769 6673 Acked Read 9 Done Done Sent false
20770 6674 Acked Read 7 Done Done Sent false
20771 6675 Acked Read 5 Done Done Sent false
20772 6676 Acked Read 5 Done Done Sent false
20773 6677 Acked Write 8 Done Done Sent false
20774 6678 Acked Flush 0 Done Done Sent false
20775 6679 Acked Write 10 Done Done Sent false
20776 6680 Acked Write 5 Done Done Sent false
20777 6681 Acked Read 3 Done Done Sent false
20778 6682 Acked Read 8 Done Done Sent false
20779 6683 Acked Read 3 Done Done Sent false
20780 6684 Acked Read 7 Done Done Sent false
20781 6685 Acked Write 6 Done Done Sent false
20782 6686 Acked Read 6 Done Done Sent false
20783 6687 Acked Read 9 Done Done Sent false
20784 6688 Acked Write 6 Done Done Sent false
20785 6689 Acked Read 1 Done Done Sent false
20786 6690 Acked Flush 0 Done Done Sent false
20787 6691 Acked Read 9 Done Done Sent false
20788 6692 Acked Write 5 Done Done Sent false
20789 6693 Acked Flush 0 Done Done Sent false
20790 6694 Acked Write 3 Done Done Sent false
20791 6695 Acked Read 4 Done Done Sent false
20792 6696 Acked Read 1 Done Done Sent false
20793 6697 Acked Read 5 Done Done Sent false
20794 6698 Acked Read 10 Done Done Sent false
20795 6699 Acked Write 9 Done Done Sent false
20796 6700 Acked Write 6 Done Done Sent false
20797 6701 Acked Read 2 Done Done Sent false
20798 6702 Acked Read 4 Done Done Sent false
20799 6703 Acked Write 8 Done Done Sent false
20800 6704 Acked Write 5 Done Done Sent false
20801 6705 Acked Write 6 Done Done Sent false
20802 6706 Acked Read 10 Done Done Sent false
20803 6707 Acked Read 6 Done Done Sent false
20804 6708 Acked Read 4 Done Done Sent false
20805 6709 Acked Write 10 Done Done Sent false
20806 6710 Acked Write 7 Done Done Sent false
20807 6711 Acked Write 7 Done Done Sent false
20808 6712 Acked Read 3 Done Done Sent false
20809 6713 Acked Write 6 Done Done Sent false
20810 6714 Acked Read 10 Done Done Sent false
20811 6715 Acked Read 1 Done Done Sent false
20812 6716 Acked Read 8 Done Done Sent false
20813 6717 Acked Write 6 Done Done Sent false
20814 6718 Acked Write 9 Done Done Sent false
20815 6719 Acked Read 5 Done Done Sent false
20816 6720 Acked Read 10 Done Done Sent false
20817 6721 Acked Write 3 Done Done Sent false
20818 6722 Acked Read 2 Done Done Sent false
20819 6723 Acked Write 10 Done Done Sent false
20820 6724 Acked Read 10 Done Done Sent false
20821 6725 Acked Write 9 Done Done Sent false
20822 6726 Acked Write 10 Done Done Sent false
20823 6727 Acked Read 9 Done Done Sent false
20824 6728 Acked Read 5 Done Done Sent false
20825 6729 Acked Read 5 Done Done Sent false
20826 6730 Acked Read 7 Done Done Sent false
20827 6731 Acked Write 8 Done Done Sent false
20828 6732 Acked Read 9 Done Done Sent false
20829 6733 Acked Read 1 Done Done Sent false
20830 STATES DS:0 DS:1 DS:2 TOTAL
20831 Sent 0 0 301 301
20832 Done 301 301 0 602
20833 Skipped 0 0 0 0
20834 Error 0 0 0 0
20835 Last Flush: 6693 6693 6432
20836 Downstairs last five completed: 6432 6431 6430 6429 6428
20837 Upstairs last five completed: 6733 6732 6731 6730 6729
20838 CLIENT: Up:0 ds:301 act:2
20839 ----------------------------------------------------------------
20840 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20841 States: Active Active Offline
20842 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20843 6433 Acked Write 5 Done Done Sent false
20844 6434 Acked Read 3 Done Done Sent false
20845 6435 Acked Read 2 Done Done Sent false
20846 6436 Acked Read 5 Done Done Sent false
20847 6437 Acked Read 5 Done Done Sent false
20848 6438 Acked Write 2 Done Done Sent false
20849 6439 Acked Write 3 Done Done Sent false
20850 6440 Acked Read 8 Done Done Sent false
20851 6441 Acked Write 4 Done Done Sent false
20852 6442 Acked Write 7 Done Done Sent false
20853 6443 Acked Read 9 Done Done Sent false
20854 6444 Acked Read 4 Done Done Sent false
20855 6445 Acked Read 6 Done Done Sent false
20856 6446 Acked Write 10 Done Done Sent false
20857 6447 Acked Write 1 Done Done Sent false
20858 6448 Acked Write 7 Done Done Sent false
20859 6449 Acked Read 6 Done Done Sent false
20860 6450 Acked Write 6 Done Done Sent false
20861 6451 Acked Write 8 Done Done Sent false
20862 6452 Acked Read 9 Done Done Sent false
20863 6453 Acked Read 3 Done Done Sent false
20864 6454 Acked Read 3 Done Done Sent false
20865 6455 Acked Read 7 Done Done Sent false
20866 6456 Acked Read 2 Done Done Sent false
20867 6457 Acked Read 5 Done Done Sent false
20868 6458 Acked Read 1 Done Done Sent false
20869 6459 Acked Write 10 Done Done Sent false
20870 6460 Acked Read 4 Done Done Sent false
20871 6461 Acked Write 1 Done Done Sent false
20872 6462 Acked Read 7 Done Done Sent false
20873 6463 Acked Write 2 Done Done Sent false
20874 6464 Acked Read 1 Done Done Sent false
20875 6465 Acked Read 5 Done Done Sent false
20876 6466 Acked Flush 0 Done Done Sent false
20877 6467 Acked Read 8 Done Done Sent false
20878 6468 Acked Read 3 Done Done Sent false
20879 6469 Acked Read 10 Done Done Sent false
20880 6470 Acked Write 1 Done Done Sent false
20881 6471 Acked Read 2 Done Done Sent false
20882 6472 Acked Write 7 Done Done Sent false
20883 6473 Acked Write 6 Done Done Sent false
20884 6474 Acked Read 2 Done Done Sent false
20885 6475 Acked Write 1 Done Done Sent false
20886 6476 Acked Write 9 Done Done Sent false
20887 6477 Acked Write 8 Done Done Sent false
20888 6478 Acked Write 5 Done Done Sent false
20889 6479 Acked Read 8 Done Done Sent false
20890 6480 Acked Read 3 Done Done Sent false
20891 6481 Acked Write 1 Done Done Sent false
20892 6482 Acked Write 5 Done Done Sent false
20893 6483 Acked Write 7 Done Done Sent false
20894 6484 Acked Read 3 Done Done Sent false
20895 6485 Acked Read 10 Done Done Sent false
20896 6486 Acked Write 8 Done Done Sent false
20897 6487 Acked Read 7 Done Done Sent false
20898 6488 Acked Read 6 Done Done Sent false
20899 6489 Acked Read 10 Done Done Sent false
20900 6490 Acked Read 9 Done Done Sent false
20901 6491 Acked Read 10 Done Done Sent false
20902 6492 Acked Write 7 Done Done Sent false
20903 6493 Acked Read 9 Done Done Sent false
20904 6494 Acked Write 9 Done Done Sent false
20905 6495 Acked Read 2 Done Done Sent false
20906 6496 Acked Read 7 Done Done Sent false
20907 6497 Acked Read 6 Done Done Sent false
20908 6498 Acked Read 8 Done Done Sent false
20909 6499 Acked Write 5 Done Done Sent false
20910 6500 Acked Write 4 Done Done Sent false
20911 6501 Acked Write 2 Done Done Sent false
20912 6502 Acked Write 9 Done Done Sent false
20913 6503 Acked Read 4 Done Done Sent false
20914 6504 Acked Read 5 Done Done Sent false
20915 6505 Acked Read 4 Done Done Sent false
20916 6506 Acked Read 9 Done Done Sent false
20917 6507 Acked Read 10 Done Done Sent false
20918 6508 Acked Write 8 Done Done Sent false
20919 6509 Acked Read 4 Done Done Sent false
20920 6510 Acked Read 5 Done Done Sent false
20921 6511 Acked Read 6 Done Done Sent false
20922 6512 Acked Read 3 Done Done Sent false
20923 6513 Acked Read 8 Done Done Sent false
20924 6514 Acked Write 1 Done Done Sent false
20925 6515 Acked Write 3 Done Done Sent false
20926 6516 Acked Write 10 Done Done Sent false
20927 6517 Acked Write 2 Done Done Sent false
20928 6518 Acked Read 4 Done Done Sent false
20929 6519 Acked Read 10 Done Done Sent false
20930 6520 Acked Write 2 Done Done Sent false
20931 6521 Acked Read 3 Done Done Sent false
20932 6522 Acked Write 9 Done Done Sent false
20933 6523 Acked Read 6 Done Done Sent false
20934 6524 Acked Write 7 Done Done Sent false
20935 6525 Acked Write 10 Done Done Sent false
20936 6526 Acked Read 8 Done Done Sent false
20937 6527 Acked Read 8 Done Done Sent false
20938 6528 Acked Write 1 Done Done Sent false
20939 6529 Acked Write 8 Done Done Sent false
20940 6530 Acked Read 10 Done Done Sent false
20941 6531 Acked Read 8 Done Done Sent false
20942 6532 Acked Read 6 Done Done Sent false
20943 6533 Acked Write 6 Done Done Sent false
20944 6534 Acked Read 5 Done Done Sent false
20945 6535 Acked Read 1 Done Done Sent false
20946 6536 Acked Read 4 Done Done Sent false
20947 6537 Acked Read 7 Done Done Sent false
20948 6538 Acked Read 4 Done Done Sent false
20949 6539 Acked Write 5 Done Done Sent false
20950 6540 Acked Read 10 Done Done Sent false
20951 6541 Acked Read 6 Done Done Sent false
20952 6542 Acked Read 9 Done Done Sent false
20953 6543 Acked Read 9 Done Done Sent false
20954 6544 Acked Read 6 Done Done Sent false
20955 6545 Acked Write 6 Done Done Sent false
20956 6546 Acked Write 8 Done Done Sent false
20957 6547 Acked Read 2 Done Done Sent false
20958 6548 Acked Write 10 Done Done Sent false
20959 6549 Acked Read 3 Done Done Sent false
20960 6550 Acked Write 6 Done Done Sent false
20961 6551 Acked Read 7 Done Done Sent false
20962 6552 Acked Read 10 Done Done Sent false
20963 6553 Acked Flush 0 Done Done Sent false
20964 6554 Acked Read 7 Done Done Sent false
20965 6555 Acked Write 5 Done Done Sent false
20966 6556 Acked Read 2 Done Done Sent false
20967 6557 Acked Read 4 Done Done Sent false
20968 6558 Acked Write 10 Done Done Sent false
20969 6559 Acked Read 9 Done Done Sent false
20970 6560 Acked Write 5 Done Done Sent false
20971 6561 Acked Read 9 Done Done Sent false
20972 6562 Acked Read 4 Done Done Sent false
20973 6563 Acked Read 1 Done Done Sent false
20974 6564 Acked Read 4 Done Done Sent false
20975 6565 Acked Read 6 Done Done Sent false
20976 6566 Acked Read 1 Done Done Sent false
20977 6567 Acked Write 5 Done Done Sent false
20978 6568 Acked Flush 0 Done Done Sent false
20979 6569 Acked Read 4 Done Done Sent false
20980 6570 Acked Read 7 Done Done Sent false
20981 6571 Acked Read 6 Done Done Sent false
20982 6572 Acked Read 5 Done Done Sent false
20983 6573 Acked Write 9 Done Done Sent false
20984 6574 Acked Write 3 Done Done Sent false
20985 6575 Acked Read 1 Done Done Sent false
20986 6576 Acked Read 3 Done Done Sent false
20987 6577 Acked Write 8 Done Done Sent false
20988 6578 Acked Read 6 Done Done Sent false
20989 6579 Acked Write 4 Done Done Sent false
20990 6580 Acked Write 2 Done Done Sent false
20991 6581 Acked Write 10 Done Done Sent false
20992 6582 Acked Read 1 Done Done Sent false
20993 6583 Acked Read 4 Done Done Sent false
20994 6584 Acked Write 10 Done Done Sent false
20995 6585 Acked Read 9 Done Done Sent false
20996 6586 Acked Read 1 Done Done Sent false
20997 6587 Acked Read 7 Done Done Sent false
20998 6588 Acked Write 8 Done Done Sent false
20999 6589 Acked Read 5 Done Done Sent false
21000 6590 Acked Read 4 Done Done Sent false
21001 6591 Acked Read 7 Done Done Sent false
21002 6592 Acked Read 2 Done Done Sent false
21003 6593 Acked Write 2 Done Done Sent false
21004 6594 Acked Read 4 Done Done Sent false
21005 6595 Acked Read 6 Done Done Sent false
21006 6596 Acked Write 6 Done Done Sent false
21007 6597 Acked Write 9 Done Done Sent false
21008 6598 Acked Read 8 Done Done Sent false
21009 6599 Acked Read 10 Done Done Sent false
21010 6600 Acked Flush 0 Done Done Sent false
21011 6601 Acked Write 7 Done Done Sent false
21012 6602 Acked Write 1 Done Done Sent false
21013 6603 Acked Flush 0 Done Done Sent false
21014 6604 Acked Write 4 Done Done Sent false
21015 6605 Acked Read 6 Done Done Sent false
21016 6606 Acked Write 5 Done Done Sent false
21017 6607 Acked Read 4 Done Done Sent false
21018 6608 Acked Flush 0 Done Done Sent false
21019 6609 Acked Read 2 Done Done Sent false
21020 6610 Acked Write 9 Done Done Sent false
21021 6611 Acked Read 1 Done Done Sent false
21022 6612 Acked Write 7 Done Done Sent false
21023 6613 Acked Read 7 Done Done Sent false
21024 6614 Acked Read 8 Done Done Sent false
21025 6615 Acked Write 9 Done Done Sent false
21026 6616 Acked Read 4 Done Done Sent false
21027 6617 Acked Read 1 Done Done Sent false
21028 6618 Acked Write 5 Done Done Sent false
21029 6619 Acked Write 6 Done Done Sent false
21030 6620 Acked Read 1 Done Done Sent false
21031 6621 Acked Read 9 Done Done Sent false
21032 6622 Acked Write 7 Done Done Sent false
21033 6623 Acked Read 3 Done Done Sent false
21034 6624 Acked Write 6 Done Done Sent false
21035 6625 Acked Write 5 Done Done Sent false
21036 6626 Acked Read 9 Done Done Sent false
21037 6627 Acked Read 4 Done Done Sent false
21038 6628 Acked Write 5 Done Done Sent false
21039 6629 Acked Write 7 Done Done Sent false
21040 6630 Acked Read 7 Done Done Sent false
21041 6631 Acked Read 10 Done Done Sent false
21042 6632 Acked Read 4 Done Done Sent false
21043 6633 Acked Read 8 Done Done Sent false
21044 6634 Acked Write 9 Done Done Sent false
21045 6635 Acked Write 8 Done Done Sent false
21046 6636 Acked Write 7 Done Done Sent false
21047 6637 Acked Write 9 Done Done Sent false
21048 6638 Acked Write 3 Done Done Sent false
21049 6639 Acked Write 10 Done Done Sent false
21050 6640 Acked Flush 0 Done Done Sent false
21051 6641 Acked Write 3 Done Done Sent false
21052 6642 Acked Read 4 Done Done Sent false
21053 6643 Acked Read 9 Done Done Sent false
21054 6644 Acked Write 6 Done Done Sent false
21055 6645 Acked Read 6 Done Done Sent false
21056 6646 Acked Read 5 Done Done Sent false
21057 6647 Acked Read 7 Done Done Sent false
21058 6648 Acked Write 9 Done Done Sent false
21059 6649 Acked Read 8 Done Done Sent false
21060 6650 Acked Write 9 Done Done Sent false
21061 6651 Acked Flush 0 Done Done Sent false
21062 6652 Acked Read 4 Done Done Sent false
21063 6653 Acked Write 3 Done Done Sent false
21064 6654 Acked Write 2 Done Done Sent false
21065 6655 Acked Write 6 Done Done Sent false
21066 6656 Acked Write 10 Done Done Sent false
21067 6657 Acked Write 9 Done Done Sent false
21068 6658 Acked Read 7 Done Done Sent false
21069 6659 Acked Write 1 Done Done Sent false
21070 6660 Acked Write 4 Done Done Sent false
21071 6661 Acked Read 5 Done Done Sent false
21072 6662 Acked Read 5 Done Done Sent false
21073 6663 Acked Read 4 Done Done Sent false
21074 6664 Acked Write 5 Done Done Sent false
21075 6665 Acked Read 1 Done Done Sent false
21076 6666 Acked Write 5 Done Done Sent false
21077 6667 Acked Read 1 Done Done Sent false
21078 6668 Acked Write 4 Done Done Sent false
21079 6669 Acked Read 7 Done Done Sent false
21080 6670 Acked Flush 0 Done Done Sent false
21081 6671 Acked Read 9 Done Done Sent false
21082 6672 Acked Read 6 Done Done Sent false
21083 6673 Acked Read 9 Done Done Sent false
21084 6674 Acked Read 7 Done Done Sent false
21085 6675 Acked Read 5 Done Done Sent false
21086 6676 Acked Read 5 Done Done Sent false
21087 6677 Acked Write 8 Done Done Sent false
21088 6678 Acked Flush 0 Done Done Sent false
21089 6679 Acked Write 10 Done Done Sent false
21090 6680 Acked Write 5 Done Done Sent false
21091 6681 Acked Read 3 Done Done Sent false
21092 6682 Acked Read 8 Done Done Sent false
21093 6683 Acked Read 3 Done Done Sent false
21094 6684 Acked Read 7 Done Done Sent false
21095 6685 Acked Write 6 Done Done Sent false
21096 6686 Acked Read 6 Done Done Sent false
21097 6687 Acked Read 9 Done Done Sent false
21098 6688 Acked Write 6 Done Done Sent false
21099 6689 Acked Read 1 Done Done Sent false
21100 6690 Acked Flush 0 Done Done Sent false
21101 6691 Acked Read 9 Done Done Sent false
21102 6692 Acked Write 5 Done Done Sent false
21103 6693 Acked Flush 0 Done Done Sent false
21104 6694 Acked Write 3 Done Done Sent false
21105 6695 Acked Read 4 Done Done Sent false
21106 6696 Acked Read 1 Done Done Sent false
21107 6697 Acked Read 5 Done Done Sent false
21108 6698 Acked Read 10 Done Done Sent false
21109 6699 Acked Write 9 Done Done Sent false
21110 6700 Acked Write 6 Done Done Sent false
21111 6701 Acked Read 2 Done Done Sent false
21112 6702 Acked Read 4 Done Done Sent false
21113 6703 Acked Write 8 Done Done Sent false
21114 6704 Acked Write 5 Done Done Sent false
21115 6705 Acked Write 6 Done Done Sent false
21116 6706 Acked Read 10 Done Done Sent false
21117 6707 Acked Read 6 Done Done Sent false
21118 6708 Acked Read 4 Done Done Sent false
21119 6709 Acked Write 10 Done Done Sent false
21120 6710 Acked Write 7 Done Done Sent false
21121 6711 Acked Write 7 Done Done Sent false
21122 6712 Acked Read 3 Done Done Sent false
21123 6713 Acked Write 6 Done Done Sent false
21124 6714 Acked Read 10 Done Done Sent false
21125 6715 Acked Read 1 Done Done Sent false
21126 6716 Acked Read 8 Done Done Sent false
21127 6717 Acked Write 6 Done Done Sent false
21128 6718 Acked Write 9 Done Done Sent false
21129 6719 Acked Read 5 Done Done Sent false
21130 6720 Acked Read 10 Done Done Sent false
21131 6721 Acked Write 3 Done Done Sent false
21132 6722 Acked Read 2 Done Done Sent false
21133 6723 Acked Write 10 Done Done Sent false
21134 6724 Acked Read 10 Done Done Sent false
21135 6725 Acked Write 9 Done Done Sent false
21136 6726 Acked Write 10 Done Done Sent false
21137 6727 Acked Read 9 Done Done Sent false
21138 6728 Acked Read 5 Done Done Sent false
21139 6729 Acked Read 5 Done Done Sent false
21140 6730 Acked Read 7 Done Done Sent false
21141 6731 Acked Write 8 Done Done Sent false
21142 6732 Acked Read 9 Done Done Sent false
21143 6733 Acked Read 1 Done Done Sent false
21144 6734 Acked Flush 0 Done Done Sent false
21145 STATES DS:0 DS:1 DS:2 TOTAL
21146 Sent 0 0 302 302
21147 Done 302 302 0 604
21148 Skipped 0 0 0 0
21149 Error 0 0 0 0
21150 Last Flush: 6734 6734 6432
21151 Downstairs last five completed: 6432 6431 6430 6429 6428
21152 Upstairs last five completed: 6734 6733 6732 6731 6730
21153 CLIENT: Up:0 ds:302 act:2
211542025-01-17T21:15:30.882ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
211552025-01-17T21:15:30.882ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
211562025-01-17T21:15:30.882ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
211572025-01-17T21:15:30.882ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
211582025-01-17T21:15:30.882ZINFOcrucible: client task is exiting = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
21159 ----------------------------------------------------------------
21160 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21161 States: Active Active Offline
21162 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21163 6433 Acked Write 5 Done Done Sent false
21164 6434 Acked Read 3 Done Done Sent false
21165 6435 Acked Read 2 Done Done Sent false
21166 6436 Acked Read 5 Done Done Sent false
21167 6437 Acked Read 5 Done Done Sent false
21168 6438 Acked Write 2 Done Done Sent false
21169 6439 Acked Write 3 Done Done Sent false
21170 6440 Acked Read 8 Done Done Sent false
21171 6441 Acked Write 4 Done Done Sent false
21172 6442 Acked Write 7 Done Done Sent false
21173 6443 Acked Read 9 Done Done Sent false
21174 6444 Acked Read 4 Done Done Sent false
21175 6445 Acked Read 6 Done Done Sent false
21176 6446 Acked Write 10 Done Done Sent false
21177 6447 Acked Write 1 Done Done Sent false
21178 6448 Acked Write 7 Done Done Sent false
21179 6449 Acked Read 6 Done Done Sent false
21180 6450 Acked Write 6 Done Done Sent false
21181 6451 Acked Write 8 Done Done Sent false
21182 6452 Acked Read 9 Done Done Sent false
21183 6453 Acked Read 3 Done Done Sent false
21184 6454 Acked Read 3 Done Done Sent false
21185 6455 Acked Read 7 Done Done Sent false
21186 6456 Acked Read 2 Done Done Sent false
21187 6457 Acked Read 5 Done Done Sent false
21188 6458 Acked Read 1 Done Done Sent false
21189 6459 Acked Write 10 Done Done Sent false
21190 6460 Acked Read 4 Done Done Sent false
21191 6461 Acked Write 1 Done Done Sent false
21192 6462 Acked Read 7 Done Done Sent false
21193 6463 Acked Write 2 Done Done Sent false
21194 6464 Acked Read 1 Done Done Sent false
21195 6465 Acked Read 5 Done Done Sent false
21196 6466 Acked Flush 0 Done Done Sent false
21197 6467 Acked Read 8 Done Done Sent false
21198 6468 Acked Read 3 Done Done Sent false
21199 6469 Acked Read 10 Done Done Sent false
21200 6470 Acked Write 1 Done Done Sent false
21201 6471 Acked Read 2 Done Done Sent false
21202 6472 Acked Write 7 Done Done Sent false
21203 6473 Acked Write 6 Done Done Sent false
21204 6474 Acked Read 2 Done Done Sent false
21205 6475 Acked Write 1 Done Done Sent false
21206 6476 Acked Write 9 Done Done Sent false
21207 6477 Acked Write 8 Done Done Sent false
21208 6478 Acked Write 5 Done Done Sent false
21209 6479 Acked Read 8 Done Done Sent false
21210 6480 Acked Read 3 Done Done Sent false
21211 6481 Acked Write 1 Done Done Sent false
21212 6482 Acked Write 5 Done Done Sent false
21213 6483 Acked Write 7 Done Done Sent false
21214 6484 Acked Read 3 Done Done Sent false
21215 6485 Acked Read 10 Done Done Sent false
21216 6486 Acked Write 8 Done Done Sent false
21217 6487 Acked Read 7 Done Done Sent false
21218 6488 Acked Read 6 Done Done Sent false
21219 6489 Acked Read 10 Done Done Sent false
21220 6490 Acked Read 9 Done Done Sent false
21221 6491 Acked Read 10 Done Done Sent false
21222 6492 Acked Write 7 Done Done Sent false
21223 6493 Acked Read 9 Done Done Sent false
21224 6494 Acked Write 9 Done Done Sent false
21225 6495 Acked Read 2 Done Done Sent false
21226 6496 Acked Read 7 Done Done Sent false
21227 6497 Acked Read 6 Done Done Sent false
21228 6498 Acked Read 8 Done Done Sent false
21229 6499 Acked Write 5 Done Done Sent false
21230 6500 Acked Write 4 Done Done Sent false
21231 6501 Acked Write 2 Done Done Sent false
21232 6502 Acked Write 9 Done Done Sent false
21233 6503 Acked Read 4 Done Done Sent false
21234 6504 Acked Read 5 Done Done Sent false
21235 6505 Acked Read 4 Done Done Sent false
21236 6506 Acked Read 9 Done Done Sent false
21237 6507 Acked Read 10 Done Done Sent false
21238 6508 Acked Write 8 Done Done Sent false
21239 6509 Acked Read 4 Done Done Sent false
21240 6510 Acked Read 5 Done Done Sent false
21241 6511 Acked Read 6 Done Done Sent false
21242 6512 Acked Read 3 Done Done Sent false
21243 6513 Acked Read 8 Done Done Sent false
21244 6514 Acked Write 1 Done Done Sent false
21245 6515 Acked Write 3 Done Done Sent false
21246 6516 Acked Write 10 Done Done Sent false
21247 6517 Acked Write 2 Done Done Sent false
21248 6518 Acked Read 4 Done Done Sent false
21249 6519 Acked Read 10 Done Done Sent false
21250 6520 Acked Write 2 Done Done Sent false
21251 6521 Acked Read 3 Done Done Sent false
21252 6522 Acked Write 9 Done Done Sent false
21253 6523 Acked Read 6 Done Done Sent false
21254 6524 Acked Write 7 Done Done Sent false
21255 6525 Acked Write 10 Done Done Sent false
21256 6526 Acked Read 8 Done Done Sent false
21257 6527 Acked Read 8 Done Done Sent false
21258 6528 Acked Write 1 Done Done Sent false
21259 6529 Acked Write 8 Done Done Sent false
21260 6530 Acked Read 10 Done Done Sent false
21261 6531 Acked Read 8 Done Done Sent false
21262 6532 Acked Read 6 Done Done Sent false
21263 6533 Acked Write 6 Done Done Sent false
21264 6534 Acked Read 5 Done Done Sent false
21265 6535 Acked Read 1 Done Done Sent false
21266 6536 Acked Read 4 Done Done Sent false
21267 6537 Acked Read 7 Done Done Sent false
21268 6538 Acked Read 4 Done Done Sent false
21269 6539 Acked Write 5 Done Done Sent false
21270 6540 Acked Read 10 Done Done Sent false
21271 6541 Acked Read 6 Done Done Sent false
21272 6542 Acked Read 9 Done Done Sent false
21273 6543 Acked Read 9 Done Done Sent false
21274 6544 Acked Read 6 Done Done Sent false
21275 6545 Acked Write 6 Done Done Sent false
21276 6546 Acked Write 8 Done Done Sent false
21277 6547 Acked Read 2 Done Done Sent false
21278 6548 Acked Write 10 Done Done Sent false
21279 6549 Acked Read 3 Done Done Sent false
21280 6550 Acked Write 6 Done Done Sent false
21281 6551 Acked Read 7 Done Done Sent false
21282 6552 Acked Read 10 Done Done Sent false
21283 6553 Acked Flush 0 Done Done Sent false
21284 6554 Acked Read 7 Done Done Sent false
21285 6555 Acked Write 5 Done Done Sent false
21286 6556 Acked Read 2 Done Done Sent false
21287 6557 Acked Read 4 Done Done Sent false
21288 6558 Acked Write 10 Done Done Sent false
21289 6559 Acked Read 9 Done Done Sent false
21290 6560 Acked Write 5 Done Done Sent false
21291 6561 Acked Read 9 Done Done Sent false
21292 6562 Acked Read 4 Done Done Sent false
21293 6563 Acked Read 1 Done Done Sent false
21294 6564 Acked Read 4 Done Done Sent false
21295 6565 Acked Read 6 Done Done Sent false
21296 6566 Acked Read 1 Done Done Sent false
21297 6567 Acked Write 5 Done Done Sent false
21298 6568 Acked Flush 0 Done Done Sent false
21299 6569 Acked Read 4 Done Done Sent false
21300 6570 Acked Read 7 Done Done Sent false
21301 6571 Acked Read 6 Done Done Sent false
21302 6572 Acked Read 5 Done Done Sent false
21303 6573 Acked Write 9 Done Done Sent false
21304 6574 Acked Write 3 Done Done Sent false
21305 6575 Acked Read 1 Done Done Sent false
21306 6576 Acked Read 3 Done Done Sent false
21307 6577 Acked Write 8 Done Done Sent false
21308 6578 Acked Read 6 Done Done Sent false
21309 6579 Acked Write 4 Done Done Sent false
21310 6580 Acked Write 2 Done Done Sent false
21311 6581 Acked Write 10 Done Done Sent false
21312 6582 Acked Read 1 Done Done Sent false
21313 6583 Acked Read 4 Done Done Sent false
21314 6584 Acked Write 10 Done Done Sent false
21315 6585 Acked Read 9 Done Done Sent false
21316 6586 Acked Read 1 Done Done Sent false
21317 6587 Acked Read 7 Done Done Sent false
21318 6588 Acked Write 8 Done Done Sent false
21319 6589 Acked Read 5 Done Done Sent false
21320 6590 Acked Read 4 Done Done Sent false
21321 6591 Acked Read 7 Done Done Sent false
21322 6592 Acked Read 2 Done Done Sent false
21323 6593 Acked Write 2 Done Done Sent false
21324 6594 Acked Read 4 Done Done Sent false
21325 6595 Acked Read 6 Done Done Sent false
21326 6596 Acked Write 6 Done Done Sent false
21327 6597 Acked Write 9 Done Done Sent false
21328 6598 Acked Read 8 Done Done Sent false
21329 6599 Acked Read 10 Done Done Sent false
21330 6600 Acked Flush 0 Done Done Sent false
21331 6601 Acked Write 7 Done Done Sent false
21332 6602 Acked Write 1 Done Done Sent false
21333 6603 Acked Flush 0 Done Done Sent false
21334 6604 Acked Write 4 Done Done Sent false
21335 6605 Acked Read 6 Done Done Sent false
21336 6606 Acked Write 5 Done Done Sent false
21337 6607 Acked Read 4 Done Done Sent false
21338 6608 Acked Flush 0 Done Done Sent false
21339 6609 Acked Read 2 Done Done Sent false
21340 6610 Acked Write 9 Done Done Sent false
21341 6611 Acked Read 1 Done Done Sent false
21342 6612 Acked Write 7 Done Done Sent false
21343 6613 Acked Read 7 Done Done Sent false
21344 6614 Acked Read 8 Done Done Sent false
21345 6615 Acked Write 9 Done Done Sent false
21346 6616 Acked Read 4 Done Done Sent false
21347 6617 Acked Read 1 Done Done Sent false
21348 6618 Acked Write 5 Done Done Sent false
21349 6619 Acked Write 6 Done Done Sent false
21350 6620 Acked Read 1 Done Done Sent false
21351 6621 Acked Read 9 Done Done Sent false
21352 6622 Acked Write 7 Done Done Sent false
21353 6623 Acked Read 3 Done Done Sent false
21354 6624 Acked Write 6 Done Done Sent false
21355 6625 Acked Write 5 Done Done Sent false
21356 6626 Acked Read 9 Done Done Sent false
21357 6627 Acked Read 4 Done Done Sent false
21358 6628 Acked Write 5 Done Done Sent false
21359 6629 Acked Write 7 Done Done Sent false
21360 6630 Acked Read 7 Done Done Sent false
21361 6631 Acked Read 10 Done Done Sent false
21362 6632 Acked Read 4 Done Done Sent false
21363 6633 Acked Read 8 Done Done Sent false
21364 6634 Acked Write 9 Done Done Sent false
21365 6635 Acked Write 8 Done Done Sent false
21366 6636 Acked Write 7 Done Done Sent false
21367 6637 Acked Write 9 Done Done Sent false
21368 6638 Acked Write 3 Done Done Sent false
21369 6639 Acked Write 10 Done Done Sent false
21370 6640 Acked Flush 0 Done Done Sent false
21371 6641 Acked Write 3 Done Done Sent false
21372 6642 Acked Read 4 Done Done Sent false
21373 6643 Acked Read 9 Done Done Sent false
21374 6644 Acked Write 6 Done Done Sent false
21375 6645 Acked Read 6 Done Done Sent false
21376 6646 Acked Read 5 Done Done Sent false
21377 6647 Acked Read 7 Done Done Sent false
21378 6648 Acked Write 9 Done Done Sent false
21379 6649 Acked Read 8 Done Done Sent false
21380 6650 Acked Write 9 Done Done Sent false
21381 6651 Acked Flush 0 Done Done Sent false
21382 6652 Acked Read 4 Done Done Sent false
21383 6653 Acked Write 3 Done Done Sent false
21384 6654 Acked Write 2 Done Done Sent false
21385 6655 Acked Write 6 Done Done Sent false
21386 6656 Acked Write 10 Done Done Sent false
21387 6657 Acked Write 9 Done Done Sent false
21388 6658 Acked Read 7 Done Done Sent false
21389 6659 Acked Write 1 Done Done Sent false
21390 6660 Acked Write 4 Done Done Sent false
21391 6661 Acked Read 5 Done Done Sent false
21392 6662 Acked Read 5 Done Done Sent false
21393 6663 Acked Read 4 Done Done Sent false
21394 6664 Acked Write 5 Done Done Sent false
21395 6665 Acked Read 1 Done Done Sent false
21396 6666 Acked Write 5 Done Done Sent false
21397 6667 Acked Read 1 Done Done Sent false
21398 6668 Acked Write 4 Done Done Sent false
21399 6669 Acked Read 7 Done Done Sent false
21400 6670 Acked Flush 0 Done Done Sent false
21401 6671 Acked Read 9 Done Done Sent false
21402 6672 Acked Read 6 Done Done Sent false
21403 6673 Acked Read 9 Done Done Sent false
21404 6674 Acked Read 7 Done Done Sent false
21405 6675 Acked Read 5 Done Done Sent false
21406 6676 Acked Read 5 Done Done Sent false
21407 6677 Acked Write 8 Done Done Sent false
21408 6678 Acked Flush 0 Done Done Sent false
21409 6679 Acked Write 10 Done Done Sent false
21410 6680 Acked Write 5 Done Done Sent false
21411 6681 Acked Read 3 Done Done Sent false
21412 6682 Acked Read 8 Done Done Sent false
21413 6683 Acked Read 3 Done Done Sent false
21414 6684 Acked Read 7 Done Done Sent false
21415 6685 Acked Write 6 Done Done Sent false
21416 6686 Acked Read 6 Done Done Sent false
21417 6687 Acked Read 9 Done Done Sent false
21418 6688 Acked Write 6 Done Done Sent false
21419 6689 Acked Read 1 Done Done Sent false
21420 6690 Acked Flush 0 Done Done Sent false
21421 6691 Acked Read 9 Done Done Sent false
21422 6692 Acked Write 5 Done Done Sent false
21423 6693 Acked Flush 0 Done Done Sent false
21424 6694 Acked Write 3 Done Done Sent false
21425 6695 Acked Read 4 Done Done Sent false
21426 6696 Acked Read 1 Done Done Sent false
21427 6697 Acked Read 5 Done Done Sent false
21428 6698 Acked Read 10 Done Done Sent false
21429 6699 Acked Write 9 Done Done Sent false
21430 6700 Acked Write 6 Done Done Sent false
21431 6701 Acked Read 2 Done Done Sent false
21432 6702 Acked Read 4 Done Done Sent false
21433 6703 Acked Write 8 Done Done Sent false
21434 6704 Acked Write 5 Done Done Sent false
21435 6705 Acked Write 6 Done Done Sent false
21436 6706 Acked Read 10 Done Done Sent false
21437 6707 Acked Read 6 Done Done Sent false
21438 6708 Acked Read 4 Done Done Sent false
21439 6709 Acked Write 10 Done Done Sent false
21440 6710 Acked Write 7 Done Done Sent false
21441 6711 Acked Write 7 Done Done Sent false
21442 6712 Acked Read 3 Done Done Sent false
21443 6713 Acked Write 6 Done Done Sent false
21444 6714 Acked Read 10 Done Done Sent false
21445 6715 Acked Read 1 Done Done Sent false
21446 6716 Acked Read 8 Done Done Sent false
21447 6717 Acked Write 6 Done Done Sent false
21448 6718 Acked Write 9 Done Done Sent false
21449 6719 Acked Read 5 Done Done Sent false
21450 6720 Acked Read 10 Done Done Sent false
21451 6721 Acked Write 3 Done Done Sent false
21452 6722 Acked Read 2 Done Done Sent false
21453 6723 Acked Write 10 Done Done Sent false
21454 6724 Acked Read 10 Done Done Sent false
21455 6725 Acked Write 9 Done Done Sent false
21456 6726 Acked Write 10 Done Done Sent false
21457 6727 Acked Read 9 Done Done Sent false
21458 6728 Acked Read 5 Done Done Sent false
21459 6729 Acked Read 5 Done Done Sent false
21460 6730 Acked Read 7 Done Done Sent false
21461 6731 Acked Write 8 Done Done Sent false
21462 6732 Acked Read 9 Done Done Sent false
21463 6733 Acked Read 1 Done Done Sent false
21464 6734 Acked Flush 0 Done Done Sent false
21465 STATES DS:0 DS:1 DS:2 TOTAL
21466 Sent 0 0 302 302
21467 Done 302 302 0 604
21468 Skipped 0 0 0 0
21469 Error 0 0 0 0
21470 Last Flush: 6734 6734 6432
21471 Downstairs last five completed: 6432 6431 6430 6429 6428
21472 Upstairs last five completed: 6734 6733 6732 6731 6730
21473 CLIENT: Up:0 ds:302 act:2
21474 [8830] got start action
21475 [8830] I am going to restart
21476 Starting downstairs at port 8830
21477 Make output file at "/tmp/dsc/downstairs-8830.txt"
21478 [8830][2] reports Starting
21479 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1219)
21480 [8830][2] reports Running
21481 ----------------------------------------------------------------
21482 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21483 States: Active Active Offline
21484 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21485 6433 Acked Write 5 Done Done Sent false
21486 6434 Acked Read 3 Done Done Sent false
21487 6435 Acked Read 2 Done Done Sent false
21488 6436 Acked Read 5 Done Done Sent false
21489 6437 Acked Read 5 Done Done Sent false
21490 6438 Acked Write 2 Done Done Sent false
21491 6439 Acked Write 3 Done Done Sent false
21492 6440 Acked Read 8 Done Done Sent false
21493 6441 Acked Write 4 Done Done Sent false
21494 6442 Acked Write 7 Done Done Sent false
21495 6443 Acked Read 9 Done Done Sent false
21496 6444 Acked Read 4 Done Done Sent false
21497 6445 Acked Read 6 Done Done Sent false
21498 6446 Acked Write 10 Done Done Sent false
21499 6447 Acked Write 1 Done Done Sent false
21500 6448 Acked Write 7 Done Done Sent false
21501 6449 Acked Read 6 Done Done Sent false
21502 6450 Acked Write 6 Done Done Sent false
21503 6451 Acked Write 8 Done Done Sent false
21504 6452 Acked Read 9 Done Done Sent false
21505 6453 Acked Read 3 Done Done Sent false
21506 6454 Acked Read 3 Done Done Sent false
21507 6455 Acked Read 7 Done Done Sent false
21508 6456 Acked Read 2 Done Done Sent false
21509 6457 Acked Read 5 Done Done Sent false
21510 6458 Acked Read 1 Done Done Sent false
21511 6459 Acked Write 10 Done Done Sent false
21512 6460 Acked Read 4 Done Done Sent false
21513 6461 Acked Write 1 Done Done Sent false
21514 6462 Acked Read 7 Done Done Sent false
21515 6463 Acked Write 2 Done Done Sent false
21516 6464 Acked Read 1 Done Done Sent false
21517 6465 Acked Read 5 Done Done Sent false
21518 6466 Acked Flush 0 Done Done Sent false
21519 6467 Acked Read 8 Done Done Sent false
21520 6468 Acked Read 3 Done Done Sent false
21521 6469 Acked Read 10 Done Done Sent false
21522 6470 Acked Write 1 Done Done Sent false
21523 6471 Acked Read 2 Done Done Sent false
21524 6472 Acked Write 7 Done Done Sent false
21525 6473 Acked Write 6 Done Done Sent false
21526 6474 Acked Read 2 Done Done Sent false
21527 6475 Acked Write 1 Done Done Sent false
21528 6476 Acked Write 9 Done Done Sent false
21529 6477 Acked Write 8 Done Done Sent false
21530 6478 Acked Write 5 Done Done Sent false
21531 6479 Acked Read 8 Done Done Sent false
21532 6480 Acked Read 3 Done Done Sent false
21533 6481 Acked Write 1 Done Done Sent false
21534 6482 Acked Write 5 Done Done Sent false
21535 6483 Acked Write 7 Done Done Sent false
21536 6484 Acked Read 3 Done Done Sent false
21537 6485 Acked Read 10 Done Done Sent false
21538 6486 Acked Write 8 Done Done Sent false
21539 6487 Acked Read 7 Done Done Sent false
21540 6488 Acked Read 6 Done Done Sent false
21541 6489 Acked Read 10 Done Done Sent false
21542 6490 Acked Read 9 Done Done Sent false
21543 6491 Acked Read 10 Done Done Sent false
21544 6492 Acked Write 7 Done Done Sent false
21545 6493 Acked Read 9 Done Done Sent false
21546 6494 Acked Write 9 Done Done Sent false
21547 6495 Acked Read 2 Done Done Sent false
21548 6496 Acked Read 7 Done Done Sent false
21549 6497 Acked Read 6 Done Done Sent false
21550 6498 Acked Read 8 Done Done Sent false
21551 6499 Acked Write 5 Done Done Sent false
21552 6500 Acked Write 4 Done Done Sent false
21553 6501 Acked Write 2 Done Done Sent false
21554 6502 Acked Write 9 Done Done Sent false
21555 6503 Acked Read 4 Done Done Sent false
21556 6504 Acked Read 5 Done Done Sent false
21557 6505 Acked Read 4 Done Done Sent false
21558 6506 Acked Read 9 Done Done Sent false
21559 6507 Acked Read 10 Done Done Sent false
21560 6508 Acked Write 8 Done Done Sent false
21561 6509 Acked Read 4 Done Done Sent false
21562 6510 Acked Read 5 Done Done Sent false
21563 6511 Acked Read 6 Done Done Sent false
21564 6512 Acked Read 3 Done Done Sent false
21565 6513 Acked Read 8 Done Done Sent false
21566 6514 Acked Write 1 Done Done Sent false
21567 6515 Acked Write 3 Done Done Sent false
21568 6516 Acked Write 10 Done Done Sent false
21569 6517 Acked Write 2 Done Done Sent false
21570 6518 Acked Read 4 Done Done Sent false
21571 6519 Acked Read 10 Done Done Sent false
21572 6520 Acked Write 2 Done Done Sent false
21573 6521 Acked Read 3 Done Done Sent false
21574 6522 Acked Write 9 Done Done Sent false
21575 6523 Acked Read 6 Done Done Sent false
21576 6524 Acked Write 7 Done Done Sent false
21577 6525 Acked Write 10 Done Done Sent false
21578 6526 Acked Read 8 Done Done Sent false
21579 6527 Acked Read 8 Done Done Sent false
21580 6528 Acked Write 1 Done Done Sent false
21581 6529 Acked Write 8 Done Done Sent false
21582 6530 Acked Read 10 Done Done Sent false
21583 6531 Acked Read 8 Done Done Sent false
21584 6532 Acked Read 6 Done Done Sent false
21585 6533 Acked Write 6 Done Done Sent false
21586 6534 Acked Read 5 Done Done Sent false
21587 6535 Acked Read 1 Done Done Sent false
21588 6536 Acked Read 4 Done Done Sent false
21589 6537 Acked Read 7 Done Done Sent false
21590 6538 Acked Read 4 Done Done Sent false
21591 6539 Acked Write 5 Done Done Sent false
21592 6540 Acked Read 10 Done Done Sent false
21593 6541 Acked Read 6 Done Done Sent false
21594 6542 Acked Read 9 Done Done Sent false
21595 6543 Acked Read 9 Done Done Sent false
21596 6544 Acked Read 6 Done Done Sent false
21597 6545 Acked Write 6 Done Done Sent false
21598 6546 Acked Write 8 Done Done Sent false
21599 6547 Acked Read 2 Done Done Sent false
21600 6548 Acked Write 10 Done Done Sent false
21601 6549 Acked Read 3 Done Done Sent false
21602 6550 Acked Write 6 Done Done Sent false
21603 6551 Acked Read 7 Done Done Sent false
21604 6552 Acked Read 10 Done Done Sent false
21605 6553 Acked Flush 0 Done Done Sent false
21606 6554 Acked Read 7 Done Done Sent false
21607 6555 Acked Write 5 Done Done Sent false
21608 6556 Acked Read 2 Done Done Sent false
21609 6557 Acked Read 4 Done Done Sent false
21610 6558 Acked Write 10 Done Done Sent false
21611 6559 Acked Read 9 Done Done Sent false
21612 6560 Acked Write 5 Done Done Sent false
21613 6561 Acked Read 9 Done Done Sent false
21614 6562 Acked Read 4 Done Done Sent false
21615 6563 Acked Read 1 Done Done Sent false
21616 6564 Acked Read 4 Done Done Sent false
21617 6565 Acked Read 6 Done Done Sent false
21618 6566 Acked Read 1 Done Done Sent false
21619 6567 Acked Write 5 Done Done Sent false
21620 6568 Acked Flush 0 Done Done Sent false
21621 6569 Acked Read 4 Done Done Sent false
21622 6570 Acked Read 7 Done Done Sent false
21623 6571 Acked Read 6 Done Done Sent false
21624 6572 Acked Read 5 Done Done Sent false
21625 6573 Acked Write 9 Done Done Sent false
21626 6574 Acked Write 3 Done Done Sent false
21627 6575 Acked Read 1 Done Done Sent false
21628 6576 Acked Read 3 Done Done Sent false
21629 6577 Acked Write 8 Done Done Sent false
21630 6578 Acked Read 6 Done Done Sent false
21631 6579 Acked Write 4 Done Done Sent false
21632 6580 Acked Write 2 Done Done Sent false
21633 6581 Acked Write 10 Done Done Sent false
21634 6582 Acked Read 1 Done Done Sent false
21635 6583 Acked Read 4 Done Done Sent false
21636 6584 Acked Write 10 Done Done Sent false
21637 6585 Acked Read 9 Done Done Sent false
21638 6586 Acked Read 1 Done Done Sent false
21639 6587 Acked Read 7 Done Done Sent false
21640 6588 Acked Write 8 Done Done Sent false
21641 6589 Acked Read 5 Done Done Sent false
21642 6590 Acked Read 4 Done Done Sent false
21643 6591 Acked Read 7 Done Done Sent false
21644 6592 Acked Read 2 Done Done Sent false
21645 6593 Acked Write 2 Done Done Sent false
21646 6594 Acked Read 4 Done Done Sent false
21647 6595 Acked Read 6 Done Done Sent false
21648 6596 Acked Write 6 Done Done Sent false
21649 6597 Acked Write 9 Done Done Sent false
21650 6598 Acked Read 8 Done Done Sent false
21651 6599 Acked Read 10 Done Done Sent false
21652 6600 Acked Flush 0 Done Done Sent false
21653 6601 Acked Write 7 Done Done Sent false
21654 6602 Acked Write 1 Done Done Sent false
21655 6603 Acked Flush 0 Done Done Sent false
21656 6604 Acked Write 4 Done Done Sent false
21657 6605 Acked Read 6 Done Done Sent false
21658 6606 Acked Write 5 Done Done Sent false
21659 6607 Acked Read 4 Done Done Sent false
21660 6608 Acked Flush 0 Done Done Sent false
21661 6609 Acked Read 2 Done Done Sent false
21662 6610 Acked Write 9 Done Done Sent false
21663 6611 Acked Read 1 Done Done Sent false
21664 6612 Acked Write 7 Done Done Sent false
21665 6613 Acked Read 7 Done Done Sent false
21666 6614 Acked Read 8 Done Done Sent false
21667 6615 Acked Write 9 Done Done Sent false
21668 6616 Acked Read 4 Done Done Sent false
21669 6617 Acked Read 1 Done Done Sent false
21670 6618 Acked Write 5 Done Done Sent false
21671 6619 Acked Write 6 Done Done Sent false
21672 6620 Acked Read 1 Done Done Sent false
21673 6621 Acked Read 9 Done Done Sent false
21674 6622 Acked Write 7 Done Done Sent false
21675 6623 Acked Read 3 Done Done Sent false
21676 6624 Acked Write 6 Done Done Sent false
21677 6625 Acked Write 5 Done Done Sent false
21678 6626 Acked Read 9 Done Done Sent false
21679 6627 Acked Read 4 Done Done Sent false
21680 6628 Acked Write 5 Done Done Sent false
21681 6629 Acked Write 7 Done Done Sent false
21682 6630 Acked Read 7 Done Done Sent false
21683 6631 Acked Read 10 Done Done Sent false
21684 6632 Acked Read 4 Done Done Sent false
21685 6633 Acked Read 8 Done Done Sent false
21686 6634 Acked Write 9 Done Done Sent false
21687 6635 Acked Write 8 Done Done Sent false
21688 6636 Acked Write 7 Done Done Sent false
21689 6637 Acked Write 9 Done Done Sent false
21690 6638 Acked Write 3 Done Done Sent false
21691 6639 Acked Write 10 Done Done Sent false
21692 6640 Acked Flush 0 Done Done Sent false
21693 6641 Acked Write 3 Done Done Sent false
21694 6642 Acked Read 4 Done Done Sent false
21695 6643 Acked Read 9 Done Done Sent false
21696 6644 Acked Write 6 Done Done Sent false
21697 6645 Acked Read 6 Done Done Sent false
21698 6646 Acked Read 5 Done Done Sent false
21699 6647 Acked Read 7 Done Done Sent false
21700 6648 Acked Write 9 Done Done Sent false
21701 6649 Acked Read 8 Done Done Sent false
21702 6650 Acked Write 9 Done Done Sent false
21703 6651 Acked Flush 0 Done Done Sent false
21704 6652 Acked Read 4 Done Done Sent false
21705 6653 Acked Write 3 Done Done Sent false
21706 6654 Acked Write 2 Done Done Sent false
21707 6655 Acked Write 6 Done Done Sent false
21708 6656 Acked Write 10 Done Done Sent false
21709 6657 Acked Write 9 Done Done Sent false
21710 6658 Acked Read 7 Done Done Sent false
21711 6659 Acked Write 1 Done Done Sent false
21712 6660 Acked Write 4 Done Done Sent false
21713 6661 Acked Read 5 Done Done Sent false
21714 6662 Acked Read 5 Done Done Sent false
21715 6663 Acked Read 4 Done Done Sent false
21716 6664 Acked Write 5 Done Done Sent false
21717 6665 Acked Read 1 Done Done Sent false
21718 6666 Acked Write 5 Done Done Sent false
21719 6667 Acked Read 1 Done Done Sent false
21720 6668 Acked Write 4 Done Done Sent false
21721 6669 Acked Read 7 Done Done Sent false
21722 6670 Acked Flush 0 Done Done Sent false
21723 6671 Acked Read 9 Done Done Sent false
21724 6672 Acked Read 6 Done Done Sent false
21725 6673 Acked Read 9 Done Done Sent false
21726 6674 Acked Read 7 Done Done Sent false
21727 6675 Acked Read 5 Done Done Sent false
21728 6676 Acked Read 5 Done Done Sent false
21729 6677 Acked Write 8 Done Done Sent false
21730 6678 Acked Flush 0 Done Done Sent false
21731 6679 Acked Write 10 Done Done Sent false
21732 6680 Acked Write 5 Done Done Sent false
21733 6681 Acked Read 3 Done Done Sent false
21734 6682 Acked Read 8 Done Done Sent false
21735 6683 Acked Read 3 Done Done Sent false
21736 6684 Acked Read 7 Done Done Sent false
21737 6685 Acked Write 6 Done Done Sent false
21738 6686 Acked Read 6 Done Done Sent false
21739 6687 Acked Read 9 Done Done Sent false
21740 6688 Acked Write 6 Done Done Sent false
21741 6689 Acked Read 1 Done Done Sent false
21742 6690 Acked Flush 0 Done Done Sent false
21743 6691 Acked Read 9 Done Done Sent false
21744 6692 Acked Write 5 Done Done Sent false
21745 6693 Acked Flush 0 Done Done Sent false
21746 6694 Acked Write 3 Done Done Sent false
21747 6695 Acked Read 4 Done Done Sent false
21748 6696 Acked Read 1 Done Done Sent false
21749 6697 Acked Read 5 Done Done Sent false
21750 6698 Acked Read 10 Done Done Sent false
21751 6699 Acked Write 9 Done Done Sent false
21752 6700 Acked Write 6 Done Done Sent false
21753 6701 Acked Read 2 Done Done Sent false
21754 6702 Acked Read 4 Done Done Sent false
21755 6703 Acked Write 8 Done Done Sent false
21756 6704 Acked Write 5 Done Done Sent false
21757 6705 Acked Write 6 Done Done Sent false
21758 6706 Acked Read 10 Done Done Sent false
21759 6707 Acked Read 6 Done Done Sent false
21760 6708 Acked Read 4 Done Done Sent false
21761 6709 Acked Write 10 Done Done Sent false
21762 6710 Acked Write 7 Done Done Sent false
21763 6711 Acked Write 7 Done Done Sent false
21764 6712 Acked Read 3 Done Done Sent false
21765 6713 Acked Write 6 Done Done Sent false
21766 6714 Acked Read 10 Done Done Sent false
21767 6715 Acked Read 1 Done Done Sent false
21768 6716 Acked Read 8 Done Done Sent false
21769 6717 Acked Write 6 Done Done Sent false
21770 6718 Acked Write 9 Done Done Sent false
21771 6719 Acked Read 5 Done Done Sent false
21772 6720 Acked Read 10 Done Done Sent false
21773 6721 Acked Write 3 Done Done Sent false
21774 6722 Acked Read 2 Done Done Sent false
21775 6723 Acked Write 10 Done Done Sent false
21776 6724 Acked Read 10 Done Done Sent false
21777 6725 Acked Write 9 Done Done Sent false
21778 6726 Acked Write 10 Done Done Sent false
21779 6727 Acked Read 9 Done Done Sent false
21780 6728 Acked Read 5 Done Done Sent false
21781 6729 Acked Read 5 Done Done Sent false
21782 6730 Acked Read 7 Done Done Sent false
21783 6731 Acked Write 8 Done Done Sent false
21784 6732 Acked Read 9 Done Done Sent false
21785 6733 Acked Read 1 Done Done Sent false
21786 6734 Acked Flush 0 Done Done Sent false
21787 STATES DS:0 DS:1 DS:2 TOTAL
21788 Sent 0 0 302 302
21789 Done 302 302 0 604
21790 Skipped 0 0 0 0
21791 Error 0 0 0 0
21792 Last Flush: 6734 6734 6432
21793 Downstairs last five completed: 6432 6431 6430 6429 6428
21794 Upstairs last five completed: 6734 6733 6732 6731 6730
21795 CLIENT: Up:0 ds:302 act:2
217962025-01-17T21:15:40.884ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
217972025-01-17T21:15:40.884ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51822) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
217982025-01-17T21:15:40.885ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
217992025-01-17T21:15:40.885ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
218002025-01-17T21:15:40.885ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
218012025-01-17T21:15:40.885ZINFOcrucible: send last flush ID to this DS: 6432 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
218022025-01-17T21:15:40.886ZINFOcrucible: Replied this last flush ID: 6432 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
218032025-01-17T21:15:40.886ZINFOcrucible: [2] client re-new 302 jobs since flush 6432 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
218042025-01-17T21:15:40.886ZINFOcrucible: [2] Replayed 302 jobs since flush: 6432 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
21805 ----------------------------------------------------------------
21806 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
21807 Downstairs last five completed: 6734 6733 6732 6731 6730
21808 Upstairs last five completed: 6734 6733 6732 6731 6730
21809 CLIENT: Up:0 ds:0 act:3
21810 Replay: All jobs finished
21811 Main task has work to do, go find it
21812 got dsc Stop(1)
21813 stop 1
21814 [8820] Got stop action so:false kr:false
21815 Jan 17 21:15:41.151 INFO request completed, latency_us: 351, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 55d973c7-84ec-4293-baa7-4352842930ee, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
218162025-01-17T21:15:41.151ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
218172025-01-17T21:15:41.151ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
218182025-01-17T21:15:41.151ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
21819 Jan 17 21:15:41.151 INFO request completed, latency_us: 144, response_code: 200, uri: /state/cid/1, method: GET, req_id: b52a6072-7f64-4342-9fee-b02c6d39ed4a, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
218202025-01-17T21:15:41.152ZINFOcrucible: client task is exiting = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
21821 [8820] Exited with: ExitStatus(unix_wait_status(9))
21822 [8820][1] reports Exit
21823 Jan 17 21:15:45.153 INFO request completed, latency_us: 213, response_code: 200, uri: /state/cid/1, method: GET, req_id: bff7e3b6-b174-427f-8faa-39d73fc1e4fd, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
21824 001/300 Write block 2046 len 24576 data: 5 6 5 5 7 6
21825 002/300 Flush
21826 003/300 Read block 1312 len 36864
21827 004/300 Read block 212 len 40960
21828 005/300 Write block 720 len 28672 data: 3 4 4 4 4 3 3
21829 006/300 Read block 3047 len 4096
21830 007/300 Read block 4414 len 8192
21831 008/300 Read block 3245 len 12288
21832 009/300 Write block 3155 len 8192 data: 4 3
21833 010/300 Write block 2723 len 28672 data: 4 3 3 3 6 7 8
21834 011/300 Read block 2269 len 24576
21835 012/300 Write block 3974 len 36864 data: 3 3 3 3 2 2 2 2 2
21836 013/300 Write block 2831 len 28672 data: 4 4 3 3 4 3 3
21837 014/300 Write block 2141 len 16384 data: 4 5 5 5
21838 015/300 Flush
21839 016/300 Read block 2391 len 12288
21840 017/300 Write block 457 len 4096 data: 5
21841 018/300 Flush
21842 019/300 Write block 4700 len 36864 data: 3 5 5 6 7 6 6 3 3
21843 020/300 Read block 803 len 28672
21844 021/300 Read block 2029 len 28672
21845 022/300 Write block 3430 len 32768 data: 3 3 3 3 3 3 4 5
21846 023/300 Write block 4859 len 20480 data: 3 4 3 3 3
21847 024/300 Write block 1481 len 20480 data: 5 5 5 6 6
21848 025/300 Read block 504 len 40960
21849 026/300 Flush
21850 027/300 Read block 2674 len 4096
21851 028/300 Write block 337 len 28672 data: 5 5 4 3 3 4 3
21852 029/300 Read block 2306 len 24576
21853 030/300 Read block 1068 len 40960
21854 031/300 Read block 4280 len 28672
21855 032/300 Write block 680 len 36864 data: 2 2 3 5 6 6 8 8 8
21856 033/300 Write block 2574 len 20480 data: 2 2 2 2 2
21857 034/300 Write block 805 len 4096 data: 5
21858 035/300 Read block 2733 len 8192
21859 036/300 Read block 4041 len 20480
21860 037/300 Flush
21861 038/300 Read block 3655 len 36864
21862 039/300 Read block 234 len 4096
21863 040/300 Read block 4386 len 12288
21864 041/300 Flush
21865 042/300 Read block 980 len 32768
21866 043/300 Read block 3963 len 4096
21867 044/300 Flush
21868 045/300 Flush
21869 046/300 Write block 931 len 16384 data: 5 5 5 6
21870 047/300 Read block 2817 len 8192
21871 048/300 Read block 7 len 24576
21872 049/300 Read block 3933 len 24576
21873 050/300 Write block 4725 len 32768 data: 3 3 2 5 4 5 4 4
21874 051/300 Write block 402 len 12288 data: 7 6 5
21875 052/300 Write block 1694 len 8192 data: 4 5
21876 053/300 Read block 4247 len 20480
21877 054/300 Read block 3586 len 36864
21878 055/300 Write block 3454 len 36864 data: 5 3 3 3 2 3 3 3 3
21879 056/300 Read block 4926 len 40960
21880 057/300 Write block 227 len 12288 data: 4 4 5
21881 058/300 Read block 3584 len 40960
21882 059/300 Read block 2675 len 40960
21883 060/300 Write block 3982 len 32768 data: 3 2 3 3 3 3 3 4
21884 061/300 Read block 4395 len 40960
21885 062/300 Read block 2843 len 16384
21886 063/300 Write block 2114 len 36864 data: 6 4 4 4 4 3 4 4 4
21887 064/300 Write block 1292 len 16384 data: 2 2 3 4
21888 065/300 Read block 2320 len 4096
21889 066/300 Write block 2912 len 20480 data: 5 4 5 5 8
21890 067/300 Write block 3271 len 36864 data: 3 3 3 2 3 4 6 6 6
21891 068/300 Read block 962 len 4096
21892 069/300 Read block 823 len 36864
21893 070/300 Write block 4890 len 32768 data: 5 5 5 4 3 3 3 3
21894 071/300 Write block 3085 len 32768 data: 6 7 7 5 6 5 4 4
21895 072/300 Flush
21896 073/300 Read block 4885 len 32768
21897 074/300 Read block 1381 len 20480
21898 075/300 Read block 3333 len 20480
21899 076/300 Write block 2182 len 12288 data: 3 5 5
21900 077/300 Write block 708 len 8192 data: 6 6
21901 078/300 Read block 1790 len 4096
21902 079/300 Write block 4732 len 36864 data: 5 6 5 5 5 4 3 3 4
21903 080/300 Write block 1239 len 36864 data: 3 3 3 3 4 5 6 6 6
21904 081/300 Read block 4992 len 32768
21905 082/300 Write block 692 len 40960 data: 5 6 6 6 5 5 5 6 4 4
21906 083/300 Flush
21907 084/300 Flush
21908 085/300 Read block 117 len 20480
21909 086/300 Write block 2872 len 36864 data: 7 6 6 4 4 4 4 3 3
21910 087/300 Write block 4302 len 4096 data: 5
21911 088/300 Read block 2843 len 32768
21912 089/300 Write block 2510 len 8192 data: 6 5
21913 090/300 Write block 2112 len 40960 data: 7 7 7 5 5 5 5 4 5 5
21914 091/300 Read block 293 len 20480
21915 092/300 Write block 623 len 32768 data: 4 4 4 4 4 4 5 6
21916 093/300 Flush
21917 094/300 Flush
21918 095/300 Read block 42 len 40960
21919 096/300 Write block 4991 len 32768 data: 3 4 5 4 4 4 4 3
21920 097/300 Read block 2966 len 32768
21921 098/300 Read block 66 len 32768
21922 099/300 Read block 381 len 16384
21923 100/300 Read block 2529 len 40960
21924 101/300 Write block 2961 len 8192 data: 2 3
21925 102/300 Write block 2367 len 32768 data: 3 2 2 3 4 5 6 5
21926 103/300 Read block 2325 len 16384
21927 104/300 Read block 1463 len 8192
21928 105/300 Read block 3104 len 36864
21929 106/300 Write block 2063 len 24576 data: 7 7 7 7 8 8
21930 107/300 Read block 1369 len 32768
21931 108/300 Write block 702 len 12288 data: 4 5 3
21932 109/300 Read block 4907 len 4096
21933 110/300 Flush
21934 111/300 Write block 1385 len 12288 data: 5 4 4
21935 112/300 Read block 1955 len 24576
21936 113/300 Read block 3886 len 4096
21937 114/300 Write block 3723 len 24576 data: 4 5 6 6 6 4
21938 115/300 Read block 1125 len 28672
21939 116/300 Read block 3996 len 36864
21940 117/300 Flush
21941 118/300 Flush
21942 119/300 Write block 4160 len 4096 data: 5
21943 120/300 Read block 3717 len 4096
21944 121/300 Read block 4778 len 40960
21945 122/300 Read block 684 len 24576
21946 123/300 Write block 2901 len 28672 data: 5 5 6 7 5 4 4
21947 124/300 Write block 2337 len 28672 data: 6 5 5 4 4 5 5
21948 125/300 Write block 4502 len 20480 data: 3 2 2 3 3
21949 126/300 Read block 2178 len 28672
21950 127/300 Flush
21951 128/300 Read block 3670 len 28672
21952 129/300 Read block 2757 len 36864
21953 130/300 Read block 848 len 12288
21954 131/300 Write block 439 len 4096 data: 3
21955 132/300 Write block 1525 len 24576 data: 7 5 4 4 4 5
21956 133/300 Flush
21957 134/300 Read block 3603 len 4096
21958 135/300 Read block 4815 len 40960
21959 136/300 Read block 838 len 28672
21960 137/300 Read block 1215 len 28672
21961 138/300 Write block 1873 len 40960 data: 5 4 4 3 3 3 3 5 5 6
21962 139/300 Read block 4710 len 16384
21963 140/300 Flush
21964 141/300 Write block 2263 len 8192 data: 5 8
21965 142/300 Read block 4877 len 24576
21966 143/300 Write block 741 len 24576 data: 5 5 4 5 4 3
21967 144/300 Write block 3920 len 24576 data: 3 4 4 4 6 6
21968 145/300 Write block 1670 len 4096 data: 3
21969 146/300 Read block 1086 len 36864
21970 147/300 Write block 4489 len 40960 data: 2 2 2 2 2 2 2 2 4 4
21971 148/300 Flush
21972 149/300 Read block 2025 len 12288
21973 150/300 Read block 700 len 8192
21974 151/300 Flush
21975 152/300 Write block 2043 len 16384 data: 4 5 5 6
21976 153/300 Read block 4335 len 20480
21977 154/300 Read block 2949 len 8192
21978 155/300 Flush
21979 156/300 Read block 899 len 12288
21980 157/300 Read block 2651 len 4096
21981 158/300 Flush
21982 159/300 Flush
21983 160/300 Read block 4036 len 4096
21984 161/300 Read block 3626 len 28672
21985 162/300 Read block 4912 len 12288
21986 163/300 Read block 2772 len 36864
21987 164/300 Read block 3514 len 32768
21988 165/300 Read block 1398 len 16384
21989 166/300 Write block 2493 len 28672 data: 5 5 4 3 3 4 3
21990 167/300 Read block 3840 len 40960
21991 168/300 Read block 54 len 32768
21992 169/300 Read block 4747 len 28672
21993 170/300 Write block 723 len 16384 data: 5 5 4 4
21994 171/300 Write block 4854 len 40960 data: 3 2 3 3 3 4 5 4 4 4
21995 172/300 Read block 1660 len 32768
21996 173/300 Write block 1049 len 12288 data: 4 4 5
21997 174/300 Read block 2736 len 36864
21998 175/300 Write block 4487 len 4096 data: 3
21999 176/300 Read block 811 len 24576
22000 177/300 Read block 2172 len 16384
22001 178/300 Read block 995 len 16384
22002 179/300 Read block 2068 len 4096
22003 180/300 Write block 4498 len 40960 data: 5 2 2 3 4 3 3 4 4 4
22004 181/300 Read block 1628 len 8192
22005 182/300 Flush
22006 183/300 Flush
22007 184/300 Read block 3843 len 8192
22008 185/300 Read block 1487 len 8192
22009 186/300 Read block 3135 len 16384
22010 187/300 Write block 2509 len 20480 data: 4 7 6 6 7
22011 188/300 Read block 1707 len 36864
22012 189/300 Read block 2214 len 16384
22013 190/300 Write block 51 len 8192 data: 4 4
22014 191/300 Write block 3303 len 12288 data: 6 5 3
22015 192/300 Read block 298 len 32768
22016 193/300 Write block 1562 len 8192 data: 5 5
22017 194/300 Read block 3405 len 32768
22018 195/300 Write block 3048 len 16384 data: 3 3 2 2
22019 196/300 Read block 4247 len 12288
22020 197/300 Read block 2405 len 28672
22021 198/300 Flush
22022 199/300 Read block 2566 len 12288
22023 200/300 Flush
22024 201/300 Read block 3199 len 36864
22025 202/300 Read block 4431 len 28672
22026 203/300 Write block 1261 len 40960 data: 6 6 4 4 4 3 4 6 6 5
22027 204/300 Read block 2601 len 32768
22028 205/300 Write block 3763 len 28672 data: 5 5 4 5 5 5 5
22029 206/300 Write block 2231 len 8192 data: 7 5
22030 207/300 Write block 4790 len 24576 data: 3 3 3 3 4 4
22031 208/300 Read block 3525 len 40960
22032 209/300 Write block 4081 len 12288 data: 4 4 3
22033 210/300 Write block 4910 len 24576 data: 3 3 3 4 4 4
22034 211/300 Write block 1230 len 4096 data: 6
22035 212/300 Read block 1214 len 16384
22036 213/300 Write block 4821 len 40960 data: 4 4 4 5 5 6 5 6 5 5
22037 214/300 Flush
22038 215/300 Read block 2221 len 8192
22039 216/300 Read block 1061 len 8192
22040 217/300 Write block 3605 len 28672 data: 5 4 4 6 8 8 9
22041 218/300 Write block 710 len 4096 data: 6
22042 219/300 Write block 2900 len 40960 data: 5 6 6 7 8 6 5 5 4 4
22043 220/300 Write block 1048 len 32768 data: 4 5 5 6 5 5 6 7
22044 221/300 Flush
22045 222/300 Read block 2578 len 12288
22046 223/300 Read block 3829 len 24576
22047 224/300 Read block 4024 len 4096
22048 225/300 Flush
22049 226/300 Write block 4408 len 36864 data: 5 4 2 2 6 7 6 6 9
22050 227/300 Write block 3001 len 32768 data: 4 5 6 6 7 7 5 6
22051 228/300 Write block 3725 len 4096 data: 7
22052 229/300 Read block 2993 len 4096
22053 230/300 Read block 1223 len 36864
22054 231/300 Write block 2814 len 4096 data: 4
22055 232/300 Read block 2203 len 40960
22056 233/300 Write block 4906 len 40960 data: 2 3 3 3 4 4 4 5 5 5
22057 234/300 Read block 3218 len 12288
22058 235/300 Write block 3043 len 12288 data: 5 3 3
22059 236/300 Write block 3952 len 32768 data: 4 4 2 2 2 2 2 3
22060 237/300 Write block 2103 len 8192 data: 4 5
22061 238/300 Read block 809 len 24576
22062 239/300 Write block 686 len 36864 data: 9 9 9 5 6 5 6 7 7
22063 240/300 Flush
22064 241/300 Flush
22065 242/300 Write block 535 len 32768 data: 5 6 4 4 3 4 5 5
22066 243/300 Write block 2462 len 24576 data: 5 5 4 5 7 5
22067 244/300 Read block 4805 len 12288
22068 245/300 Write block 3464 len 20480 data: 3 4 4 5 5
22069 246/300 Read block 1024 len 40960
22070 247/300 Read block 2386 len 12288
22071 248/300 Write block 4365 len 8192 data: 6 5
22072 249/300 Read block 968 len 40960
22073 250/300 Read block 2937 len 16384
22074 251/300 Read block 3382 len 28672
22075 252/300 Read block 4730 len 16384
22076 253/300 Write block 4317 len 28672 data: 6 5 4 4 5 6 7
22077 254/300 Read block 2315 len 40960
22078 255/300 Flush
22079 256/300 Write block 1447 len 36864 data: 5 5 5 4 4 4 5 4 4
22080 257/300 Read block 4308 len 12288
22081 258/300 Write block 1438 len 32768 data: 7 7 7 7 8 6 4 4
22082 259/300 Read block 1210 len 8192
22083 260/300 Read block 3327 len 12288
22084 261/300 Flush
22085 262/300 Read block 4167 len 20480
22086 263/300 Write block 2216 len 40960 data: 2 2 3 4 4 6 6 6 5 3
22087 264/300 Write block 3781 len 12288 data: 5 8 8
22088 265/300 Read block 2986 len 40960
22089 266/300 Flush
22090 267/300 Read block 3466 len 32768
22091 268/300 Read block 1764 len 40960
22092 269/300 Write block 770 len 40960 data: 2 2 2 2 2 2 2 3 3 3
22093 270/300 Read block 3270 len 28672
22094 271/300 Read block 4369 len 28672
22095 272/300 Write block 2793 len 40960 data: 8 7 6 5 5 4 4 3 3 3
22096 273/300 Read block 3329 len 24576
22097 274/300 Read block 2407 len 32768
22098 275/300 Write block 2213 len 12288 data: 2 2 2
22099 276/300 Write block 3721 len 28672 data: 4 5 5 6 8 7 7
22100 277/300 Write block 924 len 8192 data: 4 3
22101 278/300 Write block 2808 len 8192 data: 5 5
22102 279/300 Flush
22103 280/300 Write block 1453 len 12288 data: 6 5 5
22104 281/300 Read block 3677 len 28672
22105 282/300 Read block 2093 len 28672
22106 283/300 Write block 4696 len 28672 data: 4 4 3 3 4 6 6
22107 284/300 Read block 4394 len 20480
22108 285/300 Read block 2967 len 16384
22109 286/300 Read block 2036 len 40960
22110 287/300 Write block 4788 len 16384 data: 5 4 4 4
22111 288/300 Write block 4781 len 16384 data: 6 5 6 6
22112 289/300 Read block 536 len 28672
22113 290/300 Write block 2291 len 24576 data: 7 7 6 7 7 7
22114 291/300 Flush
22115 292/300 Read block 4209 len 32768
22116 293/300 Write block 3912 len 24576 data: 5 4 4 4 4 5
22117 294/300 Read block 4577 len 12288
22118 295/300 Write block 4206 len 20480 data: 3 3 4 3 3
22119 296/300 Flush
22120 297/300 Write block 1005 len 32768 data: 4 3 5 6 7 7 6 5
22121 298/300 Read block 2194 len 24576
22122 299/300 Write block 71 len 32768 data: 7 8 8 8 9 9 7 6
22123 300/300 Read block 475 len 20480
22124 301/300 Write block 2211 len 36864 data: 4 3 3 3 3 3 3 4 5
22125 Main task has work to do, go find it
22126 got dsc Start(1)
22127 start 1
22128 Jan 17 21:15:45.400 INFO request completed, latency_us: 314, response_code: 204, uri: /start/cid/1, method: POST, req_id: d18cc1d7-75c3-4f8a-9450-4f71502b7a85, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
22129 Replay: started 1, returned:Ok(())
22130 ----------------------------------------------------------------
22131 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
22132 States: Active Offline Active
22133 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22134 6735 Acked Write 6 Done Sent Done false
22135 6736 Acked Flush 0 Done Sent Done false
22136 6737 Acked Read 9 Done Sent Done false
22137 6738 Acked Read 10 Done Sent Done false
22138 6739 Acked Write 7 Done Sent Done false
22139 6740 Acked Read 1 Done Sent Done false
22140 6741 Acked Read 2 Done Sent Done false
22141 6742 Acked Read 3 Done Sent Done false
22142 6743 Acked Write 2 Done Sent Done false
22143 6744 Acked Write 7 Done Sent Done false
22144 6745 Acked Read 6 Done Sent Done false
22145 6746 Acked Write 9 Done Sent Done false
22146 6747 Acked Write 7 Done Sent Done false
22147 6748 Acked Write 4 Done Sent Done false
22148 6749 Acked Flush 0 Done Sent Done false
22149 6750 Acked Read 3 Done Sent Done false
22150 6751 Acked Write 1 Done Sent Done false
22151 6752 Acked Flush 0 Done Sent Done false
22152 6753 Acked Write 9 Done Sent Done false
22153 6754 Acked Read 7 Done Sent Done false
22154 6755 Acked Read 7 Done Sent Done false
22155 6756 Acked Write 8 Done Sent Done false
22156 6757 Acked Write 5 Done Sent Done false
22157 6758 Acked Write 5 Done Sent Done false
22158 6759 Acked Read 10 Done Sent Done false
22159 6760 Acked Flush 0 Done Sent Done false
22160 6761 Acked Read 1 Done Sent Done false
22161 6762 Acked Write 7 Done Sent Done false
22162 6763 Acked Read 6 Done Sent Done false
22163 6764 Acked Read 10 Done Sent Done false
22164 6765 Acked Read 7 Done Sent Done false
22165 6766 Acked Write 9 Done Sent Done false
22166 6767 Acked Write 5 Done Sent Done false
22167 6768 Acked Write 1 Done Sent Done false
22168 6769 Acked Read 2 Done Sent Done false
22169 6770 Acked Read 5 Done Sent Done false
22170 6771 Acked Flush 0 Done Sent Done false
22171 6772 Acked Read 9 Done Sent Done false
22172 6773 Acked Read 1 Done Sent Done false
22173 6774 Acked Read 3 Done Sent Done false
22174 6775 Acked Flush 0 Done Sent Done false
22175 6776 Acked Read 8 Done Sent Done false
22176 6777 Acked Read 1 Done Sent Done false
22177 6778 Acked Flush 0 Done Sent Done false
22178 6779 Acked Flush 0 Done Sent Done false
22179 6780 Acked Write 4 Done Sent Done false
22180 6781 Acked Read 2 Done Sent Done false
22181 6782 Acked Read 6 Done Sent Done false
22182 6783 Acked Read 6 Done Sent Done false
22183 6784 Acked Write 8 Done Sent Done false
22184 6785 Acked Write 3 Done Sent Done false
22185 6786 Acked Write 2 Done Sent Done false
22186 6787 Acked Read 5 Done Sent Done false
22187 6788 Acked Read 9 Done Sent Done false
22188 6789 Acked Write 9 Done Sent Done false
22189 6790 Acked Read 10 Done Sent Done false
22190 6791 Acked Write 3 Done Sent Done false
22191 6792 Acked Read 10 Done Sent Done false
22192 6793 Acked Read 10 Done Sent Done false
22193 6794 Acked Write 8 Done Sent Done false
22194 6795 Acked Read 10 Done Sent Done false
22195 6796 Acked Read 4 Done Sent Done false
22196 6797 Acked Write 9 Done Sent Done false
22197 6798 Acked Write 4 Done Sent Done false
22198 6799 Acked Read 1 Done Sent Done false
22199 6800 Acked Write 5 Done Sent Done false
22200 6801 Acked Write 9 Done Sent Done false
22201 6802 Acked Read 1 Done Sent Done false
22202 6803 Acked Read 9 Done Sent Done false
22203 6804 Acked Write 8 Done Sent Done false
22204 6805 Acked Write 8 Done Sent Done false
22205 6806 Acked Flush 0 Done Sent Done false
22206 6807 Acked Read 8 Done Sent Done false
22207 6808 Acked Read 5 Done Sent Done false
22208 6809 Acked Read 5 Done Sent Done false
22209 6810 Acked Write 3 Done Sent Done false
22210 6811 Acked Write 2 Done Sent Done false
22211 6812 Acked Read 1 Done Sent Done false
22212 6813 Acked Write 9 Done Sent Done false
22213 6814 Acked Write 9 Done Sent Done false
22214 6815 Acked Read 8 Done Sent Done false
22215 6816 Acked Write 10 Done Sent Done false
22216 6817 Acked Flush 0 Done Sent Done false
22217 6818 Acked Flush 0 Done Sent Done false
22218 6819 Acked Read 5 Done Sent Done false
22219 6820 Acked Write 9 Done Sent Done false
22220 6821 Acked Write 1 Done Sent Done false
22221 6822 Acked Read 8 Done Sent Done false
22222 6823 Acked Write 2 Done Sent Done false
22223 6824 Acked Write 10 Done Sent Done false
22224 6825 Acked Read 5 Done Sent Done false
22225 6826 Acked Write 8 Done Sent Done false
22226 6827 Acked Flush 0 Done Sent Done false
22227 6828 Acked Flush 0 Done Sent Done false
22228 6829 Acked Read 10 Done Sent Done false
22229 6830 Acked Write 8 Done Sent Done false
22230 6831 Acked Read 8 Done Sent Done false
22231 6832 Acked Read 8 Done Sent Done false
22232 6833 Acked Read 4 Done Sent Done false
22233 6834 Acked Read 10 Done Sent Done false
22234 6835 Acked Write 2 Done Sent Done false
22235 6836 Acked Write 8 Done Sent Done false
22236 6837 Acked Read 4 Done Sent Done false
22237 6838 Acked Read 2 Done Sent Done false
22238 6839 Acked Read 9 Done Sent Done false
22239 6840 Acked Write 6 Done Sent Done false
22240 6841 Acked Read 8 Done Sent Done false
22241 6842 Acked Write 3 Done Sent Done false
22242 6843 Acked Read 1 Done Sent Done false
22243 6844 Acked Flush 0 Done Sent Done false
22244 6845 Acked Write 3 Done Sent Done false
22245 6846 Acked Read 6 Done Sent Done false
22246 6847 Acked Read 1 Done Sent Done false
22247 6848 Acked Write 6 Done Sent Done false
22248 6849 Acked Read 7 Done Sent Done false
22249 6850 Acked Read 9 Done Sent Done false
22250 6851 Acked Flush 0 Done Sent Done false
22251 6852 Acked Flush 0 Done Sent Done false
22252 6853 Acked Write 1 Done Sent Done false
22253 6854 Acked Read 1 Done Sent Done false
22254 6855 Acked Read 10 Done Sent Done false
22255 6856 Acked Read 6 Done Sent Done false
22256 6857 Acked Write 7 Done Sent Done false
22257 6858 Acked Write 7 Done Sent Done false
22258 6859 Acked Write 5 Done Sent Done false
22259 6860 Acked Read 7 Done Sent Done false
22260 6861 Acked Flush 0 Done Sent Done false
22261 6862 Acked Read 7 Done Sent Done false
22262 6863 Acked Read 9 Done Sent Done false
22263 6864 Acked Read 3 Done Sent Done false
22264 6865 Acked Write 1 Done Sent Done false
22265 6866 Acked Write 6 Done Sent Done false
22266 6867 Acked Flush 0 Done Sent Done false
22267 6868 Acked Read 1 Done Sent Done false
22268 6869 Acked Read 10 Done Sent Done false
22269 6870 Acked Read 7 Done Sent Done false
22270 6871 Acked Read 7 Done Sent Done false
22271 6872 Acked Write 10 Done Sent Done false
22272 6873 Acked Read 4 Done Sent Done false
22273 6874 Acked Flush 0 Done Sent Done false
22274 6875 Acked Write 2 Done Sent Done false
22275 6876 Acked Read 6 Done Sent Done false
22276 6877 Acked Write 6 Done Sent Done false
22277 6878 Acked Write 6 Done Sent Done false
22278 6879 Acked Write 1 Done Sent Done false
22279 6880 Acked Read 9 Done Sent Done false
22280 6881 Acked Write 10 Done Sent Done false
22281 6882 Acked Flush 0 Done Sent Done false
22282 6883 Acked Read 3 Done Sent Done false
22283 6884 Acked Read 2 Done Sent Done false
22284 6885 Acked Flush 0 Done Sent Done false
22285 6886 Acked Write 4 Done Sent Done false
22286 6887 Acked Read 5 Done Sent Done false
22287 6888 Acked Read 2 Done Sent Done false
22288 6889 Acked Flush 0 Done Sent Done false
22289 6890 Acked Read 3 Done Sent Done false
22290 6891 Acked Read 1 Done Sent Done false
22291 6892 Acked Flush 0 Done Sent Done false
22292 6893 Acked Flush 0 Done Sent Done false
22293 6894 Acked Read 1 Done Sent Done false
22294 6895 Acked Read 7 Done Sent Done false
22295 6896 Acked Read 3 Done Sent Done false
22296 6897 Acked Read 9 Done Sent Done false
22297 6898 Acked Read 8 Done Sent Done false
22298 6899 Acked Read 4 Done Sent Done false
22299 6900 Acked Write 7 Done Sent Done false
22300 6901 Acked Read 10 Done Sent Done false
22301 6902 Acked Read 8 Done Sent Done false
22302 6903 Acked Read 7 Done Sent Done false
22303 6904 Acked Write 4 Done Sent Done false
22304 6905 Acked Write 10 Done Sent Done false
22305 6906 Acked Read 8 Done Sent Done false
22306 6907 Acked Write 3 Done Sent Done false
22307 6908 Acked Read 9 Done Sent Done false
22308 6909 Acked Write 1 Done Sent Done false
22309 6910 Acked Read 6 Done Sent Done false
22310 6911 Acked Read 4 Done Sent Done false
22311 6912 Acked Read 4 Done Sent Done false
22312 6913 Acked Read 1 Done Sent Done false
22313 6914 Acked Write 10 Done Sent Done false
22314 6915 Acked Read 2 Done Sent Done false
22315 6916 Acked Flush 0 Done Sent Done false
22316 6917 Acked Flush 0 Done Sent Done false
22317 6918 Acked Read 2 Done Sent Done false
22318 6919 Acked Read 2 Done Sent Done false
22319 6920 Acked Read 4 Done Sent Done false
22320 6921 Acked Write 5 Done Sent Done false
22321 6922 Acked Read 9 Done Sent Done false
22322 6923 Acked Read 4 Done Sent Done false
22323 6924 Acked Write 2 Done Sent Done false
22324 6925 Acked Write 3 Done Sent Done false
22325 6926 Acked Read 8 Done Sent Done false
22326 6927 Acked Write 2 Done Sent Done false
22327 6928 Acked Read 8 Done Sent Done false
22328 6929 Acked Write 4 Done Sent Done false
22329 6930 Acked Read 3 Done Sent Done false
22330 6931 Acked Read 7 Done Sent Done false
22331 6932 Acked Flush 0 Done Sent Done false
22332 6933 Acked Read 3 Done Sent Done false
22333 6934 Acked Flush 0 Done Sent Done false
22334 6935 Acked Read 9 Done Sent Done false
22335 6936 Acked Read 7 Done Sent Done false
22336 6937 Acked Write 10 Done Sent Done false
22337 6938 Acked Read 8 Done Sent Done false
22338 6939 Acked Write 7 Done Sent Done false
22339 6940 Acked Write 2 Done Sent Done false
22340 6941 Acked Write 6 Done Sent Done false
22341 6942 Acked Read 10 Done Sent Done false
22342 6943 Acked Write 3 Done Sent Done false
22343 6944 Acked Write 6 Done Sent Done false
22344 6945 Acked Write 1 Done Sent Done false
22345 6946 Acked Read 4 Done Sent Done false
22346 6947 Acked Write 10 Done Sent Done false
22347 6948 Acked Flush 0 Done Sent Done false
22348 6949 Acked Read 2 Done Sent Done false
22349 6950 Acked Read 2 Done Sent Done false
22350 6951 Acked Write 7 Done Sent Done false
22351 6952 Acked Write 1 Done Sent Done false
22352 6953 Acked Write 10 Done Sent Done false
22353 6954 Acked Write 8 Done Sent Done false
22354 6955 Acked Flush 0 Done Sent Done false
22355 6956 Acked Read 3 Done Sent Done false
22356 6957 Acked Read 6 Done Sent Done false
22357 6958 Acked Read 1 Done Sent Done false
22358 6959 Acked Flush 0 Done Sent Done false
22359 6960 Acked Write 9 Done Sent Done false
22360 6961 Acked Write 8 Done Sent Done false
22361 6962 Acked Write 1 Done Sent Done false
22362 6963 Acked Read 1 Done Sent Done false
22363 6964 Acked Read 9 Done Sent Done false
22364 6965 Acked Write 1 Done Sent Done false
22365 6966 Acked Read 10 Done Sent Done false
22366 6967 Acked Write 10 Done Sent Done false
22367 6968 Acked Read 3 Done Sent Done false
22368 6969 Acked Write 3 Done Sent Done false
22369 6970 Acked Write 8 Done Sent Done false
22370 6971 Acked Write 2 Done Sent Done false
22371 6972 Acked Read 6 Done Sent Done false
22372 6973 Acked Write 9 Done Sent Done false
22373 6974 Acked Flush 0 Done Sent Done false
22374 6975 Acked Flush 0 Done Sent Done false
22375 6976 Acked Write 8 Done Sent Done false
22376 6977 Acked Write 6 Done Sent Done false
22377 6978 Acked Read 3 Done Sent Done false
22378 6979 Acked Write 5 Done Sent Done false
22379 6980 Acked Read 10 Done Sent Done false
22380 6981 Acked Read 3 Done Sent Done false
22381 6982 Acked Write 2 Done Sent Done false
22382 6983 Acked Read 10 Done Sent Done false
22383 6984 Acked Read 4 Done Sent Done false
22384 6985 Acked Read 7 Done Sent Done false
22385 6986 Acked Read 4 Done Sent Done false
22386 6987 Acked Write 7 Done Sent Done false
22387 6988 Acked Read 10 Done Sent Done false
22388 6989 Acked Flush 0 Done Sent Done false
22389 6990 Acked Write 9 Done Sent Done false
22390 6991 Acked Read 3 Done Sent Done false
22391 6992 Acked Write 8 Done Sent Done false
22392 6993 Acked Read 2 Done Sent Done false
22393 6994 Acked Read 3 Done Sent Done false
22394 6995 Acked Flush 0 Done Sent Done false
22395 6996 Acked Read 5 Done Sent Done false
22396 6997 Acked Write 10 Done Sent Done false
22397 6998 Acked Write 3 Done Sent Done false
22398 6999 Acked Read 10 Done Sent Done false
22399 7000 Acked Flush 0 Done Sent Done false
22400 7001 Acked Read 8 Done Sent Done false
22401 7002 Acked Read 10 Done Sent Done false
22402 7003 Acked Write 10 Done Sent Done false
22403 7004 Acked Read 7 Done Sent Done false
22404 7005 Acked Read 7 Done Sent Done false
22405 7006 Acked Write 10 Done Sent Done false
22406 7007 Acked Read 6 Done Sent Done false
22407 7008 Acked Read 8 Done Sent Done false
22408 7009 Acked Write 3 Done Sent Done false
22409 7010 Acked Write 7 Done Sent Done false
22410 7011 Acked Write 2 Done Sent Done false
22411 7012 Acked Write 2 Done Sent Done false
22412 7013 Acked Flush 0 Done Sent Done false
22413 7014 Acked Write 3 Done Sent Done false
22414 7015 Acked Read 7 Done Sent Done false
22415 7016 Acked Read 7 Done Sent Done false
22416 7017 Acked Write 7 Done Sent Done false
22417 7018 Acked Read 5 Done Sent Done false
22418 7019 Acked Read 4 Done Sent Done false
22419 7020 Acked Read 10 Done Sent Done false
22420 7021 Acked Write 4 Done Sent Done false
22421 7022 Acked Write 4 Done Sent Done false
22422 7023 Acked Read 7 Done Sent Done false
22423 7024 Acked Write 6 Done Sent Done false
22424 7025 Acked Flush 0 Done Sent Done false
22425 7026 Acked Read 8 Done Sent Done false
22426 7027 Acked Write 6 Done Sent Done false
22427 7028 Acked Read 3 Done Sent Done false
22428 7029 Acked Write 5 Done Sent Done false
22429 7030 Acked Flush 0 Done Sent Done false
22430 7031 Acked Write 8 Done Sent Done false
22431 7032 Acked Read 6 Done Sent Done false
22432 7033 Acked Write 8 Done Sent Done false
22433 7034 Acked Read 5 Done Sent Done false
22434 7035 Acked Write 9 Done Sent Done false
22435 STATES DS:0 DS:1 DS:2 TOTAL
22436 Sent 0 301 0 301
22437 Done 301 0 301 602
22438 Skipped 0 0 0 0
22439 Error 0 0 0 0
22440 Last Flush: 7030 6734 7030
22441 Downstairs last five completed: 6734 6733 6732 6731 6730
22442 Upstairs last five completed: 7035 7034 7033 7032 7031
22443 CLIENT: Up:0 ds:301 act:2
22444 ----------------------------------------------------------------
22445 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22446 States: Active Offline Active
22447 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22448 6735 Acked Write 6 Done Sent Done false
22449 6736 Acked Flush 0 Done Sent Done false
22450 6737 Acked Read 9 Done Sent Done false
22451 6738 Acked Read 10 Done Sent Done false
22452 6739 Acked Write 7 Done Sent Done false
22453 6740 Acked Read 1 Done Sent Done false
22454 6741 Acked Read 2 Done Sent Done false
22455 6742 Acked Read 3 Done Sent Done false
22456 6743 Acked Write 2 Done Sent Done false
22457 6744 Acked Write 7 Done Sent Done false
22458 6745 Acked Read 6 Done Sent Done false
22459 6746 Acked Write 9 Done Sent Done false
22460 6747 Acked Write 7 Done Sent Done false
22461 6748 Acked Write 4 Done Sent Done false
22462 6749 Acked Flush 0 Done Sent Done false
22463 6750 Acked Read 3 Done Sent Done false
22464 6751 Acked Write 1 Done Sent Done false
22465 6752 Acked Flush 0 Done Sent Done false
22466 6753 Acked Write 9 Done Sent Done false
22467 6754 Acked Read 7 Done Sent Done false
22468 6755 Acked Read 7 Done Sent Done false
22469 6756 Acked Write 8 Done Sent Done false
22470 6757 Acked Write 5 Done Sent Done false
22471 6758 Acked Write 5 Done Sent Done false
22472 6759 Acked Read 10 Done Sent Done false
22473 6760 Acked Flush 0 Done Sent Done false
22474 6761 Acked Read 1 Done Sent Done false
22475 6762 Acked Write 7 Done Sent Done false
22476 6763 Acked Read 6 Done Sent Done false
22477 6764 Acked Read 10 Done Sent Done false
22478 6765 Acked Read 7 Done Sent Done false
22479 6766 Acked Write 9 Done Sent Done false
22480 6767 Acked Write 5 Done Sent Done false
22481 6768 Acked Write 1 Done Sent Done false
22482 6769 Acked Read 2 Done Sent Done false
22483 6770 Acked Read 5 Done Sent Done false
22484 6771 Acked Flush 0 Done Sent Done false
22485 6772 Acked Read 9 Done Sent Done false
22486 6773 Acked Read 1 Done Sent Done false
22487 6774 Acked Read 3 Done Sent Done false
22488 6775 Acked Flush 0 Done Sent Done false
22489 6776 Acked Read 8 Done Sent Done false
22490 6777 Acked Read 1 Done Sent Done false
22491 6778 Acked Flush 0 Done Sent Done false
22492 6779 Acked Flush 0 Done Sent Done false
22493 6780 Acked Write 4 Done Sent Done false
22494 6781 Acked Read 2 Done Sent Done false
22495 6782 Acked Read 6 Done Sent Done false
22496 6783 Acked Read 6 Done Sent Done false
22497 6784 Acked Write 8 Done Sent Done false
22498 6785 Acked Write 3 Done Sent Done false
22499 6786 Acked Write 2 Done Sent Done false
22500 6787 Acked Read 5 Done Sent Done false
22501 6788 Acked Read 9 Done Sent Done false
22502 6789 Acked Write 9 Done Sent Done false
22503 6790 Acked Read 10 Done Sent Done false
22504 6791 Acked Write 3 Done Sent Done false
22505 6792 Acked Read 10 Done Sent Done false
22506 6793 Acked Read 10 Done Sent Done false
22507 6794 Acked Write 8 Done Sent Done false
22508 6795 Acked Read 10 Done Sent Done false
22509 6796 Acked Read 4 Done Sent Done false
22510 6797 Acked Write 9 Done Sent Done false
22511 6798 Acked Write 4 Done Sent Done false
22512 6799 Acked Read 1 Done Sent Done false
22513 6800 Acked Write 5 Done Sent Done false
22514 6801 Acked Write 9 Done Sent Done false
22515 6802 Acked Read 1 Done Sent Done false
22516 6803 Acked Read 9 Done Sent Done false
22517 6804 Acked Write 8 Done Sent Done false
22518 6805 Acked Write 8 Done Sent Done false
22519 6806 Acked Flush 0 Done Sent Done false
22520 6807 Acked Read 8 Done Sent Done false
22521 6808 Acked Read 5 Done Sent Done false
22522 6809 Acked Read 5 Done Sent Done false
22523 6810 Acked Write 3 Done Sent Done false
22524 6811 Acked Write 2 Done Sent Done false
22525 6812 Acked Read 1 Done Sent Done false
22526 6813 Acked Write 9 Done Sent Done false
22527 6814 Acked Write 9 Done Sent Done false
22528 6815 Acked Read 8 Done Sent Done false
22529 6816 Acked Write 10 Done Sent Done false
22530 6817 Acked Flush 0 Done Sent Done false
22531 6818 Acked Flush 0 Done Sent Done false
22532 6819 Acked Read 5 Done Sent Done false
22533 6820 Acked Write 9 Done Sent Done false
22534 6821 Acked Write 1 Done Sent Done false
22535 6822 Acked Read 8 Done Sent Done false
22536 6823 Acked Write 2 Done Sent Done false
22537 6824 Acked Write 10 Done Sent Done false
22538 6825 Acked Read 5 Done Sent Done false
22539 6826 Acked Write 8 Done Sent Done false
22540 6827 Acked Flush 0 Done Sent Done false
22541 6828 Acked Flush 0 Done Sent Done false
22542 6829 Acked Read 10 Done Sent Done false
22543 6830 Acked Write 8 Done Sent Done false
22544 6831 Acked Read 8 Done Sent Done false
22545 6832 Acked Read 8 Done Sent Done false
22546 6833 Acked Read 4 Done Sent Done false
22547 6834 Acked Read 10 Done Sent Done false
22548 6835 Acked Write 2 Done Sent Done false
22549 6836 Acked Write 8 Done Sent Done false
22550 6837 Acked Read 4 Done Sent Done false
22551 6838 Acked Read 2 Done Sent Done false
22552 6839 Acked Read 9 Done Sent Done false
22553 6840 Acked Write 6 Done Sent Done false
22554 6841 Acked Read 8 Done Sent Done false
22555 6842 Acked Write 3 Done Sent Done false
22556 6843 Acked Read 1 Done Sent Done false
22557 6844 Acked Flush 0 Done Sent Done false
22558 6845 Acked Write 3 Done Sent Done false
22559 6846 Acked Read 6 Done Sent Done false
22560 6847 Acked Read 1 Done Sent Done false
22561 6848 Acked Write 6 Done Sent Done false
22562 6849 Acked Read 7 Done Sent Done false
22563 6850 Acked Read 9 Done Sent Done false
22564 6851 Acked Flush 0 Done Sent Done false
22565 6852 Acked Flush 0 Done Sent Done false
22566 6853 Acked Write 1 Done Sent Done false
22567 6854 Acked Read 1 Done Sent Done false
22568 6855 Acked Read 10 Done Sent Done false
22569 6856 Acked Read 6 Done Sent Done false
22570 6857 Acked Write 7 Done Sent Done false
22571 6858 Acked Write 7 Done Sent Done false
22572 6859 Acked Write 5 Done Sent Done false
22573 6860 Acked Read 7 Done Sent Done false
22574 6861 Acked Flush 0 Done Sent Done false
22575 6862 Acked Read 7 Done Sent Done false
22576 6863 Acked Read 9 Done Sent Done false
22577 6864 Acked Read 3 Done Sent Done false
22578 6865 Acked Write 1 Done Sent Done false
22579 6866 Acked Write 6 Done Sent Done false
22580 6867 Acked Flush 0 Done Sent Done false
22581 6868 Acked Read 1 Done Sent Done false
22582 6869 Acked Read 10 Done Sent Done false
22583 6870 Acked Read 7 Done Sent Done false
22584 6871 Acked Read 7 Done Sent Done false
22585 6872 Acked Write 10 Done Sent Done false
22586 6873 Acked Read 4 Done Sent Done false
22587 6874 Acked Flush 0 Done Sent Done false
22588 6875 Acked Write 2 Done Sent Done false
22589 6876 Acked Read 6 Done Sent Done false
22590 6877 Acked Write 6 Done Sent Done false
22591 6878 Acked Write 6 Done Sent Done false
22592 6879 Acked Write 1 Done Sent Done false
22593 6880 Acked Read 9 Done Sent Done false
22594 6881 Acked Write 10 Done Sent Done false
22595 6882 Acked Flush 0 Done Sent Done false
22596 6883 Acked Read 3 Done Sent Done false
22597 6884 Acked Read 2 Done Sent Done false
22598 6885 Acked Flush 0 Done Sent Done false
22599 6886 Acked Write 4 Done Sent Done false
22600 6887 Acked Read 5 Done Sent Done false
22601 6888 Acked Read 2 Done Sent Done false
22602 6889 Acked Flush 0 Done Sent Done false
22603 6890 Acked Read 3 Done Sent Done false
22604 6891 Acked Read 1 Done Sent Done false
22605 6892 Acked Flush 0 Done Sent Done false
22606 6893 Acked Flush 0 Done Sent Done false
22607 6894 Acked Read 1 Done Sent Done false
22608 6895 Acked Read 7 Done Sent Done false
22609 6896 Acked Read 3 Done Sent Done false
22610 6897 Acked Read 9 Done Sent Done false
22611 6898 Acked Read 8 Done Sent Done false
22612 6899 Acked Read 4 Done Sent Done false
22613 6900 Acked Write 7 Done Sent Done false
22614 6901 Acked Read 10 Done Sent Done false
22615 6902 Acked Read 8 Done Sent Done false
22616 6903 Acked Read 7 Done Sent Done false
22617 6904 Acked Write 4 Done Sent Done false
22618 6905 Acked Write 10 Done Sent Done false
22619 6906 Acked Read 8 Done Sent Done false
22620 6907 Acked Write 3 Done Sent Done false
22621 6908 Acked Read 9 Done Sent Done false
22622 6909 Acked Write 1 Done Sent Done false
22623 6910 Acked Read 6 Done Sent Done false
22624 6911 Acked Read 4 Done Sent Done false
22625 6912 Acked Read 4 Done Sent Done false
22626 6913 Acked Read 1 Done Sent Done false
22627 6914 Acked Write 10 Done Sent Done false
22628 6915 Acked Read 2 Done Sent Done false
22629 6916 Acked Flush 0 Done Sent Done false
22630 6917 Acked Flush 0 Done Sent Done false
22631 6918 Acked Read 2 Done Sent Done false
22632 6919 Acked Read 2 Done Sent Done false
22633 6920 Acked Read 4 Done Sent Done false
22634 6921 Acked Write 5 Done Sent Done false
22635 6922 Acked Read 9 Done Sent Done false
22636 6923 Acked Read 4 Done Sent Done false
22637 6924 Acked Write 2 Done Sent Done false
22638 6925 Acked Write 3 Done Sent Done false
22639 6926 Acked Read 8 Done Sent Done false
22640 6927 Acked Write 2 Done Sent Done false
22641 6928 Acked Read 8 Done Sent Done false
22642 6929 Acked Write 4 Done Sent Done false
22643 6930 Acked Read 3 Done Sent Done false
22644 6931 Acked Read 7 Done Sent Done false
22645 6932 Acked Flush 0 Done Sent Done false
22646 6933 Acked Read 3 Done Sent Done false
22647 6934 Acked Flush 0 Done Sent Done false
22648 6935 Acked Read 9 Done Sent Done false
22649 6936 Acked Read 7 Done Sent Done false
22650 6937 Acked Write 10 Done Sent Done false
22651 6938 Acked Read 8 Done Sent Done false
22652 6939 Acked Write 7 Done Sent Done false
22653 6940 Acked Write 2 Done Sent Done false
22654 6941 Acked Write 6 Done Sent Done false
22655 6942 Acked Read 10 Done Sent Done false
22656 6943 Acked Write 3 Done Sent Done false
22657 6944 Acked Write 6 Done Sent Done false
22658 6945 Acked Write 1 Done Sent Done false
22659 6946 Acked Read 4 Done Sent Done false
22660 6947 Acked Write 10 Done Sent Done false
22661 6948 Acked Flush 0 Done Sent Done false
22662 6949 Acked Read 2 Done Sent Done false
22663 6950 Acked Read 2 Done Sent Done false
22664 6951 Acked Write 7 Done Sent Done false
22665 6952 Acked Write 1 Done Sent Done false
22666 6953 Acked Write 10 Done Sent Done false
22667 6954 Acked Write 8 Done Sent Done false
22668 6955 Acked Flush 0 Done Sent Done false
22669 6956 Acked Read 3 Done Sent Done false
22670 6957 Acked Read 6 Done Sent Done false
22671 6958 Acked Read 1 Done Sent Done false
22672 6959 Acked Flush 0 Done Sent Done false
22673 6960 Acked Write 9 Done Sent Done false
22674 6961 Acked Write 8 Done Sent Done false
22675 6962 Acked Write 1 Done Sent Done false
22676 6963 Acked Read 1 Done Sent Done false
22677 6964 Acked Read 9 Done Sent Done false
22678 6965 Acked Write 1 Done Sent Done false
22679 6966 Acked Read 10 Done Sent Done false
22680 6967 Acked Write 10 Done Sent Done false
22681 6968 Acked Read 3 Done Sent Done false
22682 6969 Acked Write 3 Done Sent Done false
22683 6970 Acked Write 8 Done Sent Done false
22684 6971 Acked Write 2 Done Sent Done false
22685 6972 Acked Read 6 Done Sent Done false
22686 6973 Acked Write 9 Done Sent Done false
22687 6974 Acked Flush 0 Done Sent Done false
22688 6975 Acked Flush 0 Done Sent Done false
22689 6976 Acked Write 8 Done Sent Done false
22690 6977 Acked Write 6 Done Sent Done false
22691 6978 Acked Read 3 Done Sent Done false
22692 6979 Acked Write 5 Done Sent Done false
22693 6980 Acked Read 10 Done Sent Done false
22694 6981 Acked Read 3 Done Sent Done false
22695 6982 Acked Write 2 Done Sent Done false
22696 6983 Acked Read 10 Done Sent Done false
22697 6984 Acked Read 4 Done Sent Done false
22698 6985 Acked Read 7 Done Sent Done false
22699 6986 Acked Read 4 Done Sent Done false
22700 6987 Acked Write 7 Done Sent Done false
22701 6988 Acked Read 10 Done Sent Done false
22702 6989 Acked Flush 0 Done Sent Done false
22703 6990 Acked Write 9 Done Sent Done false
22704 6991 Acked Read 3 Done Sent Done false
22705 6992 Acked Write 8 Done Sent Done false
22706 6993 Acked Read 2 Done Sent Done false
22707 6994 Acked Read 3 Done Sent Done false
22708 6995 Acked Flush 0 Done Sent Done false
22709 6996 Acked Read 5 Done Sent Done false
22710 6997 Acked Write 10 Done Sent Done false
22711 6998 Acked Write 3 Done Sent Done false
22712 6999 Acked Read 10 Done Sent Done false
22713 7000 Acked Flush 0 Done Sent Done false
22714 7001 Acked Read 8 Done Sent Done false
22715 7002 Acked Read 10 Done Sent Done false
22716 7003 Acked Write 10 Done Sent Done false
22717 7004 Acked Read 7 Done Sent Done false
22718 7005 Acked Read 7 Done Sent Done false
22719 7006 Acked Write 10 Done Sent Done false
22720 7007 Acked Read 6 Done Sent Done false
22721 7008 Acked Read 8 Done Sent Done false
22722 7009 Acked Write 3 Done Sent Done false
22723 7010 Acked Write 7 Done Sent Done false
22724 7011 Acked Write 2 Done Sent Done false
22725 7012 Acked Write 2 Done Sent Done false
22726 7013 Acked Flush 0 Done Sent Done false
22727 7014 Acked Write 3 Done Sent Done false
22728 7015 Acked Read 7 Done Sent Done false
22729 7016 Acked Read 7 Done Sent Done false
22730 7017 Acked Write 7 Done Sent Done false
22731 7018 Acked Read 5 Done Sent Done false
22732 7019 Acked Read 4 Done Sent Done false
22733 7020 Acked Read 10 Done Sent Done false
22734 7021 Acked Write 4 Done Sent Done false
22735 7022 Acked Write 4 Done Sent Done false
22736 7023 Acked Read 7 Done Sent Done false
22737 7024 Acked Write 6 Done Sent Done false
22738 7025 Acked Flush 0 Done Sent Done false
22739 7026 Acked Read 8 Done Sent Done false
22740 7027 Acked Write 6 Done Sent Done false
22741 7028 Acked Read 3 Done Sent Done false
22742 7029 Acked Write 5 Done Sent Done false
22743 7030 Acked Flush 0 Done Sent Done false
22744 7031 Acked Write 8 Done Sent Done false
22745 7032 Acked Read 6 Done Sent Done false
22746 7033 Acked Write 8 Done Sent Done false
22747 7034 Acked Read 5 Done Sent Done false
22748 7035 Acked Write 9 Done Sent Done false
22749 7036 Acked Flush 0 Done Sent Done false
22750 STATES DS:0 DS:1 DS:2 TOTAL
22751 Sent 0 302 0 302
22752 Done 302 0 302 604
22753 Skipped 0 0 0 0
22754 Error 0 0 0 0
22755 Last Flush: 7036 6734 7036
22756 Downstairs last five completed: 6734 6733 6732 6731 6730
22757 Upstairs last five completed: 7036 7035 7034 7033 7032
22758 CLIENT: Up:0 ds:302 act:2
227592025-01-17T21:15:51.153ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
227602025-01-17T21:15:51.153ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
227612025-01-17T21:15:51.153ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
227622025-01-17T21:15:51.153ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
227632025-01-17T21:15:51.153ZINFOcrucible: client task is exiting = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
22764 ----------------------------------------------------------------
22765 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22766 States: Active Offline Active
22767 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22768 6735 Acked Write 6 Done Sent Done false
22769 6736 Acked Flush 0 Done Sent Done false
22770 6737 Acked Read 9 Done Sent Done false
22771 6738 Acked Read 10 Done Sent Done false
22772 6739 Acked Write 7 Done Sent Done false
22773 6740 Acked Read 1 Done Sent Done false
22774 6741 Acked Read 2 Done Sent Done false
22775 6742 Acked Read 3 Done Sent Done false
22776 6743 Acked Write 2 Done Sent Done false
22777 6744 Acked Write 7 Done Sent Done false
22778 6745 Acked Read 6 Done Sent Done false
22779 6746 Acked Write 9 Done Sent Done false
22780 6747 Acked Write 7 Done Sent Done false
22781 6748 Acked Write 4 Done Sent Done false
22782 6749 Acked Flush 0 Done Sent Done false
22783 6750 Acked Read 3 Done Sent Done false
22784 6751 Acked Write 1 Done Sent Done false
22785 6752 Acked Flush 0 Done Sent Done false
22786 6753 Acked Write 9 Done Sent Done false
22787 6754 Acked Read 7 Done Sent Done false
22788 6755 Acked Read 7 Done Sent Done false
22789 6756 Acked Write 8 Done Sent Done false
22790 6757 Acked Write 5 Done Sent Done false
22791 6758 Acked Write 5 Done Sent Done false
22792 6759 Acked Read 10 Done Sent Done false
22793 6760 Acked Flush 0 Done Sent Done false
22794 6761 Acked Read 1 Done Sent Done false
22795 6762 Acked Write 7 Done Sent Done false
22796 6763 Acked Read 6 Done Sent Done false
22797 6764 Acked Read 10 Done Sent Done false
22798 6765 Acked Read 7 Done Sent Done false
22799 6766 Acked Write 9 Done Sent Done false
22800 6767 Acked Write 5 Done Sent Done false
22801 6768 Acked Write 1 Done Sent Done false
22802 6769 Acked Read 2 Done Sent Done false
22803 6770 Acked Read 5 Done Sent Done false
22804 6771 Acked Flush 0 Done Sent Done false
22805 6772 Acked Read 9 Done Sent Done false
22806 6773 Acked Read 1 Done Sent Done false
22807 6774 Acked Read 3 Done Sent Done false
22808 6775 Acked Flush 0 Done Sent Done false
22809 6776 Acked Read 8 Done Sent Done false
22810 6777 Acked Read 1 Done Sent Done false
22811 6778 Acked Flush 0 Done Sent Done false
22812 6779 Acked Flush 0 Done Sent Done false
22813 6780 Acked Write 4 Done Sent Done false
22814 6781 Acked Read 2 Done Sent Done false
22815 6782 Acked Read 6 Done Sent Done false
22816 6783 Acked Read 6 Done Sent Done false
22817 6784 Acked Write 8 Done Sent Done false
22818 6785 Acked Write 3 Done Sent Done false
22819 6786 Acked Write 2 Done Sent Done false
22820 6787 Acked Read 5 Done Sent Done false
22821 6788 Acked Read 9 Done Sent Done false
22822 6789 Acked Write 9 Done Sent Done false
22823 6790 Acked Read 10 Done Sent Done false
22824 6791 Acked Write 3 Done Sent Done false
22825 6792 Acked Read 10 Done Sent Done false
22826 6793 Acked Read 10 Done Sent Done false
22827 6794 Acked Write 8 Done Sent Done false
22828 6795 Acked Read 10 Done Sent Done false
22829 6796 Acked Read 4 Done Sent Done false
22830 6797 Acked Write 9 Done Sent Done false
22831 6798 Acked Write 4 Done Sent Done false
22832 6799 Acked Read 1 Done Sent Done false
22833 6800 Acked Write 5 Done Sent Done false
22834 6801 Acked Write 9 Done Sent Done false
22835 6802 Acked Read 1 Done Sent Done false
22836 6803 Acked Read 9 Done Sent Done false
22837 6804 Acked Write 8 Done Sent Done false
22838 6805 Acked Write 8 Done Sent Done false
22839 6806 Acked Flush 0 Done Sent Done false
22840 6807 Acked Read 8 Done Sent Done false
22841 6808 Acked Read 5 Done Sent Done false
22842 6809 Acked Read 5 Done Sent Done false
22843 6810 Acked Write 3 Done Sent Done false
22844 6811 Acked Write 2 Done Sent Done false
22845 6812 Acked Read 1 Done Sent Done false
22846 6813 Acked Write 9 Done Sent Done false
22847 6814 Acked Write 9 Done Sent Done false
22848 6815 Acked Read 8 Done Sent Done false
22849 6816 Acked Write 10 Done Sent Done false
22850 6817 Acked Flush 0 Done Sent Done false
22851 6818 Acked Flush 0 Done Sent Done false
22852 6819 Acked Read 5 Done Sent Done false
22853 6820 Acked Write 9 Done Sent Done false
22854 6821 Acked Write 1 Done Sent Done false
22855 6822 Acked Read 8 Done Sent Done false
22856 6823 Acked Write 2 Done Sent Done false
22857 6824 Acked Write 10 Done Sent Done false
22858 6825 Acked Read 5 Done Sent Done false
22859 6826 Acked Write 8 Done Sent Done false
22860 6827 Acked Flush 0 Done Sent Done false
22861 6828 Acked Flush 0 Done Sent Done false
22862 6829 Acked Read 10 Done Sent Done false
22863 6830 Acked Write 8 Done Sent Done false
22864 6831 Acked Read 8 Done Sent Done false
22865 6832 Acked Read 8 Done Sent Done false
22866 6833 Acked Read 4 Done Sent Done false
22867 6834 Acked Read 10 Done Sent Done false
22868 6835 Acked Write 2 Done Sent Done false
22869 6836 Acked Write 8 Done Sent Done false
22870 6837 Acked Read 4 Done Sent Done false
22871 6838 Acked Read 2 Done Sent Done false
22872 6839 Acked Read 9 Done Sent Done false
22873 6840 Acked Write 6 Done Sent Done false
22874 6841 Acked Read 8 Done Sent Done false
22875 6842 Acked Write 3 Done Sent Done false
22876 6843 Acked Read 1 Done Sent Done false
22877 6844 Acked Flush 0 Done Sent Done false
22878 6845 Acked Write 3 Done Sent Done false
22879 6846 Acked Read 6 Done Sent Done false
22880 6847 Acked Read 1 Done Sent Done false
22881 6848 Acked Write 6 Done Sent Done false
22882 6849 Acked Read 7 Done Sent Done false
22883 6850 Acked Read 9 Done Sent Done false
22884 6851 Acked Flush 0 Done Sent Done false
22885 6852 Acked Flush 0 Done Sent Done false
22886 6853 Acked Write 1 Done Sent Done false
22887 6854 Acked Read 1 Done Sent Done false
22888 6855 Acked Read 10 Done Sent Done false
22889 6856 Acked Read 6 Done Sent Done false
22890 6857 Acked Write 7 Done Sent Done false
22891 6858 Acked Write 7 Done Sent Done false
22892 6859 Acked Write 5 Done Sent Done false
22893 6860 Acked Read 7 Done Sent Done false
22894 6861 Acked Flush 0 Done Sent Done false
22895 6862 Acked Read 7 Done Sent Done false
22896 6863 Acked Read 9 Done Sent Done false
22897 6864 Acked Read 3 Done Sent Done false
22898 6865 Acked Write 1 Done Sent Done false
22899 6866 Acked Write 6 Done Sent Done false
22900 6867 Acked Flush 0 Done Sent Done false
22901 6868 Acked Read 1 Done Sent Done false
22902 6869 Acked Read 10 Done Sent Done false
22903 6870 Acked Read 7 Done Sent Done false
22904 6871 Acked Read 7 Done Sent Done false
22905 6872 Acked Write 10 Done Sent Done false
22906 6873 Acked Read 4 Done Sent Done false
22907 6874 Acked Flush 0 Done Sent Done false
22908 6875 Acked Write 2 Done Sent Done false
22909 6876 Acked Read 6 Done Sent Done false
22910 6877 Acked Write 6 Done Sent Done false
22911 6878 Acked Write 6 Done Sent Done false
22912 6879 Acked Write 1 Done Sent Done false
22913 6880 Acked Read 9 Done Sent Done false
22914 6881 Acked Write 10 Done Sent Done false
22915 6882 Acked Flush 0 Done Sent Done false
22916 6883 Acked Read 3 Done Sent Done false
22917 6884 Acked Read 2 Done Sent Done false
22918 6885 Acked Flush 0 Done Sent Done false
22919 6886 Acked Write 4 Done Sent Done false
22920 6887 Acked Read 5 Done Sent Done false
22921 6888 Acked Read 2 Done Sent Done false
22922 6889 Acked Flush 0 Done Sent Done false
22923 6890 Acked Read 3 Done Sent Done false
22924 6891 Acked Read 1 Done Sent Done false
22925 6892 Acked Flush 0 Done Sent Done false
22926 6893 Acked Flush 0 Done Sent Done false
22927 6894 Acked Read 1 Done Sent Done false
22928 6895 Acked Read 7 Done Sent Done false
22929 6896 Acked Read 3 Done Sent Done false
22930 6897 Acked Read 9 Done Sent Done false
22931 6898 Acked Read 8 Done Sent Done false
22932 6899 Acked Read 4 Done Sent Done false
22933 6900 Acked Write 7 Done Sent Done false
22934 6901 Acked Read 10 Done Sent Done false
22935 6902 Acked Read 8 Done Sent Done false
22936 6903 Acked Read 7 Done Sent Done false
22937 6904 Acked Write 4 Done Sent Done false
22938 6905 Acked Write 10 Done Sent Done false
22939 6906 Acked Read 8 Done Sent Done false
22940 6907 Acked Write 3 Done Sent Done false
22941 6908 Acked Read 9 Done Sent Done false
22942 6909 Acked Write 1 Done Sent Done false
22943 6910 Acked Read 6 Done Sent Done false
22944 6911 Acked Read 4 Done Sent Done false
22945 6912 Acked Read 4 Done Sent Done false
22946 6913 Acked Read 1 Done Sent Done false
22947 6914 Acked Write 10 Done Sent Done false
22948 6915 Acked Read 2 Done Sent Done false
22949 6916 Acked Flush 0 Done Sent Done false
22950 6917 Acked Flush 0 Done Sent Done false
22951 6918 Acked Read 2 Done Sent Done false
22952 6919 Acked Read 2 Done Sent Done false
22953 6920 Acked Read 4 Done Sent Done false
22954 6921 Acked Write 5 Done Sent Done false
22955 6922 Acked Read 9 Done Sent Done false
22956 6923 Acked Read 4 Done Sent Done false
22957 6924 Acked Write 2 Done Sent Done false
22958 6925 Acked Write 3 Done Sent Done false
22959 6926 Acked Read 8 Done Sent Done false
22960 6927 Acked Write 2 Done Sent Done false
22961 6928 Acked Read 8 Done Sent Done false
22962 6929 Acked Write 4 Done Sent Done false
22963 6930 Acked Read 3 Done Sent Done false
22964 6931 Acked Read 7 Done Sent Done false
22965 6932 Acked Flush 0 Done Sent Done false
22966 6933 Acked Read 3 Done Sent Done false
22967 6934 Acked Flush 0 Done Sent Done false
22968 6935 Acked Read 9 Done Sent Done false
22969 6936 Acked Read 7 Done Sent Done false
22970 6937 Acked Write 10 Done Sent Done false
22971 6938 Acked Read 8 Done Sent Done false
22972 6939 Acked Write 7 Done Sent Done false
22973 6940 Acked Write 2 Done Sent Done false
22974 6941 Acked Write 6 Done Sent Done false
22975 6942 Acked Read 10 Done Sent Done false
22976 6943 Acked Write 3 Done Sent Done false
22977 6944 Acked Write 6 Done Sent Done false
22978 6945 Acked Write 1 Done Sent Done false
22979 6946 Acked Read 4 Done Sent Done false
22980 6947 Acked Write 10 Done Sent Done false
22981 6948 Acked Flush 0 Done Sent Done false
22982 6949 Acked Read 2 Done Sent Done false
22983 6950 Acked Read 2 Done Sent Done false
22984 6951 Acked Write 7 Done Sent Done false
22985 6952 Acked Write 1 Done Sent Done false
22986 6953 Acked Write 10 Done Sent Done false
22987 6954 Acked Write 8 Done Sent Done false
22988 6955 Acked Flush 0 Done Sent Done false
22989 6956 Acked Read 3 Done Sent Done false
22990 6957 Acked Read 6 Done Sent Done false
22991 6958 Acked Read 1 Done Sent Done false
22992 6959 Acked Flush 0 Done Sent Done false
22993 6960 Acked Write 9 Done Sent Done false
22994 6961 Acked Write 8 Done Sent Done false
22995 6962 Acked Write 1 Done Sent Done false
22996 6963 Acked Read 1 Done Sent Done false
22997 6964 Acked Read 9 Done Sent Done false
22998 6965 Acked Write 1 Done Sent Done false
22999 6966 Acked Read 10 Done Sent Done false
23000 6967 Acked Write 10 Done Sent Done false
23001 6968 Acked Read 3 Done Sent Done false
23002 6969 Acked Write 3 Done Sent Done false
23003 6970 Acked Write 8 Done Sent Done false
23004 6971 Acked Write 2 Done Sent Done false
23005 6972 Acked Read 6 Done Sent Done false
23006 6973 Acked Write 9 Done Sent Done false
23007 6974 Acked Flush 0 Done Sent Done false
23008 6975 Acked Flush 0 Done Sent Done false
23009 6976 Acked Write 8 Done Sent Done false
23010 6977 Acked Write 6 Done Sent Done false
23011 6978 Acked Read 3 Done Sent Done false
23012 6979 Acked Write 5 Done Sent Done false
23013 6980 Acked Read 10 Done Sent Done false
23014 6981 Acked Read 3 Done Sent Done false
23015 6982 Acked Write 2 Done Sent Done false
23016 6983 Acked Read 10 Done Sent Done false
23017 6984 Acked Read 4 Done Sent Done false
23018 6985 Acked Read 7 Done Sent Done false
23019 6986 Acked Read 4 Done Sent Done false
23020 6987 Acked Write 7 Done Sent Done false
23021 6988 Acked Read 10 Done Sent Done false
23022 6989 Acked Flush 0 Done Sent Done false
23023 6990 Acked Write 9 Done Sent Done false
23024 6991 Acked Read 3 Done Sent Done false
23025 6992 Acked Write 8 Done Sent Done false
23026 6993 Acked Read 2 Done Sent Done false
23027 6994 Acked Read 3 Done Sent Done false
23028 6995 Acked Flush 0 Done Sent Done false
23029 6996 Acked Read 5 Done Sent Done false
23030 6997 Acked Write 10 Done Sent Done false
23031 6998 Acked Write 3 Done Sent Done false
23032 6999 Acked Read 10 Done Sent Done false
23033 7000 Acked Flush 0 Done Sent Done false
23034 7001 Acked Read 8 Done Sent Done false
23035 7002 Acked Read 10 Done Sent Done false
23036 7003 Acked Write 10 Done Sent Done false
23037 7004 Acked Read 7 Done Sent Done false
23038 7005 Acked Read 7 Done Sent Done false
23039 7006 Acked Write 10 Done Sent Done false
23040 7007 Acked Read 6 Done Sent Done false
23041 7008 Acked Read 8 Done Sent Done false
23042 7009 Acked Write 3 Done Sent Done false
23043 7010 Acked Write 7 Done Sent Done false
23044 7011 Acked Write 2 Done Sent Done false
23045 7012 Acked Write 2 Done Sent Done false
23046 7013 Acked Flush 0 Done Sent Done false
23047 7014 Acked Write 3 Done Sent Done false
23048 7015 Acked Read 7 Done Sent Done false
23049 7016 Acked Read 7 Done Sent Done false
23050 7017 Acked Write 7 Done Sent Done false
23051 7018 Acked Read 5 Done Sent Done false
23052 7019 Acked Read 4 Done Sent Done false
23053 7020 Acked Read 10 Done Sent Done false
23054 7021 Acked Write 4 Done Sent Done false
23055 7022 Acked Write 4 Done Sent Done false
23056 7023 Acked Read 7 Done Sent Done false
23057 7024 Acked Write 6 Done Sent Done false
23058 7025 Acked Flush 0 Done Sent Done false
23059 7026 Acked Read 8 Done Sent Done false
23060 7027 Acked Write 6 Done Sent Done false
23061 7028 Acked Read 3 Done Sent Done false
23062 7029 Acked Write 5 Done Sent Done false
23063 7030 Acked Flush 0 Done Sent Done false
23064 7031 Acked Write 8 Done Sent Done false
23065 7032 Acked Read 6 Done Sent Done false
23066 7033 Acked Write 8 Done Sent Done false
23067 7034 Acked Read 5 Done Sent Done false
23068 7035 Acked Write 9 Done Sent Done false
23069 7036 Acked Flush 0 Done Sent Done false
23070 STATES DS:0 DS:1 DS:2 TOTAL
23071 Sent 0 302 0 302
23072 Done 302 0 302 604
23073 Skipped 0 0 0 0
23074 Error 0 0 0 0
23075 Last Flush: 7036 6734 7036
23076 Downstairs last five completed: 6734 6733 6732 6731 6730
23077 Upstairs last five completed: 7036 7035 7034 7033 7032
23078 CLIENT: Up:0 ds:302 act:2
23079 ----------------------------------------------------------------
23080 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23081 States: Active Offline Active
23082 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23083 6735 Acked Write 6 Done Sent Done false
23084 6736 Acked Flush 0 Done Sent Done false
23085 6737 Acked Read 9 Done Sent Done false
23086 6738 Acked Read 10 Done Sent Done false
23087 6739 Acked Write 7 Done Sent Done false
23088 6740 Acked Read 1 Done Sent Done false
23089 6741 Acked Read 2 Done Sent Done false
23090 6742 Acked Read 3 Done Sent Done false
23091 6743 Acked Write 2 Done Sent Done false
23092 6744 Acked Write 7 Done Sent Done false
23093 6745 Acked Read 6 Done Sent Done false
23094 6746 Acked Write 9 Done Sent Done false
23095 6747 Acked Write 7 Done Sent Done false
23096 6748 Acked Write 4 Done Sent Done false
23097 6749 Acked Flush 0 Done Sent Done false
23098 6750 Acked Read 3 Done Sent Done false
23099 6751 Acked Write 1 Done Sent Done false
23100 6752 Acked Flush 0 Done Sent Done false
23101 6753 Acked Write 9 Done Sent Done false
23102 6754 Acked Read 7 Done Sent Done false
23103 6755 Acked Read 7 Done Sent Done false
23104 6756 Acked Write 8 Done Sent Done false
23105 6757 Acked Write 5 Done Sent Done false
23106 6758 Acked Write 5 Done Sent Done false
23107 6759 Acked Read 10 Done Sent Done false
23108 6760 Acked Flush 0 Done Sent Done false
23109 6761 Acked Read 1 Done Sent Done false
23110 6762 Acked Write 7 Done Sent Done false
23111 6763 Acked Read 6 Done Sent Done false
23112 6764 Acked Read 10 Done Sent Done false
23113 6765 Acked Read 7 Done Sent Done false
23114 6766 Acked Write 9 Done Sent Done false
23115 6767 Acked Write 5 Done Sent Done false
23116 6768 Acked Write 1 Done Sent Done false
23117 6769 Acked Read 2 Done Sent Done false
23118 6770 Acked Read 5 Done Sent Done false
23119 6771 Acked Flush 0 Done Sent Done false
23120 6772 Acked Read 9 Done Sent Done false
23121 6773 Acked Read 1 Done Sent Done false
23122 6774 Acked Read 3 Done Sent Done false
23123 6775 Acked Flush 0 Done Sent Done false
23124 6776 Acked Read 8 Done Sent Done false
23125 6777 Acked Read 1 Done Sent Done false
23126 6778 Acked Flush 0 Done Sent Done false
23127 6779 Acked Flush 0 Done Sent Done false
23128 6780 Acked Write 4 Done Sent Done false
23129 6781 Acked Read 2 Done Sent Done false
23130 6782 Acked Read 6 Done Sent Done false
23131 6783 Acked Read 6 Done Sent Done false
23132 6784 Acked Write 8 Done Sent Done false
23133 6785 Acked Write 3 Done Sent Done false
23134 6786 Acked Write 2 Done Sent Done false
23135 6787 Acked Read 5 Done Sent Done false
23136 6788 Acked Read 9 Done Sent Done false
23137 6789 Acked Write 9 Done Sent Done false
23138 6790 Acked Read 10 Done Sent Done false
23139 6791 Acked Write 3 Done Sent Done false
23140 6792 Acked Read 10 Done Sent Done false
23141 6793 Acked Read 10 Done Sent Done false
23142 6794 Acked Write 8 Done Sent Done false
23143 6795 Acked Read 10 Done Sent Done false
23144 6796 Acked Read 4 Done Sent Done false
23145 6797 Acked Write 9 Done Sent Done false
23146 6798 Acked Write 4 Done Sent Done false
23147 6799 Acked Read 1 Done Sent Done false
23148 6800 Acked Write 5 Done Sent Done false
23149 6801 Acked Write 9 Done Sent Done false
23150 6802 Acked Read 1 Done Sent Done false
23151 6803 Acked Read 9 Done Sent Done false
23152 6804 Acked Write 8 Done Sent Done false
23153 6805 Acked Write 8 Done Sent Done false
23154 6806 Acked Flush 0 Done Sent Done false
23155 6807 Acked Read 8 Done Sent Done false
23156 6808 Acked Read 5 Done Sent Done false
23157 6809 Acked Read 5 Done Sent Done false
23158 6810 Acked Write 3 Done Sent Done false
23159 6811 Acked Write 2 Done Sent Done false
23160 6812 Acked Read 1 Done Sent Done false
23161 6813 Acked Write 9 Done Sent Done false
23162 6814 Acked Write 9 Done Sent Done false
23163 6815 Acked Read 8 Done Sent Done false
23164 6816 Acked Write 10 Done Sent Done false
23165 6817 Acked Flush 0 Done Sent Done false
23166 6818 Acked Flush 0 Done Sent Done false
23167 6819 Acked Read 5 Done Sent Done false
23168 6820 Acked Write 9 Done Sent Done false
23169 6821 Acked Write 1 Done Sent Done false
23170 6822 Acked Read 8 Done Sent Done false
23171 6823 Acked Write 2 Done Sent Done false
23172 6824 Acked Write 10 Done Sent Done false
23173 6825 Acked Read 5 Done Sent Done false
23174 6826 Acked Write 8 Done Sent Done false
23175 6827 Acked Flush 0 Done Sent Done false
23176 6828 Acked Flush 0 Done Sent Done false
23177 6829 Acked Read 10 Done Sent Done false
23178 6830 Acked Write 8 Done Sent Done false
23179 6831 Acked Read 8 Done Sent Done false
23180 6832 Acked Read 8 Done Sent Done false
23181 6833 Acked Read 4 Done Sent Done false
23182 6834 Acked Read 10 Done Sent Done false
23183 6835 Acked Write 2 Done Sent Done false
23184 6836 Acked Write 8 Done Sent Done false
23185 6837 Acked Read 4 Done Sent Done false
23186 6838 Acked Read 2 Done Sent Done false
23187 6839 Acked Read 9 Done Sent Done false
23188 6840 Acked Write 6 Done Sent Done false
23189 6841 Acked Read 8 Done Sent Done false
23190 6842 Acked Write 3 Done Sent Done false
23191 6843 Acked Read 1 Done Sent Done false
23192 6844 Acked Flush 0 Done Sent Done false
23193 6845 Acked Write 3 Done Sent Done false
23194 6846 Acked Read 6 Done Sent Done false
23195 6847 Acked Read 1 Done Sent Done false
23196 6848 Acked Write 6 Done Sent Done false
23197 6849 Acked Read 7 Done Sent Done false
23198 6850 Acked Read 9 Done Sent Done false
23199 6851 Acked Flush 0 Done Sent Done false
23200 6852 Acked Flush 0 Done Sent Done false
23201 6853 Acked Write 1 Done Sent Done false
23202 6854 Acked Read 1 Done Sent Done false
23203 6855 Acked Read 10 Done Sent Done false
23204 6856 Acked Read 6 Done Sent Done false
23205 6857 Acked Write 7 Done Sent Done false
23206 6858 Acked Write 7 Done Sent Done false
23207 6859 Acked Write 5 Done Sent Done false
23208 6860 Acked Read 7 Done Sent Done false
23209 6861 Acked Flush 0 Done Sent Done false
23210 6862 Acked Read 7 Done Sent Done false
23211 6863 Acked Read 9 Done Sent Done false
23212 6864 Acked Read 3 Done Sent Done false
23213 6865 Acked Write 1 Done Sent Done false
23214 6866 Acked Write 6 Done Sent Done false
23215 6867 Acked Flush 0 Done Sent Done false
23216 6868 Acked Read 1 Done Sent Done false
23217 6869 Acked Read 10 Done Sent Done false
23218 6870 Acked Read 7 Done Sent Done false
23219 6871 Acked Read 7 Done Sent Done false
23220 6872 Acked Write 10 Done Sent Done false
23221 6873 Acked Read 4 Done Sent Done false
23222 6874 Acked Flush 0 Done Sent Done false
23223 6875 Acked Write 2 Done Sent Done false
23224 6876 Acked Read 6 Done Sent Done false
23225 6877 Acked Write 6 Done Sent Done false
23226 6878 Acked Write 6 Done Sent Done false
23227 6879 Acked Write 1 Done Sent Done false
23228 6880 Acked Read 9 Done Sent Done false
23229 6881 Acked Write 10 Done Sent Done false
23230 6882 Acked Flush 0 Done Sent Done false
23231 6883 Acked Read 3 Done Sent Done false
23232 6884 Acked Read 2 Done Sent Done false
23233 6885 Acked Flush 0 Done Sent Done false
23234 6886 Acked Write 4 Done Sent Done false
23235 6887 Acked Read 5 Done Sent Done false
23236 6888 Acked Read 2 Done Sent Done false
23237 6889 Acked Flush 0 Done Sent Done false
23238 6890 Acked Read 3 Done Sent Done false
23239 6891 Acked Read 1 Done Sent Done false
23240 6892 Acked Flush 0 Done Sent Done false
23241 6893 Acked Flush 0 Done Sent Done false
23242 6894 Acked Read 1 Done Sent Done false
23243 6895 Acked Read 7 Done Sent Done false
23244 6896 Acked Read 3 Done Sent Done false
23245 6897 Acked Read 9 Done Sent Done false
23246 6898 Acked Read 8 Done Sent Done false
23247 6899 Acked Read 4 Done Sent Done false
23248 6900 Acked Write 7 Done Sent Done false
23249 6901 Acked Read 10 Done Sent Done false
23250 6902 Acked Read 8 Done Sent Done false
23251 6903 Acked Read 7 Done Sent Done false
23252 6904 Acked Write 4 Done Sent Done false
23253 6905 Acked Write 10 Done Sent Done false
23254 6906 Acked Read 8 Done Sent Done false
23255 6907 Acked Write 3 Done Sent Done false
23256 6908 Acked Read 9 Done Sent Done false
23257 6909 Acked Write 1 Done Sent Done false
23258 6910 Acked Read 6 Done Sent Done false
23259 6911 Acked Read 4 Done Sent Done false
23260 6912 Acked Read 4 Done Sent Done false
23261 6913 Acked Read 1 Done Sent Done false
23262 6914 Acked Write 10 Done Sent Done false
23263 6915 Acked Read 2 Done Sent Done false
23264 6916 Acked Flush 0 Done Sent Done false
23265 6917 Acked Flush 0 Done Sent Done false
23266 6918 Acked Read 2 Done Sent Done false
23267 6919 Acked Read 2 Done Sent Done false
23268 6920 Acked Read 4 Done Sent Done false
23269 6921 Acked Write 5 Done Sent Done false
23270 6922 Acked Read 9 Done Sent Done false
23271 6923 Acked Read 4 Done Sent Done false
23272 6924 Acked Write 2 Done Sent Done false
23273 6925 Acked Write 3 Done Sent Done false
23274 6926 Acked Read 8 Done Sent Done false
23275 6927 Acked Write 2 Done Sent Done false
23276 6928 Acked Read 8 Done Sent Done false
23277 6929 Acked Write 4 Done Sent Done false
23278 6930 Acked Read 3 Done Sent Done false
23279 6931 Acked Read 7 Done Sent Done false
23280 6932 Acked Flush 0 Done Sent Done false
23281 6933 Acked Read 3 Done Sent Done false
23282 6934 Acked Flush 0 Done Sent Done false
23283 6935 Acked Read 9 Done Sent Done false
23284 6936 Acked Read 7 Done Sent Done false
23285 6937 Acked Write 10 Done Sent Done false
23286 6938 Acked Read 8 Done Sent Done false
23287 6939 Acked Write 7 Done Sent Done false
23288 6940 Acked Write 2 Done Sent Done false
23289 6941 Acked Write 6 Done Sent Done false
23290 6942 Acked Read 10 Done Sent Done false
23291 6943 Acked Write 3 Done Sent Done false
23292 6944 Acked Write 6 Done Sent Done false
23293 6945 Acked Write 1 Done Sent Done false
23294 6946 Acked Read 4 Done Sent Done false
23295 6947 Acked Write 10 Done Sent Done false
23296 6948 Acked Flush 0 Done Sent Done false
23297 6949 Acked Read 2 Done Sent Done false
23298 6950 Acked Read 2 Done Sent Done false
23299 6951 Acked Write 7 Done Sent Done false
23300 6952 Acked Write 1 Done Sent Done false
23301 6953 Acked Write 10 Done Sent Done false
23302 6954 Acked Write 8 Done Sent Done false
23303 6955 Acked Flush 0 Done Sent Done false
23304 6956 Acked Read 3 Done Sent Done false
23305 6957 Acked Read 6 Done Sent Done false
23306 6958 Acked Read 1 Done Sent Done false
23307 6959 Acked Flush 0 Done Sent Done false
23308 6960 Acked Write 9 Done Sent Done false
23309 6961 Acked Write 8 Done Sent Done false
23310 6962 Acked Write 1 Done Sent Done false
23311 6963 Acked Read 1 Done Sent Done false
23312 6964 Acked Read 9 Done Sent Done false
23313 6965 Acked Write 1 Done Sent Done false
23314 6966 Acked Read 10 Done Sent Done false
23315 6967 Acked Write 10 Done Sent Done false
23316 6968 Acked Read 3 Done Sent Done false
23317 6969 Acked Write 3 Done Sent Done false
23318 6970 Acked Write 8 Done Sent Done false
23319 6971 Acked Write 2 Done Sent Done false
23320 6972 Acked Read 6 Done Sent Done false
23321 6973 Acked Write 9 Done Sent Done false
23322 6974 Acked Flush 0 Done Sent Done false
23323 6975 Acked Flush 0 Done Sent Done false
23324 6976 Acked Write 8 Done Sent Done false
23325 6977 Acked Write 6 Done Sent Done false
23326 6978 Acked Read 3 Done Sent Done false
23327 6979 Acked Write 5 Done Sent Done false
23328 6980 Acked Read 10 Done Sent Done false
23329 6981 Acked Read 3 Done Sent Done false
23330 6982 Acked Write 2 Done Sent Done false
23331 6983 Acked Read 10 Done Sent Done false
23332 6984 Acked Read 4 Done Sent Done false
23333 6985 Acked Read 7 Done Sent Done false
23334 6986 Acked Read 4 Done Sent Done false
23335 6987 Acked Write 7 Done Sent Done false
23336 6988 Acked Read 10 Done Sent Done false
23337 6989 Acked Flush 0 Done Sent Done false
23338 6990 Acked Write 9 Done Sent Done false
23339 6991 Acked Read 3 Done Sent Done false
23340 6992 Acked Write 8 Done Sent Done false
23341 6993 Acked Read 2 Done Sent Done false
23342 6994 Acked Read 3 Done Sent Done false
23343 6995 Acked Flush 0 Done Sent Done false
23344 6996 Acked Read 5 Done Sent Done false
23345 6997 Acked Write 10 Done Sent Done false
23346 6998 Acked Write 3 Done Sent Done false
23347 6999 Acked Read 10 Done Sent Done false
23348 7000 Acked Flush 0 Done Sent Done false
23349 7001 Acked Read 8 Done Sent Done false
23350 7002 Acked Read 10 Done Sent Done false
23351 7003 Acked Write 10 Done Sent Done false
23352 7004 Acked Read 7 Done Sent Done false
23353 7005 Acked Read 7 Done Sent Done false
23354 7006 Acked Write 10 Done Sent Done false
23355 7007 Acked Read 6 Done Sent Done false
23356 7008 Acked Read 8 Done Sent Done false
23357 7009 Acked Write 3 Done Sent Done false
23358 7010 Acked Write 7 Done Sent Done false
23359 7011 Acked Write 2 Done Sent Done false
23360 7012 Acked Write 2 Done Sent Done false
23361 7013 Acked Flush 0 Done Sent Done false
23362 7014 Acked Write 3 Done Sent Done false
23363 7015 Acked Read 7 Done Sent Done false
23364 7016 Acked Read 7 Done Sent Done false
23365 7017 Acked Write 7 Done Sent Done false
23366 7018 Acked Read 5 Done Sent Done false
23367 7019 Acked Read 4 Done Sent Done false
23368 7020 Acked Read 10 Done Sent Done false
23369 7021 Acked Write 4 Done Sent Done false
23370 7022 Acked Write 4 Done Sent Done false
23371 7023 Acked Read 7 Done Sent Done false
23372 7024 Acked Write 6 Done Sent Done false
23373 7025 Acked Flush 0 Done Sent Done false
23374 7026 Acked Read 8 Done Sent Done false
23375 7027 Acked Write 6 Done Sent Done false
23376 7028 Acked Read 3 Done Sent Done false
23377 7029 Acked Write 5 Done Sent Done false
23378 7030 Acked Flush 0 Done Sent Done false
23379 7031 Acked Write 8 Done Sent Done false
23380 7032 Acked Read 6 Done Sent Done false
23381 7033 Acked Write 8 Done Sent Done false
23382 7034 Acked Read 5 Done Sent Done false
23383 7035 Acked Write 9 Done Sent Done false
23384 7036 Acked Flush 0 Done Sent Done false
23385 STATES DS:0 DS:1 DS:2 TOTAL
23386 Sent 0 302 0 302
23387 Done 302 0 302 604
23388 Skipped 0 0 0 0
23389 Error 0 0 0 0
23390 Last Flush: 7036 6734 7036
23391 Downstairs last five completed: 6734 6733 6732 6731 6730
23392 Upstairs last five completed: 7036 7035 7034 7033 7032
23393 CLIENT: Up:0 ds:302 act:2
233942025-01-17T21:16:01.155ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
233952025-01-17T21:16:01.155ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
233962025-01-17T21:16:01.155ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
233972025-01-17T21:16:01.155ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
233982025-01-17T21:16:01.155ZINFOcrucible: client task is exiting = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
23399 [8820] got start action
23400 [8820] I am going to restart
23401 Starting downstairs at port 8820
23402 Make output file at "/tmp/dsc/downstairs-8820.txt"
23403 [8820][1] reports Starting
23404 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1220)
23405 [8820][1] reports Running
23406 ----------------------------------------------------------------
23407 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23408 States: Active Offline Active
23409 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23410 6735 Acked Write 6 Done Sent Done false
23411 6736 Acked Flush 0 Done Sent Done false
23412 6737 Acked Read 9 Done Sent Done false
23413 6738 Acked Read 10 Done Sent Done false
23414 6739 Acked Write 7 Done Sent Done false
23415 6740 Acked Read 1 Done Sent Done false
23416 6741 Acked Read 2 Done Sent Done false
23417 6742 Acked Read 3 Done Sent Done false
23418 6743 Acked Write 2 Done Sent Done false
23419 6744 Acked Write 7 Done Sent Done false
23420 6745 Acked Read 6 Done Sent Done false
23421 6746 Acked Write 9 Done Sent Done false
23422 6747 Acked Write 7 Done Sent Done false
23423 6748 Acked Write 4 Done Sent Done false
23424 6749 Acked Flush 0 Done Sent Done false
23425 6750 Acked Read 3 Done Sent Done false
23426 6751 Acked Write 1 Done Sent Done false
23427 6752 Acked Flush 0 Done Sent Done false
23428 6753 Acked Write 9 Done Sent Done false
23429 6754 Acked Read 7 Done Sent Done false
23430 6755 Acked Read 7 Done Sent Done false
23431 6756 Acked Write 8 Done Sent Done false
23432 6757 Acked Write 5 Done Sent Done false
23433 6758 Acked Write 5 Done Sent Done false
23434 6759 Acked Read 10 Done Sent Done false
23435 6760 Acked Flush 0 Done Sent Done false
23436 6761 Acked Read 1 Done Sent Done false
23437 6762 Acked Write 7 Done Sent Done false
23438 6763 Acked Read 6 Done Sent Done false
23439 6764 Acked Read 10 Done Sent Done false
23440 6765 Acked Read 7 Done Sent Done false
23441 6766 Acked Write 9 Done Sent Done false
23442 6767 Acked Write 5 Done Sent Done false
23443 6768 Acked Write 1 Done Sent Done false
23444 6769 Acked Read 2 Done Sent Done false
23445 6770 Acked Read 5 Done Sent Done false
23446 6771 Acked Flush 0 Done Sent Done false
23447 6772 Acked Read 9 Done Sent Done false
23448 6773 Acked Read 1 Done Sent Done false
23449 6774 Acked Read 3 Done Sent Done false
23450 6775 Acked Flush 0 Done Sent Done false
23451 6776 Acked Read 8 Done Sent Done false
23452 6777 Acked Read 1 Done Sent Done false
23453 6778 Acked Flush 0 Done Sent Done false
23454 6779 Acked Flush 0 Done Sent Done false
23455 6780 Acked Write 4 Done Sent Done false
23456 6781 Acked Read 2 Done Sent Done false
23457 6782 Acked Read 6 Done Sent Done false
23458 6783 Acked Read 6 Done Sent Done false
23459 6784 Acked Write 8 Done Sent Done false
23460 6785 Acked Write 3 Done Sent Done false
23461 6786 Acked Write 2 Done Sent Done false
23462 6787 Acked Read 5 Done Sent Done false
23463 6788 Acked Read 9 Done Sent Done false
23464 6789 Acked Write 9 Done Sent Done false
23465 6790 Acked Read 10 Done Sent Done false
23466 6791 Acked Write 3 Done Sent Done false
23467 6792 Acked Read 10 Done Sent Done false
23468 6793 Acked Read 10 Done Sent Done false
23469 6794 Acked Write 8 Done Sent Done false
23470 6795 Acked Read 10 Done Sent Done false
23471 6796 Acked Read 4 Done Sent Done false
23472 6797 Acked Write 9 Done Sent Done false
23473 6798 Acked Write 4 Done Sent Done false
23474 6799 Acked Read 1 Done Sent Done false
23475 6800 Acked Write 5 Done Sent Done false
23476 6801 Acked Write 9 Done Sent Done false
23477 6802 Acked Read 1 Done Sent Done false
23478 6803 Acked Read 9 Done Sent Done false
23479 6804 Acked Write 8 Done Sent Done false
23480 6805 Acked Write 8 Done Sent Done false
23481 6806 Acked Flush 0 Done Sent Done false
23482 6807 Acked Read 8 Done Sent Done false
23483 6808 Acked Read 5 Done Sent Done false
23484 6809 Acked Read 5 Done Sent Done false
23485 6810 Acked Write 3 Done Sent Done false
23486 6811 Acked Write 2 Done Sent Done false
23487 6812 Acked Read 1 Done Sent Done false
23488 6813 Acked Write 9 Done Sent Done false
23489 6814 Acked Write 9 Done Sent Done false
23490 6815 Acked Read 8 Done Sent Done false
23491 6816 Acked Write 10 Done Sent Done false
23492 6817 Acked Flush 0 Done Sent Done false
23493 6818 Acked Flush 0 Done Sent Done false
23494 6819 Acked Read 5 Done Sent Done false
23495 6820 Acked Write 9 Done Sent Done false
23496 6821 Acked Write 1 Done Sent Done false
23497 6822 Acked Read 8 Done Sent Done false
23498 6823 Acked Write 2 Done Sent Done false
23499 6824 Acked Write 10 Done Sent Done false
23500 6825 Acked Read 5 Done Sent Done false
23501 6826 Acked Write 8 Done Sent Done false
23502 6827 Acked Flush 0 Done Sent Done false
23503 6828 Acked Flush 0 Done Sent Done false
23504 6829 Acked Read 10 Done Sent Done false
23505 6830 Acked Write 8 Done Sent Done false
23506 6831 Acked Read 8 Done Sent Done false
23507 6832 Acked Read 8 Done Sent Done false
23508 6833 Acked Read 4 Done Sent Done false
23509 6834 Acked Read 10 Done Sent Done false
23510 6835 Acked Write 2 Done Sent Done false
23511 6836 Acked Write 8 Done Sent Done false
23512 6837 Acked Read 4 Done Sent Done false
23513 6838 Acked Read 2 Done Sent Done false
23514 6839 Acked Read 9 Done Sent Done false
23515 6840 Acked Write 6 Done Sent Done false
23516 6841 Acked Read 8 Done Sent Done false
23517 6842 Acked Write 3 Done Sent Done false
23518 6843 Acked Read 1 Done Sent Done false
23519 6844 Acked Flush 0 Done Sent Done false
23520 6845 Acked Write 3 Done Sent Done false
23521 6846 Acked Read 6 Done Sent Done false
23522 6847 Acked Read 1 Done Sent Done false
23523 6848 Acked Write 6 Done Sent Done false
23524 6849 Acked Read 7 Done Sent Done false
23525 6850 Acked Read 9 Done Sent Done false
23526 6851 Acked Flush 0 Done Sent Done false
23527 6852 Acked Flush 0 Done Sent Done false
23528 6853 Acked Write 1 Done Sent Done false
23529 6854 Acked Read 1 Done Sent Done false
23530 6855 Acked Read 10 Done Sent Done false
23531 6856 Acked Read 6 Done Sent Done false
23532 6857 Acked Write 7 Done Sent Done false
23533 6858 Acked Write 7 Done Sent Done false
23534 6859 Acked Write 5 Done Sent Done false
23535 6860 Acked Read 7 Done Sent Done false
23536 6861 Acked Flush 0 Done Sent Done false
23537 6862 Acked Read 7 Done Sent Done false
23538 6863 Acked Read 9 Done Sent Done false
23539 6864 Acked Read 3 Done Sent Done false
23540 6865 Acked Write 1 Done Sent Done false
23541 6866 Acked Write 6 Done Sent Done false
23542 6867 Acked Flush 0 Done Sent Done false
23543 6868 Acked Read 1 Done Sent Done false
23544 6869 Acked Read 10 Done Sent Done false
23545 6870 Acked Read 7 Done Sent Done false
23546 6871 Acked Read 7 Done Sent Done false
23547 6872 Acked Write 10 Done Sent Done false
23548 6873 Acked Read 4 Done Sent Done false
23549 6874 Acked Flush 0 Done Sent Done false
23550 6875 Acked Write 2 Done Sent Done false
23551 6876 Acked Read 6 Done Sent Done false
23552 6877 Acked Write 6 Done Sent Done false
23553 6878 Acked Write 6 Done Sent Done false
23554 6879 Acked Write 1 Done Sent Done false
23555 6880 Acked Read 9 Done Sent Done false
23556 6881 Acked Write 10 Done Sent Done false
23557 6882 Acked Flush 0 Done Sent Done false
23558 6883 Acked Read 3 Done Sent Done false
23559 6884 Acked Read 2 Done Sent Done false
23560 6885 Acked Flush 0 Done Sent Done false
23561 6886 Acked Write 4 Done Sent Done false
23562 6887 Acked Read 5 Done Sent Done false
23563 6888 Acked Read 2 Done Sent Done false
23564 6889 Acked Flush 0 Done Sent Done false
23565 6890 Acked Read 3 Done Sent Done false
23566 6891 Acked Read 1 Done Sent Done false
23567 6892 Acked Flush 0 Done Sent Done false
23568 6893 Acked Flush 0 Done Sent Done false
23569 6894 Acked Read 1 Done Sent Done false
23570 6895 Acked Read 7 Done Sent Done false
23571 6896 Acked Read 3 Done Sent Done false
23572 6897 Acked Read 9 Done Sent Done false
23573 6898 Acked Read 8 Done Sent Done false
23574 6899 Acked Read 4 Done Sent Done false
23575 6900 Acked Write 7 Done Sent Done false
23576 6901 Acked Read 10 Done Sent Done false
23577 6902 Acked Read 8 Done Sent Done false
23578 6903 Acked Read 7 Done Sent Done false
23579 6904 Acked Write 4 Done Sent Done false
23580 6905 Acked Write 10 Done Sent Done false
23581 6906 Acked Read 8 Done Sent Done false
23582 6907 Acked Write 3 Done Sent Done false
23583 6908 Acked Read 9 Done Sent Done false
23584 6909 Acked Write 1 Done Sent Done false
23585 6910 Acked Read 6 Done Sent Done false
23586 6911 Acked Read 4 Done Sent Done false
23587 6912 Acked Read 4 Done Sent Done false
23588 6913 Acked Read 1 Done Sent Done false
23589 6914 Acked Write 10 Done Sent Done false
23590 6915 Acked Read 2 Done Sent Done false
23591 6916 Acked Flush 0 Done Sent Done false
23592 6917 Acked Flush 0 Done Sent Done false
23593 6918 Acked Read 2 Done Sent Done false
23594 6919 Acked Read 2 Done Sent Done false
23595 6920 Acked Read 4 Done Sent Done false
23596 6921 Acked Write 5 Done Sent Done false
23597 6922 Acked Read 9 Done Sent Done false
23598 6923 Acked Read 4 Done Sent Done false
23599 6924 Acked Write 2 Done Sent Done false
23600 6925 Acked Write 3 Done Sent Done false
23601 6926 Acked Read 8 Done Sent Done false
23602 6927 Acked Write 2 Done Sent Done false
23603 6928 Acked Read 8 Done Sent Done false
23604 6929 Acked Write 4 Done Sent Done false
23605 6930 Acked Read 3 Done Sent Done false
23606 6931 Acked Read 7 Done Sent Done false
23607 6932 Acked Flush 0 Done Sent Done false
23608 6933 Acked Read 3 Done Sent Done false
23609 6934 Acked Flush 0 Done Sent Done false
23610 6935 Acked Read 9 Done Sent Done false
23611 6936 Acked Read 7 Done Sent Done false
23612 6937 Acked Write 10 Done Sent Done false
23613 6938 Acked Read 8 Done Sent Done false
23614 6939 Acked Write 7 Done Sent Done false
23615 6940 Acked Write 2 Done Sent Done false
23616 6941 Acked Write 6 Done Sent Done false
23617 6942 Acked Read 10 Done Sent Done false
23618 6943 Acked Write 3 Done Sent Done false
23619 6944 Acked Write 6 Done Sent Done false
23620 6945 Acked Write 1 Done Sent Done false
23621 6946 Acked Read 4 Done Sent Done false
23622 6947 Acked Write 10 Done Sent Done false
23623 6948 Acked Flush 0 Done Sent Done false
23624 6949 Acked Read 2 Done Sent Done false
23625 6950 Acked Read 2 Done Sent Done false
23626 6951 Acked Write 7 Done Sent Done false
23627 6952 Acked Write 1 Done Sent Done false
23628 6953 Acked Write 10 Done Sent Done false
23629 6954 Acked Write 8 Done Sent Done false
23630 6955 Acked Flush 0 Done Sent Done false
23631 6956 Acked Read 3 Done Sent Done false
23632 6957 Acked Read 6 Done Sent Done false
23633 6958 Acked Read 1 Done Sent Done false
23634 6959 Acked Flush 0 Done Sent Done false
23635 6960 Acked Write 9 Done Sent Done false
23636 6961 Acked Write 8 Done Sent Done false
23637 6962 Acked Write 1 Done Sent Done false
23638 6963 Acked Read 1 Done Sent Done false
23639 6964 Acked Read 9 Done Sent Done false
23640 6965 Acked Write 1 Done Sent Done false
23641 6966 Acked Read 10 Done Sent Done false
23642 6967 Acked Write 10 Done Sent Done false
23643 6968 Acked Read 3 Done Sent Done false
23644 6969 Acked Write 3 Done Sent Done false
23645 6970 Acked Write 8 Done Sent Done false
23646 6971 Acked Write 2 Done Sent Done false
23647 6972 Acked Read 6 Done Sent Done false
23648 6973 Acked Write 9 Done Sent Done false
23649 6974 Acked Flush 0 Done Sent Done false
23650 6975 Acked Flush 0 Done Sent Done false
23651 6976 Acked Write 8 Done Sent Done false
23652 6977 Acked Write 6 Done Sent Done false
23653 6978 Acked Read 3 Done Sent Done false
23654 6979 Acked Write 5 Done Sent Done false
23655 6980 Acked Read 10 Done Sent Done false
23656 6981 Acked Read 3 Done Sent Done false
23657 6982 Acked Write 2 Done Sent Done false
23658 6983 Acked Read 10 Done Sent Done false
23659 6984 Acked Read 4 Done Sent Done false
23660 6985 Acked Read 7 Done Sent Done false
23661 6986 Acked Read 4 Done Sent Done false
23662 6987 Acked Write 7 Done Sent Done false
23663 6988 Acked Read 10 Done Sent Done false
23664 6989 Acked Flush 0 Done Sent Done false
23665 6990 Acked Write 9 Done Sent Done false
23666 6991 Acked Read 3 Done Sent Done false
23667 6992 Acked Write 8 Done Sent Done false
23668 6993 Acked Read 2 Done Sent Done false
23669 6994 Acked Read 3 Done Sent Done false
23670 6995 Acked Flush 0 Done Sent Done false
23671 6996 Acked Read 5 Done Sent Done false
23672 6997 Acked Write 10 Done Sent Done false
23673 6998 Acked Write 3 Done Sent Done false
23674 6999 Acked Read 10 Done Sent Done false
23675 7000 Acked Flush 0 Done Sent Done false
23676 7001 Acked Read 8 Done Sent Done false
23677 7002 Acked Read 10 Done Sent Done false
23678 7003 Acked Write 10 Done Sent Done false
23679 7004 Acked Read 7 Done Sent Done false
23680 7005 Acked Read 7 Done Sent Done false
23681 7006 Acked Write 10 Done Sent Done false
23682 7007 Acked Read 6 Done Sent Done false
23683 7008 Acked Read 8 Done Sent Done false
23684 7009 Acked Write 3 Done Sent Done false
23685 7010 Acked Write 7 Done Sent Done false
23686 7011 Acked Write 2 Done Sent Done false
23687 7012 Acked Write 2 Done Sent Done false
23688 7013 Acked Flush 0 Done Sent Done false
23689 7014 Acked Write 3 Done Sent Done false
23690 7015 Acked Read 7 Done Sent Done false
23691 7016 Acked Read 7 Done Sent Done false
23692 7017 Acked Write 7 Done Sent Done false
23693 7018 Acked Read 5 Done Sent Done false
23694 7019 Acked Read 4 Done Sent Done false
23695 7020 Acked Read 10 Done Sent Done false
23696 7021 Acked Write 4 Done Sent Done false
23697 7022 Acked Write 4 Done Sent Done false
23698 7023 Acked Read 7 Done Sent Done false
23699 7024 Acked Write 6 Done Sent Done false
23700 7025 Acked Flush 0 Done Sent Done false
23701 7026 Acked Read 8 Done Sent Done false
23702 7027 Acked Write 6 Done Sent Done false
23703 7028 Acked Read 3 Done Sent Done false
23704 7029 Acked Write 5 Done Sent Done false
23705 7030 Acked Flush 0 Done Sent Done false
23706 7031 Acked Write 8 Done Sent Done false
23707 7032 Acked Read 6 Done Sent Done false
23708 7033 Acked Write 8 Done Sent Done false
23709 7034 Acked Read 5 Done Sent Done false
23710 7035 Acked Write 9 Done Sent Done false
23711 7036 Acked Flush 0 Done Sent Done false
23712 STATES DS:0 DS:1 DS:2 TOTAL
23713 Sent 0 302 0 302
23714 Done 302 0 302 604
23715 Skipped 0 0 0 0
23716 Error 0 0 0 0
23717 Last Flush: 7036 6734 7036
23718 Downstairs last five completed: 6734 6733 6732 6731 6730
23719 Upstairs last five completed: 7036 7035 7034 7033 7032
23720 CLIENT: Up:0 ds:302 act:2
23721 ----------------------------------------------------------------
23722 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23723 States: Active Offline Active
23724 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23725 6735 Acked Write 6 Done Sent Done false
23726 6736 Acked Flush 0 Done Sent Done false
23727 6737 Acked Read 9 Done Sent Done false
23728 6738 Acked Read 10 Done Sent Done false
23729 6739 Acked Write 7 Done Sent Done false
23730 6740 Acked Read 1 Done Sent Done false
23731 6741 Acked Read 2 Done Sent Done false
23732 6742 Acked Read 3 Done Sent Done false
23733 6743 Acked Write 2 Done Sent Done false
23734 6744 Acked Write 7 Done Sent Done false
23735 6745 Acked Read 6 Done Sent Done false
23736 6746 Acked Write 9 Done Sent Done false
23737 6747 Acked Write 7 Done Sent Done false
23738 6748 Acked Write 4 Done Sent Done false
23739 6749 Acked Flush 0 Done Sent Done false
23740 6750 Acked Read 3 Done Sent Done false
23741 6751 Acked Write 1 Done Sent Done false
23742 6752 Acked Flush 0 Done Sent Done false
23743 6753 Acked Write 9 Done Sent Done false
23744 6754 Acked Read 7 Done Sent Done false
23745 6755 Acked Read 7 Done Sent Done false
23746 6756 Acked Write 8 Done Sent Done false
23747 6757 Acked Write 5 Done Sent Done false
23748 6758 Acked Write 5 Done Sent Done false
23749 6759 Acked Read 10 Done Sent Done false
23750 6760 Acked Flush 0 Done Sent Done false
23751 6761 Acked Read 1 Done Sent Done false
23752 6762 Acked Write 7 Done Sent Done false
23753 6763 Acked Read 6 Done Sent Done false
23754 6764 Acked Read 10 Done Sent Done false
23755 6765 Acked Read 7 Done Sent Done false
23756 6766 Acked Write 9 Done Sent Done false
23757 6767 Acked Write 5 Done Sent Done false
23758 6768 Acked Write 1 Done Sent Done false
23759 6769 Acked Read 2 Done Sent Done false
23760 6770 Acked Read 5 Done Sent Done false
23761 6771 Acked Flush 0 Done Sent Done false
23762 6772 Acked Read 9 Done Sent Done false
23763 6773 Acked Read 1 Done Sent Done false
23764 6774 Acked Read 3 Done Sent Done false
23765 6775 Acked Flush 0 Done Sent Done false
23766 6776 Acked Read 8 Done Sent Done false
23767 6777 Acked Read 1 Done Sent Done false
23768 6778 Acked Flush 0 Done Sent Done false
23769 6779 Acked Flush 0 Done Sent Done false
23770 6780 Acked Write 4 Done Sent Done false
23771 6781 Acked Read 2 Done Sent Done false
23772 6782 Acked Read 6 Done Sent Done false
23773 6783 Acked Read 6 Done Sent Done false
23774 6784 Acked Write 8 Done Sent Done false
23775 6785 Acked Write 3 Done Sent Done false
23776 6786 Acked Write 2 Done Sent Done false
23777 6787 Acked Read 5 Done Sent Done false
23778 6788 Acked Read 9 Done Sent Done false
23779 6789 Acked Write 9 Done Sent Done false
23780 6790 Acked Read 10 Done Sent Done false
23781 6791 Acked Write 3 Done Sent Done false
23782 6792 Acked Read 10 Done Sent Done false
23783 6793 Acked Read 10 Done Sent Done false
23784 6794 Acked Write 8 Done Sent Done false
23785 6795 Acked Read 10 Done Sent Done false
23786 6796 Acked Read 4 Done Sent Done false
23787 6797 Acked Write 9 Done Sent Done false
23788 6798 Acked Write 4 Done Sent Done false
23789 6799 Acked Read 1 Done Sent Done false
23790 6800 Acked Write 5 Done Sent Done false
23791 6801 Acked Write 9 Done Sent Done false
23792 6802 Acked Read 1 Done Sent Done false
23793 6803 Acked Read 9 Done Sent Done false
23794 6804 Acked Write 8 Done Sent Done false
23795 6805 Acked Write 8 Done Sent Done false
23796 6806 Acked Flush 0 Done Sent Done false
23797 6807 Acked Read 8 Done Sent Done false
23798 6808 Acked Read 5 Done Sent Done false
23799 6809 Acked Read 5 Done Sent Done false
23800 6810 Acked Write 3 Done Sent Done false
23801 6811 Acked Write 2 Done Sent Done false
23802 6812 Acked Read 1 Done Sent Done false
23803 6813 Acked Write 9 Done Sent Done false
23804 6814 Acked Write 9 Done Sent Done false
23805 6815 Acked Read 8 Done Sent Done false
23806 6816 Acked Write 10 Done Sent Done false
23807 6817 Acked Flush 0 Done Sent Done false
23808 6818 Acked Flush 0 Done Sent Done false
23809 6819 Acked Read 5 Done Sent Done false
23810 6820 Acked Write 9 Done Sent Done false
23811 6821 Acked Write 1 Done Sent Done false
23812 6822 Acked Read 8 Done Sent Done false
23813 6823 Acked Write 2 Done Sent Done false
23814 6824 Acked Write 10 Done Sent Done false
23815 6825 Acked Read 5 Done Sent Done false
23816 6826 Acked Write 8 Done Sent Done false
23817 6827 Acked Flush 0 Done Sent Done false
23818 6828 Acked Flush 0 Done Sent Done false
23819 6829 Acked Read 10 Done Sent Done false
23820 6830 Acked Write 8 Done Sent Done false
23821 6831 Acked Read 8 Done Sent Done false
23822 6832 Acked Read 8 Done Sent Done false
23823 6833 Acked Read 4 Done Sent Done false
23824 6834 Acked Read 10 Done Sent Done false
23825 6835 Acked Write 2 Done Sent Done false
23826 6836 Acked Write 8 Done Sent Done false
23827 6837 Acked Read 4 Done Sent Done false
23828 6838 Acked Read 2 Done Sent Done false
23829 6839 Acked Read 9 Done Sent Done false
23830 6840 Acked Write 6 Done Sent Done false
23831 6841 Acked Read 8 Done Sent Done false
23832 6842 Acked Write 3 Done Sent Done false
23833 6843 Acked Read 1 Done Sent Done false
23834 6844 Acked Flush 0 Done Sent Done false
23835 6845 Acked Write 3 Done Sent Done false
23836 6846 Acked Read 6 Done Sent Done false
23837 6847 Acked Read 1 Done Sent Done false
23838 6848 Acked Write 6 Done Sent Done false
23839 6849 Acked Read 7 Done Sent Done false
23840 6850 Acked Read 9 Done Sent Done false
23841 6851 Acked Flush 0 Done Sent Done false
23842 6852 Acked Flush 0 Done Sent Done false
23843 6853 Acked Write 1 Done Sent Done false
23844 6854 Acked Read 1 Done Sent Done false
23845 6855 Acked Read 10 Done Sent Done false
23846 6856 Acked Read 6 Done Sent Done false
23847 6857 Acked Write 7 Done Sent Done false
23848 6858 Acked Write 7 Done Sent Done false
23849 6859 Acked Write 5 Done Sent Done false
23850 6860 Acked Read 7 Done Sent Done false
23851 6861 Acked Flush 0 Done Sent Done false
23852 6862 Acked Read 7 Done Sent Done false
23853 6863 Acked Read 9 Done Sent Done false
23854 6864 Acked Read 3 Done Sent Done false
23855 6865 Acked Write 1 Done Sent Done false
23856 6866 Acked Write 6 Done Sent Done false
23857 6867 Acked Flush 0 Done Sent Done false
23858 6868 Acked Read 1 Done Sent Done false
23859 6869 Acked Read 10 Done Sent Done false
23860 6870 Acked Read 7 Done Sent Done false
23861 6871 Acked Read 7 Done Sent Done false
23862 6872 Acked Write 10 Done Sent Done false
23863 6873 Acked Read 4 Done Sent Done false
23864 6874 Acked Flush 0 Done Sent Done false
23865 6875 Acked Write 2 Done Sent Done false
23866 6876 Acked Read 6 Done Sent Done false
23867 6877 Acked Write 6 Done Sent Done false
23868 6878 Acked Write 6 Done Sent Done false
23869 6879 Acked Write 1 Done Sent Done false
23870 6880 Acked Read 9 Done Sent Done false
23871 6881 Acked Write 10 Done Sent Done false
23872 6882 Acked Flush 0 Done Sent Done false
23873 6883 Acked Read 3 Done Sent Done false
23874 6884 Acked Read 2 Done Sent Done false
23875 6885 Acked Flush 0 Done Sent Done false
23876 6886 Acked Write 4 Done Sent Done false
23877 6887 Acked Read 5 Done Sent Done false
23878 6888 Acked Read 2 Done Sent Done false
23879 6889 Acked Flush 0 Done Sent Done false
23880 6890 Acked Read 3 Done Sent Done false
23881 6891 Acked Read 1 Done Sent Done false
23882 6892 Acked Flush 0 Done Sent Done false
23883 6893 Acked Flush 0 Done Sent Done false
23884 6894 Acked Read 1 Done Sent Done false
23885 6895 Acked Read 7 Done Sent Done false
23886 6896 Acked Read 3 Done Sent Done false
23887 6897 Acked Read 9 Done Sent Done false
23888 6898 Acked Read 8 Done Sent Done false
23889 6899 Acked Read 4 Done Sent Done false
23890 6900 Acked Write 7 Done Sent Done false
23891 6901 Acked Read 10 Done Sent Done false
23892 6902 Acked Read 8 Done Sent Done false
23893 6903 Acked Read 7 Done Sent Done false
23894 6904 Acked Write 4 Done Sent Done false
23895 6905 Acked Write 10 Done Sent Done false
23896 6906 Acked Read 8 Done Sent Done false
23897 6907 Acked Write 3 Done Sent Done false
23898 6908 Acked Read 9 Done Sent Done false
23899 6909 Acked Write 1 Done Sent Done false
23900 6910 Acked Read 6 Done Sent Done false
23901 6911 Acked Read 4 Done Sent Done false
23902 6912 Acked Read 4 Done Sent Done false
23903 6913 Acked Read 1 Done Sent Done false
23904 6914 Acked Write 10 Done Sent Done false
23905 6915 Acked Read 2 Done Sent Done false
23906 6916 Acked Flush 0 Done Sent Done false
23907 6917 Acked Flush 0 Done Sent Done false
23908 6918 Acked Read 2 Done Sent Done false
23909 6919 Acked Read 2 Done Sent Done false
23910 6920 Acked Read 4 Done Sent Done false
23911 6921 Acked Write 5 Done Sent Done false
23912 6922 Acked Read 9 Done Sent Done false
23913 6923 Acked Read 4 Done Sent Done false
23914 6924 Acked Write 2 Done Sent Done false
23915 6925 Acked Write 3 Done Sent Done false
23916 6926 Acked Read 8 Done Sent Done false
23917 6927 Acked Write 2 Done Sent Done false
23918 6928 Acked Read 8 Done Sent Done false
23919 6929 Acked Write 4 Done Sent Done false
23920 6930 Acked Read 3 Done Sent Done false
23921 6931 Acked Read 7 Done Sent Done false
23922 6932 Acked Flush 0 Done Sent Done false
23923 6933 Acked Read 3 Done Sent Done false
23924 6934 Acked Flush 0 Done Sent Done false
23925 6935 Acked Read 9 Done Sent Done false
23926 6936 Acked Read 7 Done Sent Done false
23927 6937 Acked Write 10 Done Sent Done false
23928 6938 Acked Read 8 Done Sent Done false
23929 6939 Acked Write 7 Done Sent Done false
23930 6940 Acked Write 2 Done Sent Done false
23931 6941 Acked Write 6 Done Sent Done false
23932 6942 Acked Read 10 Done Sent Done false
23933 6943 Acked Write 3 Done Sent Done false
23934 6944 Acked Write 6 Done Sent Done false
23935 6945 Acked Write 1 Done Sent Done false
23936 6946 Acked Read 4 Done Sent Done false
23937 6947 Acked Write 10 Done Sent Done false
23938 6948 Acked Flush 0 Done Sent Done false
23939 6949 Acked Read 2 Done Sent Done false
23940 6950 Acked Read 2 Done Sent Done false
23941 6951 Acked Write 7 Done Sent Done false
23942 6952 Acked Write 1 Done Sent Done false
23943 6953 Acked Write 10 Done Sent Done false
23944 6954 Acked Write 8 Done Sent Done false
23945 6955 Acked Flush 0 Done Sent Done false
23946 6956 Acked Read 3 Done Sent Done false
23947 6957 Acked Read 6 Done Sent Done false
23948 6958 Acked Read 1 Done Sent Done false
23949 6959 Acked Flush 0 Done Sent Done false
23950 6960 Acked Write 9 Done Sent Done false
23951 6961 Acked Write 8 Done Sent Done false
23952 6962 Acked Write 1 Done Sent Done false
23953 6963 Acked Read 1 Done Sent Done false
23954 6964 Acked Read 9 Done Sent Done false
23955 6965 Acked Write 1 Done Sent Done false
23956 6966 Acked Read 10 Done Sent Done false
23957 6967 Acked Write 10 Done Sent Done false
23958 6968 Acked Read 3 Done Sent Done false
23959 6969 Acked Write 3 Done Sent Done false
23960 6970 Acked Write 8 Done Sent Done false
23961 6971 Acked Write 2 Done Sent Done false
23962 6972 Acked Read 6 Done Sent Done false
23963 6973 Acked Write 9 Done Sent Done false
23964 6974 Acked Flush 0 Done Sent Done false
23965 6975 Acked Flush 0 Done Sent Done false
23966 6976 Acked Write 8 Done Sent Done false
23967 6977 Acked Write 6 Done Sent Done false
23968 6978 Acked Read 3 Done Sent Done false
23969 6979 Acked Write 5 Done Sent Done false
23970 6980 Acked Read 10 Done Sent Done false
23971 6981 Acked Read 3 Done Sent Done false
23972 6982 Acked Write 2 Done Sent Done false
23973 6983 Acked Read 10 Done Sent Done false
23974 6984 Acked Read 4 Done Sent Done false
23975 6985 Acked Read 7 Done Sent Done false
23976 6986 Acked Read 4 Done Sent Done false
23977 6987 Acked Write 7 Done Sent Done false
23978 6988 Acked Read 10 Done Sent Done false
23979 6989 Acked Flush 0 Done Sent Done false
23980 6990 Acked Write 9 Done Sent Done false
23981 6991 Acked Read 3 Done Sent Done false
23982 6992 Acked Write 8 Done Sent Done false
23983 6993 Acked Read 2 Done Sent Done false
23984 6994 Acked Read 3 Done Sent Done false
23985 6995 Acked Flush 0 Done Sent Done false
23986 6996 Acked Read 5 Done Sent Done false
23987 6997 Acked Write 10 Done Sent Done false
23988 6998 Acked Write 3 Done Sent Done false
23989 6999 Acked Read 10 Done Sent Done false
23990 7000 Acked Flush 0 Done Sent Done false
23991 7001 Acked Read 8 Done Sent Done false
23992 7002 Acked Read 10 Done Sent Done false
23993 7003 Acked Write 10 Done Sent Done false
23994 7004 Acked Read 7 Done Sent Done false
23995 7005 Acked Read 7 Done Sent Done false
23996 7006 Acked Write 10 Done Sent Done false
23997 7007 Acked Read 6 Done Sent Done false
23998 7008 Acked Read 8 Done Sent Done false
23999 7009 Acked Write 3 Done Sent Done false
24000 7010 Acked Write 7 Done Sent Done false
24001 7011 Acked Write 2 Done Sent Done false
24002 7012 Acked Write 2 Done Sent Done false
24003 7013 Acked Flush 0 Done Sent Done false
24004 7014 Acked Write 3 Done Sent Done false
24005 7015 Acked Read 7 Done Sent Done false
24006 7016 Acked Read 7 Done Sent Done false
24007 7017 Acked Write 7 Done Sent Done false
24008 7018 Acked Read 5 Done Sent Done false
24009 7019 Acked Read 4 Done Sent Done false
24010 7020 Acked Read 10 Done Sent Done false
24011 7021 Acked Write 4 Done Sent Done false
24012 7022 Acked Write 4 Done Sent Done false
24013 7023 Acked Read 7 Done Sent Done false
24014 7024 Acked Write 6 Done Sent Done false
24015 7025 Acked Flush 0 Done Sent Done false
24016 7026 Acked Read 8 Done Sent Done false
24017 7027 Acked Write 6 Done Sent Done false
24018 7028 Acked Read 3 Done Sent Done false
24019 7029 Acked Write 5 Done Sent Done false
24020 7030 Acked Flush 0 Done Sent Done false
24021 7031 Acked Write 8 Done Sent Done false
24022 7032 Acked Read 6 Done Sent Done false
24023 7033 Acked Write 8 Done Sent Done false
24024 7034 Acked Read 5 Done Sent Done false
24025 7035 Acked Write 9 Done Sent Done false
24026 7036 Acked Flush 0 Done Sent Done false
24027 STATES DS:0 DS:1 DS:2 TOTAL
24028 Sent 0 302 0 302
24029 Done 302 0 302 604
24030 Skipped 0 0 0 0
24031 Error 0 0 0 0
24032 Last Flush: 7036 6734 7036
24033 Downstairs last five completed: 6734 6733 6732 6731 6730
24034 Upstairs last five completed: 7036 7035 7034 7033 7032
24035 CLIENT: Up:0 ds:302 act:2
24036 ----------------------------------------------------------------
24037 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24038 States: Active Offline Active
24039 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24040 6735 Acked Write 6 Done Sent Done false
24041 6736 Acked Flush 0 Done Sent Done false
24042 6737 Acked Read 9 Done Sent Done false
24043 6738 Acked Read 10 Done Sent Done false
24044 6739 Acked Write 7 Done Sent Done false
24045 6740 Acked Read 1 Done Sent Done false
24046 6741 Acked Read 2 Done Sent Done false
24047 6742 Acked Read 3 Done Sent Done false
24048 6743 Acked Write 2 Done Sent Done false
24049 6744 Acked Write 7 Done Sent Done false
24050 6745 Acked Read 6 Done Sent Done false
24051 6746 Acked Write 9 Done Sent Done false
24052 6747 Acked Write 7 Done Sent Done false
24053 6748 Acked Write 4 Done Sent Done false
24054 6749 Acked Flush 0 Done Sent Done false
24055 6750 Acked Read 3 Done Sent Done false
24056 6751 Acked Write 1 Done Sent Done false
24057 6752 Acked Flush 0 Done Sent Done false
24058 6753 Acked Write 9 Done Sent Done false
24059 6754 Acked Read 7 Done Sent Done false
24060 6755 Acked Read 7 Done Sent Done false
24061 6756 Acked Write 8 Done Sent Done false
24062 6757 Acked Write 5 Done Sent Done false
24063 6758 Acked Write 5 Done Sent Done false
24064 6759 Acked Read 10 Done Sent Done false
24065 6760 Acked Flush 0 Done Sent Done false
24066 6761 Acked Read 1 Done Sent Done false
24067 6762 Acked Write 7 Done Sent Done false
24068 6763 Acked Read 6 Done Sent Done false
24069 6764 Acked Read 10 Done Sent Done false
24070 6765 Acked Read 7 Done Sent Done false
24071 6766 Acked Write 9 Done Sent Done false
24072 6767 Acked Write 5 Done Sent Done false
24073 6768 Acked Write 1 Done Sent Done false
24074 6769 Acked Read 2 Done Sent Done false
24075 6770 Acked Read 5 Done Sent Done false
24076 6771 Acked Flush 0 Done Sent Done false
24077 6772 Acked Read 9 Done Sent Done false
24078 6773 Acked Read 1 Done Sent Done false
24079 6774 Acked Read 3 Done Sent Done false
24080 6775 Acked Flush 0 Done Sent Done false
24081 6776 Acked Read 8 Done Sent Done false
24082 6777 Acked Read 1 Done Sent Done false
24083 6778 Acked Flush 0 Done Sent Done false
24084 6779 Acked Flush 0 Done Sent Done false
24085 6780 Acked Write 4 Done Sent Done false
24086 6781 Acked Read 2 Done Sent Done false
24087 6782 Acked Read 6 Done Sent Done false
24088 6783 Acked Read 6 Done Sent Done false
24089 6784 Acked Write 8 Done Sent Done false
24090 6785 Acked Write 3 Done Sent Done false
24091 6786 Acked Write 2 Done Sent Done false
24092 6787 Acked Read 5 Done Sent Done false
24093 6788 Acked Read 9 Done Sent Done false
24094 6789 Acked Write 9 Done Sent Done false
24095 6790 Acked Read 10 Done Sent Done false
24096 6791 Acked Write 3 Done Sent Done false
24097 6792 Acked Read 10 Done Sent Done false
24098 6793 Acked Read 10 Done Sent Done false
24099 6794 Acked Write 8 Done Sent Done false
24100 6795 Acked Read 10 Done Sent Done false
24101 6796 Acked Read 4 Done Sent Done false
24102 6797 Acked Write 9 Done Sent Done false
24103 6798 Acked Write 4 Done Sent Done false
24104 6799 Acked Read 1 Done Sent Done false
24105 6800 Acked Write 5 Done Sent Done false
24106 6801 Acked Write 9 Done Sent Done false
24107 6802 Acked Read 1 Done Sent Done false
24108 6803 Acked Read 9 Done Sent Done false
24109 6804 Acked Write 8 Done Sent Done false
24110 6805 Acked Write 8 Done Sent Done false
24111 6806 Acked Flush 0 Done Sent Done false
24112 6807 Acked Read 8 Done Sent Done false
24113 6808 Acked Read 5 Done Sent Done false
24114 6809 Acked Read 5 Done Sent Done false
24115 6810 Acked Write 3 Done Sent Done false
24116 6811 Acked Write 2 Done Sent Done false
24117 6812 Acked Read 1 Done Sent Done false
24118 6813 Acked Write 9 Done Sent Done false
24119 6814 Acked Write 9 Done Sent Done false
24120 6815 Acked Read 8 Done Sent Done false
24121 6816 Acked Write 10 Done Sent Done false
24122 6817 Acked Flush 0 Done Sent Done false
24123 6818 Acked Flush 0 Done Sent Done false
24124 6819 Acked Read 5 Done Sent Done false
24125 6820 Acked Write 9 Done Sent Done false
24126 6821 Acked Write 1 Done Sent Done false
24127 6822 Acked Read 8 Done Sent Done false
24128 6823 Acked Write 2 Done Sent Done false
24129 6824 Acked Write 10 Done Sent Done false
24130 6825 Acked Read 5 Done Sent Done false
24131 6826 Acked Write 8 Done Sent Done false
24132 6827 Acked Flush 0 Done Sent Done false
24133 6828 Acked Flush 0 Done Sent Done false
24134 6829 Acked Read 10 Done Sent Done false
24135 6830 Acked Write 8 Done Sent Done false
24136 6831 Acked Read 8 Done Sent Done false
24137 6832 Acked Read 8 Done Sent Done false
24138 6833 Acked Read 4 Done Sent Done false
24139 6834 Acked Read 10 Done Sent Done false
24140 6835 Acked Write 2 Done Sent Done false
24141 6836 Acked Write 8 Done Sent Done false
24142 6837 Acked Read 4 Done Sent Done false
24143 6838 Acked Read 2 Done Sent Done false
24144 6839 Acked Read 9 Done Sent Done false
24145 6840 Acked Write 6 Done Sent Done false
24146 6841 Acked Read 8 Done Sent Done false
24147 6842 Acked Write 3 Done Sent Done false
24148 6843 Acked Read 1 Done Sent Done false
24149 6844 Acked Flush 0 Done Sent Done false
24150 6845 Acked Write 3 Done Sent Done false
24151 6846 Acked Read 6 Done Sent Done false
24152 6847 Acked Read 1 Done Sent Done false
24153 6848 Acked Write 6 Done Sent Done false
24154 6849 Acked Read 7 Done Sent Done false
24155 6850 Acked Read 9 Done Sent Done false
24156 6851 Acked Flush 0 Done Sent Done false
24157 6852 Acked Flush 0 Done Sent Done false
24158 6853 Acked Write 1 Done Sent Done false
24159 6854 Acked Read 1 Done Sent Done false
24160 6855 Acked Read 10 Done Sent Done false
24161 6856 Acked Read 6 Done Sent Done false
24162 6857 Acked Write 7 Done Sent Done false
24163 6858 Acked Write 7 Done Sent Done false
24164 6859 Acked Write 5 Done Sent Done false
24165 6860 Acked Read 7 Done Sent Done false
24166 6861 Acked Flush 0 Done Sent Done false
24167 6862 Acked Read 7 Done Sent Done false
24168 6863 Acked Read 9 Done Sent Done false
24169 6864 Acked Read 3 Done Sent Done false
24170 6865 Acked Write 1 Done Sent Done false
24171 6866 Acked Write 6 Done Sent Done false
24172 6867 Acked Flush 0 Done Sent Done false
24173 6868 Acked Read 1 Done Sent Done false
24174 6869 Acked Read 10 Done Sent Done false
24175 6870 Acked Read 7 Done Sent Done false
24176 6871 Acked Read 7 Done Sent Done false
24177 6872 Acked Write 10 Done Sent Done false
24178 6873 Acked Read 4 Done Sent Done false
24179 6874 Acked Flush 0 Done Sent Done false
24180 6875 Acked Write 2 Done Sent Done false
24181 6876 Acked Read 6 Done Sent Done false
24182 6877 Acked Write 6 Done Sent Done false
24183 6878 Acked Write 6 Done Sent Done false
24184 6879 Acked Write 1 Done Sent Done false
24185 6880 Acked Read 9 Done Sent Done false
24186 6881 Acked Write 10 Done Sent Done false
24187 6882 Acked Flush 0 Done Sent Done false
24188 6883 Acked Read 3 Done Sent Done false
24189 6884 Acked Read 2 Done Sent Done false
24190 6885 Acked Flush 0 Done Sent Done false
24191 6886 Acked Write 4 Done Sent Done false
24192 6887 Acked Read 5 Done Sent Done false
24193 6888 Acked Read 2 Done Sent Done false
24194 6889 Acked Flush 0 Done Sent Done false
24195 6890 Acked Read 3 Done Sent Done false
24196 6891 Acked Read 1 Done Sent Done false
24197 6892 Acked Flush 0 Done Sent Done false
24198 6893 Acked Flush 0 Done Sent Done false
24199 6894 Acked Read 1 Done Sent Done false
24200 6895 Acked Read 7 Done Sent Done false
24201 6896 Acked Read 3 Done Sent Done false
24202 6897 Acked Read 9 Done Sent Done false
24203 6898 Acked Read 8 Done Sent Done false
24204 6899 Acked Read 4 Done Sent Done false
24205 6900 Acked Write 7 Done Sent Done false
24206 6901 Acked Read 10 Done Sent Done false
24207 6902 Acked Read 8 Done Sent Done false
24208 6903 Acked Read 7 Done Sent Done false
24209 6904 Acked Write 4 Done Sent Done false
24210 6905 Acked Write 10 Done Sent Done false
24211 6906 Acked Read 8 Done Sent Done false
24212 6907 Acked Write 3 Done Sent Done false
24213 6908 Acked Read 9 Done Sent Done false
24214 6909 Acked Write 1 Done Sent Done false
24215 6910 Acked Read 6 Done Sent Done false
24216 6911 Acked Read 4 Done Sent Done false
24217 6912 Acked Read 4 Done Sent Done false
24218 6913 Acked Read 1 Done Sent Done false
24219 6914 Acked Write 10 Done Sent Done false
24220 6915 Acked Read 2 Done Sent Done false
24221 6916 Acked Flush 0 Done Sent Done false
24222 6917 Acked Flush 0 Done Sent Done false
24223 6918 Acked Read 2 Done Sent Done false
24224 6919 Acked Read 2 Done Sent Done false
24225 6920 Acked Read 4 Done Sent Done false
24226 6921 Acked Write 5 Done Sent Done false
24227 6922 Acked Read 9 Done Sent Done false
24228 6923 Acked Read 4 Done Sent Done false
24229 6924 Acked Write 2 Done Sent Done false
24230 6925 Acked Write 3 Done Sent Done false
24231 6926 Acked Read 8 Done Sent Done false
24232 6927 Acked Write 2 Done Sent Done false
24233 6928 Acked Read 8 Done Sent Done false
24234 6929 Acked Write 4 Done Sent Done false
24235 6930 Acked Read 3 Done Sent Done false
24236 6931 Acked Read 7 Done Sent Done false
24237 6932 Acked Flush 0 Done Sent Done false
24238 6933 Acked Read 3 Done Sent Done false
24239 6934 Acked Flush 0 Done Sent Done false
24240 6935 Acked Read 9 Done Sent Done false
24241 6936 Acked Read 7 Done Sent Done false
24242 6937 Acked Write 10 Done Sent Done false
24243 6938 Acked Read 8 Done Sent Done false
24244 6939 Acked Write 7 Done Sent Done false
24245 6940 Acked Write 2 Done Sent Done false
24246 6941 Acked Write 6 Done Sent Done false
24247 6942 Acked Read 10 Done Sent Done false
24248 6943 Acked Write 3 Done Sent Done false
24249 6944 Acked Write 6 Done Sent Done false
24250 6945 Acked Write 1 Done Sent Done false
24251 6946 Acked Read 4 Done Sent Done false
24252 6947 Acked Write 10 Done Sent Done false
24253 6948 Acked Flush 0 Done Sent Done false
24254 6949 Acked Read 2 Done Sent Done false
24255 6950 Acked Read 2 Done Sent Done false
24256 6951 Acked Write 7 Done Sent Done false
24257 6952 Acked Write 1 Done Sent Done false
24258 6953 Acked Write 10 Done Sent Done false
24259 6954 Acked Write 8 Done Sent Done false
24260 6955 Acked Flush 0 Done Sent Done false
24261 6956 Acked Read 3 Done Sent Done false
24262 6957 Acked Read 6 Done Sent Done false
24263 6958 Acked Read 1 Done Sent Done false
24264 6959 Acked Flush 0 Done Sent Done false
24265 6960 Acked Write 9 Done Sent Done false
24266 6961 Acked Write 8 Done Sent Done false
24267 6962 Acked Write 1 Done Sent Done false
24268 6963 Acked Read 1 Done Sent Done false
24269 6964 Acked Read 9 Done Sent Done false
24270 6965 Acked Write 1 Done Sent Done false
24271 6966 Acked Read 10 Done Sent Done false
24272 6967 Acked Write 10 Done Sent Done false
24273 6968 Acked Read 3 Done Sent Done false
24274 6969 Acked Write 3 Done Sent Done false
24275 6970 Acked Write 8 Done Sent Done false
24276 6971 Acked Write 2 Done Sent Done false
24277 6972 Acked Read 6 Done Sent Done false
24278 6973 Acked Write 9 Done Sent Done false
24279 6974 Acked Flush 0 Done Sent Done false
24280 6975 Acked Flush 0 Done Sent Done false
24281 6976 Acked Write 8 Done Sent Done false
24282 6977 Acked Write 6 Done Sent Done false
24283 6978 Acked Read 3 Done Sent Done false
24284 6979 Acked Write 5 Done Sent Done false
24285 6980 Acked Read 10 Done Sent Done false
24286 6981 Acked Read 3 Done Sent Done false
24287 6982 Acked Write 2 Done Sent Done false
24288 6983 Acked Read 10 Done Sent Done false
24289 6984 Acked Read 4 Done Sent Done false
24290 6985 Acked Read 7 Done Sent Done false
24291 6986 Acked Read 4 Done Sent Done false
24292 6987 Acked Write 7 Done Sent Done false
24293 6988 Acked Read 10 Done Sent Done false
24294 6989 Acked Flush 0 Done Sent Done false
24295 6990 Acked Write 9 Done Sent Done false
24296 6991 Acked Read 3 Done Sent Done false
24297 6992 Acked Write 8 Done Sent Done false
24298 6993 Acked Read 2 Done Sent Done false
24299 6994 Acked Read 3 Done Sent Done false
24300 6995 Acked Flush 0 Done Sent Done false
24301 6996 Acked Read 5 Done Sent Done false
24302 6997 Acked Write 10 Done Sent Done false
24303 6998 Acked Write 3 Done Sent Done false
24304 6999 Acked Read 10 Done Sent Done false
24305 7000 Acked Flush 0 Done Sent Done false
24306 7001 Acked Read 8 Done Sent Done false
24307 7002 Acked Read 10 Done Sent Done false
24308 7003 Acked Write 10 Done Sent Done false
24309 7004 Acked Read 7 Done Sent Done false
24310 7005 Acked Read 7 Done Sent Done false
24311 7006 Acked Write 10 Done Sent Done false
24312 7007 Acked Read 6 Done Sent Done false
24313 7008 Acked Read 8 Done Sent Done false
24314 7009 Acked Write 3 Done Sent Done false
24315 7010 Acked Write 7 Done Sent Done false
24316 7011 Acked Write 2 Done Sent Done false
24317 7012 Acked Write 2 Done Sent Done false
24318 7013 Acked Flush 0 Done Sent Done false
24319 7014 Acked Write 3 Done Sent Done false
24320 7015 Acked Read 7 Done Sent Done false
24321 7016 Acked Read 7 Done Sent Done false
24322 7017 Acked Write 7 Done Sent Done false
24323 7018 Acked Read 5 Done Sent Done false
24324 7019 Acked Read 4 Done Sent Done false
24325 7020 Acked Read 10 Done Sent Done false
24326 7021 Acked Write 4 Done Sent Done false
24327 7022 Acked Write 4 Done Sent Done false
24328 7023 Acked Read 7 Done Sent Done false
24329 7024 Acked Write 6 Done Sent Done false
24330 7025 Acked Flush 0 Done Sent Done false
24331 7026 Acked Read 8 Done Sent Done false
24332 7027 Acked Write 6 Done Sent Done false
24333 7028 Acked Read 3 Done Sent Done false
24334 7029 Acked Write 5 Done Sent Done false
24335 7030 Acked Flush 0 Done Sent Done false
24336 7031 Acked Write 8 Done Sent Done false
24337 7032 Acked Read 6 Done Sent Done false
24338 7033 Acked Write 8 Done Sent Done false
24339 7034 Acked Read 5 Done Sent Done false
24340 7035 Acked Write 9 Done Sent Done false
24341 7036 Acked Flush 0 Done Sent Done false
24342 STATES DS:0 DS:1 DS:2 TOTAL
24343 Sent 0 302 0 302
24344 Done 302 0 302 604
24345 Skipped 0 0 0 0
24346 Error 0 0 0 0
24347 Last Flush: 7036 6734 7036
24348 Downstairs last five completed: 6734 6733 6732 6731 6730
24349 Upstairs last five completed: 7036 7035 7034 7033 7032
24350 CLIENT: Up:0 ds:302 act:2
243512025-01-17T21:16:11.157ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
243522025-01-17T21:16:11.157ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62750) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
243532025-01-17T21:16:11.158ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
243542025-01-17T21:16:11.159ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
243552025-01-17T21:16:11.159ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
243562025-01-17T21:16:11.159ZINFOcrucible: send last flush ID to this DS: 6734 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
243572025-01-17T21:16:11.159ZINFOcrucible: Replied this last flush ID: 6734 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
243582025-01-17T21:16:11.159ZINFOcrucible: [1] client re-new 302 jobs since flush 6734 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
243592025-01-17T21:16:11.159ZINFOcrucible: [1] Replayed 302 jobs since flush: 6734 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
24360 ----------------------------------------------------------------
24361 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
24362 Downstairs last five completed: 7036 7035 7034 7033 7032
24363 Upstairs last five completed: 7036 7035 7034 7033 7032
24364 CLIENT: Up:0 ds:0 act:3
24365 Replay: All jobs finished
24366 Main task has work to do, go find it
24367 got dsc Stop(0)
24368 stop 0
24369 Jan 17 21:16:13.415[8810] Got stop action so:false kr:false
24370 INFO request completed, latency_us: 355, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 203a107e-2049-437a-843e-e00856551b50, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
24371 Jan 17 21:16:13.416 INFO request completed, latency_us: 115, response_code: 200, uri: /state/cid/0, method: GET, req_id: 6bb8bb69-3329-4439-9965-b30d929bb455, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
243722025-01-17T21:16:13.416ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
243732025-01-17T21:16:13.416ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
243742025-01-17T21:16:13.416ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
243752025-01-17T21:16:13.416ZINFOcrucible: client task is exiting = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
24376 [8810] Exited with: ExitStatus(unix_wait_status(9))
24377 [8810][0] reports Exit
24378 Jan 17 21:16:17.418 INFO request completed, latency_us: 201, response_code: 200, uri: /state/cid/0, method: GET, req_id: 7f412a31-6ad3-4f46-a8e7-5834ae1104d8, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
24379 001/300 Write block 523 len 12288 data: 3 3 5
24380 002/300 Write block 4365 len 36864 data: 7 6 5 5 6 5 7 5 7
24381 003/300 Read block 3070 len 32768
24382 004/300 Write block 3603 len 40960 data: 5 5 6 5 5 7 9 9 10 10
24383 005/300 Write block 346 len 40960 data: 3 4 5 3 3 2 2 2 2 2
24384 006/300 Write block 65 len 32768 data: 4 4 5 6 6 6 8 9
24385 007/300 Write block 1499 len 20480 data: 3 5 5 5 4
24386 008/300 Read block 2224 len 28672
24387 009/300 Read block 3935 len 12288
24388 010/300 Write block 837 len 4096 data: 4
24389 011/300 Write block 3641 len 4096 data: 2
24390 012/300 Flush
24391 013/300 Read block 818 len 4096
24392 014/300 Read block 3599 len 8192
24393 015/300 Write block 2343 len 40960 data: 6 6 7 9 7 7 9 9 10 8
24394 016/300 Read block 1273 len 16384
24395 017/300 Write block 118 len 36864 data: 5 4 5 5 6 5 4 3 2
24396 018/300 Read block 2431 len 28672
24397 019/300 Write block 4336 len 8192 data: 5 6
24398 020/300 Read block 2903 len 8192
24399 021/300 Read block 945 len 12288
24400 022/300 Write block 4139 len 36864 data: 7 7 5 4 4 4 4 5 3
24401 023/300 Read block 3803 len 4096
24402 024/300 Read block 2203 len 24576
24403 025/300 Read block 2277 len 20480
24404 026/300 Write block 887 len 36864 data: 5 7 5 4 4 3 3 3 4
24405 027/300 Read block 2924 len 28672
24406 028/300 Read block 626 len 32768
24407 029/300 Write block 3131 len 4096 data: 3
24408 030/300 Read block 2249 len 36864
24409 031/300 Read block 2907 len 24576
24410 032/300 Read block 1132 len 24576
24411 033/300 Read block 3270 len 20480
24412 034/300 Write block 1691 len 20480 data: 3 5 3 5 6
24413 035/300 Write block 1900 len 16384 data: 5 6 6 5
24414 036/300 Write block 3348 len 8192 data: 4 3
24415 037/300 Read block 1719 len 40960
24416 038/300 Flush
24417 039/300 Read block 3952 len 4096
24418 040/300 Write block 4153 len 4096 data: 4
24419 041/300 Flush
24420 042/300 Read block 1420 len 16384
24421 043/300 Read block 2800 len 32768
24422 044/300 Read block 693 len 12288
24423 045/300 Write block 456 len 4096 data: 4
24424 046/300 Read block 1701 len 32768
24425 047/300 Write block 3929 len 8192 data: 4 5
24426 048/300 Read block 111 len 40960
24427 049/300 Read block 4140 len 12288
24428 050/300 Read block 4579 len 32768
24429 051/300 Write block 4725 len 16384 data: 4 4 3 6
24430 052/300 Write block 2678 len 20480 data: 4 3 4 5 7
24431 053/300 Write block 2408 len 16384 data: 6 5 5 5
24432 054/300 Read block 1351 len 40960
24433 055/300 Write block 2102 len 8192 data: 4 5
24434 056/300 Read block 2202 len 16384
24435 057/300 Write block 1774 len 20480 data: 2 3 3 3 2
24436 058/300 Write block 4272 len 4096 data: 4
24437 059/300 Write block 4437 len 36864 data: 5 5 5 5 4 3 3 3 3
24438 060/300 Write block 4177 len 20480 data: 9 9 7 5 5
24439 061/300 Write block 925 len 24576 data: 4 3 3 3 5 5
24440 062/300 Read block 4980 len 16384
24441 063/300 Read block 4644 len 16384
24442 064/300 Write block 1537 len 36864 data: 3 3 3 3 3 2 2 3 3
24443 065/300 Read block 3328 len 20480
24444 066/300 Read block 901 len 40960
24445 067/300 Write block 1466 len 36864 data: 6 7 7 6 6 5 7 6 6
24446 068/300 Write block 3711 len 28672 data: 5 4 4 4 5 6 6
24447 069/300 Read block 2198 len 4096
24448 070/300 Read block 2201 len 12288
24449 071/300 Read block 790 len 36864
24450 072/300 Write block 4172 len 8192 data: 6 7
24451 073/300 Read block 1021 len 20480
24452 074/300 Flush
24453 075/300 Flush
24454 076/300 Write block 2414 len 36864 data: 3 3 3 3 4 4 4 4 3
24455 077/300 Read block 1194 len 12288
24456 078/300 Write block 30 len 32768 data: 6 7 7 5 6 4 4 3
24457 079/300 Flush
24458 080/300 Write block 4815 len 4096 data: 6
24459 081/300 Write block 1061 len 8192 data: 6 8
24460 082/300 Write block 2748 len 28672 data: 5 5 5 4 4 4 4
24461 083/300 Write block 4415 len 32768 data: 7 10 8 9 6 5 6 5
24462 084/300 Read block 3435 len 12288
24463 085/300 Read block 4206 len 36864
24464 086/300 Flush
24465 087/300 Write block 245 len 28672 data: 5 6 4 4 3 5 5
24466 088/300 Write block 2642 len 36864 data: 3 4 5 6 5 3 4 5 4
24467 089/300 Write block 4327 len 20480 data: 5 4 4 5 5
24468 090/300 Read block 4574 len 32768
24469 091/300 Read block 1237 len 36864
24470 092/300 Read block 2458 len 28672
24471 093/300 Write block 4517 len 8192 data: 3 3
24472 094/300 Write block 655 len 16384 data: 4 3 3 4
24473 095/300 Write block 896 len 24576 data: 4 4 5 4 5 6
24474 096/300 Write block 4391 len 20480 data: 2 3 4 3 4
24475 097/300 Read block 2196 len 36864
24476 098/300 Read block 4338 len 32768
24477 099/300 Read block 3723 len 36864
24478 100/300 Read block 3586 len 16384
24479 101/300 Read block 4892 len 32768
24480 102/300 Read block 1715 len 28672
24481 103/300 Read block 4036 len 16384
24482 104/300 Read block 2346 len 24576
24483 105/300 Write block 2773 len 24576 data: 5 5 4 4 4 5
24484 106/300 Write block 2419 len 32768 data: 5 5 5 4 4 3 3 4
24485 107/300 Read block 4969 len 24576
24486 108/300 Write block 1852 len 16384 data: 3 3 3 4
24487 109/300 Read block 616 len 28672
24488 110/300 Write block 1516 len 40960 data: 2 4 5 5 5 5 5 6 7 8
24489 111/300 Flush
24490 112/300 Read block 2066 len 40960
24491 113/300 Read block 363 len 28672
24492 114/300 Write block 3390 len 4096 data: 2
24493 115/300 Write block 4131 len 12288 data: 6 6 5
24494 116/300 Write block 4879 len 4096 data: 5
24495 117/300 Read block 541 len 24576
24496 118/300 Read block 768 len 32768
24497 119/300 Read block 1806 len 4096
24498 120/300 Read block 841 len 20480
24499 121/300 Read block 677 len 36864
24500 122/300 Read block 1781 len 24576
24501 123/300 Write block 2067 len 36864 data: 9 9 8 6 5 6 5 5 6
24502 124/300 Read block 2737 len 28672
24503 125/300 Read block 1701 len 36864
24504 126/300 Write block 4410 len 40960 data: 3 3 7 8 7 8 11 9 10 7
24505 127/300 Read block 3341 len 40960
24506 128/300 Flush
24507 129/300 Read block 3378 len 36864
24508 130/300 Write block 4114 len 28672 data: 2 2 3 2 5 6 5
24509 131/300 Read block 71 len 28672
24510 132/300 Read block 1649 len 24576
24511 133/300 Write block 2345 len 36864 data: 8 10 8 8 10 10 11 9 6
24512 134/300 Write block 1977 len 12288 data: 6 7 6
24513 135/300 Read block 3743 len 16384
24514 136/300 Read block 3602 len 8192
24515 137/300 Write block 4244 len 4096 data: 3
24516 138/300 Read block 1411 len 12288
24517 139/300 Write block 2283 len 20480 data: 5 4 3 3 4
24518 140/300 Write block 4014 len 28672 data: 5 6 5 3 3 3 3
24519 141/300 Read block 310 len 16384
24520 142/300 Read block 3499 len 20480
24521 143/300 Write block 4813 len 4096 data: 4
24522 144/300 Read block 556 len 36864
24523 145/300 Read block 2808 len 24576
24524 146/300 Write block 1126 len 20480 data: 3 3 4 3 2
24525 147/300 Read block 1106 len 32768
24526 148/300 Read block 2226 len 20480
24527 149/300 Read block 1934 len 28672
24528 150/300 Read block 2647 len 8192
24529 151/300 Read block 425 len 24576
24530 152/300 Flush
24531 153/300 Read block 1485 len 28672
24532 154/300 Read block 530 len 28672
24533 155/300 Read block 2987 len 16384
24534 156/300 Write block 908 len 4096 data: 3
24535 157/300 Write block 1753 len 24576 data: 3 3 2 3 4 4
24536 158/300 Write block 589 len 28672 data: 5 6 6 6 4 3 4
24537 159/300 Read block 3562 len 20480
24538 160/300 Read block 784 len 32768
24539 161/300 Flush
24540 162/300 Write block 708 len 4096 data: 7
24541 163/300 Flush
24542 164/300 Read block 4054 len 4096
24543 165/300 Read block 3972 len 12288
24544 166/300 Write block 738 len 8192 data: 5 4
24545 167/300 Write block 2986 len 28672 data: 3 4 4 4 4 4 4
24546 168/300 Write block 2327 len 4096 data: 4
24547 169/300 Write block 4784 len 4096 data: 7
24548 170/300 Read block 4424 len 12288
24549 171/300 Read block 477 len 8192
24550 172/300 Read block 2545 len 12288
24551 173/300 Write block 53 len 12288 data: 5 3 2
24552 174/300 Read block 1327 len 16384
24553 175/300 Read block 926 len 36864
24554 176/300 Read block 1255 len 8192
24555 177/300 Write block 1203 len 24576 data: 5 5 4 4 4 4
24556 178/300 Read block 1044 len 24576
24557 179/300 Write block 463 len 4096 data: 4
24558 180/300 Read block 4495 len 12288
24559 181/300 Write block 108 len 32768 data: 5 5 5 4 3 3 2 3
24560 182/300 Read block 3323 len 24576
24561 183/300 Write block 1641 len 36864 data: 3 3 2 2 2 3 3 3 4
24562 184/300 Write block 2686 len 12288 data: 6 6 6
24563 185/300 Write block 2885 len 40960 data: 4 5 6 6 6 6 5 4 3 4
24564 186/300 Read block 1158 len 20480
24565 187/300 Write block 742 len 32768 data: 6 5 6 5 4 4 4 4
24566 188/300 Write block 2680 len 20480 data: 5 6 8 7 6
24567 189/300 Read block 2502 len 8192
24568 190/300 Write block 4242 len 8192 data: 3 3
24569 191/300 Read block 1434 len 24576
24570 192/300 Read block 2092 len 32768
24571 193/300 Read block 3273 len 36864
24572 194/300 Read block 1602 len 4096
24573 195/300 Read block 95 len 24576
24574 196/300 Write block 2269 len 28672 data: 5 5 5 4 2 3 4
24575 197/300 Read block 4174 len 24576
24576 198/300 Write block 1788 len 4096 data: 6
24577 199/300 Read block 3100 len 36864
24578 200/300 Write block 724 len 24576 data: 6 5 5 3 4 5
24579 201/300 Write block 4866 len 4096 data: 3
24580 202/300 Write block 4204 len 28672 data: 4 3 4 4 5 4 4
24581 203/300 Write block 1631 len 28672 data: 4 4 5 3 4 4 4
24582 204/300 Write block 4688 len 4096 data: 7
24583 205/300 Read block 1245 len 36864
24584 206/300 Write block 1515 len 28672 data: 2 3 5 6 6 6 6
24585 207/300 Read block 2584 len 16384
24586 208/300 Write block 4878 len 8192 data: 4 6
24587 209/300 Read block 1042 len 16384
24588 210/300 Read block 4944 len 4096
24589 211/300 Read block 1380 len 24576
24590 212/300 Read block 4005 len 4096
24591 213/300 Read block 3256 len 24576
24592 214/300 Write block 3100 len 36864 data: 4 5 6 7 9 9 7 7 7
24593 215/300 Read block 4155 len 24576
24594 216/300 Write block 4927 len 32768 data: 5 4 4 4 2 3 3 4
24595 217/300 Read block 56 len 16384
24596 218/300 Read block 1549 len 20480
24597 219/300 Read block 4414 len 40960
24598 220/300 Read block 4357 len 20480
24599 221/300 Flush
24600 222/300 Read block 4649 len 16384
24601 223/300 Write block 2082 len 12288 data: 4 4 3
24602 224/300 Write block 2669 len 20480 data: 3 4 4 4 4
24603 225/300 Write block 3975 len 24576 data: 4 4 4 3 3 3
24604 226/300 Read block 4733 len 32768
24605 227/300 Write block 3360 len 40960 data: 4 5 4 5 4 2 2 2 2 2
24606 228/300 Write block 4830 len 16384 data: 6 3 3 3
24607 229/300 Write block 4835 len 28672 data: 2 2 2 2 3 2 3
24608 230/300 Read block 477 len 36864
24609 231/300 Write block 102 len 20480 data: 6 7 7 6 4
24610 232/300 Write block 1220 len 24576 data: 3 3 3 4 3 4
24611 233/300 Write block 1002 len 40960 data: 4 3 4 5 4 6 7 8 8 7
24612 234/300 Read block 4068 len 4096
24613 235/300 Read block 713 len 4096
24614 236/300 Read block 1285 len 24576
24615 237/300 Read block 2963 len 28672
24616 238/300 Flush
24617 239/300 Read block 729 len 8192
24618 240/300 Flush
24619 241/300 Write block 4825 len 36864 data: 6 7 6 7 6 7 4 4 4
24620 242/300 Flush
24621 243/300 Read block 4714 len 32768
24622 244/300 Read block 134 len 4096
24623 245/300 Flush
24624 246/300 Write block 1166 len 40960 data: 2 2 3 5 5 4 3 4 3 3
24625 247/300 Read block 478 len 40960
24626 248/300 Read block 386 len 12288
24627 249/300 Write block 3595 len 4096 data: 6
24628 250/300 Read block 1621 len 24576
24629 251/300 Read block 309 len 40960
24630 252/300 Write block 1132 len 16384 data: 3 3 3 3
24631 253/300 Flush
24632 254/300 Read block 4243 len 4096
24633 255/300 Write block 2783 len 36864 data: 4 5 5 5 7 6 6 9 9
24634 256/300 Write block 4334 len 8192 data: 5 5
24635 257/300 Write block 2539 len 20480 data: 6 6 6 6 5
24636 258/300 Read block 4189 len 28672
24637 259/300 Flush
24638 260/300 Read block 3918 len 24576
24639 261/300 Flush
24640 262/300 Read block 2504 len 12288
24641 263/300 Write block 133 len 8192 data: 3 4
24642 264/300 Read block 163 len 8192
24643 265/300 Write block 2038 len 36864 data: 4 4 4 5 3 5 6 6 7
24644 266/300 Read block 3854 len 20480
24645 267/300 Write block 2312 len 8192 data: 5 4
24646 268/300 Write block 4161 len 8192 data: 3 3
24647 269/300 Write block 3258 len 8192 data: 6 3
24648 270/300 Write block 1155 len 24576 data: 8 9 11 9 9 8
24649 271/300 Read block 2123 len 4096
24650 272/300 Write block 1695 len 16384 data: 7 5 5 5
24651 273/300 Read block 1316 len 28672
24652 274/300 Read block 3759 len 24576
24653 275/300 Read block 1816 len 32768
24654 276/300 Write block 336 len 40960 data: 4 6 6 5 4 4 5 4 3 3
24655 277/300 Read block 1227 len 32768
24656 278/300 Write block 2287 len 40960 data: 5 6 6 6 8 8 7 8 8 8
24657 279/300 Write block 763 len 40960 data: 5 6 6 6 2 2 2 3 3 3
24658 280/300 Read block 128 len 28672
24659 281/300 Flush
24660 282/300 Write block 342 len 32768 data: 6 5 4 4 4 5 6 4
24661 283/300 Write block 135 len 24576 data: 3 3 3 4 4 4
24662 284/300 Read block 4576 len 32768
24663 285/300 Write block 587 len 36864 data: 6 5 6 7 7 7 5 4 5
24664 286/300 Read block 1587 len 8192
24665 287/300 Read block 1116 len 36864
24666 288/300 Write block 926 len 8192 data: 4 4
24667 289/300 Write block 4901 len 16384 data: 4 4 4 4
24668 290/300 Flush
24669 291/300 Write block 1765 len 36864 data: 4 4 3 3 3 3 3 2 2
24670 292/300 Write block 4624 len 8192 data: 6 5
24671 293/300 Read block 344 len 36864
24672 294/300 Read block 1371 len 28672
24673 295/300 Write block 1188 len 24576 data: 3 3 4 4 4 6
24674 296/300 Read block 3549 len 12288
24675 297/300 Read block 1156 len 32768
24676 298/300 Write block 2228 len 4096 data: 4
24677 299/300 Flush
24678 300/300 Write block 2074 len 8192 data: 6 7
24679 301/300 Write block 482 len 16384 data: 3 4 4 4
24680 Main task has work to do, go find it
24681 got dsc Start(0)
24682 start 0
24683 Jan 17 21:16:17.656 INFO request completed, latency_us: 213, response_code: 204, uri: /start/cid/0, method: POST, req_id: 7f04e0c8-177c-4b35-9f6f-bd62626a010e, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
24684 Replay: started 0, returned:Ok(())
24685 ----------------------------------------------------------------
24686 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
24687 States: Offline Active Active
24688 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24689 7037 Acked Write 3 Sent Done Done false
24690 7038 Acked Write 9 Sent Done Done false
24691 7039 Acked Read 8 Sent Done Done false
24692 7040 Acked Write 10 Sent Done Done false
24693 7041 Acked Write 10 Sent Done Done false
24694 7042 Acked Write 8 Sent Done Done false
24695 7043 Acked Write 5 Sent Done Done false
24696 7044 Acked Read 7 Sent Done Done false
24697 7045 Acked Read 3 Sent Done Done false
24698 7046 Acked Write 1 Sent Done Done false
24699 7047 Acked Write 1 Sent Done Done false
24700 7048 Acked Flush 0 Sent Done Done false
24701 7049 Acked Read 1 Sent Done Done false
24702 7050 Acked Read 2 Sent Done Done false
24703 7051 Acked Write 10 Sent Done Done false
24704 7052 Acked Read 4 Sent Done Done false
24705 7053 Acked Write 9 Sent Done Done false
24706 7054 Acked Read 7 Sent Done Done false
24707 7055 Acked Write 2 Sent Done Done false
24708 7056 Acked Read 2 Sent Done Done false
24709 7057 Acked Read 3 Sent Done Done false
24710 7058 Acked Write 9 Sent Done Done false
24711 7059 Acked Read 1 Sent Done Done false
24712 7060 Acked Read 6 Sent Done Done false
24713 7061 Acked Read 5 Sent Done Done false
24714 7062 Acked Write 9 Sent Done Done false
24715 7063 Acked Read 7 Sent Done Done false
24716 7064 Acked Read 8 Sent Done Done false
24717 7065 Acked Write 1 Sent Done Done false
24718 7066 Acked Read 9 Sent Done Done false
24719 7067 Acked Read 6 Sent Done Done false
24720 7068 Acked Read 6 Sent Done Done false
24721 7069 Acked Read 5 Sent Done Done false
24722 7070 Acked Write 5 Sent Done Done false
24723 7071 Acked Write 4 Sent Done Done false
24724 7072 Acked Write 2 Sent Done Done false
24725 7073 Acked Read 10 Sent Done Done false
24726 7074 Acked Flush 0 Sent Done Done false
24727 7075 Acked Read 1 Sent Done Done false
24728 7076 Acked Write 1 Sent Done Done false
24729 7077 Acked Flush 0 Sent Done Done false
24730 7078 Acked Read 4 Sent Done Done false
24731 7079 Acked Read 8 Sent Done Done false
24732 7080 Acked Read 3 Sent Done Done false
24733 7081 Acked Write 1 Sent Done Done false
24734 7082 Acked Read 8 Sent Done Done false
24735 7083 Acked Write 2 Sent Done Done false
24736 7084 Acked Read 10 Sent Done Done false
24737 7085 Acked Read 3 Sent Done Done false
24738 7086 Acked Read 8 Sent Done Done false
24739 7087 Acked Write 4 Sent Done Done false
24740 7088 Acked Write 5 Sent Done Done false
24741 7089 Acked Write 4 Sent Done Done false
24742 7090 Acked Read 10 Sent Done Done false
24743 7091 Acked Write 2 Sent Done Done false
24744 7092 Acked Read 4 Sent Done Done false
24745 7093 Acked Write 5 Sent Done Done false
24746 7094 Acked Write 1 Sent Done Done false
24747 7095 Acked Write 9 Sent Done Done false
24748 7096 Acked Write 5 Sent Done Done false
24749 7097 Acked Write 6 Sent Done Done false
24750 7098 Acked Read 4 Sent Done Done false
24751 7099 Acked Read 4 Sent Done Done false
24752 7100 Acked Write 9 Sent Done Done false
24753 7101 Acked Read 5 Sent Done Done false
24754 7102 Acked Read 10 Sent Done Done false
24755 7103 Acked Write 9 Sent Done Done false
24756 7104 Acked Write 7 Sent Done Done false
24757 7105 Acked Read 1 Sent Done Done false
24758 7106 Acked Read 3 Sent Done Done false
24759 7107 Acked Read 9 Sent Done Done false
24760 7108 Acked Write 2 Sent Done Done false
24761 7109 Acked Read 5 Sent Done Done false
24762 7110 Acked Flush 0 Sent Done Done false
24763 7111 Acked Flush 0 Sent Done Done false
24764 7112 Acked Write 9 Sent Done Done false
24765 7113 Acked Read 3 Sent Done Done false
24766 7114 Acked Write 8 Sent Done Done false
24767 7115 Acked Flush 0 Sent Done Done false
24768 7116 Acked Write 1 Sent Done Done false
24769 7117 Acked Write 2 Sent Done Done false
24770 7118 Acked Write 7 Sent Done Done false
24771 7119 Acked Write 8 Sent Done Done false
24772 7120 Acked Read 3 Sent Done Done false
24773 7121 Acked Read 9 Sent Done Done false
24774 7122 Acked Flush 0 Sent Done Done false
24775 7123 Acked Write 7 Sent Done Done false
24776 7124 Acked Write 9 Sent Done Done false
24777 7125 Acked Write 5 Sent Done Done false
24778 7126 Acked Read 8 Sent Done Done false
24779 7127 Acked Read 9 Sent Done Done false
24780 7128 Acked Read 7 Sent Done Done false
24781 7129 Acked Write 2 Sent Done Done false
24782 7130 Acked Write 4 Sent Done Done false
24783 7131 Acked Write 6 Sent Done Done false
24784 7132 Acked Write 5 Sent Done Done false
24785 7133 Acked Read 9 Sent Done Done false
24786 7134 Acked Read 8 Sent Done Done false
24787 7135 Acked Read 9 Sent Done Done false
24788 7136 Acked Read 4 Sent Done Done false
24789 7137 Acked Read 8 Sent Done Done false
24790 7138 Acked Read 7 Sent Done Done false
24791 7139 Acked Read 4 Sent Done Done false
24792 7140 Acked Read 6 Sent Done Done false
24793 7141 Acked Write 6 Sent Done Done false
24794 7142 Acked Write 8 Sent Done Done false
24795 7143 Acked Read 6 Sent Done Done false
24796 7144 Acked Write 4 Sent Done Done false
24797 7145 Acked Read 7 Sent Done Done false
24798 7146 Acked Write 10 Sent Done Done false
24799 7147 Acked Flush 0 Sent Done Done false
24800 7148 Acked Read 10 Sent Done Done false
24801 7149 Acked Read 7 Sent Done Done false
24802 7150 Acked Write 1 Sent Done Done false
24803 7151 Acked Write 3 Sent Done Done false
24804 7152 Acked Write 1 Sent Done Done false
24805 7153 Acked Read 6 Sent Done Done false
24806 7154 Acked Read 8 Sent Done Done false
24807 7155 Acked Read 1 Sent Done Done false
24808 7156 Acked Read 5 Sent Done Done false
24809 7157 Acked Read 9 Sent Done Done false
24810 7158 Acked Read 6 Sent Done Done false
24811 7159 Acked Write 9 Sent Done Done false
24812 7160 Acked Read 7 Sent Done Done false
24813 7161 Acked Read 9 Sent Done Done false
24814 7162 Acked Write 10 Sent Done Done false
24815 7163 Acked Read 10 Sent Done Done false
24816 7164 Acked Flush 0 Sent Done Done false
24817 7165 Acked Read 9 Sent Done Done false
24818 7166 Acked Write 7 Sent Done Done false
24819 7167 Acked Read 7 Sent Done Done false
24820 7168 Acked Read 6 Sent Done Done false
24821 7169 Acked Write 9 Sent Done Done false
24822 7170 Acked Write 3 Sent Done Done false
24823 7171 Acked Read 4 Sent Done Done false
24824 7172 Acked Read 2 Sent Done Done false
24825 7173 Acked Write 1 Sent Done Done false
24826 7174 Acked Read 3 Sent Done Done false
24827 7175 Acked Write 5 Sent Done Done false
24828 7176 Acked Write 7 Sent Done Done false
24829 7177 Acked Read 4 Sent Done Done false
24830 7178 Acked Read 5 Sent Done Done false
24831 7179 Acked Write 1 Sent Done Done false
24832 7180 Acked Read 9 Sent Done Done false
24833 7181 Acked Read 6 Sent Done Done false
24834 7182 Acked Write 5 Sent Done Done false
24835 7183 Acked Read 8 Sent Done Done false
24836 7184 Acked Read 5 Sent Done Done false
24837 7185 Acked Read 7 Sent Done Done false
24838 7186 Acked Read 2 Sent Done Done false
24839 7187 Acked Read 6 Sent Done Done false
24840 7188 Acked Flush 0 Sent Done Done false
24841 7189 Acked Read 7 Sent Done Done false
24842 7190 Acked Read 7 Sent Done Done false
24843 7191 Acked Read 4 Sent Done Done false
24844 7192 Acked Write 1 Sent Done Done false
24845 7193 Acked Write 6 Sent Done Done false
24846 7194 Acked Write 7 Sent Done Done false
24847 7195 Acked Read 5 Sent Done Done false
24848 7196 Acked Read 8 Sent Done Done false
24849 7197 Acked Flush 0 Sent Done Done false
24850 7198 Acked Write 1 Sent Done Done false
24851 7199 Acked Flush 0 Sent Done Done false
24852 7200 Acked Read 1 Sent Done Done false
24853 7201 Acked Read 3 Sent Done Done false
24854 7202 Acked Write 2 Sent Done Done false
24855 7203 Acked Write 7 Sent Done Done false
24856 7204 Acked Write 1 Sent Done Done false
24857 7205 Acked Write 1 Sent Done Done false
24858 7206 Acked Read 3 Sent Done Done false
24859 7207 Acked Read 2 Sent Done Done false
24860 7208 Acked Read 3 Sent Done Done false
24861 7209 Acked Write 3 Sent Done Done false
24862 7210 Acked Read 4 Sent Done Done false
24863 7211 Acked Read 9 Sent Done Done false
24864 7212 Acked Read 2 Sent Done Done false
24865 7213 Acked Write 6 Sent Done Done false
24866 7214 Acked Read 6 Sent Done Done false
24867 7215 Acked Write 1 Sent Done Done false
24868 7216 Acked Read 3 Sent Done Done false
24869 7217 Acked Write 8 Sent Done Done false
24870 7218 Acked Read 6 Sent Done Done false
24871 7219 Acked Write 9 Sent Done Done false
24872 7220 Acked Write 3 Sent Done Done false
24873 7221 Acked Write 10 Sent Done Done false
24874 7222 Acked Read 5 Sent Done Done false
24875 7223 Acked Write 8 Sent Done Done false
24876 7224 Acked Write 5 Sent Done Done false
24877 7225 Acked Read 2 Sent Done Done false
24878 7226 Acked Write 2 Sent Done Done false
24879 7227 Acked Read 6 Sent Done Done false
24880 7228 Acked Read 8 Sent Done Done false
24881 7229 Acked Read 9 Sent Done Done false
24882 7230 Acked Read 1 Sent Done Done false
24883 7231 Acked Read 6 Sent Done Done false
24884 7232 Acked Write 7 Sent Done Done false
24885 7233 Acked Read 6 Sent Done Done false
24886 7234 Acked Write 1 Sent Done Done false
24887 7235 Acked Read 9 Sent Done Done false
24888 7236 Acked Write 6 Sent Done Done false
24889 7237 Acked Write 1 Sent Done Done false
24890 7238 Acked Write 7 Sent Done Done false
24891 7239 Acked Write 7 Sent Done Done false
24892 7240 Acked Write 1 Sent Done Done false
24893 7241 Acked Read 9 Sent Done Done false
24894 7242 Acked Write 7 Sent Done Done false
24895 7243 Acked Read 4 Sent Done Done false
24896 7244 Acked Write 2 Sent Done Done false
24897 7245 Acked Read 4 Sent Done Done false
24898 7246 Acked Read 1 Sent Done Done false
24899 7247 Acked Read 6 Sent Done Done false
24900 7248 Acked Read 1 Sent Done Done false
24901 7249 Acked Read 6 Sent Done Done false
24902 7250 Acked Write 9 Sent Done Done false
24903 7251 Acked Read 6 Sent Done Done false
24904 7252 Acked Write 8 Sent Done Done false
24905 7253 Acked Read 4 Sent Done Done false
24906 7254 Acked Read 5 Sent Done Done false
24907 7255 Acked Read 10 Sent Done Done false
24908 7256 Acked Read 5 Sent Done Done false
24909 7257 Acked Flush 0 Sent Done Done false
24910 7258 Acked Read 4 Sent Done Done false
24911 7259 Acked Write 3 Sent Done Done false
24912 7260 Acked Write 5 Sent Done Done false
24913 7261 Acked Write 6 Sent Done Done false
24914 7262 Acked Read 8 Sent Done Done false
24915 7263 Acked Write 10 Sent Done Done false
24916 7264 Acked Write 4 Sent Done Done false
24917 7265 Acked Write 7 Sent Done Done false
24918 7266 Acked Read 9 Sent Done Done false
24919 7267 Acked Write 5 Sent Done Done false
24920 7268 Acked Write 6 Sent Done Done false
24921 7269 Acked Write 10 Sent Done Done false
24922 7270 Acked Read 1 Sent Done Done false
24923 7271 Acked Read 1 Sent Done Done false
24924 7272 Acked Read 6 Sent Done Done false
24925 7273 Acked Read 7 Sent Done Done false
24926 7274 Acked Flush 0 Sent Done Done false
24927 7275 Acked Read 2 Sent Done Done false
24928 7276 Acked Flush 0 Sent Done Done false
24929 7277 Acked Write 9 Sent Done Done false
24930 7278 Acked Flush 0 Sent Done Done false
24931 7279 Acked Read 8 Sent Done Done false
24932 7280 Acked Read 1 Sent Done Done false
24933 7281 Acked Flush 0 Sent Done Done false
24934 7282 Acked Write 10 Sent Done Done false
24935 7283 Acked Read 10 Sent Done Done false
24936 7284 Acked Read 3 Sent Done Done false
24937 7285 Acked Write 1 Sent Done Done false
24938 7286 Acked Read 6 Sent Done Done false
24939 7287 Acked Read 10 Sent Done Done false
24940 7288 Acked Write 4 Sent Done Done false
24941 7289 Acked Flush 0 Sent Done Done false
24942 7290 Acked Read 1 Sent Done Done false
24943 7291 Acked Write 9 Sent Done Done false
24944 7292 Acked Write 2 Sent Done Done false
24945 7293 Acked Write 5 Sent Done Done false
24946 7294 Acked Read 7 Sent Done Done false
24947 7295 Acked Flush 0 Sent Done Done false
24948 7296 Acked Read 6 Sent Done Done false
24949 7297 Acked Flush 0 Sent Done Done false
24950 7298 Acked Read 3 Sent Done Done false
24951 7299 Acked Write 2 Sent Done Done false
24952 7300 Acked Read 2 Sent Done Done false
24953 7301 Acked Write 9 Sent Done Done false
24954 7302 Acked Read 5 Sent Done Done false
24955 7303 Acked Write 2 Sent Done Done false
24956 7304 Acked Write 2 Sent Done Done false
24957 7305 Acked Write 2 Sent Done Done false
24958 7306 Acked Write 6 Sent Done Done false
24959 7307 Acked Read 1 Sent Done Done false
24960 7308 Acked Write 4 Sent Done Done false
24961 7309 Acked Read 7 Sent Done Done false
24962 7310 Acked Read 6 Sent Done Done false
24963 7311 Acked Read 8 Sent Done Done false
24964 7312 Acked Write 10 Sent Done Done false
24965 7313 Acked Read 8 Sent Done Done false
24966 7314 Acked Write 10 Sent Done Done false
24967 7315 Acked Write 10 Sent Done Done false
24968 7316 Acked Read 7 Sent Done Done false
24969 7317 Acked Flush 0 Sent Done Done false
24970 7318 Acked Write 8 Sent Done Done false
24971 7319 Acked Write 6 Sent Done Done false
24972 7320 Acked Read 8 Sent Done Done false
24973 7321 Acked Write 9 Sent Done Done false
24974 7322 Acked Read 2 Sent Done Done false
24975 7323 Acked Read 9 Sent Done Done false
24976 7324 Acked Write 2 Sent Done Done false
24977 7325 Acked Write 4 Sent Done Done false
24978 7326 Acked Flush 0 Sent Done Done false
24979 7327 Acked Write 9 Sent Done Done false
24980 7328 Acked Write 2 Sent Done Done false
24981 7329 Acked Read 9 Sent Done Done false
24982 7330 Acked Read 7 Sent Done Done false
24983 7331 Acked Write 6 Sent Done Done false
24984 7332 Acked Read 3 Sent Done Done false
24985 7333 Acked Read 8 Sent Done Done false
24986 7334 Acked Write 1 Sent Done Done false
24987 7335 Acked Flush 0 Sent Done Done false
24988 7336 Acked Write 2 Sent Done Done false
24989 7337 Acked Write 4 Sent Done Done false
24990 STATES DS:0 DS:1 DS:2 TOTAL
24991 Sent 301 0 0 301
24992 Done 0 301 301 602
24993 Skipped 0 0 0 0
24994 Error 0 0 0 0
24995 Last Flush: 7036 7335 7335
24996 Downstairs last five completed: 7036 7035 7034 7033 7032
24997 Upstairs last five completed: 7337 7336 7335 7334 7333
24998 CLIENT: Up:0 ds:301 act:2
24999 ----------------------------------------------------------------
25000 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
25001 States: Offline Active Active
25002 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25003 7037 Acked Write 3 Sent Done Done false
25004 7038 Acked Write 9 Sent Done Done false
25005 7039 Acked Read 8 Sent Done Done false
25006 7040 Acked Write 10 Sent Done Done false
25007 7041 Acked Write 10 Sent Done Done false
25008 7042 Acked Write 8 Sent Done Done false
25009 7043 Acked Write 5 Sent Done Done false
25010 7044 Acked Read 7 Sent Done Done false
25011 7045 Acked Read 3 Sent Done Done false
25012 7046 Acked Write 1 Sent Done Done false
25013 7047 Acked Write 1 Sent Done Done false
25014 7048 Acked Flush 0 Sent Done Done false
25015 7049 Acked Read 1 Sent Done Done false
25016 7050 Acked Read 2 Sent Done Done false
25017 7051 Acked Write 10 Sent Done Done false
25018 7052 Acked Read 4 Sent Done Done false
25019 7053 Acked Write 9 Sent Done Done false
25020 7054 Acked Read 7 Sent Done Done false
25021 7055 Acked Write 2 Sent Done Done false
25022 7056 Acked Read 2 Sent Done Done false
25023 7057 Acked Read 3 Sent Done Done false
25024 7058 Acked Write 9 Sent Done Done false
25025 7059 Acked Read 1 Sent Done Done false
25026 7060 Acked Read 6 Sent Done Done false
25027 7061 Acked Read 5 Sent Done Done false
25028 7062 Acked Write 9 Sent Done Done false
25029 7063 Acked Read 7 Sent Done Done false
25030 7064 Acked Read 8 Sent Done Done false
25031 7065 Acked Write 1 Sent Done Done false
25032 7066 Acked Read 9 Sent Done Done false
25033 7067 Acked Read 6 Sent Done Done false
25034 7068 Acked Read 6 Sent Done Done false
25035 7069 Acked Read 5 Sent Done Done false
25036 7070 Acked Write 5 Sent Done Done false
25037 7071 Acked Write 4 Sent Done Done false
25038 7072 Acked Write 2 Sent Done Done false
25039 7073 Acked Read 10 Sent Done Done false
25040 7074 Acked Flush 0 Sent Done Done false
25041 7075 Acked Read 1 Sent Done Done false
25042 7076 Acked Write 1 Sent Done Done false
25043 7077 Acked Flush 0 Sent Done Done false
25044 7078 Acked Read 4 Sent Done Done false
25045 7079 Acked Read 8 Sent Done Done false
25046 7080 Acked Read 3 Sent Done Done false
25047 7081 Acked Write 1 Sent Done Done false
25048 7082 Acked Read 8 Sent Done Done false
25049 7083 Acked Write 2 Sent Done Done false
25050 7084 Acked Read 10 Sent Done Done false
25051 7085 Acked Read 3 Sent Done Done false
25052 7086 Acked Read 8 Sent Done Done false
25053 7087 Acked Write 4 Sent Done Done false
25054 7088 Acked Write 5 Sent Done Done false
25055 7089 Acked Write 4 Sent Done Done false
25056 7090 Acked Read 10 Sent Done Done false
25057 7091 Acked Write 2 Sent Done Done false
25058 7092 Acked Read 4 Sent Done Done false
25059 7093 Acked Write 5 Sent Done Done false
25060 7094 Acked Write 1 Sent Done Done false
25061 7095 Acked Write 9 Sent Done Done false
25062 7096 Acked Write 5 Sent Done Done false
25063 7097 Acked Write 6 Sent Done Done false
25064 7098 Acked Read 4 Sent Done Done false
25065 7099 Acked Read 4 Sent Done Done false
25066 7100 Acked Write 9 Sent Done Done false
25067 7101 Acked Read 5 Sent Done Done false
25068 7102 Acked Read 10 Sent Done Done false
25069 7103 Acked Write 9 Sent Done Done false
25070 7104 Acked Write 7 Sent Done Done false
25071 7105 Acked Read 1 Sent Done Done false
25072 7106 Acked Read 3 Sent Done Done false
25073 7107 Acked Read 9 Sent Done Done false
25074 7108 Acked Write 2 Sent Done Done false
25075 7109 Acked Read 5 Sent Done Done false
25076 7110 Acked Flush 0 Sent Done Done false
25077 7111 Acked Flush 0 Sent Done Done false
25078 7112 Acked Write 9 Sent Done Done false
25079 7113 Acked Read 3 Sent Done Done false
25080 7114 Acked Write 8 Sent Done Done false
25081 7115 Acked Flush 0 Sent Done Done false
25082 7116 Acked Write 1 Sent Done Done false
25083 7117 Acked Write 2 Sent Done Done false
25084 7118 Acked Write 7 Sent Done Done false
25085 7119 Acked Write 8 Sent Done Done false
25086 7120 Acked Read 3 Sent Done Done false
25087 7121 Acked Read 9 Sent Done Done false
25088 7122 Acked Flush 0 Sent Done Done false
25089 7123 Acked Write 7 Sent Done Done false
25090 7124 Acked Write 9 Sent Done Done false
25091 7125 Acked Write 5 Sent Done Done false
25092 7126 Acked Read 8 Sent Done Done false
25093 7127 Acked Read 9 Sent Done Done false
25094 7128 Acked Read 7 Sent Done Done false
25095 7129 Acked Write 2 Sent Done Done false
25096 7130 Acked Write 4 Sent Done Done false
25097 7131 Acked Write 6 Sent Done Done false
25098 7132 Acked Write 5 Sent Done Done false
25099 7133 Acked Read 9 Sent Done Done false
25100 7134 Acked Read 8 Sent Done Done false
25101 7135 Acked Read 9 Sent Done Done false
25102 7136 Acked Read 4 Sent Done Done false
25103 7137 Acked Read 8 Sent Done Done false
25104 7138 Acked Read 7 Sent Done Done false
25105 7139 Acked Read 4 Sent Done Done false
25106 7140 Acked Read 6 Sent Done Done false
25107 7141 Acked Write 6 Sent Done Done false
25108 7142 Acked Write 8 Sent Done Done false
25109 7143 Acked Read 6 Sent Done Done false
25110 7144 Acked Write 4 Sent Done Done false
25111 7145 Acked Read 7 Sent Done Done false
25112 7146 Acked Write 10 Sent Done Done false
25113 7147 Acked Flush 0 Sent Done Done false
25114 7148 Acked Read 10 Sent Done Done false
25115 7149 Acked Read 7 Sent Done Done false
25116 7150 Acked Write 1 Sent Done Done false
25117 7151 Acked Write 3 Sent Done Done false
25118 7152 Acked Write 1 Sent Done Done false
25119 7153 Acked Read 6 Sent Done Done false
25120 7154 Acked Read 8 Sent Done Done false
25121 7155 Acked Read 1 Sent Done Done false
25122 7156 Acked Read 5 Sent Done Done false
25123 7157 Acked Read 9 Sent Done Done false
25124 7158 Acked Read 6 Sent Done Done false
25125 7159 Acked Write 9 Sent Done Done false
25126 7160 Acked Read 7 Sent Done Done false
25127 7161 Acked Read 9 Sent Done Done false
25128 7162 Acked Write 10 Sent Done Done false
25129 7163 Acked Read 10 Sent Done Done false
25130 7164 Acked Flush 0 Sent Done Done false
25131 7165 Acked Read 9 Sent Done Done false
25132 7166 Acked Write 7 Sent Done Done false
25133 7167 Acked Read 7 Sent Done Done false
25134 7168 Acked Read 6 Sent Done Done false
25135 7169 Acked Write 9 Sent Done Done false
25136 7170 Acked Write 3 Sent Done Done false
25137 7171 Acked Read 4 Sent Done Done false
25138 7172 Acked Read 2 Sent Done Done false
25139 7173 Acked Write 1 Sent Done Done false
25140 7174 Acked Read 3 Sent Done Done false
25141 7175 Acked Write 5 Sent Done Done false
25142 7176 Acked Write 7 Sent Done Done false
25143 7177 Acked Read 4 Sent Done Done false
25144 7178 Acked Read 5 Sent Done Done false
25145 7179 Acked Write 1 Sent Done Done false
25146 7180 Acked Read 9 Sent Done Done false
25147 7181 Acked Read 6 Sent Done Done false
25148 7182 Acked Write 5 Sent Done Done false
25149 7183 Acked Read 8 Sent Done Done false
25150 7184 Acked Read 5 Sent Done Done false
25151 7185 Acked Read 7 Sent Done Done false
25152 7186 Acked Read 2 Sent Done Done false
25153 7187 Acked Read 6 Sent Done Done false
25154 7188 Acked Flush 0 Sent Done Done false
25155 7189 Acked Read 7 Sent Done Done false
25156 7190 Acked Read 7 Sent Done Done false
25157 7191 Acked Read 4 Sent Done Done false
25158 7192 Acked Write 1 Sent Done Done false
25159 7193 Acked Write 6 Sent Done Done false
25160 7194 Acked Write 7 Sent Done Done false
25161 7195 Acked Read 5 Sent Done Done false
25162 7196 Acked Read 8 Sent Done Done false
25163 7197 Acked Flush 0 Sent Done Done false
25164 7198 Acked Write 1 Sent Done Done false
25165 7199 Acked Flush 0 Sent Done Done false
25166 7200 Acked Read 1 Sent Done Done false
25167 7201 Acked Read 3 Sent Done Done false
25168 7202 Acked Write 2 Sent Done Done false
25169 7203 Acked Write 7 Sent Done Done false
25170 7204 Acked Write 1 Sent Done Done false
25171 7205 Acked Write 1 Sent Done Done false
25172 7206 Acked Read 3 Sent Done Done false
25173 7207 Acked Read 2 Sent Done Done false
25174 7208 Acked Read 3 Sent Done Done false
25175 7209 Acked Write 3 Sent Done Done false
25176 7210 Acked Read 4 Sent Done Done false
25177 7211 Acked Read 9 Sent Done Done false
25178 7212 Acked Read 2 Sent Done Done false
25179 7213 Acked Write 6 Sent Done Done false
25180 7214 Acked Read 6 Sent Done Done false
25181 7215 Acked Write 1 Sent Done Done false
25182 7216 Acked Read 3 Sent Done Done false
25183 7217 Acked Write 8 Sent Done Done false
25184 7218 Acked Read 6 Sent Done Done false
25185 7219 Acked Write 9 Sent Done Done false
25186 7220 Acked Write 3 Sent Done Done false
25187 7221 Acked Write 10 Sent Done Done false
25188 7222 Acked Read 5 Sent Done Done false
25189 7223 Acked Write 8 Sent Done Done false
25190 7224 Acked Write 5 Sent Done Done false
25191 7225 Acked Read 2 Sent Done Done false
25192 7226 Acked Write 2 Sent Done Done false
25193 7227 Acked Read 6 Sent Done Done false
25194 7228 Acked Read 8 Sent Done Done false
25195 7229 Acked Read 9 Sent Done Done false
25196 7230 Acked Read 1 Sent Done Done false
25197 7231 Acked Read 6 Sent Done Done false
25198 7232 Acked Write 7 Sent Done Done false
25199 7233 Acked Read 6 Sent Done Done false
25200 7234 Acked Write 1 Sent Done Done false
25201 7235 Acked Read 9 Sent Done Done false
25202 7236 Acked Write 6 Sent Done Done false
25203 7237 Acked Write 1 Sent Done Done false
25204 7238 Acked Write 7 Sent Done Done false
25205 7239 Acked Write 7 Sent Done Done false
25206 7240 Acked Write 1 Sent Done Done false
25207 7241 Acked Read 9 Sent Done Done false
25208 7242 Acked Write 7 Sent Done Done false
25209 7243 Acked Read 4 Sent Done Done false
25210 7244 Acked Write 2 Sent Done Done false
25211 7245 Acked Read 4 Sent Done Done false
25212 7246 Acked Read 1 Sent Done Done false
25213 7247 Acked Read 6 Sent Done Done false
25214 7248 Acked Read 1 Sent Done Done false
25215 7249 Acked Read 6 Sent Done Done false
25216 7250 Acked Write 9 Sent Done Done false
25217 7251 Acked Read 6 Sent Done Done false
25218 7252 Acked Write 8 Sent Done Done false
25219 7253 Acked Read 4 Sent Done Done false
25220 7254 Acked Read 5 Sent Done Done false
25221 7255 Acked Read 10 Sent Done Done false
25222 7256 Acked Read 5 Sent Done Done false
25223 7257 Acked Flush 0 Sent Done Done false
25224 7258 Acked Read 4 Sent Done Done false
25225 7259 Acked Write 3 Sent Done Done false
25226 7260 Acked Write 5 Sent Done Done false
25227 7261 Acked Write 6 Sent Done Done false
25228 7262 Acked Read 8 Sent Done Done false
25229 7263 Acked Write 10 Sent Done Done false
25230 7264 Acked Write 4 Sent Done Done false
25231 7265 Acked Write 7 Sent Done Done false
25232 7266 Acked Read 9 Sent Done Done false
25233 7267 Acked Write 5 Sent Done Done false
25234 7268 Acked Write 6 Sent Done Done false
25235 7269 Acked Write 10 Sent Done Done false
25236 7270 Acked Read 1 Sent Done Done false
25237 7271 Acked Read 1 Sent Done Done false
25238 7272 Acked Read 6 Sent Done Done false
25239 7273 Acked Read 7 Sent Done Done false
25240 7274 Acked Flush 0 Sent Done Done false
25241 7275 Acked Read 2 Sent Done Done false
25242 7276 Acked Flush 0 Sent Done Done false
25243 7277 Acked Write 9 Sent Done Done false
25244 7278 Acked Flush 0 Sent Done Done false
25245 7279 Acked Read 8 Sent Done Done false
25246 7280 Acked Read 1 Sent Done Done false
25247 7281 Acked Flush 0 Sent Done Done false
25248 7282 Acked Write 10 Sent Done Done false
25249 7283 Acked Read 10 Sent Done Done false
25250 7284 Acked Read 3 Sent Done Done false
25251 7285 Acked Write 1 Sent Done Done false
25252 7286 Acked Read 6 Sent Done Done false
25253 7287 Acked Read 10 Sent Done Done false
25254 7288 Acked Write 4 Sent Done Done false
25255 7289 Acked Flush 0 Sent Done Done false
25256 7290 Acked Read 1 Sent Done Done false
25257 7291 Acked Write 9 Sent Done Done false
25258 7292 Acked Write 2 Sent Done Done false
25259 7293 Acked Write 5 Sent Done Done false
25260 7294 Acked Read 7 Sent Done Done false
25261 7295 Acked Flush 0 Sent Done Done false
25262 7296 Acked Read 6 Sent Done Done false
25263 7297 Acked Flush 0 Sent Done Done false
25264 7298 Acked Read 3 Sent Done Done false
25265 7299 Acked Write 2 Sent Done Done false
25266 7300 Acked Read 2 Sent Done Done false
25267 7301 Acked Write 9 Sent Done Done false
25268 7302 Acked Read 5 Sent Done Done false
25269 7303 Acked Write 2 Sent Done Done false
25270 7304 Acked Write 2 Sent Done Done false
25271 7305 Acked Write 2 Sent Done Done false
25272 7306 Acked Write 6 Sent Done Done false
25273 7307 Acked Read 1 Sent Done Done false
25274 7308 Acked Write 4 Sent Done Done false
25275 7309 Acked Read 7 Sent Done Done false
25276 7310 Acked Read 6 Sent Done Done false
25277 7311 Acked Read 8 Sent Done Done false
25278 7312 Acked Write 10 Sent Done Done false
25279 7313 Acked Read 8 Sent Done Done false
25280 7314 Acked Write 10 Sent Done Done false
25281 7315 Acked Write 10 Sent Done Done false
25282 7316 Acked Read 7 Sent Done Done false
25283 7317 Acked Flush 0 Sent Done Done false
25284 7318 Acked Write 8 Sent Done Done false
25285 7319 Acked Write 6 Sent Done Done false
25286 7320 Acked Read 8 Sent Done Done false
25287 7321 Acked Write 9 Sent Done Done false
25288 7322 Acked Read 2 Sent Done Done false
25289 7323 Acked Read 9 Sent Done Done false
25290 7324 Acked Write 2 Sent Done Done false
25291 7325 Acked Write 4 Sent Done Done false
25292 7326 Acked Flush 0 Sent Done Done false
25293 7327 Acked Write 9 Sent Done Done false
25294 7328 Acked Write 2 Sent Done Done false
25295 7329 Acked Read 9 Sent Done Done false
25296 7330 Acked Read 7 Sent Done Done false
25297 7331 Acked Write 6 Sent Done Done false
25298 7332 Acked Read 3 Sent Done Done false
25299 7333 Acked Read 8 Sent Done Done false
25300 7334 Acked Write 1 Sent Done Done false
25301 7335 Acked Flush 0 Sent Done Done false
25302 7336 Acked Write 2 Sent Done Done false
25303 7337 Acked Write 4 Sent Done Done false
25304 7338 Acked Flush 0 Sent Done Done false
25305 STATES DS:0 DS:1 DS:2 TOTAL
25306 Sent 302 0 0 302
25307 Done 0 302 302 604
25308 Skipped 0 0 0 0
25309 Error 0 0 0 0
25310 Last Flush: 7036 7338 7338
25311 Downstairs last five completed: 7036 7035 7034 7033 7032
25312 Upstairs last five completed: 7338 7337 7336 7335 7334
25313 CLIENT: Up:0 ds:302 act:2
253142025-01-17T21:16:23.417ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
253152025-01-17T21:16:23.417ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
253162025-01-17T21:16:23.417ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
253172025-01-17T21:16:23.417ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
253182025-01-17T21:16:23.417ZINFOcrucible: client task is exiting = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
25319 [8810] got start action
25320 [8810] I am going to restart
25321 Starting downstairs at port 8810
25322 Make output file at "/tmp/dsc/downstairs-8810.txt"
25323 [8810][0] reports Starting
25324 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1221)
25325 [8810][0] reports Running
25326 ----------------------------------------------------------------
25327 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
25328 States: Offline Active Active
25329 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25330 7037 Acked Write 3 Sent Done Done false
25331 7038 Acked Write 9 Sent Done Done false
25332 7039 Acked Read 8 Sent Done Done false
25333 7040 Acked Write 10 Sent Done Done false
25334 7041 Acked Write 10 Sent Done Done false
25335 7042 Acked Write 8 Sent Done Done false
25336 7043 Acked Write 5 Sent Done Done false
25337 7044 Acked Read 7 Sent Done Done false
25338 7045 Acked Read 3 Sent Done Done false
25339 7046 Acked Write 1 Sent Done Done false
25340 7047 Acked Write 1 Sent Done Done false
25341 7048 Acked Flush 0 Sent Done Done false
25342 7049 Acked Read 1 Sent Done Done false
25343 7050 Acked Read 2 Sent Done Done false
25344 7051 Acked Write 10 Sent Done Done false
25345 7052 Acked Read 4 Sent Done Done false
25346 7053 Acked Write 9 Sent Done Done false
25347 7054 Acked Read 7 Sent Done Done false
25348 7055 Acked Write 2 Sent Done Done false
25349 7056 Acked Read 2 Sent Done Done false
25350 7057 Acked Read 3 Sent Done Done false
25351 7058 Acked Write 9 Sent Done Done false
25352 7059 Acked Read 1 Sent Done Done false
25353 7060 Acked Read 6 Sent Done Done false
25354 7061 Acked Read 5 Sent Done Done false
25355 7062 Acked Write 9 Sent Done Done false
25356 7063 Acked Read 7 Sent Done Done false
25357 7064 Acked Read 8 Sent Done Done false
25358 7065 Acked Write 1 Sent Done Done false
25359 7066 Acked Read 9 Sent Done Done false
25360 7067 Acked Read 6 Sent Done Done false
25361 7068 Acked Read 6 Sent Done Done false
25362 7069 Acked Read 5 Sent Done Done false
25363 7070 Acked Write 5 Sent Done Done false
25364 7071 Acked Write 4 Sent Done Done false
25365 7072 Acked Write 2 Sent Done Done false
25366 7073 Acked Read 10 Sent Done Done false
25367 7074 Acked Flush 0 Sent Done Done false
25368 7075 Acked Read 1 Sent Done Done false
25369 7076 Acked Write 1 Sent Done Done false
25370 7077 Acked Flush 0 Sent Done Done false
25371 7078 Acked Read 4 Sent Done Done false
25372 7079 Acked Read 8 Sent Done Done false
25373 7080 Acked Read 3 Sent Done Done false
25374 7081 Acked Write 1 Sent Done Done false
25375 7082 Acked Read 8 Sent Done Done false
25376 7083 Acked Write 2 Sent Done Done false
25377 7084 Acked Read 10 Sent Done Done false
25378 7085 Acked Read 3 Sent Done Done false
25379 7086 Acked Read 8 Sent Done Done false
25380 7087 Acked Write 4 Sent Done Done false
25381 7088 Acked Write 5 Sent Done Done false
25382 7089 Acked Write 4 Sent Done Done false
25383 7090 Acked Read 10 Sent Done Done false
25384 7091 Acked Write 2 Sent Done Done false
25385 7092 Acked Read 4 Sent Done Done false
25386 7093 Acked Write 5 Sent Done Done false
25387 7094 Acked Write 1 Sent Done Done false
25388 7095 Acked Write 9 Sent Done Done false
25389 7096 Acked Write 5 Sent Done Done false
25390 7097 Acked Write 6 Sent Done Done false
25391 7098 Acked Read 4 Sent Done Done false
25392 7099 Acked Read 4 Sent Done Done false
25393 7100 Acked Write 9 Sent Done Done false
25394 7101 Acked Read 5 Sent Done Done false
25395 7102 Acked Read 10 Sent Done Done false
25396 7103 Acked Write 9 Sent Done Done false
25397 7104 Acked Write 7 Sent Done Done false
25398 7105 Acked Read 1 Sent Done Done false
25399 7106 Acked Read 3 Sent Done Done false
25400 7107 Acked Read 9 Sent Done Done false
25401 7108 Acked Write 2 Sent Done Done false
25402 7109 Acked Read 5 Sent Done Done false
25403 7110 Acked Flush 0 Sent Done Done false
25404 7111 Acked Flush 0 Sent Done Done false
25405 7112 Acked Write 9 Sent Done Done false
25406 7113 Acked Read 3 Sent Done Done false
25407 7114 Acked Write 8 Sent Done Done false
25408 7115 Acked Flush 0 Sent Done Done false
25409 7116 Acked Write 1 Sent Done Done false
25410 7117 Acked Write 2 Sent Done Done false
25411 7118 Acked Write 7 Sent Done Done false
25412 7119 Acked Write 8 Sent Done Done false
25413 7120 Acked Read 3 Sent Done Done false
25414 7121 Acked Read 9 Sent Done Done false
25415 7122 Acked Flush 0 Sent Done Done false
25416 7123 Acked Write 7 Sent Done Done false
25417 7124 Acked Write 9 Sent Done Done false
25418 7125 Acked Write 5 Sent Done Done false
25419 7126 Acked Read 8 Sent Done Done false
25420 7127 Acked Read 9 Sent Done Done false
25421 7128 Acked Read 7 Sent Done Done false
25422 7129 Acked Write 2 Sent Done Done false
25423 7130 Acked Write 4 Sent Done Done false
25424 7131 Acked Write 6 Sent Done Done false
25425 7132 Acked Write 5 Sent Done Done false
25426 7133 Acked Read 9 Sent Done Done false
25427 7134 Acked Read 8 Sent Done Done false
25428 7135 Acked Read 9 Sent Done Done false
25429 7136 Acked Read 4 Sent Done Done false
25430 7137 Acked Read 8 Sent Done Done false
25431 7138 Acked Read 7 Sent Done Done false
25432 7139 Acked Read 4 Sent Done Done false
25433 7140 Acked Read 6 Sent Done Done false
25434 7141 Acked Write 6 Sent Done Done false
25435 7142 Acked Write 8 Sent Done Done false
25436 7143 Acked Read 6 Sent Done Done false
25437 7144 Acked Write 4 Sent Done Done false
25438 7145 Acked Read 7 Sent Done Done false
25439 7146 Acked Write 10 Sent Done Done false
25440 7147 Acked Flush 0 Sent Done Done false
25441 7148 Acked Read 10 Sent Done Done false
25442 7149 Acked Read 7 Sent Done Done false
25443 7150 Acked Write 1 Sent Done Done false
25444 7151 Acked Write 3 Sent Done Done false
25445 7152 Acked Write 1 Sent Done Done false
25446 7153 Acked Read 6 Sent Done Done false
25447 7154 Acked Read 8 Sent Done Done false
25448 7155 Acked Read 1 Sent Done Done false
25449 7156 Acked Read 5 Sent Done Done false
25450 7157 Acked Read 9 Sent Done Done false
25451 7158 Acked Read 6 Sent Done Done false
25452 7159 Acked Write 9 Sent Done Done false
25453 7160 Acked Read 7 Sent Done Done false
25454 7161 Acked Read 9 Sent Done Done false
25455 7162 Acked Write 10 Sent Done Done false
25456 7163 Acked Read 10 Sent Done Done false
25457 7164 Acked Flush 0 Sent Done Done false
25458 7165 Acked Read 9 Sent Done Done false
25459 7166 Acked Write 7 Sent Done Done false
25460 7167 Acked Read 7 Sent Done Done false
25461 7168 Acked Read 6 Sent Done Done false
25462 7169 Acked Write 9 Sent Done Done false
25463 7170 Acked Write 3 Sent Done Done false
25464 7171 Acked Read 4 Sent Done Done false
25465 7172 Acked Read 2 Sent Done Done false
25466 7173 Acked Write 1 Sent Done Done false
25467 7174 Acked Read 3 Sent Done Done false
25468 7175 Acked Write 5 Sent Done Done false
25469 7176 Acked Write 7 Sent Done Done false
25470 7177 Acked Read 4 Sent Done Done false
25471 7178 Acked Read 5 Sent Done Done false
25472 7179 Acked Write 1 Sent Done Done false
25473 7180 Acked Read 9 Sent Done Done false
25474 7181 Acked Read 6 Sent Done Done false
25475 7182 Acked Write 5 Sent Done Done false
25476 7183 Acked Read 8 Sent Done Done false
25477 7184 Acked Read 5 Sent Done Done false
25478 7185 Acked Read 7 Sent Done Done false
25479 7186 Acked Read 2 Sent Done Done false
25480 7187 Acked Read 6 Sent Done Done false
25481 7188 Acked Flush 0 Sent Done Done false
25482 7189 Acked Read 7 Sent Done Done false
25483 7190 Acked Read 7 Sent Done Done false
25484 7191 Acked Read 4 Sent Done Done false
25485 7192 Acked Write 1 Sent Done Done false
25486 7193 Acked Write 6 Sent Done Done false
25487 7194 Acked Write 7 Sent Done Done false
25488 7195 Acked Read 5 Sent Done Done false
25489 7196 Acked Read 8 Sent Done Done false
25490 7197 Acked Flush 0 Sent Done Done false
25491 7198 Acked Write 1 Sent Done Done false
25492 7199 Acked Flush 0 Sent Done Done false
25493 7200 Acked Read 1 Sent Done Done false
25494 7201 Acked Read 3 Sent Done Done false
25495 7202 Acked Write 2 Sent Done Done false
25496 7203 Acked Write 7 Sent Done Done false
25497 7204 Acked Write 1 Sent Done Done false
25498 7205 Acked Write 1 Sent Done Done false
25499 7206 Acked Read 3 Sent Done Done false
25500 7207 Acked Read 2 Sent Done Done false
25501 7208 Acked Read 3 Sent Done Done false
25502 7209 Acked Write 3 Sent Done Done false
25503 7210 Acked Read 4 Sent Done Done false
25504 7211 Acked Read 9 Sent Done Done false
25505 7212 Acked Read 2 Sent Done Done false
25506 7213 Acked Write 6 Sent Done Done false
25507 7214 Acked Read 6 Sent Done Done false
25508 7215 Acked Write 1 Sent Done Done false
25509 7216 Acked Read 3 Sent Done Done false
25510 7217 Acked Write 8 Sent Done Done false
25511 7218 Acked Read 6 Sent Done Done false
25512 7219 Acked Write 9 Sent Done Done false
25513 7220 Acked Write 3 Sent Done Done false
25514 7221 Acked Write 10 Sent Done Done false
25515 7222 Acked Read 5 Sent Done Done false
25516 7223 Acked Write 8 Sent Done Done false
25517 7224 Acked Write 5 Sent Done Done false
25518 7225 Acked Read 2 Sent Done Done false
25519 7226 Acked Write 2 Sent Done Done false
25520 7227 Acked Read 6 Sent Done Done false
25521 7228 Acked Read 8 Sent Done Done false
25522 7229 Acked Read 9 Sent Done Done false
25523 7230 Acked Read 1 Sent Done Done false
25524 7231 Acked Read 6 Sent Done Done false
25525 7232 Acked Write 7 Sent Done Done false
25526 7233 Acked Read 6 Sent Done Done false
25527 7234 Acked Write 1 Sent Done Done false
25528 7235 Acked Read 9 Sent Done Done false
25529 7236 Acked Write 6 Sent Done Done false
25530 7237 Acked Write 1 Sent Done Done false
25531 7238 Acked Write 7 Sent Done Done false
25532 7239 Acked Write 7 Sent Done Done false
25533 7240 Acked Write 1 Sent Done Done false
25534 7241 Acked Read 9 Sent Done Done false
25535 7242 Acked Write 7 Sent Done Done false
25536 7243 Acked Read 4 Sent Done Done false
25537 7244 Acked Write 2 Sent Done Done false
25538 7245 Acked Read 4 Sent Done Done false
25539 7246 Acked Read 1 Sent Done Done false
25540 7247 Acked Read 6 Sent Done Done false
25541 7248 Acked Read 1 Sent Done Done false
25542 7249 Acked Read 6 Sent Done Done false
25543 7250 Acked Write 9 Sent Done Done false
25544 7251 Acked Read 6 Sent Done Done false
25545 7252 Acked Write 8 Sent Done Done false
25546 7253 Acked Read 4 Sent Done Done false
25547 7254 Acked Read 5 Sent Done Done false
25548 7255 Acked Read 10 Sent Done Done false
25549 7256 Acked Read 5 Sent Done Done false
25550 7257 Acked Flush 0 Sent Done Done false
25551 7258 Acked Read 4 Sent Done Done false
25552 7259 Acked Write 3 Sent Done Done false
25553 7260 Acked Write 5 Sent Done Done false
25554 7261 Acked Write 6 Sent Done Done false
25555 7262 Acked Read 8 Sent Done Done false
25556 7263 Acked Write 10 Sent Done Done false
25557 7264 Acked Write 4 Sent Done Done false
25558 7265 Acked Write 7 Sent Done Done false
25559 7266 Acked Read 9 Sent Done Done false
25560 7267 Acked Write 5 Sent Done Done false
25561 7268 Acked Write 6 Sent Done Done false
25562 7269 Acked Write 10 Sent Done Done false
25563 7270 Acked Read 1 Sent Done Done false
25564 7271 Acked Read 1 Sent Done Done false
25565 7272 Acked Read 6 Sent Done Done false
25566 7273 Acked Read 7 Sent Done Done false
25567 7274 Acked Flush 0 Sent Done Done false
25568 7275 Acked Read 2 Sent Done Done false
25569 7276 Acked Flush 0 Sent Done Done false
25570 7277 Acked Write 9 Sent Done Done false
25571 7278 Acked Flush 0 Sent Done Done false
25572 7279 Acked Read 8 Sent Done Done false
25573 7280 Acked Read 1 Sent Done Done false
25574 7281 Acked Flush 0 Sent Done Done false
25575 7282 Acked Write 10 Sent Done Done false
25576 7283 Acked Read 10 Sent Done Done false
25577 7284 Acked Read 3 Sent Done Done false
25578 7285 Acked Write 1 Sent Done Done false
25579 7286 Acked Read 6 Sent Done Done false
25580 7287 Acked Read 10 Sent Done Done false
25581 7288 Acked Write 4 Sent Done Done false
25582 7289 Acked Flush 0 Sent Done Done false
25583 7290 Acked Read 1 Sent Done Done false
25584 7291 Acked Write 9 Sent Done Done false
25585 7292 Acked Write 2 Sent Done Done false
25586 7293 Acked Write 5 Sent Done Done false
25587 7294 Acked Read 7 Sent Done Done false
25588 7295 Acked Flush 0 Sent Done Done false
25589 7296 Acked Read 6 Sent Done Done false
25590 7297 Acked Flush 0 Sent Done Done false
25591 7298 Acked Read 3 Sent Done Done false
25592 7299 Acked Write 2 Sent Done Done false
25593 7300 Acked Read 2 Sent Done Done false
25594 7301 Acked Write 9 Sent Done Done false
25595 7302 Acked Read 5 Sent Done Done false
25596 7303 Acked Write 2 Sent Done Done false
25597 7304 Acked Write 2 Sent Done Done false
25598 7305 Acked Write 2 Sent Done Done false
25599 7306 Acked Write 6 Sent Done Done false
25600 7307 Acked Read 1 Sent Done Done false
25601 7308 Acked Write 4 Sent Done Done false
25602 7309 Acked Read 7 Sent Done Done false
25603 7310 Acked Read 6 Sent Done Done false
25604 7311 Acked Read 8 Sent Done Done false
25605 7312 Acked Write 10 Sent Done Done false
25606 7313 Acked Read 8 Sent Done Done false
25607 7314 Acked Write 10 Sent Done Done false
25608 7315 Acked Write 10 Sent Done Done false
25609 7316 Acked Read 7 Sent Done Done false
25610 7317 Acked Flush 0 Sent Done Done false
25611 7318 Acked Write 8 Sent Done Done false
25612 7319 Acked Write 6 Sent Done Done false
25613 7320 Acked Read 8 Sent Done Done false
25614 7321 Acked Write 9 Sent Done Done false
25615 7322 Acked Read 2 Sent Done Done false
25616 7323 Acked Read 9 Sent Done Done false
25617 7324 Acked Write 2 Sent Done Done false
25618 7325 Acked Write 4 Sent Done Done false
25619 7326 Acked Flush 0 Sent Done Done false
25620 7327 Acked Write 9 Sent Done Done false
25621 7328 Acked Write 2 Sent Done Done false
25622 7329 Acked Read 9 Sent Done Done false
25623 7330 Acked Read 7 Sent Done Done false
25624 7331 Acked Write 6 Sent Done Done false
25625 7332 Acked Read 3 Sent Done Done false
25626 7333 Acked Read 8 Sent Done Done false
25627 7334 Acked Write 1 Sent Done Done false
25628 7335 Acked Flush 0 Sent Done Done false
25629 7336 Acked Write 2 Sent Done Done false
25630 7337 Acked Write 4 Sent Done Done false
25631 7338 Acked Flush 0 Sent Done Done false
25632 STATES DS:0 DS:1 DS:2 TOTAL
25633 Sent 302 0 0 302
25634 Done 0 302 302 604
25635 Skipped 0 0 0 0
25636 Error 0 0 0 0
25637 Last Flush: 7036 7338 7338
25638 Downstairs last five completed: 7036 7035 7034 7033 7032
25639 Upstairs last five completed: 7338 7337 7336 7335 7334
25640 CLIENT: Up:0 ds:302 act:2
25641 ----------------------------------------------------------------
25642 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
25643 States: Offline Active Active
25644 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25645 7037 Acked Write 3 Sent Done Done false
25646 7038 Acked Write 9 Sent Done Done false
25647 7039 Acked Read 8 Sent Done Done false
25648 7040 Acked Write 10 Sent Done Done false
25649 7041 Acked Write 10 Sent Done Done false
25650 7042 Acked Write 8 Sent Done Done false
25651 7043 Acked Write 5 Sent Done Done false
25652 7044 Acked Read 7 Sent Done Done false
25653 7045 Acked Read 3 Sent Done Done false
25654 7046 Acked Write 1 Sent Done Done false
25655 7047 Acked Write 1 Sent Done Done false
25656 7048 Acked Flush 0 Sent Done Done false
25657 7049 Acked Read 1 Sent Done Done false
25658 7050 Acked Read 2 Sent Done Done false
25659 7051 Acked Write 10 Sent Done Done false
25660 7052 Acked Read 4 Sent Done Done false
25661 7053 Acked Write 9 Sent Done Done false
25662 7054 Acked Read 7 Sent Done Done false
25663 7055 Acked Write 2 Sent Done Done false
25664 7056 Acked Read 2 Sent Done Done false
25665 7057 Acked Read 3 Sent Done Done false
25666 7058 Acked Write 9 Sent Done Done false
25667 7059 Acked Read 1 Sent Done Done false
25668 7060 Acked Read 6 Sent Done Done false
25669 7061 Acked Read 5 Sent Done Done false
25670 7062 Acked Write 9 Sent Done Done false
25671 7063 Acked Read 7 Sent Done Done false
25672 7064 Acked Read 8 Sent Done Done false
25673 7065 Acked Write 1 Sent Done Done false
25674 7066 Acked Read 9 Sent Done Done false
25675 7067 Acked Read 6 Sent Done Done false
25676 7068 Acked Read 6 Sent Done Done false
25677 7069 Acked Read 5 Sent Done Done false
25678 7070 Acked Write 5 Sent Done Done false
25679 7071 Acked Write 4 Sent Done Done false
25680 7072 Acked Write 2 Sent Done Done false
25681 7073 Acked Read 10 Sent Done Done false
25682 7074 Acked Flush 0 Sent Done Done false
25683 7075 Acked Read 1 Sent Done Done false
25684 7076 Acked Write 1 Sent Done Done false
25685 7077 Acked Flush 0 Sent Done Done false
25686 7078 Acked Read 4 Sent Done Done false
25687 7079 Acked Read 8 Sent Done Done false
25688 7080 Acked Read 3 Sent Done Done false
25689 7081 Acked Write 1 Sent Done Done false
25690 7082 Acked Read 8 Sent Done Done false
25691 7083 Acked Write 2 Sent Done Done false
25692 7084 Acked Read 10 Sent Done Done false
25693 7085 Acked Read 3 Sent Done Done false
25694 7086 Acked Read 8 Sent Done Done false
25695 7087 Acked Write 4 Sent Done Done false
25696 7088 Acked Write 5 Sent Done Done false
25697 7089 Acked Write 4 Sent Done Done false
25698 7090 Acked Read 10 Sent Done Done false
25699 7091 Acked Write 2 Sent Done Done false
25700 7092 Acked Read 4 Sent Done Done false
25701 7093 Acked Write 5 Sent Done Done false
25702 7094 Acked Write 1 Sent Done Done false
25703 7095 Acked Write 9 Sent Done Done false
25704 7096 Acked Write 5 Sent Done Done false
25705 7097 Acked Write 6 Sent Done Done false
25706 7098 Acked Read 4 Sent Done Done false
25707 7099 Acked Read 4 Sent Done Done false
25708 7100 Acked Write 9 Sent Done Done false
25709 7101 Acked Read 5 Sent Done Done false
25710 7102 Acked Read 10 Sent Done Done false
25711 7103 Acked Write 9 Sent Done Done false
25712 7104 Acked Write 7 Sent Done Done false
25713 7105 Acked Read 1 Sent Done Done false
25714 7106 Acked Read 3 Sent Done Done false
25715 7107 Acked Read 9 Sent Done Done false
25716 7108 Acked Write 2 Sent Done Done false
25717 7109 Acked Read 5 Sent Done Done false
25718 7110 Acked Flush 0 Sent Done Done false
25719 7111 Acked Flush 0 Sent Done Done false
25720 7112 Acked Write 9 Sent Done Done false
25721 7113 Acked Read 3 Sent Done Done false
25722 7114 Acked Write 8 Sent Done Done false
25723 7115 Acked Flush 0 Sent Done Done false
25724 7116 Acked Write 1 Sent Done Done false
25725 7117 Acked Write 2 Sent Done Done false
25726 7118 Acked Write 7 Sent Done Done false
25727 7119 Acked Write 8 Sent Done Done false
25728 7120 Acked Read 3 Sent Done Done false
25729 7121 Acked Read 9 Sent Done Done false
25730 7122 Acked Flush 0 Sent Done Done false
25731 7123 Acked Write 7 Sent Done Done false
25732 7124 Acked Write 9 Sent Done Done false
25733 7125 Acked Write 5 Sent Done Done false
25734 7126 Acked Read 8 Sent Done Done false
25735 7127 Acked Read 9 Sent Done Done false
25736 7128 Acked Read 7 Sent Done Done false
25737 7129 Acked Write 2 Sent Done Done false
25738 7130 Acked Write 4 Sent Done Done false
25739 7131 Acked Write 6 Sent Done Done false
25740 7132 Acked Write 5 Sent Done Done false
25741 7133 Acked Read 9 Sent Done Done false
25742 7134 Acked Read 8 Sent Done Done false
25743 7135 Acked Read 9 Sent Done Done false
25744 7136 Acked Read 4 Sent Done Done false
25745 7137 Acked Read 8 Sent Done Done false
25746 7138 Acked Read 7 Sent Done Done false
25747 7139 Acked Read 4 Sent Done Done false
25748 7140 Acked Read 6 Sent Done Done false
25749 7141 Acked Write 6 Sent Done Done false
25750 7142 Acked Write 8 Sent Done Done false
25751 7143 Acked Read 6 Sent Done Done false
25752 7144 Acked Write 4 Sent Done Done false
25753 7145 Acked Read 7 Sent Done Done false
25754 7146 Acked Write 10 Sent Done Done false
25755 7147 Acked Flush 0 Sent Done Done false
25756 7148 Acked Read 10 Sent Done Done false
25757 7149 Acked Read 7 Sent Done Done false
25758 7150 Acked Write 1 Sent Done Done false
25759 7151 Acked Write 3 Sent Done Done false
25760 7152 Acked Write 1 Sent Done Done false
25761 7153 Acked Read 6 Sent Done Done false
25762 7154 Acked Read 8 Sent Done Done false
25763 7155 Acked Read 1 Sent Done Done false
25764 7156 Acked Read 5 Sent Done Done false
25765 7157 Acked Read 9 Sent Done Done false
25766 7158 Acked Read 6 Sent Done Done false
25767 7159 Acked Write 9 Sent Done Done false
25768 7160 Acked Read 7 Sent Done Done false
25769 7161 Acked Read 9 Sent Done Done false
25770 7162 Acked Write 10 Sent Done Done false
25771 7163 Acked Read 10 Sent Done Done false
25772 7164 Acked Flush 0 Sent Done Done false
25773 7165 Acked Read 9 Sent Done Done false
25774 7166 Acked Write 7 Sent Done Done false
25775 7167 Acked Read 7 Sent Done Done false
25776 7168 Acked Read 6 Sent Done Done false
25777 7169 Acked Write 9 Sent Done Done false
25778 7170 Acked Write 3 Sent Done Done false
25779 7171 Acked Read 4 Sent Done Done false
25780 7172 Acked Read 2 Sent Done Done false
25781 7173 Acked Write 1 Sent Done Done false
25782 7174 Acked Read 3 Sent Done Done false
25783 7175 Acked Write 5 Sent Done Done false
25784 7176 Acked Write 7 Sent Done Done false
25785 7177 Acked Read 4 Sent Done Done false
25786 7178 Acked Read 5 Sent Done Done false
25787 7179 Acked Write 1 Sent Done Done false
25788 7180 Acked Read 9 Sent Done Done false
25789 7181 Acked Read 6 Sent Done Done false
25790 7182 Acked Write 5 Sent Done Done false
25791 7183 Acked Read 8 Sent Done Done false
25792 7184 Acked Read 5 Sent Done Done false
25793 7185 Acked Read 7 Sent Done Done false
25794 7186 Acked Read 2 Sent Done Done false
25795 7187 Acked Read 6 Sent Done Done false
25796 7188 Acked Flush 0 Sent Done Done false
25797 7189 Acked Read 7 Sent Done Done false
25798 7190 Acked Read 7 Sent Done Done false
25799 7191 Acked Read 4 Sent Done Done false
25800 7192 Acked Write 1 Sent Done Done false
25801 7193 Acked Write 6 Sent Done Done false
25802 7194 Acked Write 7 Sent Done Done false
25803 7195 Acked Read 5 Sent Done Done false
25804 7196 Acked Read 8 Sent Done Done false
25805 7197 Acked Flush 0 Sent Done Done false
25806 7198 Acked Write 1 Sent Done Done false
25807 7199 Acked Flush 0 Sent Done Done false
25808 7200 Acked Read 1 Sent Done Done false
25809 7201 Acked Read 3 Sent Done Done false
25810 7202 Acked Write 2 Sent Done Done false
25811 7203 Acked Write 7 Sent Done Done false
25812 7204 Acked Write 1 Sent Done Done false
25813 7205 Acked Write 1 Sent Done Done false
25814 7206 Acked Read 3 Sent Done Done false
25815 7207 Acked Read 2 Sent Done Done false
25816 7208 Acked Read 3 Sent Done Done false
25817 7209 Acked Write 3 Sent Done Done false
25818 7210 Acked Read 4 Sent Done Done false
25819 7211 Acked Read 9 Sent Done Done false
25820 7212 Acked Read 2 Sent Done Done false
25821 7213 Acked Write 6 Sent Done Done false
25822 7214 Acked Read 6 Sent Done Done false
25823 7215 Acked Write 1 Sent Done Done false
25824 7216 Acked Read 3 Sent Done Done false
25825 7217 Acked Write 8 Sent Done Done false
25826 7218 Acked Read 6 Sent Done Done false
25827 7219 Acked Write 9 Sent Done Done false
25828 7220 Acked Write 3 Sent Done Done false
25829 7221 Acked Write 10 Sent Done Done false
25830 7222 Acked Read 5 Sent Done Done false
25831 7223 Acked Write 8 Sent Done Done false
25832 7224 Acked Write 5 Sent Done Done false
25833 7225 Acked Read 2 Sent Done Done false
25834 7226 Acked Write 2 Sent Done Done false
25835 7227 Acked Read 6 Sent Done Done false
25836 7228 Acked Read 8 Sent Done Done false
25837 7229 Acked Read 9 Sent Done Done false
25838 7230 Acked Read 1 Sent Done Done false
25839 7231 Acked Read 6 Sent Done Done false
25840 7232 Acked Write 7 Sent Done Done false
25841 7233 Acked Read 6 Sent Done Done false
25842 7234 Acked Write 1 Sent Done Done false
25843 7235 Acked Read 9 Sent Done Done false
25844 7236 Acked Write 6 Sent Done Done false
25845 7237 Acked Write 1 Sent Done Done false
25846 7238 Acked Write 7 Sent Done Done false
25847 7239 Acked Write 7 Sent Done Done false
25848 7240 Acked Write 1 Sent Done Done false
25849 7241 Acked Read 9 Sent Done Done false
25850 7242 Acked Write 7 Sent Done Done false
25851 7243 Acked Read 4 Sent Done Done false
25852 7244 Acked Write 2 Sent Done Done false
25853 7245 Acked Read 4 Sent Done Done false
25854 7246 Acked Read 1 Sent Done Done false
25855 7247 Acked Read 6 Sent Done Done false
25856 7248 Acked Read 1 Sent Done Done false
25857 7249 Acked Read 6 Sent Done Done false
25858 7250 Acked Write 9 Sent Done Done false
25859 7251 Acked Read 6 Sent Done Done false
25860 7252 Acked Write 8 Sent Done Done false
25861 7253 Acked Read 4 Sent Done Done false
25862 7254 Acked Read 5 Sent Done Done false
25863 7255 Acked Read 10 Sent Done Done false
25864 7256 Acked Read 5 Sent Done Done false
25865 7257 Acked Flush 0 Sent Done Done false
25866 7258 Acked Read 4 Sent Done Done false
25867 7259 Acked Write 3 Sent Done Done false
25868 7260 Acked Write 5 Sent Done Done false
25869 7261 Acked Write 6 Sent Done Done false
25870 7262 Acked Read 8 Sent Done Done false
25871 7263 Acked Write 10 Sent Done Done false
25872 7264 Acked Write 4 Sent Done Done false
25873 7265 Acked Write 7 Sent Done Done false
25874 7266 Acked Read 9 Sent Done Done false
25875 7267 Acked Write 5 Sent Done Done false
25876 7268 Acked Write 6 Sent Done Done false
25877 7269 Acked Write 10 Sent Done Done false
25878 7270 Acked Read 1 Sent Done Done false
25879 7271 Acked Read 1 Sent Done Done false
25880 7272 Acked Read 6 Sent Done Done false
25881 7273 Acked Read 7 Sent Done Done false
25882 7274 Acked Flush 0 Sent Done Done false
25883 7275 Acked Read 2 Sent Done Done false
25884 7276 Acked Flush 0 Sent Done Done false
25885 7277 Acked Write 9 Sent Done Done false
25886 7278 Acked Flush 0 Sent Done Done false
25887 7279 Acked Read 8 Sent Done Done false
25888 7280 Acked Read 1 Sent Done Done false
25889 7281 Acked Flush 0 Sent Done Done false
25890 7282 Acked Write 10 Sent Done Done false
25891 7283 Acked Read 10 Sent Done Done false
25892 7284 Acked Read 3 Sent Done Done false
25893 7285 Acked Write 1 Sent Done Done false
25894 7286 Acked Read 6 Sent Done Done false
25895 7287 Acked Read 10 Sent Done Done false
25896 7288 Acked Write 4 Sent Done Done false
25897 7289 Acked Flush 0 Sent Done Done false
25898 7290 Acked Read 1 Sent Done Done false
25899 7291 Acked Write 9 Sent Done Done false
25900 7292 Acked Write 2 Sent Done Done false
25901 7293 Acked Write 5 Sent Done Done false
25902 7294 Acked Read 7 Sent Done Done false
25903 7295 Acked Flush 0 Sent Done Done false
25904 7296 Acked Read 6 Sent Done Done false
25905 7297 Acked Flush 0 Sent Done Done false
25906 7298 Acked Read 3 Sent Done Done false
25907 7299 Acked Write 2 Sent Done Done false
25908 7300 Acked Read 2 Sent Done Done false
25909 7301 Acked Write 9 Sent Done Done false
25910 7302 Acked Read 5 Sent Done Done false
25911 7303 Acked Write 2 Sent Done Done false
25912 7304 Acked Write 2 Sent Done Done false
25913 7305 Acked Write 2 Sent Done Done false
25914 7306 Acked Write 6 Sent Done Done false
25915 7307 Acked Read 1 Sent Done Done false
25916 7308 Acked Write 4 Sent Done Done false
25917 7309 Acked Read 7 Sent Done Done false
25918 7310 Acked Read 6 Sent Done Done false
25919 7311 Acked Read 8 Sent Done Done false
25920 7312 Acked Write 10 Sent Done Done false
25921 7313 Acked Read 8 Sent Done Done false
25922 7314 Acked Write 10 Sent Done Done false
25923 7315 Acked Write 10 Sent Done Done false
25924 7316 Acked Read 7 Sent Done Done false
25925 7317 Acked Flush 0 Sent Done Done false
25926 7318 Acked Write 8 Sent Done Done false
25927 7319 Acked Write 6 Sent Done Done false
25928 7320 Acked Read 8 Sent Done Done false
25929 7321 Acked Write 9 Sent Done Done false
25930 7322 Acked Read 2 Sent Done Done false
25931 7323 Acked Read 9 Sent Done Done false
25932 7324 Acked Write 2 Sent Done Done false
25933 7325 Acked Write 4 Sent Done Done false
25934 7326 Acked Flush 0 Sent Done Done false
25935 7327 Acked Write 9 Sent Done Done false
25936 7328 Acked Write 2 Sent Done Done false
25937 7329 Acked Read 9 Sent Done Done false
25938 7330 Acked Read 7 Sent Done Done false
25939 7331 Acked Write 6 Sent Done Done false
25940 7332 Acked Read 3 Sent Done Done false
25941 7333 Acked Read 8 Sent Done Done false
25942 7334 Acked Write 1 Sent Done Done false
25943 7335 Acked Flush 0 Sent Done Done false
25944 7336 Acked Write 2 Sent Done Done false
25945 7337 Acked Write 4 Sent Done Done false
25946 7338 Acked Flush 0 Sent Done Done false
25947 STATES DS:0 DS:1 DS:2 TOTAL
25948 Sent 302 0 0 302
25949 Done 0 302 302 604
25950 Skipped 0 0 0 0
25951 Error 0 0 0 0
25952 Last Flush: 7036 7338 7338
25953 Downstairs last five completed: 7036 7035 7034 7033 7032
25954 Upstairs last five completed: 7338 7337 7336 7335 7334
25955 CLIENT: Up:0 ds:302 act:2
259562025-01-17T21:16:33.419ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
259572025-01-17T21:16:33.419ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62111) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
259582025-01-17T21:16:33.420ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
259592025-01-17T21:16:33.421ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
259602025-01-17T21:16:33.421ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
259612025-01-17T21:16:33.421ZINFOcrucible: send last flush ID to this DS: 7036 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
259622025-01-17T21:16:33.421ZINFOcrucible: Replied this last flush ID: 7036 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
259632025-01-17T21:16:33.421ZINFOcrucible: [0] client re-new 302 jobs since flush 7036 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
259642025-01-17T21:16:33.421ZINFOcrucible: [0] Replayed 302 jobs since flush: 7036 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
25965 ----------------------------------------------------------------
25966 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
25967 Downstairs last five completed: 7338 7337 7336 7335 7334
25968 Upstairs last five completed: 7338 7337 7336 7335 7334
25969 CLIENT: Up:0 ds:0 act:3
25970 Replay: All jobs finished
25971 Main task has work to do, go find it
25972 got dsc Stop(2)
25973 stop 2
25974 Jan 17 21:16:33.664 INFO [8830] Got stop action so:false kr:false
25975 request completed, latency_us: 342, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 67268ed7-b159-45ba-be32-6ec253afb7c1, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
259762025-01-17T21:16:33.665ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
259772025-01-17T21:16:33.665ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
259782025-01-17T21:16:33.665ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
25979 Jan 17 21:16:33.665 INFO request completed, latency_us: 112, response_code: 200, uri: /state/cid/2, method: GET, req_id: b3599ade-2be4-4cef-aa76-f55b057a01e8, remote_addr: 127.0.0.1:57191, local_addr: 127{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-17T21:16:33.665540586Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
25980 .0.0.1:9998
25981 [8830] Exited with: ExitStatus(unix_wait_status(9))
25982 [8830][2] reports Exit
25983 Jan 17 21:16:37.668 INFO request completed, latency_us: 217, response_code: 200, uri: /state/cid/2, method: GET, req_id: 7457a1b3-1c63-44b2-a528-9a46382a3917, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
25984 001/300 Read block 131 len 36864
25985 002/300 Read block 2229 len 24576
25986 003/300 Flush
25987 004/300 Write block 3340 len 24576 data: 5 4 3 3 3 3
25988 005/300 Read block 1779 len 4096
25989 006/300 Write block 765 len 24576 data: 7 7 3 3 3 4
25990 007/300 Write block 4072 len 8192 data: 2 3
25991 008/300 Read block 1807 len 16384
25992 009/300 Write block 3886 len 16384 data: 4 3 3 5
25993 010/300 Write block 2144 len 36864 data: 6 5 5 5 5 6 5 5 6
25994 011/300 Read block 4745 len 40960
25995 012/300 Flush
25996 013/300 Write block 3375 len 32768 data: 5 5 5 4 4 4 4 4
25997 014/300 Read block 843 len 36864
25998 015/300 Read block 3529 len 24576
25999 016/300 Read block 2427 len 28672
26000 017/300 Read block 71 len 24576
26001 018/300 Write block 2895 len 36864 data: 4 4 5 4 4 6 7 7 8
26002 019/300 Flush
26003 020/300 Write block 793 len 8192 data: 5 5
26004 021/300 Read block 3794 len 28672
26005 022/300 Read block 2846 len 40960
26006 023/300 Write block 341 len 36864 data: 5 7 6 5 5 5 6 7 5
26007 024/300 Read block 1635 len 32768
26008 025/300 Write block 1720 len 40960 data: 4 5 3 4 4 5 5 5 4 4
26009 026/300 Write block 1758 len 4096 data: 5
26010 027/300 Read block 4352 len 28672
26011 028/300 Read block 33 len 12288
26012 029/300 Read block 1261 len 8192
26013 030/300 Read block 530 len 20480
26014 031/300 Read block 515 len 28672
26015 032/300 Read block 799 len 16384
26016 033/300 Read block 2135 len 8192
26017 034/300 Write block 2854 len 28672 data: 2 2 3 3 3 5 5
26018 035/300 Read block 907 len 28672
26019 036/300 Read block 1686 len 4096
26020 037/300 Write block 1384 len 24576 data: 6 6 5 5 4 4
26021 038/300 Read block 4972 len 32768
26022 039/300 Write block 4843 len 40960 data: 5 6 7 7 6 7 7 6 4 4
26023 040/300 Write block 4847 len 36864 data: 7 8 8 7 5 5 3 4 3
26024 041/300 Read block 3078 len 8192
26025 042/300 Read block 667 len 4096
26026 043/300 Read block 2126 len 4096
26027 044/300 Read block 2667 len 40960
26028 045/300 Write block 1070 len 20480 data: 5 5 4 4 4
26029 046/300 Read block 3695 len 8192
26030 047/300 Read block 1907 len 28672
26031 048/300 Write block 1922 len 12288 data: 7 6 6
26032 049/300 Read block 4195 len 36864
26033 050/300 Read block 1894 len 8192
26034 051/300 Write block 1835 len 20480 data: 5 4 4 4 4
26035 052/300 Read block 313 len 20480
26036 053/300 Read block 4710 len 28672
26037 054/300 Write block 2604 len 8192 data: 6 5
26038 055/300 Write block 4145 len 36864 data: 5 6 4 3 3 3 3 4 5
26039 056/300 Read block 3242 len 24576
26040 057/300 Read block 2420 len 8192
26041 058/300 Read block 4626 len 4096
26042 059/300 Write block 3409 len 24576 data: 5 4 4 5 5 5
26043 060/300 Read block 3212 len 20480
26044 061/300 Write block 610 len 8192 data: 3 4
26045 062/300 Read block 3145 len 12288
26046 063/300 Write block 57 len 24576 data: 2 2 3 3 4 4
26047 064/300 Read block 3410 len 32768
26048 065/300 Write block 2524 len 20480 data: 3 3 4 3 3
26049 066/300 Read block 4014 len 36864
26050 067/300 Write block 3620 len 16384 data: 6 7 6 6
26051 068/300 Flush
26052 069/300 Write block 2929 len 24576 data: 7 5 4 3 3 3
26053 070/300 Write block 2059 len 24576 data: 5 7 6 7 8 8
26054 071/300 Flush
26055 072/300 Write block 1938 len 12288 data: 3 3 3
26056 073/300 Write block 460 len 12288 data: 3 2 4
26057 074/300 Write block 2884 len 20480 data: 4 5 6 7 7
26058 075/300 Flush
26059 076/300 Flush
26060 077/300 Read block 3939 len 16384
26061 078/300 Read block 4400 len 40960
26062 079/300 Read block 1101 len 24576
26063 080/300 Write block 4892 len 16384 data: 6 5 4 4
26064 081/300 Read block 1623 len 32768
26065 082/300 Read block 1924 len 8192
26066 083/300 Write block 3102 len 24576 data: 7 8 10 10 8 8
26067 084/300 Read block 577 len 32768
26068 085/300 Read block 3785 len 32768
26069 086/300 Flush
26070 087/300 Read block 2563 len 28672
26071 088/300 Read block 1098 len 36864
26072 089/300 Read block 1784 len 40960
26073 090/300 Read block 3559 len 12288
26074 091/300 Write block 4764 len 4096 data: 10
26075 092/300 Write block 942 len 20480 data: 4 5 5 5 5
26076 093/300 Read block 1618 len 20480
26077 094/300 Write block 2669 len 32768 data: 4 5 5 5 5 5 5 5
26078 095/300 Read block 3183 len 32768
26079 096/300 Read block 3620 len 12288
26080 097/300 Read block 1625 len 16384
26081 098/300 Read block 4889 len 24576
26082 099/300 Read block 745 len 12288
26083 100/300 Write block 1827 len 8192 data: 6 5
26084 101/300 Flush
26085 102/300 Read block 2932 len 28672
26086 103/300 Write block 1586 len 36864 data: 9 8 7 8 7 8 7 5 6
26087 104/300 Read block 3768 len 12288
26088 105/300 Flush
26089 106/300 Read block 1721 len 16384
26090 107/300 Write block 3078 len 8192 data: 6 7
26091 108/300 Write block 4560 len 20480 data: 6 5 3 3 4
26092 109/300 Write block 858 len 40960 data: 4 4 6 6 7 6 5 3 2 3
26093 110/300 Write block 2147 len 40960 data: 6 6 7 6 6 7 5 5 6 6
26094 111/300 Read block 4295 len 36864
26095 112/300 Read block 3891 len 24576
26096 113/300 Flush
26097 114/300 Write block 3326 len 36864 data: 2 2 3 3 4 4 4 5 5
26098 115/300 Write block 50 len 36864 data: 4 5 5 6 4 3 2 3 3
26099 116/300 Write block 3129 len 36864 data: 3 3 4 3 5 4 4 5 5
26100 117/300 Read block 2129 len 8192
26101 118/300 Write block 1726 len 32768 data: 6 6 5 5 5 6 4 5
26102 119/300 Write block 4886 len 24576 data: 4 5 5 5 6 6
26103 120/300 Read block 23 len 20480
26104 121/300 Read block 248 len 16384
26105 122/300 Write block 3500 len 16384 data: 5 5 5 5
26106 123/300 Write block 1262 len 28672 data: 7 5 5 5 4 5 7
26107 124/300 Flush
26108 125/300 Read block 542 len 20480
26109 126/300 Read block 2133 len 28672
26110 127/300 Flush
26111 128/300 Write block 3944 len 24576 data: 2 2 3 3 4 4
26112 129/300 Read block 1306 len 28672
26113 130/300 Write block 3810 len 20480 data: 5 5 5 5 5
26114 131/300 Read block 891 len 24576
26115 132/300 Read block 2804 len 32768
26116 133/300 Write block 2019 len 28672 data: 4 4 4 5 5 5 5
26117 134/300 Write block 642 len 36864 data: 3 4 4 3 3 4 3 4 5
26118 135/300 Read block 3025 len 20480
26119 136/300 Read block 2495 len 24576
26120 137/300 Write block 3348 len 12288 data: 5 4 3
26121 138/300 Write block 642 len 4096 data: 4
26122 139/300 Write block 4385 len 28672 data: 3 3 3 3 2 2 3
26123 140/300 Read block 249 len 40960
26124 141/300 Write block 3573 len 16384 data: 4 6 6 5
26125 142/300 Write block 2664 len 8192 data: 6 5
26126 143/300 Read block 627 len 20480
26127 144/300 Write block 3117 len 12288 data: 4 5 5
26128 145/300 Read block 2200 len 12288
26129 146/300 Read block 826 len 24576
26130 147/300 Write block 3308 len 24576 data: 3 3 4 3 3 5
26131 148/300 Flush
26132 149/300 Write block 619 len 24576 data: 5 4 4 5 5 5
26133 150/300 Read block 2344 len 40960
26134 151/300 Write block 1872 len 4096 data: 3
26135 152/300 Write block 2224 len 16384 data: 6 4 2 4
26136 153/300 Read block 2652 len 8192
26137 154/300 Read block 1716 len 12288
26138 155/300 Read block 2674 len 16384
26139 156/300 Write block 1982 len 28672 data: 5 5 5 6 5 5 5
26140 157/300 Read block 475 len 40960
26141 158/300 Write block 2008 len 24576 data: 3 3 4 5 4 4
26142 159/300 Read block 1531 len 16384
26143 160/300 Read block 596 len 20480
26144 161/300 Read block 3161 len 16384
26145 162/300 Read block 4889 len 8192
26146 163/300 Write block 1196 len 20480 data: 7 6 6 6 4
26147 164/300 Read block 3107 len 28672
26148 165/300 Write block 3147 len 8192 data: 3 4
26149 166/300 Read block 1006 len 36864
26150 167/300 Read block 23 len 12288
26151 168/300 Read block 3853 len 4096
26152 169/300 Flush
26153 170/300 Read block 2775 len 24576
26154 171/300 Write block 2981 len 12288 data: 4 4 4
26155 172/300 Write block 699 len 36864 data: 7 5 5 5 6 4 3 4 5
26156 173/300 Flush
26157 174/300 Read block 3366 len 24576
26158 175/300 Write block 465 len 28672 data: 3 3 3 3 4 4 4
26159 176/300 Read block 1251 len 28672
26160 177/300 Write block 3072 len 4096 data: 5
26161 178/300 Read block 2676 len 20480
26162 179/300 Write block 2972 len 16384 data: 4 4 4 4
26163 180/300 Write block 2656 len 8192 data: 5 5
26164 181/300 Write block 3049 len 8192 data: 4 3
26165 182/300 Read block 2237 len 20480
26166 183/300 Write block 678 len 32768 data: 2 2 3 3 4 6 7 7
26167 184/300 Read block 2620 len 20480
26168 185/300 Write block 587 len 12288 data: 7 6 7
26169 186/300 Read block 259 len 32768
26170 187/300 Read block 1754 len 20480
26171 188/300 Write block 3328 len 12288 data: 4 4 5
26172 189/300 Write block 4499 len 20480 data: 3 3 4 5 4
26173 190/300 Flush
26174 191/300 Flush
26175 192/300 Read block 408 len 36864
26176 193/300 Write block 2801 len 8192 data: 4 4
26177 194/300 Write block 2074 len 24576 data: 7 8 5 4 4 4
26178 195/300 Read block 3001 len 8192
26179 196/300 Read block 4447 len 20480
26180 197/300 Read block 4689 len 24576
26181 198/300 Flush
26182 199/300 Read block 2401 len 28672
26183 200/300 Read block 1179 len 24576
26184 201/300 Read block 117 len 4096
26185 202/300 Read block 765 len 12288
26186 203/300 Read block 4596 len 4096
26187 204/300 Write block 749 len 20480 data: 5 4 3 3 2
26188 205/300 Write block 3945 len 28672 data: 3 4 4 5 5 3 4
26189 206/300 Flush
26190 207/300 Write block 4856 len 40960 data: 4 4 4 5 6 5 5 5 3 3
26191 208/300 Read block 4734 len 40960
26192 209/300 Read block 4846 len 28672
26193 210/300 Read block 2814 len 36864
26194 211/300 Write block 4823 len 8192 data: 5 6
26195 212/300 Write block 1509 len 40960 data: 4 2 3 2 2 2 3 4 6 7
26196 213/300 Read block 831 len 28672
26197 214/300 Write block 2137 len 40960 data: 2 2 2 3 5 6 6 7 6 6
26198 215/300 Read block 4279 len 24576
26199 216/300 Write block 4707 len 28672 data: 4 4 3 3 4 5 4
26200 217/300 Write block 1309 len 20480 data: 5 4 4 4 5
26201 218/300 Write block 4053 len 32768 data: 2 4 5 5 6 6 5 6
26202 219/300 Read block 1878 len 32768
26203 220/300 Write block 2610 len 32768 data: 2 2 2 2 3 3 3 3
26204 221/300 Write block 4860 len 32768 data: 7 6 6 6 4 4 4 3
26205 222/300 Read block 1570 len 28672
26206 223/300 Read block 2344 len 24576
26207 224/300 Write block 4714 len 36864 data: 5 5 5 6 4 4 4 4 4
26208 225/300 Read block 1471 len 12288
26209 226/300 Read block 2922 len 36864
26210 227/300 Read block 1531 len 8192
26211 228/300 Read block 1041 len 20480
26212 229/300 Write block 4829 len 4096 data: 7
26213 230/300 Read block 670 len 20480
26214 231/300 Read block 1603 len 16384
26215 232/300 Write block 735 len 28672 data: 5 5 5 6 5 5 6
26216 233/300 Write block 3301 len 12288 data: 7 6 7
26217 234/300 Read block 3616 len 36864
26218 235/300 Write block 2132 len 40960 data: 3 3 2 2 3 3 3 3 4 6
26219 236/300 Read block 822 len 8192
26220 237/300 Read block 2008 len 12288
26221 238/300 Write block 3751 len 12288 data: 4 4 4
26222 239/300 Write block 2255 len 16384 data: 4 5 5 6
26223 240/300 Read block 2924 len 40960
26224 241/300 Read block 240 len 40960
26225 242/300 Flush
26226 243/300 Flush
26227 244/300 Flush
26228 245/300 Write block 2783 len 20480 data: 5 6 6 6 8
26229 246/300 Read block 1191 len 8192
26230 247/300 Flush
26231 248/300 Write block 2298 len 20480 data: 5 4 4 3 2
26232 249/300 Flush
26233 250/300 Write block 3728 len 4096 data: 5
26234 251/300 Write block 1479 len 4096 data: 4
26235 252/300 Read block 346 len 24576
26236 253/300 Write block 4031 len 40960 data: 4 2 5 3 4 4 3 3 5 5
26237 254/300 Flush
26238 255/300 Write block 4868 len 12288 data: 2 3 5
26239 256/300 Write block 529 len 40960 data: 5 6 7 7 6 6 6 7 5 5
26240 257/300 Read block 3290 len 36864
26241 258/300 Write block 1385 len 16384 data: 7 6 6 5
26242 259/300 Write block 264 len 24576 data: 3 4 4 3 3 5
26243 260/300 Write block 4099 len 4096 data: 3
26244 261/300 Flush
26245 262/300 Read block 1925 len 20480
26246 263/300 Read block 850 len 32768
26247 264/300 Read block 4708 len 24576
26248 265/300 Read block 4973 len 36864
26249 266/300 Read block 1751 len 24576
26250 267/300 Read block 866 len 4096
26251 268/300 Read block 447 len 16384
26252 269/300 Write block 403 len 12288 data: 7 6 5
26253 270/300 Write block 1563 len 24576 data: 6 6 6 5 5 6
26254 271/300 Write block 4742 len 24576 data: 2 3 4 4 5 5
26255 272/300 Write block 3312 len 8192 data: 4 6
26256 273/300 Flush
26257 274/300 Write block 3774 len 32768 data: 4 4 4 3 4 5 5 6
26258 275/300 Flush
26259 276/300 Flush
26260 277/300 Read block 690 len 8192
26261 278/300 Read block 908 len 4096
26262 279/300 Write block 360 len 32768 data: 7 7 8 9 9 6 6 5
26263 280/300 Read block 4278 len 12288
26264 281/300 Read block 2988 len 28672
26265 282/300 Write block 4906 len 28672 data: 3 4 4 4 5 5 5
26266 283/300 Read block 3963 len 16384
26267 284/300 Read block 4387 len 32768
26268 285/300 Flush
26269 286/300 Read block 1173 len 8192
26270 287/300 Read block 338 len 28672
26271 288/300 Write block 246 len 20480 data: 7 5 5 4 6
26272 289/300 Write block 3205 len 8192 data: 7 5
26273 290/300 Read block 2962 len 16384
26274 291/300 Read block 4052 len 24576
26275 292/300 Write block 1719 len 40960 data: 4 5 6 4 5 5 6 7 7 6
26276 293/300 Write block 4487 len 32768 data: 4 3 3 3 3 3 3 3
26277 294/300 Write block 3879 len 20480 data: 4 4 4 4 5
26278 295/300 Write block 1919 len 28672 data: 6 7 7 8 7 7 4
26279 296/300 Write block 3516 len 24576 data: 6 4 5 4 3 3
26280 297/300 Read block 940 len 28672
26281 298/300 Write block 3538 len 16384 data: 4 4 4 5
26282 299/300 Read block 2982 len 20480
26283 300/300 Read block 1621 len 20480
26284 301/300 Read block 2265 len 24576
26285 Main task has work to do, go find it
26286 got dsc Start(2)
26287 start 2
26288 Jan 17 21:16:37.909 INFO request completed, latency_us: 216, response_code: 204, uri: /start/cid/2, method: POST, req_id: c1305df5-ce0e-4200-abe8-cd05164eef8d, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
26289 Replay: started 2, returned:Ok(())
26290 ----------------------------------------------------------------
26291 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
26292 States: Active Active Offline
26293 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26294 7339 Acked Read 9 Done Done Sent false
26295 7340 Acked Read 6 Done Done Sent false
26296 7341 Acked Flush 0 Done Done Sent false
26297 7342 Acked Write 6 Done Done Sent false
26298 7343 Acked Read 1 Done Done Sent false
26299 7344 Acked Write 6 Done Done Sent false
26300 7345 Acked Write 2 Done Done Sent false
26301 7346 Acked Read 4 Done Done Sent false
26302 7347 Acked Write 4 Done Done Sent false
26303 7348 Acked Write 9 Done Done Sent false
26304 7349 Acked Read 10 Done Done Sent false
26305 7350 Acked Flush 0 Done Done Sent false
26306 7351 Acked Write 8 Done Done Sent false
26307 7352 Acked Read 9 Done Done Sent false
26308 7353 Acked Read 6 Done Done Sent false
26309 7354 Acked Read 7 Done Done Sent false
26310 7355 Acked Read 6 Done Done Sent false
26311 7356 Acked Write 9 Done Done Sent false
26312 7357 Acked Flush 0 Done Done Sent false
26313 7358 Acked Write 2 Done Done Sent false
26314 7359 Acked Read 7 Done Done Sent false
26315 7360 Acked Read 10 Done Done Sent false
26316 7361 Acked Write 9 Done Done Sent false
26317 7362 Acked Read 8 Done Done Sent false
26318 7363 Acked Write 10 Done Done Sent false
26319 7364 Acked Write 1 Done Done Sent false
26320 7365 Acked Read 7 Done Done Sent false
26321 7366 Acked Read 3 Done Done Sent false
26322 7367 Acked Read 2 Done Done Sent false
26323 7368 Acked Read 5 Done Done Sent false
26324 7369 Acked Read 7 Done Done Sent false
26325 7370 Acked Read 4 Done Done Sent false
26326 7371 Acked Read 2 Done Done Sent false
26327 7372 Acked Write 7 Done Done Sent false
26328 7373 Acked Read 7 Done Done Sent false
26329 7374 Acked Read 1 Done Done Sent false
26330 7375 Acked Write 6 Done Done Sent false
26331 7376 Acked Read 8 Done Done Sent false
26332 7377 Acked Write 10 Done Done Sent false
26333 7378 Acked Write 9 Done Done Sent false
26334 7379 Acked Read 2 Done Done Sent false
26335 7380 Acked Read 1 Done Done Sent false
26336 7381 Acked Read 1 Done Done Sent false
26337 7382 Acked Read 10 Done Done Sent false
26338 7383 Acked Write 5 Done Done Sent false
26339 7384 Acked Read 2 Done Done Sent false
26340 7385 Acked Read 7 Done Done Sent false
26341 7386 Acked Write 3 Done Done Sent false
26342 7387 Acked Read 9 Done Done Sent false
26343 7388 Acked Read 2 Done Done Sent false
26344 7389 Acked Write 5 Done Done Sent false
26345 7390 Acked Read 5 Done Done Sent false
26346 7391 Acked Read 7 Done Done Sent false
26347 7392 Acked Write 2 Done Done Sent false
26348 7393 Acked Write 9 Done Done Sent false
26349 7394 Acked Read 6 Done Done Sent false
26350 7395 Acked Read 2 Done Done Sent false
26351 7396 Acked Read 1 Done Done Sent false
26352 7397 Acked Write 6 Done Done Sent false
26353 7398 Acked Read 5 Done Done Sent false
26354 7399 Acked Write 2 Done Done Sent false
26355 7400 Acked Read 3 Done Done Sent false
26356 7401 Acked Write 6 Done Done Sent false
26357 7402 Acked Read 8 Done Done Sent false
26358 7403 Acked Write 5 Done Done Sent false
26359 7404 Acked Read 9 Done Done Sent false
26360 7405 Acked Write 4 Done Done Sent false
26361 7406 Acked Flush 0 Done Done Sent false
26362 7407 Acked Write 6 Done Done Sent false
26363 7408 Acked Write 6 Done Done Sent false
26364 7409 Acked Flush 0 Done Done Sent false
26365 7410 Acked Write 3 Done Done Sent false
26366 7411 Acked Write 3 Done Done Sent false
26367 7412 Acked Write 5 Done Done Sent false
26368 7413 Acked Flush 0 Done Done Sent false
26369 7414 Acked Flush 0 Done Done Sent false
26370 7415 Acked Read 4 Done Done Sent false
26371 7416 Acked Read 10 Done Done Sent false
26372 7417 Acked Read 6 Done Done Sent false
26373 7418 Acked Write 4 Done Done Sent false
26374 7419 Acked Read 8 Done Done Sent false
26375 7420 Acked Read 2 Done Done Sent false
26376 7421 Acked Write 6 Done Done Sent false
26377 7422 Acked Read 8 Done Done Sent false
26378 7423 Acked Read 8 Done Done Sent false
26379 7424 Acked Flush 0 Done Done Sent false
26380 7425 Acked Read 7 Done Done Sent false
26381 7426 Acked Read 9 Done Done Sent false
26382 7427 Acked Read 10 Done Done Sent false
26383 7428 Acked Read 3 Done Done Sent false
26384 7429 Acked Write 1 Done Done Sent false
26385 7430 Acked Write 5 Done Done Sent false
26386 7431 Acked Read 5 Done Done Sent false
26387 7432 Acked Write 8 Done Done Sent false
26388 7433 Acked Read 8 Done Done Sent false
26389 7434 Acked Read 3 Done Done Sent false
26390 7435 Acked Read 4 Done Done Sent false
26391 7436 Acked Read 6 Done Done Sent false
26392 7437 Acked Read 3 Done Done Sent false
26393 7438 Acked Write 2 Done Done Sent false
26394 7439 Acked Flush 0 Done Done Sent false
26395 7440 Acked Read 7 Done Done Sent false
26396 7441 Acked Write 9 Done Done Sent false
26397 7442 Acked Read 3 Done Done Sent false
26398 7443 Acked Flush 0 Done Done Sent false
26399 7444 Acked Read 4 Done Done Sent false
26400 7445 Acked Write 2 Done Done Sent false
26401 7446 Acked Write 5 Done Done Sent false
26402 7447 Acked Write 10 Done Done Sent false
26403 7448 Acked Write 10 Done Done Sent false
26404 7449 Acked Read 9 Done Done Sent false
26405 7450 Acked Read 6 Done Done Sent false
26406 7451 Acked Flush 0 Done Done Sent false
26407 7452 Acked Write 9 Done Done Sent false
26408 7453 Acked Write 9 Done Done Sent false
26409 7454 Acked Write 9 Done Done Sent false
26410 7455 Acked Read 2 Done Done Sent false
26411 7456 Acked Write 8 Done Done Sent false
26412 7457 Acked Write 6 Done Done Sent false
26413 7458 Acked Read 5 Done Done Sent false
26414 7459 Acked Read 4 Done Done Sent false
26415 7460 Acked Write 4 Done Done Sent false
26416 7461 Acked Write 7 Done Done Sent false
26417 7462 Acked Flush 0 Done Done Sent false
26418 7463 Acked Read 5 Done Done Sent false
26419 7464 Acked Read 7 Done Done Sent false
26420 7465 Acked Flush 0 Done Done Sent false
26421 7466 Acked Write 6 Done Done Sent false
26422 7467 Acked Read 7 Done Done Sent false
26423 7468 Acked Write 5 Done Done Sent false
26424 7469 Acked Read 6 Done Done Sent false
26425 7470 Acked Read 8 Done Done Sent false
26426 7471 Acked Write 7 Done Done Sent false
26427 7472 Acked Write 9 Done Done Sent false
26428 7473 Acked Read 5 Done Done Sent false
26429 7474 Acked Read 6 Done Done Sent false
26430 7475 Acked Write 3 Done Done Sent false
26431 7476 Acked Write 1 Done Done Sent false
26432 7477 Acked Write 7 Done Done Sent false
26433 7478 Acked Read 10 Done Done Sent false
26434 7479 Acked Write 4 Done Done Sent false
26435 7480 Acked Write 2 Done Done Sent false
26436 7481 Acked Read 5 Done Done Sent false
26437 7482 Acked Write 3 Done Done Sent false
26438 7483 Acked Read 3 Done Done Sent false
26439 7484 Acked Read 6 Done Done Sent false
26440 7485 Acked Write 6 Done Done Sent false
26441 7486 Acked Flush 0 Done Done Sent false
26442 7487 Acked Write 6 Done Done Sent false
26443 7488 Acked Read 10 Done Done Sent false
26444 7489 Acked Write 1 Done Done Sent false
26445 7490 Acked Write 4 Done Done Sent false
26446 7491 Acked Read 2 Done Done Sent false
26447 7492 Acked Read 3 Done Done Sent false
26448 7493 Acked Read 4 Done Done Sent false
26449 7494 Acked Write 7 Done Done Sent false
26450 7495 Acked Read 10 Done Done Sent false
26451 7496 Acked Write 6 Done Done Sent false
26452 7497 Acked Read 4 Done Done Sent false
26453 7498 Acked Read 5 Done Done Sent false
26454 7499 Acked Read 4 Done Done Sent false
26455 7500 Acked Read 2 Done Done Sent false
26456 7501 Acked Write 5 Done Done Sent false
26457 7502 Acked Read 7 Done Done Sent false
26458 7503 Acked Write 2 Done Done Sent false
26459 7504 Acked Read 9 Done Done Sent false
26460 7505 Acked Read 3 Done Done Sent false
26461 7506 Acked Read 1 Done Done Sent false
26462 7507 Acked Flush 0 Done Done Sent false
26463 7508 Acked Read 6 Done Done Sent false
26464 7509 Acked Write 3 Done Done Sent false
26465 7510 Acked Write 9 Done Done Sent false
26466 7511 Acked Flush 0 Done Done Sent false
26467 7512 Acked Read 6 Done Done Sent false
26468 7513 Acked Write 7 Done Done Sent false
26469 7514 Acked Read 7 Done Done Sent false
26470 7515 Acked Write 1 Done Done Sent false
26471 7516 Acked Read 5 Done Done Sent false
26472 7517 Acked Write 4 Done Done Sent false
26473 7518 Acked Write 2 Done Done Sent false
26474 7519 Acked Write 2 Done Done Sent false
26475 7520 Acked Read 5 Done Done Sent false
26476 7521 Acked Write 8 Done Done Sent false
26477 7522 Acked Read 5 Done Done Sent false
26478 7523 Acked Write 3 Done Done Sent false
26479 7524 Acked Read 8 Done Done Sent false
26480 7525 Acked Read 5 Done Done Sent false
26481 7526 Acked Write 3 Done Done Sent false
26482 7527 Acked Write 5 Done Done Sent false
26483 7528 Acked Flush 0 Done Done Sent false
26484 7529 Acked Flush 0 Done Done Sent false
26485 7530 Acked Read 9 Done Done Sent false
26486 7531 Acked Write 2 Done Done Sent false
26487 7532 Acked Write 6 Done Done Sent false
26488 7533 Acked Read 2 Done Done Sent false
26489 7534 Acked Read 5 Done Done Sent false
26490 7535 Acked Read 6 Done Done Sent false
26491 7536 Acked Flush 0 Done Done Sent false
26492 7537 Acked Read 7 Done Done Sent false
26493 7538 Acked Read 6 Done Done Sent false
26494 7539 Acked Read 1 Done Done Sent false
26495 7540 Acked Read 3 Done Done Sent false
26496 7541 Acked Read 1 Done Done Sent false
26497 7542 Acked Write 5 Done Done Sent false
26498 7543 Acked Write 7 Done Done Sent false
26499 7544 Acked Flush 0 Done Done Sent false
26500 7545 Acked Write 10 Done Done Sent false
26501 7546 Acked Read 10 Done Done Sent false
26502 7547 Acked Read 7 Done Done Sent false
26503 7548 Acked Read 9 Done Done Sent false
26504 7549 Acked Write 2 Done Done Sent false
26505 7550 Acked Write 10 Done Done Sent false
26506 7551 Acked Read 7 Done Done Sent false
26507 7552 Acked Write 10 Done Done Sent false
26508 7553 Acked Read 6 Done Done Sent false
26509 7554 Acked Write 7 Done Done Sent false
26510 7555 Acked Write 5 Done Done Sent false
26511 7556 Acked Write 8 Done Done Sent false
26512 7557 Acked Read 8 Done Done Sent false
26513 7558 Acked Write 8 Done Done Sent false
26514 7559 Acked Write 8 Done Done Sent false
26515 7560 Acked Read 7 Done Done Sent false
26516 7561 Acked Read 6 Done Done Sent false
26517 7562 Acked Write 9 Done Done Sent false
26518 7563 Acked Read 3 Done Done Sent false
26519 7564 Acked Read 9 Done Done Sent false
26520 7565 Acked Read 2 Done Done Sent false
26521 7566 Acked Read 5 Done Done Sent false
26522 7567 Acked Write 1 Done Done Sent false
26523 7568 Acked Read 5 Done Done Sent false
26524 7569 Acked Read 4 Done Done Sent false
26525 7570 Acked Write 7 Done Done Sent false
26526 7571 Acked Write 3 Done Done Sent false
26527 7572 Acked Read 9 Done Done Sent false
26528 7573 Acked Write 10 Done Done Sent false
26529 7574 Acked Read 2 Done Done Sent false
26530 7575 Acked Read 3 Done Done Sent false
26531 7576 Acked Write 3 Done Done Sent false
26532 7577 Acked Write 4 Done Done Sent false
26533 7578 Acked Read 10 Done Done Sent false
26534 7579 Acked Read 10 Done Done Sent false
26535 7580 Acked Flush 0 Done Done Sent false
26536 7581 Acked Flush 0 Done Done Sent false
26537 7582 Acked Flush 0 Done Done Sent false
26538 7583 Acked Write 5 Done Done Sent false
26539 7584 Acked Read 2 Done Done Sent false
26540 7585 Acked Flush 0 Done Done Sent false
26541 7586 Acked Write 5 Done Done Sent false
26542 7587 Acked Flush 0 Done Done Sent false
26543 7588 Acked Write 1 Done Done Sent false
26544 7589 Acked Write 1 Done Done Sent false
26545 7590 Acked Read 6 Done Done Sent false
26546 7591 Acked Write 10 Done Done Sent false
26547 7592 Acked Flush 0 Done Done Sent false
26548 7593 Acked Write 3 Done Done Sent false
26549 7594 Acked Write 10 Done Done Sent false
26550 7595 Acked Read 9 Done Done Sent false
26551 7596 Acked Write 4 Done Done Sent false
26552 7597 Acked Write 6 Done Done Sent false
26553 7598 Acked Write 1 Done Done Sent false
26554 7599 Acked Flush 0 Done Done Sent false
26555 7600 Acked Read 5 Done Done Sent false
26556 7601 Acked Read 8 Done Done Sent false
26557 7602 Acked Read 6 Done Done Sent false
26558 7603 Acked Read 9 Done Done Sent false
26559 7604 Acked Read 6 Done Done Sent false
26560 7605 Acked Read 1 Done Done Sent false
26561 7606 Acked Read 4 Done Done Sent false
26562 7607 Acked Write 3 Done Done Sent false
26563 7608 Acked Write 6 Done Done Sent false
26564 7609 Acked Write 6 Done Done Sent false
26565 7610 Acked Write 2 Done Done Sent false
26566 7611 Acked Flush 0 Done Done Sent false
26567 7612 Acked Write 8 Done Done Sent false
26568 7613 Acked Flush 0 Done Done Sent false
26569 7614 Acked Flush 0 Done Done Sent false
26570 7615 Acked Read 2 Done Done Sent false
26571 7616 Acked Read 1 Done Done Sent false
26572 7617 Acked Write 8 Done Done Sent false
26573 7618 Acked Read 3 Done Done Sent false
26574 7619 Acked Read 7 Done Done Sent false
26575 7620 Acked Write 7 Done Done Sent false
26576 7621 Acked Read 4 Done Done Sent false
26577 7622 Acked Read 8 Done Done Sent false
26578 7623 Acked Flush 0 Done Done Sent false
26579 7624 Acked Read 2 Done Done Sent false
26580 7625 Acked Read 7 Done Done Sent false
26581 7626 Acked Write 5 Done Done Sent false
26582 7627 Acked Write 2 Done Done Sent false
26583 7628 Acked Read 4 Done Done Sent false
26584 7629 Acked Read 6 Done Done Sent false
26585 7630 Acked Write 10 Done Done Sent false
26586 7631 Acked Write 8 Done Done Sent false
26587 7632 Acked Write 5 Done Done Sent false
26588 7633 Acked Write 7 Done Done Sent false
26589 7634 Acked Write 6 Done Done Sent false
26590 7635 Acked Read 7 Done Done Sent false
26591 7636 Acked Write 4 Done Done Sent false
26592 7637 Acked Read 5 Done Done Sent false
26593 7638 Acked Read 5 Done Done Sent false
26594 7639 Acked Read 6 Done Done Sent false
26595 STATES DS:0 DS:1 DS:2 TOTAL
26596 Sent 0 0 301 301
26597 Done 301 301 0 602
26598 Skipped 0 0 0 0
26599 Error 0 0 0 0
26600 Last Flush: 7623 7623 7338
26601 Downstairs last five completed: 7338 7337 7336 7335 7334
26602 Upstairs last five completed: 7639 7638 7637 7636 7635
26603 CLIENT: Up:0 ds:301 act:2
26604 [8830] got start action
26605 [8830] I am going to restart
26606 Starting downstairs at port 8830
26607 Make output file at "/tmp/dsc/downstairs-8830.txt"
26608 [8830][2] reports Starting
26609 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1222)
26610 [8830][2] reports Running
26611 ----------------------------------------------------------------
26612 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26613 States: Active Active Offline
26614 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26615 7339 Acked Read 9 Done Done Sent false
26616 7340 Acked Read 6 Done Done Sent false
26617 7341 Acked Flush 0 Done Done Sent false
26618 7342 Acked Write 6 Done Done Sent false
26619 7343 Acked Read 1 Done Done Sent false
26620 7344 Acked Write 6 Done Done Sent false
26621 7345 Acked Write 2 Done Done Sent false
26622 7346 Acked Read 4 Done Done Sent false
26623 7347 Acked Write 4 Done Done Sent false
26624 7348 Acked Write 9 Done Done Sent false
26625 7349 Acked Read 10 Done Done Sent false
26626 7350 Acked Flush 0 Done Done Sent false
26627 7351 Acked Write 8 Done Done Sent false
26628 7352 Acked Read 9 Done Done Sent false
26629 7353 Acked Read 6 Done Done Sent false
26630 7354 Acked Read 7 Done Done Sent false
26631 7355 Acked Read 6 Done Done Sent false
26632 7356 Acked Write 9 Done Done Sent false
26633 7357 Acked Flush 0 Done Done Sent false
26634 7358 Acked Write 2 Done Done Sent false
26635 7359 Acked Read 7 Done Done Sent false
26636 7360 Acked Read 10 Done Done Sent false
26637 7361 Acked Write 9 Done Done Sent false
26638 7362 Acked Read 8 Done Done Sent false
26639 7363 Acked Write 10 Done Done Sent false
26640 7364 Acked Write 1 Done Done Sent false
26641 7365 Acked Read 7 Done Done Sent false
26642 7366 Acked Read 3 Done Done Sent false
26643 7367 Acked Read 2 Done Done Sent false
26644 7368 Acked Read 5 Done Done Sent false
26645 7369 Acked Read 7 Done Done Sent false
26646 7370 Acked Read 4 Done Done Sent false
26647 7371 Acked Read 2 Done Done Sent false
26648 7372 Acked Write 7 Done Done Sent false
26649 7373 Acked Read 7 Done Done Sent false
26650 7374 Acked Read 1 Done Done Sent false
26651 7375 Acked Write 6 Done Done Sent false
26652 7376 Acked Read 8 Done Done Sent false
26653 7377 Acked Write 10 Done Done Sent false
26654 7378 Acked Write 9 Done Done Sent false
26655 7379 Acked Read 2 Done Done Sent false
26656 7380 Acked Read 1 Done Done Sent false
26657 7381 Acked Read 1 Done Done Sent false
26658 7382 Acked Read 10 Done Done Sent false
26659 7383 Acked Write 5 Done Done Sent false
26660 7384 Acked Read 2 Done Done Sent false
26661 7385 Acked Read 7 Done Done Sent false
26662 7386 Acked Write 3 Done Done Sent false
26663 7387 Acked Read 9 Done Done Sent false
26664 7388 Acked Read 2 Done Done Sent false
26665 7389 Acked Write 5 Done Done Sent false
26666 7390 Acked Read 5 Done Done Sent false
26667 7391 Acked Read 7 Done Done Sent false
26668 7392 Acked Write 2 Done Done Sent false
26669 7393 Acked Write 9 Done Done Sent false
26670 7394 Acked Read 6 Done Done Sent false
26671 7395 Acked Read 2 Done Done Sent false
26672 7396 Acked Read 1 Done Done Sent false
26673 7397 Acked Write 6 Done Done Sent false
26674 7398 Acked Read 5 Done Done Sent false
26675 7399 Acked Write 2 Done Done Sent false
26676 7400 Acked Read 3 Done Done Sent false
26677 7401 Acked Write 6 Done Done Sent false
26678 7402 Acked Read 8 Done Done Sent false
26679 7403 Acked Write 5 Done Done Sent false
26680 7404 Acked Read 9 Done Done Sent false
26681 7405 Acked Write 4 Done Done Sent false
26682 7406 Acked Flush 0 Done Done Sent false
26683 7407 Acked Write 6 Done Done Sent false
26684 7408 Acked Write 6 Done Done Sent false
26685 7409 Acked Flush 0 Done Done Sent false
26686 7410 Acked Write 3 Done Done Sent false
26687 7411 Acked Write 3 Done Done Sent false
26688 7412 Acked Write 5 Done Done Sent false
26689 7413 Acked Flush 0 Done Done Sent false
26690 7414 Acked Flush 0 Done Done Sent false
26691 7415 Acked Read 4 Done Done Sent false
26692 7416 Acked Read 10 Done Done Sent false
26693 7417 Acked Read 6 Done Done Sent false
26694 7418 Acked Write 4 Done Done Sent false
26695 7419 Acked Read 8 Done Done Sent false
26696 7420 Acked Read 2 Done Done Sent false
26697 7421 Acked Write 6 Done Done Sent false
26698 7422 Acked Read 8 Done Done Sent false
26699 7423 Acked Read 8 Done Done Sent false
26700 7424 Acked Flush 0 Done Done Sent false
26701 7425 Acked Read 7 Done Done Sent false
26702 7426 Acked Read 9 Done Done Sent false
26703 7427 Acked Read 10 Done Done Sent false
26704 7428 Acked Read 3 Done Done Sent false
26705 7429 Acked Write 1 Done Done Sent false
26706 7430 Acked Write 5 Done Done Sent false
26707 7431 Acked Read 5 Done Done Sent false
26708 7432 Acked Write 8 Done Done Sent false
26709 7433 Acked Read 8 Done Done Sent false
26710 7434 Acked Read 3 Done Done Sent false
26711 7435 Acked Read 4 Done Done Sent false
26712 7436 Acked Read 6 Done Done Sent false
26713 7437 Acked Read 3 Done Done Sent false
26714 7438 Acked Write 2 Done Done Sent false
26715 7439 Acked Flush 0 Done Done Sent false
26716 7440 Acked Read 7 Done Done Sent false
26717 7441 Acked Write 9 Done Done Sent false
26718 7442 Acked Read 3 Done Done Sent false
26719 7443 Acked Flush 0 Done Done Sent false
26720 7444 Acked Read 4 Done Done Sent false
26721 7445 Acked Write 2 Done Done Sent false
26722 7446 Acked Write 5 Done Done Sent false
26723 7447 Acked Write 10 Done Done Sent false
26724 7448 Acked Write 10 Done Done Sent false
26725 7449 Acked Read 9 Done Done Sent false
26726 7450 Acked Read 6 Done Done Sent false
26727 7451 Acked Flush 0 Done Done Sent false
26728 7452 Acked Write 9 Done Done Sent false
26729 7453 Acked Write 9 Done Done Sent false
26730 7454 Acked Write 9 Done Done Sent false
26731 7455 Acked Read 2 Done Done Sent false
26732 7456 Acked Write 8 Done Done Sent false
26733 7457 Acked Write 6 Done Done Sent false
26734 7458 Acked Read 5 Done Done Sent false
26735 7459 Acked Read 4 Done Done Sent false
26736 7460 Acked Write 4 Done Done Sent false
26737 7461 Acked Write 7 Done Done Sent false
26738 7462 Acked Flush 0 Done Done Sent false
26739 7463 Acked Read 5 Done Done Sent false
26740 7464 Acked Read 7 Done Done Sent false
26741 7465 Acked Flush 0 Done Done Sent false
26742 7466 Acked Write 6 Done Done Sent false
26743 7467 Acked Read 7 Done Done Sent false
26744 7468 Acked Write 5 Done Done Sent false
26745 7469 Acked Read 6 Done Done Sent false
26746 7470 Acked Read 8 Done Done Sent false
26747 7471 Acked Write 7 Done Done Sent false
26748 7472 Acked Write 9 Done Done Sent false
26749 7473 Acked Read 5 Done Done Sent false
26750 7474 Acked Read 6 Done Done Sent false
26751 7475 Acked Write 3 Done Done Sent false
26752 7476 Acked Write 1 Done Done Sent false
26753 7477 Acked Write 7 Done Done Sent false
26754 7478 Acked Read 10 Done Done Sent false
26755 7479 Acked Write 4 Done Done Sent false
26756 7480 Acked Write 2 Done Done Sent false
26757 7481 Acked Read 5 Done Done Sent false
26758 7482 Acked Write 3 Done Done Sent false
26759 7483 Acked Read 3 Done Done Sent false
26760 7484 Acked Read 6 Done Done Sent false
26761 7485 Acked Write 6 Done Done Sent false
26762 7486 Acked Flush 0 Done Done Sent false
26763 7487 Acked Write 6 Done Done Sent false
26764 7488 Acked Read 10 Done Done Sent false
26765 7489 Acked Write 1 Done Done Sent false
26766 7490 Acked Write 4 Done Done Sent false
26767 7491 Acked Read 2 Done Done Sent false
26768 7492 Acked Read 3 Done Done Sent false
26769 7493 Acked Read 4 Done Done Sent false
26770 7494 Acked Write 7 Done Done Sent false
26771 7495 Acked Read 10 Done Done Sent false
26772 7496 Acked Write 6 Done Done Sent false
26773 7497 Acked Read 4 Done Done Sent false
26774 7498 Acked Read 5 Done Done Sent false
26775 7499 Acked Read 4 Done Done Sent false
26776 7500 Acked Read 2 Done Done Sent false
26777 7501 Acked Write 5 Done Done Sent false
26778 7502 Acked Read 7 Done Done Sent false
26779 7503 Acked Write 2 Done Done Sent false
26780 7504 Acked Read 9 Done Done Sent false
26781 7505 Acked Read 3 Done Done Sent false
26782 7506 Acked Read 1 Done Done Sent false
26783 7507 Acked Flush 0 Done Done Sent false
26784 7508 Acked Read 6 Done Done Sent false
26785 7509 Acked Write 3 Done Done Sent false
26786 7510 Acked Write 9 Done Done Sent false
26787 7511 Acked Flush 0 Done Done Sent false
26788 7512 Acked Read 6 Done Done Sent false
26789 7513 Acked Write 7 Done Done Sent false
26790 7514 Acked Read 7 Done Done Sent false
26791 7515 Acked Write 1 Done Done Sent false
26792 7516 Acked Read 5 Done Done Sent false
26793 7517 Acked Write 4 Done Done Sent false
26794 7518 Acked Write 2 Done Done Sent false
26795 7519 Acked Write 2 Done Done Sent false
26796 7520 Acked Read 5 Done Done Sent false
26797 7521 Acked Write 8 Done Done Sent false
26798 7522 Acked Read 5 Done Done Sent false
26799 7523 Acked Write 3 Done Done Sent false
26800 7524 Acked Read 8 Done Done Sent false
26801 7525 Acked Read 5 Done Done Sent false
26802 7526 Acked Write 3 Done Done Sent false
26803 7527 Acked Write 5 Done Done Sent false
26804 7528 Acked Flush 0 Done Done Sent false
26805 7529 Acked Flush 0 Done Done Sent false
26806 7530 Acked Read 9 Done Done Sent false
26807 7531 Acked Write 2 Done Done Sent false
26808 7532 Acked Write 6 Done Done Sent false
26809 7533 Acked Read 2 Done Done Sent false
26810 7534 Acked Read 5 Done Done Sent false
26811 7535 Acked Read 6 Done Done Sent false
26812 7536 Acked Flush 0 Done Done Sent false
26813 7537 Acked Read 7 Done Done Sent false
26814 7538 Acked Read 6 Done Done Sent false
26815 7539 Acked Read 1 Done Done Sent false
26816 7540 Acked Read 3 Done Done Sent false
26817 7541 Acked Read 1 Done Done Sent false
26818 7542 Acked Write 5 Done Done Sent false
26819 7543 Acked Write 7 Done Done Sent false
26820 7544 Acked Flush 0 Done Done Sent false
26821 7545 Acked Write 10 Done Done Sent false
26822 7546 Acked Read 10 Done Done Sent false
26823 7547 Acked Read 7 Done Done Sent false
26824 7548 Acked Read 9 Done Done Sent false
26825 7549 Acked Write 2 Done Done Sent false
26826 7550 Acked Write 10 Done Done Sent false
26827 7551 Acked Read 7 Done Done Sent false
26828 7552 Acked Write 10 Done Done Sent false
26829 7553 Acked Read 6 Done Done Sent false
26830 7554 Acked Write 7 Done Done Sent false
26831 7555 Acked Write 5 Done Done Sent false
26832 7556 Acked Write 8 Done Done Sent false
26833 7557 Acked Read 8 Done Done Sent false
26834 7558 Acked Write 8 Done Done Sent false
26835 7559 Acked Write 8 Done Done Sent false
26836 7560 Acked Read 7 Done Done Sent false
26837 7561 Acked Read 6 Done Done Sent false
26838 7562 Acked Write 9 Done Done Sent false
26839 7563 Acked Read 3 Done Done Sent false
26840 7564 Acked Read 9 Done Done Sent false
26841 7565 Acked Read 2 Done Done Sent false
26842 7566 Acked Read 5 Done Done Sent false
26843 7567 Acked Write 1 Done Done Sent false
26844 7568 Acked Read 5 Done Done Sent false
26845 7569 Acked Read 4 Done Done Sent false
26846 7570 Acked Write 7 Done Done Sent false
26847 7571 Acked Write 3 Done Done Sent false
26848 7572 Acked Read 9 Done Done Sent false
26849 7573 Acked Write 10 Done Done Sent false
26850 7574 Acked Read 2 Done Done Sent false
26851 7575 Acked Read 3 Done Done Sent false
26852 7576 Acked Write 3 Done Done Sent false
26853 7577 Acked Write 4 Done Done Sent false
26854 7578 Acked Read 10 Done Done Sent false
26855 7579 Acked Read 10 Done Done Sent false
26856 7580 Acked Flush 0 Done Done Sent false
26857 7581 Acked Flush 0 Done Done Sent false
26858 7582 Acked Flush 0 Done Done Sent false
26859 7583 Acked Write 5 Done Done Sent false
26860 7584 Acked Read 2 Done Done Sent false
26861 7585 Acked Flush 0 Done Done Sent false
26862 7586 Acked Write 5 Done Done Sent false
26863 7587 Acked Flush 0 Done Done Sent false
26864 7588 Acked Write 1 Done Done Sent false
26865 7589 Acked Write 1 Done Done Sent false
26866 7590 Acked Read 6 Done Done Sent false
26867 7591 Acked Write 10 Done Done Sent false
26868 7592 Acked Flush 0 Done Done Sent false
26869 7593 Acked Write 3 Done Done Sent false
26870 7594 Acked Write 10 Done Done Sent false
26871 7595 Acked Read 9 Done Done Sent false
26872 7596 Acked Write 4 Done Done Sent false
26873 7597 Acked Write 6 Done Done Sent false
26874 7598 Acked Write 1 Done Done Sent false
26875 7599 Acked Flush 0 Done Done Sent false
26876 7600 Acked Read 5 Done Done Sent false
26877 7601 Acked Read 8 Done Done Sent false
26878 7602 Acked Read 6 Done Done Sent false
26879 7603 Acked Read 9 Done Done Sent false
26880 7604 Acked Read 6 Done Done Sent false
26881 7605 Acked Read 1 Done Done Sent false
26882 7606 Acked Read 4 Done Done Sent false
26883 7607 Acked Write 3 Done Done Sent false
26884 7608 Acked Write 6 Done Done Sent false
26885 7609 Acked Write 6 Done Done Sent false
26886 7610 Acked Write 2 Done Done Sent false
26887 7611 Acked Flush 0 Done Done Sent false
26888 7612 Acked Write 8 Done Done Sent false
26889 7613 Acked Flush 0 Done Done Sent false
26890 7614 Acked Flush 0 Done Done Sent false
26891 7615 Acked Read 2 Done Done Sent false
26892 7616 Acked Read 1 Done Done Sent false
26893 7617 Acked Write 8 Done Done Sent false
26894 7618 Acked Read 3 Done Done Sent false
26895 7619 Acked Read 7 Done Done Sent false
26896 7620 Acked Write 7 Done Done Sent false
26897 7621 Acked Read 4 Done Done Sent false
26898 7622 Acked Read 8 Done Done Sent false
26899 7623 Acked Flush 0 Done Done Sent false
26900 7624 Acked Read 2 Done Done Sent false
26901 7625 Acked Read 7 Done Done Sent false
26902 7626 Acked Write 5 Done Done Sent false
26903 7627 Acked Write 2 Done Done Sent false
26904 7628 Acked Read 4 Done Done Sent false
26905 7629 Acked Read 6 Done Done Sent false
26906 7630 Acked Write 10 Done Done Sent false
26907 7631 Acked Write 8 Done Done Sent false
26908 7632 Acked Write 5 Done Done Sent false
26909 7633 Acked Write 7 Done Done Sent false
26910 7634 Acked Write 6 Done Done Sent false
26911 7635 Acked Read 7 Done Done Sent false
26912 7636 Acked Write 4 Done Done Sent false
26913 7637 Acked Read 5 Done Done Sent false
26914 7638 Acked Read 5 Done Done Sent false
26915 7639 Acked Read 6 Done Done Sent false
26916 7640 Acked Flush 0 Done Done Sent false
26917 STATES DS:0 DS:1 DS:2 TOTAL
26918 Sent 0 0 302 302
26919 Done 302 302 0 604
26920 Skipped 0 0 0 0
26921 Error 0 0 0 0
26922 Last Flush: 7640 7640 7338
26923 Downstairs last five completed: 7338 7337 7336 7335 7334
26924 Upstairs last five completed: 7640 7639 7638 7637 7636
26925 CLIENT: Up:0 ds:302 act:2
269262025-01-17T21:16:43.666ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
269272025-01-17T21:16:43.666ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40175) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
269282025-01-17T21:16:43.667ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
269292025-01-17T21:16:43.668ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
269302025-01-17T21:16:43.668ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
269312025-01-17T21:16:43.668ZINFOcrucible: send last flush ID to this DS: 7338 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
269322025-01-17T21:16:43.668ZINFOcrucible: Replied this last flush ID: 7338 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
269332025-01-17T21:16:43.668ZINFOcrucible: [2] client re-new 302 jobs since flush 7338 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
269342025-01-17T21:16:43.668ZINFOcrucible: [2] Replayed 302 jobs since flush: 7338 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
26935 ----------------------------------------------------------------
26936 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
26937 Downstairs last five completed: 7640 7639 7638 7637 7636
26938 Upstairs last five completed: 7640 7639 7638 7637 7636
26939 CLIENT: Up:0 ds:0 act:3
26940 Replay: All jobs finished
26941 Main task has work to do, go find it
26942 got dsc Stop(0)
26943 stop 0
26944 Jan 17 21:16:45.913[8810] Got stop action so:false kr:false
26945 INFO request completed, latency_us: 375, response_code: 204, uri: /stop/cid/0, method: POST, req_id: e9c307c2-0a3d-40bf-b497-7fdc01928d34, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
26946 Jan 17 21:16:45.914 INFO request completed, latency_us: 109, response_code: 200, uri: /state/cid/0, method: GET, req_id: 2568d968-81b5-4ee2-b403-e564bb92f1eb, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
269472025-01-17T21:16:45.914ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
269482025-01-17T21:16:45.914ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
269492025-01-17T21:16:45.914ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
269502025-01-17T21:16:45.914ZINFOcrucible: client task is exiting = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
26951 [8810] Exited with: ExitStatus(unix_wait_status(9))
26952 [8810][0] reports Exit
26953 Jan 17 21:16:49.916 INFO request completed, latency_us: 229, response_code: 200, uri: /state/cid/0, method: GET, req_id: 579ca14b-0fa5-437e-9da6-c499faf71d86, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
26954 001/300 Write block 207 len 40960 data: 7 7 7 7 7 6 6 4 5 5
26955 002/300 Write block 2355 len 8192 data: 4 4
26956 003/300 Write block 4249 len 4096 data: 3
26957 004/300 Read block 1561 len 8192
26958 005/300 Read block 2282 len 32768
26959 006/300 Write block 1005 len 20480 data: 6 5 7 8 9
26960 007/300 Write block 3808 len 32768 data: 4 4 6 6 6 6 6 5
26961 008/300 Read block 1243 len 28672
26962 009/300 Write block 602 len 20480 data: 3 4 4 4 4
26963 010/300 Read block 4087 len 4096
26964 011/300 Read block 682 len 24576
26965 012/300 Read block 1300 len 16384
26966 013/300 Flush
26967 014/300 Write block 848 len 24576 data: 4 3 3 2 2 3
26968 015/300 Read block 1306 len 28672
26969 016/300 Read block 309 len 40960
26970 017/300 Flush
26971 018/300 Write block 284 len 20480 data: 6 7 7 7 7
26972 019/300 Read block 38 len 4096
26973 020/300 Write block 2399 len 8192 data: 4 4
26974 021/300 Flush
26975 022/300 Write block 2021 len 16384 data: 5 6 6 6
26976 023/300 Read block 667 len 36864
26977 024/300 Flush
26978 025/300 Read block 2659 len 4096
26979 026/300 Read block 1578 len 28672
26980 027/300 Flush
26981 028/300 Read block 4168 len 24576
26982 029/300 Write block 4819 len 24576 data: 5 6 5 5 6 7
26983 030/300 Write block 1604 len 24576 data: 5 4 4 4 3 3
26984 031/300 Read block 2391 len 20480
26985 032/300 Write block 3470 len 24576 data: 4 6 6 7 7 6
26986 033/300 Read block 608 len 28672
26987 034/300 Read block 312 len 16384
26988 035/300 Write block 1070 len 32768 data: 6 6 5 5 5 4 4 4
26989 036/300 Write block 569 len 20480 data: 3 3 3 3 4
26990 037/300 Read block 2312 len 40960
26991 038/300 Write block 4980 len 28672 data: 3 3 2 2 4 4 4
26992 039/300 Read block 4383 len 24576
26993 040/300 Read block 3860 len 28672
26994 041/300 Read block 662 len 36864
26995 042/300 Write block 1749 len 32768 data: 6 6 5 3 4 4 3 4
26996 043/300 Write block 447 len 8192 data: 3 3
26997 044/300 Read block 3376 len 32768
26998 045/300 Write block 183 len 32768 data: 3 3 3 5 5 4 3 6
26999 046/300 Write block 313 len 28672 data: 4 4 4 4 4 3 3
27000 047/300 Read block 719 len 12288
27001 048/300 Read block 2344 len 12288
27002 049/300 Read block 2444 len 16384
27003 050/300 Write block 1811 len 28672 data: 4 4 4 4 4 4 2
27004 051/300 Write block 4797 len 28672 data: 2 2 2 2 2 3 3
27005 052/300 Read block 4199 len 20480
27006 053/300 Read block 1604 len 28672
27007 054/300 Write block 1038 len 8192 data: 4 4
27008 055/300 Read block 1246 len 24576
27009 056/300 Read block 2727 len 32768
27010 057/300 Write block 223 len 4096 data: 5
27011 058/300 Write block 3241 len 8192 data: 4 5
27012 059/300 Write block 2436 len 12288 data: 4 3 3
27013 060/300 Write block 1440 len 8192 data: 8 8
27014 061/300 Write block 3989 len 4096 data: 5
27015 062/300 Write block 3152 len 28672 data: 4 4 4 5 4 5 5
27016 063/300 Read block 1695 len 32768
27017 064/300 Read block 2973 len 16384
27018 065/300 Write block 4329 len 32768 data: 5 6 6 4 5 6 6 6
27019 066/300 Read block 2813 len 24576
27020 067/300 Write block 87 len 36864 data: 7 6 6 6 6 4 4 5 4
27021 068/300 Write block 4206 len 16384 data: 5 5 6 5
27022 069/300 Read block 3449 len 4096
27023 070/300 Write block 4003 len 4096 data: 6
27024 071/300 Read block 3166 len 16384
27025 072/300 Read block 4029 len 4096
27026 073/300 Read block 1643 len 32768
27027 074/300 Read block 2715 len 8192
27028 075/300 Read block 724 len 20480
27029 076/300 Read block 1250 len 8192
27030 077/300 Read block 384 len 16384
27031 078/300 Write block 1466 len 8192 data: 7 8
27032 079/300 Write block 1876 len 36864 data: 4 4 4 4 6 6 7 6 7
27033 080/300 Write block 3424 len 36864 data: 5 4 5 4 3 3 4 4 4
27034 081/300 Read block 4575 len 4096
27035 082/300 Flush
27036 083/300 Read block 355 len 16384
27037 084/300 Read block 1026 len 24576
27038 085/300 Write block 2640 len 32768 data: 3 3 4 5 6 7 6 4
27039 086/300 Flush
27040 087/300 Read block 3549 len 12288
27041 088/300 Write block 4311 len 36864 data: 6 5 4 4 5 5 7 6 5
27042 089/300 Read block 1485 len 36864
27043 090/300 Read block 3415 len 16384
27044 091/300 Write block 1436 len 28672 data: 5 6 8 8 9 9 9
27045 092/300 Read block 3614 len 4096
27046 093/300 Write block 3943 len 24576 data: 3 3 4 5 5 6
27047 094/300 Write block 235 len 16384 data: 3 2 3 3
27048 095/300 Write block 2477 len 40960 data: 5 7 6 6 5 4 5 6 5 5
27049 096/300 Read block 2958 len 24576
27050 097/300 Read block 4397 len 4096
27051 098/300 Read block 244 len 4096
27052 099/300 Read block 4166 len 36864
27053 100/300 Write block 2753 len 4096 data: 5
27054 101/300 Read block 2313 len 40960
27055 102/300 Write block 530 len 4096 data: 7
27056 103/300 Write block 59 len 16384 data: 4 4 5 5
27057 104/300 Write block 2902 len 32768 data: 8 9 9 7 6 6 5 5
27058 105/300 Read block 237 len 32768
27059 106/300 Write block 1127 len 24576 data: 4 5 4 3 2 4
27060 107/300 Read block 1041 len 24576
27061 108/300 Read block 2567 len 32768
27062 109/300 Flush
27063 110/300 Write block 3493 len 8192 data: 3 4
27064 111/300 Read block 3728 len 16384
27065 112/300 Read block 4147 len 8192
27066 113/300 Write block 1869 len 12288 data: 3 2 3
27067 114/300 Read block 2377 len 12288
27068 115/300 Write block 4304 len 12288 data: 3 2 3
27069 116/300 Write block 4593 len 12288 data: 3 3 3
27070 117/300 Read block 3468 len 12288
27071 118/300 Flush
27072 119/300 Write block 1595 len 16384 data: 5 5 4 3
27073 120/300 Read block 4229 len 32768
27074 121/300 Read block 4832 len 4096
27075 122/300 Write block 1955 len 36864 data: 8 6 6 6 7 7 5 4 5
27076 123/300 Read block 1067 len 28672
27077 124/300 Read block 2130 len 4096
27078 125/300 Write block 881 len 16384 data: 7 5 4 5
27079 126/300 Read block 489 len 8192
27080 127/300 Read block 3836 len 24576
27081 128/300 Read block 2459 len 4096
27082 129/300 Read block 4648 len 28672
27083 130/300 Read block 653 len 8192
27084 131/300 Write block 4171 len 36864 data: 6 7 8 8 9 7 10 10 8
27085 132/300 Read block 1810 len 20480
27086 133/300 Flush
27087 134/300 Read block 2377 len 8192
27088 135/300 Flush
27089 136/300 Read block 3621 len 24576
27090 137/300 Write block 928 len 24576 data: 4 6 6 6 6 6
27091 138/300 Flush
27092 139/300 Read block 4441 len 40960
27093 140/300 Write block 1623 len 36864 data: 4 5 4 5 3 5 4 4 5
27094 141/300 Read block 3579 len 28672
27095 142/300 Read block 1428 len 20480
27096 143/300 Write block 3190 len 40960 data: 5 5 6 5 5 5 5 5 3 4
27097 144/300 Write block 677 len 40960 data: 2 3 3 4 4 5 7 8 8 10
27098 145/300 Read block 2691 len 4096
27099 146/300 Read block 1659 len 36864
27100 147/300 Write block 4736 len 28672 data: 6 5 4 4 5 4 3
27101 148/300 Write block 2405 len 40960 data: 5 5 6 7 6 6 6 4 3 4
27102 149/300 Read block 3045 len 12288
27103 150/300 Flush
27104 151/300 Flush
27105 152/300 Write block 379 len 12288 data: 3 3 4
27106 153/300 Read block 2739 len 28672
27107 154/300 Read block 4473 len 36864
27108 155/300 Write block 2926 len 24576 data: 6 5 7 8 6 5
27109 156/300 Read block 1285 len 28672
27110 157/300 Flush
27111 158/300 Read block 2518 len 20480
27112 159/300 Flush
27113 160/300 Read block 3681 len 24576
27114 161/300 Write block 3789 len 24576 data: 6 6 4 4 3 3
27115 162/300 Read block 1317 len 12288
27116 163/300 Write block 255 len 16384 data: 4 4 4 4
27117 164/300 Write block 502 len 36864 data: 4 4 5 6 5 6 7 7 8
27118 165/300 Read block 2125 len 12288
27119 166/300 Write block 2994 len 40960 data: 5 5 4 3 3 3 4 5 6 7
27120 167/300 Read block 4490 len 8192
27121 168/300 Read block 4241 len 16384
27122 169/300 Write block 3548 len 8192 data: 2 2
27123 170/300 Read block 1214 len 36864
27124 171/300 Read block 3334 len 36864
27125 172/300 Read block 3218 len 24576
27126 173/300 Write block 2472 len 8192 data: 5 6
27127 174/300 Read block 645 len 8192
27128 175/300 Flush
27129 176/300 Read block 4913 len 36864
27130 177/300 Write block 862 len 36864 data: 8 7 6 4 3 4 4 3 4
27131 178/300 Write block 3804 len 4096 data: 4
27132 179/300 Read block 2239 len 8192
27133 180/300 Read block 4989 len 36864
27134 181/300 Read block 4496 len 28672
27135 182/300 Write block 4246 len 12288 data: 3 3 3
27136 183/300 Flush
27137 184/300 Read block 2182 len 36864
27138 185/300 Write block 802 len 20480 data: 4 5 6 6 5
27139 186/300 Read block 3191 len 32768
27140 187/300 Flush
27141 188/300 Write block 1993 len 4096 data: 4
27142 189/300 Read block 1328 len 28672
27143 190/300 Read block 2186 len 8192
27144 191/300 Flush
27145 192/300 Read block 4640 len 4096
27146 193/300 Write block 2939 len 28672 data: 3 3 3 4 6 7 6
27147 194/300 Write block 922 len 20480 data: 6 6 5 5 5
27148 195/300 Read block 2115 len 24576
27149 196/300 Read block 3717 len 20480
27150 197/300 Read block 3433 len 28672
27151 198/300 Write block 3658 len 16384 data: 2 2 2 3
27152 199/300 Write block 2319 len 24576 data: 6 6 5 7 6 6
27153 200/300 Read block 4858 len 8192
27154 201/300 Read block 1301 len 28672
27155 202/300 Read block 3714 len 12288
27156 203/300 Flush
27157 204/300 Read block 3084 len 28672
27158 205/300 Read block 726 len 36864
27159 206/300 Read block 3089 len 16384
27160 207/300 Write block 2535 len 4096 data: 4
27161 208/300 Write block 3754 len 28672 data: 6 5 6 6 4 3 3
27162 209/300 Flush
27163 210/300 Write block 1224 len 20480 data: 4 5 4 4 4
27164 211/300 Flush
27165 212/300 Read block 58 len 12288
27166 213/300 Flush
27167 214/300 Read block 943 len 12288
27168 215/300 Read block 2350 len 28672
27169 216/300 Read block 1660 len 16384
27170 217/300 Write block 2204 len 28672 data: 5 5 5 5 4 3 4
27171 218/300 Write block 2064 len 32768 data: 9 8 8 10 10 9 7 6
27172 219/300 Flush
27173 220/300 Read block 4915 len 20480
27174 221/300 Read block 3510 len 8192
27175 222/300 Read block 4257 len 40960
27176 223/300 Read block 163 len 4096
27177 224/300 Read block 4606 len 8192
27178 225/300 Read block 2787 len 36864
27179 226/300 Read block 4931 len 32768
27180 227/300 Read block 74 len 20480
27181 228/300 Read block 888 len 40960
27182 229/300 Write block 4154 len 32768 data: 3 5 7 7 6 5 6 4
27183 230/300 Write block 3935 len 12288 data: 8 8 6
27184 231/300 Write block 2367 len 36864 data: 4 3 3 4 5 6 7 6 5
27185 232/300 Write block 4346 len 8192 data: 4 5
27186 233/300 Read block 4424 len 16384
27187 234/300 Write block 2117 len 36864 data: 6 6 5 6 6 5 5 5 5
27188 235/300 Read block 1059 len 12288
27189 236/300 Read block 1446 len 8192
27190 237/300 Write block 3163 len 4096 data: 6
27191 238/300 Read block 3174 len 4096
27192 239/300 Read block 3830 len 12288
27193 240/300 Read block 2338 len 40960
27194 241/300 Write block 3786 len 36864 data: 8 8 8 7 7 5 5 4 4
27195 242/300 Write block 217 len 32768 data: 5 4 4 4 4 4 6 5
27196 243/300 Read block 3563 len 8192
27197 244/300 Write block 684 len 40960 data: 9 9 11 10 10 6 7 6 7 8
27198 245/300 Read block 3921 len 20480
27199 246/300 Write block 1522 len 20480 data: 6 7 8 9 6
27200 247/300 Flush
27201 248/300 Read block 3371 len 24576
27202 249/300 Flush
27203 250/300 Write block 4774 len 32768 data: 4 2 4 4 5 7 5 7
27204 251/300 Read block 310 len 20480
27205 252/300 Write block 3863 len 32768 data: 3 3 3 3 3 3 4 4
27206 253/300 Read block 1173 len 40960
27207 254/300 Read block 470 len 12288
27208 255/300 Read block 1851 len 40960
27209 256/300 Read block 3950 len 20480
27210 257/300 Write block 201 len 20480 data: 5 4 3 4 5
27211 258/300 Read block 3112 len 8192
27212 259/300 Read block 868 len 40960
27213 260/300 Read block 964 len 32768
27214 261/300 Read block 5 len 32768
27215 262/300 Read block 3531 len 8192
27216 263/300 Write block 2804 len 20480 data: 3 4 4 5 6
27217 264/300 Read block 2685 len 32768
27218 265/300 Read block 4459 len 8192
27219 266/300 Write block 4086 len 8192 data: 4 4
27220 267/300 Flush
27221 268/300 Read block 2695 len 24576
27222 269/300 Read block 967 len 20480
27223 270/300 Write block 836 len 32768 data: 4 5 4 4 4 4 6 5
27224 271/300 Read block 554 len 12288
27225 272/300 Read block 4720 len 20480
27226 273/300 Write block 4591 len 40960 data: 3 3 4 4 4 4 3 3 4 4
27227 274/300 Flush
27228 275/300 Read block 2550 len 12288
27229 276/300 Write block 2580 len 32768 data: 2 3 3 5 4 6 9 9
27230 277/300 Write block 4831 len 24576 data: 5 5 5 2 3 3
27231 278/300 Flush
27232 279/300 Read block 667 len 20480
27233 280/300 Write block 3805 len 32768 data: 4 4 5 5 5 7 7 7
27234 281/300 Write block 4048 len 16384 data: 3 2 2 2
27235 282/300 Read block 846 len 40960
27236 283/300 Read block 3237 len 24576
27237 284/300 Read block 2672 len 4096
27238 285/300 Read block 423 len 36864
27239 286/300 Read block 3294 len 12288
27240 287/300 Read block 3481 len 4096
27241 288/300 Write block 382 len 12288 data: 5 6 6
27242 289/300 Read block 4660 len 4096
27243 290/300 Read block 1773 len 20480
27244 291/300 Write block 2679 len 32768 data: 4 6 7 9 8 7 7 7
27245 292/300 Read block 200 len 24576
27246 293/300 Read block 1419 len 20480
27247 294/300 Flush
27248 295/300 Write block 277 len 28672 data: 6 5 4 6 6 7 6
27249 296/300 Read block 3764 len 24576
27250 297/300 Flush
27251 298/300 Flush
27252 299/300 Read block 3541 len 4096
27253 300/300 Read block 1390 len 36864
27254 301/300 Read block 12 len 4096
27255 Main task has work to do, go find it
27256 got dsc Start(0)
27257 start 0
27258 Jan 17 21:16:50.159 INFO request completed, latency_us: 248, response_code: 204, uri: /start/cid/0, method: POST, req_id: 476df4cc-74a1-46cd-aa5e-ef2e8e29b618, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
27259 Replay: started 0, returned:Ok(())
27260 ----------------------------------------------------------------
27261 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
27262 States: Offline Active Active
27263 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27264 7641 Acked Write 10 Sent Done Done false
27265 7642 Acked Write 2 Sent Done Done false
27266 7643 Acked Write 1 Sent Done Done false
27267 7644 Acked Read 2 Sent Done Done false
27268 7645 Acked Read 8 Sent Done Done false
27269 7646 Acked Write 5 Sent Done Done false
27270 7647 Acked Write 8 Sent Done Done false
27271 7648 Acked Read 7 Sent Done Done false
27272 7649 Acked Write 5 Sent Done Done false
27273 7650 Acked Read 1 Sent Done Done false
27274 7651 Acked Read 6 Sent Done Done false
27275 7652 Acked Read 4 Sent Done Done false
27276 7653 Acked Flush 0 Sent Done Done false
27277 7654 Acked Write 6 Sent Done Done false
27278 7655 Acked Read 7 Sent Done Done false
27279 7656 Acked Read 10 Sent Done Done false
27280 7657 Acked Flush 0 Sent Done Done false
27281 7658 Acked Write 5 Sent Done Done false
27282 7659 Acked Read 1 Sent Done Done false
27283 7660 Acked Write 2 Sent Done Done false
27284 7661 Acked Flush 0 Sent Done Done false
27285 7662 Acked Write 4 Sent Done Done false
27286 7663 Acked Read 9 Sent Done Done false
27287 7664 Acked Flush 0 Sent Done Done false
27288 7665 Acked Read 1 Sent Done Done false
27289 7666 Acked Read 7 Sent Done Done false
27290 7667 Acked Flush 0 Sent Done Done false
27291 7668 Acked Read 6 Sent Done Done false
27292 7669 Acked Write 6 Sent Done Done false
27293 7670 Acked Write 6 Sent Done Done false
27294 7671 Acked Read 5 Sent Done Done false
27295 7672 Acked Write 6 Sent Done Done false
27296 7673 Acked Read 7 Sent Done Done false
27297 7674 Acked Read 4 Sent Done Done false
27298 7675 Acked Write 8 Sent Done Done false
27299 7676 Acked Write 5 Sent Done Done false
27300 7677 Acked Read 10 Sent Done Done false
27301 7678 Acked Write 7 Sent Done Done false
27302 7679 Acked Read 6 Sent Done Done false
27303 7680 Acked Read 7 Sent Done Done false
27304 7681 Acked Read 9 Sent Done Done false
27305 7682 Acked Write 8 Sent Done Done false
27306 7683 Acked Write 2 Sent Done Done false
27307 7684 Acked Read 8 Sent Done Done false
27308 7685 Acked Write 8 Sent Done Done false
27309 7686 Acked Write 7 Sent Done Done false
27310 7687 Acked Read 3 Sent Done Done false
27311 7688 Acked Read 3 Sent Done Done false
27312 7689 Acked Read 4 Sent Done Done false
27313 7690 Acked Write 7 Sent Done Done false
27314 7691 Acked Write 7 Sent Done Done false
27315 7692 Acked Read 5 Sent Done Done false
27316 7693 Acked Read 7 Sent Done Done false
27317 7694 Acked Write 2 Sent Done Done false
27318 7695 Acked Read 6 Sent Done Done false
27319 7696 Acked Read 8 Sent Done Done false
27320 7697 Acked Write 1 Sent Done Done false
27321 7698 Acked Write 2 Sent Done Done false
27322 7699 Acked Write 3 Sent Done Done false
27323 7700 Acked Write 2 Sent Done Done false
27324 7701 Acked Write 1 Sent Done Done false
27325 7702 Acked Write 7 Sent Done Done false
27326 7703 Acked Read 8 Sent Done Done false
27327 7704 Acked Read 4 Sent Done Done false
27328 7705 Acked Write 8 Sent Done Done false
27329 7706 Acked Read 6 Sent Done Done false
27330 7707 Acked Write 9 Sent Done Done false
27331 7708 Acked Write 4 Sent Done Done false
27332 7709 Acked Read 1 Sent Done Done false
27333 7710 Acked Write 1 Sent Done Done false
27334 7711 Acked Read 4 Sent Done Done false
27335 7712 Acked Read 1 Sent Done Done false
27336 7713 Acked Read 8 Sent Done Done false
27337 7714 Acked Read 2 Sent Done Done false
27338 7715 Acked Read 5 Sent Done Done false
27339 7716 Acked Read 2 Sent Done Done false
27340 7717 Acked Read 4 Sent Done Done false
27341 7718 Acked Write 2 Sent Done Done false
27342 7719 Acked Write 9 Sent Done Done false
27343 7720 Acked Write 9 Sent Done Done false
27344 7721 Acked Read 1 Sent Done Done false
27345 7722 Acked Flush 0 Sent Done Done false
27346 7723 Acked Read 4 Sent Done Done false
27347 7724 Acked Read 6 Sent Done Done false
27348 7725 Acked Write 8 Sent Done Done false
27349 7726 Acked Flush 0 Sent Done Done false
27350 7727 Acked Read 3 Sent Done Done false
27351 7728 Acked Write 9 Sent Done Done false
27352 7729 Acked Read 9 Sent Done Done false
27353 7730 Acked Read 4 Sent Done Done false
27354 7731 Acked Write 7 Sent Done Done false
27355 7732 Acked Read 1 Sent Done Done false
27356 7733 Acked Write 6 Sent Done Done false
27357 7734 Acked Write 4 Sent Done Done false
27358 7735 Acked Write 10 Sent Done Done false
27359 7736 Acked Read 6 Sent Done Done false
27360 7737 Acked Read 1 Sent Done Done false
27361 7738 Acked Read 1 Sent Done Done false
27362 7739 Acked Read 9 Sent Done Done false
27363 7740 Acked Write 1 Sent Done Done false
27364 7741 Acked Read 10 Sent Done Done false
27365 7742 Acked Write 1 Sent Done Done false
27366 7743 Acked Write 4 Sent Done Done false
27367 7744 Acked Write 8 Sent Done Done false
27368 7745 Acked Read 8 Sent Done Done false
27369 7746 Acked Write 6 Sent Done Done false
27370 7747 Acked Read 6 Sent Done Done false
27371 7748 Acked Read 8 Sent Done Done false
27372 7749 Acked Flush 0 Sent Done Done false
27373 7750 Acked Write 2 Sent Done Done false
27374 7751 Acked Read 4 Sent Done Done false
27375 7752 Acked Read 2 Sent Done Done false
27376 7753 Acked Write 3 Sent Done Done false
27377 7754 Acked Read 3 Sent Done Done false
27378 7755 Acked Write 3 Sent Done Done false
27379 7756 Acked Write 3 Sent Done Done false
27380 7757 Acked Read 3 Sent Done Done false
27381 7758 Acked Flush 0 Sent Done Done false
27382 7759 Acked Write 4 Sent Done Done false
27383 7760 Acked Read 8 Sent Done Done false
27384 7761 Acked Read 1 Sent Done Done false
27385 7762 Acked Write 9 Sent Done Done false
27386 7763 Acked Read 7 Sent Done Done false
27387 7764 Acked Read 1 Sent Done Done false
27388 7765 Acked Write 4 Sent Done Done false
27389 7766 Acked Read 2 Sent Done Done false
27390 7767 Acked Read 6 Sent Done Done false
27391 7768 Acked Read 1 Sent Done Done false
27392 7769 Acked Read 7 Sent Done Done false
27393 7770 Acked Read 2 Sent Done Done false
27394 7771 Acked Write 9 Sent Done Done false
27395 7772 Acked Read 5 Sent Done Done false
27396 7773 Acked Flush 0 Sent Done Done false
27397 7774 Acked Read 2 Sent Done Done false
27398 7775 Acked Flush 0 Sent Done Done false
27399 7776 Acked Read 6 Sent Done Done false
27400 7777 Acked Write 6 Sent Done Done false
27401 7778 Acked Flush 0 Sent Done Done false
27402 7779 Acked Read 10 Sent Done Done false
27403 7780 Acked Write 9 Sent Done Done false
27404 7781 Acked Read 7 Sent Done Done false
27405 7782 Acked Read 5 Sent Done Done false
27406 7783 Acked Write 10 Sent Done Done false
27407 7784 Acked Write 10 Sent Done Done false
27408 7785 Acked Read 1 Sent Done Done false
27409 7786 Acked Read 9 Sent Done Done false
27410 7787 Acked Write 7 Sent Done Done false
27411 7788 Acked Write 10 Sent Done Done false
27412 7789 Acked Read 3 Sent Done Done false
27413 7790 Acked Flush 0 Sent Done Done false
27414 7791 Acked Flush 0 Sent Done Done false
27415 7792 Acked Write 3 Sent Done Done false
27416 7793 Acked Read 7 Sent Done Done false
27417 7794 Acked Read 9 Sent Done Done false
27418 7795 Acked Write 6 Sent Done Done false
27419 7796 Acked Read 7 Sent Done Done false
27420 7797 Acked Flush 0 Sent Done Done false
27421 7798 Acked Read 5 Sent Done Done false
27422 7799 Acked Flush 0 Sent Done Done false
27423 7800 Acked Read 6 Sent Done Done false
27424 7801 Acked Write 6 Sent Done Done false
27425 7802 Acked Read 3 Sent Done Done false
27426 7803 Acked Write 4 Sent Done Done false
27427 7804 Acked Write 9 Sent Done Done false
27428 7805 Acked Read 3 Sent Done Done false
27429 7806 Acked Write 10 Sent Done Done false
27430 7807 Acked Read 2 Sent Done Done false
27431 7808 Acked Read 4 Sent Done Done false
27432 7809 Acked Write 2 Sent Done Done false
27433 7810 Acked Read 9 Sent Done Done false
27434 7811 Acked Read 9 Sent Done Done false
27435 7812 Acked Read 6 Sent Done Done false
27436 7813 Acked Write 2 Sent Done Done false
27437 7814 Acked Read 2 Sent Done Done false
27438 7815 Acked Flush 0 Sent Done Done false
27439 7816 Acked Read 9 Sent Done Done false
27440 7817 Acked Write 9 Sent Done Done false
27441 7818 Acked Write 1 Sent Done Done false
27442 7819 Acked Read 2 Sent Done Done false
27443 7820 Acked Read 9 Sent Done Done false
27444 7821 Acked Read 7 Sent Done Done false
27445 7822 Acked Write 3 Sent Done Done false
27446 7823 Acked Flush 0 Sent Done Done false
27447 7824 Acked Read 9 Sent Done Done false
27448 7825 Acked Write 5 Sent Done Done false
27449 7826 Acked Read 8 Sent Done Done false
27450 7827 Acked Flush 0 Sent Done Done false
27451 7828 Acked Write 1 Sent Done Done false
27452 7829 Acked Read 7 Sent Done Done false
27453 7830 Acked Read 2 Sent Done Done false
27454 7831 Acked Flush 0 Sent Done Done false
27455 7832 Acked Read 1 Sent Done Done false
27456 7833 Acked Write 7 Sent Done Done false
27457 7834 Acked Write 5 Sent Done Done false
27458 7835 Acked Read 6 Sent Done Done false
27459 7836 Acked Read 5 Sent Done Done false
27460 7837 Acked Read 7 Sent Done Done false
27461 7838 Acked Write 4 Sent Done Done false
27462 7839 Acked Write 6 Sent Done Done false
27463 7840 Acked Read 2 Sent Done Done false
27464 7841 Acked Read 7 Sent Done Done false
27465 7842 Acked Read 3 Sent Done Done false
27466 7843 Acked Flush 0 Sent Done Done false
27467 7844 Acked Read 7 Sent Done Done false
27468 7845 Acked Read 9 Sent Done Done false
27469 7846 Acked Read 4 Sent Done Done false
27470 7847 Acked Write 1 Sent Done Done false
27471 7848 Acked Write 7 Sent Done Done false
27472 7849 Acked Flush 0 Sent Done Done false
27473 7850 Acked Write 5 Sent Done Done false
27474 7851 Acked Flush 0 Sent Done Done false
27475 7852 Acked Read 3 Sent Done Done false
27476 7853 Acked Flush 0 Sent Done Done false
27477 7854 Acked Read 3 Sent Done Done false
27478 7855 Acked Read 7 Sent Done Done false
27479 7856 Acked Read 4 Sent Done Done false
27480 7857 Acked Write 7 Sent Done Done false
27481 7858 Acked Write 8 Sent Done Done false
27482 7859 Acked Flush 0 Sent Done Done false
27483 7860 Acked Read 5 Sent Done Done false
27484 7861 Acked Read 2 Sent Done Done false
27485 7862 Acked Read 10 Sent Done Done false
27486 7863 Acked Read 1 Sent Done Done false
27487 7864 Acked Read 2 Sent Done Done false
27488 7865 Acked Read 9 Sent Done Done false
27489 7866 Acked Read 8 Sent Done Done false
27490 7867 Acked Read 5 Sent Done Done false
27491 7868 Acked Read 10 Sent Done Done false
27492 7869 Acked Write 8 Sent Done Done false
27493 7870 Acked Write 3 Sent Done Done false
27494 7871 Acked Write 9 Sent Done Done false
27495 7872 Acked Write 2 Sent Done Done false
27496 7873 Acked Read 4 Sent Done Done false
27497 7874 Acked Write 9 Sent Done Done false
27498 7875 Acked Read 3 Sent Done Done false
27499 7876 Acked Read 2 Sent Done Done false
27500 7877 Acked Write 1 Sent Done Done false
27501 7878 Acked Read 1 Sent Done Done false
27502 7879 Acked Read 3 Sent Done Done false
27503 7880 Acked Read 10 Sent Done Done false
27504 7881 Acked Write 9 Sent Done Done false
27505 7882 Acked Write 8 Sent Done Done false
27506 7883 Acked Read 2 Sent Done Done false
27507 7884 Acked Write 10 Sent Done Done false
27508 7885 Acked Read 5 Sent Done Done false
27509 7886 Acked Write 5 Sent Done Done false
27510 7887 Acked Flush 0 Sent Done Done false
27511 7888 Acked Read 6 Sent Done Done false
27512 7889 Acked Flush 0 Sent Done Done false
27513 7890 Acked Write 8 Sent Done Done false
27514 7891 Acked Read 5 Sent Done Done false
27515 7892 Acked Write 8 Sent Done Done false
27516 7893 Acked Read 10 Sent Done Done false
27517 7894 Acked Read 3 Sent Done Done false
27518 7895 Acked Read 10 Sent Done Done false
27519 7896 Acked Read 5 Sent Done Done false
27520 7897 Acked Write 5 Sent Done Done false
27521 7898 Acked Read 2 Sent Done Done false
27522 7899 Acked Read 10 Sent Done Done false
27523 7900 Acked Read 8 Sent Done Done false
27524 7901 Acked Read 8 Sent Done Done false
27525 7902 Acked Read 2 Sent Done Done false
27526 7903 Acked Write 5 Sent Done Done false
27527 7904 Acked Read 8 Sent Done Done false
27528 7905 Acked Read 2 Sent Done Done false
27529 7906 Acked Write 2 Sent Done Done false
27530 7907 Acked Flush 0 Sent Done Done false
27531 7908 Acked Read 6 Sent Done Done false
27532 7909 Acked Read 5 Sent Done Done false
27533 7910 Acked Write 8 Sent Done Done false
27534 7911 Acked Read 3 Sent Done Done false
27535 7912 Acked Read 5 Sent Done Done false
27536 7913 Acked Write 10 Sent Done Done false
27537 7914 Acked Flush 0 Sent Done Done false
27538 7915 Acked Read 3 Sent Done Done false
27539 7916 Acked Write 8 Sent Done Done false
27540 7917 Acked Write 6 Sent Done Done false
27541 7918 Acked Flush 0 Sent Done Done false
27542 7919 Acked Read 5 Sent Done Done false
27543 7920 Acked Write 8 Sent Done Done false
27544 7921 Acked Write 4 Sent Done Done false
27545 7922 Acked Read 10 Sent Done Done false
27546 7923 Acked Read 6 Sent Done Done false
27547 7924 Acked Read 1 Sent Done Done false
27548 7925 Acked Read 9 Sent Done Done false
27549 7926 Acked Read 3 Sent Done Done false
27550 7927 Acked Read 1 Sent Done Done false
27551 7928 Acked Write 3 Sent Done Done false
27552 7929 Acked Read 1 Sent Done Done false
27553 7930 Acked Read 5 Sent Done Done false
27554 7931 Acked Write 8 Sent Done Done false
27555 7932 Acked Read 6 Sent Done Done false
27556 7933 Acked Read 5 Sent Done Done false
27557 7934 Acked Flush 0 Sent Done Done false
27558 7935 Acked Write 7 Sent Done Done false
27559 7936 Acked Read 6 Sent Done Done false
27560 7937 Acked Flush 0 Sent Done Done false
27561 7938 Acked Flush 0 Sent Done Done false
27562 7939 Acked Read 1 Sent Done Done false
27563 7940 Acked Read 9 Sent Done Done false
27564 7941 Acked Read 1 Sent Done Done false
27565 STATES DS:0 DS:1 DS:2 TOTAL
27566 Sent 301 0 0 301
27567 Done 0 301 301 602
27568 Skipped 0 0 0 0
27569 Error 0 0 0 0
27570 Last Flush: 7640 7938 7938
27571 Downstairs last five completed: 7640 7639 7638 7637 7636
27572 Upstairs last five completed: 7941 7940 7939 7938 7937
27573 CLIENT: Up:0 ds:301 act:2
27574 [8810] got start action
27575 [8810] I am going to restart
27576 Starting downstairs at port 8810
27577 Make output file at "/tmp/dsc/downstairs-8810.txt"
27578 [8810][0] reports Starting
27579 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1223)
27580 [8810][0] reports Running
27581 ----------------------------------------------------------------
27582 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27583 States: Offline Active Active
27584 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27585 7641 Acked Write 10 Sent Done Done false
27586 7642 Acked Write 2 Sent Done Done false
27587 7643 Acked Write 1 Sent Done Done false
27588 7644 Acked Read 2 Sent Done Done false
27589 7645 Acked Read 8 Sent Done Done false
27590 7646 Acked Write 5 Sent Done Done false
27591 7647 Acked Write 8 Sent Done Done false
27592 7648 Acked Read 7 Sent Done Done false
27593 7649 Acked Write 5 Sent Done Done false
27594 7650 Acked Read 1 Sent Done Done false
27595 7651 Acked Read 6 Sent Done Done false
27596 7652 Acked Read 4 Sent Done Done false
27597 7653 Acked Flush 0 Sent Done Done false
27598 7654 Acked Write 6 Sent Done Done false
27599 7655 Acked Read 7 Sent Done Done false
27600 7656 Acked Read 10 Sent Done Done false
27601 7657 Acked Flush 0 Sent Done Done false
27602 7658 Acked Write 5 Sent Done Done false
27603 7659 Acked Read 1 Sent Done Done false
27604 7660 Acked Write 2 Sent Done Done false
27605 7661 Acked Flush 0 Sent Done Done false
27606 7662 Acked Write 4 Sent Done Done false
27607 7663 Acked Read 9 Sent Done Done false
27608 7664 Acked Flush 0 Sent Done Done false
27609 7665 Acked Read 1 Sent Done Done false
27610 7666 Acked Read 7 Sent Done Done false
27611 7667 Acked Flush 0 Sent Done Done false
27612 7668 Acked Read 6 Sent Done Done false
27613 7669 Acked Write 6 Sent Done Done false
27614 7670 Acked Write 6 Sent Done Done false
27615 7671 Acked Read 5 Sent Done Done false
27616 7672 Acked Write 6 Sent Done Done false
27617 7673 Acked Read 7 Sent Done Done false
27618 7674 Acked Read 4 Sent Done Done false
27619 7675 Acked Write 8 Sent Done Done false
27620 7676 Acked Write 5 Sent Done Done false
27621 7677 Acked Read 10 Sent Done Done false
27622 7678 Acked Write 7 Sent Done Done false
27623 7679 Acked Read 6 Sent Done Done false
27624 7680 Acked Read 7 Sent Done Done false
27625 7681 Acked Read 9 Sent Done Done false
27626 7682 Acked Write 8 Sent Done Done false
27627 7683 Acked Write 2 Sent Done Done false
27628 7684 Acked Read 8 Sent Done Done false
27629 7685 Acked Write 8 Sent Done Done false
27630 7686 Acked Write 7 Sent Done Done false
27631 7687 Acked Read 3 Sent Done Done false
27632 7688 Acked Read 3 Sent Done Done false
27633 7689 Acked Read 4 Sent Done Done false
27634 7690 Acked Write 7 Sent Done Done false
27635 7691 Acked Write 7 Sent Done Done false
27636 7692 Acked Read 5 Sent Done Done false
27637 7693 Acked Read 7 Sent Done Done false
27638 7694 Acked Write 2 Sent Done Done false
27639 7695 Acked Read 6 Sent Done Done false
27640 7696 Acked Read 8 Sent Done Done false
27641 7697 Acked Write 1 Sent Done Done false
27642 7698 Acked Write 2 Sent Done Done false
27643 7699 Acked Write 3 Sent Done Done false
27644 7700 Acked Write 2 Sent Done Done false
27645 7701 Acked Write 1 Sent Done Done false
27646 7702 Acked Write 7 Sent Done Done false
27647 7703 Acked Read 8 Sent Done Done false
27648 7704 Acked Read 4 Sent Done Done false
27649 7705 Acked Write 8 Sent Done Done false
27650 7706 Acked Read 6 Sent Done Done false
27651 7707 Acked Write 9 Sent Done Done false
27652 7708 Acked Write 4 Sent Done Done false
27653 7709 Acked Read 1 Sent Done Done false
27654 7710 Acked Write 1 Sent Done Done false
27655 7711 Acked Read 4 Sent Done Done false
27656 7712 Acked Read 1 Sent Done Done false
27657 7713 Acked Read 8 Sent Done Done false
27658 7714 Acked Read 2 Sent Done Done false
27659 7715 Acked Read 5 Sent Done Done false
27660 7716 Acked Read 2 Sent Done Done false
27661 7717 Acked Read 4 Sent Done Done false
27662 7718 Acked Write 2 Sent Done Done false
27663 7719 Acked Write 9 Sent Done Done false
27664 7720 Acked Write 9 Sent Done Done false
27665 7721 Acked Read 1 Sent Done Done false
27666 7722 Acked Flush 0 Sent Done Done false
27667 7723 Acked Read 4 Sent Done Done false
27668 7724 Acked Read 6 Sent Done Done false
27669 7725 Acked Write 8 Sent Done Done false
27670 7726 Acked Flush 0 Sent Done Done false
27671 7727 Acked Read 3 Sent Done Done false
27672 7728 Acked Write 9 Sent Done Done false
27673 7729 Acked Read 9 Sent Done Done false
27674 7730 Acked Read 4 Sent Done Done false
27675 7731 Acked Write 7 Sent Done Done false
27676 7732 Acked Read 1 Sent Done Done false
27677 7733 Acked Write 6 Sent Done Done false
27678 7734 Acked Write 4 Sent Done Done false
27679 7735 Acked Write 10 Sent Done Done false
27680 7736 Acked Read 6 Sent Done Done false
27681 7737 Acked Read 1 Sent Done Done false
27682 7738 Acked Read 1 Sent Done Done false
27683 7739 Acked Read 9 Sent Done Done false
27684 7740 Acked Write 1 Sent Done Done false
27685 7741 Acked Read 10 Sent Done Done false
27686 7742 Acked Write 1 Sent Done Done false
27687 7743 Acked Write 4 Sent Done Done false
27688 7744 Acked Write 8 Sent Done Done false
27689 7745 Acked Read 8 Sent Done Done false
27690 7746 Acked Write 6 Sent Done Done false
27691 7747 Acked Read 6 Sent Done Done false
27692 7748 Acked Read 8 Sent Done Done false
27693 7749 Acked Flush 0 Sent Done Done false
27694 7750 Acked Write 2 Sent Done Done false
27695 7751 Acked Read 4 Sent Done Done false
27696 7752 Acked Read 2 Sent Done Done false
27697 7753 Acked Write 3 Sent Done Done false
27698 7754 Acked Read 3 Sent Done Done false
27699 7755 Acked Write 3 Sent Done Done false
27700 7756 Acked Write 3 Sent Done Done false
27701 7757 Acked Read 3 Sent Done Done false
27702 7758 Acked Flush 0 Sent Done Done false
27703 7759 Acked Write 4 Sent Done Done false
27704 7760 Acked Read 8 Sent Done Done false
27705 7761 Acked Read 1 Sent Done Done false
27706 7762 Acked Write 9 Sent Done Done false
27707 7763 Acked Read 7 Sent Done Done false
27708 7764 Acked Read 1 Sent Done Done false
27709 7765 Acked Write 4 Sent Done Done false
27710 7766 Acked Read 2 Sent Done Done false
27711 7767 Acked Read 6 Sent Done Done false
27712 7768 Acked Read 1 Sent Done Done false
27713 7769 Acked Read 7 Sent Done Done false
27714 7770 Acked Read 2 Sent Done Done false
27715 7771 Acked Write 9 Sent Done Done false
27716 7772 Acked Read 5 Sent Done Done false
27717 7773 Acked Flush 0 Sent Done Done false
27718 7774 Acked Read 2 Sent Done Done false
27719 7775 Acked Flush 0 Sent Done Done false
27720 7776 Acked Read 6 Sent Done Done false
27721 7777 Acked Write 6 Sent Done Done false
27722 7778 Acked Flush 0 Sent Done Done false
27723 7779 Acked Read 10 Sent Done Done false
27724 7780 Acked Write 9 Sent Done Done false
27725 7781 Acked Read 7 Sent Done Done false
27726 7782 Acked Read 5 Sent Done Done false
27727 7783 Acked Write 10 Sent Done Done false
27728 7784 Acked Write 10 Sent Done Done false
27729 7785 Acked Read 1 Sent Done Done false
27730 7786 Acked Read 9 Sent Done Done false
27731 7787 Acked Write 7 Sent Done Done false
27732 7788 Acked Write 10 Sent Done Done false
27733 7789 Acked Read 3 Sent Done Done false
27734 7790 Acked Flush 0 Sent Done Done false
27735 7791 Acked Flush 0 Sent Done Done false
27736 7792 Acked Write 3 Sent Done Done false
27737 7793 Acked Read 7 Sent Done Done false
27738 7794 Acked Read 9 Sent Done Done false
27739 7795 Acked Write 6 Sent Done Done false
27740 7796 Acked Read 7 Sent Done Done false
27741 7797 Acked Flush 0 Sent Done Done false
27742 7798 Acked Read 5 Sent Done Done false
27743 7799 Acked Flush 0 Sent Done Done false
27744 7800 Acked Read 6 Sent Done Done false
27745 7801 Acked Write 6 Sent Done Done false
27746 7802 Acked Read 3 Sent Done Done false
27747 7803 Acked Write 4 Sent Done Done false
27748 7804 Acked Write 9 Sent Done Done false
27749 7805 Acked Read 3 Sent Done Done false
27750 7806 Acked Write 10 Sent Done Done false
27751 7807 Acked Read 2 Sent Done Done false
27752 7808 Acked Read 4 Sent Done Done false
27753 7809 Acked Write 2 Sent Done Done false
27754 7810 Acked Read 9 Sent Done Done false
27755 7811 Acked Read 9 Sent Done Done false
27756 7812 Acked Read 6 Sent Done Done false
27757 7813 Acked Write 2 Sent Done Done false
27758 7814 Acked Read 2 Sent Done Done false
27759 7815 Acked Flush 0 Sent Done Done false
27760 7816 Acked Read 9 Sent Done Done false
27761 7817 Acked Write 9 Sent Done Done false
27762 7818 Acked Write 1 Sent Done Done false
27763 7819 Acked Read 2 Sent Done Done false
27764 7820 Acked Read 9 Sent Done Done false
27765 7821 Acked Read 7 Sent Done Done false
27766 7822 Acked Write 3 Sent Done Done false
27767 7823 Acked Flush 0 Sent Done Done false
27768 7824 Acked Read 9 Sent Done Done false
27769 7825 Acked Write 5 Sent Done Done false
27770 7826 Acked Read 8 Sent Done Done false
27771 7827 Acked Flush 0 Sent Done Done false
27772 7828 Acked Write 1 Sent Done Done false
27773 7829 Acked Read 7 Sent Done Done false
27774 7830 Acked Read 2 Sent Done Done false
27775 7831 Acked Flush 0 Sent Done Done false
27776 7832 Acked Read 1 Sent Done Done false
27777 7833 Acked Write 7 Sent Done Done false
27778 7834 Acked Write 5 Sent Done Done false
27779 7835 Acked Read 6 Sent Done Done false
27780 7836 Acked Read 5 Sent Done Done false
27781 7837 Acked Read 7 Sent Done Done false
27782 7838 Acked Write 4 Sent Done Done false
27783 7839 Acked Write 6 Sent Done Done false
27784 7840 Acked Read 2 Sent Done Done false
27785 7841 Acked Read 7 Sent Done Done false
27786 7842 Acked Read 3 Sent Done Done false
27787 7843 Acked Flush 0 Sent Done Done false
27788 7844 Acked Read 7 Sent Done Done false
27789 7845 Acked Read 9 Sent Done Done false
27790 7846 Acked Read 4 Sent Done Done false
27791 7847 Acked Write 1 Sent Done Done false
27792 7848 Acked Write 7 Sent Done Done false
27793 7849 Acked Flush 0 Sent Done Done false
27794 7850 Acked Write 5 Sent Done Done false
27795 7851 Acked Flush 0 Sent Done Done false
27796 7852 Acked Read 3 Sent Done Done false
27797 7853 Acked Flush 0 Sent Done Done false
27798 7854 Acked Read 3 Sent Done Done false
27799 7855 Acked Read 7 Sent Done Done false
27800 7856 Acked Read 4 Sent Done Done false
27801 7857 Acked Write 7 Sent Done Done false
27802 7858 Acked Write 8 Sent Done Done false
27803 7859 Acked Flush 0 Sent Done Done false
27804 7860 Acked Read 5 Sent Done Done false
27805 7861 Acked Read 2 Sent Done Done false
27806 7862 Acked Read 10 Sent Done Done false
27807 7863 Acked Read 1 Sent Done Done false
27808 7864 Acked Read 2 Sent Done Done false
27809 7865 Acked Read 9 Sent Done Done false
27810 7866 Acked Read 8 Sent Done Done false
27811 7867 Acked Read 5 Sent Done Done false
27812 7868 Acked Read 10 Sent Done Done false
27813 7869 Acked Write 8 Sent Done Done false
27814 7870 Acked Write 3 Sent Done Done false
27815 7871 Acked Write 9 Sent Done Done false
27816 7872 Acked Write 2 Sent Done Done false
27817 7873 Acked Read 4 Sent Done Done false
27818 7874 Acked Write 9 Sent Done Done false
27819 7875 Acked Read 3 Sent Done Done false
27820 7876 Acked Read 2 Sent Done Done false
27821 7877 Acked Write 1 Sent Done Done false
27822 7878 Acked Read 1 Sent Done Done false
27823 7879 Acked Read 3 Sent Done Done false
27824 7880 Acked Read 10 Sent Done Done false
27825 7881 Acked Write 9 Sent Done Done false
27826 7882 Acked Write 8 Sent Done Done false
27827 7883 Acked Read 2 Sent Done Done false
27828 7884 Acked Write 10 Sent Done Done false
27829 7885 Acked Read 5 Sent Done Done false
27830 7886 Acked Write 5 Sent Done Done false
27831 7887 Acked Flush 0 Sent Done Done false
27832 7888 Acked Read 6 Sent Done Done false
27833 7889 Acked Flush 0 Sent Done Done false
27834 7890 Acked Write 8 Sent Done Done false
27835 7891 Acked Read 5 Sent Done Done false
27836 7892 Acked Write 8 Sent Done Done false
27837 7893 Acked Read 10 Sent Done Done false
27838 7894 Acked Read 3 Sent Done Done false
27839 7895 Acked Read 10 Sent Done Done false
27840 7896 Acked Read 5 Sent Done Done false
27841 7897 Acked Write 5 Sent Done Done false
27842 7898 Acked Read 2 Sent Done Done false
27843 7899 Acked Read 10 Sent Done Done false
27844 7900 Acked Read 8 Sent Done Done false
27845 7901 Acked Read 8 Sent Done Done false
27846 7902 Acked Read 2 Sent Done Done false
27847 7903 Acked Write 5 Sent Done Done false
27848 7904 Acked Read 8 Sent Done Done false
27849 7905 Acked Read 2 Sent Done Done false
27850 7906 Acked Write 2 Sent Done Done false
27851 7907 Acked Flush 0 Sent Done Done false
27852 7908 Acked Read 6 Sent Done Done false
27853 7909 Acked Read 5 Sent Done Done false
27854 7910 Acked Write 8 Sent Done Done false
27855 7911 Acked Read 3 Sent Done Done false
27856 7912 Acked Read 5 Sent Done Done false
27857 7913 Acked Write 10 Sent Done Done false
27858 7914 Acked Flush 0 Sent Done Done false
27859 7915 Acked Read 3 Sent Done Done false
27860 7916 Acked Write 8 Sent Done Done false
27861 7917 Acked Write 6 Sent Done Done false
27862 7918 Acked Flush 0 Sent Done Done false
27863 7919 Acked Read 5 Sent Done Done false
27864 7920 Acked Write 8 Sent Done Done false
27865 7921 Acked Write 4 Sent Done Done false
27866 7922 Acked Read 10 Sent Done Done false
27867 7923 Acked Read 6 Sent Done Done false
27868 7924 Acked Read 1 Sent Done Done false
27869 7925 Acked Read 9 Sent Done Done false
27870 7926 Acked Read 3 Sent Done Done false
27871 7927 Acked Read 1 Sent Done Done false
27872 7928 Acked Write 3 Sent Done Done false
27873 7929 Acked Read 1 Sent Done Done false
27874 7930 Acked Read 5 Sent Done Done false
27875 7931 Acked Write 8 Sent Done Done false
27876 7932 Acked Read 6 Sent Done Done false
27877 7933 Acked Read 5 Sent Done Done false
27878 7934 Acked Flush 0 Sent Done Done false
27879 7935 Acked Write 7 Sent Done Done false
27880 7936 Acked Read 6 Sent Done Done false
27881 7937 Acked Flush 0 Sent Done Done false
27882 7938 Acked Flush 0 Sent Done Done false
27883 7939 Acked Read 1 Sent Done Done false
27884 7940 Acked Read 9 Sent Done Done false
27885 7941 Acked Read 1 Sent Done Done false
27886 7942 Acked Flush 0 Sent Done Done false
27887 STATES DS:0 DS:1 DS:2 TOTAL
27888 Sent 302 0 0 302
27889 Done 0 302 302 604
27890 Skipped 0 0 0 0
27891 Error 0 0 0 0
27892 Last Flush: 7640 7942 7942
27893 Downstairs last five completed: 7640 7639 7638 7637 7636
27894 Upstairs last five completed: 7942 7941 7940 7939 7938
27895 CLIENT: Up:0 ds:302 act:2
278962025-01-17T21:16:55.915ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
278972025-01-17T21:16:55.915ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64250) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
278982025-01-17T21:16:55.916ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
278992025-01-17T21:16:55.917ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
279002025-01-17T21:16:55.917ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
279012025-01-17T21:16:55.917ZINFOcrucible: send last flush ID to this DS: 7640 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
279022025-01-17T21:16:55.917ZINFOcrucible: Replied this last flush ID: 7640 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
279032025-01-17T21:16:55.917ZINFOcrucible: [0] client re-new 302 jobs since flush 7640 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
279042025-01-17T21:16:55.918ZINFOcrucible: [0] Replayed 302 jobs since flush: 7640 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
27905 ----------------------------------------------------------------
27906 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
27907 Downstairs last five completed: 7942 7941 7940 7939 7938
27908 Upstairs last five completed: 7942 7941 7940 7939 7938
27909 CLIENT: Up:0 ds:0 act:3
27910 Replay: All jobs finished
27911 Main task has work to do, go find it
27912 got dsc Stop(2)
27913 stop 2
27914 [8830] Got stop action so:false kr:false
27915 Jan 17 21:16:58.165 INFO request completed, latency_us: 365, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 8fc81bbb-fb2f-461e-b9d1-890607172a0f, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
279162025-01-17T21:16:58.166ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
279172025-01-17T21:16:58.166ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
279182025-01-17T21:16:58.166ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
27919 Jan 17 21:16:58.166 INFO request completed, latency_us: 158, response_code: 200, uri: /state/cid/2, method: GET, req_id: 8e2b6c71-da53-4442-9fd8-c26973015af6, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
279202025-01-17T21:16:58.166ZINFOcrucible: client task is exiting = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
27921 [8830] Exited with: ExitStatus(unix_wait_status(9))
27922 [8830][2] reports Exit
27923 Jan 17 21:17:02.168 INFO request completed, latency_us: 208, response_code: 200, uri: /state/cid/2, method: GET, req_id: a789252b-e43f-48a7-94b4-2fd8739a15cd, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
27924 001/300 Write block 4408 len 28672 data: 6 5 4 4 8 9 8
27925 002/300 Flush
27926 003/300 Write block 1629 len 4096 data: 5
27927 004/300 Write block 128 len 28672 data: 2 2 2 2 2 4 5
27928 005/300 Write block 279 len 8192 data: 5 7
27929 006/300 Read block 2639 len 16384
27930 007/300 Read block 4633 len 20480
27931 008/300 Write block 2987 len 28672 data: 5 5 5 5 5 5 5
27932 009/300 Read block 1435 len 4096
27933 010/300 Write block 3442 len 28672 data: 5 5 6 10 11 10 9
27934 011/300 Read block 3824 len 32768
27935 012/300 Write block 2956 len 4096 data: 3
27936 013/300 Write block 3245 len 20480 data: 4 4 3 3 3
27937 014/300 Read block 4587 len 28672
27938 015/300 Read block 2207 len 4096
27939 016/300 Flush
27940 017/300 Read block 1075 len 28672
27941 018/300 Read block 2308 len 40960
27942 019/300 Write block 2325 len 20480 data: 6 5 5 4 5
27943 020/300 Write block 3841 len 28672 data: 7 9 7 5 3 3 3
27944 021/300 Read block 655 len 40960
27945 022/300 Read block 2465 len 36864
27946 023/300 Read block 2207 len 12288
27947 024/300 Write block 611 len 24576 data: 5 6 4 4 4 3
27948 025/300 Read block 730 len 28672
27949 026/300 Read block 2274 len 8192
27950 027/300 Write block 4882 len 4096 data: 6
27951 028/300 Read block 575 len 4096
27952 029/300 Write block 3244 len 36864 data: 4 5 5 4 4 4 4 6 6
27953 030/300 Write block 4431 len 24576 data: 4 4 3 4 5 5
27954 031/300 Read block 2494 len 8192
27955 032/300 Read block 4829 len 40960
27956 033/300 Write block 1095 len 28672 data: 4 5 5 6 7 6 6
27957 034/300 Flush
27958 035/300 Write block 1137 len 4096 data: 4
27959 036/300 Read block 3753 len 4096
27960 037/300 Write block 4301 len 20480 data: 7 6 5 4 3
27961 038/300 Read block 1257 len 20480
27962 039/300 Read block 3518 len 28672
27963 040/300 Read block 4135 len 36864
27964 041/300 Flush
27965 042/300 Write block 3841 len 28672 data: 8 10 8 6 4 4 4
27966 043/300 Write block 875 len 28672 data: 5 5 7 8 7 8 8
27967 044/300 Read block 721 len 28672
27968 045/300 Write block 1396 len 8192 data: 6 7
27969 046/300 Read block 3719 len 28672
27970 047/300 Write block 1332 len 36864 data: 7 5 4 4 5 5 3 3 3
27971 048/300 Read block 1534 len 36864
27972 049/300 Write block 567 len 4096 data: 3
27973 050/300 Write block 4418 len 12288 data: 11 8 6
27974 051/300 Write block 4430 len 32768 data: 6 5 5 4 5 6 6 6
27975 052/300 Write block 3146 len 28672 data: 4 4 5 5 5 5 5
27976 053/300 Write block 3780 len 36864 data: 6 7 9 9 7 8 9 9 9
27977 054/300 Flush
27978 055/300 Read block 531 len 28672
27979 056/300 Read block 4193 len 36864
27980 057/300 Read block 3343 len 36864
27981 058/300 Read block 1350 len 12288
27982 059/300 Read block 182 len 12288
27983 060/300 Write block 3447 len 24576 data: 11 10 8 9 8 7
27984 061/300 Write block 2627 len 28672 data: 5 6 5 6 6 4 4
27985 062/300 Read block 4950 len 32768
27986 063/300 Write block 39 len 12288 data: 4 4 4
27987 064/300 Write block 2151 len 28672 data: 7 8 6 6 7 7 5
27988 065/300 Read block 4466 len 8192
27989 066/300 Read block 4269 len 28672
27990 067/300 Write block 177 len 20480 data: 5 5 5 6 5
27991 068/300 Read block 2405 len 12288
27992 069/300 Read block 1199 len 24576
27993 070/300 Write block 9 len 16384 data: 2 2 3 5
27994 071/300 Read block 2234 len 40960
27995 072/300 Flush
27996 073/300 Read block 3216 len 4096
27997 074/300 Write block 3180 len 36864 data: 5 4 3 3 3 3 5 5 5
27998 075/300 Flush
27999 076/300 Read block 1393 len 24576
28000 077/300 Read block 1546 len 32768
28001 078/300 Flush
28002 079/300 Read block 2987 len 32768
28003 080/300 Read block 3341 len 4096
28004 081/300 Flush
28005 082/300 Read block 2810 len 36864
28006 083/300 Write block 735 len 4096 data: 6
28007 084/300 Read block 4356 len 40960
28008 085/300 Read block 1741 len 16384
28009 086/300 Flush
28010 087/300 Flush
28011 088/300 Write block 4516 len 36864 data: 3 4 4 2 2 3 3 3 4
28012 089/300 Read block 4403 len 28672
28013 090/300 Read block 3005 len 12288
28014 091/300 Read block 1633 len 8192
28015 092/300 Write block 4719 len 8192 data: 5 5
28016 093/300 Read block 2171 len 36864
28017 094/300 Write block 2445 len 36864 data: 5 5 6 6 6 4 3 3 3
28018 095/300 Read block 2264 len 12288
28019 096/300 Read block 3857 len 16384
28020 097/300 Write block 900 len 32768 data: 6 7 6 5 4 5 4 4
28021 098/300 Read block 4502 len 40960
28022 099/300 Write block 2789 len 16384 data: 7 10 10 10
28023 100/300 Write block 3327 len 36864 data: 3 5 5 6 5 5 6 6 6
28024 101/300 Write block 914 len 36864 data: 4 5 5 4 5 7 7 5 7
28025 102/300 Read block 3083 len 24576
28026 103/300 Write block 3339 len 4096 data: 4
28027 104/300 Read block 4251 len 36864
28028 105/300 Write block 1407 len 28672 data: 2 2 2 2 2 2 2
28029 106/300 Write block 3295 len 20480 data: 6 7 8 8 8
28030 107/300 Read block 1048 len 24576
28031 108/300 Write block 3362 len 8192 data: 5 6
28032 109/300 Write block 4872 len 4096 data: 4
28033 110/300 Read block 3855 len 28672
28034 111/300 Write block 3587 len 24576 data: 3 4 5 5 4 4
28035 112/300 Write block 1274 len 40960 data: 4 3 3 4 3 3 3 3 3 3
28036 113/300 Flush
28037 114/300 Write block 2739 len 32768 data: 6 5 5 6 7 6 6 5
28038 115/300 Write block 3389 len 16384 data: 3 3 3 4
28039 116/300 Read block 3530 len 32768
28040 117/300 Read block 3314 len 28672
28041 118/300 Read block 4031 len 16384
28042 119/300 Flush
28043 120/300 Read block 409 len 16384
28044 121/300 Read block 3803 len 24576
28045 122/300 Write block 3956 len 40960 data: 3 3 3 4 4 3 4 4 4 5
28046 123/300 Write block 4815 len 20480 data: 7 5 5 6 6
28047 124/300 Read block 2635 len 12288
28048 125/300 Read block 3628 len 20480
28049 126/300 Write block 600 len 12288 data: 3 3 4
28050 127/300 Flush
28051 128/300 Read block 2569 len 28672
28052 129/300 Read block 4762 len 16384
28053 130/300 Write block 4716 len 24576 data: 6 7 5 6 6 5
28054 131/300 Read block 183 len 24576
28055 132/300 Read block 2743 len 36864
28056 133/300 Read block 2479 len 24576
28057 134/300 Read block 1924 len 4096
28058 135/300 Read block 3195 len 40960
28059 136/300 Flush
28060 137/300 Flush
28061 138/300 Write block 1590 len 8192 data: 8 9
28062 139/300 Flush
28063 140/300 Write block 513 len 20480 data: 5 4 4 4 5
28064 141/300 Write block 4839 len 24576 data: 4 3 4 3 6 7
28065 142/300 Read block 412 len 20480
28066 143/300 Write block 4422 len 24576 data: 6 6 8 6 7 7
28067 144/300 Read block 4757 len 40960
28068 145/300 Write block 2287 len 28672 data: 6 7 7 7 9 9 8
28069 146/300 Read block 4301 len 12288
28070 147/300 Read block 4865 len 20480
28071 148/300 Read block 3834 len 20480
28072 149/300 Read block 274 len 36864
28073 150/300 Read block 3553 len 20480
28074 151/300 Write block 1507 len 24576 data: 5 5 5 3 4 3
28075 152/300 Write block 3926 len 40960 data: 3 3 3 5 6 8 8 10 9 9
28076 153/300 Flush
28077 154/300 Write block 2201 len 4096 data: 5
28078 155/300 Write block 3781 len 28672 data: 8 10 10 8 9 10 10
28079 156/300 Read block 1838 len 32768
28080 157/300 Write block 2962 len 8192 data: 4 3
28081 158/300 Read block 867 len 8192
28082 159/300 Write block 4841 len 36864 data: 5 4 7 8 8 8 8 9 9
28083 160/300 Read block 4090 len 20480
28084 161/300 Read block 782 len 20480
28085 162/300 Write block 2943 len 20480 data: 7 8 7 5 4
28086 163/300 Read block 746 len 28672
28087 164/300 Read block 808 len 16384
28088 165/300 Read block 4845 len 24576
28089 166/300 Flush
28090 167/300 Read block 4094 len 16384
28091 168/300 Write block 2233 len 4096 data: 4
28092 169/300 Read block 3372 len 40960
28093 170/300 Read block 155 len 12288
28094 171/300 Write block 1439 len 20480 data: 9 10 10 10 7
28095 172/300 Read block 212 len 32768
28096 173/300 Write block 249 len 8192 data: 5 7
28097 174/300 Write block 1366 len 24576 data: 3 4 5 5 4 4
28098 175/300 Read block 2635 len 28672
28099 176/300 Read block 1042 len 12288
28100 177/300 Flush
28101 178/300 Read block 1033 len 24576
28102 179/300 Write block 3250 len 40960 data: 5 7 7 5 6 6 6 6 7 4
28103 180/300 Read block 2170 len 24576
28104 181/300 Read block 4075 len 4096
28105 182/300 Read block 1508 len 16384
28106 183/300 Read block 1938 len 24576
28107 184/300 Write block 4484 len 20480 data: 6 4 4 5 4
28108 185/300 Write block 4064 len 36864 data: 4 5 5 5 4 4 3 2 3
28109 186/300 Write block 4526 len 40960 data: 4 4 3 3 4 6 7 6 6 6
28110 187/300 Read block 4913 len 20480
28111 188/300 Read block 1640 len 28672
28112 189/300 Flush
28113 190/300 Write block 1301 len 12288 data: 2 3 3
28114 191/300 Read block 2570 len 40960
28115 192/300 Read block 1640 len 16384
28116 193/300 Write block 1504 len 4096 data: 5
28117 194/300 Write block 4867 len 16384 data: 4 3 4 6
28118 195/300 Read block 2508 len 28672
28119 196/300 Read block 2625 len 20480
28120 197/300 Read block 4019 len 24576
28121 198/300 Write block 4555 len 24576 data: 7 8 9 7 6 7
28122 199/300 Write block 4004 len 16384 data: 8 7 9 9
28123 200/300 Read block 73 len 24576
28124 201/300 Read block 2198 len 12288
28125 202/300 Flush
28126 203/300 Write block 286 len 28672 data: 8 8 8 7 5 6 6
28127 204/300 Read block 1079 len 20480
28128 205/300 Flush
28129 206/300 Read block 4926 len 4096
28130 207/300 Write block 1658 len 40960 data: 5 6 7 7 6 6 7 7 7 4
28131 208/300 Read block 1554 len 16384
28132 209/300 Read block 4504 len 28672
28133 210/300 Read block 1374 len 24576
28134 211/300 Write block 4727 len 32768 data: 4 7 5 6 5 6 7 6
28135 212/300 Read block 3725 len 32768
28136 213/300 Write block 3076 len 12288 data: 5 5 7
28137 214/300 Write block 749 len 28672 data: 6 5 4 4 3 2 4
28138 215/300 Read block 4277 len 40960
28139 216/300 Flush
28140 217/300 Write block 3574 len 24576 data: 7 7 6 5 3 3
28141 218/300 Read block 4318 len 24576
28142 219/300 Write block 2063 len 40960 data: 9 10 9 9 11 11 10 8 7 7
28143 220/300 Read block 1255 len 28672
28144 221/300 Flush
28145 222/300 Write block 781 len 32768 data: 3 3 2 3 3 4 4 4
28146 223/300 Write block 703 len 8192 data: 7 5
28147 224/300 Write block 2740 len 36864 data: 6 6 7 8 7 7 6 4 6
28148 225/300 Read block 4754 len 20480
28149 226/300 Read block 1078 len 4096
28150 227/300 Read block 3562 len 36864
28151 228/300 Read block 4530 len 32768
28152 229/300 Write block 2311 len 40960 data: 6 6 5 4 3 4 4 6 7 7
28153 230/300 Write block 2824 len 8192 data: 3 3
28154 231/300 Read block 3831 len 20480
28155 232/300 Write block 3716 len 40960 data: 7 7 5 5 4 5 6 6 7 9
28156 233/300 Read block 350 len 28672
28157 234/300 Flush
28158 235/300 Read block 3488 len 16384
28159 236/300 Write block 1924 len 32768 data: 8 5 3 3 3 2 2 3
28160 237/300 Read block 299 len 28672
28161 238/300 Read block 4152 len 32768
28162 239/300 Flush
28163 240/300 Flush
28164 241/300 Read block 2963 len 28672
28165 242/300 Write block 3811 len 16384 data: 8 8 7 7
28166 243/300 Write block 3415 len 24576 data: 4 4 4 5 5 3
28167 244/300 Read block 1555 len 32768
28168 245/300 Write block 3082 len 40960 data: 8 7 5 7 8 8 6 7 6 5
28169 246/300 Read block 2599 len 4096
28170 247/300 Read block 1194 len 8192
28171 248/300 Read block 2988 len 8192
28172 249/300 Read block 230 len 4096
28173 250/300 Read block 1161 len 4096
28174 251/300 Read block 799 len 16384
28175 252/300 Write block 36 len 32768 data: 5 4 3 5 5 5 4 4
28176 253/300 Read block 2904 len 16384
28177 254/300 Read block 1578 len 20480
28178 255/300 Flush
28179 256/300 Write block 4986 len 28672 data: 5 3 3 3 4 4 5
28180 257/300 Write block 1228 len 16384 data: 5 5 7 7
28181 258/300 Write block 2036 len 28672 data: 3 4 5 5 5 6 4
28182 259/300 Read block 4476 len 16384
28183 260/300 Read block 1922 len 4096
28184 261/300 Read block 4122 len 8192
28185 262/300 Read block 4184 len 32768
28186 263/300 Read block 2989 len 28672
28187 264/300 Write block 2451 len 12288 data: 4 4 4
28188 265/300 Write block 1894 len 32768 data: 3 3 3 3 3 3 6 7
28189 266/300 Read block 4353 len 20480
28190 267/300 Read block 4041 len 36864
28191 268/300 Read block 2214 len 40960
28192 269/300 Read block 240 len 4096
28193 270/300 Read block 3061 len 40960
28194 271/300 Read block 276 len 12288
28195 272/300 Write block 173 len 24576 data: 6 6 7 7 6 6
28196 273/300 Write block 1442 len 16384 data: 11 8 5 5
28197 274/300 Write block 219 len 36864 data: 5 5 5 5 7 6 5 4 5
28198 275/300 Read block 3956 len 32768
28199 276/300 Write block 1820 len 32768 data: 3 4 4 5 6 7 7 7
28200 277/300 Write block 375 len 16384 data: 2 2 2 3
28201 278/300 Write block 871 len 24576 data: 5 5 6 6 6 6
28202 279/300 Read block 2698 len 40960
28203 280/300 Read block 4517 len 8192
28204 281/300 Read block 3200 len 8192
28205 282/300 Read block 2226 len 12288
28206 283/300 Write block 407 len 40960 data: 4 4 4 4 4 3 3 3 3 4
28207 284/300 Write block 1637 len 28672 data: 5 4 4 3 4 4 3
28208 285/300 Read block 2420 len 24576
28209 286/300 Flush
28210 287/300 Flush
28211 288/300 Read block 505 len 36864
28212 289/300 Flush
28213 290/300 Read block 2245 len 8192
28214 291/300 Read block 4144 len 32768
28215 292/300 Read block 2811 len 24576
28216 293/300 Read block 4890 len 32768
28217 294/300 Write block 3633 len 36864 data: 5 4 5 5 4 3 3 4 3
28218 295/300 Flush
28219 296/300 Read block 4989 len 40960
28220 297/300 Read block 4822 len 20480
28221 298/300 Write block 550 len 32768 data: 6 4 4 3 3 3 3 3
28222 299/300 Read block 2480 len 28672
28223 300/300 Read block 4622 len 24576
28224 301/300 Read block 4835 len 4096
28225 Main task has work to do, go find it
28226 got dsc Start(2)
28227 start 2
28228 Jan 17 21:17:02.421 INFO request completed, latency_us: 231, response_code: 204, uri: /start/cid/2, method: POST, req_id: ee1a861f-cf76-4cf3-906b-d1dca5b4ea1b, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
28229 Replay: started 2, returned:Ok(())
28230 ----------------------------------------------------------------
28231 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
28232 States: Active Active Offline
28233 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28234 7943 Acked Write 7 Done Done Sent false
28235 7944 Acked Flush 0 Done Done Sent false
28236 7945 Acked Write 1 Done Done Sent false
28237 7946 Acked Write 7 Done Done Sent false
28238 7947 Acked Write 2 Done Done Sent false
28239 7948 Acked Read 4 Done Done Sent false
28240 7949 Acked Read 5 Done Done Sent false
28241 7950 Acked Write 7 Done Done Sent false
28242 7951 Acked Read 1 Done Done Sent false
28243 7952 Acked Write 7 Done Done Sent false
28244 7953 Acked Read 8 Done Done Sent false
28245 7954 Acked Write 1 Done Done Sent false
28246 7955 Acked Write 5 Done Done Sent false
28247 7956 Acked Read 7 Done Done Sent false
28248 7957 Acked Read 1 Done Done Sent false
28249 7958 Acked Flush 0 Done Done Sent false
28250 7959 Acked Read 7 Done Done Sent false
28251 7960 Acked Read 10 Done Done Sent false
28252 7961 Acked Write 5 Done Done Sent false
28253 7962 Acked Write 7 Done Done Sent false
28254 7963 Acked Read 10 Done Done Sent false
28255 7964 Acked Read 9 Done Done Sent false
28256 7965 Acked Read 3 Done Done Sent false
28257 7966 Acked Write 6 Done Done Sent false
28258 7967 Acked Read 7 Done Done Sent false
28259 7968 Acked Read 2 Done Done Sent false
28260 7969 Acked Write 1 Done Done Sent false
28261 7970 Acked Read 1 Done Done Sent false
28262 7971 Acked Write 9 Done Done Sent false
28263 7972 Acked Write 6 Done Done Sent false
28264 7973 Acked Read 2 Done Done Sent false
28265 7974 Acked Read 10 Done Done Sent false
28266 7975 Acked Write 7 Done Done Sent false
28267 7976 Acked Flush 0 Done Done Sent false
28268 7977 Acked Write 1 Done Done Sent false
28269 7978 Acked Read 1 Done Done Sent false
28270 7979 Acked Write 5 Done Done Sent false
28271 7980 Acked Read 5 Done Done Sent false
28272 7981 Acked Read 7 Done Done Sent false
28273 7982 Acked Read 9 Done Done Sent false
28274 7983 Acked Flush 0 Done Done Sent false
28275 7984 Acked Write 7 Done Done Sent false
28276 7985 Acked Write 7 Done Done Sent false
28277 7986 Acked Read 7 Done Done Sent false
28278 7987 Acked Write 2 Done Done Sent false
28279 7988 Acked Read 7 Done Done Sent false
28280 7989 Acked Write 9 Done Done Sent false
28281 7990 Acked Read 9 Done Done Sent false
28282 7991 Acked Write 1 Done Done Sent false
28283 7992 Acked Write 3 Done Done Sent false
28284 7993 Acked Write 8 Done Done Sent false
28285 7994 Acked Write 7 Done Done Sent false
28286 7995 Acked Write 9 Done Done Sent false
28287 7996 Acked Flush 0 Done Done Sent false
28288 7997 Acked Read 7 Done Done Sent false
28289 7998 Acked Read 9 Done Done Sent false
28290 7999 Acked Read 9 Done Done Sent false
28291 8000 Acked Read 3 Done Done Sent false
28292 8001 Acked Read 3 Done Done Sent false
28293 8002 Acked Write 6 Done Done Sent false
28294 8003 Acked Write 7 Done Done Sent false
28295 8004 Acked Read 8 Done Done Sent false
28296 8005 Acked Write 3 Done Done Sent false
28297 8006 Acked Write 7 Done Done Sent false
28298 8007 Acked Read 2 Done Done Sent false
28299 8008 Acked Read 7 Done Done Sent false
28300 8009 Acked Write 5 Done Done Sent false
28301 8010 Acked Read 3 Done Done Sent false
28302 8011 Acked Read 6 Done Done Sent false
28303 8012 Acked Write 4 Done Done Sent false
28304 8013 Acked Read 10 Done Done Sent false
28305 8014 Acked Flush 0 Done Done Sent false
28306 8015 Acked Read 1 Done Done Sent false
28307 8016 Acked Write 9 Done Done Sent false
28308 8017 Acked Flush 0 Done Done Sent false
28309 8018 Acked Read 6 Done Done Sent false
28310 8019 Acked Read 8 Done Done Sent false
28311 8020 Acked Flush 0 Done Done Sent false
28312 8021 Acked Read 8 Done Done Sent false
28313 8022 Acked Read 1 Done Done Sent false
28314 8023 Acked Flush 0 Done Done Sent false
28315 8024 Acked Read 9 Done Done Sent false
28316 8025 Acked Write 1 Done Done Sent false
28317 8026 Acked Read 10 Done Done Sent false
28318 8027 Acked Read 4 Done Done Sent false
28319 8028 Acked Flush 0 Done Done Sent false
28320 8029 Acked Flush 0 Done Done Sent false
28321 8030 Acked Write 9 Done Done Sent false
28322 8031 Acked Read 7 Done Done Sent false
28323 8032 Acked Read 3 Done Done Sent false
28324 8033 Acked Read 2 Done Done Sent false
28325 8034 Acked Write 2 Done Done Sent false
28326 8035 Acked Read 9 Done Done Sent false
28327 8036 Acked Write 9 Done Done Sent false
28328 8037 Acked Read 3 Done Done Sent false
28329 8038 Acked Read 4 Done Done Sent false
28330 8039 Acked Write 8 Done Done Sent false
28331 8040 Acked Read 10 Done Done Sent false
28332 8041 Acked Write 4 Done Done Sent false
28333 8042 Acked Write 9 Done Done Sent false
28334 8043 Acked Write 9 Done Done Sent false
28335 8044 Acked Read 6 Done Done Sent false
28336 8045 Acked Write 1 Done Done Sent false
28337 8046 Acked Read 9 Done Done Sent false
28338 8047 Acked Write 7 Done Done Sent false
28339 8048 Acked Write 5 Done Done Sent false
28340 8049 Acked Read 6 Done Done Sent false
28341 8050 Acked Write 2 Done Done Sent false
28342 8051 Acked Write 1 Done Done Sent false
28343 8052 Acked Read 7 Done Done Sent false
28344 8053 Acked Write 6 Done Done Sent false
28345 8054 Acked Write 10 Done Done Sent false
28346 8055 Acked Flush 0 Done Done Sent false
28347 8056 Acked Write 8 Done Done Sent false
28348 8057 Acked Write 4 Done Done Sent false
28349 8058 Acked Read 8 Done Done Sent false
28350 8059 Acked Read 7 Done Done Sent false
28351 8060 Acked Read 4 Done Done Sent false
28352 8061 Acked Flush 0 Done Done Sent false
28353 8062 Acked Read 4 Done Done Sent false
28354 8063 Acked Read 6 Done Done Sent false
28355 8064 Acked Write 10 Done Done Sent false
28356 8065 Acked Write 5 Done Done Sent false
28357 8066 Acked Read 3 Done Done Sent false
28358 8067 Acked Read 5 Done Done Sent false
28359 8068 Acked Write 3 Done Done Sent false
28360 8069 Acked Flush 0 Done Done Sent false
28361 8070 Acked Read 7 Done Done Sent false
28362 8071 Acked Read 4 Done Done Sent false
28363 8072 Acked Write 6 Done Done Sent false
28364 8073 Acked Read 6 Done Done Sent false
28365 8074 Acked Read 9 Done Done Sent false
28366 8075 Acked Read 6 Done Done Sent false
28367 8076 Acked Read 1 Done Done Sent false
28368 8077 Acked Read 10 Done Done Sent false
28369 8078 Acked Flush 0 Done Done Sent false
28370 8079 Acked Flush 0 Done Done Sent false
28371 8080 Acked Write 2 Done Done Sent false
28372 8081 Acked Flush 0 Done Done Sent false
28373 8082 Acked Write 5 Done Done Sent false
28374 8083 Acked Write 6 Done Done Sent false
28375 8084 Acked Read 5 Done Done Sent false
28376 8085 Acked Write 6 Done Done Sent false
28377 8086 Acked Read 10 Done Done Sent false
28378 8087 Acked Write 7 Done Done Sent false
28379 8088 Acked Read 3 Done Done Sent false
28380 8089 Acked Read 5 Done Done Sent false
28381 8090 Acked Read 5 Done Done Sent false
28382 8091 Acked Read 9 Done Done Sent false
28383 8092 Acked Read 5 Done Done Sent false
28384 8093 Acked Write 6 Done Done Sent false
28385 8094 Acked Write 10 Done Done Sent false
28386 8095 Acked Flush 0 Done Done Sent false
28387 8096 Acked Write 1 Done Done Sent false
28388 8097 Acked Write 7 Done Done Sent false
28389 8098 Acked Read 8 Done Done Sent false
28390 8099 Acked Write 2 Done Done Sent false
28391 8100 Acked Read 2 Done Done Sent false
28392 8101 Acked Write 9 Done Done Sent false
28393 8102 Acked Read 5 Done Done Sent false
28394 8103 Acked Read 5 Done Done Sent false
28395 8104 Acked Write 5 Done Done Sent false
28396 8105 Acked Read 7 Done Done Sent false
28397 8106 Acked Read 4 Done Done Sent false
28398 8107 Acked Read 6 Done Done Sent false
28399 8108 Acked Flush 0 Done Done Sent false
28400 8109 Acked Read 4 Done Done Sent false
28401 8110 Acked Write 1 Done Done Sent false
28402 8111 Acked Read 10 Done Done Sent false
28403 8112 Acked Read 3 Done Done Sent false
28404 8113 Acked Write 5 Done Done Sent false
28405 8114 Acked Read 8 Done Done Sent false
28406 8115 Acked Write 2 Done Done Sent false
28407 8116 Acked Write 6 Done Done Sent false
28408 8117 Acked Read 7 Done Done Sent false
28409 8118 Acked Read 3 Done Done Sent false
28410 8119 Acked Flush 0 Done Done Sent false
28411 8120 Acked Read 6 Done Done Sent false
28412 8121 Acked Write 10 Done Done Sent false
28413 8122 Acked Read 6 Done Done Sent false
28414 8123 Acked Read 1 Done Done Sent false
28415 8124 Acked Read 4 Done Done Sent false
28416 8125 Acked Read 6 Done Done Sent false
28417 8126 Acked Write 5 Done Done Sent false
28418 8127 Acked Write 9 Done Done Sent false
28419 8128 Acked Write 10 Done Done Sent false
28420 8129 Acked Read 5 Done Done Sent false
28421 8130 Acked Read 7 Done Done Sent false
28422 8131 Acked Flush 0 Done Done Sent false
28423 8132 Acked Write 3 Done Done Sent false
28424 8133 Acked Read 10 Done Done Sent false
28425 8134 Acked Read 4 Done Done Sent false
28426 8135 Acked Write 1 Done Done Sent false
28427 8136 Acked Write 4 Done Done Sent false
28428 8137 Acked Read 7 Done Done Sent false
28429 8138 Acked Read 5 Done Done Sent false
28430 8139 Acked Read 6 Done Done Sent false
28431 8140 Acked Write 6 Done Done Sent false
28432 8141 Acked Write 4 Done Done Sent false
28433 8142 Acked Read 6 Done Done Sent false
28434 8143 Acked Read 3 Done Done Sent false
28435 8144 Acked Flush 0 Done Done Sent false
28436 8145 Acked Write 7 Done Done Sent false
28437 8146 Acked Read 5 Done Done Sent false
28438 8147 Acked Flush 0 Done Done Sent false
28439 8148 Acked Read 1 Done Done Sent false
28440 8149 Acked Write 10 Done Done Sent false
28441 8150 Acked Read 4 Done Done Sent false
28442 8151 Acked Read 7 Done Done Sent false
28443 8152 Acked Read 6 Done Done Sent false
28444 8153 Acked Write 8 Done Done Sent false
28445 8154 Acked Read 8 Done Done Sent false
28446 8155 Acked Write 3 Done Done Sent false
28447 8156 Acked Write 7 Done Done Sent false
28448 8157 Acked Read 10 Done Done Sent false
28449 8158 Acked Flush 0 Done Done Sent false
28450 8159 Acked Write 6 Done Done Sent false
28451 8160 Acked Read 6 Done Done Sent false
28452 8161 Acked Write 10 Done Done Sent false
28453 8162 Acked Read 7 Done Done Sent false
28454 8163 Acked Flush 0 Done Done Sent false
28455 8164 Acked Write 8 Done Done Sent false
28456 8165 Acked Write 2 Done Done Sent false
28457 8166 Acked Write 9 Done Done Sent false
28458 8167 Acked Read 5 Done Done Sent false
28459 8168 Acked Read 1 Done Done Sent false
28460 8169 Acked Read 9 Done Done Sent false
28461 8170 Acked Read 8 Done Done Sent false
28462 8171 Acked Write 10 Done Done Sent false
28463 8172 Acked Write 2 Done Done Sent false
28464 8173 Acked Read 5 Done Done Sent false
28465 8174 Acked Write 10 Done Done Sent false
28466 8175 Acked Read 7 Done Done Sent false
28467 8176 Acked Flush 0 Done Done Sent false
28468 8177 Acked Read 4 Done Done Sent false
28469 8178 Acked Write 8 Done Done Sent false
28470 8179 Acked Read 7 Done Done Sent false
28471 8180 Acked Read 8 Done Done Sent false
28472 8181 Acked Flush 0 Done Done Sent false
28473 8182 Acked Flush 0 Done Done Sent false
28474 8183 Acked Read 7 Done Done Sent false
28475 8184 Acked Write 4 Done Done Sent false
28476 8185 Acked Write 6 Done Done Sent false
28477 8186 Acked Read 8 Done Done Sent false
28478 8187 Acked Write 10 Done Done Sent false
28479 8188 Acked Read 1 Done Done Sent false
28480 8189 Acked Read 2 Done Done Sent false
28481 8190 Acked Read 2 Done Done Sent false
28482 8191 Acked Read 1 Done Done Sent false
28483 8192 Acked Read 1 Done Done Sent false
28484 8193 Acked Read 4 Done Done Sent false
28485 8194 Acked Write 8 Done Done Sent false
28486 8195 Acked Read 4 Done Done Sent false
28487 8196 Acked Read 5 Done Done Sent false
28488 8197 Acked Flush 0 Done Done Sent false
28489 8198 Acked Write 7 Done Done Sent false
28490 8199 Acked Write 4 Done Done Sent false
28491 8200 Acked Write 7 Done Done Sent false
28492 8201 Acked Read 4 Done Done Sent false
28493 8202 Acked Read 1 Done Done Sent false
28494 8203 Acked Read 2 Done Done Sent false
28495 8204 Acked Read 8 Done Done Sent false
28496 8205 Acked Read 7 Done Done Sent false
28497 8206 Acked Write 3 Done Done Sent false
28498 8207 Acked Write 8 Done Done Sent false
28499 8208 Acked Read 5 Done Done Sent false
28500 8209 Acked Read 9 Done Done Sent false
28501 8210 Acked Read 10 Done Done Sent false
28502 8211 Acked Read 1 Done Done Sent false
28503 8212 Acked Read 10 Done Done Sent false
28504 8213 Acked Read 3 Done Done Sent false
28505 8214 Acked Write 6 Done Done Sent false
28506 8215 Acked Write 4 Done Done Sent false
28507 8216 Acked Write 9 Done Done Sent false
28508 8217 Acked Read 8 Done Done Sent false
28509 8218 Acked Write 8 Done Done Sent false
28510 8219 Acked Write 4 Done Done Sent false
28511 8220 Acked Write 6 Done Done Sent false
28512 8221 Acked Read 10 Done Done Sent false
28513 8222 Acked Read 2 Done Done Sent false
28514 8223 Acked Read 2 Done Done Sent false
28515 8224 Acked Read 3 Done Done Sent false
28516 8225 Acked Write 10 Done Done Sent false
28517 8226 Acked Write 7 Done Done Sent false
28518 8227 Acked Read 6 Done Done Sent false
28519 8228 Acked Flush 0 Done Done Sent false
28520 8229 Acked Flush 0 Done Done Sent false
28521 8230 Acked Read 9 Done Done Sent false
28522 8231 Acked Flush 0 Done Done Sent false
28523 8232 Acked Read 2 Done Done Sent false
28524 8233 Acked Read 8 Done Done Sent false
28525 8234 Acked Read 6 Done Done Sent false
28526 8235 Acked Read 8 Done Done Sent false
28527 8236 Acked Write 9 Done Done Sent false
28528 8237 Acked Flush 0 Done Done Sent false
28529 8238 Acked Read 10 Done Done Sent false
28530 8239 Acked Read 5 Done Done Sent false
28531 8240 Acked Write 8 Done Done Sent false
28532 8241 Acked Read 7 Done Done Sent false
28533 8242 Acked Read 6 Done Done Sent false
28534 8243 Acked Read 1 Done Done Sent false
28535 STATES DS:0 DS:1 DS:2 TOTAL
28536 Sent 0 0 301 301
28537 Done 301 301 0 602
28538 Skipped 0 0 0 0
28539 Error 0 0 0 0
28540 Last Flush: 8237 8237 7942
28541 Downstairs last five completed: 7942 7941 7940 7939 7938
28542 Upstairs last five completed: 8243 8242 8241 8240 8239
28543 CLIENT: Up:0 ds:301 act:2
28544 [8830] got start action
28545 [8830] I am going to restart
28546 Starting downstairs at port 8830
28547 Make output file at "/tmp/dsc/downstairs-8830.txt"
28548 [8830][2] reports Starting
28549 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1224)
28550 [8830][2] reports Running
28551 ----------------------------------------------------------------
28552 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28553 States: Active Active Offline
28554 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28555 7943 Acked Write 7 Done Done Sent false
28556 7944 Acked Flush 0 Done Done Sent false
28557 7945 Acked Write 1 Done Done Sent false
28558 7946 Acked Write 7 Done Done Sent false
28559 7947 Acked Write 2 Done Done Sent false
28560 7948 Acked Read 4 Done Done Sent false
28561 7949 Acked Read 5 Done Done Sent false
28562 7950 Acked Write 7 Done Done Sent false
28563 7951 Acked Read 1 Done Done Sent false
28564 7952 Acked Write 7 Done Done Sent false
28565 7953 Acked Read 8 Done Done Sent false
28566 7954 Acked Write 1 Done Done Sent false
28567 7955 Acked Write 5 Done Done Sent false
28568 7956 Acked Read 7 Done Done Sent false
28569 7957 Acked Read 1 Done Done Sent false
28570 7958 Acked Flush 0 Done Done Sent false
28571 7959 Acked Read 7 Done Done Sent false
28572 7960 Acked Read 10 Done Done Sent false
28573 7961 Acked Write 5 Done Done Sent false
28574 7962 Acked Write 7 Done Done Sent false
28575 7963 Acked Read 10 Done Done Sent false
28576 7964 Acked Read 9 Done Done Sent false
28577 7965 Acked Read 3 Done Done Sent false
28578 7966 Acked Write 6 Done Done Sent false
28579 7967 Acked Read 7 Done Done Sent false
28580 7968 Acked Read 2 Done Done Sent false
28581 7969 Acked Write 1 Done Done Sent false
28582 7970 Acked Read 1 Done Done Sent false
28583 7971 Acked Write 9 Done Done Sent false
28584 7972 Acked Write 6 Done Done Sent false
28585 7973 Acked Read 2 Done Done Sent false
28586 7974 Acked Read 10 Done Done Sent false
28587 7975 Acked Write 7 Done Done Sent false
28588 7976 Acked Flush 0 Done Done Sent false
28589 7977 Acked Write 1 Done Done Sent false
28590 7978 Acked Read 1 Done Done Sent false
28591 7979 Acked Write 5 Done Done Sent false
28592 7980 Acked Read 5 Done Done Sent false
28593 7981 Acked Read 7 Done Done Sent false
28594 7982 Acked Read 9 Done Done Sent false
28595 7983 Acked Flush 0 Done Done Sent false
28596 7984 Acked Write 7 Done Done Sent false
28597 7985 Acked Write 7 Done Done Sent false
28598 7986 Acked Read 7 Done Done Sent false
28599 7987 Acked Write 2 Done Done Sent false
28600 7988 Acked Read 7 Done Done Sent false
28601 7989 Acked Write 9 Done Done Sent false
28602 7990 Acked Read 9 Done Done Sent false
28603 7991 Acked Write 1 Done Done Sent false
28604 7992 Acked Write 3 Done Done Sent false
28605 7993 Acked Write 8 Done Done Sent false
28606 7994 Acked Write 7 Done Done Sent false
28607 7995 Acked Write 9 Done Done Sent false
28608 7996 Acked Flush 0 Done Done Sent false
28609 7997 Acked Read 7 Done Done Sent false
28610 7998 Acked Read 9 Done Done Sent false
28611 7999 Acked Read 9 Done Done Sent false
28612 8000 Acked Read 3 Done Done Sent false
28613 8001 Acked Read 3 Done Done Sent false
28614 8002 Acked Write 6 Done Done Sent false
28615 8003 Acked Write 7 Done Done Sent false
28616 8004 Acked Read 8 Done Done Sent false
28617 8005 Acked Write 3 Done Done Sent false
28618 8006 Acked Write 7 Done Done Sent false
28619 8007 Acked Read 2 Done Done Sent false
28620 8008 Acked Read 7 Done Done Sent false
28621 8009 Acked Write 5 Done Done Sent false
28622 8010 Acked Read 3 Done Done Sent false
28623 8011 Acked Read 6 Done Done Sent false
28624 8012 Acked Write 4 Done Done Sent false
28625 8013 Acked Read 10 Done Done Sent false
28626 8014 Acked Flush 0 Done Done Sent false
28627 8015 Acked Read 1 Done Done Sent false
28628 8016 Acked Write 9 Done Done Sent false
28629 8017 Acked Flush 0 Done Done Sent false
28630 8018 Acked Read 6 Done Done Sent false
28631 8019 Acked Read 8 Done Done Sent false
28632 8020 Acked Flush 0 Done Done Sent false
28633 8021 Acked Read 8 Done Done Sent false
28634 8022 Acked Read 1 Done Done Sent false
28635 8023 Acked Flush 0 Done Done Sent false
28636 8024 Acked Read 9 Done Done Sent false
28637 8025 Acked Write 1 Done Done Sent false
28638 8026 Acked Read 10 Done Done Sent false
28639 8027 Acked Read 4 Done Done Sent false
28640 8028 Acked Flush 0 Done Done Sent false
28641 8029 Acked Flush 0 Done Done Sent false
28642 8030 Acked Write 9 Done Done Sent false
28643 8031 Acked Read 7 Done Done Sent false
28644 8032 Acked Read 3 Done Done Sent false
28645 8033 Acked Read 2 Done Done Sent false
28646 8034 Acked Write 2 Done Done Sent false
28647 8035 Acked Read 9 Done Done Sent false
28648 8036 Acked Write 9 Done Done Sent false
28649 8037 Acked Read 3 Done Done Sent false
28650 8038 Acked Read 4 Done Done Sent false
28651 8039 Acked Write 8 Done Done Sent false
28652 8040 Acked Read 10 Done Done Sent false
28653 8041 Acked Write 4 Done Done Sent false
28654 8042 Acked Write 9 Done Done Sent false
28655 8043 Acked Write 9 Done Done Sent false
28656 8044 Acked Read 6 Done Done Sent false
28657 8045 Acked Write 1 Done Done Sent false
28658 8046 Acked Read 9 Done Done Sent false
28659 8047 Acked Write 7 Done Done Sent false
28660 8048 Acked Write 5 Done Done Sent false
28661 8049 Acked Read 6 Done Done Sent false
28662 8050 Acked Write 2 Done Done Sent false
28663 8051 Acked Write 1 Done Done Sent false
28664 8052 Acked Read 7 Done Done Sent false
28665 8053 Acked Write 6 Done Done Sent false
28666 8054 Acked Write 10 Done Done Sent false
28667 8055 Acked Flush 0 Done Done Sent false
28668 8056 Acked Write 8 Done Done Sent false
28669 8057 Acked Write 4 Done Done Sent false
28670 8058 Acked Read 8 Done Done Sent false
28671 8059 Acked Read 7 Done Done Sent false
28672 8060 Acked Read 4 Done Done Sent false
28673 8061 Acked Flush 0 Done Done Sent false
28674 8062 Acked Read 4 Done Done Sent false
28675 8063 Acked Read 6 Done Done Sent false
28676 8064 Acked Write 10 Done Done Sent false
28677 8065 Acked Write 5 Done Done Sent false
28678 8066 Acked Read 3 Done Done Sent false
28679 8067 Acked Read 5 Done Done Sent false
28680 8068 Acked Write 3 Done Done Sent false
28681 8069 Acked Flush 0 Done Done Sent false
28682 8070 Acked Read 7 Done Done Sent false
28683 8071 Acked Read 4 Done Done Sent false
28684 8072 Acked Write 6 Done Done Sent false
28685 8073 Acked Read 6 Done Done Sent false
28686 8074 Acked Read 9 Done Done Sent false
28687 8075 Acked Read 6 Done Done Sent false
28688 8076 Acked Read 1 Done Done Sent false
28689 8077 Acked Read 10 Done Done Sent false
28690 8078 Acked Flush 0 Done Done Sent false
28691 8079 Acked Flush 0 Done Done Sent false
28692 8080 Acked Write 2 Done Done Sent false
28693 8081 Acked Flush 0 Done Done Sent false
28694 8082 Acked Write 5 Done Done Sent false
28695 8083 Acked Write 6 Done Done Sent false
28696 8084 Acked Read 5 Done Done Sent false
28697 8085 Acked Write 6 Done Done Sent false
28698 8086 Acked Read 10 Done Done Sent false
28699 8087 Acked Write 7 Done Done Sent false
28700 8088 Acked Read 3 Done Done Sent false
28701 8089 Acked Read 5 Done Done Sent false
28702 8090 Acked Read 5 Done Done Sent false
28703 8091 Acked Read 9 Done Done Sent false
28704 8092 Acked Read 5 Done Done Sent false
28705 8093 Acked Write 6 Done Done Sent false
28706 8094 Acked Write 10 Done Done Sent false
28707 8095 Acked Flush 0 Done Done Sent false
28708 8096 Acked Write 1 Done Done Sent false
28709 8097 Acked Write 7 Done Done Sent false
28710 8098 Acked Read 8 Done Done Sent false
28711 8099 Acked Write 2 Done Done Sent false
28712 8100 Acked Read 2 Done Done Sent false
28713 8101 Acked Write 9 Done Done Sent false
28714 8102 Acked Read 5 Done Done Sent false
28715 8103 Acked Read 5 Done Done Sent false
28716 8104 Acked Write 5 Done Done Sent false
28717 8105 Acked Read 7 Done Done Sent false
28718 8106 Acked Read 4 Done Done Sent false
28719 8107 Acked Read 6 Done Done Sent false
28720 8108 Acked Flush 0 Done Done Sent false
28721 8109 Acked Read 4 Done Done Sent false
28722 8110 Acked Write 1 Done Done Sent false
28723 8111 Acked Read 10 Done Done Sent false
28724 8112 Acked Read 3 Done Done Sent false
28725 8113 Acked Write 5 Done Done Sent false
28726 8114 Acked Read 8 Done Done Sent false
28727 8115 Acked Write 2 Done Done Sent false
28728 8116 Acked Write 6 Done Done Sent false
28729 8117 Acked Read 7 Done Done Sent false
28730 8118 Acked Read 3 Done Done Sent false
28731 8119 Acked Flush 0 Done Done Sent false
28732 8120 Acked Read 6 Done Done Sent false
28733 8121 Acked Write 10 Done Done Sent false
28734 8122 Acked Read 6 Done Done Sent false
28735 8123 Acked Read 1 Done Done Sent false
28736 8124 Acked Read 4 Done Done Sent false
28737 8125 Acked Read 6 Done Done Sent false
28738 8126 Acked Write 5 Done Done Sent false
28739 8127 Acked Write 9 Done Done Sent false
28740 8128 Acked Write 10 Done Done Sent false
28741 8129 Acked Read 5 Done Done Sent false
28742 8130 Acked Read 7 Done Done Sent false
28743 8131 Acked Flush 0 Done Done Sent false
28744 8132 Acked Write 3 Done Done Sent false
28745 8133 Acked Read 10 Done Done Sent false
28746 8134 Acked Read 4 Done Done Sent false
28747 8135 Acked Write 1 Done Done Sent false
28748 8136 Acked Write 4 Done Done Sent false
28749 8137 Acked Read 7 Done Done Sent false
28750 8138 Acked Read 5 Done Done Sent false
28751 8139 Acked Read 6 Done Done Sent false
28752 8140 Acked Write 6 Done Done Sent false
28753 8141 Acked Write 4 Done Done Sent false
28754 8142 Acked Read 6 Done Done Sent false
28755 8143 Acked Read 3 Done Done Sent false
28756 8144 Acked Flush 0 Done Done Sent false
28757 8145 Acked Write 7 Done Done Sent false
28758 8146 Acked Read 5 Done Done Sent false
28759 8147 Acked Flush 0 Done Done Sent false
28760 8148 Acked Read 1 Done Done Sent false
28761 8149 Acked Write 10 Done Done Sent false
28762 8150 Acked Read 4 Done Done Sent false
28763 8151 Acked Read 7 Done Done Sent false
28764 8152 Acked Read 6 Done Done Sent false
28765 8153 Acked Write 8 Done Done Sent false
28766 8154 Acked Read 8 Done Done Sent false
28767 8155 Acked Write 3 Done Done Sent false
28768 8156 Acked Write 7 Done Done Sent false
28769 8157 Acked Read 10 Done Done Sent false
28770 8158 Acked Flush 0 Done Done Sent false
28771 8159 Acked Write 6 Done Done Sent false
28772 8160 Acked Read 6 Done Done Sent false
28773 8161 Acked Write 10 Done Done Sent false
28774 8162 Acked Read 7 Done Done Sent false
28775 8163 Acked Flush 0 Done Done Sent false
28776 8164 Acked Write 8 Done Done Sent false
28777 8165 Acked Write 2 Done Done Sent false
28778 8166 Acked Write 9 Done Done Sent false
28779 8167 Acked Read 5 Done Done Sent false
28780 8168 Acked Read 1 Done Done Sent false
28781 8169 Acked Read 9 Done Done Sent false
28782 8170 Acked Read 8 Done Done Sent false
28783 8171 Acked Write 10 Done Done Sent false
28784 8172 Acked Write 2 Done Done Sent false
28785 8173 Acked Read 5 Done Done Sent false
28786 8174 Acked Write 10 Done Done Sent false
28787 8175 Acked Read 7 Done Done Sent false
28788 8176 Acked Flush 0 Done Done Sent false
28789 8177 Acked Read 4 Done Done Sent false
28790 8178 Acked Write 8 Done Done Sent false
28791 8179 Acked Read 7 Done Done Sent false
28792 8180 Acked Read 8 Done Done Sent false
28793 8181 Acked Flush 0 Done Done Sent false
28794 8182 Acked Flush 0 Done Done Sent false
28795 8183 Acked Read 7 Done Done Sent false
28796 8184 Acked Write 4 Done Done Sent false
28797 8185 Acked Write 6 Done Done Sent false
28798 8186 Acked Read 8 Done Done Sent false
28799 8187 Acked Write 10 Done Done Sent false
28800 8188 Acked Read 1 Done Done Sent false
28801 8189 Acked Read 2 Done Done Sent false
28802 8190 Acked Read 2 Done Done Sent false
28803 8191 Acked Read 1 Done Done Sent false
28804 8192 Acked Read 1 Done Done Sent false
28805 8193 Acked Read 4 Done Done Sent false
28806 8194 Acked Write 8 Done Done Sent false
28807 8195 Acked Read 4 Done Done Sent false
28808 8196 Acked Read 5 Done Done Sent false
28809 8197 Acked Flush 0 Done Done Sent false
28810 8198 Acked Write 7 Done Done Sent false
28811 8199 Acked Write 4 Done Done Sent false
28812 8200 Acked Write 7 Done Done Sent false
28813 8201 Acked Read 4 Done Done Sent false
28814 8202 Acked Read 1 Done Done Sent false
28815 8203 Acked Read 2 Done Done Sent false
28816 8204 Acked Read 8 Done Done Sent false
28817 8205 Acked Read 7 Done Done Sent false
28818 8206 Acked Write 3 Done Done Sent false
28819 8207 Acked Write 8 Done Done Sent false
28820 8208 Acked Read 5 Done Done Sent false
28821 8209 Acked Read 9 Done Done Sent false
28822 8210 Acked Read 10 Done Done Sent false
28823 8211 Acked Read 1 Done Done Sent false
28824 8212 Acked Read 10 Done Done Sent false
28825 8213 Acked Read 3 Done Done Sent false
28826 8214 Acked Write 6 Done Done Sent false
28827 8215 Acked Write 4 Done Done Sent false
28828 8216 Acked Write 9 Done Done Sent false
28829 8217 Acked Read 8 Done Done Sent false
28830 8218 Acked Write 8 Done Done Sent false
28831 8219 Acked Write 4 Done Done Sent false
28832 8220 Acked Write 6 Done Done Sent false
28833 8221 Acked Read 10 Done Done Sent false
28834 8222 Acked Read 2 Done Done Sent false
28835 8223 Acked Read 2 Done Done Sent false
28836 8224 Acked Read 3 Done Done Sent false
28837 8225 Acked Write 10 Done Done Sent false
28838 8226 Acked Write 7 Done Done Sent false
28839 8227 Acked Read 6 Done Done Sent false
28840 8228 Acked Flush 0 Done Done Sent false
28841 8229 Acked Flush 0 Done Done Sent false
28842 8230 Acked Read 9 Done Done Sent false
28843 8231 Acked Flush 0 Done Done Sent false
28844 8232 Acked Read 2 Done Done Sent false
28845 8233 Acked Read 8 Done Done Sent false
28846 8234 Acked Read 6 Done Done Sent false
28847 8235 Acked Read 8 Done Done Sent false
28848 8236 Acked Write 9 Done Done Sent false
28849 8237 Acked Flush 0 Done Done Sent false
28850 8238 Acked Read 10 Done Done Sent false
28851 8239 Acked Read 5 Done Done Sent false
28852 8240 Acked Write 8 Done Done Sent false
28853 8241 Acked Read 7 Done Done Sent false
28854 8242 Acked Read 6 Done Done Sent false
28855 8243 Acked Read 1 Done Done Sent false
28856 8244 Acked Flush 0 Done Done Sent false
28857 STATES DS:0 DS:1 DS:2 TOTAL
28858 Sent 0 0 302 302
28859 Done 302 302 0 604
28860 Skipped 0 0 0 0
28861 Error 0 0 0 0
28862 Last Flush: 8244 8244 7942
28863 Downstairs last five completed: 7942 7941 7940 7939 7938
28864 Upstairs last five completed: 8244 8243 8242 8241 8240
28865 CLIENT: Up:0 ds:302 act:2
288662025-01-17T21:17:08.168ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
288672025-01-17T21:17:08.168ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41826) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
288682025-01-17T21:17:08.169ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
288692025-01-17T21:17:08.169ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
288702025-01-17T21:17:08.169ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
288712025-01-17T21:17:08.170ZINFOcrucible: send last flush ID to this DS: 7942 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
288722025-01-17T21:17:08.170ZINFOcrucible: Replied this last flush ID: 7942 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
288732025-01-17T21:17:08.170ZINFOcrucible: [2] client re-new 302 jobs since flush 7942 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
288742025-01-17T21:17:08.170ZINFOcrucible: [2] Replayed 302 jobs since flush: 7942 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
28875 ----------------------------------------------------------------
28876 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
28877 Downstairs last five completed: 8244 8243 8242 8241 8240
28878 Upstairs last five completed: 8244 8243 8242 8241 8240
28879 CLIENT: Up:0 ds:0 act:3
28880 Replay: All jobs finished
28881 Main task has work to do, go find it
28882 got dsc Stop(1)
28883 stop 1
28884 Jan 17 21:17:10.426 INFO request completed, latency_us: 329[8820] Got stop action so:false kr:false
28885 , response_code: 204, uri: /stop/cid/1, method: POST, req_id: f0ef6837-7afb-40ee-8936-0fdeb3b900c7, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
288862025-01-17T21:17:10.426ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
28887 Jan 17 21:17:10.426 INFO request completed, latency_us: 141, response_code: {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:17:10.42684304Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
28888 200, uri: /state/cid/1, method: GET, req_id: {"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:17:10.426883195Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
28889 47edf6a7-7b72-48ff-a283-c7c6f22ddab7, remote_addr: 127.0.0.1:57191, {"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-17T21:17:10.426927029Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
28890 local_addr: 127.0.0.1:9998
28891 [8820] Exited with: ExitStatus(unix_wait_status(9))
28892 [8820][1] reports Exit
28893 Jan 17 21:17:14.428 INFO request completed, latency_us: 235, response_code: 200, uri: /state/cid/1, method: GET, req_id: 4bf33e1a-cda1-4d94-88fe-d24b5b64dd88, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
28894 001/300 Read block 3224 len 24576
28895 002/300 Read block 4464 len 24576
28896 003/300 Read block 2980 len 8192
28897 004/300 Write block 3640 len 24576 data: 5 4 2 4 4 4
28898 005/300 Read block 4908 len 8192
28899 006/300 Write block 3267 len 4096 data: 3
28900 007/300 Write block 303 len 40960 data: 4 5 5 5 5 4 3 3 3 2
28901 008/300 Read block 2405 len 4096
28902 009/300 Write block 3005 len 36864 data: 8 8 6 7 5 6 5 6 6
28903 010/300 Write block 1177 len 4096 data: 5
28904 011/300 Write block 2362 len 12288 data: 3 3 3
28905 012/300 Flush
28906 013/300 Read block 4543 len 20480
28907 014/300 Read block 2906 len 16384
28908 015/300 Read block 2029 len 24576
28909 016/300 Flush
28910 017/300 Flush
28911 018/300 Read block 2837 len 28672
28912 019/300 Read block 4882 len 40960
28913 020/300 Read block 3530 len 24576
28914 021/300 Write block 248 len 4096 data: 6
28915 022/300 Flush
28916 023/300 Read block 851 len 16384
28917 024/300 Read block 4482 len 24576
28918 025/300 Read block 2332 len 8192
28919 026/300 Read block 600 len 24576
28920 027/300 Read block 1917 len 12288
28921 028/300 Flush
28922 029/300 Flush
28923 030/300 Flush
28924 031/300 Read block 35 len 8192
28925 032/300 Read block 3065 len 4096
28926 033/300 Write block 3206 len 40960 data: 6 6 5 5 6 7 8 7 5 4
28927 034/300 Write block 536 len 24576 data: 8 6 6 4 5 6
28928 035/300 Read block 697 len 12288
28929 036/300 Read block 1853 len 36864
28930 037/300 Write block 991 len 24576 data: 4 5 5 4 4 4
28931 038/300 Flush
28932 039/300 Read block 2483 len 8192
28933 040/300 Read block 4599 len 36864
28934 041/300 Read block 501 len 20480
28935 042/300 Read block 4940 len 32768
28936 043/300 Read block 3302 len 4096
28937 044/300 Read block 2314 len 12288
28938 045/300 Flush
28939 046/300 Write block 4931 len 20480 data: 3 4 4 5 5
28940 047/300 Read block 1628 len 8192
28941 048/300 Write block 1306 len 32768 data: 4 5 5 6 5 5 5 6
28942 049/300 Read block 3759 len 32768
28943 050/300 Read block 2183 len 4096
28944 051/300 Read block 1449 len 20480
28945 052/300 Write block 2037 len 36864 data: 5 6 6 6 7 5 6 7 7
28946 053/300 Flush
28947 054/300 Write block 4971 len 20480 data: 3 3 3 3 4
28948 055/300 Read block 1702 len 12288
28949 056/300 Write block 685 len 20480 data: 10 12 11 11 7
28950 057/300 Write block 1234 len 24576 data: 5 5 5 5 3 4
28951 058/300 Read block 91 len 32768
28952 059/300 Write block 4975 len 36864 data: 5 4 5 4 4 4 4 3 3
28953 060/300 Read block 3473 len 24576
28954 061/300 Read block 1954 len 16384
28955 062/300 Write block 4701 len 4096 data: 7
28956 063/300 Write block 3737 len 12288 data: 4 4 4
28957 064/300 Read block 2094 len 36864
28958 065/300 Flush
28959 066/300 Flush
28960 067/300 Read block 3000 len 36864
28961 068/300 Read block 3209 len 12288
28962 069/300 Write block 687 len 20480 data: 12 12 8 8 7
28963 070/300 Read block 3135 len 20480
28964 071/300 Read block 217 len 4096
28965 072/300 Read block 257 len 28672
28966 073/300 Read block 4099 len 4096
28967 074/300 Write block 4583 len 4096 data: 4
28968 075/300 Read block 3758 len 12288
28969 076/300 Read block 2931 len 8192
28970 077/300 Write block 2962 len 4096 data: 5
28971 078/300 Write block 4909 len 36864 data: 5 6 6 6 6 6 6 3 2
28972 079/300 Read block 1707 len 8192
28973 080/300 Flush
28974 081/300 Read block 4062 len 40960
28975 082/300 Flush
28976 083/300 Flush
28977 084/300 Flush
28978 085/300 Flush
28979 086/300 Flush
28980 087/300 Write block 1837 len 12288 data: 5 5 5
28981 088/300 Write block 3292 len 8192 data: 4 5
28982 089/300 Write block 920 len 32768 data: 8 6 8 7 6 6 6 5
28983 090/300 Flush
28984 091/300 Write block 2922 len 4096 data: 4
28985 092/300 Write block 1434 len 24576 data: 5 5 6 7 9 10
28986 093/300 Read block 4741 len 20480
28987 094/300 Read block 3349 len 36864
28988 095/300 Read block 2722 len 36864
28989 096/300 Write block 4800 len 32768 data: 3 3 4 4 3 3 5 5
28990 097/300 Write block 2495 len 16384 data: 5 4 4 5
28991 098/300 Read block 1633 len 40960
28992 099/300 Write block 2201 len 12288 data: 6 5 4
28993 100/300 Write block 3315 len 24576 data: 6 6 6 6 7 8
28994 101/300 Write block 4821 len 4096 data: 6
28995 102/300 Read block 2054 len 16384
28996 103/300 Write block 1655 len 36864 data: 6 4 4 6 7 8 8 7 7
28997 104/300 Write block 4714 len 32768 data: 6 6 7 8 6 7 7 6
28998 105/300 Read block 1944 len 32768
28999 106/300 Write block 1593 len 32768 data: 6 7 6 6 5 4 5 5
29000 107/300 Read block 3940 len 20480
29001 108/300 Write block 4535 len 8192 data: 7 6
29002 109/300 Read block 1069 len 20480
29003 110/300 Read block 4186 len 32768
29004 111/300 Write block 701 len 28672 data: 6 6 8 6 4 5 6
29005 112/300 Read block 1015 len 16384
29006 113/300 Write block 116 len 8192 data: 6 6
29007 114/300 Read block 2432 len 20480
29008 115/300 Write block 2666 len 20480 data: 4 3 3 5 6
29009 116/300 Read block 1792 len 4096
29010 117/300 Write block 593 len 20480 data: 6 5 6 4 4
29011 118/300 Flush
29012 119/300 Write block 2552 len 40960 data: 7 7 5 3 3 4 4 3 4 3
29013 120/300 Read block 971 len 20480
29014 121/300 Read block 1563 len 24576
29015 122/300 Write block 4343 len 40960 data: 4 5 4 5 6 5 5 5 4 3
29016 123/300 Read block 1714 len 24576
29017 124/300 Read block 1060 len 8192
29018 125/300 Write block 3081 len 28672 data: 7 9 8 6 8 9 9
29019 126/300 Read block 2490 len 16384
29020 127/300 Write block 2489 len 28672 data: 6 7 8 7 6 6 6
29021 128/300 Read block 1306 len 36864
29022 129/300 Write block 2140 len 8192 data: 5 7
29023 130/300 Read block 4292 len 28672
29024 131/300 Write block 264 len 40960 data: 4 5 5 4 4 6 7 6 6 6
29025 132/300 Write block 3418 len 28672 data: 6 6 4 4 4 5 6
29026 133/300 Write block 2593 len 4096 data: 4
29027 134/300 Flush
29028 135/300 Read block 545 len 8192
29029 136/300 Write block 955 len 36864 data: 7 7 7 6 5 6 6 7 5
29030 137/300 Flush
29031 138/300 Write block 1312 len 36864 data: 6 7 5 4 3 3 3 3 3
29032 139/300 Write block 2686 len 36864 data: 8 7 7 5 4 4 4 4 4
29033 140/300 Read block 144 len 28672
29034 141/300 Read block 1790 len 16384
29035 142/300 Read block 4355 len 20480
29036 143/300 Read block 4138 len 36864
29037 144/300 Write block 2573 len 28672 data: 3 3 3 3 3 3 2
29038 145/300 Write block 2757 len 12288 data: 4 3 3
29039 146/300 Write block 2457 len 28672 data: 2 3 3 4 5 6 6
29040 147/300 Write block 3673 len 28672 data: 3 3 3 3 3 4 5
29041 148/300 Read block 2814 len 4096
29042 149/300 Read block 3658 len 8192
29043 150/300 Read block 2653 len 8192
29044 151/300 Write block 4052 len 36864 data: 2 3 5 6 6 7 7 6 7
29045 152/300 Read block 2753 len 20480
29046 153/300 Read block 293 len 16384
29047 154/300 Read block 1088 len 12288
29048 155/300 Read block 4617 len 32768
29049 156/300 Read block 2948 len 8192
29050 157/300 Read block 1078 len 4096
29051 158/300 Flush
29052 159/300 Write block 1242 len 4096 data: 4
29053 160/300 Read block 4883 len 40960
29054 161/300 Flush
29055 162/300 Write block 4572 len 28672 data: 3 3 3 4 6 5 5
29056 163/300 Flush
29057 164/300 Read block 267 len 32768
29058 165/300 Write block 2103 len 40960 data: 6 6 6 7 7 7 9 7 7 8
29059 166/300 Read block 1088 len 36864
29060 167/300 Read block 1142 len 36864
29061 168/300 Read block 154 len 24576
29062 169/300 Read block 4169 len 32768
29063 170/300 Write block 811 len 24576 data: 5 7 7 6 5 6
29064 171/300 Read block 729 len 28672
29065 172/300 Flush
29066 173/300 Write block 1977 len 4096 data: 7
29067 174/300 Write block 4322 len 8192 data: 7 8
29068 175/300 Read block 3532 len 24576
29069 176/300 Write block 1217 len 16384 data: 3 3 3 4
29070 177/300 Read block 4876 len 12288
29071 178/300 Write block 468 len 32768 data: 4 5 5 5 4 5 5 4
29072 179/300 Read block 506 len 36864
29073 180/300 Read block 263 len 20480
29074 181/300 Write block 3621 len 28672 data: 8 7 7 6 5 3 4
29075 182/300 Read block 3084 len 40960
29076 183/300 Write block 2054 len 12288 data: 5 6 6
29077 184/300 Read block 4015 len 40960
29078 185/300 Write block 46 len 20480 data: 3 3 3 3 5
29079 186/300 Write block 925 len 20480 data: 7 7 6 5 7
29080 187/300 Write block 1666 len 4096 data: 8
29081 188/300 Read block 3148 len 40960
29082 189/300 Write block 398 len 16384 data: 7 7 7 7
29083 190/300 Read block 1015 len 8192
29084 191/300 Write block 2489 len 32768 data: 7 8 9 8 7 7 7 5
29085 192/300 Read block 4466 len 8192
29086 193/300 Read block 3653 len 16384
29087 194/300 Write block 2386 len 40960 data: 5 4 5 5 6 6 6 7 8 8
29088 195/300 Write block 4128 len 4096 data: 4
29089 196/300 Read block 320 len 28672
29090 197/300 Flush
29091 198/300 Write block 3547 len 4096 data: 2
29092 199/300 Read block 1131 len 12288
29093 200/300 Write block 536 len 24576 data: 9 7 7 5 6 7
29094 201/300 Read block 4137 len 16384
29095 202/300 Read block 2261 len 40960
29096 203/300 Read block 2284 len 4096
29097 204/300 Read block 604 len 16384
29098 205/300 Read block 1315 len 8192
29099 206/300 Write block 392 len 40960 data: 9 8 7 4 4 5 8 8 8 8
29100 207/300 Flush
29101 208/300 Read block 2024 len 28672
29102 209/300 Write block 2406 len 16384 data: 6 7 8 7
29103 210/300 Write block 2767 len 4096 data: 7
29104 211/300 Read block 2488 len 16384
29105 212/300 Write block 2145 len 24576 data: 7 7 7 7 8 7
29106 213/300 Write block 2232 len 4096 data: 6
29107 214/300 Read block 4961 len 20480
29108 215/300 Write block 811 len 32768 data: 6 8 8 7 6 7 5 4
29109 216/300 Write block 3911 len 4096 data: 5
29110 217/300 Read block 980 len 36864
29111 218/300 Read block 3838 len 24576
29112 219/300 Read block 3661 len 16384
29113 220/300 Read block 128 len 36864
29114 221/300 Write block 1946 len 4096 data: 2
29115 222/300 Read block 3596 len 40960
29116 223/300 Read block 3034 len 8192
29117 224/300 Write block 752 len 20480 data: 5 4 3 5 5
29118 225/300 Read block 1961 len 24576
29119 226/300 Flush
29120 227/300 Write block 1354 len 20480 data: 3 6 4 4 6
29121 228/300 Read block 2331 len 32768
29122 229/300 Write block 4921 len 8192 data: 4 5
29123 230/300 Write block 4021 len 36864 data: 4 4 3 4 5 4 4 6 6
29124 231/300 Write block 2140 len 36864 data: 6 8 7 7 8 8 8 8 8
29125 232/300 Read block 970 len 28672
29126 233/300 Write block 3249 len 12288 data: 5 6 8
29127 234/300 Read block 934 len 36864
29128 235/300 Read block 400 len 36864
29129 236/300 Read block 3157 len 24576
29130 237/300 Read block 1570 len 8192
29131 238/300 Write block 4366 len 32768 data: 7 6 6 7 6 8 6 8
29132 239/300 Read block 2084 len 4096
29133 240/300 Write block 1381 len 12288 data: 5 6 7
29134 241/300 Flush
29135 242/300 Read block 588 len 4096
29136 243/300 Flush
29137 244/300 Write block 2292 len 8192 data: 10 9
29138 245/300 Write block 281 len 36864 data: 7 8 7 7 8 9 9 9 8
29139 246/300 Read block 3796 len 24576
29140 247/300 Write block 3393 len 20480 data: 4 4 4 3 3
29141 248/300 Write block 4951 len 40960 data: 2 5 4 2 3 4 4 5 5 5
29142 249/300 Write block 3378 len 28672 data: 5 5 5 5 5 4 4
29143 250/300 Read block 3263 len 12288
29144 251/300 Read block 2964 len 24576
29145 252/300 Read block 3189 len 24576
29146 253/300 Flush
29147 254/300 Write block 2711 len 24576 data: 5 4 5 4 4 4
29148 255/300 Write block 686 len 36864 data: 13 13 13 9 9 8 8 9 8
29149 256/300 Write block 3516 len 24576 data: 7 5 6 5 4 4
29150 257/300 Write block 659 len 4096 data: 5
29151 258/300 Write block 59 len 40960 data: 5 5 6 6 3 3 5 5 6 7
29152 259/300 Read block 4765 len 36864
29153 260/300 Write block 1495 len 12288 data: 2 2 3
29154 261/300 Flush
29155 262/300 Flush
29156 263/300 Write block 1772 len 20480 data: 3 3 3 4 4
29157 264/300 Read block 1128 len 8192
29158 265/300 Write block 1369 len 40960 data: 6 5 5 3 2 2 4 5 5 5
29159 266/300 Read block 1081 len 24576
29160 267/300 Read block 552 len 28672
29161 268/300 Read block 4098 len 28672
29162 269/300 Write block 4003 len 36864 data: 7 9 8 10 10 10 10 8 8
29163 270/300 Flush
29164 271/300 Read block 612 len 32768
29165 272/300 Read block 2557 len 28672
29166 273/300 Read block 1637 len 28672
29167 274/300 Read block 1186 len 8192
29168 275/300 Read block 4820 len 36864
29169 276/300 Read block 4405 len 40960
29170 277/300 Write block 1434 len 20480 data: 6 6 7 8 10
29171 278/300 Read block 2182 len 28672
29172 279/300 Read block 2216 len 12288
29173 280/300 Flush
29174 281/300 Read block 859 len 36864
29175 282/300 Read block 2402 len 36864
29176 283/300 Write block 2231 len 16384 data: 8 7 5 5
29177 284/300 Read block 2981 len 16384
29178 285/300 Read block 367 len 16384
29179 286/300 Read block 2229 len 20480
29180 287/300 Read block 1460 len 8192
29181 288/300 Write block 1098 len 28672 data: 7 8 7 7 7 7 7
29182 289/300 Write block 1169 len 8192 data: 6 6
29183 290/300 Write block 2069 len 8192 data: 11 9
29184 291/300 Write block 3698 len 12288 data: 5 5 6
29185 292/300 Write block 1161 len 12288 data: 6 5 4
29186 293/300 Read block 3504 len 20480
29187 294/300 Read block 1453 len 8192
29188 295/300 Read block 3564 len 20480
29189 296/300 Write block 1939 len 28672 data: 4 4 3 4 4 4 3
29190 297/300 Write block 2026 len 4096 data: 3
29191 298/300 Write block 3603 len 28672 data: 6 6 7 6 6 8 10
29192 299/300 Read block 715 len 20480
29193 300/300 Read block 4490 len 32768
29194 301/300 Write block 621 len 4096 data: 5
29195 Main task has work to do, go find it
29196 got dsc Start(1)
29197 start 1
29198 Jan 17 21:17:14.666 INFO request completed, latency_us: 201, response_code: 204, uri: /start/cid/1, method: POST, req_id: 3b44bd40-7688-4444-9053-59b6d27ee84a, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
29199 Replay: started 1, returned:Ok(())
29200 ----------------------------------------------------------------
29201 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
29202 States: Active Offline Active
29203 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29204 8245 Acked Read 6 Done Sent Done false
29205 8246 Acked Read 6 Done Sent Done false
29206 8247 Acked Read 2 Done Sent Done false
29207 8248 Acked Write 6 Done Sent Done false
29208 8249 Acked Read 2 Done Sent Done false
29209 8250 Acked Write 1 Done Sent Done false
29210 8251 Acked Write 10 Done Sent Done false
29211 8252 Acked Read 1 Done Sent Done false
29212 8253 Acked Write 9 Done Sent Done false
29213 8254 Acked Write 1 Done Sent Done false
29214 8255 Acked Write 3 Done Sent Done false
29215 8256 Acked Flush 0 Done Sent Done false
29216 8257 Acked Read 5 Done Sent Done false
29217 8258 Acked Read 4 Done Sent Done false
29218 8259 Acked Read 6 Done Sent Done false
29219 8260 Acked Flush 0 Done Sent Done false
29220 8261 Acked Flush 0 Done Sent Done false
29221 8262 Acked Read 7 Done Sent Done false
29222 8263 Acked Read 10 Done Sent Done false
29223 8264 Acked Read 6 Done Sent Done false
29224 8265 Acked Write 1 Done Sent Done false
29225 8266 Acked Flush 0 Done Sent Done false
29226 8267 Acked Read 4 Done Sent Done false
29227 8268 Acked Read 6 Done Sent Done false
29228 8269 Acked Read 2 Done Sent Done false
29229 8270 Acked Read 6 Done Sent Done false
29230 8271 Acked Read 3 Done Sent Done false
29231 8272 Acked Flush 0 Done Sent Done false
29232 8273 Acked Flush 0 Done Sent Done false
29233 8274 Acked Flush 0 Done Sent Done false
29234 8275 Acked Read 2 Done Sent Done false
29235 8276 Acked Read 1 Done Sent Done false
29236 8277 Acked Write 10 Done Sent Done false
29237 8278 Acked Write 6 Done Sent Done false
29238 8279 Acked Read 3 Done Sent Done false
29239 8280 Acked Read 9 Done Sent Done false
29240 8281 Acked Write 6 Done Sent Done false
29241 8282 Acked Flush 0 Done Sent Done false
29242 8283 Acked Read 2 Done Sent Done false
29243 8284 Acked Read 9 Done Sent Done false
29244 8285 Acked Read 5 Done Sent Done false
29245 8286 Acked Read 8 Done Sent Done false
29246 8287 Acked Read 1 Done Sent Done false
29247 8288 Acked Read 3 Done Sent Done false
29248 8289 Acked Flush 0 Done Sent Done false
29249 8290 Acked Write 5 Done Sent Done false
29250 8291 Acked Read 2 Done Sent Done false
29251 8292 Acked Write 8 Done Sent Done false
29252 8293 Acked Read 8 Done Sent Done false
29253 8294 Acked Read 1 Done Sent Done false
29254 8295 Acked Read 5 Done Sent Done false
29255 8296 Acked Write 9 Done Sent Done false
29256 8297 Acked Flush 0 Done Sent Done false
29257 8298 Acked Write 5 Done Sent Done false
29258 8299 Acked Read 3 Done Sent Done false
29259 8300 Acked Write 5 Done Sent Done false
29260 8301 Acked Write 6 Done Sent Done false
29261 8302 Acked Read 8 Done Sent Done false
29262 8303 Acked Write 9 Done Sent Done false
29263 8304 Acked Read 6 Done Sent Done false
29264 8305 Acked Read 4 Done Sent Done false
29265 8306 Acked Write 1 Done Sent Done false
29266 8307 Acked Write 3 Done Sent Done false
29267 8308 Acked Read 9 Done Sent Done false
29268 8309 Acked Flush 0 Done Sent Done false
29269 8310 Acked Flush 0 Done Sent Done false
29270 8311 Acked Read 9 Done Sent Done false
29271 8312 Acked Read 3 Done Sent Done false
29272 8313 Acked Write 5 Done Sent Done false
29273 8314 Acked Read 5 Done Sent Done false
29274 8315 Acked Read 1 Done Sent Done false
29275 8316 Acked Read 7 Done Sent Done false
29276 8317 Acked Read 1 Done Sent Done false
29277 8318 Acked Write 1 Done Sent Done false
29278 8319 Acked Read 3 Done Sent Done false
29279 8320 Acked Read 2 Done Sent Done false
29280 8321 Acked Write 1 Done Sent Done false
29281 8322 Acked Write 9 Done Sent Done false
29282 8323 Acked Read 2 Done Sent Done false
29283 8324 Acked Flush 0 Done Sent Done false
29284 8325 Acked Read 10 Done Sent Done false
29285 8326 Acked Flush 0 Done Sent Done false
29286 8327 Acked Flush 0 Done Sent Done false
29287 8328 Acked Flush 0 Done Sent Done false
29288 8329 Acked Flush 0 Done Sent Done false
29289 8330 Acked Flush 0 Done Sent Done false
29290 8331 Acked Write 3 Done Sent Done false
29291 8332 Acked Write 2 Done Sent Done false
29292 8333 Acked Write 8 Done Sent Done false
29293 8334 Acked Flush 0 Done Sent Done false
29294 8335 Acked Write 1 Done Sent Done false
29295 8336 Acked Write 6 Done Sent Done false
29296 8337 Acked Read 5 Done Sent Done false
29297 8338 Acked Read 9 Done Sent Done false
29298 8339 Acked Read 9 Done Sent Done false
29299 8340 Acked Write 8 Done Sent Done false
29300 8341 Acked Write 4 Done Sent Done false
29301 8342 Acked Read 10 Done Sent Done false
29302 8343 Acked Write 3 Done Sent Done false
29303 8344 Acked Write 6 Done Sent Done false
29304 8345 Acked Write 1 Done Sent Done false
29305 8346 Acked Read 4 Done Sent Done false
29306 8347 Acked Write 9 Done Sent Done false
29307 8348 Acked Write 8 Done Sent Done false
29308 8349 Acked Read 8 Done Sent Done false
29309 8350 Acked Write 8 Done Sent Done false
29310 8351 Acked Read 5 Done Sent Done false
29311 8352 Acked Write 2 Done Sent Done false
29312 8353 Acked Read 5 Done Sent Done false
29313 8354 Acked Read 8 Done Sent Done false
29314 8355 Acked Write 7 Done Sent Done false
29315 8356 Acked Read 4 Done Sent Done false
29316 8357 Acked Write 2 Done Sent Done false
29317 8358 Acked Read 5 Done Sent Done false
29318 8359 Acked Write 5 Done Sent Done false
29319 8360 Acked Read 1 Done Sent Done false
29320 8361 Acked Write 5 Done Sent Done false
29321 8362 Acked Flush 0 Done Sent Done false
29322 8363 Acked Write 10 Done Sent Done false
29323 8364 Acked Read 5 Done Sent Done false
29324 8365 Acked Read 6 Done Sent Done false
29325 8366 Acked Write 10 Done Sent Done false
29326 8367 Acked Read 6 Done Sent Done false
29327 8368 Acked Read 2 Done Sent Done false
29328 8369 Acked Write 7 Done Sent Done false
29329 8370 Acked Read 4 Done Sent Done false
29330 8371 Acked Write 7 Done Sent Done false
29331 8372 Acked Read 9 Done Sent Done false
29332 8373 Acked Write 2 Done Sent Done false
29333 8374 Acked Read 7 Done Sent Done false
29334 8375 Acked Write 10 Done Sent Done false
29335 8376 Acked Write 7 Done Sent Done false
29336 8377 Acked Write 1 Done Sent Done false
29337 8378 Acked Flush 0 Done Sent Done false
29338 8379 Acked Read 2 Done Sent Done false
29339 8380 Acked Write 9 Done Sent Done false
29340 8381 Acked Flush 0 Done Sent Done false
29341 8382 Acked Write 9 Done Sent Done false
29342 8383 Acked Write 9 Done Sent Done false
29343 8384 Acked Read 7 Done Sent Done false
29344 8385 Acked Read 4 Done Sent Done false
29345 8386 Acked Read 5 Done Sent Done false
29346 8387 Acked Read 9 Done Sent Done false
29347 8388 Acked Write 7 Done Sent Done false
29348 8389 Acked Write 3 Done Sent Done false
29349 8390 Acked Write 7 Done Sent Done false
29350 8391 Acked Write 7 Done Sent Done false
29351 8392 Acked Read 1 Done Sent Done false
29352 8393 Acked Read 2 Done Sent Done false
29353 8394 Acked Read 2 Done Sent Done false
29354 8395 Acked Write 9 Done Sent Done false
29355 8396 Acked Read 5 Done Sent Done false
29356 8397 Acked Read 4 Done Sent Done false
29357 8398 Acked Read 3 Done Sent Done false
29358 8399 Acked Read 8 Done Sent Done false
29359 8400 Acked Read 2 Done Sent Done false
29360 8401 Acked Read 1 Done Sent Done false
29361 8402 Acked Flush 0 Done Sent Done false
29362 8403 Acked Write 1 Done Sent Done false
29363 8404 Acked Read 10 Done Sent Done false
29364 8405 Acked Flush 0 Done Sent Done false
29365 8406 Acked Write 7 Done Sent Done false
29366 8407 Acked Flush 0 Done Sent Done false
29367 8408 Acked Read 8 Done Sent Done false
29368 8409 Acked Write 10 Done Sent Done false
29369 8410 Acked Read 9 Done Sent Done false
29370 8411 Acked Read 9 Done Sent Done false
29371 8412 Acked Read 6 Done Sent Done false
29372 8413 Acked Read 8 Done Sent Done false
29373 8414 Acked Write 6 Done Sent Done false
29374 8415 Acked Read 7 Done Sent Done false
29375 8416 Acked Flush 0 Done Sent Done false
29376 8417 Acked Write 1 Done Sent Done false
29377 8418 Acked Write 2 Done Sent Done false
29378 8419 Acked Read 6 Done Sent Done false
29379 8420 Acked Write 4 Done Sent Done false
29380 8421 Acked Read 3 Done Sent Done false
29381 8422 Acked Write 8 Done Sent Done false
29382 8423 Acked Read 9 Done Sent Done false
29383 8424 Acked Read 5 Done Sent Done false
29384 8425 Acked Write 7 Done Sent Done false
29385 8426 Acked Read 10 Done Sent Done false
29386 8427 Acked Write 3 Done Sent Done false
29387 8428 Acked Read 10 Done Sent Done false
29388 8429 Acked Write 5 Done Sent Done false
29389 8430 Acked Write 5 Done Sent Done false
29390 8431 Acked Write 1 Done Sent Done false
29391 8432 Acked Read 10 Done Sent Done false
29392 8433 Acked Write 4 Done Sent Done false
29393 8434 Acked Read 2 Done Sent Done false
29394 8435 Acked Write 8 Done Sent Done false
29395 8436 Acked Read 2 Done Sent Done false
29396 8437 Acked Read 4 Done Sent Done false
29397 8438 Acked Write 10 Done Sent Done false
29398 8439 Acked Write 1 Done Sent Done false
29399 8440 Acked Read 7 Done Sent Done false
29400 8441 Acked Flush 0 Done Sent Done false
29401 8442 Acked Write 1 Done Sent Done false
29402 8443 Acked Read 3 Done Sent Done false
29403 8444 Acked Write 6 Done Sent Done false
29404 8445 Acked Read 4 Done Sent Done false
29405 8446 Acked Read 10 Done Sent Done false
29406 8447 Acked Read 1 Done Sent Done false
29407 8448 Acked Read 4 Done Sent Done false
29408 8449 Acked Read 2 Done Sent Done false
29409 8450 Acked Write 10 Done Sent Done false
29410 8451 Acked Flush 0 Done Sent Done false
29411 8452 Acked Read 7 Done Sent Done false
29412 8453 Acked Write 4 Done Sent Done false
29413 8454 Acked Write 1 Done Sent Done false
29414 8455 Acked Read 4 Done Sent Done false
29415 8456 Acked Write 6 Done Sent Done false
29416 8457 Acked Write 1 Done Sent Done false
29417 8458 Acked Read 5 Done Sent Done false
29418 8459 Acked Write 8 Done Sent Done false
29419 8460 Acked Write 1 Done Sent Done false
29420 8461 Acked Read 9 Done Sent Done false
29421 8462 Acked Read 6 Done Sent Done false
29422 8463 Acked Read 4 Done Sent Done false
29423 8464 Acked Read 9 Done Sent Done false
29424 8465 Acked Write 1 Done Sent Done false
29425 8466 Acked Read 10 Done Sent Done false
29426 8467 Acked Read 2 Done Sent Done false
29427 8468 Acked Write 5 Done Sent Done false
29428 8469 Acked Read 6 Done Sent Done false
29429 8470 Acked Flush 0 Done Sent Done false
29430 8471 Acked Write 5 Done Sent Done false
29431 8472 Acked Read 8 Done Sent Done false
29432 8473 Acked Write 2 Done Sent Done false
29433 8474 Acked Write 9 Done Sent Done false
29434 8475 Acked Write 9 Done Sent Done false
29435 8476 Acked Read 7 Done Sent Done false
29436 8477 Acked Write 3 Done Sent Done false
29437 8478 Acked Read 9 Done Sent Done false
29438 8479 Acked Read 9 Done Sent Done false
29439 8480 Acked Read 6 Done Sent Done false
29440 8481 Acked Read 2 Done Sent Done false
29441 8482 Acked Write 8 Done Sent Done false
29442 8483 Acked Read 1 Done Sent Done false
29443 8484 Acked Write 3 Done Sent Done false
29444 8485 Acked Flush 0 Done Sent Done false
29445 8486 Acked Read 1 Done Sent Done false
29446 8487 Acked Flush 0 Done Sent Done false
29447 8488 Acked Write 2 Done Sent Done false
29448 8489 Acked Write 9 Done Sent Done false
29449 8490 Acked Read 6 Done Sent Done false
29450 8491 Acked Write 5 Done Sent Done false
29451 8492 Acked Write 10 Done Sent Done false
29452 8493 Acked Write 7 Done Sent Done false
29453 8494 Acked Read 3 Done Sent Done false
29454 8495 Acked Read 6 Done Sent Done false
29455 8496 Acked Read 6 Done Sent Done false
29456 8497 Acked Flush 0 Done Sent Done false
29457 8498 Acked Write 6 Done Sent Done false
29458 8499 Acked Write 9 Done Sent Done false
29459 8500 Acked Write 6 Done Sent Done false
29460 8501 Acked Write 1 Done Sent Done false
29461 8502 Acked Write 10 Done Sent Done false
29462 8503 Acked Read 9 Done Sent Done false
29463 8504 Acked Write 3 Done Sent Done false
29464 8505 Acked Flush 0 Done Sent Done false
29465 8506 Acked Flush 0 Done Sent Done false
29466 8507 Acked Write 5 Done Sent Done false
29467 8508 Acked Read 2 Done Sent Done false
29468 8509 Acked Write 10 Done Sent Done false
29469 8510 Acked Read 6 Done Sent Done false
29470 8511 Acked Read 7 Done Sent Done false
29471 8512 Acked Read 7 Done Sent Done false
29472 8513 Acked Write 9 Done Sent Done false
29473 8514 Acked Flush 0 Done Sent Done false
29474 8515 Acked Read 8 Done Sent Done false
29475 8516 Acked Read 7 Done Sent Done false
29476 8517 Acked Read 7 Done Sent Done false
29477 8518 Acked Read 2 Done Sent Done false
29478 8519 Acked Read 9 Done Sent Done false
29479 8520 Acked Read 10 Done Sent Done false
29480 8521 Acked Write 5 Done Sent Done false
29481 8522 Acked Read 7 Done Sent Done false
29482 8523 Acked Read 3 Done Sent Done false
29483 8524 Acked Flush 0 Done Sent Done false
29484 8525 Acked Read 9 Done Sent Done false
29485 8526 Acked Read 9 Done Sent Done false
29486 8527 Acked Write 4 Done Sent Done false
29487 8528 Acked Read 4 Done Sent Done false
29488 8529 Acked Read 4 Done Sent Done false
29489 8530 Acked Read 5 Done Sent Done false
29490 8531 Acked Read 2 Done Sent Done false
29491 8532 Acked Write 7 Done Sent Done false
29492 8533 Acked Write 2 Done Sent Done false
29493 8534 Acked Write 2 Done Sent Done false
29494 8535 Acked Write 3 Done Sent Done false
29495 8536 Acked Write 3 Done Sent Done false
29496 8537 Acked Read 5 Done Sent Done false
29497 8538 Acked Read 2 Done Sent Done false
29498 8539 Acked Read 5 Done Sent Done false
29499 8540 Acked Write 7 Done Sent Done false
29500 8541 Acked Write 1 Done Sent Done false
29501 8542 Acked Write 7 Done Sent Done false
29502 8543 Acked Read 5 Done Sent Done false
29503 8544 Acked Read 8 Done Sent Done false
29504 8545 Acked Write 1 Done Sent Done false
29505 STATES DS:0 DS:1 DS:2 TOTAL
29506 Sent 0 301 0 301
29507 Done 301 0 301 602
29508 Skipped 0 0 0 0
29509 Error 0 0 0 0
29510 Last Flush: 8524 8244 8524
29511 Downstairs last five completed: 8244 8243 8242 8241 8240
29512 Upstairs last five completed: 8545 8544 8543 8542 8541
29513 CLIENT: Up:0 ds:301 act:2
29514 [8820] got start action
29515 [8820] I am going to restart
29516 Starting downstairs at port 8820
29517 Make output file at "/tmp/dsc/downstairs-8820.txt"
29518 [8820][1] reports Starting
29519 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1225)
29520 [8820][1] reports Running
29521 ----------------------------------------------------------------
29522 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29523 States: Active Offline Active
29524 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29525 8245 Acked Read 6 Done Sent Done false
29526 8246 Acked Read 6 Done Sent Done false
29527 8247 Acked Read 2 Done Sent Done false
29528 8248 Acked Write 6 Done Sent Done false
29529 8249 Acked Read 2 Done Sent Done false
29530 8250 Acked Write 1 Done Sent Done false
29531 8251 Acked Write 10 Done Sent Done false
29532 8252 Acked Read 1 Done Sent Done false
29533 8253 Acked Write 9 Done Sent Done false
29534 8254 Acked Write 1 Done Sent Done false
29535 8255 Acked Write 3 Done Sent Done false
29536 8256 Acked Flush 0 Done Sent Done false
29537 8257 Acked Read 5 Done Sent Done false
29538 8258 Acked Read 4 Done Sent Done false
29539 8259 Acked Read 6 Done Sent Done false
29540 8260 Acked Flush 0 Done Sent Done false
29541 8261 Acked Flush 0 Done Sent Done false
29542 8262 Acked Read 7 Done Sent Done false
29543 8263 Acked Read 10 Done Sent Done false
29544 8264 Acked Read 6 Done Sent Done false
29545 8265 Acked Write 1 Done Sent Done false
29546 8266 Acked Flush 0 Done Sent Done false
29547 8267 Acked Read 4 Done Sent Done false
29548 8268 Acked Read 6 Done Sent Done false
29549 8269 Acked Read 2 Done Sent Done false
29550 8270 Acked Read 6 Done Sent Done false
29551 8271 Acked Read 3 Done Sent Done false
29552 8272 Acked Flush 0 Done Sent Done false
29553 8273 Acked Flush 0 Done Sent Done false
29554 8274 Acked Flush 0 Done Sent Done false
29555 8275 Acked Read 2 Done Sent Done false
29556 8276 Acked Read 1 Done Sent Done false
29557 8277 Acked Write 10 Done Sent Done false
29558 8278 Acked Write 6 Done Sent Done false
29559 8279 Acked Read 3 Done Sent Done false
29560 8280 Acked Read 9 Done Sent Done false
29561 8281 Acked Write 6 Done Sent Done false
29562 8282 Acked Flush 0 Done Sent Done false
29563 8283 Acked Read 2 Done Sent Done false
29564 8284 Acked Read 9 Done Sent Done false
29565 8285 Acked Read 5 Done Sent Done false
29566 8286 Acked Read 8 Done Sent Done false
29567 8287 Acked Read 1 Done Sent Done false
29568 8288 Acked Read 3 Done Sent Done false
29569 8289 Acked Flush 0 Done Sent Done false
29570 8290 Acked Write 5 Done Sent Done false
29571 8291 Acked Read 2 Done Sent Done false
29572 8292 Acked Write 8 Done Sent Done false
29573 8293 Acked Read 8 Done Sent Done false
29574 8294 Acked Read 1 Done Sent Done false
29575 8295 Acked Read 5 Done Sent Done false
29576 8296 Acked Write 9 Done Sent Done false
29577 8297 Acked Flush 0 Done Sent Done false
29578 8298 Acked Write 5 Done Sent Done false
29579 8299 Acked Read 3 Done Sent Done false
29580 8300 Acked Write 5 Done Sent Done false
29581 8301 Acked Write 6 Done Sent Done false
29582 8302 Acked Read 8 Done Sent Done false
29583 8303 Acked Write 9 Done Sent Done false
29584 8304 Acked Read 6 Done Sent Done false
29585 8305 Acked Read 4 Done Sent Done false
29586 8306 Acked Write 1 Done Sent Done false
29587 8307 Acked Write 3 Done Sent Done false
29588 8308 Acked Read 9 Done Sent Done false
29589 8309 Acked Flush 0 Done Sent Done false
29590 8310 Acked Flush 0 Done Sent Done false
29591 8311 Acked Read 9 Done Sent Done false
29592 8312 Acked Read 3 Done Sent Done false
29593 8313 Acked Write 5 Done Sent Done false
29594 8314 Acked Read 5 Done Sent Done false
29595 8315 Acked Read 1 Done Sent Done false
29596 8316 Acked Read 7 Done Sent Done false
29597 8317 Acked Read 1 Done Sent Done false
29598 8318 Acked Write 1 Done Sent Done false
29599 8319 Acked Read 3 Done Sent Done false
29600 8320 Acked Read 2 Done Sent Done false
29601 8321 Acked Write 1 Done Sent Done false
29602 8322 Acked Write 9 Done Sent Done false
29603 8323 Acked Read 2 Done Sent Done false
29604 8324 Acked Flush 0 Done Sent Done false
29605 8325 Acked Read 10 Done Sent Done false
29606 8326 Acked Flush 0 Done Sent Done false
29607 8327 Acked Flush 0 Done Sent Done false
29608 8328 Acked Flush 0 Done Sent Done false
29609 8329 Acked Flush 0 Done Sent Done false
29610 8330 Acked Flush 0 Done Sent Done false
29611 8331 Acked Write 3 Done Sent Done false
29612 8332 Acked Write 2 Done Sent Done false
29613 8333 Acked Write 8 Done Sent Done false
29614 8334 Acked Flush 0 Done Sent Done false
29615 8335 Acked Write 1 Done Sent Done false
29616 8336 Acked Write 6 Done Sent Done false
29617 8337 Acked Read 5 Done Sent Done false
29618 8338 Acked Read 9 Done Sent Done false
29619 8339 Acked Read 9 Done Sent Done false
29620 8340 Acked Write 8 Done Sent Done false
29621 8341 Acked Write 4 Done Sent Done false
29622 8342 Acked Read 10 Done Sent Done false
29623 8343 Acked Write 3 Done Sent Done false
29624 8344 Acked Write 6 Done Sent Done false
29625 8345 Acked Write 1 Done Sent Done false
29626 8346 Acked Read 4 Done Sent Done false
29627 8347 Acked Write 9 Done Sent Done false
29628 8348 Acked Write 8 Done Sent Done false
29629 8349 Acked Read 8 Done Sent Done false
29630 8350 Acked Write 8 Done Sent Done false
29631 8351 Acked Read 5 Done Sent Done false
29632 8352 Acked Write 2 Done Sent Done false
29633 8353 Acked Read 5 Done Sent Done false
29634 8354 Acked Read 8 Done Sent Done false
29635 8355 Acked Write 7 Done Sent Done false
29636 8356 Acked Read 4 Done Sent Done false
29637 8357 Acked Write 2 Done Sent Done false
29638 8358 Acked Read 5 Done Sent Done false
29639 8359 Acked Write 5 Done Sent Done false
29640 8360 Acked Read 1 Done Sent Done false
29641 8361 Acked Write 5 Done Sent Done false
29642 8362 Acked Flush 0 Done Sent Done false
29643 8363 Acked Write 10 Done Sent Done false
29644 8364 Acked Read 5 Done Sent Done false
29645 8365 Acked Read 6 Done Sent Done false
29646 8366 Acked Write 10 Done Sent Done false
29647 8367 Acked Read 6 Done Sent Done false
29648 8368 Acked Read 2 Done Sent Done false
29649 8369 Acked Write 7 Done Sent Done false
29650 8370 Acked Read 4 Done Sent Done false
29651 8371 Acked Write 7 Done Sent Done false
29652 8372 Acked Read 9 Done Sent Done false
29653 8373 Acked Write 2 Done Sent Done false
29654 8374 Acked Read 7 Done Sent Done false
29655 8375 Acked Write 10 Done Sent Done false
29656 8376 Acked Write 7 Done Sent Done false
29657 8377 Acked Write 1 Done Sent Done false
29658 8378 Acked Flush 0 Done Sent Done false
29659 8379 Acked Read 2 Done Sent Done false
29660 8380 Acked Write 9 Done Sent Done false
29661 8381 Acked Flush 0 Done Sent Done false
29662 8382 Acked Write 9 Done Sent Done false
29663 8383 Acked Write 9 Done Sent Done false
29664 8384 Acked Read 7 Done Sent Done false
29665 8385 Acked Read 4 Done Sent Done false
29666 8386 Acked Read 5 Done Sent Done false
29667 8387 Acked Read 9 Done Sent Done false
29668 8388 Acked Write 7 Done Sent Done false
29669 8389 Acked Write 3 Done Sent Done false
29670 8390 Acked Write 7 Done Sent Done false
29671 8391 Acked Write 7 Done Sent Done false
29672 8392 Acked Read 1 Done Sent Done false
29673 8393 Acked Read 2 Done Sent Done false
29674 8394 Acked Read 2 Done Sent Done false
29675 8395 Acked Write 9 Done Sent Done false
29676 8396 Acked Read 5 Done Sent Done false
29677 8397 Acked Read 4 Done Sent Done false
29678 8398 Acked Read 3 Done Sent Done false
29679 8399 Acked Read 8 Done Sent Done false
29680 8400 Acked Read 2 Done Sent Done false
29681 8401 Acked Read 1 Done Sent Done false
29682 8402 Acked Flush 0 Done Sent Done false
29683 8403 Acked Write 1 Done Sent Done false
29684 8404 Acked Read 10 Done Sent Done false
29685 8405 Acked Flush 0 Done Sent Done false
29686 8406 Acked Write 7 Done Sent Done false
29687 8407 Acked Flush 0 Done Sent Done false
29688 8408 Acked Read 8 Done Sent Done false
29689 8409 Acked Write 10 Done Sent Done false
29690 8410 Acked Read 9 Done Sent Done false
29691 8411 Acked Read 9 Done Sent Done false
29692 8412 Acked Read 6 Done Sent Done false
29693 8413 Acked Read 8 Done Sent Done false
29694 8414 Acked Write 6 Done Sent Done false
29695 8415 Acked Read 7 Done Sent Done false
29696 8416 Acked Flush 0 Done Sent Done false
29697 8417 Acked Write 1 Done Sent Done false
29698 8418 Acked Write 2 Done Sent Done false
29699 8419 Acked Read 6 Done Sent Done false
29700 8420 Acked Write 4 Done Sent Done false
29701 8421 Acked Read 3 Done Sent Done false
29702 8422 Acked Write 8 Done Sent Done false
29703 8423 Acked Read 9 Done Sent Done false
29704 8424 Acked Read 5 Done Sent Done false
29705 8425 Acked Write 7 Done Sent Done false
29706 8426 Acked Read 10 Done Sent Done false
29707 8427 Acked Write 3 Done Sent Done false
29708 8428 Acked Read 10 Done Sent Done false
29709 8429 Acked Write 5 Done Sent Done false
29710 8430 Acked Write 5 Done Sent Done false
29711 8431 Acked Write 1 Done Sent Done false
29712 8432 Acked Read 10 Done Sent Done false
29713 8433 Acked Write 4 Done Sent Done false
29714 8434 Acked Read 2 Done Sent Done false
29715 8435 Acked Write 8 Done Sent Done false
29716 8436 Acked Read 2 Done Sent Done false
29717 8437 Acked Read 4 Done Sent Done false
29718 8438 Acked Write 10 Done Sent Done false
29719 8439 Acked Write 1 Done Sent Done false
29720 8440 Acked Read 7 Done Sent Done false
29721 8441 Acked Flush 0 Done Sent Done false
29722 8442 Acked Write 1 Done Sent Done false
29723 8443 Acked Read 3 Done Sent Done false
29724 8444 Acked Write 6 Done Sent Done false
29725 8445 Acked Read 4 Done Sent Done false
29726 8446 Acked Read 10 Done Sent Done false
29727 8447 Acked Read 1 Done Sent Done false
29728 8448 Acked Read 4 Done Sent Done false
29729 8449 Acked Read 2 Done Sent Done false
29730 8450 Acked Write 10 Done Sent Done false
29731 8451 Acked Flush 0 Done Sent Done false
29732 8452 Acked Read 7 Done Sent Done false
29733 8453 Acked Write 4 Done Sent Done false
29734 8454 Acked Write 1 Done Sent Done false
29735 8455 Acked Read 4 Done Sent Done false
29736 8456 Acked Write 6 Done Sent Done false
29737 8457 Acked Write 1 Done Sent Done false
29738 8458 Acked Read 5 Done Sent Done false
29739 8459 Acked Write 8 Done Sent Done false
29740 8460 Acked Write 1 Done Sent Done false
29741 8461 Acked Read 9 Done Sent Done false
29742 8462 Acked Read 6 Done Sent Done false
29743 8463 Acked Read 4 Done Sent Done false
29744 8464 Acked Read 9 Done Sent Done false
29745 8465 Acked Write 1 Done Sent Done false
29746 8466 Acked Read 10 Done Sent Done false
29747 8467 Acked Read 2 Done Sent Done false
29748 8468 Acked Write 5 Done Sent Done false
29749 8469 Acked Read 6 Done Sent Done false
29750 8470 Acked Flush 0 Done Sent Done false
29751 8471 Acked Write 5 Done Sent Done false
29752 8472 Acked Read 8 Done Sent Done false
29753 8473 Acked Write 2 Done Sent Done false
29754 8474 Acked Write 9 Done Sent Done false
29755 8475 Acked Write 9 Done Sent Done false
29756 8476 Acked Read 7 Done Sent Done false
29757 8477 Acked Write 3 Done Sent Done false
29758 8478 Acked Read 9 Done Sent Done false
29759 8479 Acked Read 9 Done Sent Done false
29760 8480 Acked Read 6 Done Sent Done false
29761 8481 Acked Read 2 Done Sent Done false
29762 8482 Acked Write 8 Done Sent Done false
29763 8483 Acked Read 1 Done Sent Done false
29764 8484 Acked Write 3 Done Sent Done false
29765 8485 Acked Flush 0 Done Sent Done false
29766 8486 Acked Read 1 Done Sent Done false
29767 8487 Acked Flush 0 Done Sent Done false
29768 8488 Acked Write 2 Done Sent Done false
29769 8489 Acked Write 9 Done Sent Done false
29770 8490 Acked Read 6 Done Sent Done false
29771 8491 Acked Write 5 Done Sent Done false
29772 8492 Acked Write 10 Done Sent Done false
29773 8493 Acked Write 7 Done Sent Done false
29774 8494 Acked Read 3 Done Sent Done false
29775 8495 Acked Read 6 Done Sent Done false
29776 8496 Acked Read 6 Done Sent Done false
29777 8497 Acked Flush 0 Done Sent Done false
29778 8498 Acked Write 6 Done Sent Done false
29779 8499 Acked Write 9 Done Sent Done false
29780 8500 Acked Write 6 Done Sent Done false
29781 8501 Acked Write 1 Done Sent Done false
29782 8502 Acked Write 10 Done Sent Done false
29783 8503 Acked Read 9 Done Sent Done false
29784 8504 Acked Write 3 Done Sent Done false
29785 8505 Acked Flush 0 Done Sent Done false
29786 8506 Acked Flush 0 Done Sent Done false
29787 8507 Acked Write 5 Done Sent Done false
29788 8508 Acked Read 2 Done Sent Done false
29789 8509 Acked Write 10 Done Sent Done false
29790 8510 Acked Read 6 Done Sent Done false
29791 8511 Acked Read 7 Done Sent Done false
29792 8512 Acked Read 7 Done Sent Done false
29793 8513 Acked Write 9 Done Sent Done false
29794 8514 Acked Flush 0 Done Sent Done false
29795 8515 Acked Read 8 Done Sent Done false
29796 8516 Acked Read 7 Done Sent Done false
29797 8517 Acked Read 7 Done Sent Done false
29798 8518 Acked Read 2 Done Sent Done false
29799 8519 Acked Read 9 Done Sent Done false
29800 8520 Acked Read 10 Done Sent Done false
29801 8521 Acked Write 5 Done Sent Done false
29802 8522 Acked Read 7 Done Sent Done false
29803 8523 Acked Read 3 Done Sent Done false
29804 8524 Acked Flush 0 Done Sent Done false
29805 8525 Acked Read 9 Done Sent Done false
29806 8526 Acked Read 9 Done Sent Done false
29807 8527 Acked Write 4 Done Sent Done false
29808 8528 Acked Read 4 Done Sent Done false
29809 8529 Acked Read 4 Done Sent Done false
29810 8530 Acked Read 5 Done Sent Done false
29811 8531 Acked Read 2 Done Sent Done false
29812 8532 Acked Write 7 Done Sent Done false
29813 8533 Acked Write 2 Done Sent Done false
29814 8534 Acked Write 2 Done Sent Done false
29815 8535 Acked Write 3 Done Sent Done false
29816 8536 Acked Write 3 Done Sent Done false
29817 8537 Acked Read 5 Done Sent Done false
29818 8538 Acked Read 2 Done Sent Done false
29819 8539 Acked Read 5 Done Sent Done false
29820 8540 Acked Write 7 Done Sent Done false
29821 8541 Acked Write 1 Done Sent Done false
29822 8542 Acked Write 7 Done Sent Done false
29823 8543 Acked Read 5 Done Sent Done false
29824 8544 Acked Read 8 Done Sent Done false
29825 8545 Acked Write 1 Done Sent Done false
29826 8546 Acked Flush 0 Done Sent Done false
29827 STATES DS:0 DS:1 DS:2 TOTAL
29828 Sent 0 302 0 302
29829 Done 302 0 302 604
29830 Skipped 0 0 0 0
29831 Error 0 0 0 0
29832 Last Flush: 8546 8244 8546
29833 Downstairs last five completed: 8244 8243 8242 8241 8240
29834 Upstairs last five completed: 8546 8545 8544 8543 8542
29835 CLIENT: Up:0 ds:302 act:2
298362025-01-17T21:17:20.428ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
298372025-01-17T21:17:20.428ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:35400) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
298382025-01-17T21:17:20.429ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
298392025-01-17T21:17:20.430ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
298402025-01-17T21:17:20.430ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
298412025-01-17T21:17:20.430ZINFOcrucible: send last flush ID to this DS: 8244 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
298422025-01-17T21:17:20.430ZINFOcrucible: Replied this last flush ID: 8244 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
298432025-01-17T21:17:20.430ZINFOcrucible: [1] client re-new 302 jobs since flush 8244 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
298442025-01-17T21:17:20.430ZINFOcrucible: [1] Replayed 302 jobs since flush: 8244 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
29845 ----------------------------------------------------------------
29846 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
29847 Downstairs last five completed: 8546 8545 8544 8543 8542
29848 Upstairs last five completed: 8546 8545 8544 8543 8542
29849 CLIENT: Up:0 ds:0 act:3
29850 Replay: All jobs finished
29851 Main task has work to do, go find it
29852 got dsc Stop(0)
29853 stop 0
29854 Jan 17 21:17:22.671 INFO request completed, latency_us: 328, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 173385a0-209c-4626-8697-6d53c730801a, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
29855 [8810] Got stop action so:false kr:false
29856 Jan 17 21:17:22.672 INFO request completed, latency_us: 203, response_code: 200, uri: /state/cid/0, method: GET, req_id: 44d29e48-a009-4b0b-9457-78975cb54eb9, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
298572025-01-17T21:17:22.672ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
298582025-01-17T21:17:22.672ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
298592025-01-17T21:17:22.672ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
298602025-01-17T21:17:22.672ZINFOcrucible: client task is exiting = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
29861 [8810] Exited with: ExitStatus(unix_wait_status(9))
29862 [8810][0] reports Exit
29863 Jan 17 21:17:26.674 INFO request completed, latency_us: 188, response_code: 200, uri: /state/cid/0, method: GET, req_id: 473e0366-c48e-4564-ac46-41ea54344960, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
29864 001/300 Flush
29865 002/300 Write block 1718 len 20480 data: 3 5 6 7 5
29866 003/300 Read block 3243 len 24576
29867 004/300 Flush
29868 005/300 Write block 911 len 20480 data: 3 3 4 5 6
29869 006/300 Write block 616 len 16384 data: 4 4 4 6
29870 007/300 Write block 3648 len 40960 data: 5 4 4 4 3 3 2 2 2 2
29871 008/300 Read block 4280 len 8192
29872 009/300 Write block 2355 len 16384 data: 5 5 4 4
29873 010/300 Read block 77 len 28672
29874 011/300 Write block 1059 len 24576 data: 8 7 7 9 7 8
29875 012/300 Read block 3957 len 20480
29876 013/300 Write block 4931 len 8192 data: 4 5
29877 014/300 Read block 1642 len 24576
29878 015/300 Read block 1870 len 40960
29879 016/300 Read block 4395 len 24576
29880 017/300 Write block 1682 len 40960 data: 4 4 4 3 3 3 4 3 3 4
29881 018/300 Flush
29882 019/300 Flush
29883 020/300 Read block 97 len 12288
29884 021/300 Write block 1671 len 36864 data: 3 3 3 4 6 6 6 7 8
29885 022/300 Read block 2004 len 4096
29886 023/300 Read block 1101 len 24576
29887 024/300 Read block 1569 len 32768
29888 025/300 Write block 4493 len 4096 data: 4
29889 026/300 Flush
29890 027/300 Read block 1473 len 36864
29891 028/300 Flush
29892 029/300 Write block 4395 len 20480 data: 5 4 6 5 5
29893 030/300 Flush
29894 031/300 Flush
29895 032/300 Read block 1756 len 12288
29896 033/300 Read block 3928 len 16384
29897 034/300 Write block 1640 len 32768 data: 4 5 5 4 3 3 4 4
29898 035/300 Flush
29899 036/300 Read block 514 len 8192
29900 037/300 Write block 1705 len 8192 data: 4 5
29901 038/300 Read block 4227 len 40960
29902 039/300 Read block 721 len 24576
29903 040/300 Write block 3486 len 28672 data: 3 4 4 4 3 3 3
29904 041/300 Write block 4688 len 12288 data: 8 6 6
29905 042/300 Write block 2944 len 40960 data: 9 8 6 5 5 4 3 3 3 3
29906 043/300 Flush
29907 044/300 Read block 2410 len 12288
29908 045/300 Write block 965 len 20480 data: 7 6 6 6 7
29909 046/300 Read block 3974 len 12288
29910 047/300 Read block 2000 len 36864
29911 048/300 Write block 2003 len 40960 data: 2 2 2 2 2 4 4 5 6 5
29912 049/300 Read block 4924 len 12288
29913 050/300 Read block 4039 len 20480
29914 051/300 Read block 605 len 24576
29915 052/300 Read block 1633 len 8192
29916 053/300 Write block 3939 len 24576 data: 6 6 3 3 4 4
29917 054/300 Write block 1518 len 28672 data: 8 7 7 7 7 8 9
29918 055/300 Write block 2956 len 24576 data: 4 4 4 3 3 3
29919 056/300 Write block 2333 len 28672 data: 8 7 7 6 7 6 6
29920 057/300 Write block 408 len 12288 data: 5 5 5
29921 058/300 Write block 1752 len 40960 data: 4 5 5 4 5 5 6 3 3 3
29922 059/300 Flush
29923 060/300 Flush
29924 061/300 Write block 4237 len 32768 data: 6 4 4 2 2 4 4 4
29925 062/300 Read block 2968 len 20480
29926 063/300 Read block 1616 len 20480
29927 064/300 Read block 4676 len 20480
29928 065/300 Read block 3954 len 36864
29929 066/300 Flush
29930 067/300 Read block 844 len 40960
29931 068/300 Read block 590 len 36864
29932 069/300 Write block 4361 len 32768 data: 4 5 6 7 8 8 7 7
29933 070/300 Write block 109 len 4096 data: 6
29934 071/300 Flush
29935 072/300 Write block 3205 len 4096 data: 8
29936 073/300 Flush
29937 074/300 Read block 3992 len 4096
29938 075/300 Flush
29939 076/300 Write block 2613 len 28672 data: 3 4 4 4 4 3 4
29940 077/300 Read block 4177 len 32768
29941 078/300 Write block 310 len 16384 data: 4 4 3 5
29942 079/300 Read block 1191 len 32768
29943 080/300 Read block 4547 len 20480
29944 081/300 Write block 1093 len 40960 data: 4 3 5 6 6 8 9 8 8 8
29945 082/300 Write block 4257 len 32768 data: 3 3 3 3 7 7 8 4
29946 083/300 Read block 2220 len 4096
29947 084/300 Write block 1129 len 12288 data: 5 4 3
29948 085/300 Write block 1858 len 28672 data: 5 5 4 6 5 5 6
29949 086/300 Write block 1548 len 40960 data: 4 2 2 2 2 3 4 4 4 6
29950 087/300 Read block 4743 len 8192
29951 088/300 Read block 4690 len 32768
29952 089/300 Read block 3806 len 20480
29953 090/300 Read block 946 len 24576
29954 091/300 Write block 2374 len 28672 data: 7 6 6 6 5 6 5
29955 092/300 Read block 470 len 16384
29956 093/300 Write block 1194 len 40960 data: 6 7 8 7 7 7 5 4 3 6
29957 094/300 Read block 2188 len 4096
29958 095/300 Write block 2410 len 28672 data: 7 7 5 4 5 4 4
29959 096/300 Write block 1852 len 40960 data: 4 4 4 5 4 4 6 6 5 7
29960 097/300 Read block 3232 len 28672
29961 098/300 Read block 2054 len 36864
29962 099/300 Read block 188 len 4096
29963 100/300 Read block 3133 len 12288
29964 101/300 Write block 289 len 4096 data: 9
29965 102/300 Write block 903 len 36864 data: 6 5 6 5 5 4 2 2 4
29966 103/300 Write block 2627 len 4096 data: 6
29967 104/300 Write block 3187 len 20480 data: 6 6 4 6 6
29968 105/300 Read block 1365 len 12288
29969 106/300 Read block 1587 len 40960
29970 107/300 Read block 1362 len 16384
29971 108/300 Write block 4854 len 16384 data: 5 4 5 5
29972 109/300 Flush
29973 110/300 Write block 799 len 32768 data: 3 3 3 5 6 7 7 6
29974 111/300 Write block 4328 len 40960 data: 5 6 7 7 5 6 7 7 7 7
29975 112/300 Read block 2859 len 8192
29976 113/300 Read block 2204 len 32768
29977 114/300 Read block 4777 len 28672
29978 115/300 Read block 3096 len 16384
29979 116/300 Read block 4175 len 8192
29980 117/300 Write block 4452 len 8192 data: 4 4
29981 118/300 Flush
29982 119/300 Read block 4944 len 32768
29983 120/300 Read block 1230 len 28672
29984 121/300 Write block 945 len 24576 data: 6 6 4 4 4 5
29985 122/300 Read block 4100 len 8192
29986 123/300 Write block 4743 len 24576 data: 4 5 5 6 6 4
29987 124/300 Flush
29988 125/300 Read block 752 len 24576
29989 126/300 Write block 2643 len 36864 data: 6 7 8 7 5 5 6 5 5
29990 127/300 Read block 3528 len 40960
29991 128/300 Read block 1872 len 20480
29992 129/300 Write block 213 len 40960 data: 7 5 6 6 6 5 6 6 6 6
29993 130/300 Write block 1722 len 4096 data: 6
29994 131/300 Read block 686 len 24576
29995 132/300 Write block 4367 len 8192 data: 8 8
29996 133/300 Write block 2096 len 4096 data: 4
29997 134/300 Read block 4814 len 28672
29998 135/300 Read block 1952 len 40960
29999 136/300 Write block 2226 len 4096 data: 3
30000 137/300 Read block 1388 len 8192
30001 138/300 Write block 3621 len 4096 data: 9
30002 139/300 Read block 2362 len 36864
30003 140/300 Write block 3242 len 40960 data: 6 4 5 6 6 5 5 6 7 9
30004 141/300 Read block 2801 len 4096
30005 142/300 Write block 3216 len 32768 data: 5 5 6 4 2 4 4 5
30006 143/300 Read block 772 len 32768
30007 144/300 Write block 3122 len 12288 data: 8 7 6
30008 145/300 Read block 2202 len 40960
30009 146/300 Read block 2265 len 40960
30010 147/300 Write block 700 len 36864 data: 6 7 7 9 7 5 6 7 8
30011 148/300 Flush
30012 149/300 Read block 2945 len 36864
30013 150/300 Write block 2006 len 12288 data: 3 3 5
30014 151/300 Write block 3599 len 32768 data: 3 3 3 3 7 7 8 7
30015 152/300 Write block 1047 len 12288 data: 4 5 6
30016 153/300 Write block 393 len 4096 data: 9
30017 154/300 Write block 443 len 4096 data: 3
30018 155/300 Write block 1370 len 40960 data: 6 6 4 3 3 5 6 6 6 5
30019 156/300 Read block 2529 len 20480
30020 157/300 Flush
30021 158/300 Read block 1128 len 36864
30022 159/300 Read block 1477 len 12288
30023 160/300 Write block 2949 len 12288 data: 5 4 4
30024 161/300 Write block 676 len 40960 data: 2 3 4 4 5 5 6 8 10 11
30025 162/300 Read block 2914 len 40960
30026 163/300 Read block 1674 len 16384
30027 164/300 Read block 3999 len 24576
30028 165/300 Write block 946 len 8192 data: 7 5
30029 166/300 Read block 2619 len 4096
30030 167/300 Write block 4921 len 24576 data: 5 6 5 5 5 5
30031 168/300 Write block 1087 len 32768 data: 6 5 4 4 3 4 5 4
30032 169/300 Read block 2450 len 12288
30033 170/300 Read block 4504 len 32768
30034 171/300 Write block 597 len 8192 data: 5 4
30035 172/300 Read block 1554 len 12288
30036 173/300 Write block 2579 len 16384 data: 3 3 4 4
30037 174/300 Write block 621 len 8192 data: 6 6
30038 175/300 Read block 3010 len 40960
30039 176/300 Write block 2312 len 8192 data: 7 6
30040 177/300 Read block 3768 len 24576
30041 178/300 Read block 4969 len 4096
30042 179/300 Read block 2244 len 32768
30043 180/300 Read block 3332 len 24576
30044 181/300 Write block 486 len 24576 data: 3 2 3 2 2 2
30045 182/300 Flush
30046 183/300 Write block 2062 len 8192 data: 8 10
30047 184/300 Write block 3777 len 16384 data: 4 5 6 7
30048 185/300 Write block 997 len 32768 data: 5 5 5 4 4 5 4 5
30049 186/300 Read block 417 len 8192
30050 187/300 Write block 1968 len 12288 data: 5 6 4
30051 188/300 Flush
30052 189/300 Read block 103 len 28672
30053 190/300 Read block 2616 len 36864
30054 191/300 Write block 2551 len 12288 data: 7 8 8
30055 192/300 Read block 929 len 12288
30056 193/300 Read block 3982 len 32768
30057 194/300 Write block 1233 len 16384 data: 4 6 6 6
30058 195/300 Read block 2107 len 16384
30059 196/300 Write block 2606 len 4096 data: 5
30060 197/300 Write block 4095 len 4096 data: 2
30061 198/300 Read block 344 len 4096
30062 199/300 Read block 2366 len 32768
30063 200/300 Write block 4143 len 36864 data: 5 5 6 7 5 4 4 4 4
30064 201/300 Read block 340 len 20480
30065 202/300 Write block 4842 len 12288 data: 5 8 9
30066 203/300 Read block 2800 len 12288
30067 204/300 Flush
30068 205/300 Write block 1385 len 4096 data: 8
30069 206/300 Read block 3471 len 12288
30070 207/300 Write block 2062 len 36864 data: 9 11 11 10 10 12 12 12 10
30071 208/300 Write block 4298 len 16384 data: 7 7 7 8
30072 209/300 Read block 2069 len 16384
30073 210/300 Flush
30074 211/300 Write block 1526 len 32768 data: 7 5 5 5 6 6 6 4
30075 212/300 Flush
30076 213/300 Write block 1158 len 16384 data: 10 10 9 7
30077 214/300 Read block 1681 len 40960
30078 215/300 Write block 976 len 36864 data: 5 4 4 3 4 3 3 3 4
30079 216/300 Write block 11 len 32768 data: 4 6 5 4 4 4 4 4
30080 217/300 Flush
30081 218/300 Write block 3335 len 8192 data: 7 6
30082 219/300 Write block 1435 len 12288 data: 7 8 9
30083 220/300 Write block 1239 len 12288 data: 5 4 4
30084 221/300 Read block 2652 len 12288
30085 222/300 Write block 3115 len 8192 data: 6 5
30086 223/300 Read block 3644 len 4096
30087 224/300 Read block 1448 len 24576
30088 225/300 Write block 3530 len 4096 data: 6
30089 226/300 Write block 2081 len 12288 data: 3 5 5
30090 227/300 Write block 3427 len 28672 data: 5 4 4 5 5 5 4
30091 228/300 Read block 1828 len 24576
30092 229/300 Write block 1628 len 24576 data: 6 6 5 6 5 6
30093 230/300 Flush
30094 231/300 Read block 1659 len 8192
30095 232/300 Read block 907 len 32768
30096 233/300 Read block 366 len 4096
30097 234/300 Read block 3070 len 36864
30098 235/300 Write block 3518 len 20480 data: 7 6 5 5 3
30099 236/300 Read block 2006 len 16384
30100 237/300 Write block 2990 len 24576 data: 6 6 6 6 6 6
30101 238/300 Write block 861 len 8192 data: 7 9
30102 239/300 Read block 2721 len 8192
30103 240/300 Write block 2070 len 28672 data: 11 8 8 6 8 9 6
30104 241/300 Write block 691 len 28672 data: 9 9 10 9 7 6 6
30105 242/300 Read block 3047 len 32768
30106 243/300 Flush
30107 244/300 Flush
30108 245/300 Flush
30109 246/300 Read block 3172 len 12288
30110 247/300 Flush
30111 248/300 Read block 4839 len 20480
30112 249/300 Write block 2306 len 4096 data: 4
30113 250/300 Read block 222 len 20480
30114 251/300 Write block 154 len 4096 data: 7
30115 252/300 Read block 413 len 4096
30116 253/300 Write block 1550 len 24576 data: 3 3 3 4 5 5
30117 254/300 Read block 2592 len 12288
30118 255/300 Read block 1233 len 4096
30119 256/300 Write block 2988 len 28672 data: 6 6 7 7 7 7 7
30120 257/300 Read block 4820 len 28672
30121 258/300 Read block 4377 len 28672
30122 259/300 Write block 4370 len 40960 data: 7 9 7 9 6 6 7 6 5 4
30123 260/300 Read block 3719 len 40960
30124 261/300 Read block 2700 len 36864
30125 262/300 Flush
30126 263/300 Read block 195 len 4096
30127 264/300 Write block 2898 len 24576 data: 5 5 7 8 9 10
30128 265/300 Read block 2937 len 12288
30129 266/300 Read block 1870 len 32768
30130 267/300 Read block 3387 len 4096
30131 268/300 Read block 1960 len 8192
30132 269/300 Write block 3644 len 28672 data: 5 5 4 4 6 5 5
30133 270/300 Flush
30134 271/300 Write block 113 len 24576 data: 4 3 4 7 7 6
30135 272/300 Read block 2827 len 4096
30136 273/300 Write block 1861 len 32768 data: 8 6 6 7 4 3 4 3
30137 274/300 Read block 3035 len 20480
30138 275/300 Write block 2622 len 24576 data: 6 5 5 5 4 7
30139 276/300 Read block 1637 len 36864
30140 277/300 Read block 4773 len 8192
30141 278/300 Flush
30142 279/300 Write block 1752 len 12288 data: 5 6 6
30143 280/300 Read block 4045 len 12288
30144 281/300 Read block 4818 len 16384
30145 282/300 Write block 938 len 4096 data: 4
30146 283/300 Write block 3769 len 16384 data: 6 4 4 3
30147 284/300 Write block 4479 len 8192 data: 5 5
30148 285/300 Flush
30149 286/300 Write block 576 len 36864 data: 5 4 4 4 5 5 3 3 3
30150 287/300 Read block 2422 len 4096
30151 288/300 Read block 4932 len 8192
30152 289/300 Read block 4822 len 12288
30153 290/300 Read block 4594 len 36864
30154 291/300 Write block 3264 len 8192 data: 3 2
30155 292/300 Read block 4467 len 32768
30156 293/300 Flush
30157 294/300 Write block 476 len 24576 data: 3 3 2 2 3 3
30158 295/300 Write block 3486 len 20480 data: 4 5 5 5 4
30159 296/300 Flush
30160 297/300 Read block 1161 len 8192
30161 298/300 Read block 927 len 12288
30162 299/300 Read block 2169 len 12288
30163 300/300 Write block 3433 len 28672 data: 5 4 4 5 6 4 4
30164 301/300 Read block 2698 len 20480
30165 Main task has work to do, go find it
30166 got dsc Start(0)
30167 start 0
30168 Jan 17 21:17:26.904 INFO request completed, latency_us: 216, response_code: 204, uri: /start/cid/0, method: POST, req_id: a269dbf1-89ff-43c7-9ff8-7d7ecd99f37e, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
30169 Replay: started 0, returned:Ok(())
30170 ----------------------------------------------------------------
30171 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
30172 States: Offline Active Active
30173 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30174 8547 Acked Flush 0 Sent Done Done false
30175 8548 Acked Write 5 Sent Done Done false
30176 8549 Acked Read 6 Sent Done Done false
30177 8550 Acked Flush 0 Sent Done Done false
30178 8551 Acked Write 5 Sent Done Done false
30179 8552 Acked Write 4 Sent Done Done false
30180 8553 Acked Write 10 Sent Done Done false
30181 8554 Acked Read 2 Sent Done Done false
30182 8555 Acked Write 4 Sent Done Done false
30183 8556 Acked Read 7 Sent Done Done false
30184 8557 Acked Write 6 Sent Done Done false
30185 8558 Acked Read 5 Sent Done Done false
30186 8559 Acked Write 2 Sent Done Done false
30187 8560 Acked Read 6 Sent Done Done false
30188 8561 Acked Read 10 Sent Done Done false
30189 8562 Acked Read 6 Sent Done Done false
30190 8563 Acked Write 10 Sent Done Done false
30191 8564 Acked Flush 0 Sent Done Done false
30192 8565 Acked Flush 0 Sent Done Done false
30193 8566 Acked Read 3 Sent Done Done false
30194 8567 Acked Write 9 Sent Done Done false
30195 8568 Acked Read 1 Sent Done Done false
30196 8569 Acked Read 6 Sent Done Done false
30197 8570 Acked Read 8 Sent Done Done false
30198 8571 Acked Write 1 Sent Done Done false
30199 8572 Acked Flush 0 Sent Done Done false
30200 8573 Acked Read 9 Sent Done Done false
30201 8574 Acked Flush 0 Sent Done Done false
30202 8575 Acked Write 5 Sent Done Done false
30203 8576 Acked Flush 0 Sent Done Done false
30204 8577 Acked Flush 0 Sent Done Done false
30205 8578 Acked Read 3 Sent Done Done false
30206 8579 Acked Read 4 Sent Done Done false
30207 8580 Acked Write 8 Sent Done Done false
30208 8581 Acked Flush 0 Sent Done Done false
30209 8582 Acked Read 2 Sent Done Done false
30210 8583 Acked Write 2 Sent Done Done false
30211 8584 Acked Read 10 Sent Done Done false
30212 8585 Acked Read 6 Sent Done Done false
30213 8586 Acked Write 7 Sent Done Done false
30214 8587 Acked Write 3 Sent Done Done false
30215 8588 Acked Write 10 Sent Done Done false
30216 8589 Acked Flush 0 Sent Done Done false
30217 8590 Acked Read 3 Sent Done Done false
30218 8591 Acked Write 5 Sent Done Done false
30219 8592 Acked Read 3 Sent Done Done false
30220 8593 Acked Read 9 Sent Done Done false
30221 8594 Acked Write 10 Sent Done Done false
30222 8595 Acked Read 3 Sent Done Done false
30223 8596 Acked Read 5 Sent Done Done false
30224 8597 Acked Read 6 Sent Done Done false
30225 8598 Acked Read 2 Sent Done Done false
30226 8599 Acked Write 6 Sent Done Done false
30227 8600 Acked Write 7 Sent Done Done false
30228 8601 Acked Write 6 Sent Done Done false
30229 8602 Acked Write 7 Sent Done Done false
30230 8603 Acked Write 3 Sent Done Done false
30231 8604 Acked Write 10 Sent Done Done false
30232 8605 Acked Flush 0 Sent Done Done false
30233 8606 Acked Flush 0 Sent Done Done false
30234 8607 Acked Write 8 Sent Done Done false
30235 8608 Acked Read 5 Sent Done Done false
30236 8609 Acked Read 5 Sent Done Done false
30237 8610 Acked Read 5 Sent Done Done false
30238 8611 Acked Read 9 Sent Done Done false
30239 8612 Acked Flush 0 Sent Done Done false
30240 8613 Acked Read 10 Sent Done Done false
30241 8614 Acked Read 9 Sent Done Done false
30242 8615 Acked Write 8 Sent Done Done false
30243 8616 Acked Write 1 Sent Done Done false
30244 8617 Acked Flush 0 Sent Done Done false
30245 8618 Acked Write 1 Sent Done Done false
30246 8619 Acked Flush 0 Sent Done Done false
30247 8620 Acked Read 1 Sent Done Done false
30248 8621 Acked Flush 0 Sent Done Done false
30249 8622 Acked Write 7 Sent Done Done false
30250 8623 Acked Read 8 Sent Done Done false
30251 8624 Acked Write 4 Sent Done Done false
30252 8625 Acked Read 8 Sent Done Done false
30253 8626 Acked Read 5 Sent Done Done false
30254 8627 Acked Write 10 Sent Done Done false
30255 8628 Acked Write 8 Sent Done Done false
30256 8629 Acked Read 1 Sent Done Done false
30257 8630 Acked Write 3 Sent Done Done false
30258 8631 Acked Write 7 Sent Done Done false
30259 8632 Acked Write 10 Sent Done Done false
30260 8633 Acked Read 2 Sent Done Done false
30261 8634 Acked Read 8 Sent Done Done false
30262 8635 Acked Read 5 Sent Done Done false
30263 8636 Acked Read 6 Sent Done Done false
30264 8637 Acked Write 7 Sent Done Done false
30265 8638 Acked Read 4 Sent Done Done false
30266 8639 Acked Write 10 Sent Done Done false
30267 8640 Acked Read 1 Sent Done Done false
30268 8641 Acked Write 7 Sent Done Done false
30269 8642 Acked Write 10 Sent Done Done false
30270 8643 Acked Read 7 Sent Done Done false
30271 8644 Acked Read 9 Sent Done Done false
30272 8645 Acked Read 1 Sent Done Done false
30273 8646 Acked Read 3 Sent Done Done false
30274 8647 Acked Write 1 Sent Done Done false
30275 8648 Acked Write 9 Sent Done Done false
30276 8649 Acked Write 1 Sent Done Done false
30277 8650 Acked Write 5 Sent Done Done false
30278 8651 Acked Read 3 Sent Done Done false
30279 8652 Acked Read 10 Sent Done Done false
30280 8653 Acked Read 4 Sent Done Done false
30281 8654 Acked Write 4 Sent Done Done false
30282 8655 Acked Flush 0 Sent Done Done false
30283 8656 Acked Write 8 Sent Done Done false
30284 8657 Acked Write 10 Sent Done Done false
30285 8658 Acked Read 2 Sent Done Done false
30286 8659 Acked Read 8 Sent Done Done false
30287 8660 Acked Read 7 Sent Done Done false
30288 8661 Acked Read 4 Sent Done Done false
30289 8662 Acked Read 2 Sent Done Done false
30290 8663 Acked Write 2 Sent Done Done false
30291 8664 Acked Flush 0 Sent Done Done false
30292 8665 Acked Read 8 Sent Done Done false
30293 8666 Acked Read 7 Sent Done Done false
30294 8667 Acked Write 6 Sent Done Done false
30295 8668 Acked Read 2 Sent Done Done false
30296 8669 Acked Write 6 Sent Done Done false
30297 8670 Acked Flush 0 Sent Done Done false
30298 8671 Acked Read 6 Sent Done Done false
30299 8672 Acked Write 9 Sent Done Done false
30300 8673 Acked Read 10 Sent Done Done false
30301 8674 Acked Read 5 Sent Done Done false
30302 8675 Acked Write 10 Sent Done Done false
30303 8676 Acked Write 1 Sent Done Done false
30304 8677 Acked Read 6 Sent Done Done false
30305 8678 Acked Write 2 Sent Done Done false
30306 8679 Acked Write 1 Sent Done Done false
30307 8680 Acked Read 7 Sent Done Done false
30308 8681 Acked Read 10 Sent Done Done false
30309 8682 Acked Write 1 Sent Done Done false
30310 8683 Acked Read 2 Sent Done Done false
30311 8684 Acked Write 1 Sent Done Done false
30312 8685 Acked Read 9 Sent Done Done false
30313 8686 Acked Write 10 Sent Done Done false
30314 8687 Acked Read 1 Sent Done Done false
30315 8688 Acked Write 8 Sent Done Done false
30316 8689 Acked Read 8 Sent Done Done false
30317 8690 Acked Write 3 Sent Done Done false
30318 8691 Acked Read 10 Sent Done Done false
30319 8692 Acked Read 10 Sent Done Done false
30320 8693 Acked Write 9 Sent Done Done false
30321 8694 Acked Flush 0 Sent Done Done false
30322 8695 Acked Read 9 Sent Done Done false
30323 8696 Acked Write 3 Sent Done Done false
30324 8697 Acked Write 8 Sent Done Done false
30325 8698 Acked Write 3 Sent Done Done false
30326 8699 Acked Write 1 Sent Done Done false
30327 8700 Acked Write 1 Sent Done Done false
30328 8701 Acked Write 10 Sent Done Done false
30329 8702 Acked Read 5 Sent Done Done false
30330 8703 Acked Flush 0 Sent Done Done false
30331 8704 Acked Read 9 Sent Done Done false
30332 8705 Acked Read 3 Sent Done Done false
30333 8706 Acked Write 3 Sent Done Done false
30334 8707 Acked Write 10 Sent Done Done false
30335 8708 Acked Read 10 Sent Done Done false
30336 8709 Acked Read 4 Sent Done Done false
30337 8710 Acked Read 6 Sent Done Done false
30338 8711 Acked Write 2 Sent Done Done false
30339 8712 Acked Read 1 Sent Done Done false
30340 8713 Acked Write 6 Sent Done Done false
30341 8714 Acked Write 8 Sent Done Done false
30342 8715 Acked Read 3 Sent Done Done false
30343 8716 Acked Read 8 Sent Done Done false
30344 8717 Acked Write 2 Sent Done Done false
30345 8718 Acked Read 3 Sent Done Done false
30346 8719 Acked Write 4 Sent Done Done false
30347 8720 Acked Write 2 Sent Done Done false
30348 8721 Acked Read 10 Sent Done Done false
30349 8722 Acked Write 2 Sent Done Done false
30350 8723 Acked Read 6 Sent Done Done false
30351 8724 Acked Read 1 Sent Done Done false
30352 8725 Acked Read 8 Sent Done Done false
30353 8726 Acked Read 6 Sent Done Done false
30354 8727 Acked Write 6 Sent Done Done false
30355 8728 Acked Flush 0 Sent Done Done false
30356 8729 Acked Write 2 Sent Done Done false
30357 8730 Acked Write 4 Sent Done Done false
30358 8731 Acked Write 8 Sent Done Done false
30359 8732 Acked Read 2 Sent Done Done false
30360 8733 Acked Write 3 Sent Done Done false
30361 8734 Acked Flush 0 Sent Done Done false
30362 8735 Acked Read 7 Sent Done Done false
30363 8736 Acked Read 9 Sent Done Done false
30364 8737 Acked Write 3 Sent Done Done false
30365 8738 Acked Read 3 Sent Done Done false
30366 8739 Acked Read 8 Sent Done Done false
30367 8740 Acked Write 4 Sent Done Done false
30368 8741 Acked Read 4 Sent Done Done false
30369 8742 Acked Write 1 Sent Done Done false
30370 8743 Acked Write 1 Sent Done Done false
30371 8744 Acked Read 1 Sent Done Done false
30372 8745 Acked Read 8 Sent Done Done false
30373 8746 Acked Write 9 Sent Done Done false
30374 8747 Acked Read 5 Sent Done Done false
30375 8748 Acked Write 3 Sent Done Done false
30376 8749 Acked Read 3 Sent Done Done false
30377 8750 Acked Flush 0 Sent Done Done false
30378 8751 Acked Write 1 Sent Done Done false
30379 8752 Acked Read 3 Sent Done Done false
30380 8753 Acked Write 9 Sent Done Done false
30381 8754 Acked Write 4 Sent Done Done false
30382 8755 Acked Read 4 Sent Done Done false
30383 8756 Acked Flush 0 Sent Done Done false
30384 8757 Acked Write 8 Sent Done Done false
30385 8758 Acked Flush 0 Sent Done Done false
30386 8759 Acked Write 4 Sent Done Done false
30387 8760 Acked Read 10 Sent Done Done false
30388 8761 Acked Write 9 Sent Done Done false
30389 8762 Acked Write 8 Sent Done Done false
30390 8763 Acked Flush 0 Sent Done Done false
30391 8764 Acked Write 2 Sent Done Done false
30392 8765 Acked Write 3 Sent Done Done false
30393 8766 Acked Write 3 Sent Done Done false
30394 8767 Acked Read 3 Sent Done Done false
30395 8768 Acked Write 2 Sent Done Done false
30396 8769 Acked Read 1 Sent Done Done false
30397 8770 Acked Read 6 Sent Done Done false
30398 8771 Acked Write 1 Sent Done Done false
30399 8772 Acked Write 3 Sent Done Done false
30400 8773 Acked Write 7 Sent Done Done false
30401 8774 Acked Read 6 Sent Done Done false
30402 8775 Acked Write 6 Sent Done Done false
30403 8776 Acked Flush 0 Sent Done Done false
30404 8777 Acked Read 2 Sent Done Done false
30405 8778 Acked Read 8 Sent Done Done false
30406 8779 Acked Read 1 Sent Done Done false
30407 8780 Acked Read 9 Sent Done Done false
30408 8781 Acked Write 5 Sent Done Done false
30409 8782 Acked Read 4 Sent Done Done false
30410 8783 Acked Write 6 Sent Done Done false
30411 8784 Acked Write 2 Sent Done Done false
30412 8785 Acked Read 2 Sent Done Done false
30413 8786 Acked Write 7 Sent Done Done false
30414 8787 Acked Write 7 Sent Done Done false
30415 8788 Acked Read 8 Sent Done Done false
30416 8789 Acked Flush 0 Sent Done Done false
30417 8790 Acked Flush 0 Sent Done Done false
30418 8791 Acked Flush 0 Sent Done Done false
30419 8792 Acked Read 3 Sent Done Done false
30420 8793 Acked Flush 0 Sent Done Done false
30421 8794 Acked Read 5 Sent Done Done false
30422 8795 Acked Write 1 Sent Done Done false
30423 8796 Acked Read 5 Sent Done Done false
30424 8797 Acked Write 1 Sent Done Done false
30425 8798 Acked Read 1 Sent Done Done false
30426 8799 Acked Write 6 Sent Done Done false
30427 8800 Acked Read 3 Sent Done Done false
30428 8801 Acked Read 1 Sent Done Done false
30429 8802 Acked Write 7 Sent Done Done false
30430 8803 Acked Read 7 Sent Done Done false
30431 8804 Acked Read 7 Sent Done Done false
30432 8805 Acked Write 10 Sent Done Done false
30433 8806 Acked Read 10 Sent Done Done false
30434 8807 Acked Read 9 Sent Done Done false
30435 8808 Acked Flush 0 Sent Done Done false
30436 8809 Acked Read 1 Sent Done Done false
30437 8810 Acked Write 6 Sent Done Done false
30438 8811 Acked Read 3 Sent Done Done false
30439 8812 Acked Read 8 Sent Done Done false
30440 8813 Acked Read 1 Sent Done Done false
30441 8814 Acked Read 2 Sent Done Done false
30442 8815 Acked Write 7 Sent Done Done false
30443 8816 Acked Flush 0 Sent Done Done false
30444 8817 Acked Write 6 Sent Done Done false
30445 8818 Acked Read 1 Sent Done Done false
30446 8819 Acked Write 8 Sent Done Done false
30447 8820 Acked Read 5 Sent Done Done false
30448 8821 Acked Write 6 Sent Done Done false
30449 8822 Acked Read 9 Sent Done Done false
30450 8823 Acked Read 2 Sent Done Done false
30451 8824 Acked Flush 0 Sent Done Done false
30452 8825 Acked Write 3 Sent Done Done false
30453 8826 Acked Read 3 Sent Done Done false
30454 8827 Acked Read 4 Sent Done Done false
30455 8828 Acked Write 1 Sent Done Done false
30456 8829 Acked Write 4 Sent Done Done false
30457 8830 Acked Write 2 Sent Done Done false
30458 8831 Acked Flush 0 Sent Done Done false
30459 8832 Acked Write 9 Sent Done Done false
30460 8833 Acked Read 1 Sent Done Done false
30461 8834 Acked Read 2 Sent Done Done false
30462 8835 Acked Read 3 Sent Done Done false
30463 8836 Acked Read 9 Sent Done Done false
30464 8837 Acked Write 2 Sent Done Done false
30465 8838 Acked Read 8 Sent Done Done false
30466 8839 Acked Flush 0 Sent Done Done false
30467 8840 Acked Write 6 Sent Done Done false
30468 8841 Acked Write 5 Sent Done Done false
30469 8842 Acked Flush 0 Sent Done Done false
30470 8843 Acked Read 2 Sent Done Done false
30471 8844 Acked Read 3 Sent Done Done false
30472 8845 Acked Read 3 Sent Done Done false
30473 8846 Acked Write 7 Sent Done Done false
30474 8847 Acked Read 5 Sent Done Done false
30475 STATES DS:0 DS:1 DS:2 TOTAL
30476 Sent 301 0 0 301
30477 Done 0 301 301 602
30478 Skipped 0 0 0 0
30479 Error 0 0 0 0
30480 Last Flush: 8546 8842 8842
30481 Downstairs last five completed: 8546 8545 8544 8543 8542
30482 Upstairs last five completed: 8847 8846 8845 8844 8843
30483 CLIENT: Up:0 ds:301 act:2
30484 [8810] got start action
30485 [8810] I am going to restart
30486 Starting downstairs at port 8810
30487 Make output file at "/tmp/dsc/downstairs-8810.txt"
30488 [8810][0] reports Starting
30489 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1226)
30490 [8810][0] reports Running
30491 ----------------------------------------------------------------
30492 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
30493 States: Offline Active Active
30494 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30495 8547 Acked Flush 0 Sent Done Done false
30496 8548 Acked Write 5 Sent Done Done false
30497 8549 Acked Read 6 Sent Done Done false
30498 8550 Acked Flush 0 Sent Done Done false
30499 8551 Acked Write 5 Sent Done Done false
30500 8552 Acked Write 4 Sent Done Done false
30501 8553 Acked Write 10 Sent Done Done false
30502 8554 Acked Read 2 Sent Done Done false
30503 8555 Acked Write 4 Sent Done Done false
30504 8556 Acked Read 7 Sent Done Done false
30505 8557 Acked Write 6 Sent Done Done false
30506 8558 Acked Read 5 Sent Done Done false
30507 8559 Acked Write 2 Sent Done Done false
30508 8560 Acked Read 6 Sent Done Done false
30509 8561 Acked Read 10 Sent Done Done false
30510 8562 Acked Read 6 Sent Done Done false
30511 8563 Acked Write 10 Sent Done Done false
30512 8564 Acked Flush 0 Sent Done Done false
30513 8565 Acked Flush 0 Sent Done Done false
30514 8566 Acked Read 3 Sent Done Done false
30515 8567 Acked Write 9 Sent Done Done false
30516 8568 Acked Read 1 Sent Done Done false
30517 8569 Acked Read 6 Sent Done Done false
30518 8570 Acked Read 8 Sent Done Done false
30519 8571 Acked Write 1 Sent Done Done false
30520 8572 Acked Flush 0 Sent Done Done false
30521 8573 Acked Read 9 Sent Done Done false
30522 8574 Acked Flush 0 Sent Done Done false
30523 8575 Acked Write 5 Sent Done Done false
30524 8576 Acked Flush 0 Sent Done Done false
30525 8577 Acked Flush 0 Sent Done Done false
30526 8578 Acked Read 3 Sent Done Done false
30527 8579 Acked Read 4 Sent Done Done false
30528 8580 Acked Write 8 Sent Done Done false
30529 8581 Acked Flush 0 Sent Done Done false
30530 8582 Acked Read 2 Sent Done Done false
30531 8583 Acked Write 2 Sent Done Done false
30532 8584 Acked Read 10 Sent Done Done false
30533 8585 Acked Read 6 Sent Done Done false
30534 8586 Acked Write 7 Sent Done Done false
30535 8587 Acked Write 3 Sent Done Done false
30536 8588 Acked Write 10 Sent Done Done false
30537 8589 Acked Flush 0 Sent Done Done false
30538 8590 Acked Read 3 Sent Done Done false
30539 8591 Acked Write 5 Sent Done Done false
30540 8592 Acked Read 3 Sent Done Done false
30541 8593 Acked Read 9 Sent Done Done false
30542 8594 Acked Write 10 Sent Done Done false
30543 8595 Acked Read 3 Sent Done Done false
30544 8596 Acked Read 5 Sent Done Done false
30545 8597 Acked Read 6 Sent Done Done false
30546 8598 Acked Read 2 Sent Done Done false
30547 8599 Acked Write 6 Sent Done Done false
30548 8600 Acked Write 7 Sent Done Done false
30549 8601 Acked Write 6 Sent Done Done false
30550 8602 Acked Write 7 Sent Done Done false
30551 8603 Acked Write 3 Sent Done Done false
30552 8604 Acked Write 10 Sent Done Done false
30553 8605 Acked Flush 0 Sent Done Done false
30554 8606 Acked Flush 0 Sent Done Done false
30555 8607 Acked Write 8 Sent Done Done false
30556 8608 Acked Read 5 Sent Done Done false
30557 8609 Acked Read 5 Sent Done Done false
30558 8610 Acked Read 5 Sent Done Done false
30559 8611 Acked Read 9 Sent Done Done false
30560 8612 Acked Flush 0 Sent Done Done false
30561 8613 Acked Read 10 Sent Done Done false
30562 8614 Acked Read 9 Sent Done Done false
30563 8615 Acked Write 8 Sent Done Done false
30564 8616 Acked Write 1 Sent Done Done false
30565 8617 Acked Flush 0 Sent Done Done false
30566 8618 Acked Write 1 Sent Done Done false
30567 8619 Acked Flush 0 Sent Done Done false
30568 8620 Acked Read 1 Sent Done Done false
30569 8621 Acked Flush 0 Sent Done Done false
30570 8622 Acked Write 7 Sent Done Done false
30571 8623 Acked Read 8 Sent Done Done false
30572 8624 Acked Write 4 Sent Done Done false
30573 8625 Acked Read 8 Sent Done Done false
30574 8626 Acked Read 5 Sent Done Done false
30575 8627 Acked Write 10 Sent Done Done false
30576 8628 Acked Write 8 Sent Done Done false
30577 8629 Acked Read 1 Sent Done Done false
30578 8630 Acked Write 3 Sent Done Done false
30579 8631 Acked Write 7 Sent Done Done false
30580 8632 Acked Write 10 Sent Done Done false
30581 8633 Acked Read 2 Sent Done Done false
30582 8634 Acked Read 8 Sent Done Done false
30583 8635 Acked Read 5 Sent Done Done false
30584 8636 Acked Read 6 Sent Done Done false
30585 8637 Acked Write 7 Sent Done Done false
30586 8638 Acked Read 4 Sent Done Done false
30587 8639 Acked Write 10 Sent Done Done false
30588 8640 Acked Read 1 Sent Done Done false
30589 8641 Acked Write 7 Sent Done Done false
30590 8642 Acked Write 10 Sent Done Done false
30591 8643 Acked Read 7 Sent Done Done false
30592 8644 Acked Read 9 Sent Done Done false
30593 8645 Acked Read 1 Sent Done Done false
30594 8646 Acked Read 3 Sent Done Done false
30595 8647 Acked Write 1 Sent Done Done false
30596 8648 Acked Write 9 Sent Done Done false
30597 8649 Acked Write 1 Sent Done Done false
30598 8650 Acked Write 5 Sent Done Done false
30599 8651 Acked Read 3 Sent Done Done false
30600 8652 Acked Read 10 Sent Done Done false
30601 8653 Acked Read 4 Sent Done Done false
30602 8654 Acked Write 4 Sent Done Done false
30603 8655 Acked Flush 0 Sent Done Done false
30604 8656 Acked Write 8 Sent Done Done false
30605 8657 Acked Write 10 Sent Done Done false
30606 8658 Acked Read 2 Sent Done Done false
30607 8659 Acked Read 8 Sent Done Done false
30608 8660 Acked Read 7 Sent Done Done false
30609 8661 Acked Read 4 Sent Done Done false
30610 8662 Acked Read 2 Sent Done Done false
30611 8663 Acked Write 2 Sent Done Done false
30612 8664 Acked Flush 0 Sent Done Done false
30613 8665 Acked Read 8 Sent Done Done false
30614 8666 Acked Read 7 Sent Done Done false
30615 8667 Acked Write 6 Sent Done Done false
30616 8668 Acked Read 2 Sent Done Done false
30617 8669 Acked Write 6 Sent Done Done false
30618 8670 Acked Flush 0 Sent Done Done false
30619 8671 Acked Read 6 Sent Done Done false
30620 8672 Acked Write 9 Sent Done Done false
30621 8673 Acked Read 10 Sent Done Done false
30622 8674 Acked Read 5 Sent Done Done false
30623 8675 Acked Write 10 Sent Done Done false
30624 8676 Acked Write 1 Sent Done Done false
30625 8677 Acked Read 6 Sent Done Done false
30626 8678 Acked Write 2 Sent Done Done false
30627 8679 Acked Write 1 Sent Done Done false
30628 8680 Acked Read 7 Sent Done Done false
30629 8681 Acked Read 10 Sent Done Done false
30630 8682 Acked Write 1 Sent Done Done false
30631 8683 Acked Read 2 Sent Done Done false
30632 8684 Acked Write 1 Sent Done Done false
30633 8685 Acked Read 9 Sent Done Done false
30634 8686 Acked Write 10 Sent Done Done false
30635 8687 Acked Read 1 Sent Done Done false
30636 8688 Acked Write 8 Sent Done Done false
30637 8689 Acked Read 8 Sent Done Done false
30638 8690 Acked Write 3 Sent Done Done false
30639 8691 Acked Read 10 Sent Done Done false
30640 8692 Acked Read 10 Sent Done Done false
30641 8693 Acked Write 9 Sent Done Done false
30642 8694 Acked Flush 0 Sent Done Done false
30643 8695 Acked Read 9 Sent Done Done false
30644 8696 Acked Write 3 Sent Done Done false
30645 8697 Acked Write 8 Sent Done Done false
30646 8698 Acked Write 3 Sent Done Done false
30647 8699 Acked Write 1 Sent Done Done false
30648 8700 Acked Write 1 Sent Done Done false
30649 8701 Acked Write 10 Sent Done Done false
30650 8702 Acked Read 5 Sent Done Done false
30651 8703 Acked Flush 0 Sent Done Done false
30652 8704 Acked Read 9 Sent Done Done false
30653 8705 Acked Read 3 Sent Done Done false
30654 8706 Acked Write 3 Sent Done Done false
30655 8707 Acked Write 10 Sent Done Done false
30656 8708 Acked Read 10 Sent Done Done false
30657 8709 Acked Read 4 Sent Done Done false
30658 8710 Acked Read 6 Sent Done Done false
30659 8711 Acked Write 2 Sent Done Done false
30660 8712 Acked Read 1 Sent Done Done false
30661 8713 Acked Write 6 Sent Done Done false
30662 8714 Acked Write 8 Sent Done Done false
30663 8715 Acked Read 3 Sent Done Done false
30664 8716 Acked Read 8 Sent Done Done false
30665 8717 Acked Write 2 Sent Done Done false
30666 8718 Acked Read 3 Sent Done Done false
30667 8719 Acked Write 4 Sent Done Done false
30668 8720 Acked Write 2 Sent Done Done false
30669 8721 Acked Read 10 Sent Done Done false
30670 8722 Acked Write 2 Sent Done Done false
30671 8723 Acked Read 6 Sent Done Done false
30672 8724 Acked Read 1 Sent Done Done false
30673 8725 Acked Read 8 Sent Done Done false
30674 8726 Acked Read 6 Sent Done Done false
30675 8727 Acked Write 6 Sent Done Done false
30676 8728 Acked Flush 0 Sent Done Done false
30677 8729 Acked Write 2 Sent Done Done false
30678 8730 Acked Write 4 Sent Done Done false
30679 8731 Acked Write 8 Sent Done Done false
30680 8732 Acked Read 2 Sent Done Done false
30681 8733 Acked Write 3 Sent Done Done false
30682 8734 Acked Flush 0 Sent Done Done false
30683 8735 Acked Read 7 Sent Done Done false
30684 8736 Acked Read 9 Sent Done Done false
30685 8737 Acked Write 3 Sent Done Done false
30686 8738 Acked Read 3 Sent Done Done false
30687 8739 Acked Read 8 Sent Done Done false
30688 8740 Acked Write 4 Sent Done Done false
30689 8741 Acked Read 4 Sent Done Done false
30690 8742 Acked Write 1 Sent Done Done false
30691 8743 Acked Write 1 Sent Done Done false
30692 8744 Acked Read 1 Sent Done Done false
30693 8745 Acked Read 8 Sent Done Done false
30694 8746 Acked Write 9 Sent Done Done false
30695 8747 Acked Read 5 Sent Done Done false
30696 8748 Acked Write 3 Sent Done Done false
30697 8749 Acked Read 3 Sent Done Done false
30698 8750 Acked Flush 0 Sent Done Done false
30699 8751 Acked Write 1 Sent Done Done false
30700 8752 Acked Read 3 Sent Done Done false
30701 8753 Acked Write 9 Sent Done Done false
30702 8754 Acked Write 4 Sent Done Done false
30703 8755 Acked Read 4 Sent Done Done false
30704 8756 Acked Flush 0 Sent Done Done false
30705 8757 Acked Write 8 Sent Done Done false
30706 8758 Acked Flush 0 Sent Done Done false
30707 8759 Acked Write 4 Sent Done Done false
30708 8760 Acked Read 10 Sent Done Done false
30709 8761 Acked Write 9 Sent Done Done false
30710 8762 Acked Write 8 Sent Done Done false
30711 8763 Acked Flush 0 Sent Done Done false
30712 8764 Acked Write 2 Sent Done Done false
30713 8765 Acked Write 3 Sent Done Done false
30714 8766 Acked Write 3 Sent Done Done false
30715 8767 Acked Read 3 Sent Done Done false
30716 8768 Acked Write 2 Sent Done Done false
30717 8769 Acked Read 1 Sent Done Done false
30718 8770 Acked Read 6 Sent Done Done false
30719 8771 Acked Write 1 Sent Done Done false
30720 8772 Acked Write 3 Sent Done Done false
30721 8773 Acked Write 7 Sent Done Done false
30722 8774 Acked Read 6 Sent Done Done false
30723 8775 Acked Write 6 Sent Done Done false
30724 8776 Acked Flush 0 Sent Done Done false
30725 8777 Acked Read 2 Sent Done Done false
30726 8778 Acked Read 8 Sent Done Done false
30727 8779 Acked Read 1 Sent Done Done false
30728 8780 Acked Read 9 Sent Done Done false
30729 8781 Acked Write 5 Sent Done Done false
30730 8782 Acked Read 4 Sent Done Done false
30731 8783 Acked Write 6 Sent Done Done false
30732 8784 Acked Write 2 Sent Done Done false
30733 8785 Acked Read 2 Sent Done Done false
30734 8786 Acked Write 7 Sent Done Done false
30735 8787 Acked Write 7 Sent Done Done false
30736 8788 Acked Read 8 Sent Done Done false
30737 8789 Acked Flush 0 Sent Done Done false
30738 8790 Acked Flush 0 Sent Done Done false
30739 8791 Acked Flush 0 Sent Done Done false
30740 8792 Acked Read 3 Sent Done Done false
30741 8793 Acked Flush 0 Sent Done Done false
30742 8794 Acked Read 5 Sent Done Done false
30743 8795 Acked Write 1 Sent Done Done false
30744 8796 Acked Read 5 Sent Done Done false
30745 8797 Acked Write 1 Sent Done Done false
30746 8798 Acked Read 1 Sent Done Done false
30747 8799 Acked Write 6 Sent Done Done false
30748 8800 Acked Read 3 Sent Done Done false
30749 8801 Acked Read 1 Sent Done Done false
30750 8802 Acked Write 7 Sent Done Done false
30751 8803 Acked Read 7 Sent Done Done false
30752 8804 Acked Read 7 Sent Done Done false
30753 8805 Acked Write 10 Sent Done Done false
30754 8806 Acked Read 10 Sent Done Done false
30755 8807 Acked Read 9 Sent Done Done false
30756 8808 Acked Flush 0 Sent Done Done false
30757 8809 Acked Read 1 Sent Done Done false
30758 8810 Acked Write 6 Sent Done Done false
30759 8811 Acked Read 3 Sent Done Done false
30760 8812 Acked Read 8 Sent Done Done false
30761 8813 Acked Read 1 Sent Done Done false
30762 8814 Acked Read 2 Sent Done Done false
30763 8815 Acked Write 7 Sent Done Done false
30764 8816 Acked Flush 0 Sent Done Done false
30765 8817 Acked Write 6 Sent Done Done false
30766 8818 Acked Read 1 Sent Done Done false
30767 8819 Acked Write 8 Sent Done Done false
30768 8820 Acked Read 5 Sent Done Done false
30769 8821 Acked Write 6 Sent Done Done false
30770 8822 Acked Read 9 Sent Done Done false
30771 8823 Acked Read 2 Sent Done Done false
30772 8824 Acked Flush 0 Sent Done Done false
30773 8825 Acked Write 3 Sent Done Done false
30774 8826 Acked Read 3 Sent Done Done false
30775 8827 Acked Read 4 Sent Done Done false
30776 8828 Acked Write 1 Sent Done Done false
30777 8829 Acked Write 4 Sent Done Done false
30778 8830 Acked Write 2 Sent Done Done false
30779 8831 Acked Flush 0 Sent Done Done false
30780 8832 Acked Write 9 Sent Done Done false
30781 8833 Acked Read 1 Sent Done Done false
30782 8834 Acked Read 2 Sent Done Done false
30783 8835 Acked Read 3 Sent Done Done false
30784 8836 Acked Read 9 Sent Done Done false
30785 8837 Acked Write 2 Sent Done Done false
30786 8838 Acked Read 8 Sent Done Done false
30787 8839 Acked Flush 0 Sent Done Done false
30788 8840 Acked Write 6 Sent Done Done false
30789 8841 Acked Write 5 Sent Done Done false
30790 8842 Acked Flush 0 Sent Done Done false
30791 8843 Acked Read 2 Sent Done Done false
30792 8844 Acked Read 3 Sent Done Done false
30793 8845 Acked Read 3 Sent Done Done false
30794 8846 Acked Write 7 Sent Done Done false
30795 8847 Acked Read 5 Sent Done Done false
30796 8848 Acked Flush 0 Sent Done Done false
30797 STATES DS:0 DS:1 DS:2 TOTAL
30798 Sent 302 0 0 302
30799 Done 0 302 302 604
30800 Skipped 0 0 0 0
30801 Error 0 0 0 0
30802 Last Flush: 8546 8848 8848
30803 Downstairs last five completed: 8546 8545 8544 8543 8542
30804 Upstairs last five completed: 8848 8847 8846 8845 8844
30805 CLIENT: Up:0 ds:302 act:2
308062025-01-17T21:17:32.673ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
308072025-01-17T21:17:32.673ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55018) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
308082025-01-17T21:17:32.675ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
308092025-01-17T21:17:32.675ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
308102025-01-17T21:17:32.675ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
308112025-01-17T21:17:32.675ZINFOcrucible: send last flush ID to this DS: 8546 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
308122025-01-17T21:17:32.675ZINFOcrucible: Replied this last flush ID: 8546 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
308132025-01-17T21:17:32.675ZINFOcrucible: [0] client re-new 302 jobs since flush 8546 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
308142025-01-17T21:17:32.676ZINFOcrucible: [0] Replayed 302 jobs since flush: 8546 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
30815 ----------------------------------------------------------------
30816 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
30817 Downstairs last five completed: 8848 8847 8846 8845 8844
30818 Upstairs last five completed: 8848 8847 8846 8845 8844
30819 CLIENT: Up:0 ds:0 act:3
30820 Replay: All jobs finished
30821 Main task has work to do, go find it
30822 got dsc Stop(0)
30823 stop 0
30824 [8810] Got stop action so:false kr:false
30825 Jan 17 21:17:34.909 INFO request completed, latency_us: 347, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 05f607ea-212c-4d85-ac40-1bde5e012042, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
30826 Jan 17 21:17:34.909{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:17:34.909936518Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
30827 INFO request completed, latency_us:{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:17:34.910010447Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
308282025-01-17T21:17:34.910ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
30829 111, response_code: 200{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-17T21:17:34.910089807Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
30830 , uri: /state/cid/0, method: GET, req_id: a9add909-ee7e-42be-96d2-193e295d23eb, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
30831 [8810] Exited with: ExitStatus(unix_wait_status(9))
30832 [8810][0] reports Exit
30833 Jan 17 21:17:38.911 INFO request completed, latency_us: 218, response_code: 200, uri: /state/cid/0, method: GET, req_id: 90219b69-29e0-40d2-b3bb-75dd37ca2196, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
30834 001/300 Read block 4064 len 4096
30835 002/300 Read block 4403 len 28672
30836 003/300 Write block 199 len 8192 data: 6 5
30837 004/300 Write block 4696 len 32768 data: 5 5 4 4 5 8 7 7
30838 005/300 Read block 4698 len 16384
30839 006/300 Read block 2976 len 28672
30840 007/300 Write block 3682 len 24576 data: 6 5 5 4 4 4
30841 008/300 Read block 1039 len 4096
30842 009/300 Flush
30843 010/300 Flush
30844 011/300 Write block 1164 len 16384 data: 3 2 3 3
30845 012/300 Read block 784 len 36864
30846 013/300 Read block 2643 len 4096
30847 014/300 Flush
30848 015/300 Read block 4666 len 24576
30849 016/300 Write block 637 len 32768 data: 4 3 3 4 3 5 5 5
30850 017/300 Flush
30851 018/300 Write block 3564 len 8192 data: 2 3
30852 019/300 Flush
30853 020/300 Read block 2266 len 8192
30854 021/300 Read block 2605 len 24576
30855 022/300 Write block 1186 len 36864 data: 5 4 4 4 5 5 5 7 7
30856 023/300 Write block 1706 len 8192 data: 6 6
30857 024/300 Write block 2348 len 20480 data: 9 11 11 12 10
30858 025/300 Flush
30859 026/300 Write block 3995 len 16384 data: 10 10 9 7
30860 027/300 Read block 1065 len 20480
30861 028/300 Write block 1047 len 20480 data: 5 6 7 6 7
30862 029/300 Flush
30863 030/300 Read block 3792 len 20480
30864 031/300 Read block 434 len 40960
30865 032/300 Read block 1801 len 20480
30866 033/300 Write block 4841 len 20480 data: 6 6 9 10 9
30867 034/300 Write block 3004 len 20480 data: 7 9 9 7 8
30868 035/300 Read block 3231 len 32768
30869 036/300 Write block 4522 len 36864 data: 4 4 5 4 5 5 4 4 5
30870 037/300 Write block 1915 len 20480 data: 6 5 7 7 7
30871 038/300 Write block 563 len 40960 data: 6 5 4 4 4 3 4 4 4 4
30872 039/300 Read block 2524 len 12288
30873 040/300 Write block 2535 len 28672 data: 5 4 6 5 7 7 7
30874 041/300 Write block 4205 len 20480 data: 4 6 6 7 6
30875 042/300 Write block 2414 len 8192 data: 6 5
30876 043/300 Read block 325 len 36864
30877 044/300 Read block 1544 len 4096
30878 045/300 Read block 3744 len 20480
30879 046/300 Write block 4935 len 12288 data: 6 3 3
30880 047/300 Write block 2979 len 16384 data: 4 5 5 5
30881 048/300 Read block 1108 len 24576
30882 049/300 Read block 2555 len 36864
30883 050/300 Read block 1828 len 32768
30884 051/300 Read block 4835 len 40960
30885 052/300 Write block 3890 len 16384 data: 6 6 7 8
30886 053/300 Write block 2730 len 28672 data: 8 7 4 3 2 3 3
30887 054/300 Write block 4404 len 28672 data: 5 4 4 4 7 6 5
30888 055/300 Read block 3333 len 28672
30889 056/300 Flush
30890 057/300 Write block 2920 len 24576 data: 6 5 5 4 4 5
30891 058/300 Flush
30892 059/300 Write block 195 len 36864 data: 10 10 10 8 7 6 6 5 4
30893 060/300 Read block 2058 len 36864
30894 061/300 Read block 2108 len 28672
30895 062/300 Write block 903 len 16384 data: 7 6 7 6
30896 063/300 Read block 4648 len 20480
30897 064/300 Flush
30898 065/300 Read block 2018 len 40960
30899 066/300 Write block 653 len 28672 data: 5 4 5 4 4 5 6
30900 067/300 Read block 2822 len 24576
30901 068/300 Read block 1639 len 24576
30902 069/300 Write block 4526 len 16384 data: 6 6 5 5
30903 070/300 Flush
30904 071/300 Write block 565 len 16384 data: 5 5 5 4
30905 072/300 Write block 836 len 36864 data: 5 6 5 5 5 5 7 6 3
30906 073/300 Write block 1116 len 8192 data: 4 5
30907 074/300 Write block 2936 len 36864 data: 3 2 3 4 4 4 5 8 10
30908 075/300 Write block 4917 len 12288 data: 3 2 2
30909 076/300 Flush
30910 077/300 Read block 3973 len 28672
30911 078/300 Write block 644 len 24576 data: 6 4 4 5 4 5
30912 079/300 Flush
30913 080/300 Write block 340 len 32768 data: 5 6 8 7 6 6 6 7
30914 081/300 Read block 737 len 28672
30915 082/300 Write block 1288 len 8192 data: 2 2
30916 083/300 Read block 2111 len 8192
30917 084/300 Flush
30918 085/300 Read block 3460 len 12288
30919 086/300 Read block 1674 len 36864
30920 087/300 Read block 2675 len 28672
30921 088/300 Read block 2164 len 20480
30922 089/300 Flush
30923 090/300 Read block 959 len 32768
30924 091/300 Read block 342 len 40960
30925 092/300 Write block 822 len 36864 data: 4 4 5 4 5 6 7 6 6
30926 093/300 Write block 3398 len 36864 data: 3 3 3 2 2 2 2 2 4
30927 094/300 Write block 4034 len 8192 data: 4 5
30928 095/300 Flush
30929 096/300 Write block 2814 len 8192 data: 5 5
30930 097/300 Flush
30931 098/300 Write block 482 len 4096 data: 4
30932 099/300 Read block 4614 len 36864
30933 100/300 Write block 2842 len 36864 data: 2 2 4 4 5 4 4 3 3
30934 101/300 Read block 130 len 4096
30935 102/300 Read block 3691 len 20480
30936 103/300 Read block 683 len 24576
30937 104/300 Write block 2997 len 4096 data: 4
30938 105/300 Write block 1415 len 24576 data: 3 4 4 4 4 4
30939 106/300 Read block 1340 len 12288
30940 107/300 Read block 3237 len 8192
30941 108/300 Write block 1227 len 36864 data: 5 6 6 8 8 6 5 7 7
30942 109/300 Read block 3793 len 12288
30943 110/300 Write block 76 len 20480 data: 10 8 7 4 5
30944 111/300 Read block 3780 len 40960
30945 112/300 Read block 4362 len 28672
30946 113/300 Flush
30947 114/300 Write block 1558 len 16384 data: 5 6 6 4
30948 115/300 Read block 3433 len 24576
30949 116/300 Read block 3604 len 36864
30950 117/300 Read block 827 len 16384
30951 118/300 Write block 1751 len 12288 data: 6 6 7
30952 119/300 Read block 2369 len 40960
30953 120/300 Read block 4365 len 16384
30954 121/300 Read block 3351 len 16384
30955 122/300 Flush
30956 123/300 Write block 921 len 28672 data: 7 9 8 7 8 8 7
30957 124/300 Write block 3292 len 16384 data: 5 6 7 7
30958 125/300 Read block 247 len 20480
30959 126/300 Read block 4856 len 40960
30960 127/300 Write block 1371 len 24576 data: 7 5 4 4 6 7
30961 128/300 Write block 4 len 40960 data: 5 4 4 3 4 3 3 5 7 6
30962 129/300 Read block 941 len 36864
30963 130/300 Write block 3050 len 12288 data: 4 3 2
30964 131/300 Flush
30965 132/300 Read block 2735 len 32768
30966 133/300 Write block 2931 len 40960 data: 6 4 4 4 4 4 3 4 5 5
30967 134/300 Flush
30968 135/300 Read block 1235 len 8192
30969 136/300 Flush
30970 137/300 Read block 2237 len 32768
30971 138/300 Write block 2437 len 16384 data: 4 4 3 4
30972 139/300 Write block 1849 len 16384 data: 4 4 3 5
30973 140/300 Read block 233 len 28672
30974 141/300 Read block 3731 len 16384
30975 142/300 Flush
30976 143/300 Write block 3392 len 40960 data: 5 5 5 5 4 4 4 4 4 3
30977 144/300 Read block 1603 len 40960
30978 145/300 Read block 1947 len 16384
30979 146/300 Read block 1270 len 24576
30980 147/300 Read block 2333 len 8192
30981 148/300 Write block 1935 len 8192 data: 2 2
30982 149/300 Read block 3014 len 32768
30983 150/300 Read block 2129 len 24576
30984 151/300 Flush
30985 152/300 Read block 4850 len 28672
30986 153/300 Read block 573 len 8192
30987 154/300 Read block 2535 len 20480
30988 155/300 Write block 565 len 40960 data: 6 6 6 5 5 5 5 5 5 4
30989 156/300 Write block 4002 len 36864 data: 5 8 10 9 11 11 11 11 9
30990 157/300 Read block 1455 len 36864
30991 158/300 Read block 1209 len 28672
30992 159/300 Read block 4129 len 8192
30993 160/300 Read block 4658 len 4096
30994 161/300 Read block 4881 len 28672
30995 162/300 Read block 468 len 36864
30996 163/300 Read block 2569 len 4096
30997 164/300 Read block 4031 len 4096
30998 165/300 Read block 1300 len 36864
30999 166/300 Write block 222 len 32768 data: 7 8 7 6 5 6 5 6
31000 167/300 Write block 562 len 40960 data: 7 7 6 7 7 7 6 6 6 6
31001 168/300 Flush
31002 169/300 Read block 816 len 24576
31003 170/300 Read block 2492 len 32768
31004 171/300 Write block 4593 len 36864 data: 5 5 5 5 4 4 5 5 4
31005 172/300 Read block 2118 len 28672
31006 173/300 Write block 4472 len 20480 data: 10 9 10 8 7
31007 174/300 Read block 4454 len 40960
31008 175/300 Write block 4991 len 4096 data: 5
31009 176/300 Flush
31010 177/300 Flush
31011 178/300 Write block 3561 len 16384 data: 3 3 3 3
31012 179/300 Write block 4045 len 12288 data: 5 5 5
31013 180/300 Read block 1149 len 8192
31014 181/300 Write block 45 len 20480 data: 4 4 4 4 4
31015 182/300 Write block 1490 len 16384 data: 4 4 3 2
31016 183/300 Write block 1250 len 4096 data: 4
31017 184/300 Write block 2548 len 24576 data: 9 7 8 8 9 9
31018 185/300 Read block 1 len 12288
31019 186/300 Write block 3800 len 32768 data: 5 5 5 5 5 5 5 6
31020 187/300 Read block 1849 len 36864
31021 188/300 Write block 3932 len 28672 data: 9 11 10 10 9 7 7
31022 189/300 Read block 2268 len 8192
31023 190/300 Read block 229 len 20480
31024 191/300 Read block 1461 len 12288
31025 192/300 Read block 715 len 16384
31026 193/300 Read block 2165 len 4096
31027 194/300 Read block 3863 len 12288
31028 195/300 Read block 1746 len 12288
31029 196/300 Flush
31030 197/300 Flush
31031 198/300 Write block 249 len 36864 data: 6 8 6 5 3 4 5 5 5
31032 199/300 Flush
31033 200/300 Read block 822 len 28672
31034 201/300 Write block 1909 len 28672 data: 4 4 4 6 6 6 7
31035 202/300 Write block 1309 len 24576 data: 7 6 6 7 8 6
31036 203/300 Read block 4859 len 4096
31037 204/300 Read block 4564 len 40960
31038 205/300 Read block 1236 len 4096
31039 206/300 Read block 3234 len 8192
31040 207/300 Write block 2320 len 12288 data: 8 6 8
31041 208/300 Read block 346 len 4096
31042 209/300 Write block 3409 len 4096 data: 6
31043 210/300 Read block 3299 len 24576
31044 211/300 Write block 3059 len 16384 data: 3 3 3 3
31045 212/300 Write block 3735 len 16384 data: 3 4 5 5
31046 213/300 Read block 297 len 12288
31047 214/300 Write block 3964 len 32768 data: 5 6 5 4 3 3 2 3
31048 215/300 Read block 4585 len 20480
31049 216/300 Write block 4757 len 16384 data: 8 7 10 10
31050 217/300 Read block 2849 len 32768
31051 218/300 Read block 2944 len 28672
31052 219/300 Write block 4858 len 16384 data: 5 6 8 7
31053 220/300 Write block 4024 len 32768 data: 5 6 5 5 7 7 4 5
31054 221/300 Read block 3771 len 8192
31055 222/300 Read block 943 len 8192
31056 223/300 Read block 1859 len 28672
31057 224/300 Read block 1896 len 24576
31058 225/300 Flush
31059 226/300 Write block 3997 len 28672 data: 10 8 6 6 4 6 9
31060 227/300 Read block 4604 len 4096
31061 228/300 Read block 1045 len 36864
31062 229/300 Read block 2019 len 28672
31063 230/300 Read block 4917 len 24576
31064 231/300 Read block 1191 len 12288
31065 232/300 Read block 4713 len 12288
31066 233/300 Flush
31067 234/300 Write block 233 len 24576 data: 4 3 4 3 4 4
31068 235/300 Write block 4101 len 12288 data: 4 5 5
31069 236/300 Read block 469 len 20480
31070 237/300 Write block 1627 len 16384 data: 4 7 7 6
31071 238/300 Write block 2529 len 4096 data: 3
31072 239/300 Write block 2751 len 32768 data: 5 5 6 5 5 4 5 4
31073 240/300 Read block 667 len 28672
31074 241/300 Write block 2753 len 8192 data: 7 6
31075 242/300 Flush
31076 243/300 Write block 3846 len 4096 data: 5
31077 244/300 Write block 2659 len 16384 data: 4 4 4 5
31078 245/300 Read block 4858 len 16384
31079 246/300 Flush
31080 247/300 Flush
31081 248/300 Read block 1329 len 40960
31082 249/300 Write block 1944 len 16384 data: 5 4 3 3
31083 250/300 Read block 138 len 12288
31084 251/300 Flush
31085 252/300 Write block 2248 len 28672 data: 3 4 3 3 3 3 3
31086 253/300 Write block 3255 len 36864 data: 7 7 7 8 5 2 2 3 3
31087 254/300 Write block 1964 len 36864 data: 7 7 6 6 6 7 5 4 3
31088 255/300 Read block 1561 len 16384
31089 256/300 Read block 3671 len 4096
31090 257/300 Write block 795 len 4096 data: 5
31091 258/300 Read block 1067 len 16384
31092 259/300 Write block 1520 len 12288 data: 8 8 8
31093 260/300 Flush
31094 261/300 Write block 2915 len 8192 data: 6 9
31095 262/300 Write block 1976 len 20480 data: 4 8 8 7 5
31096 263/300 Write block 2772 len 4096 data: 4
31097 264/300 Write block 4421 len 12288 data: 7 7 7
31098 265/300 Read block 3544 len 20480
31099 266/300 Write block 4776 len 8192 data: 5 5
31100 267/300 Read block 1781 len 36864
31101 268/300 Read block 462 len 12288
31102 269/300 Read block 1386 len 4096
31103 270/300 Read block 2176 len 40960
31104 271/300 Write block 2279 len 32768 data: 6 6 5 5 6 5 4 4
31105 272/300 Read block 4872 len 20480
31106 273/300 Write block 1599 len 24576 data: 6 6 5 4 4 6
31107 274/300 Read block 319 len 28672
31108 275/300 Write block 2661 len 4096 data: 5
31109 276/300 Read block 1738 len 32768
31110 277/300 Write block 108 len 28672 data: 6 7 6 5 4 5 4
31111 278/300 Write block 2881 len 32768 data: 5 5 4 5 6 7 8 8
31112 279/300 Read block 883 len 24576
31113 280/300 Read block 1728 len 20480
31114 281/300 Read block 1387 len 28672
31115 282/300 Write block 679 len 40960 data: 5 6 6 7 9 11 12 14 14 14
31116 283/300 Flush
31117 284/300 Read block 581 len 12288
31118 285/300 Flush
31119 286/300 Read block 1009 len 12288
31120 287/300 Write block 1137 len 40960 data: 5 5 4 3 3 3 3 3 3 3
31121 288/300 Read block 3215 len 20480
31122 289/300 Write block 3962 len 20480 data: 5 5 6 7 6
31123 290/300 Flush
31124 291/300 Read block 3175 len 16384
31125 292/300 Write block 1796 len 12288 data: 5 4 4
31126 293/300 Write block 2741 len 16384 data: 7 8 9 8
31127 294/300 Write block 3026 len 20480 data: 2 2 3 2 2
31128 295/300 Read block 4699 len 28672
31129 296/300 Read block 2574 len 20480
31130 297/300 Flush
31131 298/300 Write block 4271 len 12288 data: 5 5 5
31132 299/300 Write block 785 len 8192 data: 4 5
31133 300/300 Write block 1179 len 20480 data: 3 3 4 4 4
31134 301/300 Read block 809 len 4096
31135 Main task has work to do, go find it
31136 got dsc Start(0)
31137 start 0
31138 Jan 17 21:17:39.145 INFO request completed, latency_us: 210, response_code: 204, uri: /start/cid/0, method: POST, req_id: ef6cbc8d-c8cb-4115-9b80-db98844ea636, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
31139 Replay: started 0, returned:Ok(())
31140 ----------------------------------------------------------------
31141 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
31142 States: Offline Active Active
31143 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31144 8849 Acked Read 1 Sent Done Done false
31145 8850 Acked Read 7 Sent Done Done false
31146 8851 Acked Write 2 Sent Done Done false
31147 8852 Acked Write 8 Sent Done Done false
31148 8853 Acked Read 4 Sent Done Done false
31149 8854 Acked Read 7 Sent Done Done false
31150 8855 Acked Write 6 Sent Done Done false
31151 8856 Acked Read 1 Sent Done Done false
31152 8857 Acked Flush 0 Sent Done Done false
31153 8858 Acked Flush 0 Sent Done Done false
31154 8859 Acked Write 4 Sent Done Done false
31155 8860 Acked Read 9 Sent Done Done false
31156 8861 Acked Read 1 Sent Done Done false
31157 8862 Acked Flush 0 Sent Done Done false
31158 8863 Acked Read 6 Sent Done Done false
31159 8864 Acked Write 8 Sent Done Done false
31160 8865 Acked Flush 0 Sent Done Done false
31161 8866 Acked Write 2 Sent Done Done false
31162 8867 Acked Flush 0 Sent Done Done false
31163 8868 Acked Read 2 Sent Done Done false
31164 8869 Acked Read 6 Sent Done Done false
31165 8870 Acked Write 9 Sent Done Done false
31166 8871 Acked Write 2 Sent Done Done false
31167 8872 Acked Write 5 Sent Done Done false
31168 8873 Acked Flush 0 Sent Done Done false
31169 8874 Acked Write 4 Sent Done Done false
31170 8875 Acked Read 5 Sent Done Done false
31171 8876 Acked Write 5 Sent Done Done false
31172 8877 Acked Flush 0 Sent Done Done false
31173 8878 Acked Read 5 Sent Done Done false
31174 8879 Acked Read 10 Sent Done Done false
31175 8880 Acked Read 5 Sent Done Done false
31176 8881 Acked Write 5 Sent Done Done false
31177 8882 Acked Write 5 Sent Done Done false
31178 8883 Acked Read 8 Sent Done Done false
31179 8884 Acked Write 9 Sent Done Done false
31180 8885 Acked Write 5 Sent Done Done false
31181 8886 Acked Write 10 Sent Done Done false
31182 8887 Acked Read 3 Sent Done Done false
31183 8888 Acked Write 7 Sent Done Done false
31184 8889 Acked Write 5 Sent Done Done false
31185 8890 Acked Write 2 Sent Done Done false
31186 8891 Acked Read 9 Sent Done Done false
31187 8892 Acked Read 1 Sent Done Done false
31188 8893 Acked Read 5 Sent Done Done false
31189 8894 Acked Write 3 Sent Done Done false
31190 8895 Acked Write 4 Sent Done Done false
31191 8896 Acked Read 6 Sent Done Done false
31192 8897 Acked Read 9 Sent Done Done false
31193 8898 Acked Read 8 Sent Done Done false
31194 8899 Acked Read 10 Sent Done Done false
31195 8900 Acked Write 4 Sent Done Done false
31196 8901 Acked Write 7 Sent Done Done false
31197 8902 Acked Write 7 Sent Done Done false
31198 8903 Acked Read 7 Sent Done Done false
31199 8904 Acked Flush 0 Sent Done Done false
31200 8905 Acked Write 6 Sent Done Done false
31201 8906 Acked Flush 0 Sent Done Done false
31202 8907 Acked Write 9 Sent Done Done false
31203 8908 Acked Read 9 Sent Done Done false
31204 8909 Acked Read 7 Sent Done Done false
31205 8910 Acked Write 4 Sent Done Done false
31206 8911 Acked Read 5 Sent Done Done false
31207 8912 Acked Flush 0 Sent Done Done false
31208 8913 Acked Read 10 Sent Done Done false
31209 8914 Acked Write 7 Sent Done Done false
31210 8915 Acked Read 6 Sent Done Done false
31211 8916 Acked Read 6 Sent Done Done false
31212 8917 Acked Write 4 Sent Done Done false
31213 8918 Acked Flush 0 Sent Done Done false
31214 8919 Acked Write 4 Sent Done Done false
31215 8920 Acked Write 9 Sent Done Done false
31216 8921 Acked Write 2 Sent Done Done false
31217 8922 Acked Write 9 Sent Done Done false
31218 8923 Acked Write 3 Sent Done Done false
31219 8924 Acked Flush 0 Sent Done Done false
31220 8925 Acked Read 7 Sent Done Done false
31221 8926 Acked Write 6 Sent Done Done false
31222 8927 Acked Flush 0 Sent Done Done false
31223 8928 Acked Write 8 Sent Done Done false
31224 8929 Acked Read 7 Sent Done Done false
31225 8930 Acked Write 2 Sent Done Done false
31226 8931 Acked Read 2 Sent Done Done false
31227 8932 Acked Flush 0 Sent Done Done false
31228 8933 Acked Read 3 Sent Done Done false
31229 8934 Acked Read 9 Sent Done Done false
31230 8935 Acked Read 7 Sent Done Done false
31231 8936 Acked Read 5 Sent Done Done false
31232 8937 Acked Flush 0 Sent Done Done false
31233 8938 Acked Read 8 Sent Done Done false
31234 8939 Acked Read 10 Sent Done Done false
31235 8940 Acked Write 9 Sent Done Done false
31236 8941 Acked Write 9 Sent Done Done false
31237 8942 Acked Write 2 Sent Done Done false
31238 8943 Acked Flush 0 Sent Done Done false
31239 8944 Acked Write 2 Sent Done Done false
31240 8945 Acked Flush 0 Sent Done Done false
31241 8946 Acked Write 1 Sent Done Done false
31242 8947 Acked Read 9 Sent Done Done false
31243 8948 Acked Write 9 Sent Done Done false
31244 8949 Acked Read 1 Sent Done Done false
31245 8950 Acked Read 5 Sent Done Done false
31246 8951 Acked Read 6 Sent Done Done false
31247 8952 Acked Write 1 Sent Done Done false
31248 8953 Acked Write 6 Sent Done Done false
31249 8954 Acked Read 3 Sent Done Done false
31250 8955 Acked Read 2 Sent Done Done false
31251 8956 Acked Write 9 Sent Done Done false
31252 8957 Acked Read 3 Sent Done Done false
31253 8958 Acked Write 5 Sent Done Done false
31254 8959 Acked Read 10 Sent Done Done false
31255 8960 Acked Read 7 Sent Done Done false
31256 8961 Acked Flush 0 Sent Done Done false
31257 8962 Acked Write 4 Sent Done Done false
31258 8963 Acked Read 6 Sent Done Done false
31259 8964 Acked Read 9 Sent Done Done false
31260 8965 Acked Read 4 Sent Done Done false
31261 8966 Acked Write 3 Sent Done Done false
31262 8967 Acked Read 10 Sent Done Done false
31263 8968 Acked Read 4 Sent Done Done false
31264 8969 Acked Read 4 Sent Done Done false
31265 8970 Acked Flush 0 Sent Done Done false
31266 8971 Acked Write 7 Sent Done Done false
31267 8972 Acked Write 4 Sent Done Done false
31268 8973 Acked Read 5 Sent Done Done false
31269 8974 Acked Read 10 Sent Done Done false
31270 8975 Acked Write 6 Sent Done Done false
31271 8976 Acked Write 10 Sent Done Done false
31272 8977 Acked Read 9 Sent Done Done false
31273 8978 Acked Write 3 Sent Done Done false
31274 8979 Acked Flush 0 Sent Done Done false
31275 8980 Acked Read 8 Sent Done Done false
31276 8981 Acked Write 10 Sent Done Done false
31277 8982 Acked Flush 0 Sent Done Done false
31278 8983 Acked Read 2 Sent Done Done false
31279 8984 Acked Flush 0 Sent Done Done false
31280 8985 Acked Read 8 Sent Done Done false
31281 8986 Acked Write 4 Sent Done Done false
31282 8987 Acked Write 4 Sent Done Done false
31283 8988 Acked Read 7 Sent Done Done false
31284 8989 Acked Read 4 Sent Done Done false
31285 8990 Acked Flush 0 Sent Done Done false
31286 8991 Acked Write 10 Sent Done Done false
31287 8992 Acked Read 10 Sent Done Done false
31288 8993 Acked Read 4 Sent Done Done false
31289 8994 Acked Read 6 Sent Done Done false
31290 8995 Acked Read 2 Sent Done Done false
31291 8996 Acked Write 2 Sent Done Done false
31292 8997 Acked Read 8 Sent Done Done false
31293 8998 Acked Read 6 Sent Done Done false
31294 8999 Acked Flush 0 Sent Done Done false
31295 9000 Acked Read 7 Sent Done Done false
31296 9001 Acked Read 2 Sent Done Done false
31297 9002 Acked Read 5 Sent Done Done false
31298 9003 Acked Write 10 Sent Done Done false
31299 9004 Acked Write 9 Sent Done Done false
31300 9005 Acked Read 9 Sent Done Done false
31301 9006 Acked Read 7 Sent Done Done false
31302 9007 Acked Read 2 Sent Done Done false
31303 9008 Acked Read 1 Sent Done Done false
31304 9009 Acked Read 7 Sent Done Done false
31305 9010 Acked Read 9 Sent Done Done false
31306 9011 Acked Read 1 Sent Done Done false
31307 9012 Acked Read 1 Sent Done Done false
31308 9013 Acked Read 9 Sent Done Done false
31309 9014 Acked Write 8 Sent Done Done false
31310 9015 Acked Write 10 Sent Done Done false
31311 9016 Acked Flush 0 Sent Done Done false
31312 9017 Acked Read 6 Sent Done Done false
31313 9018 Acked Read 8 Sent Done Done false
31314 9019 Acked Write 9 Sent Done Done false
31315 9020 Acked Read 7 Sent Done Done false
31316 9021 Acked Write 5 Sent Done Done false
31317 9022 Acked Read 10 Sent Done Done false
31318 9023 Acked Write 1 Sent Done Done false
31319 9024 Acked Flush 0 Sent Done Done false
31320 9025 Acked Flush 0 Sent Done Done false
31321 9026 Acked Write 4 Sent Done Done false
31322 9027 Acked Write 3 Sent Done Done false
31323 9028 Acked Read 2 Sent Done Done false
31324 9029 Acked Write 5 Sent Done Done false
31325 9030 Acked Write 4 Sent Done Done false
31326 9031 Acked Write 1 Sent Done Done false
31327 9032 Acked Write 6 Sent Done Done false
31328 9033 Acked Read 3 Sent Done Done false
31329 9034 Acked Write 8 Sent Done Done false
31330 9035 Acked Read 9 Sent Done Done false
31331 9036 Acked Write 7 Sent Done Done false
31332 9037 Acked Read 2 Sent Done Done false
31333 9038 Acked Read 5 Sent Done Done false
31334 9039 Acked Read 3 Sent Done Done false
31335 9040 Acked Read 4 Sent Done Done false
31336 9041 Acked Read 1 Sent Done Done false
31337 9042 Acked Read 3 Sent Done Done false
31338 9043 Acked Read 3 Sent Done Done false
31339 9044 Acked Flush 0 Sent Done Done false
31340 9045 Acked Flush 0 Sent Done Done false
31341 9046 Acked Write 9 Sent Done Done false
31342 9047 Acked Flush 0 Sent Done Done false
31343 9048 Acked Read 7 Sent Done Done false
31344 9049 Acked Write 7 Sent Done Done false
31345 9050 Acked Write 6 Sent Done Done false
31346 9051 Acked Read 1 Sent Done Done false
31347 9052 Acked Read 10 Sent Done Done false
31348 9053 Acked Read 1 Sent Done Done false
31349 9054 Acked Read 2 Sent Done Done false
31350 9055 Acked Write 3 Sent Done Done false
31351 9056 Acked Read 1 Sent Done Done false
31352 9057 Acked Write 1 Sent Done Done false
31353 9058 Acked Read 6 Sent Done Done false
31354 9059 Acked Write 4 Sent Done Done false
31355 9060 Acked Write 4 Sent Done Done false
31356 9061 Acked Read 3 Sent Done Done false
31357 9062 Acked Write 8 Sent Done Done false
31358 9063 Acked Read 5 Sent Done Done false
31359 9064 Acked Write 4 Sent Done Done false
31360 9065 Acked Read 8 Sent Done Done false
31361 9066 Acked Read 7 Sent Done Done false
31362 9067 Acked Write 4 Sent Done Done false
31363 9068 Acked Write 8 Sent Done Done false
31364 9069 Acked Read 2 Sent Done Done false
31365 9070 Acked Read 2 Sent Done Done false
31366 9071 Acked Read 7 Sent Done Done false
31367 9072 Acked Read 6 Sent Done Done false
31368 9073 Acked Flush 0 Sent Done Done false
31369 9074 Acked Write 7 Sent Done Done false
31370 9075 Acked Read 1 Sent Done Done false
31371 9076 Acked Read 9 Sent Done Done false
31372 9077 Acked Read 7 Sent Done Done false
31373 9078 Acked Read 6 Sent Done Done false
31374 9079 Acked Read 3 Sent Done Done false
31375 9080 Acked Read 3 Sent Done Done false
31376 9081 Acked Flush 0 Sent Done Done false
31377 9082 Acked Write 6 Sent Done Done false
31378 9083 Acked Write 3 Sent Done Done false
31379 9084 Acked Read 5 Sent Done Done false
31380 9085 Acked Write 4 Sent Done Done false
31381 9086 Acked Write 1 Sent Done Done false
31382 9087 Acked Write 8 Sent Done Done false
31383 9088 Acked Read 7 Sent Done Done false
31384 9089 Acked Write 2 Sent Done Done false
31385 9090 Acked Flush 0 Sent Done Done false
31386 9091 Acked Write 1 Sent Done Done false
31387 9092 Acked Write 4 Sent Done Done false
31388 9093 Acked Read 4 Sent Done Done false
31389 9094 Acked Flush 0 Sent Done Done false
31390 9095 Acked Flush 0 Sent Done Done false
31391 9096 Acked Read 10 Sent Done Done false
31392 9097 Acked Write 4 Sent Done Done false
31393 9098 Acked Read 3 Sent Done Done false
31394 9099 Acked Flush 0 Sent Done Done false
31395 9100 Acked Write 7 Sent Done Done false
31396 9101 Acked Write 9 Sent Done Done false
31397 9102 Acked Write 9 Sent Done Done false
31398 9103 Acked Read 4 Sent Done Done false
31399 9104 Acked Read 1 Sent Done Done false
31400 9105 Acked Write 1 Sent Done Done false
31401 9106 Acked Read 4 Sent Done Done false
31402 9107 Acked Write 3 Sent Done Done false
31403 9108 Acked Flush 0 Sent Done Done false
31404 9109 Acked Write 2 Sent Done Done false
31405 9110 Acked Write 5 Sent Done Done false
31406 9111 Acked Write 1 Sent Done Done false
31407 9112 Acked Write 3 Sent Done Done false
31408 9113 Acked Read 5 Sent Done Done false
31409 9114 Acked Write 2 Sent Done Done false
31410 9115 Acked Read 9 Sent Done Done false
31411 9116 Acked Read 3 Sent Done Done false
31412 9117 Acked Read 1 Sent Done Done false
31413 9118 Acked Read 10 Sent Done Done false
31414 9119 Acked Write 8 Sent Done Done false
31415 9120 Acked Read 5 Sent Done Done false
31416 9121 Acked Write 6 Sent Done Done false
31417 9122 Acked Read 7 Sent Done Done false
31418 9123 Acked Write 1 Sent Done Done false
31419 9124 Acked Read 8 Sent Done Done false
31420 9125 Acked Write 7 Sent Done Done false
31421 9126 Acked Write 8 Sent Done Done false
31422 9127 Acked Read 6 Sent Done Done false
31423 9128 Acked Read 5 Sent Done Done false
31424 9129 Acked Read 7 Sent Done Done false
31425 9130 Acked Write 10 Sent Done Done false
31426 9131 Acked Flush 0 Sent Done Done false
31427 9132 Acked Read 3 Sent Done Done false
31428 9133 Acked Flush 0 Sent Done Done false
31429 9134 Acked Read 3 Sent Done Done false
31430 9135 Acked Write 10 Sent Done Done false
31431 9136 Acked Read 5 Sent Done Done false
31432 9137 Acked Write 5 Sent Done Done false
31433 9138 Acked Flush 0 Sent Done Done false
31434 9139 Acked Read 4 Sent Done Done false
31435 9140 Acked Write 3 Sent Done Done false
31436 9141 Acked Write 4 Sent Done Done false
31437 9142 Acked Write 5 Sent Done Done false
31438 9143 Acked Read 7 Sent Done Done false
31439 9144 Acked Read 5 Sent Done Done false
31440 9145 Acked Flush 0 Sent Done Done false
31441 9146 Acked Write 3 Sent Done Done false
31442 9147 Acked Write 2 Sent Done Done false
31443 9148 Acked Write 5 Sent Done Done false
31444 9149 Acked Read 1 Sent Done Done false
31445 STATES DS:0 DS:1 DS:2 TOTAL
31446 Sent 301 0 0 301
31447 Done 0 301 301 602
31448 Skipped 0 0 0 0
31449 Error 0 0 0 0
31450 Last Flush: 8848 9145 9145
31451 Downstairs last five completed: 8848 8847 8846 8845 8844
31452 Upstairs last five completed: 9149 9148 9147 9146 9145
31453 CLIENT: Up:0 ds:301 act:2
31454 [8810] got start action
31455 [8810] I am going to restart
31456 Starting downstairs at port 8810
31457 Make output file at "/tmp/dsc/downstairs-8810.txt"
31458 [8810][0] reports Starting
31459 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1227)
31460 [8810][0] reports Running
31461 ----------------------------------------------------------------
31462 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31463 States: Offline Active Active
31464 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31465 8849 Acked Read 1 Sent Done Done false
31466 8850 Acked Read 7 Sent Done Done false
31467 8851 Acked Write 2 Sent Done Done false
31468 8852 Acked Write 8 Sent Done Done false
31469 8853 Acked Read 4 Sent Done Done false
31470 8854 Acked Read 7 Sent Done Done false
31471 8855 Acked Write 6 Sent Done Done false
31472 8856 Acked Read 1 Sent Done Done false
31473 8857 Acked Flush 0 Sent Done Done false
31474 8858 Acked Flush 0 Sent Done Done false
31475 8859 Acked Write 4 Sent Done Done false
31476 8860 Acked Read 9 Sent Done Done false
31477 8861 Acked Read 1 Sent Done Done false
31478 8862 Acked Flush 0 Sent Done Done false
31479 8863 Acked Read 6 Sent Done Done false
31480 8864 Acked Write 8 Sent Done Done false
31481 8865 Acked Flush 0 Sent Done Done false
31482 8866 Acked Write 2 Sent Done Done false
31483 8867 Acked Flush 0 Sent Done Done false
31484 8868 Acked Read 2 Sent Done Done false
31485 8869 Acked Read 6 Sent Done Done false
31486 8870 Acked Write 9 Sent Done Done false
31487 8871 Acked Write 2 Sent Done Done false
31488 8872 Acked Write 5 Sent Done Done false
31489 8873 Acked Flush 0 Sent Done Done false
31490 8874 Acked Write 4 Sent Done Done false
31491 8875 Acked Read 5 Sent Done Done false
31492 8876 Acked Write 5 Sent Done Done false
31493 8877 Acked Flush 0 Sent Done Done false
31494 8878 Acked Read 5 Sent Done Done false
31495 8879 Acked Read 10 Sent Done Done false
31496 8880 Acked Read 5 Sent Done Done false
31497 8881 Acked Write 5 Sent Done Done false
31498 8882 Acked Write 5 Sent Done Done false
31499 8883 Acked Read 8 Sent Done Done false
31500 8884 Acked Write 9 Sent Done Done false
31501 8885 Acked Write 5 Sent Done Done false
31502 8886 Acked Write 10 Sent Done Done false
31503 8887 Acked Read 3 Sent Done Done false
31504 8888 Acked Write 7 Sent Done Done false
31505 8889 Acked Write 5 Sent Done Done false
31506 8890 Acked Write 2 Sent Done Done false
31507 8891 Acked Read 9 Sent Done Done false
31508 8892 Acked Read 1 Sent Done Done false
31509 8893 Acked Read 5 Sent Done Done false
31510 8894 Acked Write 3 Sent Done Done false
31511 8895 Acked Write 4 Sent Done Done false
31512 8896 Acked Read 6 Sent Done Done false
31513 8897 Acked Read 9 Sent Done Done false
31514 8898 Acked Read 8 Sent Done Done false
31515 8899 Acked Read 10 Sent Done Done false
31516 8900 Acked Write 4 Sent Done Done false
31517 8901 Acked Write 7 Sent Done Done false
31518 8902 Acked Write 7 Sent Done Done false
31519 8903 Acked Read 7 Sent Done Done false
31520 8904 Acked Flush 0 Sent Done Done false
31521 8905 Acked Write 6 Sent Done Done false
31522 8906 Acked Flush 0 Sent Done Done false
31523 8907 Acked Write 9 Sent Done Done false
31524 8908 Acked Read 9 Sent Done Done false
31525 8909 Acked Read 7 Sent Done Done false
31526 8910 Acked Write 4 Sent Done Done false
31527 8911 Acked Read 5 Sent Done Done false
31528 8912 Acked Flush 0 Sent Done Done false
31529 8913 Acked Read 10 Sent Done Done false
31530 8914 Acked Write 7 Sent Done Done false
31531 8915 Acked Read 6 Sent Done Done false
31532 8916 Acked Read 6 Sent Done Done false
31533 8917 Acked Write 4 Sent Done Done false
31534 8918 Acked Flush 0 Sent Done Done false
31535 8919 Acked Write 4 Sent Done Done false
31536 8920 Acked Write 9 Sent Done Done false
31537 8921 Acked Write 2 Sent Done Done false
31538 8922 Acked Write 9 Sent Done Done false
31539 8923 Acked Write 3 Sent Done Done false
31540 8924 Acked Flush 0 Sent Done Done false
31541 8925 Acked Read 7 Sent Done Done false
31542 8926 Acked Write 6 Sent Done Done false
31543 8927 Acked Flush 0 Sent Done Done false
31544 8928 Acked Write 8 Sent Done Done false
31545 8929 Acked Read 7 Sent Done Done false
31546 8930 Acked Write 2 Sent Done Done false
31547 8931 Acked Read 2 Sent Done Done false
31548 8932 Acked Flush 0 Sent Done Done false
31549 8933 Acked Read 3 Sent Done Done false
31550 8934 Acked Read 9 Sent Done Done false
31551 8935 Acked Read 7 Sent Done Done false
31552 8936 Acked Read 5 Sent Done Done false
31553 8937 Acked Flush 0 Sent Done Done false
31554 8938 Acked Read 8 Sent Done Done false
31555 8939 Acked Read 10 Sent Done Done false
31556 8940 Acked Write 9 Sent Done Done false
31557 8941 Acked Write 9 Sent Done Done false
31558 8942 Acked Write 2 Sent Done Done false
31559 8943 Acked Flush 0 Sent Done Done false
31560 8944 Acked Write 2 Sent Done Done false
31561 8945 Acked Flush 0 Sent Done Done false
31562 8946 Acked Write 1 Sent Done Done false
31563 8947 Acked Read 9 Sent Done Done false
31564 8948 Acked Write 9 Sent Done Done false
31565 8949 Acked Read 1 Sent Done Done false
31566 8950 Acked Read 5 Sent Done Done false
31567 8951 Acked Read 6 Sent Done Done false
31568 8952 Acked Write 1 Sent Done Done false
31569 8953 Acked Write 6 Sent Done Done false
31570 8954 Acked Read 3 Sent Done Done false
31571 8955 Acked Read 2 Sent Done Done false
31572 8956 Acked Write 9 Sent Done Done false
31573 8957 Acked Read 3 Sent Done Done false
31574 8958 Acked Write 5 Sent Done Done false
31575 8959 Acked Read 10 Sent Done Done false
31576 8960 Acked Read 7 Sent Done Done false
31577 8961 Acked Flush 0 Sent Done Done false
31578 8962 Acked Write 4 Sent Done Done false
31579 8963 Acked Read 6 Sent Done Done false
31580 8964 Acked Read 9 Sent Done Done false
31581 8965 Acked Read 4 Sent Done Done false
31582 8966 Acked Write 3 Sent Done Done false
31583 8967 Acked Read 10 Sent Done Done false
31584 8968 Acked Read 4 Sent Done Done false
31585 8969 Acked Read 4 Sent Done Done false
31586 8970 Acked Flush 0 Sent Done Done false
31587 8971 Acked Write 7 Sent Done Done false
31588 8972 Acked Write 4 Sent Done Done false
31589 8973 Acked Read 5 Sent Done Done false
31590 8974 Acked Read 10 Sent Done Done false
31591 8975 Acked Write 6 Sent Done Done false
31592 8976 Acked Write 10 Sent Done Done false
31593 8977 Acked Read 9 Sent Done Done false
31594 8978 Acked Write 3 Sent Done Done false
31595 8979 Acked Flush 0 Sent Done Done false
31596 8980 Acked Read 8 Sent Done Done false
31597 8981 Acked Write 10 Sent Done Done false
31598 8982 Acked Flush 0 Sent Done Done false
31599 8983 Acked Read 2 Sent Done Done false
31600 8984 Acked Flush 0 Sent Done Done false
31601 8985 Acked Read 8 Sent Done Done false
31602 8986 Acked Write 4 Sent Done Done false
31603 8987 Acked Write 4 Sent Done Done false
31604 8988 Acked Read 7 Sent Done Done false
31605 8989 Acked Read 4 Sent Done Done false
31606 8990 Acked Flush 0 Sent Done Done false
31607 8991 Acked Write 10 Sent Done Done false
31608 8992 Acked Read 10 Sent Done Done false
31609 8993 Acked Read 4 Sent Done Done false
31610 8994 Acked Read 6 Sent Done Done false
31611 8995 Acked Read 2 Sent Done Done false
31612 8996 Acked Write 2 Sent Done Done false
31613 8997 Acked Read 8 Sent Done Done false
31614 8998 Acked Read 6 Sent Done Done false
31615 8999 Acked Flush 0 Sent Done Done false
31616 9000 Acked Read 7 Sent Done Done false
31617 9001 Acked Read 2 Sent Done Done false
31618 9002 Acked Read 5 Sent Done Done false
31619 9003 Acked Write 10 Sent Done Done false
31620 9004 Acked Write 9 Sent Done Done false
31621 9005 Acked Read 9 Sent Done Done false
31622 9006 Acked Read 7 Sent Done Done false
31623 9007 Acked Read 2 Sent Done Done false
31624 9008 Acked Read 1 Sent Done Done false
31625 9009 Acked Read 7 Sent Done Done false
31626 9010 Acked Read 9 Sent Done Done false
31627 9011 Acked Read 1 Sent Done Done false
31628 9012 Acked Read 1 Sent Done Done false
31629 9013 Acked Read 9 Sent Done Done false
31630 9014 Acked Write 8 Sent Done Done false
31631 9015 Acked Write 10 Sent Done Done false
31632 9016 Acked Flush 0 Sent Done Done false
31633 9017 Acked Read 6 Sent Done Done false
31634 9018 Acked Read 8 Sent Done Done false
31635 9019 Acked Write 9 Sent Done Done false
31636 9020 Acked Read 7 Sent Done Done false
31637 9021 Acked Write 5 Sent Done Done false
31638 9022 Acked Read 10 Sent Done Done false
31639 9023 Acked Write 1 Sent Done Done false
31640 9024 Acked Flush 0 Sent Done Done false
31641 9025 Acked Flush 0 Sent Done Done false
31642 9026 Acked Write 4 Sent Done Done false
31643 9027 Acked Write 3 Sent Done Done false
31644 9028 Acked Read 2 Sent Done Done false
31645 9029 Acked Write 5 Sent Done Done false
31646 9030 Acked Write 4 Sent Done Done false
31647 9031 Acked Write 1 Sent Done Done false
31648 9032 Acked Write 6 Sent Done Done false
31649 9033 Acked Read 3 Sent Done Done false
31650 9034 Acked Write 8 Sent Done Done false
31651 9035 Acked Read 9 Sent Done Done false
31652 9036 Acked Write 7 Sent Done Done false
31653 9037 Acked Read 2 Sent Done Done false
31654 9038 Acked Read 5 Sent Done Done false
31655 9039 Acked Read 3 Sent Done Done false
31656 9040 Acked Read 4 Sent Done Done false
31657 9041 Acked Read 1 Sent Done Done false
31658 9042 Acked Read 3 Sent Done Done false
31659 9043 Acked Read 3 Sent Done Done false
31660 9044 Acked Flush 0 Sent Done Done false
31661 9045 Acked Flush 0 Sent Done Done false
31662 9046 Acked Write 9 Sent Done Done false
31663 9047 Acked Flush 0 Sent Done Done false
31664 9048 Acked Read 7 Sent Done Done false
31665 9049 Acked Write 7 Sent Done Done false
31666 9050 Acked Write 6 Sent Done Done false
31667 9051 Acked Read 1 Sent Done Done false
31668 9052 Acked Read 10 Sent Done Done false
31669 9053 Acked Read 1 Sent Done Done false
31670 9054 Acked Read 2 Sent Done Done false
31671 9055 Acked Write 3 Sent Done Done false
31672 9056 Acked Read 1 Sent Done Done false
31673 9057 Acked Write 1 Sent Done Done false
31674 9058 Acked Read 6 Sent Done Done false
31675 9059 Acked Write 4 Sent Done Done false
31676 9060 Acked Write 4 Sent Done Done false
31677 9061 Acked Read 3 Sent Done Done false
31678 9062 Acked Write 8 Sent Done Done false
31679 9063 Acked Read 5 Sent Done Done false
31680 9064 Acked Write 4 Sent Done Done false
31681 9065 Acked Read 8 Sent Done Done false
31682 9066 Acked Read 7 Sent Done Done false
31683 9067 Acked Write 4 Sent Done Done false
31684 9068 Acked Write 8 Sent Done Done false
31685 9069 Acked Read 2 Sent Done Done false
31686 9070 Acked Read 2 Sent Done Done false
31687 9071 Acked Read 7 Sent Done Done false
31688 9072 Acked Read 6 Sent Done Done false
31689 9073 Acked Flush 0 Sent Done Done false
31690 9074 Acked Write 7 Sent Done Done false
31691 9075 Acked Read 1 Sent Done Done false
31692 9076 Acked Read 9 Sent Done Done false
31693 9077 Acked Read 7 Sent Done Done false
31694 9078 Acked Read 6 Sent Done Done false
31695 9079 Acked Read 3 Sent Done Done false
31696 9080 Acked Read 3 Sent Done Done false
31697 9081 Acked Flush 0 Sent Done Done false
31698 9082 Acked Write 6 Sent Done Done false
31699 9083 Acked Write 3 Sent Done Done false
31700 9084 Acked Read 5 Sent Done Done false
31701 9085 Acked Write 4 Sent Done Done false
31702 9086 Acked Write 1 Sent Done Done false
31703 9087 Acked Write 8 Sent Done Done false
31704 9088 Acked Read 7 Sent Done Done false
31705 9089 Acked Write 2 Sent Done Done false
31706 9090 Acked Flush 0 Sent Done Done false
31707 9091 Acked Write 1 Sent Done Done false
31708 9092 Acked Write 4 Sent Done Done false
31709 9093 Acked Read 4 Sent Done Done false
31710 9094 Acked Flush 0 Sent Done Done false
31711 9095 Acked Flush 0 Sent Done Done false
31712 9096 Acked Read 10 Sent Done Done false
31713 9097 Acked Write 4 Sent Done Done false
31714 9098 Acked Read 3 Sent Done Done false
31715 9099 Acked Flush 0 Sent Done Done false
31716 9100 Acked Write 7 Sent Done Done false
31717 9101 Acked Write 9 Sent Done Done false
31718 9102 Acked Write 9 Sent Done Done false
31719 9103 Acked Read 4 Sent Done Done false
31720 9104 Acked Read 1 Sent Done Done false
31721 9105 Acked Write 1 Sent Done Done false
31722 9106 Acked Read 4 Sent Done Done false
31723 9107 Acked Write 3 Sent Done Done false
31724 9108 Acked Flush 0 Sent Done Done false
31725 9109 Acked Write 2 Sent Done Done false
31726 9110 Acked Write 5 Sent Done Done false
31727 9111 Acked Write 1 Sent Done Done false
31728 9112 Acked Write 3 Sent Done Done false
31729 9113 Acked Read 5 Sent Done Done false
31730 9114 Acked Write 2 Sent Done Done false
31731 9115 Acked Read 9 Sent Done Done false
31732 9116 Acked Read 3 Sent Done Done false
31733 9117 Acked Read 1 Sent Done Done false
31734 9118 Acked Read 10 Sent Done Done false
31735 9119 Acked Write 8 Sent Done Done false
31736 9120 Acked Read 5 Sent Done Done false
31737 9121 Acked Write 6 Sent Done Done false
31738 9122 Acked Read 7 Sent Done Done false
31739 9123 Acked Write 1 Sent Done Done false
31740 9124 Acked Read 8 Sent Done Done false
31741 9125 Acked Write 7 Sent Done Done false
31742 9126 Acked Write 8 Sent Done Done false
31743 9127 Acked Read 6 Sent Done Done false
31744 9128 Acked Read 5 Sent Done Done false
31745 9129 Acked Read 7 Sent Done Done false
31746 9130 Acked Write 10 Sent Done Done false
31747 9131 Acked Flush 0 Sent Done Done false
31748 9132 Acked Read 3 Sent Done Done false
31749 9133 Acked Flush 0 Sent Done Done false
31750 9134 Acked Read 3 Sent Done Done false
31751 9135 Acked Write 10 Sent Done Done false
31752 9136 Acked Read 5 Sent Done Done false
31753 9137 Acked Write 5 Sent Done Done false
31754 9138 Acked Flush 0 Sent Done Done false
31755 9139 Acked Read 4 Sent Done Done false
31756 9140 Acked Write 3 Sent Done Done false
31757 9141 Acked Write 4 Sent Done Done false
31758 9142 Acked Write 5 Sent Done Done false
31759 9143 Acked Read 7 Sent Done Done false
31760 9144 Acked Read 5 Sent Done Done false
31761 9145 Acked Flush 0 Sent Done Done false
31762 9146 Acked Write 3 Sent Done Done false
31763 9147 Acked Write 2 Sent Done Done false
31764 9148 Acked Write 5 Sent Done Done false
31765 9149 Acked Read 1 Sent Done Done false
31766 9150 Acked Flush 0 Sent Done Done false
31767 STATES DS:0 DS:1 DS:2 TOTAL
31768 Sent 302 0 0 302
31769 Done 0 302 302 604
31770 Skipped 0 0 0 0
31771 Error 0 0 0 0
31772 Last Flush: 8848 9150 9150
31773 Downstairs last five completed: 8848 8847 8846 8845 8844
31774 Upstairs last five completed: 9150 9149 9148 9147 9146
31775 CLIENT: Up:0 ds:302 act:2
317762025-01-17T21:17:44.911ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
317772025-01-17T21:17:44.911ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39672) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
317782025-01-17T21:17:44.912ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
317792025-01-17T21:17:44.913ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
317802025-01-17T21:17:44.913ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
317812025-01-17T21:17:44.913ZINFOcrucible: send last flush ID to this DS: 8848 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
317822025-01-17T21:17:44.913ZINFOcrucible: Replied this last flush ID: 8848 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
317832025-01-17T21:17:44.913ZINFOcrucible: [0] client re-new 302 jobs since flush 8848 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
317842025-01-17T21:17:44.913ZINFOcrucible: [0] Replayed 302 jobs since flush: 8848 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
31785 ----------------------------------------------------------------
31786 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
31787 Downstairs last five completed: 9150 9149 9148 9147 9146
31788 Upstairs last five completed: 9150 9149 9148 9147 9146
31789 CLIENT: Up:0 ds:0 act:3
31790 Replay: All jobs finished
31791 Main task has work to do, go find it
31792 got dsc Stop(1)
31793 stop 1
31794 Jan 17 21:17:47.151 INFO request completed, latency_us: 382, response_code: [8820] Got stop action so:false kr:false
31795 204, uri: /stop/cid/1, method: POST, req_id: f03f32eb-f8e5-404f-b979-337bf4e0e89a, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
31796 Jan 17 21:17:47.152 INFO request completed, latency_us: 142, response_code: 200, uri: /state/cid/1{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:17:47.152273836Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
31797 , method: GET, req_id: a4d810f6-1266-497b-933b-809f9f9d2631, remote_addr: 127.0.0.1{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:17:47.152332358Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
31798 :{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:17:47.152359134Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
31799 57191, local_addr: 127.0.0.1:9998
318002025-01-17T21:17:47.152ZINFOcrucible: client task is exiting = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
31801 [8820] Exited with: ExitStatus(unix_wait_status(9))
31802 [8820][1] reports Exit
31803 Jan 17 21:17:51.154 INFO request completed, latency_us: 217, response_code: 200, uri: /state/cid/1, method: GET, req_id: 8d3583c0-db09-4a43-b7a3-5b21adff445b, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
31804 001/300 Write block 3747 len 32768 data: 5 6 6 5 5 5 5 7
31805 002/300 Write block 2878 len 8192 data: 5 4
31806 003/300 Flush
31807 004/300 Flush
31808 005/300 Flush
31809 006/300 Flush
31810 007/300 Read block 2240 len 28672
31811 008/300 Flush
31812 009/300 Read block 1828 len 40960
31813 010/300 Read block 552 len 24576
31814 011/300 Read block 1956 len 24576
31815 012/300 Write block 1617 len 12288 data: 3 3 3
31816 013/300 Read block 1337 len 40960
31817 014/300 Write block 708 len 4096 data: 9
31818 015/300 Flush
31819 016/300 Read block 2467 len 4096
31820 017/300 Read block 4935 len 16384
31821 018/300 Read block 4910 len 8192
31822 019/300 Read block 4615 len 20480
31823 020/300 Read block 3969 len 40960
31824 021/300 Write block 4448 len 20480 data: 3 4 5 4 5
31825 022/300 Read block 3698 len 4096
31826 023/300 Read block 3773 len 16384
31827 024/300 Read block 2510 len 12288
31828 025/300 Write block 4832 len 20480 data: 6 6 3 4 4
31829 026/300 Write block 3698 len 36864 data: 6 6 7 7 8 8 9 7 7
31830 027/300 Read block 3059 len 8192
31831 028/300 Flush
31832 029/300 Write block 2802 len 40960 data: 5 3 4 5 5 6 7 6 4 4
31833 030/300 Read block 951 len 28672
31834 031/300 Write block 4840 len 4096 data: 4
31835 032/300 Read block 4707 len 40960
31836 033/300 Write block 1655 len 28672 data: 7 5 5 7 8 9 9
31837 034/300 Read block 4296 len 28672
31838 035/300 Read block 1567 len 40960
31839 036/300 Read block 3677 len 40960
31840 037/300 Read block 2294 len 32768
31841 038/300 Read block 2025 len 12288
31842 039/300 Read block 2028 len 12288
31843 040/300 Write block 1293 len 8192 data: 3 4
31844 041/300 Read block 113 len 16384
31845 042/300 Write block 1708 len 24576 data: 7 9 8 6 5 3
31846 043/300 Write block 2093 len 28672 data: 4 4 3 5 5 6 5
31847 044/300 Write block 1392 len 40960 data: 6 5 5 6 7 8 7 7 4 4
31848 045/300 Read block 2578 len 4096
31849 046/300 Write block 1012 len 8192 data: 6 5
31850 047/300 Write block 4522 len 20480 data: 5 5 6 5 7
31851 048/300 Write block 4153 len 16384 data: 6 4 6 8
31852 049/300 Write block 701 len 8192 data: 8 8
31853 050/300 Write block 2333 len 20480 data: 9 8 8 7 8
31854 051/300 Read block 892 len 12288
31855 052/300 Write block 4817 len 24576 data: 6 7 7 7 7 6
31856 053/300 Read block 3134 len 24576
31857 054/300 Write block 3364 len 16384 data: 5 3 3 3
31858 055/300 Flush
31859 056/300 Write block 4361 len 16384 data: 5 6 7 8
31860 057/300 Write block 28 len 16384 data: 9 8 7 8
31861 058/300 Write block 253 len 24576 data: 4 5 6 6 6 5
31862 059/300 Read block 204 len 36864
31863 060/300 Read block 4425 len 40960
31864 061/300 Read block 194 len 16384
31865 062/300 Flush
31866 063/300 Read block 3992 len 36864
31867 064/300 Write block 3607 len 20480 data: 7 9 11 10 11
31868 065/300 Read block 4483 len 20480
31869 066/300 Read block 575 len 8192
31870 067/300 Read block 1595 len 32768
31871 068/300 Write block 1284 len 12288 data: 3 3 3
31872 069/300 Read block 916 len 32768
31873 070/300 Write block 3448 len 16384 data: 11 9 10 9
31874 071/300 Read block 3223 len 4096
31875 072/300 Read block 611 len 24576
31876 073/300 Write block 138 len 20480 data: 5 5 5 3 4
31877 074/300 Flush
31878 075/300 Read block 4984 len 12288
31879 076/300 Read block 253 len 28672
31880 077/300 Read block 4407 len 40960
31881 078/300 Read block 4250 len 8192
31882 079/300 Write block 567 len 12288 data: 8 7 7
31883 080/300 Write block 2953 len 4096 data: 4
31884 081/300 Write block 938 len 32768 data: 5 3 4 3 5 6 6 7
31885 082/300 Write block 4782 len 24576 data: 6 7 8 7 6 5
31886 083/300 Read block 2526 len 36864
31887 084/300 Read block 1858 len 20480
31888 085/300 Flush
31889 086/300 Write block 3674 len 20480 data: 4 4 4 4 5
31890 087/300 Write block 2686 len 4096 data: 9
31891 088/300 Write block 336 len 16384 data: 5 7 7 6
31892 089/300 Write block 2948 len 32768 data: 6 6 5 5 4 5 3 3
31893 090/300 Write block 4808 len 4096 data: 7
31894 091/300 Read block 1708 len 24576
31895 092/300 Read block 4055 len 28672
31896 093/300 Write block 3835 len 16384 data: 6 8 7 7
31897 094/300 Write block 2023 len 40960 data: 7 7 6 4 3 3 3 3 4 4
31898 095/300 Read block 1747 len 24576
31899 096/300 Read block 2324 len 36864
31900 097/300 Write block 4483 len 8192 data: 7 7
31901 098/300 Write block 4663 len 16384 data: 5 5 5 6
31902 099/300 Read block 2159 len 28672
31903 100/300 Read block 1851 len 40960
31904 101/300 Write block 3038 len 20480 data: 5 4 4 3 4
31905 102/300 Write block 184 len 32768 data: 4 4 6 6 5 4 7 5
31906 103/300 Write block 1932 len 32768 data: 3 3 3 3 3 2 4 5
31907 104/300 Write block 4697 len 32768 data: 6 5 5 6 9 8 8 8
31908 105/300 Flush
31909 106/300 Read block 1834 len 20480
31910 107/300 Write block 2797 len 20480 data: 6 5 5 4 5
31911 108/300 Write block 2723 len 40960 data: 5 4 4 4 7 8 9 9 8 5
31912 109/300 Read block 3851 len 36864
31913 110/300 Write block 1063 len 16384 data: 8 9 7 4
31914 111/300 Read block 3276 len 8192
31915 112/300 Read block 869 len 32768
31916 113/300 Write block 4282 len 8192 data: 4 4
31917 114/300 Read block 2473 len 20480
31918 115/300 Write block 2754 len 8192 data: 7 6
31919 116/300 Read block 98 len 28672
31920 117/300 Read block 847 len 32768
31921 118/300 Read block 1768 len 12288
31922 119/300 Write block 2806 len 16384 data: 6 7 8 7
31923 120/300 Write block 1978 len 8192 data: 9 8
31924 121/300 Write block 4348 len 12288 data: 6 6 6
31925 122/300 Write block 2450 len 24576 data: 5 5 5 5 3 3
31926 123/300 Read block 675 len 4096
31927 124/300 Read block 1426 len 12288
31928 125/300 Write block 3001 len 24576 data: 6 7 8 8 10 10
31929 126/300 Write block 1596 len 28672 data: 7 6 5 7 7 6 5
31930 127/300 Write block 2129 len 20480 data: 3 4 3 4 4
31931 128/300 Write block 3121 len 16384 data: 7 9 8 7
31932 129/300 Write block 1647 len 4096 data: 5
31933 130/300 Write block 1237 len 4096 data: 6
31934 131/300 Write block 273 len 16384 data: 7 5 6 6
31935 132/300 Read block 1865 len 24576
31936 133/300 Read block 437 len 40960
31937 134/300 Write block 437 len 40960 data: 4 4 4 4 4 4 4 3 4 3
31938 135/300 Read block 2431 len 36864
31939 136/300 Read block 3799 len 20480
31940 137/300 Read block 2849 len 12288
31941 138/300 Flush
31942 139/300 Read block 4089 len 36864
31943 140/300 Write block 3102 len 36864 data: 8 9 11 11 9 9 8 6 6
31944 141/300 Read block 3812 len 40960
31945 142/300 Read block 253 len 36864
31946 143/300 Read block 2988 len 40960
31947 144/300 Write block 2600 len 36864 data: 5 4 3 4 7 6 6 4 2
31948 145/300 Flush
31949 146/300 Read block 3247 len 36864
31950 147/300 Read block 2346 len 32768
31951 148/300 Write block 294 len 36864 data: 7 6 4 4 5 6 4 5 5
31952 149/300 Write block 1390 len 12288 data: 4 4 7
31953 150/300 Write block 2809 len 8192 data: 8 5
31954 151/300 Read block 2474 len 16384
31955 152/300 Write block 3154 len 4096 data: 5
31956 153/300 Write block 671 len 12288 data: 4 5 4
31957 154/300 Write block 1415 len 20480 data: 4 5 5 5 5
31958 155/300 Write block 3295 len 24576 data: 8 8 9 9 9 7
31959 156/300 Read block 2917 len 28672
31960 157/300 Write block 3608 len 24576 data: 10 12 11 12 11 7
31961 158/300 Read block 1524 len 12288
31962 159/300 Read block 3102 len 24576
31963 160/300 Read block 1412 len 4096
31964 161/300 Read block 4162 len 28672
31965 162/300 Read block 1693 len 20480
31966 163/300 Write block 3989 len 36864 data: 6 7 8 9 10 9 11 11 11
31967 164/300 Flush
31968 165/300 Read block 3897 len 12288
31969 166/300 Write block 4228 len 32768 data: 3 4 4 4 4 5 4 6
31970 167/300 Read block 2981 len 12288
31971 168/300 Write block 4979 len 36864 data: 5 5 5 4 4 5 5 6 4
31972 169/300 Read block 4496 len 24576
31973 170/300 Flush
31974 171/300 Read block 4434 len 12288
31975 172/300 Write block 3303 len 28672 data: 8 6 4 3 3 4 4
31976 173/300 Write block 2401 len 28672 data: 5 5 4 5 6 7 8
31977 174/300 Write block 1330 len 4096 data: 7
31978 175/300 Write block 1919 len 40960 data: 8 8 8 9 8 9 6 4 4 4
31979 176/300 Read block 3508 len 32768
31980 177/300 Read block 4303 len 20480
31981 178/300 Write block 3890 len 20480 data: 7 7 8 9 7
31982 179/300 Read block 182 len 4096
31983 180/300 Write block 1977 len 36864 data: 9 10 9 6 7 6 6 6 7
31984 181/300 Write block 2226 len 12288 data: 4 5 5
31985 182/300 Read block 2060 len 36864
31986 183/300 Write block 971 len 36864 data: 7 7 6 4 5 6 5 5 4
31987 184/300 Write block 1593 len 8192 data: 7 8
31988 185/300 Read block 4646 len 4096
31989 186/300 Read block 4853 len 16384
31990 187/300 Read block 4296 len 20480
31991 188/300 Read block 623 len 40960
31992 189/300 Read block 149 len 4096
31993 190/300 Read block 4596 len 32768
31994 191/300 Write block 3724 len 24576 data: 8 10 8 8 6 4
31995 192/300 Read block 2739 len 20480
31996 193/300 Read block 3171 len 4096
31997 194/300 Read block 3517 len 20480
31998 195/300 Write block 3568 len 40960 data: 2 2 4 4 4 5 8 8 7 6
31999 196/300 Flush
32000 197/300 Read block 3207 len 20480
32001 198/300 Write block 2448 len 12288 data: 7 7 6
32002 199/300 Read block 4667 len 32768
32003 200/300 Read block 4115 len 36864
32004 201/300 Read block 1275 len 4096
32005 202/300 Read block 2520 len 8192
32006 203/300 Read block 240 len 12288
32007 204/300 Write block 3909 len 4096 data: 5
32008 205/300 Write block 3702 len 28672 data: 9 9 10 8 8 9 9
32009 206/300 Write block 4336 len 16384 data: 8 8 6 7
32010 207/300 Write block 2834 len 16384 data: 4 5 4 4
32011 208/300 Write block 1945 len 32768 data: 5 4 4 3 4 5 6 8
32012 209/300 Write block 1085 len 20480 data: 7 6 7 6 5
32013 210/300 Read block 379 len 16384
32014 211/300 Write block 3915 len 32768 data: 5 5 6 5 4 4 5 5
32015 212/300 Read block 2062 len 4096
32016 213/300 Read block 1698 len 36864
32017 214/300 Read block 3830 len 12288
32018 215/300 Write block 4945 len 40960 data: 3 3 2 2 2 2 3 6 5 3
32019 216/300 Write block 864 len 12288 data: 7 5 4
32020 217/300 Write block 4711 len 4096 data: 5
32021 218/300 Flush
32022 219/300 Read block 3538 len 28672
32023 220/300 Read block 478 len 8192
32024 221/300 Flush
32025 222/300 Write block 1830 len 40960 data: 4 4 5 3 4 6 5 6 6 6
32026 223/300 Write block 3792 len 40960 data: 6 5 5 5 5 4 5 5 6 6
32027 224/300 Read block 3425 len 24576
32028 225/300 Read block 2524 len 16384
32029 226/300 Read block 4526 len 28672
32030 227/300 Write block 727 len 16384 data: 4 5 6 5
32031 228/300 Write block 4122 len 16384 data: 6 6 4 3
32032 229/300 Write block 3331 len 16384 data: 6 6 7 7
32033 230/300 Write block 2248 len 16384 data: 4 5 4 4
32034 231/300 Write block 2380 len 36864 data: 6 5 5 5 5 5 6 5 6
32035 232/300 Write block 1594 len 28672 data: 9 7 8 7 6 8 8
32036 233/300 Flush
32037 234/300 Read block 51 len 24576
32038 235/300 Write block 898 len 28672 data: 6 5 7 8 7 8 7
32039 236/300 Write block 1091 len 32768 data: 4 5 6 5 6 7 7 9
32040 237/300 Write block 4023 len 16384 data: 4 6 7 6
32041 238/300 Write block 3262 len 24576 data: 4 4 4 3 2 4
32042 239/300 Write block 3999 len 36864 data: 7 7 5 7 10 11 10 12 12
32043 240/300 Read block 1967 len 20480
32044 241/300 Write block 1146 len 12288 data: 4 4 6
32045 242/300 Flush
32046 243/300 Write block 2878 len 8192 data: 6 5
32047 244/300 Read block 4232 len 28672
32048 245/300 Read block 4167 len 36864
32049 246/300 Read block 4927 len 28672
32050 247/300 Write block 3759 len 12288 data: 4 4 5
32051 248/300 Read block 4453 len 24576
32052 249/300 Flush
32053 250/300 Flush
32054 251/300 Write block 2642 len 16384 data: 5 7 8 9
32055 252/300 Write block 2479 len 16384 data: 7 7 6 5
32056 253/300 Write block 2571 len 20480 data: 3 4 4 4 4
32057 254/300 Read block 2099 len 12288
32058 255/300 Write block 829 len 40960 data: 7 7 6 6 6 4 4 6 7 6
32059 256/300 Write block 4388 len 16384 data: 4 3 3 4
32060 257/300 Read block 2979 len 24576
32061 258/300 Flush
32062 259/300 Read block 4085 len 24576
32063 260/300 Write block 4548 len 24576 data: 4 5 5 5 6 8
32064 261/300 Read block 1583 len 8192
32065 262/300 Read block 3511 len 24576
32066 263/300 Write block 4435 len 4096 data: 7
32067 264/300 Read block 2956 len 36864
32068 265/300 Write block 1342 len 28672 data: 5 5 4 4 4 5 6
32069 266/300 Flush
32070 267/300 Read block 4377 len 36864
32071 268/300 Read block 3213 len 24576
32072 269/300 Flush
32073 270/300 Write block 1095 len 20480 data: 7 8 8 10 10
32074 271/300 Read block 2835 len 8192
32075 272/300 Write block 1988 len 12288 data: 6 8 7
32076 273/300 Write block 1006 len 8192 data: 6 8
32077 274/300 Read block 1325 len 28672
32078 275/300 Write block 1777 len 24576 data: 4 3 2 2 2 2
32079 276/300 Write block 2369 len 16384 data: 4 5 6 7
32080 277/300 Write block 3922 len 12288 data: 6 5 7
32081 278/300 Read block 525 len 24576
32082 279/300 Write block 544 len 28672 data: 4 4 4 5 5 6 7
32083 280/300 Write block 2454 len 40960 data: 4 4 2 3 4 4 5 6 7 7
32084 281/300 Write block 4154 len 24576 data: 5 7 9 8 7 6
32085 282/300 Write block 4980 len 20480 data: 6 6 5 5 6
32086 283/300 Read block 2486 len 36864
32087 284/300 Read block 4568 len 8192
32088 285/300 Read block 2683 len 20480
32089 286/300 Write block 949 len 8192 data: 5 6
32090 287/300 Write block 4075 len 12288 data: 2 3 4
32091 288/300 Read block 2662 len 28672
32092 289/300 Write block 3635 len 4096 data: 6
32093 290/300 Read block 3001 len 32768
32094 291/300 Write block 4399 len 4096 data: 6
32095 292/300 Read block 4273 len 16384
32096 293/300 Read block 650 len 24576
32097 294/300 Read block 3179 len 4096
32098 295/300 Write block 4506 len 32768 data: 5 5 3 3 3 3 3 3
32099 296/300 Read block 4116 len 12288
32100 297/300 Write block 41 len 16384 data: 6 5 5 4
32101 298/300 Read block 3362 len 16384
32102 299/300 Write block 730 len 20480 data: 6 4 4 5 6
32103 300/300 Write block 4395 len 20480 data: 6 5 7 6 7
32104 301/300 Read block 1375 len 20480
32105 Main task has work to do, go find it
32106 got dsc Start(1)
32107 start 1
32108 Jan 17 21:17:51.382 INFO request completed, latency_us: 200, response_code: 204, uri: /start/cid/1, method: POST, req_id: 88d2cebf-e26e-4876-a900-4145c00da8af, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
32109 Replay: started 1, returned:Ok(())
32110 ----------------------------------------------------------------
32111 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
32112 States: Active Offline Active
32113 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32114 9151 Acked Write 8 Done Sent Done false
32115 9152 Acked Write 2 Done Sent Done false
32116 9153 Acked Flush 0 Done Sent Done false
32117 9154 Acked Flush 0 Done Sent Done false
32118 9155 Acked Flush 0 Done Sent Done false
32119 9156 Acked Flush 0 Done Sent Done false
32120 9157 Acked Read 7 Done Sent Done false
32121 9158 Acked Flush 0 Done Sent Done false
32122 9159 Acked Read 10 Done Sent Done false
32123 9160 Acked Read 6 Done Sent Done false
32124 9161 Acked Read 6 Done Sent Done false
32125 9162 Acked Write 3 Done Sent Done false
32126 9163 Acked Read 10 Done Sent Done false
32127 9164 Acked Write 1 Done Sent Done false
32128 9165 Acked Flush 0 Done Sent Done false
32129 9166 Acked Read 1 Done Sent Done false
32130 9167 Acked Read 4 Done Sent Done false
32131 9168 Acked Read 2 Done Sent Done false
32132 9169 Acked Read 5 Done Sent Done false
32133 9170 Acked Read 10 Done Sent Done false
32134 9171 Acked Write 5 Done Sent Done false
32135 9172 Acked Read 1 Done Sent Done false
32136 9173 Acked Read 4 Done Sent Done false
32137 9174 Acked Read 3 Done Sent Done false
32138 9175 Acked Write 5 Done Sent Done false
32139 9176 Acked Write 9 Done Sent Done false
32140 9177 Acked Read 2 Done Sent Done false
32141 9178 Acked Flush 0 Done Sent Done false
32142 9179 Acked Write 10 Done Sent Done false
32143 9180 Acked Read 7 Done Sent Done false
32144 9181 Acked Write 1 Done Sent Done false
32145 9182 Acked Read 10 Done Sent Done false
32146 9183 Acked Write 7 Done Sent Done false
32147 9184 Acked Read 7 Done Sent Done false
32148 9185 Acked Read 10 Done Sent Done false
32149 9186 Acked Read 10 Done Sent Done false
32150 9187 Acked Read 8 Done Sent Done false
32151 9188 Acked Read 3 Done Sent Done false
32152 9189 Acked Read 3 Done Sent Done false
32153 9190 Acked Write 2 Done Sent Done false
32154 9191 Acked Read 4 Done Sent Done false
32155 9192 Acked Write 6 Done Sent Done false
32156 9193 Acked Write 7 Done Sent Done false
32157 9194 Acked Write 10 Done Sent Done false
32158 9195 Acked Read 1 Done Sent Done false
32159 9196 Acked Write 2 Done Sent Done false
32160 9197 Acked Write 5 Done Sent Done false
32161 9198 Acked Write 4 Done Sent Done false
32162 9199 Acked Write 2 Done Sent Done false
32163 9200 Acked Write 5 Done Sent Done false
32164 9201 Acked Read 3 Done Sent Done false
32165 9202 Acked Write 6 Done Sent Done false
32166 9203 Acked Read 6 Done Sent Done false
32167 9204 Acked Write 4 Done Sent Done false
32168 9205 Acked Flush 0 Done Sent Done false
32169 9206 Acked Write 4 Done Sent Done false
32170 9207 Acked Write 4 Done Sent Done false
32171 9208 Acked Write 6 Done Sent Done false
32172 9209 Acked Read 9 Done Sent Done false
32173 9210 Acked Read 10 Done Sent Done false
32174 9211 Acked Read 4 Done Sent Done false
32175 9212 Acked Flush 0 Done Sent Done false
32176 9213 Acked Read 9 Done Sent Done false
32177 9214 Acked Write 5 Done Sent Done false
32178 9215 Acked Read 5 Done Sent Done false
32179 9216 Acked Read 2 Done Sent Done false
32180 9217 Acked Read 8 Done Sent Done false
32181 9218 Acked Write 3 Done Sent Done false
32182 9219 Acked Read 8 Done Sent Done false
32183 9220 Acked Write 4 Done Sent Done false
32184 9221 Acked Read 1 Done Sent Done false
32185 9222 Acked Read 6 Done Sent Done false
32186 9223 Acked Write 5 Done Sent Done false
32187 9224 Acked Flush 0 Done Sent Done false
32188 9225 Acked Read 3 Done Sent Done false
32189 9226 Acked Read 7 Done Sent Done false
32190 9227 Acked Read 10 Done Sent Done false
32191 9228 Acked Read 2 Done Sent Done false
32192 9229 Acked Write 3 Done Sent Done false
32193 9230 Acked Write 1 Done Sent Done false
32194 9231 Acked Write 8 Done Sent Done false
32195 9232 Acked Write 6 Done Sent Done false
32196 9233 Acked Read 9 Done Sent Done false
32197 9234 Acked Read 5 Done Sent Done false
32198 9235 Acked Flush 0 Done Sent Done false
32199 9236 Acked Write 5 Done Sent Done false
32200 9237 Acked Write 1 Done Sent Done false
32201 9238 Acked Write 4 Done Sent Done false
32202 9239 Acked Write 8 Done Sent Done false
32203 9240 Acked Write 1 Done Sent Done false
32204 9241 Acked Read 6 Done Sent Done false
32205 9242 Acked Read 7 Done Sent Done false
32206 9243 Acked Write 4 Done Sent Done false
32207 9244 Acked Write 10 Done Sent Done false
32208 9245 Acked Read 6 Done Sent Done false
32209 9246 Acked Read 9 Done Sent Done false
32210 9247 Acked Write 2 Done Sent Done false
32211 9248 Acked Write 4 Done Sent Done false
32212 9249 Acked Read 7 Done Sent Done false
32213 9250 Acked Read 10 Done Sent Done false
32214 9251 Acked Write 5 Done Sent Done false
32215 9252 Acked Write 8 Done Sent Done false
32216 9253 Acked Write 8 Done Sent Done false
32217 9254 Acked Write 8 Done Sent Done false
32218 9255 Acked Flush 0 Done Sent Done false
32219 9256 Acked Read 5 Done Sent Done false
32220 9257 Acked Write 5 Done Sent Done false
32221 9258 Acked Write 10 Done Sent Done false
32222 9259 Acked Read 9 Done Sent Done false
32223 9260 Acked Write 4 Done Sent Done false
32224 9261 Acked Read 2 Done Sent Done false
32225 9262 Acked Read 8 Done Sent Done false
32226 9263 Acked Write 2 Done Sent Done false
32227 9264 Acked Read 5 Done Sent Done false
32228 9265 Acked Write 2 Done Sent Done false
32229 9266 Acked Read 7 Done Sent Done false
32230 9267 Acked Read 8 Done Sent Done false
32231 9268 Acked Read 3 Done Sent Done false
32232 9269 Acked Write 4 Done Sent Done false
32233 9270 Acked Write 2 Done Sent Done false
32234 9271 Acked Write 3 Done Sent Done false
32235 9272 Acked Write 6 Done Sent Done false
32236 9273 Acked Read 1 Done Sent Done false
32237 9274 Acked Read 3 Done Sent Done false
32238 9275 Acked Write 6 Done Sent Done false
32239 9276 Acked Write 7 Done Sent Done false
32240 9277 Acked Write 5 Done Sent Done false
32241 9278 Acked Write 4 Done Sent Done false
32242 9279 Acked Write 1 Done Sent Done false
32243 9280 Acked Write 1 Done Sent Done false
32244 9281 Acked Write 4 Done Sent Done false
32245 9282 Acked Read 6 Done Sent Done false
32246 9283 Acked Read 10 Done Sent Done false
32247 9284 Acked Write 10 Done Sent Done false
32248 9285 Acked Read 9 Done Sent Done false
32249 9286 Acked Read 5 Done Sent Done false
32250 9287 Acked Read 3 Done Sent Done false
32251 9288 Acked Flush 0 Done Sent Done false
32252 9289 Acked Read 9 Done Sent Done false
32253 9290 Acked Write 9 Done Sent Done false
32254 9291 Acked Read 10 Done Sent Done false
32255 9292 Acked Read 9 Done Sent Done false
32256 9293 Acked Read 10 Done Sent Done false
32257 9294 Acked Write 9 Done Sent Done false
32258 9295 Acked Flush 0 Done Sent Done false
32259 9296 Acked Read 9 Done Sent Done false
32260 9297 Acked Read 8 Done Sent Done false
32261 9298 Acked Write 9 Done Sent Done false
32262 9299 Acked Write 3 Done Sent Done false
32263 9300 Acked Write 2 Done Sent Done false
32264 9301 Acked Read 4 Done Sent Done false
32265 9302 Acked Write 1 Done Sent Done false
32266 9303 Acked Write 3 Done Sent Done false
32267 9304 Acked Write 5 Done Sent Done false
32268 9305 Acked Write 6 Done Sent Done false
32269 9306 Acked Read 7 Done Sent Done false
32270 9307 Acked Write 6 Done Sent Done false
32271 9308 Acked Read 3 Done Sent Done false
32272 9309 Acked Read 6 Done Sent Done false
32273 9310 Acked Read 1 Done Sent Done false
32274 9311 Acked Read 7 Done Sent Done false
32275 9312 Acked Read 5 Done Sent Done false
32276 9313 Acked Write 9 Done Sent Done false
32277 9314 Acked Flush 0 Done Sent Done false
32278 9315 Acked Read 3 Done Sent Done false
32279 9316 Acked Write 8 Done Sent Done false
32280 9317 Acked Read 3 Done Sent Done false
32281 9318 Acked Write 9 Done Sent Done false
32282 9319 Acked Read 6 Done Sent Done false
32283 9320 Acked Flush 0 Done Sent Done false
32284 9321 Acked Read 3 Done Sent Done false
32285 9322 Acked Write 7 Done Sent Done false
32286 9323 Acked Write 7 Done Sent Done false
32287 9324 Acked Write 1 Done Sent Done false
32288 9325 Acked Write 10 Done Sent Done false
32289 9326 Acked Read 8 Done Sent Done false
32290 9327 Acked Read 5 Done Sent Done false
32291 9328 Acked Write 5 Done Sent Done false
32292 9329 Acked Read 1 Done Sent Done false
32293 9330 Acked Write 9 Done Sent Done false
32294 9331 Acked Write 3 Done Sent Done false
32295 9332 Acked Read 9 Done Sent Done false
32296 9333 Acked Write 9 Done Sent Done false
32297 9334 Acked Write 2 Done Sent Done false
32298 9335 Acked Read 1 Done Sent Done false
32299 9336 Acked Read 4 Done Sent Done false
32300 9337 Acked Read 5 Done Sent Done false
32301 9338 Acked Read 10 Done Sent Done false
32302 9339 Acked Read 1 Done Sent Done false
32303 9340 Acked Read 8 Done Sent Done false
32304 9341 Acked Write 6 Done Sent Done false
32305 9342 Acked Read 5 Done Sent Done false
32306 9343 Acked Read 1 Done Sent Done false
32307 9344 Acked Read 5 Done Sent Done false
32308 9345 Acked Write 10 Done Sent Done false
32309 9346 Acked Flush 0 Done Sent Done false
32310 9347 Acked Read 5 Done Sent Done false
32311 9348 Acked Write 3 Done Sent Done false
32312 9349 Acked Read 8 Done Sent Done false
32313 9350 Acked Read 9 Done Sent Done false
32314 9351 Acked Read 1 Done Sent Done false
32315 9352 Acked Read 2 Done Sent Done false
32316 9353 Acked Read 3 Done Sent Done false
32317 9354 Acked Write 1 Done Sent Done false
32318 9355 Acked Write 7 Done Sent Done false
32319 9356 Acked Write 4 Done Sent Done false
32320 9357 Acked Write 4 Done Sent Done false
32321 9358 Acked Write 8 Done Sent Done false
32322 9359 Acked Write 5 Done Sent Done false
32323 9360 Acked Read 4 Done Sent Done false
32324 9361 Acked Write 8 Done Sent Done false
32325 9362 Acked Read 1 Done Sent Done false
32326 9363 Acked Read 9 Done Sent Done false
32327 9364 Acked Read 3 Done Sent Done false
32328 9365 Acked Write 10 Done Sent Done false
32329 9366 Acked Write 3 Done Sent Done false
32330 9367 Acked Write 1 Done Sent Done false
32331 9368 Acked Flush 0 Done Sent Done false
32332 9369 Acked Read 7 Done Sent Done false
32333 9370 Acked Read 2 Done Sent Done false
32334 9371 Acked Flush 0 Done Sent Done false
32335 9372 Acked Write 10 Done Sent Done false
32336 9373 Acked Write 10 Done Sent Done false
32337 9374 Acked Read 6 Done Sent Done false
32338 9375 Acked Read 4 Done Sent Done false
32339 9376 Acked Read 7 Done Sent Done false
32340 9377 Acked Write 4 Done Sent Done false
32341 9378 Acked Write 4 Done Sent Done false
32342 9379 Acked Write 4 Done Sent Done false
32343 9380 Acked Write 4 Done Sent Done false
32344 9381 Acked Write 9 Done Sent Done false
32345 9382 Acked Write 7 Done Sent Done false
32346 9383 Acked Flush 0 Done Sent Done false
32347 9384 Acked Read 6 Done Sent Done false
32348 9385 Acked Write 7 Done Sent Done false
32349 9386 Acked Write 8 Done Sent Done false
32350 9387 Acked Write 4 Done Sent Done false
32351 9388 Acked Write 6 Done Sent Done false
32352 9389 Acked Write 9 Done Sent Done false
32353 9390 Acked Read 5 Done Sent Done false
32354 9391 Acked Write 3 Done Sent Done false
32355 9392 Acked Flush 0 Done Sent Done false
32356 9393 Acked Write 2 Done Sent Done false
32357 9394 Acked Read 7 Done Sent Done false
32358 9395 Acked Read 9 Done Sent Done false
32359 9396 Acked Read 7 Done Sent Done false
32360 9397 Acked Write 3 Done Sent Done false
32361 9398 Acked Read 6 Done Sent Done false
32362 9399 Acked Flush 0 Done Sent Done false
32363 9400 Acked Flush 0 Done Sent Done false
32364 9401 Acked Write 4 Done Sent Done false
32365 9402 Acked Write 4 Done Sent Done false
32366 9403 Acked Write 5 Done Sent Done false
32367 9404 Acked Read 3 Done Sent Done false
32368 9405 Acked Write 10 Done Sent Done false
32369 9406 Acked Write 4 Done Sent Done false
32370 9407 Acked Read 6 Done Sent Done false
32371 9408 Acked Flush 0 Done Sent Done false
32372 9409 Acked Read 6 Done Sent Done false
32373 9410 Acked Write 6 Done Sent Done false
32374 9411 Acked Read 2 Done Sent Done false
32375 9412 Acked Read 6 Done Sent Done false
32376 9413 Acked Write 1 Done Sent Done false
32377 9414 Acked Read 9 Done Sent Done false
32378 9415 Acked Write 7 Done Sent Done false
32379 9416 Acked Flush 0 Done Sent Done false
32380 9417 Acked Read 9 Done Sent Done false
32381 9418 Acked Read 6 Done Sent Done false
32382 9419 Acked Flush 0 Done Sent Done false
32383 9420 Acked Write 5 Done Sent Done false
32384 9421 Acked Read 2 Done Sent Done false
32385 9422 Acked Write 3 Done Sent Done false
32386 9423 Acked Write 2 Done Sent Done false
32387 9424 Acked Read 7 Done Sent Done false
32388 9425 Acked Write 6 Done Sent Done false
32389 9426 Acked Write 4 Done Sent Done false
32390 9427 Acked Write 3 Done Sent Done false
32391 9428 Acked Read 6 Done Sent Done false
32392 9429 Acked Write 7 Done Sent Done false
32393 9430 Acked Write 10 Done Sent Done false
32394 9431 Acked Write 6 Done Sent Done false
32395 9432 Acked Write 5 Done Sent Done false
32396 9433 Acked Read 9 Done Sent Done false
32397 9434 Acked Read 2 Done Sent Done false
32398 9435 Acked Read 5 Done Sent Done false
32399 9436 Acked Write 2 Done Sent Done false
32400 9437 Acked Write 3 Done Sent Done false
32401 9438 Acked Read 7 Done Sent Done false
32402 9439 Acked Write 1 Done Sent Done false
32403 9440 Acked Read 8 Done Sent Done false
32404 9441 Acked Write 1 Done Sent Done false
32405 9442 Acked Read 4 Done Sent Done false
32406 9443 Acked Read 6 Done Sent Done false
32407 9444 Acked Read 1 Done Sent Done false
32408 9445 Acked Write 8 Done Sent Done false
32409 9446 Acked Read 3 Done Sent Done false
32410 9447 Acked Write 4 Done Sent Done false
32411 9448 Acked Read 4 Done Sent Done false
32412 9449 Acked Write 5 Done Sent Done false
32413 9450 Acked Write 5 Done Sent Done false
32414 9451 Acked Read 5 Done Sent Done false
32415 STATES DS:0 DS:1 DS:2 TOTAL
32416 Sent 0 301 0 301
32417 Done 301 0 301 602
32418 Skipped 0 0 0 0
32419 Error 0 0 0 0
32420 Last Flush: 9419 9150 9419
32421 Downstairs last five completed: 9150 9149 9148 9147 9146
32422 Upstairs last five completed: 9451 9450 9449 9448 9447
32423 CLIENT: Up:0 ds:301 act:2
32424 [8820] got start action
32425 [8820] I am going to restart
32426 Starting downstairs at port 8820
32427 Make output file at "/tmp/dsc/downstairs-8820.txt"
32428 [8820][1] reports Starting
32429 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1228)
32430 [8820][1] reports Running
32431 ----------------------------------------------------------------
32432 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32433 States: Active Offline Active
32434 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32435 9151 Acked Write 8 Done Sent Done false
32436 9152 Acked Write 2 Done Sent Done false
32437 9153 Acked Flush 0 Done Sent Done false
32438 9154 Acked Flush 0 Done Sent Done false
32439 9155 Acked Flush 0 Done Sent Done false
32440 9156 Acked Flush 0 Done Sent Done false
32441 9157 Acked Read 7 Done Sent Done false
32442 9158 Acked Flush 0 Done Sent Done false
32443 9159 Acked Read 10 Done Sent Done false
32444 9160 Acked Read 6 Done Sent Done false
32445 9161 Acked Read 6 Done Sent Done false
32446 9162 Acked Write 3 Done Sent Done false
32447 9163 Acked Read 10 Done Sent Done false
32448 9164 Acked Write 1 Done Sent Done false
32449 9165 Acked Flush 0 Done Sent Done false
32450 9166 Acked Read 1 Done Sent Done false
32451 9167 Acked Read 4 Done Sent Done false
32452 9168 Acked Read 2 Done Sent Done false
32453 9169 Acked Read 5 Done Sent Done false
32454 9170 Acked Read 10 Done Sent Done false
32455 9171 Acked Write 5 Done Sent Done false
32456 9172 Acked Read 1 Done Sent Done false
32457 9173 Acked Read 4 Done Sent Done false
32458 9174 Acked Read 3 Done Sent Done false
32459 9175 Acked Write 5 Done Sent Done false
32460 9176 Acked Write 9 Done Sent Done false
32461 9177 Acked Read 2 Done Sent Done false
32462 9178 Acked Flush 0 Done Sent Done false
32463 9179 Acked Write 10 Done Sent Done false
32464 9180 Acked Read 7 Done Sent Done false
32465 9181 Acked Write 1 Done Sent Done false
32466 9182 Acked Read 10 Done Sent Done false
32467 9183 Acked Write 7 Done Sent Done false
32468 9184 Acked Read 7 Done Sent Done false
32469 9185 Acked Read 10 Done Sent Done false
32470 9186 Acked Read 10 Done Sent Done false
32471 9187 Acked Read 8 Done Sent Done false
32472 9188 Acked Read 3 Done Sent Done false
32473 9189 Acked Read 3 Done Sent Done false
32474 9190 Acked Write 2 Done Sent Done false
32475 9191 Acked Read 4 Done Sent Done false
32476 9192 Acked Write 6 Done Sent Done false
32477 9193 Acked Write 7 Done Sent Done false
32478 9194 Acked Write 10 Done Sent Done false
32479 9195 Acked Read 1 Done Sent Done false
32480 9196 Acked Write 2 Done Sent Done false
32481 9197 Acked Write 5 Done Sent Done false
32482 9198 Acked Write 4 Done Sent Done false
32483 9199 Acked Write 2 Done Sent Done false
32484 9200 Acked Write 5 Done Sent Done false
32485 9201 Acked Read 3 Done Sent Done false
32486 9202 Acked Write 6 Done Sent Done false
32487 9203 Acked Read 6 Done Sent Done false
32488 9204 Acked Write 4 Done Sent Done false
32489 9205 Acked Flush 0 Done Sent Done false
32490 9206 Acked Write 4 Done Sent Done false
32491 9207 Acked Write 4 Done Sent Done false
32492 9208 Acked Write 6 Done Sent Done false
32493 9209 Acked Read 9 Done Sent Done false
32494 9210 Acked Read 10 Done Sent Done false
32495 9211 Acked Read 4 Done Sent Done false
32496 9212 Acked Flush 0 Done Sent Done false
32497 9213 Acked Read 9 Done Sent Done false
32498 9214 Acked Write 5 Done Sent Done false
32499 9215 Acked Read 5 Done Sent Done false
32500 9216 Acked Read 2 Done Sent Done false
32501 9217 Acked Read 8 Done Sent Done false
32502 9218 Acked Write 3 Done Sent Done false
32503 9219 Acked Read 8 Done Sent Done false
32504 9220 Acked Write 4 Done Sent Done false
32505 9221 Acked Read 1 Done Sent Done false
32506 9222 Acked Read 6 Done Sent Done false
32507 9223 Acked Write 5 Done Sent Done false
32508 9224 Acked Flush 0 Done Sent Done false
32509 9225 Acked Read 3 Done Sent Done false
32510 9226 Acked Read 7 Done Sent Done false
32511 9227 Acked Read 10 Done Sent Done false
32512 9228 Acked Read 2 Done Sent Done false
32513 9229 Acked Write 3 Done Sent Done false
32514 9230 Acked Write 1 Done Sent Done false
32515 9231 Acked Write 8 Done Sent Done false
32516 9232 Acked Write 6 Done Sent Done false
32517 9233 Acked Read 9 Done Sent Done false
32518 9234 Acked Read 5 Done Sent Done false
32519 9235 Acked Flush 0 Done Sent Done false
32520 9236 Acked Write 5 Done Sent Done false
32521 9237 Acked Write 1 Done Sent Done false
32522 9238 Acked Write 4 Done Sent Done false
32523 9239 Acked Write 8 Done Sent Done false
32524 9240 Acked Write 1 Done Sent Done false
32525 9241 Acked Read 6 Done Sent Done false
32526 9242 Acked Read 7 Done Sent Done false
32527 9243 Acked Write 4 Done Sent Done false
32528 9244 Acked Write 10 Done Sent Done false
32529 9245 Acked Read 6 Done Sent Done false
32530 9246 Acked Read 9 Done Sent Done false
32531 9247 Acked Write 2 Done Sent Done false
32532 9248 Acked Write 4 Done Sent Done false
32533 9249 Acked Read 7 Done Sent Done false
32534 9250 Acked Read 10 Done Sent Done false
32535 9251 Acked Write 5 Done Sent Done false
32536 9252 Acked Write 8 Done Sent Done false
32537 9253 Acked Write 8 Done Sent Done false
32538 9254 Acked Write 8 Done Sent Done false
32539 9255 Acked Flush 0 Done Sent Done false
32540 9256 Acked Read 5 Done Sent Done false
32541 9257 Acked Write 5 Done Sent Done false
32542 9258 Acked Write 10 Done Sent Done false
32543 9259 Acked Read 9 Done Sent Done false
32544 9260 Acked Write 4 Done Sent Done false
32545 9261 Acked Read 2 Done Sent Done false
32546 9262 Acked Read 8 Done Sent Done false
32547 9263 Acked Write 2 Done Sent Done false
32548 9264 Acked Read 5 Done Sent Done false
32549 9265 Acked Write 2 Done Sent Done false
32550 9266 Acked Read 7 Done Sent Done false
32551 9267 Acked Read 8 Done Sent Done false
32552 9268 Acked Read 3 Done Sent Done false
32553 9269 Acked Write 4 Done Sent Done false
32554 9270 Acked Write 2 Done Sent Done false
32555 9271 Acked Write 3 Done Sent Done false
32556 9272 Acked Write 6 Done Sent Done false
32557 9273 Acked Read 1 Done Sent Done false
32558 9274 Acked Read 3 Done Sent Done false
32559 9275 Acked Write 6 Done Sent Done false
32560 9276 Acked Write 7 Done Sent Done false
32561 9277 Acked Write 5 Done Sent Done false
32562 9278 Acked Write 4 Done Sent Done false
32563 9279 Acked Write 1 Done Sent Done false
32564 9280 Acked Write 1 Done Sent Done false
32565 9281 Acked Write 4 Done Sent Done false
32566 9282 Acked Read 6 Done Sent Done false
32567 9283 Acked Read 10 Done Sent Done false
32568 9284 Acked Write 10 Done Sent Done false
32569 9285 Acked Read 9 Done Sent Done false
32570 9286 Acked Read 5 Done Sent Done false
32571 9287 Acked Read 3 Done Sent Done false
32572 9288 Acked Flush 0 Done Sent Done false
32573 9289 Acked Read 9 Done Sent Done false
32574 9290 Acked Write 9 Done Sent Done false
32575 9291 Acked Read 10 Done Sent Done false
32576 9292 Acked Read 9 Done Sent Done false
32577 9293 Acked Read 10 Done Sent Done false
32578 9294 Acked Write 9 Done Sent Done false
32579 9295 Acked Flush 0 Done Sent Done false
32580 9296 Acked Read 9 Done Sent Done false
32581 9297 Acked Read 8 Done Sent Done false
32582 9298 Acked Write 9 Done Sent Done false
32583 9299 Acked Write 3 Done Sent Done false
32584 9300 Acked Write 2 Done Sent Done false
32585 9301 Acked Read 4 Done Sent Done false
32586 9302 Acked Write 1 Done Sent Done false
32587 9303 Acked Write 3 Done Sent Done false
32588 9304 Acked Write 5 Done Sent Done false
32589 9305 Acked Write 6 Done Sent Done false
32590 9306 Acked Read 7 Done Sent Done false
32591 9307 Acked Write 6 Done Sent Done false
32592 9308 Acked Read 3 Done Sent Done false
32593 9309 Acked Read 6 Done Sent Done false
32594 9310 Acked Read 1 Done Sent Done false
32595 9311 Acked Read 7 Done Sent Done false
32596 9312 Acked Read 5 Done Sent Done false
32597 9313 Acked Write 9 Done Sent Done false
32598 9314 Acked Flush 0 Done Sent Done false
32599 9315 Acked Read 3 Done Sent Done false
32600 9316 Acked Write 8 Done Sent Done false
32601 9317 Acked Read 3 Done Sent Done false
32602 9318 Acked Write 9 Done Sent Done false
32603 9319 Acked Read 6 Done Sent Done false
32604 9320 Acked Flush 0 Done Sent Done false
32605 9321 Acked Read 3 Done Sent Done false
32606 9322 Acked Write 7 Done Sent Done false
32607 9323 Acked Write 7 Done Sent Done false
32608 9324 Acked Write 1 Done Sent Done false
32609 9325 Acked Write 10 Done Sent Done false
32610 9326 Acked Read 8 Done Sent Done false
32611 9327 Acked Read 5 Done Sent Done false
32612 9328 Acked Write 5 Done Sent Done false
32613 9329 Acked Read 1 Done Sent Done false
32614 9330 Acked Write 9 Done Sent Done false
32615 9331 Acked Write 3 Done Sent Done false
32616 9332 Acked Read 9 Done Sent Done false
32617 9333 Acked Write 9 Done Sent Done false
32618 9334 Acked Write 2 Done Sent Done false
32619 9335 Acked Read 1 Done Sent Done false
32620 9336 Acked Read 4 Done Sent Done false
32621 9337 Acked Read 5 Done Sent Done false
32622 9338 Acked Read 10 Done Sent Done false
32623 9339 Acked Read 1 Done Sent Done false
32624 9340 Acked Read 8 Done Sent Done false
32625 9341 Acked Write 6 Done Sent Done false
32626 9342 Acked Read 5 Done Sent Done false
32627 9343 Acked Read 1 Done Sent Done false
32628 9344 Acked Read 5 Done Sent Done false
32629 9345 Acked Write 10 Done Sent Done false
32630 9346 Acked Flush 0 Done Sent Done false
32631 9347 Acked Read 5 Done Sent Done false
32632 9348 Acked Write 3 Done Sent Done false
32633 9349 Acked Read 8 Done Sent Done false
32634 9350 Acked Read 9 Done Sent Done false
32635 9351 Acked Read 1 Done Sent Done false
32636 9352 Acked Read 2 Done Sent Done false
32637 9353 Acked Read 3 Done Sent Done false
32638 9354 Acked Write 1 Done Sent Done false
32639 9355 Acked Write 7 Done Sent Done false
32640 9356 Acked Write 4 Done Sent Done false
32641 9357 Acked Write 4 Done Sent Done false
32642 9358 Acked Write 8 Done Sent Done false
32643 9359 Acked Write 5 Done Sent Done false
32644 9360 Acked Read 4 Done Sent Done false
32645 9361 Acked Write 8 Done Sent Done false
32646 9362 Acked Read 1 Done Sent Done false
32647 9363 Acked Read 9 Done Sent Done false
32648 9364 Acked Read 3 Done Sent Done false
32649 9365 Acked Write 10 Done Sent Done false
32650 9366 Acked Write 3 Done Sent Done false
32651 9367 Acked Write 1 Done Sent Done false
32652 9368 Acked Flush 0 Done Sent Done false
32653 9369 Acked Read 7 Done Sent Done false
32654 9370 Acked Read 2 Done Sent Done false
32655 9371 Acked Flush 0 Done Sent Done false
32656 9372 Acked Write 10 Done Sent Done false
32657 9373 Acked Write 10 Done Sent Done false
32658 9374 Acked Read 6 Done Sent Done false
32659 9375 Acked Read 4 Done Sent Done false
32660 9376 Acked Read 7 Done Sent Done false
32661 9377 Acked Write 4 Done Sent Done false
32662 9378 Acked Write 4 Done Sent Done false
32663 9379 Acked Write 4 Done Sent Done false
32664 9380 Acked Write 4 Done Sent Done false
32665 9381 Acked Write 9 Done Sent Done false
32666 9382 Acked Write 7 Done Sent Done false
32667 9383 Acked Flush 0 Done Sent Done false
32668 9384 Acked Read 6 Done Sent Done false
32669 9385 Acked Write 7 Done Sent Done false
32670 9386 Acked Write 8 Done Sent Done false
32671 9387 Acked Write 4 Done Sent Done false
32672 9388 Acked Write 6 Done Sent Done false
32673 9389 Acked Write 9 Done Sent Done false
32674 9390 Acked Read 5 Done Sent Done false
32675 9391 Acked Write 3 Done Sent Done false
32676 9392 Acked Flush 0 Done Sent Done false
32677 9393 Acked Write 2 Done Sent Done false
32678 9394 Acked Read 7 Done Sent Done false
32679 9395 Acked Read 9 Done Sent Done false
32680 9396 Acked Read 7 Done Sent Done false
32681 9397 Acked Write 3 Done Sent Done false
32682 9398 Acked Read 6 Done Sent Done false
32683 9399 Acked Flush 0 Done Sent Done false
32684 9400 Acked Flush 0 Done Sent Done false
32685 9401 Acked Write 4 Done Sent Done false
32686 9402 Acked Write 4 Done Sent Done false
32687 9403 Acked Write 5 Done Sent Done false
32688 9404 Acked Read 3 Done Sent Done false
32689 9405 Acked Write 10 Done Sent Done false
32690 9406 Acked Write 4 Done Sent Done false
32691 9407 Acked Read 6 Done Sent Done false
32692 9408 Acked Flush 0 Done Sent Done false
32693 9409 Acked Read 6 Done Sent Done false
32694 9410 Acked Write 6 Done Sent Done false
32695 9411 Acked Read 2 Done Sent Done false
32696 9412 Acked Read 6 Done Sent Done false
32697 9413 Acked Write 1 Done Sent Done false
32698 9414 Acked Read 9 Done Sent Done false
32699 9415 Acked Write 7 Done Sent Done false
32700 9416 Acked Flush 0 Done Sent Done false
32701 9417 Acked Read 9 Done Sent Done false
32702 9418 Acked Read 6 Done Sent Done false
32703 9419 Acked Flush 0 Done Sent Done false
32704 9420 Acked Write 5 Done Sent Done false
32705 9421 Acked Read 2 Done Sent Done false
32706 9422 Acked Write 3 Done Sent Done false
32707 9423 Acked Write 2 Done Sent Done false
32708 9424 Acked Read 7 Done Sent Done false
32709 9425 Acked Write 6 Done Sent Done false
32710 9426 Acked Write 4 Done Sent Done false
32711 9427 Acked Write 3 Done Sent Done false
32712 9428 Acked Read 6 Done Sent Done false
32713 9429 Acked Write 7 Done Sent Done false
32714 9430 Acked Write 10 Done Sent Done false
32715 9431 Acked Write 6 Done Sent Done false
32716 9432 Acked Write 5 Done Sent Done false
32717 9433 Acked Read 9 Done Sent Done false
32718 9434 Acked Read 2 Done Sent Done false
32719 9435 Acked Read 5 Done Sent Done false
32720 9436 Acked Write 2 Done Sent Done false
32721 9437 Acked Write 3 Done Sent Done false
32722 9438 Acked Read 7 Done Sent Done false
32723 9439 Acked Write 1 Done Sent Done false
32724 9440 Acked Read 8 Done Sent Done false
32725 9441 Acked Write 1 Done Sent Done false
32726 9442 Acked Read 4 Done Sent Done false
32727 9443 Acked Read 6 Done Sent Done false
32728 9444 Acked Read 1 Done Sent Done false
32729 9445 Acked Write 8 Done Sent Done false
32730 9446 Acked Read 3 Done Sent Done false
32731 9447 Acked Write 4 Done Sent Done false
32732 9448 Acked Read 4 Done Sent Done false
32733 9449 Acked Write 5 Done Sent Done false
32734 9450 Acked Write 5 Done Sent Done false
32735 9451 Acked Read 5 Done Sent Done false
32736 9452 Acked Flush 0 Done Sent Done false
32737 STATES DS:0 DS:1 DS:2 TOTAL
32738 Sent 0 302 0 302
32739 Done 302 0 302 604
32740 Skipped 0 0 0 0
32741 Error 0 0 0 0
32742 Last Flush: 9452 9150 9452
32743 Downstairs last five completed: 9150 9149 9148 9147 9146
32744 Upstairs last five completed: 9452 9451 9450 9449 9448
32745 CLIENT: Up:0 ds:302 act:2
327462025-01-17T21:17:57.154ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
327472025-01-17T21:17:57.154ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59804) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
327482025-01-17T21:17:57.155ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
327492025-01-17T21:17:57.155ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
327502025-01-17T21:17:57.155ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
327512025-01-17T21:17:57.155ZINFOcrucible: send last flush ID to this DS: 9150 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
327522025-01-17T21:17:57.156ZINFOcrucible: Replied this last flush ID: 9150 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
327532025-01-17T21:17:57.156ZINFOcrucible: [1] client re-new 302 jobs since flush 9150 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
327542025-01-17T21:17:57.156ZINFOcrucible: [1] Replayed 302 jobs since flush: 9150 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
32755 ----------------------------------------------------------------
32756 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
32757 Downstairs last five completed: 9452 9451 9450 9449 9448
32758 Upstairs last five completed: 9452 9451 9450 9449 9448
32759 CLIENT: Up:0 ds:0 act:3
32760 Replay: All jobs finished
32761 Main task has work to do, go find it
32762 got dsc Stop(0)
32763 stop 0
32764 Jan 17 21:17:59.388 INFO [8810] Got stop action so:false kr:false
32765 request completed, latency_us: 358, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 58663974-c335-46d1-ac81-aff79740c35b, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
32766 Jan 17 21:17:59.389 INFO request completed, latency_us{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:17:59.388992413Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
32767 : 127, response_code: 200, uri: /state/cid/0, method: GET, req_id: 277dce3c-8612-401f-b579-7276245fe5a6, remote_addr: 127.0.0.1:57191, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:17:59.38908358Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
32768 local_addr: 127.0.0.{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:17:59.389140372Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
32769 1:9998
327702025-01-17T21:17:59.389ZINFOcrucible: client task is exiting = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
32771 [8810] Exited with: ExitStatus(unix_wait_status(9))
32772 [8810][0] reports Exit
32773 Jan 17 21:18:03.391 INFO request completed, latency_us: 209, response_code: 200, uri: /state/cid/0, method: GET, req_id: 257b32fc-ac21-4cd8-a5b3-bca98ab7a5c0, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
32774 001/300 Write block 419 len 4096 data: 6
32775 002/300 Read block 4830 len 16384
32776 003/300 Read block 1927 len 40960
32777 004/300 Read block 1364 len 24576
32778 005/300 Write block 1131 len 12288 data: 4 5 4
32779 006/300 Write block 4476 len 4096 data: 8
32780 007/300 Write block 4936 len 4096 data: 4
32781 008/300 Write block 3017 len 32768 data: 2 2 2 2 2 2 2 2
32782 009/300 Read block 3311 len 20480
32783 010/300 Read block 2343 len 12288
32784 011/300 Flush
32785 012/300 Write block 1578 len 40960 data: 4 5 5 6 10 9 11 11 10 9
32786 013/300 Read block 2220 len 32768
32787 014/300 Read block 3786 len 32768
32788 015/300 Read block 3457 len 20480
32789 016/300 Read block 1187 len 28672
32790 017/300 Read block 1764 len 32768
32791 018/300 Write block 144 len 28672 data: 4 5 5 6 5 6 5
32792 019/300 Write block 3557 len 8192 data: 2 2
32793 020/300 Read block 1489 len 20480
32794 021/300 Read block 1906 len 4096
32795 022/300 Flush
32796 023/300 Flush
32797 024/300 Read block 3982 len 16384
32798 025/300 Read block 4441 len 4096
32799 026/300 Write block 3868 len 24576 data: 4 5 5 5 5 5
32800 027/300 Read block 1359 len 24576
32801 028/300 Read block 1651 len 36864
32802 029/300 Write block 2654 len 16384 data: 6 6 6 6
32803 030/300 Read block 4855 len 8192
32804 031/300 Flush
32805 032/300 Read block 4480 len 12288
32806 033/300 Write block 4718 len 40960 data: 7 8 8 7 5 3 3 5 5 5
32807 034/300 Read block 2726 len 20480
32808 035/300 Write block 1443 len 40960 data: 9 6 6 4 6 6 6 5 5 5
32809 036/300 Read block 4709 len 40960
32810 037/300 Read block 867 len 20480
32811 038/300 Read block 1862 len 24576
32812 039/300 Write block 2424 len 36864 data: 4 4 5 6 7 7 6 6 6
32813 040/300 Read block 1118 len 36864
32814 041/300 Read block 2409 len 28672
32815 042/300 Read block 904 len 16384
32816 043/300 Flush
32817 044/300 Write block 2263 len 40960 data: 6 9 8 9 7 5 6 6 6 5
32818 045/300 Flush
32819 046/300 Write block 1987 len 8192 data: 6 7
32820 047/300 Flush
32821 048/300 Write block 4253 len 4096 data: 5
32822 049/300 Read block 4904 len 20480
32823 050/300 Read block 3107 len 20480
32824 051/300 Read block 3170 len 20480
32825 052/300 Flush
32826 053/300 Write block 4347 len 20480 data: 7 7 7 7 5
32827 054/300 Read block 1059 len 32768
32828 055/300 Write block 1335 len 36864 data: 5 6 6 4 4 4 3 6 6
32829 056/300 Read block 2089 len 24576
32830 057/300 Write block 1268 len 36864 data: 8 7 6 5 5 4 5 4 4
32831 058/300 Flush
32832 059/300 Read block 2358 len 32768
32833 060/300 Read block 2360 len 36864
32834 061/300 Read block 4957 len 36864
32835 062/300 Read block 3045 len 12288
32836 063/300 Read block 4605 len 36864
32837 064/300 Read block 3413 len 8192
32838 065/300 Write block 2506 len 28672 data: 4 4 4 5 8 7 7
32839 066/300 Write block 1979 len 4096 data: 10
32840 067/300 Flush
32841 068/300 Write block 1640 len 36864 data: 5 6 6 5 4 4 5 6 4
32842 069/300 Write block 2767 len 36864 data: 8 7 6 4 4 5 6 6 5
32843 070/300 Read block 3143 len 40960
32844 071/300 Read block 3190 len 8192
32845 072/300 Read block 4298 len 28672
32846 073/300 Write block 416 len 4096 data: 5
32847 074/300 Write block 609 len 20480 data: 5 4 6 7 5
32848 075/300 Flush
32849 076/300 Read block 4891 len 4096
32850 077/300 Read block 4238 len 12288
32851 078/300 Write block 4848 len 12288 data: 10 10 8
32852 079/300 Write block 3284 len 32768 data: 6 5 4 3 3 3 3 3
32853 080/300 Write block 4791 len 40960 data: 5 4 4 5 5 4 3 3 3 4
32854 081/300 Read block 3233 len 32768
32855 082/300 Write block 244 len 28672 data: 4 6 8 6 7 7 9
32856 083/300 Write block 2344 len 8192 data: 7 9
32857 084/300 Flush
32858 085/300 Read block 1146 len 40960
32859 086/300 Read block 3381 len 16384
32860 087/300 Write block 405 len 32768 data: 6 4 5 6 6 6 5 4
32861 088/300 Read block 2663 len 28672
32862 089/300 Read block 2176 len 32768
32863 090/300 Read block 259 len 28672
32864 091/300 Read block 41 len 4096
32865 092/300 Write block 3656 len 32768 data: 3 3 3 3 3 4 3 3
32866 093/300 Write block 1453 len 36864 data: 7 6 6 3 3 3 3 3 4
32867 094/300 Read block 1294 len 8192
32868 095/300 Read block 3573 len 40960
32869 096/300 Read block 1737 len 32768
32870 097/300 Write block 2417 len 32768 data: 4 5 6 6 6 5 5 5
32871 098/300 Read block 1305 len 36864
32872 099/300 Read block 1471 len 40960
32873 100/300 Flush
32874 101/300 Read block 2784 len 20480
32875 102/300 Write block 3279 len 32768 data: 7 6 5 5 6 7 6 5
32876 103/300 Flush
32877 104/300 Read block 917 len 24576
32878 105/300 Write block 1122 len 24576 data: 5 4 4 4 4 5
32879 106/300 Write block 1654 len 20480 data: 4 8 6 6 8
32880 107/300 Write block 2882 len 24576 data: 6 5 6 7 8 9
32881 108/300 Read block 2558 len 36864
32882 109/300 Flush
32883 110/300 Read block 2814 len 16384
32884 111/300 Flush
32885 112/300 Read block 620 len 8192
32886 113/300 Read block 2934 len 12288
32887 114/300 Write block 2176 len 24576 data: 4 5 4 3 3 2
32888 115/300 Read block 2650 len 32768
32889 116/300 Flush
32890 117/300 Write block 1966 len 32768 data: 7 7 7 8 6 5 4 3
32891 118/300 Read block 4162 len 4096
32892 119/300 Read block 4703 len 4096
32893 120/300 Read block 2205 len 32768
32894 121/300 Write block 4891 len 8192 data: 7 7
32895 122/300 Write block 1146 len 4096 data: 5
32896 123/300 Read block 4890 len 32768
32897 124/300 Read block 3742 len 36864
32898 125/300 Read block 2563 len 12288
32899 126/300 Write block 1171 len 32768 data: 5 4 5 4 4 5 6 3
32900 127/300 Read block 327 len 8192
32901 128/300 Read block 2311 len 28672
32902 129/300 Read block 484 len 36864
32903 130/300 Read block 440 len 16384
32904 131/300 Read block 2678 len 20480
32905 132/300 Read block 2454 len 28672
32906 133/300 Read block 640 len 8192
32907 134/300 Flush
32908 135/300 Flush
32909 136/300 Read block 4754 len 12288
32910 137/300 Write block 1686 len 12288 data: 4 4 5
32911 138/300 Read block 2344 len 28672
32912 139/300 Write block 1144 len 16384 data: 4 4 6 5
32913 140/300 Read block 4062 len 36864
32914 141/300 Read block 3372 len 4096
32915 142/300 Write block 936 len 24576 data: 6 6 6 4 5 4
32916 143/300 Flush
32917 144/300 Write block 3357 len 4096 data: 4
32918 145/300 Write block 2848 len 40960 data: 5 4 4 4 3 3 3 3 4 4
32919 146/300 Read block 4900 len 16384
32920 147/300 Write block 1812 len 12288 data: 5 5 5
32921 148/300 Read block 2705 len 4096
32922 149/300 Read block 4914 len 40960
32923 150/300 Read block 536 len 20480
32924 151/300 Write block 3754 len 36864 data: 8 6 7 7 5 5 5 6 5
32925 152/300 Read block 1377 len 40960
32926 153/300 Read block 161 len 24576
32927 154/300 Read block 2392 len 24576
32928 155/300 Read block 185 len 28672
32929 156/300 Write block 2015 len 28672 data: 4 4 5 5 5 5 6
32930 157/300 Write block 4700 len 24576 data: 7 10 9 9 9 7
32931 158/300 Write block 640 len 32768 data: 5 4 6 6 7 5 5 6
32932 159/300 Read block 1147 len 32768
32933 160/300 Write block 2339 len 28672 data: 7 5 5 6 7 8 10
32934 161/300 Flush
32935 162/300 Read block 1495 len 4096
32936 163/300 Read block 2435 len 20480
32937 164/300 Read block 2704 len 32768
32938 165/300 Write block 3976 len 4096 data: 5
32939 166/300 Write block 884 len 8192 data: 6 4
32940 167/300 Write block 383 len 16384 data: 7 7 6 7
32941 168/300 Read block 3949 len 16384
32942 169/300 Write block 1141 len 36864 data: 4 4 4 5 5 7 6 7 7
32943 170/300 Flush
32944 171/300 Write block 4254 len 40960 data: 6 3 3 4 4 4 4 8 8 9
32945 172/300 Read block 3645 len 8192
32946 173/300 Write block 1244 len 32768 data: 6 7 7 7 4 3 5 4
32947 174/300 Write block 2246 len 8192 data: 4 3
32948 175/300 Read block 2717 len 12288
32949 176/300 Read block 1361 len 4096
32950 177/300 Write block 4573 len 32768 data: 4 4 5 7 6 6 7 6
32951 178/300 Read block 1813 len 4096
32952 179/300 Read block 1037 len 32768
32953 180/300 Write block 7 len 12288 data: 4 5 4
32954 181/300 Write block 3234 len 8192 data: 5 6
32955 182/300 Write block 3616 len 40960 data: 7 7 8 7 7 10 8 8 7 6
32956 183/300 Write block 3978 len 16384 data: 4 4 4 3
32957 184/300 Read block 3058 len 24576
32958 185/300 Read block 1108 len 24576
32959 186/300 Write block 601 len 36864 data: 4 5 5 5 5 5 4 5 6
32960 187/300 Read block 2006 len 28672
32961 188/300 Write block 2905 len 32768 data: 8 7 7 6 6 4 5 6
32962 189/300 Read block 2472 len 12288
32963 190/300 Read block 4094 len 28672
32964 191/300 Read block 1312 len 28672
32965 192/300 Write block 1188 len 8192 data: 5 5
32966 193/300 Read block 1936 len 36864
32967 194/300 Read block 418 len 16384
32968 195/300 Write block 2739 len 24576 data: 7 7 8 9 10 9
32969 196/300 Flush
32970 197/300 Read block 1160 len 32768
32971 198/300 Write block 1582 len 24576 data: 11 10 12 12 11 10
32972 199/300 Read block 3196 len 36864
32973 200/300 Write block 939 len 20480 data: 5 6 5 6 7
32974 201/300 Read block 2028 len 4096
32975 202/300 Write block 291 len 36864 data: 7 7 7 8 7 5 5 6 7
32976 203/300 Write block 1128 len 16384 data: 6 6 5 5
32977 204/300 Write block 205 len 40960 data: 6 8 8 8 8 8 8 7 8 6
32978 205/300 Write block 572 len 24576 data: 6 6 5 6 6 5
32979 206/300 Read block 2947 len 16384
32980 207/300 Write block 2472 len 32768 data: 6 7 7 6 6 6 8 8
32981 208/300 Write block 114 len 36864 data: 5 5 8 8 7 5 6 6 7
32982 209/300 Read block 2388 len 28672
32983 210/300 Read block 3608 len 36864
32984 211/300 Read block 4393 len 20480
32985 212/300 Read block 3814 len 8192
32986 213/300 Read block 1466 len 4096
32987 214/300 Write block 4958 len 36864 data: 6 6 6 5 5 5 4 4 3
32988 215/300 Read block 42 len 12288
32989 216/300 Write block 4440 len 4096 data: 6
32990 217/300 Write block 4132 len 32768 data: 7 6 6 5 4 6 6 8
32991 218/300 Write block 3451 len 28672 data: 10 8 6 6 4 4 4
32992 219/300 Read block 891 len 12288
32993 220/300 Write block 1136 len 36864 data: 5 6 6 5 4 5 5 5 6
32994 221/300 Read block 4924 len 20480
32995 222/300 Read block 70 len 36864
32996 223/300 Read block 2011 len 36864
32997 224/300 Write block 1950 len 32768 data: 6 7 9 8 8 9 7 7
32998 225/300 Write block 3490 len 24576 data: 5 4 4 4 5 6
32999 226/300 Read block 1654 len 36864
33000 227/300 Write block 1193 len 16384 data: 8 8 8 9
33001 228/300 Write block 2142 len 36864 data: 8 8 9 9 9 9 9 9 8
33002 229/300 Read block 1144 len 24576
33003 230/300 Write block 2200 len 4096 data: 5
33004 231/300 Write block 2510 len 8192 data: 9 8
33005 232/300 Flush
33006 233/300 Write block 1777 len 16384 data: 5 4 3 3
33007 234/300 Write block 4829 len 28672 data: 8 8 6 7 7 4 5
33008 235/300 Read block 705 len 36864
33009 236/300 Write block 1346 len 20480 data: 5 6 7 5 3
33010 237/300 Write block 3565 len 28672 data: 4 3 2 3 3 5 5
33011 238/300 Flush
33012 239/300 Write block 103 len 8192 data: 8 8
33013 240/300 Read block 1750 len 32768
33014 241/300 Flush
33015 242/300 Read block 434 len 32768
33016 243/300 Read block 2743 len 36864
33017 244/300 Write block 2547 len 4096 data: 7
33018 245/300 Read block 3762 len 36864
33019 246/300 Write block 4348 len 32768 data: 8 8 8 6 4 2 2 2
33020 247/300 Write block 4033 len 12288 data: 6 5 6
33021 248/300 Read block 2706 len 24576
33022 249/300 Read block 2767 len 28672
33023 250/300 Read block 4208 len 8192
33024 251/300 Read block 297 len 36864
33025 252/300 Flush
33026 253/300 Read block 4687 len 28672
33027 254/300 Write block 207 len 24576 data: 9 9 9 9 9 8
33028 255/300 Flush
33029 256/300 Flush
33030 257/300 Write block 4852 len 4096 data: 6
33031 258/300 Write block 4594 len 36864 data: 6 6 6 5 5 6 6 5 4
33032 259/300 Flush
33033 260/300 Read block 3550 len 36864
33034 261/300 Flush
33035 262/300 Write block 2222 len 28672 data: 7 7 7 5 5 6 6
33036 263/300 Write block 1818 len 36864 data: 2 2 4 5 5 6 7 8 8
33037 264/300 Write block 229 len 12288 data: 7 5 5
33038 265/300 Read block 691 len 20480
33039 266/300 Read block 802 len 20480
33040 267/300 Write block 3070 len 24576 data: 4 5 6 4 5 5
33041 268/300 Read block 3303 len 40960
33042 269/300 Read block 3226 len 8192
33043 270/300 Read block 535 len 24576
33044 271/300 Read block 4213 len 28672
33045 272/300 Flush
33046 273/300 Write block 1978 len 28672 data: 11 11 7 8 7 7 7
33047 274/300 Write block 1268 len 16384 data: 9 8 7 6
33048 275/300 Write block 2982 len 24576 data: 6 5 4 3 4 6
33049 276/300 Write block 4695 len 20480 data: 5 6 7 6 6
33050 277/300 Write block 2494 len 36864 data: 8 8 6 5 6 4 3 3 3
33051 278/300 Read block 3888 len 20480
33052 279/300 Read block 1032 len 28672
33053 280/300 Read block 1551 len 4096
33054 281/300 Write block 983 len 24576 data: 4 5 3 3 3 3
33055 282/300 Write block 2580 len 28672 data: 4 5 5 6 5 7 10
33056 283/300 Read block 4408 len 24576
33057 284/300 Write block 1298 len 24576 data: 4 3 3 3 4 4
33058 285/300 Read block 3045 len 16384
33059 286/300 Read block 1803 len 36864
33060 287/300 Write block 4284 len 36864 data: 4 3 3 4 4 4 4 4 4
33061 288/300 Write block 2821 len 36864 data: 3 2 2 4 4 5 5 5 5
33062 289/300 Flush
33063 290/300 Read block 4881 len 36864
33064 291/300 Write block 1696 len 16384 data: 6 6 6 4
33065 292/300 Read block 4991 len 24576
33066 293/300 Read block 4770 len 40960
33067 294/300 Flush
33068 295/300 Read block 3339 len 20480
33069 296/300 Read block 3698 len 20480
33070 297/300 Read block 1358 len 40960
33071 298/300 Flush
33072 299/300 Read block 1858 len 36864
33073 300/300 Flush
33074 301/300 Read block 4831 len 28672
33075 Main task has work to do, go find it
33076 got dsc Start(0)
33077 start 0
33078 Jan 17 21:18:03.642 INFO request completed, latency_us: 201, response_code: 204, uri: /start/cid/0, method: POST, req_id: 465fc5e0-de2e-4f8b-bb45-b8e612798998, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
33079 Replay: started 0, returned:Ok(())
33080 ----------------------------------------------------------------
33081 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
33082 States: Offline Active Active
33083 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33084 9453 Acked Write 1 Sent Done Done false
33085 9454 Acked Read 4 Sent Done Done false
33086 9455 Acked Read 10 Sent Done Done false
33087 9456 Acked Read 6 Sent Done Done false
33088 9457 Acked Write 3 Sent Done Done false
33089 9458 Acked Write 1 Sent Done Done false
33090 9459 Acked Write 1 Sent Done Done false
33091 9460 Acked Write 8 Sent Done Done false
33092 9461 Acked Read 5 Sent Done Done false
33093 9462 Acked Read 3 Sent Done Done false
33094 9463 Acked Flush 0 Sent Done Done false
33095 9464 Acked Write 10 Sent Done Done false
33096 9465 Acked Read 8 Sent Done Done false
33097 9466 Acked Read 8 Sent Done Done false
33098 9467 Acked Read 5 Sent Done Done false
33099 9468 Acked Read 7 Sent Done Done false
33100 9469 Acked Read 8 Sent Done Done false
33101 9470 Acked Write 7 Sent Done Done false
33102 9471 Acked Write 2 Sent Done Done false
33103 9472 Acked Read 5 Sent Done Done false
33104 9473 Acked Read 1 Sent Done Done false
33105 9474 Acked Flush 0 Sent Done Done false
33106 9475 Acked Flush 0 Sent Done Done false
33107 9476 Acked Read 4 Sent Done Done false
33108 9477 Acked Read 1 Sent Done Done false
33109 9478 Acked Write 6 Sent Done Done false
33110 9479 Acked Read 6 Sent Done Done false
33111 9480 Acked Read 9 Sent Done Done false
33112 9481 Acked Write 4 Sent Done Done false
33113 9482 Acked Read 2 Sent Done Done false
33114 9483 Acked Flush 0 Sent Done Done false
33115 9484 Acked Read 3 Sent Done Done false
33116 9485 Acked Write 10 Sent Done Done false
33117 9486 Acked Read 5 Sent Done Done false
33118 9487 Acked Write 10 Sent Done Done false
33119 9488 Acked Read 10 Sent Done Done false
33120 9489 Acked Read 5 Sent Done Done false
33121 9490 Acked Read 6 Sent Done Done false
33122 9491 Acked Write 9 Sent Done Done false
33123 9492 Acked Read 9 Sent Done Done false
33124 9493 Acked Read 7 Sent Done Done false
33125 9494 Acked Read 4 Sent Done Done false
33126 9495 Acked Flush 0 Sent Done Done false
33127 9496 Acked Write 10 Sent Done Done false
33128 9497 Acked Flush 0 Sent Done Done false
33129 9498 Acked Write 2 Sent Done Done false
33130 9499 Acked Flush 0 Sent Done Done false
33131 9500 Acked Write 1 Sent Done Done false
33132 9501 Acked Read 5 Sent Done Done false
33133 9502 Acked Read 5 Sent Done Done false
33134 9503 Acked Read 5 Sent Done Done false
33135 9504 Acked Flush 0 Sent Done Done false
33136 9505 Acked Write 5 Sent Done Done false
33137 9506 Acked Read 8 Sent Done Done false
33138 9507 Acked Write 9 Sent Done Done false
33139 9508 Acked Read 6 Sent Done Done false
33140 9509 Acked Write 9 Sent Done Done false
33141 9510 Acked Flush 0 Sent Done Done false
33142 9511 Acked Read 8 Sent Done Done false
33143 9512 Acked Read 9 Sent Done Done false
33144 9513 Acked Read 9 Sent Done Done false
33145 9514 Acked Read 3 Sent Done Done false
33146 9515 Acked Read 9 Sent Done Done false
33147 9516 Acked Read 2 Sent Done Done false
33148 9517 Acked Write 7 Sent Done Done false
33149 9518 Acked Write 1 Sent Done Done false
33150 9519 Acked Flush 0 Sent Done Done false
33151 9520 Acked Write 9 Sent Done Done false
33152 9521 Acked Write 9 Sent Done Done false
33153 9522 Acked Read 10 Sent Done Done false
33154 9523 Acked Read 2 Sent Done Done false
33155 9524 Acked Read 7 Sent Done Done false
33156 9525 Acked Write 1 Sent Done Done false
33157 9526 Acked Write 5 Sent Done Done false
33158 9527 Acked Flush 0 Sent Done Done false
33159 9528 Acked Read 1 Sent Done Done false
33160 9529 Acked Read 3 Sent Done Done false
33161 9530 Acked Write 3 Sent Done Done false
33162 9531 Acked Write 8 Sent Done Done false
33163 9532 Acked Write 10 Sent Done Done false
33164 9533 Acked Read 8 Sent Done Done false
33165 9534 Acked Write 7 Sent Done Done false
33166 9535 Acked Write 2 Sent Done Done false
33167 9536 Acked Flush 0 Sent Done Done false
33168 9537 Acked Read 10 Sent Done Done false
33169 9538 Acked Read 4 Sent Done Done false
33170 9539 Acked Write 8 Sent Done Done false
33171 9540 Acked Read 7 Sent Done Done false
33172 9541 Acked Read 8 Sent Done Done false
33173 9542 Acked Read 7 Sent Done Done false
33174 9543 Acked Read 1 Sent Done Done false
33175 9544 Acked Write 8 Sent Done Done false
33176 9545 Acked Write 9 Sent Done Done false
33177 9546 Acked Read 2 Sent Done Done false
33178 9547 Acked Read 10 Sent Done Done false
33179 9548 Acked Read 8 Sent Done Done false
33180 9549 Acked Write 8 Sent Done Done false
33181 9550 Acked Read 9 Sent Done Done false
33182 9551 Acked Read 10 Sent Done Done false
33183 9552 Acked Flush 0 Sent Done Done false
33184 9553 Acked Read 5 Sent Done Done false
33185 9554 Acked Write 8 Sent Done Done false
33186 9555 Acked Flush 0 Sent Done Done false
33187 9556 Acked Read 6 Sent Done Done false
33188 9557 Acked Write 6 Sent Done Done false
33189 9558 Acked Write 5 Sent Done Done false
33190 9559 Acked Write 6 Sent Done Done false
33191 9560 Acked Read 9 Sent Done Done false
33192 9561 Acked Flush 0 Sent Done Done false
33193 9562 Acked Read 4 Sent Done Done false
33194 9563 Acked Flush 0 Sent Done Done false
33195 9564 Acked Read 2 Sent Done Done false
33196 9565 Acked Read 3 Sent Done Done false
33197 9566 Acked Write 6 Sent Done Done false
33198 9567 Acked Read 8 Sent Done Done false
33199 9568 Acked Flush 0 Sent Done Done false
33200 9569 Acked Write 8 Sent Done Done false
33201 9570 Acked Read 1 Sent Done Done false
33202 9571 Acked Read 1 Sent Done Done false
33203 9572 Acked Read 8 Sent Done Done false
33204 9573 Acked Write 2 Sent Done Done false
33205 9574 Acked Write 1 Sent Done Done false
33206 9575 Acked Read 8 Sent Done Done false
33207 9576 Acked Read 9 Sent Done Done false
33208 9577 Acked Read 3 Sent Done Done false
33209 9578 Acked Write 8 Sent Done Done false
33210 9579 Acked Read 2 Sent Done Done false
33211 9580 Acked Read 7 Sent Done Done false
33212 9581 Acked Read 9 Sent Done Done false
33213 9582 Acked Read 4 Sent Done Done false
33214 9583 Acked Read 5 Sent Done Done false
33215 9584 Acked Read 7 Sent Done Done false
33216 9585 Acked Read 2 Sent Done Done false
33217 9586 Acked Flush 0 Sent Done Done false
33218 9587 Acked Flush 0 Sent Done Done false
33219 9588 Acked Read 3 Sent Done Done false
33220 9589 Acked Write 3 Sent Done Done false
33221 9590 Acked Read 7 Sent Done Done false
33222 9591 Acked Write 4 Sent Done Done false
33223 9592 Acked Read 9 Sent Done Done false
33224 9593 Acked Read 1 Sent Done Done false
33225 9594 Acked Write 6 Sent Done Done false
33226 9595 Acked Flush 0 Sent Done Done false
33227 9596 Acked Write 1 Sent Done Done false
33228 9597 Acked Write 10 Sent Done Done false
33229 9598 Acked Read 4 Sent Done Done false
33230 9599 Acked Write 3 Sent Done Done false
33231 9600 Acked Read 1 Sent Done Done false
33232 9601 Acked Read 10 Sent Done Done false
33233 9602 Acked Read 5 Sent Done Done false
33234 9603 Acked Write 9 Sent Done Done false
33235 9604 Acked Read 10 Sent Done Done false
33236 9605 Acked Read 6 Sent Done Done false
33237 9606 Acked Read 6 Sent Done Done false
33238 9607 Acked Read 7 Sent Done Done false
33239 9608 Acked Write 7 Sent Done Done false
33240 9609 Acked Write 6 Sent Done Done false
33241 9610 Acked Write 8 Sent Done Done false
33242 9611 Acked Read 8 Sent Done Done false
33243 9612 Acked Write 7 Sent Done Done false
33244 9613 Acked Flush 0 Sent Done Done false
33245 9614 Acked Read 1 Sent Done Done false
33246 9615 Acked Read 5 Sent Done Done false
33247 9616 Acked Read 8 Sent Done Done false
33248 9617 Acked Write 1 Sent Done Done false
33249 9618 Acked Write 2 Sent Done Done false
33250 9619 Acked Write 4 Sent Done Done false
33251 9620 Acked Read 4 Sent Done Done false
33252 9621 Acked Write 9 Sent Done Done false
33253 9622 Acked Flush 0 Sent Done Done false
33254 9623 Acked Write 10 Sent Done Done false
33255 9624 Acked Read 2 Sent Done Done false
33256 9625 Acked Write 8 Sent Done Done false
33257 9626 Acked Write 2 Sent Done Done false
33258 9627 Acked Read 3 Sent Done Done false
33259 9628 Acked Read 1 Sent Done Done false
33260 9629 Acked Write 8 Sent Done Done false
33261 9630 Acked Read 1 Sent Done Done false
33262 9631 Acked Read 8 Sent Done Done false
33263 9632 Acked Write 3 Sent Done Done false
33264 9633 Acked Write 2 Sent Done Done false
33265 9634 Acked Write 10 Sent Done Done false
33266 9635 Acked Write 4 Sent Done Done false
33267 9636 Acked Read 6 Sent Done Done false
33268 9637 Acked Read 6 Sent Done Done false
33269 9638 Acked Write 9 Sent Done Done false
33270 9639 Acked Read 7 Sent Done Done false
33271 9640 Acked Write 8 Sent Done Done false
33272 9641 Acked Read 3 Sent Done Done false
33273 9642 Acked Read 7 Sent Done Done false
33274 9643 Acked Read 7 Sent Done Done false
33275 9644 Acked Write 2 Sent Done Done false
33276 9645 Acked Read 9 Sent Done Done false
33277 9646 Acked Read 4 Sent Done Done false
33278 9647 Acked Write 6 Sent Done Done false
33279 9648 Acked Flush 0 Sent Done Done false
33280 9649 Acked Read 8 Sent Done Done false
33281 9650 Acked Write 6 Sent Done Done false
33282 9651 Acked Read 9 Sent Done Done false
33283 9652 Acked Write 5 Sent Done Done false
33284 9653 Acked Read 1 Sent Done Done false
33285 9654 Acked Write 9 Sent Done Done false
33286 9655 Acked Write 4 Sent Done Done false
33287 9656 Acked Write 10 Sent Done Done false
33288 9657 Acked Write 6 Sent Done Done false
33289 9658 Acked Read 4 Sent Done Done false
33290 9659 Acked Write 8 Sent Done Done false
33291 9660 Acked Write 9 Sent Done Done false
33292 9661 Acked Read 7 Sent Done Done false
33293 9662 Acked Read 9 Sent Done Done false
33294 9663 Acked Read 5 Sent Done Done false
33295 9664 Acked Read 2 Sent Done Done false
33296 9665 Acked Read 1 Sent Done Done false
33297 9666 Acked Write 9 Sent Done Done false
33298 9667 Acked Read 3 Sent Done Done false
33299 9668 Acked Write 1 Sent Done Done false
33300 9669 Acked Write 8 Sent Done Done false
33301 9670 Acked Write 7 Sent Done Done false
33302 9671 Acked Read 3 Sent Done Done false
33303 9672 Acked Write 9 Sent Done Done false
33304 9673 Acked Read 5 Sent Done Done false
33305 9674 Acked Read 9 Sent Done Done false
33306 9675 Acked Read 9 Sent Done Done false
33307 9676 Acked Write 8 Sent Done Done false
33308 9677 Acked Write 6 Sent Done Done false
33309 9678 Acked Read 9 Sent Done Done false
33310 9679 Acked Write 4 Sent Done Done false
33311 9680 Acked Write 9 Sent Done Done false
33312 9681 Acked Read 6 Sent Done Done false
33313 9682 Acked Write 1 Sent Done Done false
33314 9683 Acked Write 2 Sent Done Done false
33315 9684 Acked Flush 0 Sent Done Done false
33316 9685 Acked Write 4 Sent Done Done false
33317 9686 Acked Write 7 Sent Done Done false
33318 9687 Acked Read 9 Sent Done Done false
33319 9688 Acked Write 5 Sent Done Done false
33320 9689 Acked Write 7 Sent Done Done false
33321 9690 Acked Flush 0 Sent Done Done false
33322 9691 Acked Write 2 Sent Done Done false
33323 9692 Acked Read 8 Sent Done Done false
33324 9693 Acked Flush 0 Sent Done Done false
33325 9694 Acked Read 8 Sent Done Done false
33326 9695 Acked Read 9 Sent Done Done false
33327 9696 Acked Write 1 Sent Done Done false
33328 9697 Acked Read 9 Sent Done Done false
33329 9698 Acked Write 8 Sent Done Done false
33330 9699 Acked Write 3 Sent Done Done false
33331 9700 Acked Read 6 Sent Done Done false
33332 9701 Acked Read 7 Sent Done Done false
33333 9702 Acked Read 2 Sent Done Done false
33334 9703 Acked Read 9 Sent Done Done false
33335 9704 Acked Flush 0 Sent Done Done false
33336 9705 Acked Read 7 Sent Done Done false
33337 9706 Acked Write 6 Sent Done Done false
33338 9707 Acked Flush 0 Sent Done Done false
33339 9708 Acked Flush 0 Sent Done Done false
33340 9709 Acked Write 1 Sent Done Done false
33341 9710 Acked Write 9 Sent Done Done false
33342 9711 Acked Flush 0 Sent Done Done false
33343 9712 Acked Read 9 Sent Done Done false
33344 9713 Acked Flush 0 Sent Done Done false
33345 9714 Acked Write 7 Sent Done Done false
33346 9715 Acked Write 9 Sent Done Done false
33347 9716 Acked Write 3 Sent Done Done false
33348 9717 Acked Read 5 Sent Done Done false
33349 9718 Acked Read 5 Sent Done Done false
33350 9719 Acked Write 6 Sent Done Done false
33351 9720 Acked Read 10 Sent Done Done false
33352 9721 Acked Read 2 Sent Done Done false
33353 9722 Acked Read 6 Sent Done Done false
33354 9723 Acked Read 7 Sent Done Done false
33355 9724 Acked Flush 0 Sent Done Done false
33356 9725 Acked Write 7 Sent Done Done false
33357 9726 Acked Write 4 Sent Done Done false
33358 9727 Acked Write 6 Sent Done Done false
33359 9728 Acked Write 5 Sent Done Done false
33360 9729 Acked Write 9 Sent Done Done false
33361 9730 Acked Read 5 Sent Done Done false
33362 9731 Acked Read 7 Sent Done Done false
33363 9732 Acked Read 1 Sent Done Done false
33364 9733 Acked Write 6 Sent Done Done false
33365 9734 Acked Write 7 Sent Done Done false
33366 9735 Acked Read 6 Sent Done Done false
33367 9736 Acked Write 6 Sent Done Done false
33368 9737 Acked Read 4 Sent Done Done false
33369 9738 Acked Read 9 Sent Done Done false
33370 9739 Acked Write 9 Sent Done Done false
33371 9740 Acked Write 9 Sent Done Done false
33372 9741 Acked Flush 0 Sent Done Done false
33373 9742 Acked Read 9 Sent Done Done false
33374 9743 Acked Write 4 Sent Done Done false
33375 9744 Acked Read 6 Sent Done Done false
33376 9745 Acked Read 10 Sent Done Done false
33377 9746 Acked Flush 0 Sent Done Done false
33378 9747 Acked Read 5 Sent Done Done false
33379 9748 Acked Read 5 Sent Done Done false
33380 9749 Acked Read 10 Sent Done Done false
33381 9750 Acked Flush 0 Sent Done Done false
33382 9751 Acked Read 9 Sent Done Done false
33383 9752 Acked Flush 0 Sent Done Done false
33384 9753 Acked Read 7 Sent Done Done false
33385 STATES DS:0 DS:1 DS:2 TOTAL
33386 Sent 301 0 0 301
33387 Done 0 301 301 602
33388 Skipped 0 0 0 0
33389 Error 0 0 0 0
33390 Last Flush: 9452 9752 9752
33391 Downstairs last five completed: 9452 9451 9450 9449 9448
33392 Upstairs last five completed: 9753 9752 9751 9750 9749
33393 CLIENT: Up:0 ds:301 act:2
33394 [8810] got start action
33395 [8810] I am going to restart
33396 Starting downstairs at port 8810
33397 Make output file at "/tmp/dsc/downstairs-8810.txt"
33398 [8810][0] reports Starting
33399 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1229)
33400 [8810][0] reports Running
33401 ----------------------------------------------------------------
33402 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
33403 States: Offline Active Active
33404 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33405 9453 Acked Write 1 Sent Done Done false
33406 9454 Acked Read 4 Sent Done Done false
33407 9455 Acked Read 10 Sent Done Done false
33408 9456 Acked Read 6 Sent Done Done false
33409 9457 Acked Write 3 Sent Done Done false
33410 9458 Acked Write 1 Sent Done Done false
33411 9459 Acked Write 1 Sent Done Done false
33412 9460 Acked Write 8 Sent Done Done false
33413 9461 Acked Read 5 Sent Done Done false
33414 9462 Acked Read 3 Sent Done Done false
33415 9463 Acked Flush 0 Sent Done Done false
33416 9464 Acked Write 10 Sent Done Done false
33417 9465 Acked Read 8 Sent Done Done false
33418 9466 Acked Read 8 Sent Done Done false
33419 9467 Acked Read 5 Sent Done Done false
33420 9468 Acked Read 7 Sent Done Done false
33421 9469 Acked Read 8 Sent Done Done false
33422 9470 Acked Write 7 Sent Done Done false
33423 9471 Acked Write 2 Sent Done Done false
33424 9472 Acked Read 5 Sent Done Done false
33425 9473 Acked Read 1 Sent Done Done false
33426 9474 Acked Flush 0 Sent Done Done false
33427 9475 Acked Flush 0 Sent Done Done false
33428 9476 Acked Read 4 Sent Done Done false
33429 9477 Acked Read 1 Sent Done Done false
33430 9478 Acked Write 6 Sent Done Done false
33431 9479 Acked Read 6 Sent Done Done false
33432 9480 Acked Read 9 Sent Done Done false
33433 9481 Acked Write 4 Sent Done Done false
33434 9482 Acked Read 2 Sent Done Done false
33435 9483 Acked Flush 0 Sent Done Done false
33436 9484 Acked Read 3 Sent Done Done false
33437 9485 Acked Write 10 Sent Done Done false
33438 9486 Acked Read 5 Sent Done Done false
33439 9487 Acked Write 10 Sent Done Done false
33440 9488 Acked Read 10 Sent Done Done false
33441 9489 Acked Read 5 Sent Done Done false
33442 9490 Acked Read 6 Sent Done Done false
33443 9491 Acked Write 9 Sent Done Done false
33444 9492 Acked Read 9 Sent Done Done false
33445 9493 Acked Read 7 Sent Done Done false
33446 9494 Acked Read 4 Sent Done Done false
33447 9495 Acked Flush 0 Sent Done Done false
33448 9496 Acked Write 10 Sent Done Done false
33449 9497 Acked Flush 0 Sent Done Done false
33450 9498 Acked Write 2 Sent Done Done false
33451 9499 Acked Flush 0 Sent Done Done false
33452 9500 Acked Write 1 Sent Done Done false
33453 9501 Acked Read 5 Sent Done Done false
33454 9502 Acked Read 5 Sent Done Done false
33455 9503 Acked Read 5 Sent Done Done false
33456 9504 Acked Flush 0 Sent Done Done false
33457 9505 Acked Write 5 Sent Done Done false
33458 9506 Acked Read 8 Sent Done Done false
33459 9507 Acked Write 9 Sent Done Done false
33460 9508 Acked Read 6 Sent Done Done false
33461 9509 Acked Write 9 Sent Done Done false
33462 9510 Acked Flush 0 Sent Done Done false
33463 9511 Acked Read 8 Sent Done Done false
33464 9512 Acked Read 9 Sent Done Done false
33465 9513 Acked Read 9 Sent Done Done false
33466 9514 Acked Read 3 Sent Done Done false
33467 9515 Acked Read 9 Sent Done Done false
33468 9516 Acked Read 2 Sent Done Done false
33469 9517 Acked Write 7 Sent Done Done false
33470 9518 Acked Write 1 Sent Done Done false
33471 9519 Acked Flush 0 Sent Done Done false
33472 9520 Acked Write 9 Sent Done Done false
33473 9521 Acked Write 9 Sent Done Done false
33474 9522 Acked Read 10 Sent Done Done false
33475 9523 Acked Read 2 Sent Done Done false
33476 9524 Acked Read 7 Sent Done Done false
33477 9525 Acked Write 1 Sent Done Done false
33478 9526 Acked Write 5 Sent Done Done false
33479 9527 Acked Flush 0 Sent Done Done false
33480 9528 Acked Read 1 Sent Done Done false
33481 9529 Acked Read 3 Sent Done Done false
33482 9530 Acked Write 3 Sent Done Done false
33483 9531 Acked Write 8 Sent Done Done false
33484 9532 Acked Write 10 Sent Done Done false
33485 9533 Acked Read 8 Sent Done Done false
33486 9534 Acked Write 7 Sent Done Done false
33487 9535 Acked Write 2 Sent Done Done false
33488 9536 Acked Flush 0 Sent Done Done false
33489 9537 Acked Read 10 Sent Done Done false
33490 9538 Acked Read 4 Sent Done Done false
33491 9539 Acked Write 8 Sent Done Done false
33492 9540 Acked Read 7 Sent Done Done false
33493 9541 Acked Read 8 Sent Done Done false
33494 9542 Acked Read 7 Sent Done Done false
33495 9543 Acked Read 1 Sent Done Done false
33496 9544 Acked Write 8 Sent Done Done false
33497 9545 Acked Write 9 Sent Done Done false
33498 9546 Acked Read 2 Sent Done Done false
33499 9547 Acked Read 10 Sent Done Done false
33500 9548 Acked Read 8 Sent Done Done false
33501 9549 Acked Write 8 Sent Done Done false
33502 9550 Acked Read 9 Sent Done Done false
33503 9551 Acked Read 10 Sent Done Done false
33504 9552 Acked Flush 0 Sent Done Done false
33505 9553 Acked Read 5 Sent Done Done false
33506 9554 Acked Write 8 Sent Done Done false
33507 9555 Acked Flush 0 Sent Done Done false
33508 9556 Acked Read 6 Sent Done Done false
33509 9557 Acked Write 6 Sent Done Done false
33510 9558 Acked Write 5 Sent Done Done false
33511 9559 Acked Write 6 Sent Done Done false
33512 9560 Acked Read 9 Sent Done Done false
33513 9561 Acked Flush 0 Sent Done Done false
33514 9562 Acked Read 4 Sent Done Done false
33515 9563 Acked Flush 0 Sent Done Done false
33516 9564 Acked Read 2 Sent Done Done false
33517 9565 Acked Read 3 Sent Done Done false
33518 9566 Acked Write 6 Sent Done Done false
33519 9567 Acked Read 8 Sent Done Done false
33520 9568 Acked Flush 0 Sent Done Done false
33521 9569 Acked Write 8 Sent Done Done false
33522 9570 Acked Read 1 Sent Done Done false
33523 9571 Acked Read 1 Sent Done Done false
33524 9572 Acked Read 8 Sent Done Done false
33525 9573 Acked Write 2 Sent Done Done false
33526 9574 Acked Write 1 Sent Done Done false
33527 9575 Acked Read 8 Sent Done Done false
33528 9576 Acked Read 9 Sent Done Done false
33529 9577 Acked Read 3 Sent Done Done false
33530 9578 Acked Write 8 Sent Done Done false
33531 9579 Acked Read 2 Sent Done Done false
33532 9580 Acked Read 7 Sent Done Done false
33533 9581 Acked Read 9 Sent Done Done false
33534 9582 Acked Read 4 Sent Done Done false
33535 9583 Acked Read 5 Sent Done Done false
33536 9584 Acked Read 7 Sent Done Done false
33537 9585 Acked Read 2 Sent Done Done false
33538 9586 Acked Flush 0 Sent Done Done false
33539 9587 Acked Flush 0 Sent Done Done false
33540 9588 Acked Read 3 Sent Done Done false
33541 9589 Acked Write 3 Sent Done Done false
33542 9590 Acked Read 7 Sent Done Done false
33543 9591 Acked Write 4 Sent Done Done false
33544 9592 Acked Read 9 Sent Done Done false
33545 9593 Acked Read 1 Sent Done Done false
33546 9594 Acked Write 6 Sent Done Done false
33547 9595 Acked Flush 0 Sent Done Done false
33548 9596 Acked Write 1 Sent Done Done false
33549 9597 Acked Write 10 Sent Done Done false
33550 9598 Acked Read 4 Sent Done Done false
33551 9599 Acked Write 3 Sent Done Done false
33552 9600 Acked Read 1 Sent Done Done false
33553 9601 Acked Read 10 Sent Done Done false
33554 9602 Acked Read 5 Sent Done Done false
33555 9603 Acked Write 9 Sent Done Done false
33556 9604 Acked Read 10 Sent Done Done false
33557 9605 Acked Read 6 Sent Done Done false
33558 9606 Acked Read 6 Sent Done Done false
33559 9607 Acked Read 7 Sent Done Done false
33560 9608 Acked Write 7 Sent Done Done false
33561 9609 Acked Write 6 Sent Done Done false
33562 9610 Acked Write 8 Sent Done Done false
33563 9611 Acked Read 8 Sent Done Done false
33564 9612 Acked Write 7 Sent Done Done false
33565 9613 Acked Flush 0 Sent Done Done false
33566 9614 Acked Read 1 Sent Done Done false
33567 9615 Acked Read 5 Sent Done Done false
33568 9616 Acked Read 8 Sent Done Done false
33569 9617 Acked Write 1 Sent Done Done false
33570 9618 Acked Write 2 Sent Done Done false
33571 9619 Acked Write 4 Sent Done Done false
33572 9620 Acked Read 4 Sent Done Done false
33573 9621 Acked Write 9 Sent Done Done false
33574 9622 Acked Flush 0 Sent Done Done false
33575 9623 Acked Write 10 Sent Done Done false
33576 9624 Acked Read 2 Sent Done Done false
33577 9625 Acked Write 8 Sent Done Done false
33578 9626 Acked Write 2 Sent Done Done false
33579 9627 Acked Read 3 Sent Done Done false
33580 9628 Acked Read 1 Sent Done Done false
33581 9629 Acked Write 8 Sent Done Done false
33582 9630 Acked Read 1 Sent Done Done false
33583 9631 Acked Read 8 Sent Done Done false
33584 9632 Acked Write 3 Sent Done Done false
33585 9633 Acked Write 2 Sent Done Done false
33586 9634 Acked Write 10 Sent Done Done false
33587 9635 Acked Write 4 Sent Done Done false
33588 9636 Acked Read 6 Sent Done Done false
33589 9637 Acked Read 6 Sent Done Done false
33590 9638 Acked Write 9 Sent Done Done false
33591 9639 Acked Read 7 Sent Done Done false
33592 9640 Acked Write 8 Sent Done Done false
33593 9641 Acked Read 3 Sent Done Done false
33594 9642 Acked Read 7 Sent Done Done false
33595 9643 Acked Read 7 Sent Done Done false
33596 9644 Acked Write 2 Sent Done Done false
33597 9645 Acked Read 9 Sent Done Done false
33598 9646 Acked Read 4 Sent Done Done false
33599 9647 Acked Write 6 Sent Done Done false
33600 9648 Acked Flush 0 Sent Done Done false
33601 9649 Acked Read 8 Sent Done Done false
33602 9650 Acked Write 6 Sent Done Done false
33603 9651 Acked Read 9 Sent Done Done false
33604 9652 Acked Write 5 Sent Done Done false
33605 9653 Acked Read 1 Sent Done Done false
33606 9654 Acked Write 9 Sent Done Done false
33607 9655 Acked Write 4 Sent Done Done false
33608 9656 Acked Write 10 Sent Done Done false
33609 9657 Acked Write 6 Sent Done Done false
33610 9658 Acked Read 4 Sent Done Done false
33611 9659 Acked Write 8 Sent Done Done false
33612 9660 Acked Write 9 Sent Done Done false
33613 9661 Acked Read 7 Sent Done Done false
33614 9662 Acked Read 9 Sent Done Done false
33615 9663 Acked Read 5 Sent Done Done false
33616 9664 Acked Read 2 Sent Done Done false
33617 9665 Acked Read 1 Sent Done Done false
33618 9666 Acked Write 9 Sent Done Done false
33619 9667 Acked Read 3 Sent Done Done false
33620 9668 Acked Write 1 Sent Done Done false
33621 9669 Acked Write 8 Sent Done Done false
33622 9670 Acked Write 7 Sent Done Done false
33623 9671 Acked Read 3 Sent Done Done false
33624 9672 Acked Write 9 Sent Done Done false
33625 9673 Acked Read 5 Sent Done Done false
33626 9674 Acked Read 9 Sent Done Done false
33627 9675 Acked Read 9 Sent Done Done false
33628 9676 Acked Write 8 Sent Done Done false
33629 9677 Acked Write 6 Sent Done Done false
33630 9678 Acked Read 9 Sent Done Done false
33631 9679 Acked Write 4 Sent Done Done false
33632 9680 Acked Write 9 Sent Done Done false
33633 9681 Acked Read 6 Sent Done Done false
33634 9682 Acked Write 1 Sent Done Done false
33635 9683 Acked Write 2 Sent Done Done false
33636 9684 Acked Flush 0 Sent Done Done false
33637 9685 Acked Write 4 Sent Done Done false
33638 9686 Acked Write 7 Sent Done Done false
33639 9687 Acked Read 9 Sent Done Done false
33640 9688 Acked Write 5 Sent Done Done false
33641 9689 Acked Write 7 Sent Done Done false
33642 9690 Acked Flush 0 Sent Done Done false
33643 9691 Acked Write 2 Sent Done Done false
33644 9692 Acked Read 8 Sent Done Done false
33645 9693 Acked Flush 0 Sent Done Done false
33646 9694 Acked Read 8 Sent Done Done false
33647 9695 Acked Read 9 Sent Done Done false
33648 9696 Acked Write 1 Sent Done Done false
33649 9697 Acked Read 9 Sent Done Done false
33650 9698 Acked Write 8 Sent Done Done false
33651 9699 Acked Write 3 Sent Done Done false
33652 9700 Acked Read 6 Sent Done Done false
33653 9701 Acked Read 7 Sent Done Done false
33654 9702 Acked Read 2 Sent Done Done false
33655 9703 Acked Read 9 Sent Done Done false
33656 9704 Acked Flush 0 Sent Done Done false
33657 9705 Acked Read 7 Sent Done Done false
33658 9706 Acked Write 6 Sent Done Done false
33659 9707 Acked Flush 0 Sent Done Done false
33660 9708 Acked Flush 0 Sent Done Done false
33661 9709 Acked Write 1 Sent Done Done false
33662 9710 Acked Write 9 Sent Done Done false
33663 9711 Acked Flush 0 Sent Done Done false
33664 9712 Acked Read 9 Sent Done Done false
33665 9713 Acked Flush 0 Sent Done Done false
33666 9714 Acked Write 7 Sent Done Done false
33667 9715 Acked Write 9 Sent Done Done false
33668 9716 Acked Write 3 Sent Done Done false
33669 9717 Acked Read 5 Sent Done Done false
33670 9718 Acked Read 5 Sent Done Done false
33671 9719 Acked Write 6 Sent Done Done false
33672 9720 Acked Read 10 Sent Done Done false
33673 9721 Acked Read 2 Sent Done Done false
33674 9722 Acked Read 6 Sent Done Done false
33675 9723 Acked Read 7 Sent Done Done false
33676 9724 Acked Flush 0 Sent Done Done false
33677 9725 Acked Write 7 Sent Done Done false
33678 9726 Acked Write 4 Sent Done Done false
33679 9727 Acked Write 6 Sent Done Done false
33680 9728 Acked Write 5 Sent Done Done false
33681 9729 Acked Write 9 Sent Done Done false
33682 9730 Acked Read 5 Sent Done Done false
33683 9731 Acked Read 7 Sent Done Done false
33684 9732 Acked Read 1 Sent Done Done false
33685 9733 Acked Write 6 Sent Done Done false
33686 9734 Acked Write 7 Sent Done Done false
33687 9735 Acked Read 6 Sent Done Done false
33688 9736 Acked Write 6 Sent Done Done false
33689 9737 Acked Read 4 Sent Done Done false
33690 9738 Acked Read 9 Sent Done Done false
33691 9739 Acked Write 9 Sent Done Done false
33692 9740 Acked Write 9 Sent Done Done false
33693 9741 Acked Flush 0 Sent Done Done false
33694 9742 Acked Read 9 Sent Done Done false
33695 9743 Acked Write 4 Sent Done Done false
33696 9744 Acked Read 6 Sent Done Done false
33697 9745 Acked Read 10 Sent Done Done false
33698 9746 Acked Flush 0 Sent Done Done false
33699 9747 Acked Read 5 Sent Done Done false
33700 9748 Acked Read 5 Sent Done Done false
33701 9749 Acked Read 10 Sent Done Done false
33702 9750 Acked Flush 0 Sent Done Done false
33703 9751 Acked Read 9 Sent Done Done false
33704 9752 Acked Flush 0 Sent Done Done false
33705 9753 Acked Read 7 Sent Done Done false
33706 9754 Acked Flush 0 Sent Done Done false
33707 STATES DS:0 DS:1 DS:2 TOTAL
33708 Sent 302 0 0 302
33709 Done 0 302 302 604
33710 Skipped 0 0 0 0
33711 Error 0 0 0 0
33712 Last Flush: 9452 9754 9754
33713 Downstairs last five completed: 9452 9451 9450 9449 9448
33714 Upstairs last five completed: 9754 9753 9752 9751 9750
33715 CLIENT: Up:0 ds:302 act:2
337162025-01-17T21:18:09.389ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
337172025-01-17T21:18:09.389ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:43677) = io task client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
337182025-01-17T21:18:09.391ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
337192025-01-17T21:18:09.391ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
337202025-01-17T21:18:09.391ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
337212025-01-17T21:18:09.391ZINFOcrucible: send last flush ID to this DS: 9452 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
337222025-01-17T21:18:09.391ZINFOcrucible: Replied this last flush ID: 9452 = downstairs client = 0 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
337232025-01-17T21:18:09.391ZINFOcrucible: [0] client re-new 302 jobs since flush 9452 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
337242025-01-17T21:18:09.392ZINFOcrucible: [0] Replayed 302 jobs since flush: 9452 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
33725 ----------------------------------------------------------------
33726 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
33727 Downstairs last five completed: 9754 9753 9752 9751 9750
33728 Upstairs last five completed: 9754 9753 9752 9751 9750
33729 CLIENT: Up:0 ds:0 act:3
33730 Replay: All jobs finished
33731 Main task has work to do, go find it
33732 got dsc Stop(2)
33733 stop 2
33734 [8830] Got stop action so:false kr:false
33735 Jan 17 21:18:11.647 INFO request completed, latency_us: 333, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 557fb14f-eb7f-4ddd-afa3-bbfad65cfe5d, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
33736 Jan 17 21:18:11.648 INFO request completed, latency_us: 179, response_code: 200, uri: /state/cid/2, method: {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:18:11.648330775Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
33737 GET, req_id: c631651c-298b-464f-b147-7d708c2139cb, remote_addr: 127.0.0.1:57191{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-17T21:18:11.648421823Z","hostname":"w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"64f44984-0f34-4cfb-a401-a0494f2ea105"}
33738 , local_addr: 127.0.0.1:9998
337392025-01-17T21:18:11.648ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
337402025-01-17T21:18:11.648ZINFOcrucible: client task is exiting = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
33741 [8830] Exited with: ExitStatus(unix_wait_status(9))
33742 [8830][2] reports Exit
33743 Jan 17 21:18:15.650 INFO request completed, latency_us: 200, response_code: 200, uri: /state/cid/2, method: GET, req_id: 44804c80-303b-427d-8453-8caf6962b4bb, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
33744 001/300 Write block 459 len 40960 data: 3 4 3 5 5 4 4 4 4 5
33745 002/300 Read block 2127 len 24576
33746 003/300 Read block 1421 len 24576
33747 004/300 Write block 270 len 8192 data: 8 7
33748 005/300 Write block 3696 len 8192 data: 5 5
33749 006/300 Read block 2850 len 8192
33750 007/300 Write block 1867 len 24576 data: 5 4 4 3 4 4
33751 008/300 Read block 4050 len 28672
33752 009/300 Read block 1709 len 28672
33753 010/300 Read block 4897 len 40960
33754 011/300 Write block 4562 len 4096 data: 4
33755 012/300 Write block 1752 len 16384 data: 7 8 7 5
33756 013/300 Read block 4683 len 40960
33757 014/300 Read block 835 len 16384
33758 015/300 Read block 1904 len 12288
33759 016/300 Flush
33760 017/300 Write block 3997 len 4096 data: 12
33761 018/300 Write block 2777 len 24576 data: 5 6 4 4 4 3
33762 019/300 Write block 4575 len 32768 data: 6 8 7 7 8 7 7 5
33763 020/300 Read block 3065 len 20480
33764 021/300 Read block 1471 len 8192
33765 022/300 Read block 1307 len 20480
33766 023/300 Read block 2571 len 32768
33767 024/300 Write block 1554 len 4096 data: 6
33768 025/300 Flush
33769 026/300 Read block 20 len 36864
33770 027/300 Write block 1445 len 4096 data: 7
33771 028/300 Write block 1440 len 28672 data: 11 11 12 10 7 8 5
33772 029/300 Read block 4787 len 36864
33773 030/300 Read block 3561 len 12288
33774 031/300 Write block 4056 len 32768 data: 7 8 8 7 8 4 4 5
33775 032/300 Read block 2571 len 4096
33776 033/300 Write block 3994 len 12288 data: 10 12 12
33777 034/300 Write block 769 len 24576 data: 4 5 4 4 3 3
33778 035/300 Read block 836 len 16384
33779 036/300 Read block 1408 len 28672
33780 037/300 Read block 3955 len 16384
33781 038/300 Read block 1515 len 8192
33782 039/300 Read block 3492 len 24576
33783 040/300 Read block 4748 len 4096
33784 041/300 Write block 2638 len 36864 data: 5 5 4 4 6 8 9 10 8
33785 042/300 Flush
33786 043/300 Read block 2714 len 32768
33787 044/300 Read block 3734 len 24576
33788 045/300 Read block 2601 len 12288
33789 046/300 Read block 956 len 24576
33790 047/300 Write block 2210 len 24576 data: 5 5 4 4 4 4
33791 048/300 Write block 3466 len 36864 data: 5 6 6 4 5 7 7 8 8
33792 049/300 Read block 2166 len 16384
33793 050/300 Flush
33794 051/300 Read block 4444 len 28672
33795 052/300 Write block 149 len 36864 data: 7 6 6 7 8 8 7 6 6
33796 053/300 Read block 2411 len 12288
33797 054/300 Flush
33798 055/300 Write block 4351 len 4096 data: 7
33799 056/300 Write block 750 len 28672 data: 6 5 6 5 4 6 6
33800 057/300 Write block 966 len 20480 data: 7 7 7 8 7
33801 058/300 Write block 423 len 24576 data: 8 8 5 6 5 5
33802 059/300 Read block 3629 len 36864
33803 060/300 Write block 2049 len 16384 data: 6 8 7 4
33804 061/300 Read block 3923 len 8192
33805 062/300 Flush
33806 063/300 Read block 3162 len 40960
33807 064/300 Write block 1386 len 28672 data: 7 7 6 5 5 5 8
33808 065/300 Read block 181 len 4096
33809 066/300 Read block 2793 len 16384
33810 067/300 Read block 3428 len 20480
33811 068/300 Write block 3304 len 40960 data: 7 5 4 4 5 5 5 4 5 7
33812 069/300 Flush
33813 070/300 Read block 4726 len 4096
33814 071/300 Flush
33815 072/300 Flush
33816 073/300 Flush
33817 074/300 Write block 2696 len 36864 data: 3 2 2 4 3 3 2 2 4
33818 075/300 Write block 701 len 28672 data: 9 9 10 8 6 7 8
33819 076/300 Write block 3799 len 36864 data: 6 7 7 6 6 6 6 6 7
33820 077/300 Read block 1825 len 4096
33821 078/300 Read block 1851 len 12288
33822 079/300 Read block 2271 len 24576
33823 080/300 Read block 3741 len 36864
33824 081/300 Read block 1678 len 8192
33825 082/300 Read block 3909 len 8192
33826 083/300 Read block 4135 len 40960
33827 084/300 Read block 4942 len 12288
33828 085/300 Write block 4443 len 32768 data: 4 4 4 3 3 4 5 6
33829 086/300 Read block 2665 len 24576
33830 087/300 Read block 3553 len 32768
33831 088/300 Read block 2137 len 36864
33832 089/300 Read block 4002 len 40960
33833 090/300 Read block 4346 len 36864
33834 091/300 Write block 3624 len 28672 data: 8 7 4 5 4 5 4
33835 092/300 Write block 2923 len 16384 data: 5 5 6 7
33836 093/300 Read block 1794 len 28672
33837 094/300 Write block 536 len 20480 data: 10 8 8 6 7
33838 095/300 Write block 4630 len 36864 data: 3 3 3 3 3 4 5 6 6
33839 096/300 Write block 141 len 8192 data: 4 5
33840 097/300 Read block 4046 len 32768
33841 098/300 Read block 2568 len 12288
33842 099/300 Flush
33843 100/300 Write block 1367 len 24576 data: 5 6 7 7 8 6
33844 101/300 Write block 2607 len 4096 data: 5
33845 102/300 Read block 3217 len 40960
33846 103/300 Write block 773 len 4096 data: 4
33847 104/300 Write block 2521 len 24576 data: 3 2 2 4 4 5
33848 105/300 Write block 3542 len 40960 data: 5 5 5 4 2 3 3 3 3 3
33849 106/300 Read block 4451 len 16384
33850 107/300 Write block 4574 len 16384 data: 5 7 9 8
33851 108/300 Write block 3065 len 20480 data: 4 3 3 4 4
33852 109/300 Write block 2627 len 4096 data: 8
33853 110/300 Write block 95 len 8192 data: 5 5
33854 111/300 Write block 1575 len 40960 data: 5 4 4 5 6 6 7 12 11 13
33855 112/300 Read block 4756 len 12288
33856 113/300 Read block 871 len 8192
33857 114/300 Write block 1136 len 40960 data: 6 7 7 6 5 6 6 6 7 6
33858 115/300 Write block 3261 len 28672 data: 3 5 5 5 4 3 5
33859 116/300 Write block 663 len 20480 data: 4 5 7 7 7
33860 117/300 Flush
33861 118/300 Write block 4322 len 16384 data: 8 9 7 5
33862 119/300 Read block 4346 len 16384
33863 120/300 Write block 721 len 12288 data: 5 5 6
33864 121/300 Write block 3525 len 12288 data: 4 5 6
33865 122/300 Write block 1577 len 4096 data: 5
33866 123/300 Write block 1750 len 16384 data: 7 7 8 9
33867 124/300 Read block 693 len 8192
33868 125/300 Flush
33869 126/300 Read block 2393 len 8192
33870 127/300 Read block 1192 len 24576
33871 128/300 Read block 3382 len 12288
33872 129/300 Write block 598 len 8192 data: 5 4
33873 130/300 Read block 3506 len 40960
33874 131/300 Read block 3156 len 16384
33875 132/300 Flush
33876 133/300 Read block 1732 len 12288
33877 134/300 Read block 1122 len 28672
33878 135/300 Write block 958 len 32768 data: 7 6 7 7 8 6 7 8
33879 136/300 Read block 774 len 36864
33880 137/300 Flush
33881 138/300 Read block 1768 len 36864
33882 139/300 Write block 4119 len 8192 data: 7 6
33883 140/300 Read block 4371 len 12288
33884 141/300 Read block 1241 len 24576
33885 142/300 Read block 3997 len 40960
33886 143/300 Flush
33887 144/300 Read block 4322 len 32768
33888 145/300 Flush
33889 146/300 Read block 1862 len 4096
33890 147/300 Read block 1733 len 20480
33891 148/300 Read block 3475 len 12288
33892 149/300 Read block 290 len 24576
33893 150/300 Read block 847 len 4096
33894 151/300 Read block 2778 len 40960
33895 152/300 Write block 4442 len 20480 data: 4 5 5 5 4
33896 153/300 Read block 2474 len 28672
33897 154/300 Write block 971 len 24576 data: 8 8 7 5 6 7
33898 155/300 Read block 3253 len 40960
33899 156/300 Read block 4632 len 32768
33900 157/300 Read block 4148 len 12288
33901 158/300 Read block 4798 len 28672
33902 159/300 Read block 4094 len 20480
33903 160/300 Write block 4731 len 12288 data: 6 7 8
33904 161/300 Write block 3513 len 12288 data: 6 6 7
33905 162/300 Flush
33906 163/300 Read block 994 len 8192
33907 164/300 Write block 3805 len 36864 data: 7 7 8 6 6 8 9 9 8
33908 165/300 Write block 2418 len 20480 data: 6 7 7 7 6
33909 166/300 Read block 3175 len 12288
33910 167/300 Write block 998 len 32768 data: 6 6 5 5 6 5 6 7
33911 168/300 Read block 4117 len 36864
33912 169/300 Write block 4954 len 28672 data: 4 4 5 5 7 7 7
33913 170/300 Read block 3592 len 16384
33914 171/300 Read block 1290 len 4096
33915 172/300 Read block 512 len 16384
33916 173/300 Write block 3840 len 12288 data: 8 9 11
33917 174/300 Write block 2300 len 20480 data: 5 4 3 2 2
33918 175/300 Read block 1813 len 40960
33919 176/300 Read block 518 len 40960
33920 177/300 Write block 4666 len 32768 data: 7 6 5 5 4 5 5 6
33921 178/300 Write block 137 len 40960 data: 4 6 6 6 5 6 4 5 6 6
33922 179/300 Read block 2345 len 32768
33923 180/300 Flush
33924 181/300 Write block 579 len 12288 data: 5 6 6
33925 182/300 Flush
33926 183/300 Read block 2714 len 4096
33927 184/300 Write block 72 len 32768 data: 10 9 9 10 11 9 8 5
33928 185/300 Read block 1664 len 4096
33929 186/300 Read block 1916 len 16384
33930 187/300 Write block 3202 len 24576 data: 5 8 7 9 7 7
33931 188/300 Read block 2338 len 28672
33932 189/300 Read block 205 len 28672
33933 190/300 Write block 4925 len 40960 data: 6 6 6 5 5 5 5 6 5 6
33934 191/300 Write block 3343 len 24576 data: 4 4 4 5 4 6
33935 192/300 Flush
33936 193/300 Read block 1859 len 24576
33937 194/300 Read block 916 len 16384
33938 195/300 Read block 2950 len 20480
33939 196/300 Read block 1549 len 12288
33940 197/300 Write block 3721 len 16384 data: 6 7 7 9
33941 198/300 Write block 1140 len 28672 data: 6 7 7 7 8 7 8
33942 199/300 Write block 4660 len 20480 data: 5 4 6 6 6
33943 200/300 Read block 2011 len 8192
33944 201/300 Read block 3630 len 4096
33945 202/300 Read block 2105 len 16384
33946 203/300 Write block 1224 len 40960 data: 5 6 5 6 7 7 9 9 7 6
33947 204/300 Read block 4072 len 20480
33948 205/300 Write block 4060 len 8192 data: 9 5
33949 206/300 Write block 1954 len 32768 data: 9 10 8 8 7 8 8 6
33950 207/300 Write block 1221 len 8192 data: 4 4
33951 208/300 Flush
33952 209/300 Read block 1575 len 12288
33953 210/300 Write block 3926 len 16384 data: 4 4 4 6
33954 211/300 Read block 2638 len 36864
33955 212/300 Write block 3861 len 4096 data: 3
33956 213/300 Write block 1405 len 40960 data: 2 2 3 3 3 3 3 3 3 3
33957 214/300 Write block 4597 len 24576 data: 6 6 7 7 6 5
33958 215/300 Write block 119 len 24576 data: 6 7 7 8 6 5
33959 216/300 Write block 4888 len 16384 data: 6 6 7 8
33960 217/300 Read block 3656 len 4096
33961 218/300 Read block 384 len 20480
33962 219/300 Write block 4233 len 24576 data: 6 5 7 6 7 5
33963 220/300 Write block 2115 len 16384 data: 6 6 7 7
33964 221/300 Write block 333 len 4096 data: 3
33965 222/300 Read block 2404 len 40960
33966 223/300 Write block 4199 len 40960 data: 4 4 4 5 5 5 5 7 7 8
33967 224/300 Read block 4370 len 28672
33968 225/300 Flush
33969 226/300 Read block 4135 len 40960
33970 227/300 Read block 681 len 16384
33971 228/300 Read block 151 len 12288
33972 229/300 Write block 2712 len 4096 data: 5
33973 230/300 Read block 3534 len 8192
33974 231/300 Flush
33975 232/300 Write block 3967 len 8192 data: 5 4
33976 233/300 Write block 2487 len 36864 data: 6 7 8 9 10 9 8 9 9
33977 234/300 Read block 2186 len 4096
33978 235/300 Write block 396 len 16384 data: 5 6 9 9
33979 236/300 Read block 1160 len 40960
33980 237/300 Read block 874 len 20480
33981 238/300 Read block 4943 len 12288
33982 239/300 Read block 435 len 36864
33983 240/300 Flush
33984 241/300 Write block 3797 len 8192 data: 5 6
33985 242/300 Write block 1413 len 12288 data: 4 4 5
33986 243/300 Write block 3440 len 4096 data: 5
33987 244/300 Write block 4512 len 28672 data: 4 4 3 3 4 5 5
33988 245/300 Write block 4135 len 24576 data: 6 5 7 7 9 8
33989 246/300 Read block 2820 len 24576
33990 247/300 Read block 1925 len 36864
33991 248/300 Read block 2088 len 8192
33992 249/300 Write block 3401 len 4096 data: 4
33993 250/300 Read block 280 len 4096
33994 251/300 Write block 1368 len 28672 data: 7 8 8 9 7 5 5
33995 252/300 Write block 4179 len 4096 data: 9
33996 253/300 Write block 4742 len 4096 data: 4
33997 254/300 Read block 355 len 28672
33998 255/300 Read block 4716 len 24576
33999 256/300 Read block 3834 len 4096
34000 257/300 Write block 3794 len 32768 data: 6 6 6 6 7 7 8 8
34001 258/300 Read block 925 len 24576
34002 259/300 Write block 3919 len 12288 data: 5 5 6
34003 260/300 Flush
34004 261/300 Read block 4667 len 4096
34005 262/300 Write block 2850 len 8192 data: 5 5
34006 263/300 Flush
34007 264/300 Write block 2813 len 24576 data: 4 6 6 6 5 5
34008 265/300 Read block 2005 len 4096
34009 266/300 Write block 453 len 12288 data: 2 3 3
34010 267/300 Write block 3756 len 16384 data: 8 8 6 6
34011 268/300 Read block 3212 len 20480
34012 269/300 Read block 881 len 40960
34013 270/300 Write block 3896 len 24576 data: 7 5 4 4 3 4
34014 271/300 Flush
34015 272/300 Write block 3862 len 40960 data: 3 4 4 4 4 4 5 6 6 6
34016 273/300 Write block 4607 len 12288 data: 6 7 7
34017 274/300 Write block 3807 len 28672 data: 9 7 7 9 10 10 9
34018 275/300 Write block 2161 len 16384 data: 7 7 6 7
34019 276/300 Read block 1068 len 4096
34020 277/300 Read block 3838 len 4096
34021 278/300 Read block 3705 len 12288
34022 279/300 Write block 4056 len 8192 data: 8 9
34023 280/300 Write block 2686 len 28672 data: 10 8 8 6 5 5 5
34024 281/300 Write block 3322 len 28672 data: 5 4 5 3 3 4 6
34025 282/300 Read block 1539 len 20480
34026 283/300 Write block 2982 len 4096 data: 7
34027 284/300 Read block 4118 len 36864
34028 285/300 Flush
34029 286/300 Read block 3767 len 24576
34030 287/300 Write block 580 len 16384 data: 7 7 4 4
34031 288/300 Write block 41 len 20480 data: 7 6 6 5 5
34032 289/300 Read block 4686 len 16384
34033 290/300 Write block 1040 len 12288 data: 3 3 2
34034 291/300 Read block 3302 len 12288
34035 292/300 Write block 81 len 24576 data: 6 5 4 4 4 6
34036 293/300 Read block 3902 len 28672
34037 294/300 Read block 435 len 8192
34038 295/300 Write block 2940 len 36864 data: 6 5 6 9 11 9 7 6 7
34039 296/300 Write block 880 len 20480 data: 9 9 6 5 7
34040 297/300 Flush
34041 298/300 Write block 2067 len 20480 data: 13 13 13 12 9
34042 299/300 Write block 2170 len 12288 data: 9 7 7
34043 300/300 Write block 859 len 16384 data: 5 7 8 10
34044 301/300 Flush
34045 Main task has work to do, go find it
34046 got dsc Start(2)
34047 start 2
34048 Jan 17 21:18:15.879 INFO request completed, latency_us: 223, response_code: 204, uri: /start/cid/2, method: POST, req_id: b66e03d6-5057-4655-8eb1-9ff37416570e, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
34049 Replay: started 2, returned:Ok(())
34050 ----------------------------------------------------------------
34051 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
34052 States: Active Active Offline
34053 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34054 9755 Acked Write 10 Done Done Sent false
34055 9756 Acked Read 6 Done Done Sent false
34056 9757 Acked Read 6 Done Done Sent false
34057 9758 Acked Write 2 Done Done Sent false
34058 9759 Acked Write 2 Done Done Sent false
34059 9760 Acked Read 2 Done Done Sent false
34060 9761 Acked Write 6 Done Done Sent false
34061 9762 Acked Read 7 Done Done Sent false
34062 9763 Acked Read 7 Done Done Sent false
34063 9764 Acked Read 10 Done Done Sent false
34064 9765 Acked Write 1 Done Done Sent false
34065 9766 Acked Write 4 Done Done Sent false
34066 9767 Acked Read 10 Done Done Sent false
34067 9768 Acked Read 4 Done Done Sent false
34068 9769 Acked Read 3 Done Done Sent false
34069 9770 Acked Flush 0 Done Done Sent false
34070 9771 Acked Write 1 Done Done Sent false
34071 9772 Acked Write 6 Done Done Sent false
34072 9773 Acked Write 8 Done Done Sent false
34073 9774 Acked Read 5 Done Done Sent false
34074 9775 Acked Read 2 Done Done Sent false
34075 9776 Acked Read 5 Done Done Sent false
34076 9777 Acked Read 8 Done Done Sent false
34077 9778 Acked Write 1 Done Done Sent false
34078 9779 Acked Flush 0 Done Done Sent false
34079 9780 Acked Read 9 Done Done Sent false
34080 9781 Acked Write 1 Done Done Sent false
34081 9782 Acked Write 7 Done Done Sent false
34082 9783 Acked Read 9 Done Done Sent false
34083 9784 Acked Read 3 Done Done Sent false
34084 9785 Acked Write 8 Done Done Sent false
34085 9786 Acked Read 1 Done Done Sent false
34086 9787 Acked Write 3 Done Done Sent false
34087 9788 Acked Write 6 Done Done Sent false
34088 9789 Acked Read 4 Done Done Sent false
34089 9790 Acked Read 7 Done Done Sent false
34090 9791 Acked Read 4 Done Done Sent false
34091 9792 Acked Read 2 Done Done Sent false
34092 9793 Acked Read 6 Done Done Sent false
34093 9794 Acked Read 1 Done Done Sent false
34094 9795 Acked Write 9 Done Done Sent false
34095 9796 Acked Flush 0 Done Done Sent false
34096 9797 Acked Read 8 Done Done Sent false
34097 9798 Acked Read 6 Done Done Sent false
34098 9799 Acked Read 3 Done Done Sent false
34099 9800 Acked Read 6 Done Done Sent false
34100 9801 Acked Write 6 Done Done Sent false
34101 9802 Acked Write 9 Done Done Sent false
34102 9803 Acked Read 4 Done Done Sent false
34103 9804 Acked Flush 0 Done Done Sent false
34104 9805 Acked Read 7 Done Done Sent false
34105 9806 Acked Write 9 Done Done Sent false
34106 9807 Acked Read 3 Done Done Sent false
34107 9808 Acked Flush 0 Done Done Sent false
34108 9809 Acked Write 1 Done Done Sent false
34109 9810 Acked Write 7 Done Done Sent false
34110 9811 Acked Write 5 Done Done Sent false
34111 9812 Acked Write 6 Done Done Sent false
34112 9813 Acked Read 9 Done Done Sent false
34113 9814 Acked Write 4 Done Done Sent false
34114 9815 Acked Read 2 Done Done Sent false
34115 9816 Acked Flush 0 Done Done Sent false
34116 9817 Acked Read 10 Done Done Sent false
34117 9818 Acked Write 7 Done Done Sent false
34118 9819 Acked Read 1 Done Done Sent false
34119 9820 Acked Read 4 Done Done Sent false
34120 9821 Acked Read 5 Done Done Sent false
34121 9822 Acked Write 10 Done Done Sent false
34122 9823 Acked Flush 0 Done Done Sent false
34123 9824 Acked Read 1 Done Done Sent false
34124 9825 Acked Flush 0 Done Done Sent false
34125 9826 Acked Flush 0 Done Done Sent false
34126 9827 Acked Flush 0 Done Done Sent false
34127 9828 Acked Write 9 Done Done Sent false
34128 9829 Acked Write 7 Done Done Sent false
34129 9830 Acked Write 9 Done Done Sent false
34130 9831 Acked Read 1 Done Done Sent false
34131 9832 Acked Read 3 Done Done Sent false
34132 9833 Acked Read 6 Done Done Sent false
34133 9834 Acked Read 9 Done Done Sent false
34134 9835 Acked Read 2 Done Done Sent false
34135 9836 Acked Read 2 Done Done Sent false
34136 9837 Acked Read 10 Done Done Sent false
34137 9838 Acked Read 3 Done Done Sent false
34138 9839 Acked Write 8 Done Done Sent false
34139 9840 Acked Read 6 Done Done Sent false
34140 9841 Acked Read 8 Done Done Sent false
34141 9842 Acked Read 9 Done Done Sent false
34142 9843 Acked Read 10 Done Done Sent false
34143 9844 Acked Read 9 Done Done Sent false
34144 9845 Acked Write 7 Done Done Sent false
34145 9846 Acked Write 4 Done Done Sent false
34146 9847 Acked Read 7 Done Done Sent false
34147 9848 Acked Write 5 Done Done Sent false
34148 9849 Acked Write 9 Done Done Sent false
34149 9850 Acked Write 2 Done Done Sent false
34150 9851 Acked Read 8 Done Done Sent false
34151 9852 Acked Read 3 Done Done Sent false
34152 9853 Acked Flush 0 Done Done Sent false
34153 9854 Acked Write 6 Done Done Sent false
34154 9855 Acked Write 1 Done Done Sent false
34155 9856 Acked Read 10 Done Done Sent false
34156 9857 Acked Write 1 Done Done Sent false
34157 9858 Acked Write 6 Done Done Sent false
34158 9859 Acked Write 10 Done Done Sent false
34159 9860 Acked Read 4 Done Done Sent false
34160 9861 Acked Write 4 Done Done Sent false
34161 9862 Acked Write 5 Done Done Sent false
34162 9863 Acked Write 1 Done Done Sent false
34163 9864 Acked Write 2 Done Done Sent false
34164 9865 Acked Write 10 Done Done Sent false
34165 9866 Acked Read 3 Done Done Sent false
34166 9867 Acked Read 2 Done Done Sent false
34167 9868 Acked Write 10 Done Done Sent false
34168 9869 Acked Write 7 Done Done Sent false
34169 9870 Acked Write 5 Done Done Sent false
34170 9871 Acked Flush 0 Done Done Sent false
34171 9872 Acked Write 4 Done Done Sent false
34172 9873 Acked Read 4 Done Done Sent false
34173 9874 Acked Write 3 Done Done Sent false
34174 9875 Acked Write 3 Done Done Sent false
34175 9876 Acked Write 1 Done Done Sent false
34176 9877 Acked Write 4 Done Done Sent false
34177 9878 Acked Read 2 Done Done Sent false
34178 9879 Acked Flush 0 Done Done Sent false
34179 9880 Acked Read 2 Done Done Sent false
34180 9881 Acked Read 6 Done Done Sent false
34181 9882 Acked Read 3 Done Done Sent false
34182 9883 Acked Write 2 Done Done Sent false
34183 9884 Acked Read 10 Done Done Sent false
34184 9885 Acked Read 4 Done Done Sent false
34185 9886 Acked Flush 0 Done Done Sent false
34186 9887 Acked Read 3 Done Done Sent false
34187 9888 Acked Read 7 Done Done Sent false
34188 9889 Acked Write 8 Done Done Sent false
34189 9890 Acked Read 9 Done Done Sent false
34190 9891 Acked Flush 0 Done Done Sent false
34191 9892 Acked Read 9 Done Done Sent false
34192 9893 Acked Write 2 Done Done Sent false
34193 9894 Acked Read 3 Done Done Sent false
34194 9895 Acked Read 6 Done Done Sent false
34195 9896 Acked Read 10 Done Done Sent false
34196 9897 Acked Flush 0 Done Done Sent false
34197 9898 Acked Read 8 Done Done Sent false
34198 9899 Acked Flush 0 Done Done Sent false
34199 9900 Acked Read 1 Done Done Sent false
34200 9901 Acked Read 5 Done Done Sent false
34201 9902 Acked Read 3 Done Done Sent false
34202 9903 Acked Read 6 Done Done Sent false
34203 9904 Acked Read 1 Done Done Sent false
34204 9905 Acked Read 10 Done Done Sent false
34205 9906 Acked Write 5 Done Done Sent false
34206 9907 Acked Read 7 Done Done Sent false
34207 9908 Acked Write 6 Done Done Sent false
34208 9909 Acked Read 10 Done Done Sent false
34209 9910 Acked Read 8 Done Done Sent false
34210 9911 Acked Read 3 Done Done Sent false
34211 9912 Acked Read 7 Done Done Sent false
34212 9913 Acked Read 5 Done Done Sent false
34213 9914 Acked Write 3 Done Done Sent false
34214 9915 Acked Write 3 Done Done Sent false
34215 9916 Acked Flush 0 Done Done Sent false
34216 9917 Acked Read 2 Done Done Sent false
34217 9918 Acked Write 9 Done Done Sent false
34218 9919 Acked Write 5 Done Done Sent false
34219 9920 Acked Read 3 Done Done Sent false
34220 9921 Acked Write 8 Done Done Sent false
34221 9922 Acked Read 9 Done Done Sent false
34222 9923 Acked Write 7 Done Done Sent false
34223 9924 Acked Read 4 Done Done Sent false
34224 9925 Acked Read 1 Done Done Sent false
34225 9926 Acked Read 4 Done Done Sent false
34226 9927 Acked Write 3 Done Done Sent false
34227 9928 Acked Write 5 Done Done Sent false
34228 9929 Acked Read 10 Done Done Sent false
34229 9930 Acked Read 10 Done Done Sent false
34230 9931 Acked Write 8 Done Done Sent false
34231 9932 Acked Write 10 Done Done Sent false
34232 9933 Acked Read 8 Done Done Sent false
34233 9934 Acked Flush 0 Done Done Sent false
34234 9935 Acked Write 3 Done Done Sent false
34235 9936 Acked Flush 0 Done Done Sent false
34236 9937 Acked Read 1 Done Done Sent false
34237 9938 Acked Write 8 Done Done Sent false
34238 9939 Acked Read 1 Done Done Sent false
34239 9940 Acked Read 4 Done Done Sent false
34240 9941 Acked Write 6 Done Done Sent false
34241 9942 Acked Read 7 Done Done Sent false
34242 9943 Acked Read 7 Done Done Sent false
34243 9944 Acked Write 10 Done Done Sent false
34244 9945 Acked Write 6 Done Done Sent false
34245 9946 Acked Flush 0 Done Done Sent false
34246 9947 Acked Read 6 Done Done Sent false
34247 9948 Acked Read 4 Done Done Sent false
34248 9949 Acked Read 5 Done Done Sent false
34249 9950 Acked Read 3 Done Done Sent false
34250 9951 Acked Write 4 Done Done Sent false
34251 9952 Acked Write 7 Done Done Sent false
34252 9953 Acked Write 5 Done Done Sent false
34253 9954 Acked Read 2 Done Done Sent false
34254 9955 Acked Read 1 Done Done Sent false
34255 9956 Acked Read 4 Done Done Sent false
34256 9957 Acked Write 10 Done Done Sent false
34257 9958 Acked Read 5 Done Done Sent false
34258 9959 Acked Write 2 Done Done Sent false
34259 9960 Acked Write 8 Done Done Sent false
34260 9961 Acked Write 2 Done Done Sent false
34261 9962 Acked Flush 0 Done Done Sent false
34262 9963 Acked Read 3 Done Done Sent false
34263 9964 Acked Write 4 Done Done Sent false
34264 9965 Acked Read 9 Done Done Sent false
34265 9966 Acked Write 1 Done Done Sent false
34266 9967 Acked Write 10 Done Done Sent false
34267 9968 Acked Write 6 Done Done Sent false
34268 9969 Acked Write 6 Done Done Sent false
34269 9970 Acked Write 4 Done Done Sent false
34270 9971 Acked Read 1 Done Done Sent false
34271 9972 Acked Read 5 Done Done Sent false
34272 9973 Acked Write 6 Done Done Sent false
34273 9974 Acked Write 4 Done Done Sent false
34274 9975 Acked Write 1 Done Done Sent false
34275 9976 Acked Read 10 Done Done Sent false
34276 9977 Acked Write 10 Done Done Sent false
34277 9978 Acked Read 7 Done Done Sent false
34278 9979 Acked Flush 0 Done Done Sent false
34279 9980 Acked Read 10 Done Done Sent false
34280 9981 Acked Read 4 Done Done Sent false
34281 9982 Acked Read 3 Done Done Sent false
34282 9983 Acked Write 1 Done Done Sent false
34283 9984 Acked Read 2 Done Done Sent false
34284 9985 Acked Flush 0 Done Done Sent false
34285 9986 Acked Write 2 Done Done Sent false
34286 9987 Acked Write 9 Done Done Sent false
34287 9988 Acked Read 1 Done Done Sent false
34288 9989 Acked Write 4 Done Done Sent false
34289 9990 Acked Read 10 Done Done Sent false
34290 9991 Acked Read 5 Done Done Sent false
34291 9992 Acked Read 3 Done Done Sent false
34292 9993 Acked Read 9 Done Done Sent false
34293 9994 Acked Flush 0 Done Done Sent false
34294 9995 Acked Write 2 Done Done Sent false
34295 9996 Acked Write 3 Done Done Sent false
34296 9997 Acked Write 1 Done Done Sent false
34297 9998 Acked Write 7 Done Done Sent false
34298 9999 Acked Write 6 Done Done Sent false
34299 10000 Acked Read 6 Done Done Sent false
34300 10001 Acked Read 9 Done Done Sent false
34301 10002 Acked Read 2 Done Done Sent false
34302 10003 Acked Write 1 Done Done Sent false
34303 10004 Acked Read 1 Done Done Sent false
34304 10005 Acked Write 7 Done Done Sent false
34305 10006 Acked Write 1 Done Done Sent false
34306 10007 Acked Write 1 Done Done Sent false
34307 10008 Acked Read 7 Done Done Sent false
34308 10009 Acked Read 6 Done Done Sent false
34309 10010 Acked Read 1 Done Done Sent false
34310 10011 Acked Write 8 Done Done Sent false
34311 10012 Acked Read 6 Done Done Sent false
34312 10013 Acked Write 3 Done Done Sent false
34313 10014 Acked Flush 0 Done Done Sent false
34314 10015 Acked Read 1 Done Done Sent false
34315 10016 Acked Write 2 Done Done Sent false
34316 10017 Acked Flush 0 Done Done Sent false
34317 10018 Acked Write 6 Done Done Sent false
34318 10019 Acked Read 1 Done Done Sent false
34319 10020 Acked Write 3 Done Done Sent false
34320 10021 Acked Write 4 Done Done Sent false
34321 10022 Acked Read 5 Done Done Sent false
34322 10023 Acked Read 10 Done Done Sent false
34323 10024 Acked Write 6 Done Done Sent false
34324 10025 Acked Flush 0 Done Done Sent false
34325 10026 Acked Write 10 Done Done Sent false
34326 10027 Acked Write 3 Done Done Sent false
34327 10028 Acked Write 7 Done Done Sent false
34328 10029 Acked Write 4 Done Done Sent false
34329 10030 Acked Read 1 Done Done Sent false
34330 10031 Acked Read 1 Done Done Sent false
34331 10032 Acked Read 3 Done Done Sent false
34332 10033 Acked Write 2 Done Done Sent false
34333 10034 Acked Write 7 Done Done Sent false
34334 10035 Acked Write 7 Done Done Sent false
34335 10036 Acked Read 5 Done Done Sent false
34336 10037 Acked Write 1 Done Done Sent false
34337 10038 Acked Read 9 Done Done Sent false
34338 10039 Acked Flush 0 Done Done Sent false
34339 10040 Acked Read 6 Done Done Sent false
34340 10041 Acked Write 4 Done Done Sent false
34341 10042 Acked Write 5 Done Done Sent false
34342 10043 Acked Read 4 Done Done Sent false
34343 10044 Acked Write 3 Done Done Sent false
34344 10045 Acked Read 3 Done Done Sent false
34345 10046 Acked Write 6 Done Done Sent false
34346 10047 Acked Read 7 Done Done Sent false
34347 10048 Acked Read 2 Done Done Sent false
34348 10049 Acked Write 9 Done Done Sent false
34349 10050 Acked Write 5 Done Done Sent false
34350 10051 Acked Flush 0 Done Done Sent false
34351 10052 Acked Write 5 Done Done Sent false
34352 10053 Acked Write 3 Done Done Sent false
34353 10054 Acked Write 4 Done Done Sent false
34354 10055 Acked Flush 0 Done Done Sent false
34355 STATES DS:0 DS:1 DS:2 TOTAL
34356 Sent 0 0 301 301
34357 Done 301 301 0 602
34358 Skipped 0 0 0 0
34359 Error 0 0 0 0
34360 Last Flush: 10055 10055 9754
34361 Downstairs last five completed: 9754 9753 9752 9751 9750
34362 Upstairs last five completed: 10055 10054 10053 10052 10051
34363 CLIENT: Up:0 ds:301 act:2
34364 [8830] got start action
34365 [8830] I am going to restart
34366 Starting downstairs at port 8830
34367 Make output file at "/tmp/dsc/downstairs-8830.txt"
34368 [8830][2] reports Starting
34369 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1230)
34370 [8830][2] reports Running
34371 ----------------------------------------------------------------
34372 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
34373 States: Active Active Offline
34374 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34375 9755 Acked Write 10 Done Done Sent false
34376 9756 Acked Read 6 Done Done Sent false
34377 9757 Acked Read 6 Done Done Sent false
34378 9758 Acked Write 2 Done Done Sent false
34379 9759 Acked Write 2 Done Done Sent false
34380 9760 Acked Read 2 Done Done Sent false
34381 9761 Acked Write 6 Done Done Sent false
34382 9762 Acked Read 7 Done Done Sent false
34383 9763 Acked Read 7 Done Done Sent false
34384 9764 Acked Read 10 Done Done Sent false
34385 9765 Acked Write 1 Done Done Sent false
34386 9766 Acked Write 4 Done Done Sent false
34387 9767 Acked Read 10 Done Done Sent false
34388 9768 Acked Read 4 Done Done Sent false
34389 9769 Acked Read 3 Done Done Sent false
34390 9770 Acked Flush 0 Done Done Sent false
34391 9771 Acked Write 1 Done Done Sent false
34392 9772 Acked Write 6 Done Done Sent false
34393 9773 Acked Write 8 Done Done Sent false
34394 9774 Acked Read 5 Done Done Sent false
34395 9775 Acked Read 2 Done Done Sent false
34396 9776 Acked Read 5 Done Done Sent false
34397 9777 Acked Read 8 Done Done Sent false
34398 9778 Acked Write 1 Done Done Sent false
34399 9779 Acked Flush 0 Done Done Sent false
34400 9780 Acked Read 9 Done Done Sent false
34401 9781 Acked Write 1 Done Done Sent false
34402 9782 Acked Write 7 Done Done Sent false
34403 9783 Acked Read 9 Done Done Sent false
34404 9784 Acked Read 3 Done Done Sent false
34405 9785 Acked Write 8 Done Done Sent false
34406 9786 Acked Read 1 Done Done Sent false
34407 9787 Acked Write 3 Done Done Sent false
34408 9788 Acked Write 6 Done Done Sent false
34409 9789 Acked Read 4 Done Done Sent false
34410 9790 Acked Read 7 Done Done Sent false
34411 9791 Acked Read 4 Done Done Sent false
34412 9792 Acked Read 2 Done Done Sent false
34413 9793 Acked Read 6 Done Done Sent false
34414 9794 Acked Read 1 Done Done Sent false
34415 9795 Acked Write 9 Done Done Sent false
34416 9796 Acked Flush 0 Done Done Sent false
34417 9797 Acked Read 8 Done Done Sent false
34418 9798 Acked Read 6 Done Done Sent false
34419 9799 Acked Read 3 Done Done Sent false
34420 9800 Acked Read 6 Done Done Sent false
34421 9801 Acked Write 6 Done Done Sent false
34422 9802 Acked Write 9 Done Done Sent false
34423 9803 Acked Read 4 Done Done Sent false
34424 9804 Acked Flush 0 Done Done Sent false
34425 9805 Acked Read 7 Done Done Sent false
34426 9806 Acked Write 9 Done Done Sent false
34427 9807 Acked Read 3 Done Done Sent false
34428 9808 Acked Flush 0 Done Done Sent false
34429 9809 Acked Write 1 Done Done Sent false
34430 9810 Acked Write 7 Done Done Sent false
34431 9811 Acked Write 5 Done Done Sent false
34432 9812 Acked Write 6 Done Done Sent false
34433 9813 Acked Read 9 Done Done Sent false
34434 9814 Acked Write 4 Done Done Sent false
34435 9815 Acked Read 2 Done Done Sent false
34436 9816 Acked Flush 0 Done Done Sent false
34437 9817 Acked Read 10 Done Done Sent false
34438 9818 Acked Write 7 Done Done Sent false
34439 9819 Acked Read 1 Done Done Sent false
34440 9820 Acked Read 4 Done Done Sent false
34441 9821 Acked Read 5 Done Done Sent false
34442 9822 Acked Write 10 Done Done Sent false
34443 9823 Acked Flush 0 Done Done Sent false
34444 9824 Acked Read 1 Done Done Sent false
34445 9825 Acked Flush 0 Done Done Sent false
34446 9826 Acked Flush 0 Done Done Sent false
34447 9827 Acked Flush 0 Done Done Sent false
34448 9828 Acked Write 9 Done Done Sent false
34449 9829 Acked Write 7 Done Done Sent false
34450 9830 Acked Write 9 Done Done Sent false
34451 9831 Acked Read 1 Done Done Sent false
34452 9832 Acked Read 3 Done Done Sent false
34453 9833 Acked Read 6 Done Done Sent false
34454 9834 Acked Read 9 Done Done Sent false
34455 9835 Acked Read 2 Done Done Sent false
34456 9836 Acked Read 2 Done Done Sent false
34457 9837 Acked Read 10 Done Done Sent false
34458 9838 Acked Read 3 Done Done Sent false
34459 9839 Acked Write 8 Done Done Sent false
34460 9840 Acked Read 6 Done Done Sent false
34461 9841 Acked Read 8 Done Done Sent false
34462 9842 Acked Read 9 Done Done Sent false
34463 9843 Acked Read 10 Done Done Sent false
34464 9844 Acked Read 9 Done Done Sent false
34465 9845 Acked Write 7 Done Done Sent false
34466 9846 Acked Write 4 Done Done Sent false
34467 9847 Acked Read 7 Done Done Sent false
34468 9848 Acked Write 5 Done Done Sent false
34469 9849 Acked Write 9 Done Done Sent false
34470 9850 Acked Write 2 Done Done Sent false
34471 9851 Acked Read 8 Done Done Sent false
34472 9852 Acked Read 3 Done Done Sent false
34473 9853 Acked Flush 0 Done Done Sent false
34474 9854 Acked Write 6 Done Done Sent false
34475 9855 Acked Write 1 Done Done Sent false
34476 9856 Acked Read 10 Done Done Sent false
34477 9857 Acked Write 1 Done Done Sent false
34478 9858 Acked Write 6 Done Done Sent false
34479 9859 Acked Write 10 Done Done Sent false
34480 9860 Acked Read 4 Done Done Sent false
34481 9861 Acked Write 4 Done Done Sent false
34482 9862 Acked Write 5 Done Done Sent false
34483 9863 Acked Write 1 Done Done Sent false
34484 9864 Acked Write 2 Done Done Sent false
34485 9865 Acked Write 10 Done Done Sent false
34486 9866 Acked Read 3 Done Done Sent false
34487 9867 Acked Read 2 Done Done Sent false
34488 9868 Acked Write 10 Done Done Sent false
34489 9869 Acked Write 7 Done Done Sent false
34490 9870 Acked Write 5 Done Done Sent false
34491 9871 Acked Flush 0 Done Done Sent false
34492 9872 Acked Write 4 Done Done Sent false
34493 9873 Acked Read 4 Done Done Sent false
34494 9874 Acked Write 3 Done Done Sent false
34495 9875 Acked Write 3 Done Done Sent false
34496 9876 Acked Write 1 Done Done Sent false
34497 9877 Acked Write 4 Done Done Sent false
34498 9878 Acked Read 2 Done Done Sent false
34499 9879 Acked Flush 0 Done Done Sent false
34500 9880 Acked Read 2 Done Done Sent false
34501 9881 Acked Read 6 Done Done Sent false
34502 9882 Acked Read 3 Done Done Sent false
34503 9883 Acked Write 2 Done Done Sent false
34504 9884 Acked Read 10 Done Done Sent false
34505 9885 Acked Read 4 Done Done Sent false
34506 9886 Acked Flush 0 Done Done Sent false
34507 9887 Acked Read 3 Done Done Sent false
34508 9888 Acked Read 7 Done Done Sent false
34509 9889 Acked Write 8 Done Done Sent false
34510 9890 Acked Read 9 Done Done Sent false
34511 9891 Acked Flush 0 Done Done Sent false
34512 9892 Acked Read 9 Done Done Sent false
34513 9893 Acked Write 2 Done Done Sent false
34514 9894 Acked Read 3 Done Done Sent false
34515 9895 Acked Read 6 Done Done Sent false
34516 9896 Acked Read 10 Done Done Sent false
34517 9897 Acked Flush 0 Done Done Sent false
34518 9898 Acked Read 8 Done Done Sent false
34519 9899 Acked Flush 0 Done Done Sent false
34520 9900 Acked Read 1 Done Done Sent false
34521 9901 Acked Read 5 Done Done Sent false
34522 9902 Acked Read 3 Done Done Sent false
34523 9903 Acked Read 6 Done Done Sent false
34524 9904 Acked Read 1 Done Done Sent false
34525 9905 Acked Read 10 Done Done Sent false
34526 9906 Acked Write 5 Done Done Sent false
34527 9907 Acked Read 7 Done Done Sent false
34528 9908 Acked Write 6 Done Done Sent false
34529 9909 Acked Read 10 Done Done Sent false
34530 9910 Acked Read 8 Done Done Sent false
34531 9911 Acked Read 3 Done Done Sent false
34532 9912 Acked Read 7 Done Done Sent false
34533 9913 Acked Read 5 Done Done Sent false
34534 9914 Acked Write 3 Done Done Sent false
34535 9915 Acked Write 3 Done Done Sent false
34536 9916 Acked Flush 0 Done Done Sent false
34537 9917 Acked Read 2 Done Done Sent false
34538 9918 Acked Write 9 Done Done Sent false
34539 9919 Acked Write 5 Done Done Sent false
34540 9920 Acked Read 3 Done Done Sent false
34541 9921 Acked Write 8 Done Done Sent false
34542 9922 Acked Read 9 Done Done Sent false
34543 9923 Acked Write 7 Done Done Sent false
34544 9924 Acked Read 4 Done Done Sent false
34545 9925 Acked Read 1 Done Done Sent false
34546 9926 Acked Read 4 Done Done Sent false
34547 9927 Acked Write 3 Done Done Sent false
34548 9928 Acked Write 5 Done Done Sent false
34549 9929 Acked Read 10 Done Done Sent false
34550 9930 Acked Read 10 Done Done Sent false
34551 9931 Acked Write 8 Done Done Sent false
34552 9932 Acked Write 10 Done Done Sent false
34553 9933 Acked Read 8 Done Done Sent false
34554 9934 Acked Flush 0 Done Done Sent false
34555 9935 Acked Write 3 Done Done Sent false
34556 9936 Acked Flush 0 Done Done Sent false
34557 9937 Acked Read 1 Done Done Sent false
34558 9938 Acked Write 8 Done Done Sent false
34559 9939 Acked Read 1 Done Done Sent false
34560 9940 Acked Read 4 Done Done Sent false
34561 9941 Acked Write 6 Done Done Sent false
34562 9942 Acked Read 7 Done Done Sent false
34563 9943 Acked Read 7 Done Done Sent false
34564 9944 Acked Write 10 Done Done Sent false
34565 9945 Acked Write 6 Done Done Sent false
34566 9946 Acked Flush 0 Done Done Sent false
34567 9947 Acked Read 6 Done Done Sent false
34568 9948 Acked Read 4 Done Done Sent false
34569 9949 Acked Read 5 Done Done Sent false
34570 9950 Acked Read 3 Done Done Sent false
34571 9951 Acked Write 4 Done Done Sent false
34572 9952 Acked Write 7 Done Done Sent false
34573 9953 Acked Write 5 Done Done Sent false
34574 9954 Acked Read 2 Done Done Sent false
34575 9955 Acked Read 1 Done Done Sent false
34576 9956 Acked Read 4 Done Done Sent false
34577 9957 Acked Write 10 Done Done Sent false
34578 9958 Acked Read 5 Done Done Sent false
34579 9959 Acked Write 2 Done Done Sent false
34580 9960 Acked Write 8 Done Done Sent false
34581 9961 Acked Write 2 Done Done Sent false
34582 9962 Acked Flush 0 Done Done Sent false
34583 9963 Acked Read 3 Done Done Sent false
34584 9964 Acked Write 4 Done Done Sent false
34585 9965 Acked Read 9 Done Done Sent false
34586 9966 Acked Write 1 Done Done Sent false
34587 9967 Acked Write 10 Done Done Sent false
34588 9968 Acked Write 6 Done Done Sent false
34589 9969 Acked Write 6 Done Done Sent false
34590 9970 Acked Write 4 Done Done Sent false
34591 9971 Acked Read 1 Done Done Sent false
34592 9972 Acked Read 5 Done Done Sent false
34593 9973 Acked Write 6 Done Done Sent false
34594 9974 Acked Write 4 Done Done Sent false
34595 9975 Acked Write 1 Done Done Sent false
34596 9976 Acked Read 10 Done Done Sent false
34597 9977 Acked Write 10 Done Done Sent false
34598 9978 Acked Read 7 Done Done Sent false
34599 9979 Acked Flush 0 Done Done Sent false
34600 9980 Acked Read 10 Done Done Sent false
34601 9981 Acked Read 4 Done Done Sent false
34602 9982 Acked Read 3 Done Done Sent false
34603 9983 Acked Write 1 Done Done Sent false
34604 9984 Acked Read 2 Done Done Sent false
34605 9985 Acked Flush 0 Done Done Sent false
34606 9986 Acked Write 2 Done Done Sent false
34607 9987 Acked Write 9 Done Done Sent false
34608 9988 Acked Read 1 Done Done Sent false
34609 9989 Acked Write 4 Done Done Sent false
34610 9990 Acked Read 10 Done Done Sent false
34611 9991 Acked Read 5 Done Done Sent false
34612 9992 Acked Read 3 Done Done Sent false
34613 9993 Acked Read 9 Done Done Sent false
34614 9994 Acked Flush 0 Done Done Sent false
34615 9995 Acked Write 2 Done Done Sent false
34616 9996 Acked Write 3 Done Done Sent false
34617 9997 Acked Write 1 Done Done Sent false
34618 9998 Acked Write 7 Done Done Sent false
34619 9999 Acked Write 6 Done Done Sent false
34620 10000 Acked Read 6 Done Done Sent false
34621 10001 Acked Read 9 Done Done Sent false
34622 10002 Acked Read 2 Done Done Sent false
34623 10003 Acked Write 1 Done Done Sent false
34624 10004 Acked Read 1 Done Done Sent false
34625 10005 Acked Write 7 Done Done Sent false
34626 10006 Acked Write 1 Done Done Sent false
34627 10007 Acked Write 1 Done Done Sent false
34628 10008 Acked Read 7 Done Done Sent false
34629 10009 Acked Read 6 Done Done Sent false
34630 10010 Acked Read 1 Done Done Sent false
34631 10011 Acked Write 8 Done Done Sent false
34632 10012 Acked Read 6 Done Done Sent false
34633 10013 Acked Write 3 Done Done Sent false
34634 10014 Acked Flush 0 Done Done Sent false
34635 10015 Acked Read 1 Done Done Sent false
34636 10016 Acked Write 2 Done Done Sent false
34637 10017 Acked Flush 0 Done Done Sent false
34638 10018 Acked Write 6 Done Done Sent false
34639 10019 Acked Read 1 Done Done Sent false
34640 10020 Acked Write 3 Done Done Sent false
34641 10021 Acked Write 4 Done Done Sent false
34642 10022 Acked Read 5 Done Done Sent false
34643 10023 Acked Read 10 Done Done Sent false
34644 10024 Acked Write 6 Done Done Sent false
34645 10025 Acked Flush 0 Done Done Sent false
34646 10026 Acked Write 10 Done Done Sent false
34647 10027 Acked Write 3 Done Done Sent false
34648 10028 Acked Write 7 Done Done Sent false
34649 10029 Acked Write 4 Done Done Sent false
34650 10030 Acked Read 1 Done Done Sent false
34651 10031 Acked Read 1 Done Done Sent false
34652 10032 Acked Read 3 Done Done Sent false
34653 10033 Acked Write 2 Done Done Sent false
34654 10034 Acked Write 7 Done Done Sent false
34655 10035 Acked Write 7 Done Done Sent false
34656 10036 Acked Read 5 Done Done Sent false
34657 10037 Acked Write 1 Done Done Sent false
34658 10038 Acked Read 9 Done Done Sent false
34659 10039 Acked Flush 0 Done Done Sent false
34660 10040 Acked Read 6 Done Done Sent false
34661 10041 Acked Write 4 Done Done Sent false
34662 10042 Acked Write 5 Done Done Sent false
34663 10043 Acked Read 4 Done Done Sent false
34664 10044 Acked Write 3 Done Done Sent false
34665 10045 Acked Read 3 Done Done Sent false
34666 10046 Acked Write 6 Done Done Sent false
34667 10047 Acked Read 7 Done Done Sent false
34668 10048 Acked Read 2 Done Done Sent false
34669 10049 Acked Write 9 Done Done Sent false
34670 10050 Acked Write 5 Done Done Sent false
34671 10051 Acked Flush 0 Done Done Sent false
34672 10052 Acked Write 5 Done Done Sent false
34673 10053 Acked Write 3 Done Done Sent false
34674 10054 Acked Write 4 Done Done Sent false
34675 10055 Acked Flush 0 Done Done Sent false
34676 STATES DS:0 DS:1 DS:2 TOTAL
34677 Sent 0 0 301 301
34678 Done 301 301 0 602
34679 Skipped 0 0 0 0
34680 Error 0 0 0 0
34681 Last Flush: 10055 10055 9754
34682 Downstairs last five completed: 9754 9753 9752 9751 9750
34683 Upstairs last five completed: 10055 10054 10053 10052 10051
34684 CLIENT: Up:0 ds:301 act:2
346852025-01-17T21:18:21.650ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
346862025-01-17T21:18:21.650ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64127) = io task client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
346872025-01-17T21:18:21.651ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
346882025-01-17T21:18:21.651ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
346892025-01-17T21:18:21.652ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
346902025-01-17T21:18:21.652ZINFOcrucible: send last flush ID to this DS: 9754 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
346912025-01-17T21:18:21.652ZINFOcrucible: Replied this last flush ID: 9754 = downstairs client = 2 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
346922025-01-17T21:18:21.652ZINFOcrucible: [2] client re-new 301 jobs since flush 9754 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
346932025-01-17T21:18:21.652ZINFOcrucible: [2] Replayed 301 jobs since flush: 9754 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
34694 ----------------------------------------------------------------
34695 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
34696 Downstairs last five completed: 10055 10054 10053 10052 10051
34697 Upstairs last five completed: 10055 10054 10053 10052 10051
34698 CLIENT: Up:0 ds:0 act:3
34699 Replay: All jobs finished
34700 Main task has work to do, go find it
34701 got dsc Stop(1)
34702 stop 1
34703 [8820] Got stop action so:false kr:false
34704 Jan 17 21:18:23.884 INFO request completed, latency_us: 382, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 5a789960-a3ce-4f5a-bd25-6ade11dcd350, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
347052025-01-17T21:18:23.885ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
347062025-01-17T21:18:23.885ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
347072025-01-17T21:18:23.885ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
347082025-01-17T21:18:23.885ZINFOcrucible: client task is exiting = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
34709 Jan 17 21:18:23.885 INFO request completed, latency_us: 132, response_code: 200, uri: /state/cid/1, method: GET, req_id: 6d932b3a-3af6-4928-ba09-c70d9be01557, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
34710 [8820] Exited with: ExitStatus(unix_wait_status(9))
34711 [8820][1] reports Exit
34712 Jan 17 21:18:27.887 INFO request completed, latency_us: 190, response_code: 200, uri: /state/cid/1, method: GET, req_id: 5b91b283-e018-4e04-8428-a5975697f22e, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
34713 001/300 Read block 78 len 20480
34714 002/300 Read block 110 len 36864
34715 003/300 Read block 1625 len 36864
34716 004/300 Write block 1846 len 8192 data: 4 4
34717 005/300 Flush
34718 006/300 Write block 1795 len 20480 data: 7 6 5 5 3
34719 007/300 Read block 3287 len 20480
34720 008/300 Read block 4996 len 4096
34721 009/300 Write block 1272 len 4096 data: 6
34722 010/300 Flush
34723 011/300 Write block 3846 len 28672 data: 6 5 3 3 3 4 4
34724 012/300 Write block 1718 len 36864 data: 4 6 7 8 7 6 6 7 8
34725 013/300 Write block 592 len 16384 data: 8 7 6 7
34726 014/300 Read block 3595 len 8192
34727 015/300 Write block 1203 len 32768 data: 7 6 5 5 5 5 4 4
34728 016/300 Write block 3029 len 40960 data: 3 3 2 2 2 2 3 5 4 6
34729 017/300 Read block 1966 len 4096
34730 018/300 Write block 2420 len 12288 data: 8 8 7
34731 019/300 Read block 3975 len 40960
34732 020/300 Write block 3604 len 24576 data: 8 9 8 8 11 13
34733 021/300 Read block 814 len 20480
34734 022/300 Read block 1047 len 32768
34735 023/300 Read block 2742 len 40960
34736 024/300 Read block 4916 len 24576
34737 025/300 Read block 3803 len 36864
34738 026/300 Read block 2586 len 20480
34739 027/300 Write block 1472 len 24576 data: 8 7 7 5 3 4
34740 028/300 Read block 4093 len 28672
34741 029/300 Read block 2290 len 12288
34742 030/300 Write block 4647 len 36864 data: 3 3 3 2 4 5 5 5 5
34743 031/300 Read block 1344 len 24576
34744 032/300 Write block 4272 len 12288 data: 6 6 5
34745 033/300 Read block 3263 len 20480
34746 034/300 Write block 3852 len 4096 data: 5
34747 035/300 Write block 639 len 12288 data: 4 6 5
34748 036/300 Write block 4915 len 12288 data: 7 4 4
34749 037/300 Read block 3536 len 32768
34750 038/300 Write block 3330 len 40960 data: 7 7 7 8 8 8 7 6 4 5
34751 039/300 Read block 3685 len 36864
34752 040/300 Read block 3321 len 8192
34753 041/300 Write block 3602 len 28672 data: 4 8 9 10 9 9 12
34754 042/300 Write block 1712 len 16384 data: 6 4 4 4
34755 043/300 Read block 1079 len 40960
34756 044/300 Flush
34757 045/300 Write block 3995 len 12288 data: 13 13 13
34758 046/300 Write block 3766 len 40960 data: 6 6 6 7 5 5 4 3 5 5
34759 047/300 Read block 2542 len 16384
34760 048/300 Write block 924 len 24576 data: 8 9 9 8 6 8
34761 049/300 Write block 4356 len 36864 data: 4 4 3 3 4 6 7 8 9
34762 050/300 Write block 4469 len 16384 data: 6 8 9 11
34763 051/300 Read block 610 len 32768
34764 052/300 Read block 3139 len 32768
34765 053/300 Write block 1184 len 28672 data: 6 6 6 5 6 6 6
34766 054/300 Flush
34767 055/300 Write block 2278 len 24576 data: 6 7 7 6 6 7
34768 056/300 Write block 352 len 20480 data: 3 3 3 3 3
34769 057/300 Write block 2996 len 28672 data: 5 5 4 4 5 7 8
34770 058/300 Flush
34771 059/300 Write block 2048 len 16384 data: 6 7 9 8
34772 060/300 Write block 1772 len 20480 data: 4 4 4 5 5
34773 061/300 Write block 1904 len 4096 data: 6
34774 062/300 Read block 178 len 24576
34775 063/300 Write block 4523 len 12288 data: 6 7 6
34776 064/300 Write block 2607 len 24576 data: 6 3 2 3 3 3
34777 065/300 Read block 1502 len 4096
34778 066/300 Write block 2232 len 24576 data: 8 6 6 4 4 3
34779 067/300 Flush
34780 068/300 Write block 1365 len 28672 data: 3 4 6 8 9 9 10
34781 069/300 Read block 3655 len 24576
34782 070/300 Write block 3860 len 4096 data: 4
34783 071/300 Flush
34784 072/300 Write block 2430 len 36864 data: 7 7 7 5 5 5 5 5 5
34785 073/300 Write block 4622 len 24576 data: 7 7 7 6 4 4
34786 074/300 Write block 4765 len 4096 data: 8
34787 075/300 Read block 4876 len 40960
34788 076/300 Write block 1411 len 40960 data: 4 4 5 5 6 6 6 6 6 5
34789 077/300 Read block 2019 len 40960
34790 078/300 Write block 492 len 32768 data: 3 3 4 3 4 4 5 4
34791 079/300 Write block 1224 len 8192 data: 6 7
34792 080/300 Read block 2469 len 20480
34793 081/300 Write block 1933 len 8192 data: 4 4
34794 082/300 Write block 2352 len 24576 data: 11 7 5 6 6 5
34795 083/300 Write block 2207 len 16384 data: 6 5 4 6
34796 084/300 Read block 2247 len 28672
34797 085/300 Write block 122 len 24576 data: 9 7 6 4 3 2
34798 086/300 Write block 4083 len 36864 data: 4 3 3 5 5 4 4 3 3
34799 087/300 Read block 3206 len 40960
34800 088/300 Flush
34801 089/300 Write block 2403 len 28672 data: 5 6 7 8 9 9 8
34802 090/300 Read block 664 len 16384
34803 091/300 Read block 4891 len 4096
34804 092/300 Read block 915 len 32768
34805 093/300 Read block 150 len 28672
34806 094/300 Write block 4375 len 24576 data: 7 8 7 6 5 4
34807 095/300 Read block 1291 len 40960
34808 096/300 Write block 2782 len 40960 data: 4 6 7 7 7 9 7 8 11 11
34809 097/300 Flush
34810 098/300 Read block 2240 len 4096
34811 099/300 Write block 152 len 24576 data: 8 9 9 8 7 7
34812 100/300 Flush
34813 101/300 Read block 2027 len 24576
34814 102/300 Read block 779 len 16384
34815 103/300 Write block 1496 len 20480 data: 3 4 4 4 6
34816 104/300 Write block 1245 len 12288 data: 8 8 8
34817 105/300 Write block 4932 len 12288 data: 7 6 7
34818 106/300 Read block 3539 len 28672
34819 107/300 Read block 2835 len 32768
34820 108/300 Read block 101 len 24576
34821 109/300 Write block 3588 len 8192 data: 5 6
34822 110/300 Write block 2182 len 20480 data: 4 6 6 5 4
34823 111/300 Write block 962 len 8192 data: 9 7
34824 112/300 Flush
34825 113/300 Read block 3407 len 24576
34826 114/300 Write block 1265 len 8192 data: 6 5
34827 115/300 Read block 2008 len 20480
34828 116/300 Read block 3487 len 32768
34829 117/300 Write block 3036 len 24576 data: 6 5 7 5 5 4
34830 118/300 Write block 2932 len 20480 data: 5 5 5 5 5
34831 119/300 Read block 2395 len 32768
34832 120/300 Read block 4642 len 28672
34833 121/300 Read block 1921 len 12288
34834 122/300 Write block 329 len 16384 data: 3 3 3 3
34835 123/300 Write block 1160 len 20480 data: 10 8 6 5 4
34836 124/300 Read block 1977 len 8192
34837 125/300 Read block 4938 len 20480
34838 126/300 Read block 169 len 4096
34839 127/300 Write block 263 len 8192 data: 4 5
34840 128/300 Write block 2901 len 20480 data: 9 10 11 10 9
34841 129/300 Read block 453 len 32768
34842 130/300 Write block 2784 len 32768 data: 8 8 8 10 8 9 12 12
34843 131/300 Read block 4979 len 32768
34844 132/300 Read block 4330 len 8192
34845 133/300 Read block 4959 len 32768
34846 134/300 Read block 1427 len 36864
34847 135/300 Write block 2591 len 32768 data: 6 6 5 4 4 4 4 4
34848 136/300 Write block 3961 len 36864 data: 4 6 6 7 8 7 6 5 4
34849 137/300 Read block 4839 len 24576
34850 138/300 Read block 3449 len 40960
34851 139/300 Write block 2080 len 40960 data: 2 4 6 6 4 5 6 6 6 5
34852 140/300 Read block 2448 len 20480
34853 141/300 Write block 264 len 36864 data: 6 6 6 5 5 7 9 8 7
34854 142/300 Write block 1430 len 24576 data: 3 3 3 3 7 8
34855 143/300 Write block 3295 len 24576 data: 9 9 10 10 10 8
34856 144/300 Write block 119 len 16384 data: 7 8 8 10
34857 145/300 Read block 1561 len 8192
34858 146/300 Write block 1263 len 28672 data: 6 6 7 6 6 10 9
34859 147/300 Read block 2204 len 12288
34860 148/300 Write block 1364 len 32768 data: 3 4 5 7 9 10 10 11
34861 149/300 Read block 504 len 32768
34862 150/300 Read block 980 len 4096
34863 151/300 Write block 830 len 4096 data: 8
34864 152/300 Read block 4968 len 8192
34865 153/300 Flush
34866 154/300 Read block 3860 len 4096
34867 155/300 Write block 3538 len 20480 data: 5 5 5 6 6
34868 156/300 Read block 2159 len 4096
34869 157/300 Read block 1666 len 36864
34870 158/300 Write block 3799 len 12288 data: 8 9 9
34871 159/300 Write block 750 len 8192 data: 7 6
34872 160/300 Read block 205 len 36864
34873 161/300 Write block 465 len 8192 data: 5 5
34874 162/300 Write block 3570 len 20480 data: 6 6 5 6 9
34875 163/300 Write block 2825 len 24576 data: 5 6 6 6 6 5
34876 164/300 Write block 1804 len 24576 data: 2 2 2 3 3 3
34877 165/300 Flush
34878 166/300 Write block 2774 len 4096 data: 7
34879 167/300 Flush
34880 168/300 Read block 4929 len 24576
34881 169/300 Write block 4564 len 24576 data: 5 3 2 2 2 2
34882 170/300 Write block 450 len 20480 data: 4 4 2 3 4
34883 171/300 Read block 2149 len 16384
34884 172/300 Write block 1456 len 12288 data: 4 4 4
34885 173/300 Write block 3270 len 24576 data: 4 4 4 4 3 4
34886 174/300 Read block 3244 len 24576
34887 175/300 Read block 3614 len 24576
34888 176/300 Read block 4169 len 32768
34889 177/300 Write block 3279 len 12288 data: 8 7 6
34890 178/300 Write block 803 len 12288 data: 7 8 8
34891 179/300 Write block 2 len 36864 data: 2 4 6 5 5 5 6 5 4
34892 180/300 Read block 1878 len 32768
34893 181/300 Write block 382 len 40960 data: 6 8 8 7 8 6 6 7 8 9
34894 182/300 Write block 2591 len 40960 data: 7 7 6 5 5 5 5 5 5 6
34895 183/300 Read block 24 len 32768
34896 184/300 Read block 3352 len 20480
34897 185/300 Read block 3067 len 32768
34898 186/300 Write block 672 len 4096 data: 6
34899 187/300 Read block 4150 len 36864
34900 188/300 Read block 3081 len 12288
34901 189/300 Write block 2330 len 4096 data: 7
34902 190/300 Write block 89 len 20480 data: 7 7 7 5 5
34903 191/300 Read block 947 len 12288
34904 192/300 Read block 1726 len 20480
34905 193/300 Write block 3015 len 8192 data: 5 3
34906 194/300 Read block 2079 len 40960
34907 195/300 Write block 3294 len 12288 data: 8 10 10
34908 196/300 Write block 14 len 20480 data: 5 5 5 5 5
34909 197/300 Write block 3775 len 36864 data: 6 5 5 6 7 8 9 11 11
34910 198/300 Read block 4892 len 28672
34911 199/300 Write block 3566 len 8192 data: 4 3
34912 200/300 Read block 1683 len 12288
34913 201/300 Read block 3776 len 4096
34914 202/300 Read block 111 len 32768
34915 203/300 Read block 2821 len 4096
34916 204/300 Write block 398 len 20480 data: 10 10 9 9 8
34917 205/300 Read block 3699 len 12288
34918 206/300 Write block 3405 len 24576 data: 3 5 5 5 7 5
34919 207/300 Read block 3391 len 36864
34920 208/300 Read block 3865 len 20480
34921 209/300 Write block 2546 len 12288 data: 6 8 10
34922 210/300 Write block 336 len 12288 data: 6 8 8
34923 211/300 Flush
34924 212/300 Write block 1993 len 4096 data: 5
34925 213/300 Write block 4248 len 16384 data: 4 4 3 3
34926 214/300 Read block 4924 len 40960
34927 215/300 Write block 4974 len 36864 data: 4 6 5 6 5 6 7 7 6
34928 216/300 Read block 2306 len 4096
34929 217/300 Flush
34930 218/300 Flush
34931 219/300 Read block 4707 len 36864
34932 220/300 Read block 3954 len 36864
34933 221/300 Write block 4265 len 4096 data: 5
34934 222/300 Read block 1774 len 4096
34935 223/300 Flush
34936 224/300 Write block 1096 len 4096 data: 9
34937 225/300 Write block 1646 len 20480 data: 6 7 5 5 4
34938 226/300 Write block 2187 len 36864 data: 4 4 4 6 5 5 5 5 4
34939 227/300 Read block 1495 len 12288
34940 228/300 Write block 272 len 4096 data: 8
34941 229/300 Read block 996 len 8192
34942 230/300 Read block 3339 len 28672
34943 231/300 Read block 3744 len 36864
34944 232/300 Read block 2718 len 20480
34945 233/300 Read block 3263 len 8192
34946 234/300 Write block 13 len 40960 data: 7 6 6 6 6 6 4 4 5 5
34947 235/300 Write block 3371 len 20480 data: 3 3 3 4 6
34948 236/300 Read block 462 len 36864
34949 237/300 Read block 2245 len 8192
34950 238/300 Write block 1089 len 8192 data: 6 5
34951 239/300 Read block 2503 len 16384
34952 240/300 Read block 4118 len 20480
34953 241/300 Write block 2665 len 40960 data: 6 5 4 4 6 7 6 6 6 6
34954 242/300 Read block 861 len 12288
34955 243/300 Read block 4355 len 24576
34956 244/300 Write block 2076 len 12288 data: 7 5 5
34957 245/300 Write block 4357 len 40960 data: 5 4 4 5 7 8 9 10 9 9
34958 246/300 Read block 3800 len 12288
34959 247/300 Write block 3949 len 32768 data: 6 4 5 5 5 3 3 4
34960 248/300 Write block 4939 len 20480 data: 3 3 3 5 4
34961 249/300 Write block 4281 len 20480 data: 4 5 5 5 4
34962 250/300 Write block 1738 len 32768 data: 7 7 6 7 8 8 9 9
34963 251/300 Write block 2171 len 32768 data: 8 8 7 5 5 5 6 5
34964 252/300 Write block 4404 len 4096 data: 6
34965 253/300 Read block 4325 len 16384
34966 254/300 Write block 1137 len 28672 data: 8 8 7 7 8 8 8
34967 255/300 Write block 1780 len 20480 data: 4 3 3 3 4
34968 256/300 Write block 1806 len 40960 data: 3 4 4 4 4 5 6 6 6 5
34969 257/300 Write block 1855 len 32768 data: 6 5 5 7 7 6 9 7
34970 258/300 Read block 4819 len 32768
34971 259/300 Read block 4795 len 20480
34972 260/300 Flush
34973 261/300 Flush
34974 262/300 Read block 2881 len 32768
34975 263/300 Write block 2081 len 20480 data: 5 7 7 5 6
34976 264/300 Write block 3256 len 20480 data: 8 8 9 6 3
34977 265/300 Read block 842 len 32768
34978 266/300 Write block 2976 len 40960 data: 6 4 4 5 6 6 8 6 5 4
34979 267/300 Flush
34980 268/300 Write block 1243 len 28672 data: 5 7 9 9 9 5 4
34981 269/300 Write block 3104 len 4096 data: 12
34982 270/300 Read block 2722 len 16384
34983 271/300 Read block 4246 len 40960
34984 272/300 Read block 4457 len 36864
34985 273/300 Read block 2557 len 12288
34986 274/300 Write block 708 len 16384 data: 10 7 7 6
34987 275/300 Write block 2037 len 40960 data: 6 7 7 7 8 6 7 8 8 8
34988 276/300 Write block 272 len 32768 data: 9 8 6 7 7 7 6 6
34989 277/300 Write block 4708 len 4096 data: 5
34990 278/300 Read block 3107 len 8192
34991 279/300 Read block 3158 len 4096
34992 280/300 Read block 2621 len 28672
34993 281/300 Read block 3213 len 8192
34994 282/300 Write block 702 len 40960 data: 10 11 9 7 8 9 11 8 8 7
34995 283/300 Write block 4730 len 20480 data: 7 7 8 9 7
34996 284/300 Write block 101 len 8192 data: 7 7
34997 285/300 Write block 4049 len 28672 data: 3 3 3 3 4 6 7
34998 286/300 Read block 4767 len 20480
34999 287/300 Read block 1068 len 40960
35000 288/300 Read block 1870 len 32768
35001 289/300 Read block 2575 len 12288
35002 290/300 Read block 450 len 36864
35003 291/300 Flush
35004 292/300 Flush
35005 293/300 Read block 2914 len 40960
35006 294/300 Flush
35007 295/300 Read block 212 len 36864
35008 296/300 Read block 1837 len 20480
35009 297/300 Write block 2087 len 20480 data: 7 7 6 6 5
35010 298/300 Flush
35011 299/300 Read block 2039 len 20480
35012 300/300 Write block 917 len 28672 data: 5 6 8 9 8 10 9
35013 301/300 Read block 2383 len 24576
35014 Main task has work to do, go find it
35015 got dsc Start(1)
35016 start 1
35017 Jan 17 21:18:28.119 INFO request completed, latency_us: 231, response_code: 204, uri: /start/cid/1, method: POST, req_id: 881cb0be-aed9-44e0-b2f9-d4a5bf3c3ace, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
35018 Replay: started 1, returned:Ok(())
35019 ----------------------------------------------------------------
35020 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
35021 States: Active Offline Active
35022 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35023 10056 Acked Read 5 Done Sent Done false
35024 10057 Acked Read 9 Done Sent Done false
35025 10058 Acked Read 9 Done Sent Done false
35026 10059 Acked Write 2 Done Sent Done false
35027 10060 Acked Flush 0 Done Sent Done false
35028 10061 Acked Write 5 Done Sent Done false
35029 10062 Acked Read 5 Done Sent Done false
35030 10063 Acked Read 1 Done Sent Done false
35031 10064 Acked Write 1 Done Sent Done false
35032 10065 Acked Flush 0 Done Sent Done false
35033 10066 Acked Write 7 Done Sent Done false
35034 10067 Acked Write 9 Done Sent Done false
35035 10068 Acked Write 4 Done Sent Done false
35036 10069 Acked Read 2 Done Sent Done false
35037 10070 Acked Write 8 Done Sent Done false
35038 10071 Acked Write 10 Done Sent Done false
35039 10072 Acked Read 1 Done Sent Done false
35040 10073 Acked Write 3 Done Sent Done false
35041 10074 Acked Read 10 Done Sent Done false
35042 10075 Acked Write 6 Done Sent Done false
35043 10076 Acked Read 5 Done Sent Done false
35044 10077 Acked Read 8 Done Sent Done false
35045 10078 Acked Read 10 Done Sent Done false
35046 10079 Acked Read 6 Done Sent Done false
35047 10080 Acked Read 9 Done Sent Done false
35048 10081 Acked Read 5 Done Sent Done false
35049 10082 Acked Write 6 Done Sent Done false
35050 10083 Acked Read 7 Done Sent Done false
35051 10084 Acked Read 3 Done Sent Done false
35052 10085 Acked Write 9 Done Sent Done false
35053 10086 Acked Read 6 Done Sent Done false
35054 10087 Acked Write 3 Done Sent Done false
35055 10088 Acked Read 5 Done Sent Done false
35056 10089 Acked Write 1 Done Sent Done false
35057 10090 Acked Write 3 Done Sent Done false
35058 10091 Acked Write 3 Done Sent Done false
35059 10092 Acked Read 8 Done Sent Done false
35060 10093 Acked Write 10 Done Sent Done false
35061 10094 Acked Read 9 Done Sent Done false
35062 10095 Acked Read 2 Done Sent Done false
35063 10096 Acked Write 7 Done Sent Done false
35064 10097 Acked Write 4 Done Sent Done false
35065 10098 Acked Read 10 Done Sent Done false
35066 10099 Acked Flush 0 Done Sent Done false
35067 10100 Acked Write 3 Done Sent Done false
35068 10101 Acked Write 10 Done Sent Done false
35069 10102 Acked Read 4 Done Sent Done false
35070 10103 Acked Write 6 Done Sent Done false
35071 10104 Acked Write 9 Done Sent Done false
35072 10105 Acked Write 4 Done Sent Done false
35073 10106 Acked Read 8 Done Sent Done false
35074 10107 Acked Read 8 Done Sent Done false
35075 10108 Acked Write 7 Done Sent Done false
35076 10109 Acked Flush 0 Done Sent Done false
35077 10110 Acked Write 6 Done Sent Done false
35078 10111 Acked Write 5 Done Sent Done false
35079 10112 Acked Write 7 Done Sent Done false
35080 10113 Acked Flush 0 Done Sent Done false
35081 10114 Acked Write 4 Done Sent Done false
35082 10115 Acked Write 5 Done Sent Done false
35083 10116 Acked Write 1 Done Sent Done false
35084 10117 Acked Read 6 Done Sent Done false
35085 10118 Acked Write 3 Done Sent Done false
35086 10119 Acked Write 6 Done Sent Done false
35087 10120 Acked Read 1 Done Sent Done false
35088 10121 Acked Write 6 Done Sent Done false
35089 10122 Acked Flush 0 Done Sent Done false
35090 10123 Acked Write 7 Done Sent Done false
35091 10124 Acked Read 6 Done Sent Done false
35092 10125 Acked Write 1 Done Sent Done false
35093 10126 Acked Flush 0 Done Sent Done false
35094 10127 Acked Write 9 Done Sent Done false
35095 10128 Acked Write 6 Done Sent Done false
35096 10129 Acked Write 1 Done Sent Done false
35097 10130 Acked Read 10 Done Sent Done false
35098 10131 Acked Write 10 Done Sent Done false
35099 10132 Acked Read 10 Done Sent Done false
35100 10133 Acked Write 8 Done Sent Done false
35101 10134 Acked Write 2 Done Sent Done false
35102 10135 Acked Read 5 Done Sent Done false
35103 10136 Acked Write 2 Done Sent Done false
35104 10137 Acked Write 6 Done Sent Done false
35105 10138 Acked Write 4 Done Sent Done false
35106 10139 Acked Read 7 Done Sent Done false
35107 10140 Acked Write 6 Done Sent Done false
35108 10141 Acked Write 9 Done Sent Done false
35109 10142 Acked Read 10 Done Sent Done false
35110 10143 Acked Flush 0 Done Sent Done false
35111 10144 Acked Write 7 Done Sent Done false
35112 10145 Acked Read 4 Done Sent Done false
35113 10146 Acked Read 1 Done Sent Done false
35114 10147 Acked Read 8 Done Sent Done false
35115 10148 Acked Read 7 Done Sent Done false
35116 10149 Acked Write 6 Done Sent Done false
35117 10150 Acked Read 10 Done Sent Done false
35118 10151 Acked Write 10 Done Sent Done false
35119 10152 Acked Flush 0 Done Sent Done false
35120 10153 Acked Read 1 Done Sent Done false
35121 10154 Acked Write 6 Done Sent Done false
35122 10155 Acked Flush 0 Done Sent Done false
35123 10156 Acked Read 6 Done Sent Done false
35124 10157 Acked Read 4 Done Sent Done false
35125 10158 Acked Write 5 Done Sent Done false
35126 10159 Acked Write 3 Done Sent Done false
35127 10160 Acked Write 3 Done Sent Done false
35128 10161 Acked Read 7 Done Sent Done false
35129 10162 Acked Read 8 Done Sent Done false
35130 10163 Acked Read 6 Done Sent Done false
35131 10164 Acked Write 2 Done Sent Done false
35132 10165 Acked Write 5 Done Sent Done false
35133 10166 Acked Write 2 Done Sent Done false
35134 10167 Acked Flush 0 Done Sent Done false
35135 10168 Acked Read 6 Done Sent Done false
35136 10169 Acked Write 2 Done Sent Done false
35137 10170 Acked Read 5 Done Sent Done false
35138 10171 Acked Read 8 Done Sent Done false
35139 10172 Acked Write 6 Done Sent Done false
35140 10173 Acked Write 5 Done Sent Done false
35141 10174 Acked Read 8 Done Sent Done false
35142 10175 Acked Read 7 Done Sent Done false
35143 10176 Acked Read 3 Done Sent Done false
35144 10177 Acked Write 4 Done Sent Done false
35145 10178 Acked Write 5 Done Sent Done false
35146 10179 Acked Read 2 Done Sent Done false
35147 10180 Acked Read 5 Done Sent Done false
35148 10181 Acked Read 1 Done Sent Done false
35149 10182 Acked Write 2 Done Sent Done false
35150 10183 Acked Write 5 Done Sent Done false
35151 10184 Acked Read 8 Done Sent Done false
35152 10185 Acked Write 8 Done Sent Done false
35153 10186 Acked Read 8 Done Sent Done false
35154 10187 Acked Read 2 Done Sent Done false
35155 10188 Acked Read 8 Done Sent Done false
35156 10189 Acked Read 9 Done Sent Done false
35157 10190 Acked Write 8 Done Sent Done false
35158 10191 Acked Write 9 Done Sent Done false
35159 10192 Acked Read 6 Done Sent Done false
35160 10193 Acked Read 10 Done Sent Done false
35161 10194 Acked Write 10 Done Sent Done false
35162 10195 Acked Read 5 Done Sent Done false
35163 10196 Acked Write 9 Done Sent Done false
35164 10197 Acked Write 6 Done Sent Done false
35165 10198 Acked Write 6 Done Sent Done false
35166 10199 Acked Write 4 Done Sent Done false
35167 10200 Acked Read 2 Done Sent Done false
35168 10201 Acked Write 7 Done Sent Done false
35169 10202 Acked Read 3 Done Sent Done false
35170 10203 Acked Write 8 Done Sent Done false
35171 10204 Acked Read 8 Done Sent Done false
35172 10205 Acked Read 1 Done Sent Done false
35173 10206 Acked Write 1 Done Sent Done false
35174 10207 Acked Read 2 Done Sent Done false
35175 10208 Acked Flush 0 Done Sent Done false
35176 10209 Acked Read 1 Done Sent Done false
35177 10210 Acked Write 5 Done Sent Done false
35178 10211 Acked Read 1 Done Sent Done false
35179 10212 Acked Read 9 Done Sent Done false
35180 10213 Acked Write 3 Done Sent Done false
35181 10214 Acked Write 2 Done Sent Done false
35182 10215 Acked Read 9 Done Sent Done false
35183 10216 Acked Write 2 Done Sent Done false
35184 10217 Acked Write 5 Done Sent Done false
35185 10218 Acked Write 6 Done Sent Done false
35186 10219 Acked Write 6 Done Sent Done false
35187 10220 Acked Flush 0 Done Sent Done false
35188 10221 Acked Write 1 Done Sent Done false
35189 10222 Acked Flush 0 Done Sent Done false
35190 10223 Acked Read 6 Done Sent Done false
35191 10224 Acked Write 6 Done Sent Done false
35192 10225 Acked Write 5 Done Sent Done false
35193 10226 Acked Read 4 Done Sent Done false
35194 10227 Acked Write 3 Done Sent Done false
35195 10228 Acked Write 6 Done Sent Done false
35196 10229 Acked Read 6 Done Sent Done false
35197 10230 Acked Read 6 Done Sent Done false
35198 10231 Acked Read 8 Done Sent Done false
35199 10232 Acked Write 3 Done Sent Done false
35200 10233 Acked Write 3 Done Sent Done false
35201 10234 Acked Write 9 Done Sent Done false
35202 10235 Acked Read 8 Done Sent Done false
35203 10236 Acked Write 10 Done Sent Done false
35204 10237 Acked Write 10 Done Sent Done false
35205 10238 Acked Read 8 Done Sent Done false
35206 10239 Acked Read 5 Done Sent Done false
35207 10240 Acked Read 8 Done Sent Done false
35208 10241 Acked Write 1 Done Sent Done false
35209 10242 Acked Read 9 Done Sent Done false
35210 10243 Acked Read 3 Done Sent Done false
35211 10244 Acked Write 1 Done Sent Done false
35212 10245 Acked Write 5 Done Sent Done false
35213 10246 Acked Read 3 Done Sent Done false
35214 10247 Acked Read 5 Done Sent Done false
35215 10248 Acked Write 2 Done Sent Done false
35216 10249 Acked Read 10 Done Sent Done false
35217 10250 Acked Write 3 Done Sent Done false
35218 10251 Acked Write 5 Done Sent Done false
35219 10252 Acked Write 9 Done Sent Done false
35220 10253 Acked Read 7 Done Sent Done false
35221 10254 Acked Write 2 Done Sent Done false
35222 10255 Acked Read 3 Done Sent Done false
35223 10256 Acked Read 1 Done Sent Done false
35224 10257 Acked Read 8 Done Sent Done false
35225 10258 Acked Read 1 Done Sent Done false
35226 10259 Acked Write 5 Done Sent Done false
35227 10260 Acked Read 3 Done Sent Done false
35228 10261 Acked Write 6 Done Sent Done false
35229 10262 Acked Read 9 Done Sent Done false
35230 10263 Acked Read 5 Done Sent Done false
35231 10264 Acked Write 3 Done Sent Done false
35232 10265 Acked Write 3 Done Sent Done false
35233 10266 Acked Flush 0 Done Sent Done false
35234 10267 Acked Write 1 Done Sent Done false
35235 10268 Acked Write 4 Done Sent Done false
35236 10269 Acked Read 10 Done Sent Done false
35237 10270 Acked Write 9 Done Sent Done false
35238 10271 Acked Read 1 Done Sent Done false
35239 10272 Acked Flush 0 Done Sent Done false
35240 10273 Acked Flush 0 Done Sent Done false
35241 10274 Acked Read 9 Done Sent Done false
35242 10275 Acked Read 9 Done Sent Done false
35243 10276 Acked Write 1 Done Sent Done false
35244 10277 Acked Read 1 Done Sent Done false
35245 10278 Acked Flush 0 Done Sent Done false
35246 10279 Acked Write 1 Done Sent Done false
35247 10280 Acked Write 5 Done Sent Done false
35248 10281 Acked Write 9 Done Sent Done false
35249 10282 Acked Read 3 Done Sent Done false
35250 10283 Acked Write 1 Done Sent Done false
35251 10284 Acked Read 2 Done Sent Done false
35252 10285 Acked Read 7 Done Sent Done false
35253 10286 Acked Read 9 Done Sent Done false
35254 10287 Acked Read 5 Done Sent Done false
35255 10288 Acked Read 2 Done Sent Done false
35256 10289 Acked Write 10 Done Sent Done false
35257 10290 Acked Write 5 Done Sent Done false
35258 10291 Acked Read 9 Done Sent Done false
35259 10292 Acked Read 2 Done Sent Done false
35260 10293 Acked Write 2 Done Sent Done false
35261 10294 Acked Read 4 Done Sent Done false
35262 10295 Acked Read 5 Done Sent Done false
35263 10296 Acked Write 10 Done Sent Done false
35264 10297 Acked Read 3 Done Sent Done false
35265 10298 Acked Read 6 Done Sent Done false
35266 10299 Acked Write 3 Done Sent Done false
35267 10300 Acked Write 10 Done Sent Done false
35268 10301 Acked Read 3 Done Sent Done false
35269 10302 Acked Write 8 Done Sent Done false
35270 10303 Acked Write 5 Done Sent Done false
35271 10304 Acked Write 5 Done Sent Done false
35272 10305 Acked Write 8 Done Sent Done false
35273 10306 Acked Write 8 Done Sent Done false
35274 10307 Acked Write 1 Done Sent Done false
35275 10308 Acked Read 4 Done Sent Done false
35276 10309 Acked Write 7 Done Sent Done false
35277 10310 Acked Write 5 Done Sent Done false
35278 10311 Acked Write 10 Done Sent Done false
35279 10312 Acked Write 8 Done Sent Done false
35280 10313 Acked Read 8 Done Sent Done false
35281 10314 Acked Read 5 Done Sent Done false
35282 10315 Acked Flush 0 Done Sent Done false
35283 10316 Acked Flush 0 Done Sent Done false
35284 10317 Acked Read 8 Done Sent Done false
35285 10318 Acked Write 5 Done Sent Done false
35286 10319 Acked Write 5 Done Sent Done false
35287 10320 Acked Read 8 Done Sent Done false
35288 10321 Acked Write 10 Done Sent Done false
35289 10322 Acked Flush 0 Done Sent Done false
35290 10323 Acked Write 7 Done Sent Done false
35291 10324 Acked Write 1 Done Sent Done false
35292 10325 Acked Read 4 Done Sent Done false
35293 10326 Acked Read 10 Done Sent Done false
35294 10327 Acked Read 9 Done Sent Done false
35295 10328 Acked Read 3 Done Sent Done false
35296 10329 Acked Write 4 Done Sent Done false
35297 10330 Acked Write 10 Done Sent Done false
35298 10331 Acked Write 8 Done Sent Done false
35299 10332 Acked Write 1 Done Sent Done false
35300 10333 Acked Read 2 Done Sent Done false
35301 10334 Acked Read 1 Done Sent Done false
35302 10335 Acked Read 7 Done Sent Done false
35303 10336 Acked Read 2 Done Sent Done false
35304 10337 Acked Write 10 Done Sent Done false
35305 10338 Acked Write 5 Done Sent Done false
35306 10339 Acked Write 2 Done Sent Done false
35307 10340 Acked Write 7 Done Sent Done false
35308 10341 Acked Read 5 Done Sent Done false
35309 10342 Acked Read 10 Done Sent Done false
35310 10343 Acked Read 8 Done Sent Done false
35311 10344 Acked Read 3 Done Sent Done false
35312 10345 Acked Read 9 Done Sent Done false
35313 10346 Acked Flush 0 Done Sent Done false
35314 10347 Acked Flush 0 Done Sent Done false
35315 10348 Acked Read 10 Done Sent Done false
35316 10349 Acked Flush 0 Done Sent Done false
35317 10350 Acked Read 9 Done Sent Done false
35318 10351 Acked Read 5 Done Sent Done false
35319 10352 Acked Write 5 Done Sent Done false
35320 10353 Acked Flush 0 Done Sent Done false
35321 10354 Acked Read 5 Done Sent Done false
35322 10355 Acked Write 7 Done Sent Done false
35323 10356 Acked Read 6 Done Sent Done false
35324 STATES DS:0 DS:1 DS:2 TOTAL
35325 Sent 0 301 0 301
35326 Done 301 0 301 602
35327 Skipped 0 0 0 0
35328 Error 0 0 0 0
35329 Last Flush: 10353 10055 10353
35330 Downstairs last five completed: 10055 10054 10053 10052 10051
35331 Upstairs last five completed: 10356 10355 10354 10353 10352
35332 CLIENT: Up:0 ds:301 act:2
35333 ----------------------------------------------------------------
35334 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
35335 States: Active Offline Active
35336 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35337 10056 Acked Read 5 Done Sent Done false
35338 10057 Acked Read 9 Done Sent Done false
35339 10058 Acked Read 9 Done Sent Done false
35340 10059 Acked Write 2 Done Sent Done false
35341 10060 Acked Flush 0 Done Sent Done false
35342 10061 Acked Write 5 Done Sent Done false
35343 10062 Acked Read 5 Done Sent Done false
35344 10063 Acked Read 1 Done Sent Done false
35345 10064 Acked Write 1 Done Sent Done false
35346 10065 Acked Flush 0 Done Sent Done false
35347 10066 Acked Write 7 Done Sent Done false
35348 10067 Acked Write 9 Done Sent Done false
35349 10068 Acked Write 4 Done Sent Done false
35350 10069 Acked Read 2 Done Sent Done false
35351 10070 Acked Write 8 Done Sent Done false
35352 10071 Acked Write 10 Done Sent Done false
35353 10072 Acked Read 1 Done Sent Done false
35354 10073 Acked Write 3 Done Sent Done false
35355 10074 Acked Read 10 Done Sent Done false
35356 10075 Acked Write 6 Done Sent Done false
35357 10076 Acked Read 5 Done Sent Done false
35358 10077 Acked Read 8 Done Sent Done false
35359 10078 Acked Read 10 Done Sent Done false
35360 10079 Acked Read 6 Done Sent Done false
35361 10080 Acked Read 9 Done Sent Done false
35362 10081 Acked Read 5 Done Sent Done false
35363 10082 Acked Write 6 Done Sent Done false
35364 10083 Acked Read 7 Done Sent Done false
35365 10084 Acked Read 3 Done Sent Done false
35366 10085 Acked Write 9 Done Sent Done false
35367 10086 Acked Read 6 Done Sent Done false
35368 10087 Acked Write 3 Done Sent Done false
35369 10088 Acked Read 5 Done Sent Done false
35370 10089 Acked Write 1 Done Sent Done false
35371 10090 Acked Write 3 Done Sent Done false
35372 10091 Acked Write 3 Done Sent Done false
35373 10092 Acked Read 8 Done Sent Done false
35374 10093 Acked Write 10 Done Sent Done false
35375 10094 Acked Read 9 Done Sent Done false
35376 10095 Acked Read 2 Done Sent Done false
35377 10096 Acked Write 7 Done Sent Done false
35378 10097 Acked Write 4 Done Sent Done false
35379 10098 Acked Read 10 Done Sent Done false
35380 10099 Acked Flush 0 Done Sent Done false
35381 10100 Acked Write 3 Done Sent Done false
35382 10101 Acked Write 10 Done Sent Done false
35383 10102 Acked Read 4 Done Sent Done false
35384 10103 Acked Write 6 Done Sent Done false
35385 10104 Acked Write 9 Done Sent Done false
35386 10105 Acked Write 4 Done Sent Done false
35387 10106 Acked Read 8 Done Sent Done false
35388 10107 Acked Read 8 Done Sent Done false
35389 10108 Acked Write 7 Done Sent Done false
35390 10109 Acked Flush 0 Done Sent Done false
35391 10110 Acked Write 6 Done Sent Done false
35392 10111 Acked Write 5 Done Sent Done false
35393 10112 Acked Write 7 Done Sent Done false
35394 10113 Acked Flush 0 Done Sent Done false
35395 10114 Acked Write 4 Done Sent Done false
35396 10115 Acked Write 5 Done Sent Done false
35397 10116 Acked Write 1 Done Sent Done false
35398 10117 Acked Read 6 Done Sent Done false
35399 10118 Acked Write 3 Done Sent Done false
35400 10119 Acked Write 6 Done Sent Done false
35401 10120 Acked Read 1 Done Sent Done false
35402 10121 Acked Write 6 Done Sent Done false
35403 10122 Acked Flush 0 Done Sent Done false
35404 10123 Acked Write 7 Done Sent Done false
35405 10124 Acked Read 6 Done Sent Done false
35406 10125 Acked Write 1 Done Sent Done false
35407 10126 Acked Flush 0 Done Sent Done false
35408 10127 Acked Write 9 Done Sent Done false
35409 10128 Acked Write 6 Done Sent Done false
35410 10129 Acked Write 1 Done Sent Done false
35411 10130 Acked Read 10 Done Sent Done false
35412 10131 Acked Write 10 Done Sent Done false
35413 10132 Acked Read 10 Done Sent Done false
35414 10133 Acked Write 8 Done Sent Done false
35415 10134 Acked Write 2 Done Sent Done false
35416 10135 Acked Read 5 Done Sent Done false
35417 10136 Acked Write 2 Done Sent Done false
35418 10137 Acked Write 6 Done Sent Done false
35419 10138 Acked Write 4 Done Sent Done false
35420 10139 Acked Read 7 Done Sent Done false
35421 10140 Acked Write 6 Done Sent Done false
35422 10141 Acked Write 9 Done Sent Done false
35423 10142 Acked Read 10 Done Sent Done false
35424 10143 Acked Flush 0 Done Sent Done false
35425 10144 Acked Write 7 Done Sent Done false
35426 10145 Acked Read 4 Done Sent Done false
35427 10146 Acked Read 1 Done Sent Done false
35428 10147 Acked Read 8 Done Sent Done false
35429 10148 Acked Read 7 Done Sent Done false
35430 10149 Acked Write 6 Done Sent Done false
35431 10150 Acked Read 10 Done Sent Done false
35432 10151 Acked Write 10 Done Sent Done false
35433 10152 Acked Flush 0 Done Sent Done false
35434 10153 Acked Read 1 Done Sent Done false
35435 10154 Acked Write 6 Done Sent Done false
35436 10155 Acked Flush 0 Done Sent Done false
35437 10156 Acked Read 6 Done Sent Done false
35438 10157 Acked Read 4 Done Sent Done false
35439 10158 Acked Write 5 Done Sent Done false
35440 10159 Acked Write 3 Done Sent Done false
35441 10160 Acked Write 3 Done Sent Done false
35442 10161 Acked Read 7 Done Sent Done false
35443 10162 Acked Read 8 Done Sent Done false
35444 10163 Acked Read 6 Done Sent Done false
35445 10164 Acked Write 2 Done Sent Done false
35446 10165 Acked Write 5 Done Sent Done false
35447 10166 Acked Write 2 Done Sent Done false
35448 10167 Acked Flush 0 Done Sent Done false
35449 10168 Acked Read 6 Done Sent Done false
35450 10169 Acked Write 2 Done Sent Done false
35451 10170 Acked Read 5 Done Sent Done false
35452 10171 Acked Read 8 Done Sent Done false
35453 10172 Acked Write 6 Done Sent Done false
35454 10173 Acked Write 5 Done Sent Done false
35455 10174 Acked Read 8 Done Sent Done false
35456 10175 Acked Read 7 Done Sent Done false
35457 10176 Acked Read 3 Done Sent Done false
35458 10177 Acked Write 4 Done Sent Done false
35459 10178 Acked Write 5 Done Sent Done false
35460 10179 Acked Read 2 Done Sent Done false
35461 10180 Acked Read 5 Done Sent Done false
35462 10181 Acked Read 1 Done Sent Done false
35463 10182 Acked Write 2 Done Sent Done false
35464 10183 Acked Write 5 Done Sent Done false
35465 10184 Acked Read 8 Done Sent Done false
35466 10185 Acked Write 8 Done Sent Done false
35467 10186 Acked Read 8 Done Sent Done false
35468 10187 Acked Read 2 Done Sent Done false
35469 10188 Acked Read 8 Done Sent Done false
35470 10189 Acked Read 9 Done Sent Done false
35471 10190 Acked Write 8 Done Sent Done false
35472 10191 Acked Write 9 Done Sent Done false
35473 10192 Acked Read 6 Done Sent Done false
35474 10193 Acked Read 10 Done Sent Done false
35475 10194 Acked Write 10 Done Sent Done false
35476 10195 Acked Read 5 Done Sent Done false
35477 10196 Acked Write 9 Done Sent Done false
35478 10197 Acked Write 6 Done Sent Done false
35479 10198 Acked Write 6 Done Sent Done false
35480 10199 Acked Write 4 Done Sent Done false
35481 10200 Acked Read 2 Done Sent Done false
35482 10201 Acked Write 7 Done Sent Done false
35483 10202 Acked Read 3 Done Sent Done false
35484 10203 Acked Write 8 Done Sent Done false
35485 10204 Acked Read 8 Done Sent Done false
35486 10205 Acked Read 1 Done Sent Done false
35487 10206 Acked Write 1 Done Sent Done false
35488 10207 Acked Read 2 Done Sent Done false
35489 10208 Acked Flush 0 Done Sent Done false
35490 10209 Acked Read 1 Done Sent Done false
35491 10210 Acked Write 5 Done Sent Done false
35492 10211 Acked Read 1 Done Sent Done false
35493 10212 Acked Read 9 Done Sent Done false
35494 10213 Acked Write 3 Done Sent Done false
35495 10214 Acked Write 2 Done Sent Done false
35496 10215 Acked Read 9 Done Sent Done false
35497 10216 Acked Write 2 Done Sent Done false
35498 10217 Acked Write 5 Done Sent Done false
35499 10218 Acked Write 6 Done Sent Done false
35500 10219 Acked Write 6 Done Sent Done false
35501 10220 Acked Flush 0 Done Sent Done false
35502 10221 Acked Write 1 Done Sent Done false
35503 10222 Acked Flush 0 Done Sent Done false
35504 10223 Acked Read 6 Done Sent Done false
35505 10224 Acked Write 6 Done Sent Done false
35506 10225 Acked Write 5 Done Sent Done false
35507 10226 Acked Read 4 Done Sent Done false
35508 10227 Acked Write 3 Done Sent Done false
35509 10228 Acked Write 6 Done Sent Done false
35510 10229 Acked Read 6 Done Sent Done false
35511 10230 Acked Read 6 Done Sent Done false
35512 10231 Acked Read 8 Done Sent Done false
35513 10232 Acked Write 3 Done Sent Done false
35514 10233 Acked Write 3 Done Sent Done false
35515 10234 Acked Write 9 Done Sent Done false
35516 10235 Acked Read 8 Done Sent Done false
35517 10236 Acked Write 10 Done Sent Done false
35518 10237 Acked Write 10 Done Sent Done false
35519 10238 Acked Read 8 Done Sent Done false
35520 10239 Acked Read 5 Done Sent Done false
35521 10240 Acked Read 8 Done Sent Done false
35522 10241 Acked Write 1 Done Sent Done false
35523 10242 Acked Read 9 Done Sent Done false
35524 10243 Acked Read 3 Done Sent Done false
35525 10244 Acked Write 1 Done Sent Done false
35526 10245 Acked Write 5 Done Sent Done false
35527 10246 Acked Read 3 Done Sent Done false
35528 10247 Acked Read 5 Done Sent Done false
35529 10248 Acked Write 2 Done Sent Done false
35530 10249 Acked Read 10 Done Sent Done false
35531 10250 Acked Write 3 Done Sent Done false
35532 10251 Acked Write 5 Done Sent Done false
35533 10252 Acked Write 9 Done Sent Done false
35534 10253 Acked Read 7 Done Sent Done false
35535 10254 Acked Write 2 Done Sent Done false
35536 10255 Acked Read 3 Done Sent Done false
35537 10256 Acked Read 1 Done Sent Done false
35538 10257 Acked Read 8 Done Sent Done false
35539 10258 Acked Read 1 Done Sent Done false
35540 10259 Acked Write 5 Done Sent Done false
35541 10260 Acked Read 3 Done Sent Done false
35542 10261 Acked Write 6 Done Sent Done false
35543 10262 Acked Read 9 Done Sent Done false
35544 10263 Acked Read 5 Done Sent Done false
35545 10264 Acked Write 3 Done Sent Done false
35546 10265 Acked Write 3 Done Sent Done false
35547 10266 Acked Flush 0 Done Sent Done false
35548 10267 Acked Write 1 Done Sent Done false
35549 10268 Acked Write 4 Done Sent Done false
35550 10269 Acked Read 10 Done Sent Done false
35551 10270 Acked Write 9 Done Sent Done false
35552 10271 Acked Read 1 Done Sent Done false
35553 10272 Acked Flush 0 Done Sent Done false
35554 10273 Acked Flush 0 Done Sent Done false
35555 10274 Acked Read 9 Done Sent Done false
35556 10275 Acked Read 9 Done Sent Done false
35557 10276 Acked Write 1 Done Sent Done false
35558 10277 Acked Read 1 Done Sent Done false
35559 10278 Acked Flush 0 Done Sent Done false
35560 10279 Acked Write 1 Done Sent Done false
35561 10280 Acked Write 5 Done Sent Done false
35562 10281 Acked Write 9 Done Sent Done false
35563 10282 Acked Read 3 Done Sent Done false
35564 10283 Acked Write 1 Done Sent Done false
35565 10284 Acked Read 2 Done Sent Done false
35566 10285 Acked Read 7 Done Sent Done false
35567 10286 Acked Read 9 Done Sent Done false
35568 10287 Acked Read 5 Done Sent Done false
35569 10288 Acked Read 2 Done Sent Done false
35570 10289 Acked Write 10 Done Sent Done false
35571 10290 Acked Write 5 Done Sent Done false
35572 10291 Acked Read 9 Done Sent Done false
35573 10292 Acked Read 2 Done Sent Done false
35574 10293 Acked Write 2 Done Sent Done false
35575 10294 Acked Read 4 Done Sent Done false
35576 10295 Acked Read 5 Done Sent Done false
35577 10296 Acked Write 10 Done Sent Done false
35578 10297 Acked Read 3 Done Sent Done false
35579 10298 Acked Read 6 Done Sent Done false
35580 10299 Acked Write 3 Done Sent Done false
35581 10300 Acked Write 10 Done Sent Done false
35582 10301 Acked Read 3 Done Sent Done false
35583 10302 Acked Write 8 Done Sent Done false
35584 10303 Acked Write 5 Done Sent Done false
35585 10304 Acked Write 5 Done Sent Done false
35586 10305 Acked Write 8 Done Sent Done false
35587 10306 Acked Write 8 Done Sent Done false
35588 10307 Acked Write 1 Done Sent Done false
35589 10308 Acked Read 4 Done Sent Done false
35590 10309 Acked Write 7 Done Sent Done false
35591 10310 Acked Write 5 Done Sent Done false
35592 10311 Acked Write 10 Done Sent Done false
35593 10312 Acked Write 8 Done Sent Done false
35594 10313 Acked Read 8 Done Sent Done false
35595 10314 Acked Read 5 Done Sent Done false
35596 10315 Acked Flush 0 Done Sent Done false
35597 10316 Acked Flush 0 Done Sent Done false
35598 10317 Acked Read 8 Done Sent Done false
35599 10318 Acked Write 5 Done Sent Done false
35600 10319 Acked Write 5 Done Sent Done false
35601 10320 Acked Read 8 Done Sent Done false
35602 10321 Acked Write 10 Done Sent Done false
35603 10322 Acked Flush 0 Done Sent Done false
35604 10323 Acked Write 7 Done Sent Done false
35605 10324 Acked Write 1 Done Sent Done false
35606 10325 Acked Read 4 Done Sent Done false
35607 10326 Acked Read 10 Done Sent Done false
35608 10327 Acked Read 9 Done Sent Done false
35609 10328 Acked Read 3 Done Sent Done false
35610 10329 Acked Write 4 Done Sent Done false
35611 10330 Acked Write 10 Done Sent Done false
35612 10331 Acked Write 8 Done Sent Done false
35613 10332 Acked Write 1 Done Sent Done false
35614 10333 Acked Read 2 Done Sent Done false
35615 10334 Acked Read 1 Done Sent Done false
35616 10335 Acked Read 7 Done Sent Done false
35617 10336 Acked Read 2 Done Sent Done false
35618 10337 Acked Write 10 Done Sent Done false
35619 10338 Acked Write 5 Done Sent Done false
35620 10339 Acked Write 2 Done Sent Done false
35621 10340 Acked Write 7 Done Sent Done false
35622 10341 Acked Read 5 Done Sent Done false
35623 10342 Acked Read 10 Done Sent Done false
35624 10343 Acked Read 8 Done Sent Done false
35625 10344 Acked Read 3 Done Sent Done false
35626 10345 Acked Read 9 Done Sent Done false
35627 10346 Acked Flush 0 Done Sent Done false
35628 10347 Acked Flush 0 Done Sent Done false
35629 10348 Acked Read 10 Done Sent Done false
35630 10349 Acked Flush 0 Done Sent Done false
35631 10350 Acked Read 9 Done Sent Done false
35632 10351 Acked Read 5 Done Sent Done false
35633 10352 Acked Write 5 Done Sent Done false
35634 10353 Acked Flush 0 Done Sent Done false
35635 10354 Acked Read 5 Done Sent Done false
35636 10355 Acked Write 7 Done Sent Done false
35637 10356 Acked Read 6 Done Sent Done false
35638 10357 Acked Flush 0 Done Sent Done false
35639 STATES DS:0 DS:1 DS:2 TOTAL
35640 Sent 0 302 0 302
35641 Done 302 0 302 604
35642 Skipped 0 0 0 0
35643 Error 0 0 0 0
35644 Last Flush: 10357 10055 10357
35645 Downstairs last five completed: 10055 10054 10053 10052 10051
35646 Upstairs last five completed: 10357 10356 10355 10354 10353
35647 CLIENT: Up:0 ds:302 act:2
356482025-01-17T21:18:33.885ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
356492025-01-17T21:18:33.885ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
356502025-01-17T21:18:33.885ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
356512025-01-17T21:18:33.885ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
356522025-01-17T21:18:33.885ZINFOcrucible: client task is exiting = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
35653 [8820] got start action
35654 [8820] I am going to restart
35655 Starting downstairs at port 8820
35656 Make output file at "/tmp/dsc/downstairs-8820.txt"
35657 [8820][1] reports Starting
35658 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1231)
35659 [8820][1] reports Running
35660 ----------------------------------------------------------------
35661 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
35662 States: Active Offline Active
35663 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35664 10056 Acked Read 5 Done Sent Done false
35665 10057 Acked Read 9 Done Sent Done false
35666 10058 Acked Read 9 Done Sent Done false
35667 10059 Acked Write 2 Done Sent Done false
35668 10060 Acked Flush 0 Done Sent Done false
35669 10061 Acked Write 5 Done Sent Done false
35670 10062 Acked Read 5 Done Sent Done false
35671 10063 Acked Read 1 Done Sent Done false
35672 10064 Acked Write 1 Done Sent Done false
35673 10065 Acked Flush 0 Done Sent Done false
35674 10066 Acked Write 7 Done Sent Done false
35675 10067 Acked Write 9 Done Sent Done false
35676 10068 Acked Write 4 Done Sent Done false
35677 10069 Acked Read 2 Done Sent Done false
35678 10070 Acked Write 8 Done Sent Done false
35679 10071 Acked Write 10 Done Sent Done false
35680 10072 Acked Read 1 Done Sent Done false
35681 10073 Acked Write 3 Done Sent Done false
35682 10074 Acked Read 10 Done Sent Done false
35683 10075 Acked Write 6 Done Sent Done false
35684 10076 Acked Read 5 Done Sent Done false
35685 10077 Acked Read 8 Done Sent Done false
35686 10078 Acked Read 10 Done Sent Done false
35687 10079 Acked Read 6 Done Sent Done false
35688 10080 Acked Read 9 Done Sent Done false
35689 10081 Acked Read 5 Done Sent Done false
35690 10082 Acked Write 6 Done Sent Done false
35691 10083 Acked Read 7 Done Sent Done false
35692 10084 Acked Read 3 Done Sent Done false
35693 10085 Acked Write 9 Done Sent Done false
35694 10086 Acked Read 6 Done Sent Done false
35695 10087 Acked Write 3 Done Sent Done false
35696 10088 Acked Read 5 Done Sent Done false
35697 10089 Acked Write 1 Done Sent Done false
35698 10090 Acked Write 3 Done Sent Done false
35699 10091 Acked Write 3 Done Sent Done false
35700 10092 Acked Read 8 Done Sent Done false
35701 10093 Acked Write 10 Done Sent Done false
35702 10094 Acked Read 9 Done Sent Done false
35703 10095 Acked Read 2 Done Sent Done false
35704 10096 Acked Write 7 Done Sent Done false
35705 10097 Acked Write 4 Done Sent Done false
35706 10098 Acked Read 10 Done Sent Done false
35707 10099 Acked Flush 0 Done Sent Done false
35708 10100 Acked Write 3 Done Sent Done false
35709 10101 Acked Write 10 Done Sent Done false
35710 10102 Acked Read 4 Done Sent Done false
35711 10103 Acked Write 6 Done Sent Done false
35712 10104 Acked Write 9 Done Sent Done false
35713 10105 Acked Write 4 Done Sent Done false
35714 10106 Acked Read 8 Done Sent Done false
35715 10107 Acked Read 8 Done Sent Done false
35716 10108 Acked Write 7 Done Sent Done false
35717 10109 Acked Flush 0 Done Sent Done false
35718 10110 Acked Write 6 Done Sent Done false
35719 10111 Acked Write 5 Done Sent Done false
35720 10112 Acked Write 7 Done Sent Done false
35721 10113 Acked Flush 0 Done Sent Done false
35722 10114 Acked Write 4 Done Sent Done false
35723 10115 Acked Write 5 Done Sent Done false
35724 10116 Acked Write 1 Done Sent Done false
35725 10117 Acked Read 6 Done Sent Done false
35726 10118 Acked Write 3 Done Sent Done false
35727 10119 Acked Write 6 Done Sent Done false
35728 10120 Acked Read 1 Done Sent Done false
35729 10121 Acked Write 6 Done Sent Done false
35730 10122 Acked Flush 0 Done Sent Done false
35731 10123 Acked Write 7 Done Sent Done false
35732 10124 Acked Read 6 Done Sent Done false
35733 10125 Acked Write 1 Done Sent Done false
35734 10126 Acked Flush 0 Done Sent Done false
35735 10127 Acked Write 9 Done Sent Done false
35736 10128 Acked Write 6 Done Sent Done false
35737 10129 Acked Write 1 Done Sent Done false
35738 10130 Acked Read 10 Done Sent Done false
35739 10131 Acked Write 10 Done Sent Done false
35740 10132 Acked Read 10 Done Sent Done false
35741 10133 Acked Write 8 Done Sent Done false
35742 10134 Acked Write 2 Done Sent Done false
35743 10135 Acked Read 5 Done Sent Done false
35744 10136 Acked Write 2 Done Sent Done false
35745 10137 Acked Write 6 Done Sent Done false
35746 10138 Acked Write 4 Done Sent Done false
35747 10139 Acked Read 7 Done Sent Done false
35748 10140 Acked Write 6 Done Sent Done false
35749 10141 Acked Write 9 Done Sent Done false
35750 10142 Acked Read 10 Done Sent Done false
35751 10143 Acked Flush 0 Done Sent Done false
35752 10144 Acked Write 7 Done Sent Done false
35753 10145 Acked Read 4 Done Sent Done false
35754 10146 Acked Read 1 Done Sent Done false
35755 10147 Acked Read 8 Done Sent Done false
35756 10148 Acked Read 7 Done Sent Done false
35757 10149 Acked Write 6 Done Sent Done false
35758 10150 Acked Read 10 Done Sent Done false
35759 10151 Acked Write 10 Done Sent Done false
35760 10152 Acked Flush 0 Done Sent Done false
35761 10153 Acked Read 1 Done Sent Done false
35762 10154 Acked Write 6 Done Sent Done false
35763 10155 Acked Flush 0 Done Sent Done false
35764 10156 Acked Read 6 Done Sent Done false
35765 10157 Acked Read 4 Done Sent Done false
35766 10158 Acked Write 5 Done Sent Done false
35767 10159 Acked Write 3 Done Sent Done false
35768 10160 Acked Write 3 Done Sent Done false
35769 10161 Acked Read 7 Done Sent Done false
35770 10162 Acked Read 8 Done Sent Done false
35771 10163 Acked Read 6 Done Sent Done false
35772 10164 Acked Write 2 Done Sent Done false
35773 10165 Acked Write 5 Done Sent Done false
35774 10166 Acked Write 2 Done Sent Done false
35775 10167 Acked Flush 0 Done Sent Done false
35776 10168 Acked Read 6 Done Sent Done false
35777 10169 Acked Write 2 Done Sent Done false
35778 10170 Acked Read 5 Done Sent Done false
35779 10171 Acked Read 8 Done Sent Done false
35780 10172 Acked Write 6 Done Sent Done false
35781 10173 Acked Write 5 Done Sent Done false
35782 10174 Acked Read 8 Done Sent Done false
35783 10175 Acked Read 7 Done Sent Done false
35784 10176 Acked Read 3 Done Sent Done false
35785 10177 Acked Write 4 Done Sent Done false
35786 10178 Acked Write 5 Done Sent Done false
35787 10179 Acked Read 2 Done Sent Done false
35788 10180 Acked Read 5 Done Sent Done false
35789 10181 Acked Read 1 Done Sent Done false
35790 10182 Acked Write 2 Done Sent Done false
35791 10183 Acked Write 5 Done Sent Done false
35792 10184 Acked Read 8 Done Sent Done false
35793 10185 Acked Write 8 Done Sent Done false
35794 10186 Acked Read 8 Done Sent Done false
35795 10187 Acked Read 2 Done Sent Done false
35796 10188 Acked Read 8 Done Sent Done false
35797 10189 Acked Read 9 Done Sent Done false
35798 10190 Acked Write 8 Done Sent Done false
35799 10191 Acked Write 9 Done Sent Done false
35800 10192 Acked Read 6 Done Sent Done false
35801 10193 Acked Read 10 Done Sent Done false
35802 10194 Acked Write 10 Done Sent Done false
35803 10195 Acked Read 5 Done Sent Done false
35804 10196 Acked Write 9 Done Sent Done false
35805 10197 Acked Write 6 Done Sent Done false
35806 10198 Acked Write 6 Done Sent Done false
35807 10199 Acked Write 4 Done Sent Done false
35808 10200 Acked Read 2 Done Sent Done false
35809 10201 Acked Write 7 Done Sent Done false
35810 10202 Acked Read 3 Done Sent Done false
35811 10203 Acked Write 8 Done Sent Done false
35812 10204 Acked Read 8 Done Sent Done false
35813 10205 Acked Read 1 Done Sent Done false
35814 10206 Acked Write 1 Done Sent Done false
35815 10207 Acked Read 2 Done Sent Done false
35816 10208 Acked Flush 0 Done Sent Done false
35817 10209 Acked Read 1 Done Sent Done false
35818 10210 Acked Write 5 Done Sent Done false
35819 10211 Acked Read 1 Done Sent Done false
35820 10212 Acked Read 9 Done Sent Done false
35821 10213 Acked Write 3 Done Sent Done false
35822 10214 Acked Write 2 Done Sent Done false
35823 10215 Acked Read 9 Done Sent Done false
35824 10216 Acked Write 2 Done Sent Done false
35825 10217 Acked Write 5 Done Sent Done false
35826 10218 Acked Write 6 Done Sent Done false
35827 10219 Acked Write 6 Done Sent Done false
35828 10220 Acked Flush 0 Done Sent Done false
35829 10221 Acked Write 1 Done Sent Done false
35830 10222 Acked Flush 0 Done Sent Done false
35831 10223 Acked Read 6 Done Sent Done false
35832 10224 Acked Write 6 Done Sent Done false
35833 10225 Acked Write 5 Done Sent Done false
35834 10226 Acked Read 4 Done Sent Done false
35835 10227 Acked Write 3 Done Sent Done false
35836 10228 Acked Write 6 Done Sent Done false
35837 10229 Acked Read 6 Done Sent Done false
35838 10230 Acked Read 6 Done Sent Done false
35839 10231 Acked Read 8 Done Sent Done false
35840 10232 Acked Write 3 Done Sent Done false
35841 10233 Acked Write 3 Done Sent Done false
35842 10234 Acked Write 9 Done Sent Done false
35843 10235 Acked Read 8 Done Sent Done false
35844 10236 Acked Write 10 Done Sent Done false
35845 10237 Acked Write 10 Done Sent Done false
35846 10238 Acked Read 8 Done Sent Done false
35847 10239 Acked Read 5 Done Sent Done false
35848 10240 Acked Read 8 Done Sent Done false
35849 10241 Acked Write 1 Done Sent Done false
35850 10242 Acked Read 9 Done Sent Done false
35851 10243 Acked Read 3 Done Sent Done false
35852 10244 Acked Write 1 Done Sent Done false
35853 10245 Acked Write 5 Done Sent Done false
35854 10246 Acked Read 3 Done Sent Done false
35855 10247 Acked Read 5 Done Sent Done false
35856 10248 Acked Write 2 Done Sent Done false
35857 10249 Acked Read 10 Done Sent Done false
35858 10250 Acked Write 3 Done Sent Done false
35859 10251 Acked Write 5 Done Sent Done false
35860 10252 Acked Write 9 Done Sent Done false
35861 10253 Acked Read 7 Done Sent Done false
35862 10254 Acked Write 2 Done Sent Done false
35863 10255 Acked Read 3 Done Sent Done false
35864 10256 Acked Read 1 Done Sent Done false
35865 10257 Acked Read 8 Done Sent Done false
35866 10258 Acked Read 1 Done Sent Done false
35867 10259 Acked Write 5 Done Sent Done false
35868 10260 Acked Read 3 Done Sent Done false
35869 10261 Acked Write 6 Done Sent Done false
35870 10262 Acked Read 9 Done Sent Done false
35871 10263 Acked Read 5 Done Sent Done false
35872 10264 Acked Write 3 Done Sent Done false
35873 10265 Acked Write 3 Done Sent Done false
35874 10266 Acked Flush 0 Done Sent Done false
35875 10267 Acked Write 1 Done Sent Done false
35876 10268 Acked Write 4 Done Sent Done false
35877 10269 Acked Read 10 Done Sent Done false
35878 10270 Acked Write 9 Done Sent Done false
35879 10271 Acked Read 1 Done Sent Done false
35880 10272 Acked Flush 0 Done Sent Done false
35881 10273 Acked Flush 0 Done Sent Done false
35882 10274 Acked Read 9 Done Sent Done false
35883 10275 Acked Read 9 Done Sent Done false
35884 10276 Acked Write 1 Done Sent Done false
35885 10277 Acked Read 1 Done Sent Done false
35886 10278 Acked Flush 0 Done Sent Done false
35887 10279 Acked Write 1 Done Sent Done false
35888 10280 Acked Write 5 Done Sent Done false
35889 10281 Acked Write 9 Done Sent Done false
35890 10282 Acked Read 3 Done Sent Done false
35891 10283 Acked Write 1 Done Sent Done false
35892 10284 Acked Read 2 Done Sent Done false
35893 10285 Acked Read 7 Done Sent Done false
35894 10286 Acked Read 9 Done Sent Done false
35895 10287 Acked Read 5 Done Sent Done false
35896 10288 Acked Read 2 Done Sent Done false
35897 10289 Acked Write 10 Done Sent Done false
35898 10290 Acked Write 5 Done Sent Done false
35899 10291 Acked Read 9 Done Sent Done false
35900 10292 Acked Read 2 Done Sent Done false
35901 10293 Acked Write 2 Done Sent Done false
35902 10294 Acked Read 4 Done Sent Done false
35903 10295 Acked Read 5 Done Sent Done false
35904 10296 Acked Write 10 Done Sent Done false
35905 10297 Acked Read 3 Done Sent Done false
35906 10298 Acked Read 6 Done Sent Done false
35907 10299 Acked Write 3 Done Sent Done false
35908 10300 Acked Write 10 Done Sent Done false
35909 10301 Acked Read 3 Done Sent Done false
35910 10302 Acked Write 8 Done Sent Done false
35911 10303 Acked Write 5 Done Sent Done false
35912 10304 Acked Write 5 Done Sent Done false
35913 10305 Acked Write 8 Done Sent Done false
35914 10306 Acked Write 8 Done Sent Done false
35915 10307 Acked Write 1 Done Sent Done false
35916 10308 Acked Read 4 Done Sent Done false
35917 10309 Acked Write 7 Done Sent Done false
35918 10310 Acked Write 5 Done Sent Done false
35919 10311 Acked Write 10 Done Sent Done false
35920 10312 Acked Write 8 Done Sent Done false
35921 10313 Acked Read 8 Done Sent Done false
35922 10314 Acked Read 5 Done Sent Done false
35923 10315 Acked Flush 0 Done Sent Done false
35924 10316 Acked Flush 0 Done Sent Done false
35925 10317 Acked Read 8 Done Sent Done false
35926 10318 Acked Write 5 Done Sent Done false
35927 10319 Acked Write 5 Done Sent Done false
35928 10320 Acked Read 8 Done Sent Done false
35929 10321 Acked Write 10 Done Sent Done false
35930 10322 Acked Flush 0 Done Sent Done false
35931 10323 Acked Write 7 Done Sent Done false
35932 10324 Acked Write 1 Done Sent Done false
35933 10325 Acked Read 4 Done Sent Done false
35934 10326 Acked Read 10 Done Sent Done false
35935 10327 Acked Read 9 Done Sent Done false
35936 10328 Acked Read 3 Done Sent Done false
35937 10329 Acked Write 4 Done Sent Done false
35938 10330 Acked Write 10 Done Sent Done false
35939 10331 Acked Write 8 Done Sent Done false
35940 10332 Acked Write 1 Done Sent Done false
35941 10333 Acked Read 2 Done Sent Done false
35942 10334 Acked Read 1 Done Sent Done false
35943 10335 Acked Read 7 Done Sent Done false
35944 10336 Acked Read 2 Done Sent Done false
35945 10337 Acked Write 10 Done Sent Done false
35946 10338 Acked Write 5 Done Sent Done false
35947 10339 Acked Write 2 Done Sent Done false
35948 10340 Acked Write 7 Done Sent Done false
35949 10341 Acked Read 5 Done Sent Done false
35950 10342 Acked Read 10 Done Sent Done false
35951 10343 Acked Read 8 Done Sent Done false
35952 10344 Acked Read 3 Done Sent Done false
35953 10345 Acked Read 9 Done Sent Done false
35954 10346 Acked Flush 0 Done Sent Done false
35955 10347 Acked Flush 0 Done Sent Done false
35956 10348 Acked Read 10 Done Sent Done false
35957 10349 Acked Flush 0 Done Sent Done false
35958 10350 Acked Read 9 Done Sent Done false
35959 10351 Acked Read 5 Done Sent Done false
35960 10352 Acked Write 5 Done Sent Done false
35961 10353 Acked Flush 0 Done Sent Done false
35962 10354 Acked Read 5 Done Sent Done false
35963 10355 Acked Write 7 Done Sent Done false
35964 10356 Acked Read 6 Done Sent Done false
35965 10357 Acked Flush 0 Done Sent Done false
35966 STATES DS:0 DS:1 DS:2 TOTAL
35967 Sent 0 302 0 302
35968 Done 302 0 302 604
35969 Skipped 0 0 0 0
35970 Error 0 0 0 0
35971 Last Flush: 10357 10055 10357
35972 Downstairs last five completed: 10055 10054 10053 10052 10051
35973 Upstairs last five completed: 10357 10356 10355 10354 10353
35974 CLIENT: Up:0 ds:302 act:2
35975 ----------------------------------------------------------------
35976 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
35977 States: Active Offline Active
35978 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35979 10056 Acked Read 5 Done Sent Done false
35980 10057 Acked Read 9 Done Sent Done false
35981 10058 Acked Read 9 Done Sent Done false
35982 10059 Acked Write 2 Done Sent Done false
35983 10060 Acked Flush 0 Done Sent Done false
35984 10061 Acked Write 5 Done Sent Done false
35985 10062 Acked Read 5 Done Sent Done false
35986 10063 Acked Read 1 Done Sent Done false
35987 10064 Acked Write 1 Done Sent Done false
35988 10065 Acked Flush 0 Done Sent Done false
35989 10066 Acked Write 7 Done Sent Done false
35990 10067 Acked Write 9 Done Sent Done false
35991 10068 Acked Write 4 Done Sent Done false
35992 10069 Acked Read 2 Done Sent Done false
35993 10070 Acked Write 8 Done Sent Done false
35994 10071 Acked Write 10 Done Sent Done false
35995 10072 Acked Read 1 Done Sent Done false
35996 10073 Acked Write 3 Done Sent Done false
35997 10074 Acked Read 10 Done Sent Done false
35998 10075 Acked Write 6 Done Sent Done false
35999 10076 Acked Read 5 Done Sent Done false
36000 10077 Acked Read 8 Done Sent Done false
36001 10078 Acked Read 10 Done Sent Done false
36002 10079 Acked Read 6 Done Sent Done false
36003 10080 Acked Read 9 Done Sent Done false
36004 10081 Acked Read 5 Done Sent Done false
36005 10082 Acked Write 6 Done Sent Done false
36006 10083 Acked Read 7 Done Sent Done false
36007 10084 Acked Read 3 Done Sent Done false
36008 10085 Acked Write 9 Done Sent Done false
36009 10086 Acked Read 6 Done Sent Done false
36010 10087 Acked Write 3 Done Sent Done false
36011 10088 Acked Read 5 Done Sent Done false
36012 10089 Acked Write 1 Done Sent Done false
36013 10090 Acked Write 3 Done Sent Done false
36014 10091 Acked Write 3 Done Sent Done false
36015 10092 Acked Read 8 Done Sent Done false
36016 10093 Acked Write 10 Done Sent Done false
36017 10094 Acked Read 9 Done Sent Done false
36018 10095 Acked Read 2 Done Sent Done false
36019 10096 Acked Write 7 Done Sent Done false
36020 10097 Acked Write 4 Done Sent Done false
36021 10098 Acked Read 10 Done Sent Done false
36022 10099 Acked Flush 0 Done Sent Done false
36023 10100 Acked Write 3 Done Sent Done false
36024 10101 Acked Write 10 Done Sent Done false
36025 10102 Acked Read 4 Done Sent Done false
36026 10103 Acked Write 6 Done Sent Done false
36027 10104 Acked Write 9 Done Sent Done false
36028 10105 Acked Write 4 Done Sent Done false
36029 10106 Acked Read 8 Done Sent Done false
36030 10107 Acked Read 8 Done Sent Done false
36031 10108 Acked Write 7 Done Sent Done false
36032 10109 Acked Flush 0 Done Sent Done false
36033 10110 Acked Write 6 Done Sent Done false
36034 10111 Acked Write 5 Done Sent Done false
36035 10112 Acked Write 7 Done Sent Done false
36036 10113 Acked Flush 0 Done Sent Done false
36037 10114 Acked Write 4 Done Sent Done false
36038 10115 Acked Write 5 Done Sent Done false
36039 10116 Acked Write 1 Done Sent Done false
36040 10117 Acked Read 6 Done Sent Done false
36041 10118 Acked Write 3 Done Sent Done false
36042 10119 Acked Write 6 Done Sent Done false
36043 10120 Acked Read 1 Done Sent Done false
36044 10121 Acked Write 6 Done Sent Done false
36045 10122 Acked Flush 0 Done Sent Done false
36046 10123 Acked Write 7 Done Sent Done false
36047 10124 Acked Read 6 Done Sent Done false
36048 10125 Acked Write 1 Done Sent Done false
36049 10126 Acked Flush 0 Done Sent Done false
36050 10127 Acked Write 9 Done Sent Done false
36051 10128 Acked Write 6 Done Sent Done false
36052 10129 Acked Write 1 Done Sent Done false
36053 10130 Acked Read 10 Done Sent Done false
36054 10131 Acked Write 10 Done Sent Done false
36055 10132 Acked Read 10 Done Sent Done false
36056 10133 Acked Write 8 Done Sent Done false
36057 10134 Acked Write 2 Done Sent Done false
36058 10135 Acked Read 5 Done Sent Done false
36059 10136 Acked Write 2 Done Sent Done false
36060 10137 Acked Write 6 Done Sent Done false
36061 10138 Acked Write 4 Done Sent Done false
36062 10139 Acked Read 7 Done Sent Done false
36063 10140 Acked Write 6 Done Sent Done false
36064 10141 Acked Write 9 Done Sent Done false
36065 10142 Acked Read 10 Done Sent Done false
36066 10143 Acked Flush 0 Done Sent Done false
36067 10144 Acked Write 7 Done Sent Done false
36068 10145 Acked Read 4 Done Sent Done false
36069 10146 Acked Read 1 Done Sent Done false
36070 10147 Acked Read 8 Done Sent Done false
36071 10148 Acked Read 7 Done Sent Done false
36072 10149 Acked Write 6 Done Sent Done false
36073 10150 Acked Read 10 Done Sent Done false
36074 10151 Acked Write 10 Done Sent Done false
36075 10152 Acked Flush 0 Done Sent Done false
36076 10153 Acked Read 1 Done Sent Done false
36077 10154 Acked Write 6 Done Sent Done false
36078 10155 Acked Flush 0 Done Sent Done false
36079 10156 Acked Read 6 Done Sent Done false
36080 10157 Acked Read 4 Done Sent Done false
36081 10158 Acked Write 5 Done Sent Done false
36082 10159 Acked Write 3 Done Sent Done false
36083 10160 Acked Write 3 Done Sent Done false
36084 10161 Acked Read 7 Done Sent Done false
36085 10162 Acked Read 8 Done Sent Done false
36086 10163 Acked Read 6 Done Sent Done false
36087 10164 Acked Write 2 Done Sent Done false
36088 10165 Acked Write 5 Done Sent Done false
36089 10166 Acked Write 2 Done Sent Done false
36090 10167 Acked Flush 0 Done Sent Done false
36091 10168 Acked Read 6 Done Sent Done false
36092 10169 Acked Write 2 Done Sent Done false
36093 10170 Acked Read 5 Done Sent Done false
36094 10171 Acked Read 8 Done Sent Done false
36095 10172 Acked Write 6 Done Sent Done false
36096 10173 Acked Write 5 Done Sent Done false
36097 10174 Acked Read 8 Done Sent Done false
36098 10175 Acked Read 7 Done Sent Done false
36099 10176 Acked Read 3 Done Sent Done false
36100 10177 Acked Write 4 Done Sent Done false
36101 10178 Acked Write 5 Done Sent Done false
36102 10179 Acked Read 2 Done Sent Done false
36103 10180 Acked Read 5 Done Sent Done false
36104 10181 Acked Read 1 Done Sent Done false
36105 10182 Acked Write 2 Done Sent Done false
36106 10183 Acked Write 5 Done Sent Done false
36107 10184 Acked Read 8 Done Sent Done false
36108 10185 Acked Write 8 Done Sent Done false
36109 10186 Acked Read 8 Done Sent Done false
36110 10187 Acked Read 2 Done Sent Done false
36111 10188 Acked Read 8 Done Sent Done false
36112 10189 Acked Read 9 Done Sent Done false
36113 10190 Acked Write 8 Done Sent Done false
36114 10191 Acked Write 9 Done Sent Done false
36115 10192 Acked Read 6 Done Sent Done false
36116 10193 Acked Read 10 Done Sent Done false
36117 10194 Acked Write 10 Done Sent Done false
36118 10195 Acked Read 5 Done Sent Done false
36119 10196 Acked Write 9 Done Sent Done false
36120 10197 Acked Write 6 Done Sent Done false
36121 10198 Acked Write 6 Done Sent Done false
36122 10199 Acked Write 4 Done Sent Done false
36123 10200 Acked Read 2 Done Sent Done false
36124 10201 Acked Write 7 Done Sent Done false
36125 10202 Acked Read 3 Done Sent Done false
36126 10203 Acked Write 8 Done Sent Done false
36127 10204 Acked Read 8 Done Sent Done false
36128 10205 Acked Read 1 Done Sent Done false
36129 10206 Acked Write 1 Done Sent Done false
36130 10207 Acked Read 2 Done Sent Done false
36131 10208 Acked Flush 0 Done Sent Done false
36132 10209 Acked Read 1 Done Sent Done false
36133 10210 Acked Write 5 Done Sent Done false
36134 10211 Acked Read 1 Done Sent Done false
36135 10212 Acked Read 9 Done Sent Done false
36136 10213 Acked Write 3 Done Sent Done false
36137 10214 Acked Write 2 Done Sent Done false
36138 10215 Acked Read 9 Done Sent Done false
36139 10216 Acked Write 2 Done Sent Done false
36140 10217 Acked Write 5 Done Sent Done false
36141 10218 Acked Write 6 Done Sent Done false
36142 10219 Acked Write 6 Done Sent Done false
36143 10220 Acked Flush 0 Done Sent Done false
36144 10221 Acked Write 1 Done Sent Done false
36145 10222 Acked Flush 0 Done Sent Done false
36146 10223 Acked Read 6 Done Sent Done false
36147 10224 Acked Write 6 Done Sent Done false
36148 10225 Acked Write 5 Done Sent Done false
36149 10226 Acked Read 4 Done Sent Done false
36150 10227 Acked Write 3 Done Sent Done false
36151 10228 Acked Write 6 Done Sent Done false
36152 10229 Acked Read 6 Done Sent Done false
36153 10230 Acked Read 6 Done Sent Done false
36154 10231 Acked Read 8 Done Sent Done false
36155 10232 Acked Write 3 Done Sent Done false
36156 10233 Acked Write 3 Done Sent Done false
36157 10234 Acked Write 9 Done Sent Done false
36158 10235 Acked Read 8 Done Sent Done false
36159 10236 Acked Write 10 Done Sent Done false
36160 10237 Acked Write 10 Done Sent Done false
36161 10238 Acked Read 8 Done Sent Done false
36162 10239 Acked Read 5 Done Sent Done false
36163 10240 Acked Read 8 Done Sent Done false
36164 10241 Acked Write 1 Done Sent Done false
36165 10242 Acked Read 9 Done Sent Done false
36166 10243 Acked Read 3 Done Sent Done false
36167 10244 Acked Write 1 Done Sent Done false
36168 10245 Acked Write 5 Done Sent Done false
36169 10246 Acked Read 3 Done Sent Done false
36170 10247 Acked Read 5 Done Sent Done false
36171 10248 Acked Write 2 Done Sent Done false
36172 10249 Acked Read 10 Done Sent Done false
36173 10250 Acked Write 3 Done Sent Done false
36174 10251 Acked Write 5 Done Sent Done false
36175 10252 Acked Write 9 Done Sent Done false
36176 10253 Acked Read 7 Done Sent Done false
36177 10254 Acked Write 2 Done Sent Done false
36178 10255 Acked Read 3 Done Sent Done false
36179 10256 Acked Read 1 Done Sent Done false
36180 10257 Acked Read 8 Done Sent Done false
36181 10258 Acked Read 1 Done Sent Done false
36182 10259 Acked Write 5 Done Sent Done false
36183 10260 Acked Read 3 Done Sent Done false
36184 10261 Acked Write 6 Done Sent Done false
36185 10262 Acked Read 9 Done Sent Done false
36186 10263 Acked Read 5 Done Sent Done false
36187 10264 Acked Write 3 Done Sent Done false
36188 10265 Acked Write 3 Done Sent Done false
36189 10266 Acked Flush 0 Done Sent Done false
36190 10267 Acked Write 1 Done Sent Done false
36191 10268 Acked Write 4 Done Sent Done false
36192 10269 Acked Read 10 Done Sent Done false
36193 10270 Acked Write 9 Done Sent Done false
36194 10271 Acked Read 1 Done Sent Done false
36195 10272 Acked Flush 0 Done Sent Done false
36196 10273 Acked Flush 0 Done Sent Done false
36197 10274 Acked Read 9 Done Sent Done false
36198 10275 Acked Read 9 Done Sent Done false
36199 10276 Acked Write 1 Done Sent Done false
36200 10277 Acked Read 1 Done Sent Done false
36201 10278 Acked Flush 0 Done Sent Done false
36202 10279 Acked Write 1 Done Sent Done false
36203 10280 Acked Write 5 Done Sent Done false
36204 10281 Acked Write 9 Done Sent Done false
36205 10282 Acked Read 3 Done Sent Done false
36206 10283 Acked Write 1 Done Sent Done false
36207 10284 Acked Read 2 Done Sent Done false
36208 10285 Acked Read 7 Done Sent Done false
36209 10286 Acked Read 9 Done Sent Done false
36210 10287 Acked Read 5 Done Sent Done false
36211 10288 Acked Read 2 Done Sent Done false
36212 10289 Acked Write 10 Done Sent Done false
36213 10290 Acked Write 5 Done Sent Done false
36214 10291 Acked Read 9 Done Sent Done false
36215 10292 Acked Read 2 Done Sent Done false
36216 10293 Acked Write 2 Done Sent Done false
36217 10294 Acked Read 4 Done Sent Done false
36218 10295 Acked Read 5 Done Sent Done false
36219 10296 Acked Write 10 Done Sent Done false
36220 10297 Acked Read 3 Done Sent Done false
36221 10298 Acked Read 6 Done Sent Done false
36222 10299 Acked Write 3 Done Sent Done false
36223 10300 Acked Write 10 Done Sent Done false
36224 10301 Acked Read 3 Done Sent Done false
36225 10302 Acked Write 8 Done Sent Done false
36226 10303 Acked Write 5 Done Sent Done false
36227 10304 Acked Write 5 Done Sent Done false
36228 10305 Acked Write 8 Done Sent Done false
36229 10306 Acked Write 8 Done Sent Done false
36230 10307 Acked Write 1 Done Sent Done false
36231 10308 Acked Read 4 Done Sent Done false
36232 10309 Acked Write 7 Done Sent Done false
36233 10310 Acked Write 5 Done Sent Done false
36234 10311 Acked Write 10 Done Sent Done false
36235 10312 Acked Write 8 Done Sent Done false
36236 10313 Acked Read 8 Done Sent Done false
36237 10314 Acked Read 5 Done Sent Done false
36238 10315 Acked Flush 0 Done Sent Done false
36239 10316 Acked Flush 0 Done Sent Done false
36240 10317 Acked Read 8 Done Sent Done false
36241 10318 Acked Write 5 Done Sent Done false
36242 10319 Acked Write 5 Done Sent Done false
36243 10320 Acked Read 8 Done Sent Done false
36244 10321 Acked Write 10 Done Sent Done false
36245 10322 Acked Flush 0 Done Sent Done false
36246 10323 Acked Write 7 Done Sent Done false
36247 10324 Acked Write 1 Done Sent Done false
36248 10325 Acked Read 4 Done Sent Done false
36249 10326 Acked Read 10 Done Sent Done false
36250 10327 Acked Read 9 Done Sent Done false
36251 10328 Acked Read 3 Done Sent Done false
36252 10329 Acked Write 4 Done Sent Done false
36253 10330 Acked Write 10 Done Sent Done false
36254 10331 Acked Write 8 Done Sent Done false
36255 10332 Acked Write 1 Done Sent Done false
36256 10333 Acked Read 2 Done Sent Done false
36257 10334 Acked Read 1 Done Sent Done false
36258 10335 Acked Read 7 Done Sent Done false
36259 10336 Acked Read 2 Done Sent Done false
36260 10337 Acked Write 10 Done Sent Done false
36261 10338 Acked Write 5 Done Sent Done false
36262 10339 Acked Write 2 Done Sent Done false
36263 10340 Acked Write 7 Done Sent Done false
36264 10341 Acked Read 5 Done Sent Done false
36265 10342 Acked Read 10 Done Sent Done false
36266 10343 Acked Read 8 Done Sent Done false
36267 10344 Acked Read 3 Done Sent Done false
36268 10345 Acked Read 9 Done Sent Done false
36269 10346 Acked Flush 0 Done Sent Done false
36270 10347 Acked Flush 0 Done Sent Done false
36271 10348 Acked Read 10 Done Sent Done false
36272 10349 Acked Flush 0 Done Sent Done false
36273 10350 Acked Read 9 Done Sent Done false
36274 10351 Acked Read 5 Done Sent Done false
36275 10352 Acked Write 5 Done Sent Done false
36276 10353 Acked Flush 0 Done Sent Done false
36277 10354 Acked Read 5 Done Sent Done false
36278 10355 Acked Write 7 Done Sent Done false
36279 10356 Acked Read 6 Done Sent Done false
36280 10357 Acked Flush 0 Done Sent Done false
36281 STATES DS:0 DS:1 DS:2 TOTAL
36282 Sent 0 302 0 302
36283 Done 302 0 302 604
36284 Skipped 0 0 0 0
36285 Error 0 0 0 0
36286 Last Flush: 10357 10055 10357
36287 Downstairs last five completed: 10055 10054 10053 10052 10051
36288 Upstairs last five completed: 10357 10356 10355 10354 10353
36289 CLIENT: Up:0 ds:302 act:2
362902025-01-17T21:18:43.886ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
362912025-01-17T21:18:43.886ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58557) = io task client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
362922025-01-17T21:18:43.887ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
362932025-01-17T21:18:43.888ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
362942025-01-17T21:18:43.888ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
362952025-01-17T21:18:43.888ZINFOcrucible: send last flush ID to this DS: 10055 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
362962025-01-17T21:18:43.888ZINFOcrucible: Replied this last flush ID: 10055 = downstairs client = 1 session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
362972025-01-17T21:18:43.888ZINFOcrucible: [1] client re-new 302 jobs since flush 10055 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
362982025-01-17T21:18:43.888ZINFOcrucible: [1] Replayed 302 jobs since flush: 10055 = downstairs session_id = 64f44984-0f34-4cfb-a401-a0494f2ea105
36299 ----------------------------------------------------------------
36300 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
36301 Downstairs last five completed: 10357 10356 10355 10354 10353
36302 Upstairs last five completed: 10357 10356 10355 10354 10353
36303 CLIENT: Up:0 ds:0 act:3
36304 Replay: All jobs finished
36305 Test replay has completed
36306 Wrote out file "/tmp/test_replay_verify.log"
36307 CLIENT: Tests done. All submitted work has been ACK'd
36308 ----------------------------------------------------------------
36309 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
36310 Downstairs last five completed: 10357 10356 10355 10354 10353
36311 Upstairs last five completed: 10357 10356 10355 10354 10353
36312 CLIENT: Up:0 ds:0 act:3
36313 CLIENT: All jobs finished, all DS active.
36314 Replays:30 time: 7:36
36315 Do final verify
36316 Jan 17 21:18:44.204 INFO accepted connection, remote_addr: 127.0.0.1:63790, local_addr: 127.0.0.1:9998
36317 Jan 17 21:18:44.205 INFO request completed, latency_us: 202, response_code: 200, uri: /regioninfo, method: GET, req_id: c2dfcd37-bbe6-414b-a177-2d59f0c84b00, remote_addr: 127.0.0.1:63790, local_addr: 127.0.0.1:9998
363182025-01-17T21:18:44.205ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
36319 Jan 17 21:18:44.205 INFO request completed, latency_us: 87, response_code: 200, uri: /regioncount, method: GET, req_id: 976d016e-7bc2-4e70-9cf3-7f8eac1b7c4c, remote_addr: 127.0.0.1:63790, local_addr: 127.0.0.1:9998
363202025-01-17T21:18:44.206ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
36321 Jan 17 21:18:44.206 INFO request completed, latency_us: 108, response_code: 200, uri: /port/cid/0, method: GET, req_id: 8a9eb4df-a473-45de-93ce-26b8a8af582f, remote_addr: 127.0.0.1:63790, local_addr: 127.0.0.1:9998
36322 Jan 17 21:18:44.207 INFO request completed, latency_us: 102, response_code: 200, uri: /port/cid/1, method: GET, req_id: 6d4b431c-7956-4635-93f6-6f782631cd87, remote_addr: 127.0.0.1:63790, local_addr: 127.0.0.1:9998
36323 Jan 17 21:18:44.207 INFO request completed, latency_us: 97, response_code: 200, uri: /port/cid/2, method: GET, req_id: 7b51ae5d-1d7d-445a-9d53-78f0f52c91b1, remote_addr: 127.0.0.1:63790, local_addr: 127.0.0.1:9998
363242025-01-17T21:18:44.207ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
363252025-01-17T21:18:44.208ZINFOcrucible: Upstairs starts
363262025-01-17T21:18:44.208ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "abd5430f1508af0bc008143ea2566277ef714ec2", git_commit_timestamp: "2025-01-17T20:55:03.000000000Z", git_branch: "alan/tokio-to-140", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
363272025-01-17T21:18:44.208ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
363282025-01-17T21:18:44.208ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
363292025-01-17T21:18:44.208ZINFOcrucible: Crucible 4094caf7-3c76-49c3-8f47-8519da60bb7a has session id: 6b9b33aa-cc91-4029-a760-42b945e6ab80 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363302025-01-17T21:18:44.208ZINFOcrucible: Upstairs opts: Upstairs UUID: 4094caf7-3c76-49c3-8f47-8519da60bb7a, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363312025-01-17T21:18:44.208ZINFOcrucible: Crucible stats registered with UUID: 4094caf7-3c76-49c3-8f47-8519da60bb7a session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363322025-01-17T21:18:44.208ZWARNcrucible: could not find Downstairs address for Nexus session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363332025-01-17T21:18:44.209ZINFOcrucible: Downstairs in volume = 3 task = crutest
363342025-01-17T21:18:44.209ZINFOcrucible: The guest has requested activation with gen:3
363352025-01-17T21:18:44.209ZINFOcrucible: 4094caf7-3c76-49c3-8f47-8519da60bb7a active request set session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363362025-01-17T21:18:44.209ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363372025-01-17T21:18:44.209ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363382025-01-17T21:18:44.209ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363392025-01-17T21:18:44.209ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363402025-01-17T21:18:44.209ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363412025-01-17T21:18:44.209ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363422025-01-17T21:18:44.209ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363432025-01-17T21:18:44.209ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363442025-01-17T21:18:44.209ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363452025-01-17T21:18:44.209ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:48705) = io task client = 2 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363462025-01-17T21:18:44.209ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57089) = io task client = 0 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363472025-01-17T21:18:44.210ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:45222) = io task client = 1 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363482025-01-17T21:18:44.210ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363492025-01-17T21:18:44.210ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363502025-01-17T21:18:44.210ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363512025-01-17T21:18:44.211ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363522025-01-17T21:18:44.211ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363532025-01-17T21:18:44.211ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363542025-01-17T21:18:44.212ZINFOcrucible: Waiting for more clients to be ready session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363552025-01-17T21:18:44.212ZINFOcrucible: Waiting for more clients to be ready session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363562025-01-17T21:18:44.212ZINFOcrucible: [0]R flush_numbers[0..12]: [1015, 1018, 1018, 1011, 1011, 999, 1011, 1018, 1011, 1022, 1015, 1015] = downstairs session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363572025-01-17T21:18:44.212ZINFOcrucible: [0]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363582025-01-17T21:18:44.212ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363592025-01-17T21:18:44.212ZINFOcrucible: [1]R flush_numbers[0..12]: [1015, 1018, 1018, 1011, 1011, 999, 1011, 1018, 1011, 1022, 1015, 1015] = downstairs session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363602025-01-17T21:18:44.212ZINFOcrucible: [1]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363612025-01-17T21:18:44.212ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363622025-01-17T21:18:44.212ZINFOcrucible: [2]R flush_numbers[0..12]: [1015, 1018, 1018, 1011, 1011, 999, 1011, 1018, 1011, 1022, 1015, 1015] = downstairs session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363632025-01-17T21:18:44.212ZINFOcrucible: [2]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363642025-01-17T21:18:44.212ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363652025-01-17T21:18:44.212ZINFOcrucible: Max found gen is 3 = downstairs session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363662025-01-17T21:18:44.212ZINFOcrucible: Generation requested: 3 >= found:3 = downstairs session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363672025-01-17T21:18:44.212ZINFOcrucible: Next flush: 1023 = downstairs session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
36368 {"msg":"All extents match","v":0,"name":"crucible","level":30,"time":"2025-01-17T21:18:44.212537134Z","hostname":"Wait for a query_work_queue command to finish before sending IO
36369 w-01JHV1KEJDK2FKADHFW4CFET5M","pid":1236,"":"downstairs","":"downstairs","session_id":"6b9b33aa-cc91-4029-a760-42b945e6ab80"}
363702025-01-17T21:18:44.212ZINFOcrucible: No downstairs reconciliation required session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363712025-01-17T21:18:44.212ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363722025-01-17T21:18:44.212ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363732025-01-17T21:18:44.212ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363742025-01-17T21:18:44.212ZINFOcrucible: All required reconciliation work is completed session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363752025-01-17T21:18:44.212ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363762025-01-17T21:18:44.212ZINFOcrucible: 4094caf7-3c76-49c3-8f47-8519da60bb7a is now active with session: 6b9b33aa-cc91-4029-a760-42b945e6ab80 session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363772025-01-17T21:18:44.212ZINFOcrucible: Set Active after no reconciliation session_id = 6b9b33aa-cc91-4029-a760-42b945e6ab80
363782025-01-17T21:18:44.212ZINFOcrucible: The guest has finished waiting for activation with:3
36379 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
36380 Loading write count information from file "/tmp/test_replay_verify.log"
36381 Read and Verify all blocks (0..5000 range:false)
36382 Wrote out file "/tmp/test_replay_verify.log"
36383 Verify test completed
36384 Wrote out file "/tmp/test_replay_verify.log"
36385 CLIENT: Tests done. All submitted work has been ACK'd
36386 ----------------------------------------------------------------
36387 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:20
36388 States: Active Active Active
36389 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
36390 1000 Acked Read 256 Done Done Done false
36391 1001 Acked Read 256 Done Done Done false
36392 1002 Acked Read 256 Done Done Done false
36393 1003 Acked Read 256 Done Done Done false
36394 1004 Acked Read 256 Done Done Done false
36395 1005 Acked Read 256 Done Done Done false
36396 1006 Acked Read 256 Done Done Done false
36397 1007 Acked Read 256 Done Done Done false
36398 1008 Acked Read 256 Done Done Done false
36399 1009 Acked Read 256 Done Done Done false
36400 1010 Acked Read 256 Done Done Done false
36401 1011 Acked Read 256 Done Done Done false
36402 1012 Acked Read 256 Done Done Done false
36403 1013 Acked Read 256 Done Done Done false
36404 1014 Acked Read 256 Done Done Done false
36405 1015 Acked Read 256 Done Done Done false
36406 1016 Acked Read 256 Done Done Done false
36407 1017 Acked Read 256 Done Done Done false
36408 1018 Acked Read 256 Done Done Done false
36409 1019 Acked Read 136 Done Done Done false
36410 STATES DS:0 DS:1 DS:2 TOTAL
36411 Sent 0 0 0 0
36412 Done 20 20 20 60
36413 Skipped 0 0 0 0
36414 Error 0 0 0 0
36415 Last Flush: 0 0 0
36416 Downstairs last five completed:
36417 Upstairs last five completed: 1019 1018 1017 1016 1015
36418 CLIENT: Up:0 ds:20 act:3
36419 ----------------------------------------------------------------
36420 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:0
36421 Downstairs last five completed: 1020 1019 1018 1017 1016
36422 Upstairs last five completed: 1020 1019 1018 1017 1016
36423 CLIENT: Up:0 ds:0 act:3
36424 CLIENT: All crucible jobs finished, exiting program
364252025-01-17T21:18:48.612ZWARNcrucible: Guest handle has been dropped
36426 Jan 17 21:18:48.676 INFO accepted connection, remote_addr: 127.0.0.1:61670, local_addr: 127.0.0.1:9998
36427 Main task has work to do, go find it
36428 got dsc Shutdown
36429 Shutdown
36430 Jan 17 21:18:48.677 INFO request completed, latency_us: 146, response_code:Shut it down
36431 [8810] Disable keep_running
36432 204, uri: /shutdown, method: POST[8810] Got stop action so:false kr:false
36433 , req_id: 3c561d52-fc04-4304-832a-26f28402108c, remote_addr: 127.0.0.1:61670, local_addr: 127.0.0.1:9998
36434 [8820] Disable keep_running
36435 [8820] Got stop action so:false kr:false
36436 [8830] Disable keep_running
36437 [8830] Got stop action so:false kr:false
36438 [8820] Exited with: ExitStatus(unix_wait_status(9))
36439 [8830] Exited with: ExitStatus(unix_wait_status(9))
36440 [8820][1] reports Exit
36441 [8830][2] reports Exit
36442 [8810] Exited with: ExitStatus(unix_wait_status(9))
36443 [8810][0] reports Exit
36444 January 17, 2025 at 09:18:53 PM UTC Test ends with 0